Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen
Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen University of Dortmund, Germany Madhu Sudan Massachusetts Institute of Technology, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany
5612
Julie A. Jacko (Ed.)
Human-Computer Interaction Ambient, Ubiquitous and Intelligent Interaction 13th International Conference, HCI International 2009 San Diego, CA, USA, July 19-24, 2009 Proceedings, Part III
13
Volume Editor Julie A. Jacko University of Minnesota Institute of Health Informatics MMC 912, 420 Delaware Street S.E., Minneapolis, MN 55455, USA E-mail:
[email protected]
Library of Congress Control Number: 2009929048 CR Subject Classification (1998): H.5, I.3, I.7.5, I.5, I.2.10 LNCS Sublibrary: SL 3 – Information Systems and Application, incl. Internet/Web and HCI ISSN ISBN-10 ISBN-13
0302-9743 3-642-02579-X Springer Berlin Heidelberg New York 978-3-642-02579-2 Springer Berlin Heidelberg New York
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2009 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 12707225 06/3180 543210
Foreword
The 13th International Conference on Human–Computer Interaction, HCI International 2009, was held in San Diego, California, USA, July 19–24, 2009, jointly with the Symposium on Human Interface (Japan) 2009, the 8th International Conference on Engineering Psychology and Cognitive Ergonomics, the 5th International Conference on Universal Access in Human–Computer Interaction, the Third International Conference on Virtual and Mixed Reality, the Third International Conference on Internationalization, Design and Global Development, the Third International Conference on Online Communities and Social Computing, the 5th International Conference on Augmented Cognition, the Second International Conference on Digital Human Modeling, and the First International Conference on Human Centered Design. A total of 4,348 individuals from academia, research institutes, industry and governmental agencies from 73 countries submitted contributions, and 1,397 papers that were judged to be of high scientific quality were included in the program. These papers address the latest research and development efforts and highlight the human aspects of the design and use of computing systems. The papers accepted for presentation thoroughly cover the entire field of human–computer interaction, addressing major advances in the knowledge and effective use of computers in a variety of application areas. This volume, edited by Julie A. Jacko, contains papers in the thematic area of Human–Computer Interaction, addressing the following major topics: • • • • • •
Mobile Interaction In-vehicle Interaction and Environment Navigation Agents, Avatars and Personalization Ambient Interaction Affect, Emotion and Engagement Smart and Wearable Materials and Devices
The remaining volumes of the HCI International 2009 proceedings are: • • • • •
Volume 1, LNCS 5610, Human–Computer Interaction––New Trends (Part I), edited by Julie A. Jacko Volume 2, LNCS 5611, Human–Computer Interaction––Novel Interaction Methods and Techniques (Part II), edited by Julie A. Jacko Volume 4, LNCS 5613, Human–Computer Interaction––Interacting in Various Application Domains (Part IV), edited by Julie A. Jacko Volume 5, LNCS 5614, Universal Access in Human–Computer Interaction––Addressing Diversity (Part I), edited by Constantine Stephanidis Volume 6, LNCS 5615, Universal Access in Human–Computer Interaction––Intelligent and Ubiquitous Interaction Environments (Part II), edited by Constantine Stephanidis
VI
Foreword
• • • • • • • • • • •
Volume 7, LNCS 5616, Universal Access in Human–Computer Interaction––Applications and Services (Part III), edited by Constantine Stephanidis Volume 8, LNCS 5617, Human Interface and the Management of Information––Designing Information Environments (Part I), edited by Michael J. Smith and Gavriel Salvendy Volume 9, LNCS 5618, Human Interface and the Management of Information––Information and Interaction (Part II), edited by Gavriel Salvendy and Michael J. Smith Volume 10, LNCS 5619, Human Centered Design, edited by Masaaki Kurosu Volume 11, LNCS 5620, Digital Human Modeling, edited by Vincent G. Duffy Volume 12, LNCS 5621, Online Communities and Social Computing, edited by A. Ant Ozok and Panayiotis Zaphiris Volume 13, LNCS 5622, Virtual and Mixed Reality, edited by Randall Shumaker Volume 14, LNCS 5623, Internationalization, Design and Global Development, edited by Nuray Aykin Volume 15, LNCS 5624, Ergonomics and Health Aspects of Work with Computers, edited by Ben-Tzion Karsh Volume 16, LNAI 5638, The Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, edited by Dylan Schmorrow, Ivy Estabrooke and Marc Grootjen Volume 17, LNAI 5639, Engineering Psychology and Cognitive Ergonomics, edited by Don Harris
I would like to thank the Program Chairs and the members of the Program Boards of all thematic areas, listed below, for their contribution to the highest scientific quality and the overall success of HCI International 2009.
Ergonomics and Health Aspects of Work with Computers Program Chair: Ben-Tzion Karsh Arne Aarås, Norway Pascale Carayon, USA Barbara G.F. Cohen, USA Wolfgang Friesdorf, Germany John Gosbee, USA Martin Helander, Singapore Ed Israelski, USA Waldemar Karwowski, USA Peter Kern, Germany Danuta Koradecka, Poland Kari Lindström, Finland
Holger Luczak, Germany Aura C. Matias, Philippines Kyung (Ken) Park, Korea Michelle M. Robertson, USA Michelle L. Rogers, USA Steven L. Sauter, USA Dominique L. Scapin, France Naomi Swanson, USA Peter Vink, The Netherlands John Wilson, UK Teresa Zayas-Cabán, USA
Foreword
Human Interface and the Management of Information Program Chair: Michael J. Smith Gunilla Bradley, Sweden Hans-Jörg Bullinger, Germany Alan Chan, Hong Kong Klaus-Peter Fähnrich, Germany Michitaka Hirose, Japan Jhilmil Jain, USA Yasufumi Kume, Japan Mark Lehto, USA Fiona Fui-Hoon Nah, USA Shogo Nishida, Japan Robert Proctor, USA Youngho Rhee, Korea
Anxo Cereijo Roibás, UK Katsunori Shimohara, Japan Dieter Spath, Germany Tsutomu Tabe, Japan Alvaro D. Taveira, USA Kim-Phuong L. Vu, USA Tomio Watanabe, Japan Sakae Yamamoto, Japan Hidekazu Yoshikawa, Japan Li Zheng, P.R. China Bernhard Zimolong, Germany
Human–Computer Interaction Program Chair: Julie A. Jacko Sebastiano Bagnara, Italy Sherry Y. Chen, UK Marvin J. Dainoff, USA Jianming Dong, USA John Eklund, Australia Xiaowen Fang, USA Ayse Gurses, USA Vicki L. Hanson, UK Sheue-Ling Hwang, Taiwan Wonil Hwang, Korea Yong Gu Ji, Korea Steven Landry, USA
Gitte Lindgaard, Canada Chen Ling, USA Yan Liu, USA Chang S. Nam, USA Celestine A. Ntuen, USA Philippe Palanque, France P.L. Patrick Rau, P.R. China Ling Rothrock, USA Guangfeng Song, USA Steffen Staab, Germany Wan Chul Yoon, Korea Wenli Zhu, P.R. China
Engineering Psychology and Cognitive Ergonomics Program Chair: Don Harris Guy A. Boy, USA John Huddlestone, UK Kenji Itoh, Japan Hung-Sying Jing, Taiwan Ron Laughery, USA Wen-Chin Li, Taiwan James T. Luxhøj, USA
Nicolas Marmaras, Greece Sundaram Narayanan, USA Mark A. Neerincx, The Netherlands Jan M. Noyes, UK Kjell Ohlsson, Sweden Axel Schulte, Germany Sarah C. Sharples, UK
VII
VIII
Foreword
Neville A. Stanton, UK Xianghong Sun, P.R. China Andrew Thatcher, South Africa
Matthew J.W. Thomas, Australia Mark Young, UK
Universal Access in Human–Computer Interaction Program Chair: Constantine Stephanidis Julio Abascal, Spain Ray Adams, UK Elisabeth André, Germany Margherita Antona, Greece Chieko Asakawa, Japan Christian Bühler, Germany Noelle Carbonell, France Jerzy Charytonowicz, Poland Pier Luigi Emiliani, Italy Michael Fairhurst, UK Dimitris Grammenos, Greece Andreas Holzinger, Austria Arthur I. Karshmer, USA Simeon Keates, Denmark Georgios Kouroupetroglou, Greece Sri Kurniawan, USA
Patrick M. Langdon, UK Seongil Lee, Korea Zhengjie Liu, P.R. China Klaus Miesenberger, Austria Helen Petrie, UK Michael Pieper, Germany Anthony Savidis, Greece Andrew Sears, USA Christian Stary, Austria Hirotada Ueda, Japan Jean Vanderdonckt, Belgium Gregg C. Vanderheiden, USA Gerhard Weber, Germany Harald Weber, Germany Toshiki Yamaoka, Japan Panayiotis Zaphiris, UK
Virtual and Mixed Reality Program Chair: Randall Shumaker Pat Banerjee, USA Mark Billinghurst, New Zealand Charles E. Hughes, USA David Kaber, USA Hirokazu Kato, Japan Robert S. Kennedy, USA Young J. Kim, Korea Ben Lawson, USA
Gordon M. Mair, UK Miguel A. Otaduy, Switzerland David Pratt, UK Albert “Skip” Rizzo, USA Lawrence Rosenblum, USA Dieter Schmalstieg, Austria Dylan Schmorrow, USA Mark Wiederhold, USA
Internationalization, Design and Global Development Program Chair: Nuray Aykin Michael L. Best, USA Ram Bishu, USA Alan Chan, Hong Kong Andy M. Dearden, UK
Susan M. Dray, USA Vanessa Evers, The Netherlands Paul Fu, USA Emilie Gould, USA
Foreword
Sung H. Han, Korea Veikko Ikonen, Finland Esin Kiris, USA Masaaki Kurosu, Japan Apala Lahiri Chavan, USA James R. Lewis, USA Ann Light, UK James J.W. Lin, USA Rungtai Lin, Taiwan Zhengjie Liu, P.R. China Aaron Marcus, USA Allen E. Milewski, USA
Elizabeth D. Mynatt, USA Oguzhan Ozcan, Turkey Girish Prabhu, India Kerstin Röse, Germany Eunice Ratna Sari, Indonesia Supriya Singh, Australia Christian Sturm, Spain Adi Tedjasaputra, Singapore Kentaro Toyama, India Alvin W. Yeo, Malaysia Chen Zhao, P.R. China Wei Zhou, P.R. China
Online Communities and Social Computing Program Chairs: A. Ant Ozok, Panayiotis Zaphiris Chadia N. Abras, USA Chee Siang Ang, UK Amy Bruckman, USA Peter Day, UK Fiorella De Cindio, Italy Michael Gurstein, Canada Tom Horan, USA Anita Komlodi, USA Piet A.M. Kommers, The Netherlands Jonathan Lazar, USA Stefanie Lindstaedt, Austria
Gabriele Meiselwitz, USA Hideyuki Nakanishi, Japan Anthony F. Norcio, USA Jennifer Preece, USA Elaine M. Raybourn, USA Douglas Schuler, USA Gilson Schwartz, Brazil Sergei Stafeev, Russia Charalambos Vrasidas, Cyprus Cheng-Yen Wang, Taiwan
Augmented Cognition Program Chair: Dylan D. Schmorrow Andy Bellenkes, USA Andrew Belyavin, UK Joseph Cohn, USA Martha E. Crosby, USA Tjerk de Greef, The Netherlands Blair Dickson, UK Traci Downs, USA Julie Drexler, USA Ivy Estabrooke, USA Cali Fidopiastis, USA Chris Forsythe, USA Wai Tat Fu, USA Henry Girolamo, USA
Marc Grootjen, The Netherlands Taro Kanno, Japan Wilhelm E. Kincses, Germany David Kobus, USA Santosh Mathan, USA Rob Matthews, Australia Dennis McBride, USA Robert McCann, USA Jeff Morrison, USA Eric Muth, USA Mark A. Neerincx, The Netherlands Denise Nicholson, USA Glenn Osga, USA
IX
X
Foreword
Dennis Proffitt, USA Leah Reeves, USA Mike Russo, USA Kay Stanney, USA Roy Stripling, USA Mike Swetnam, USA Rob Taylor, UK
Maria L.Thomas, USA Peter-Paul van Maanen, The Netherlands Karl van Orden, USA Roman Vilimek, Germany Glenn Wilson, USA Thorsten Zander, Germany
Digital Human Modeling Program Chair: Vincent G. Duffy Karim Abdel-Malek, USA Thomas J. Armstrong, USA Norm Badler, USA Kathryn Cormican, Ireland Afzal Godil, USA Ravindra Goonetilleke, Hong Kong Anand Gramopadhye, USA Sung H. Han, Korea Lars Hanson, Sweden Pheng Ann Heng, Hong Kong Tianzi Jiang, P.R. China
Kang Li, USA Zhizhong Li, P.R. China Timo J. Määttä, Finland Woojin Park, USA Matthew Parkinson, USA Jim Potvin, Canada Rajesh Subramanian, USA Xuguang Wang, France John F. Wiechel, USA Jingzhou (James) Yang, USA Xiu-gan Yuan, P.R. China
Human Centered Design Program Chair: Masaaki Kurosu Gerhard Fischer, USA Tom Gross, Germany Naotake Hirasawa, Japan Yasuhiro Horibe, Japan Minna Isomursu, Finland Mitsuhiko Karashima, Japan Tadashi Kobayashi, Japan
Kun-Pyo Lee, Korea Loïc Martínez-Normand, Spain Dominique L. Scapin, France Haruhiko Urokohara, Japan Gerrit C. van der Veer, The Netherlands Kazuhiko Yamazaki, Japan
In addition to the members of the Program Boards above, I also wish to thank the following volunteer external reviewers: Gavin Lew from the USA, Daniel Su from the UK, and Ilia Adami, Ioannis Basdekis, Yannis Georgalis, Panagiotis Karampelas, Iosif Klironomos, Alexandros Mourouzis, and Stavroula Ntoa from Greece. This conference could not have been possible without the continuous support and advice of the Conference Scientific Advisor, Prof. Gavriel Salvendy, as well as the dedicated work and outstanding efforts of the Communications Chair and Editor of HCI International News, Abbas Moallem.
Foreword
XI
I would also like to thank for their contribution toward the organization of the HCI International 2009 conference the members of the Human–Computer Interaction Laboratory of ICS-FORTH, and in particular Margherita Antona, George Paparoulis, Maria Pitsoulaki, Stavroula Ntoa, and Maria Bouhli. Constantine Stephanidis
HCI International 2011
The 14th International Conference on Human–Computer Interaction, HCI International 2011, will be held jointly with the affiliated conferences in the summer of 2011. It will cover a broad spectrum of themes related to human–computer interaction, including theoretical issues, methods, tools, processes and case studies in HCI design, as well as novel interaction techniques, interfaces and applications. The proceedings will be published by Springer. More information about the topics, as well as the venue and dates of the conference, will be announced through the HCI International Conference series website: http://www.hci-international.org/
General Chair Professor Constantine Stephanidis University of Crete and ICS-FORTH Heraklion, Crete, Greece Email:
[email protected]
Table of Contents
Part I: Mobile Interaction BigKey: A Virtual Keyboard for Mobile Devices . . . . . . . . . . . . . . . . . . . . . Khaldoun Al Faraj, Mustapha Mojahid, and Nadine Vigouroux
3
TringIt: Easy Triggering of Web Actions from a Phone . . . . . . . . . . . . . . . . Vinod Anupam
11
Context Awareness and Perceived Interactivity in Multimedia Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiao Dong and Pei-Luen Patrick Rau
21
Human Computer Interaction with a PIM Application: Merging Activity, Location and Social Setting into Context . . . . . . . . . . . . . . . . . . . Tor-Morten Grønli and Gheorghita Ghinea
30
CLURD: A New Character-Inputting System Using One 5-Way Key Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hyunjin Ji and Taeyong Kim
39
Menu Design in Cell Phones: Use of 3D Menus . . . . . . . . . . . . . . . . . . . . . . Kyungdoh Kim, Robert W. Proctor, and Gavriel Salvendy
48
Mobile Interfaces in Tangible Mnemonics Interaction . . . . . . . . . . . . . . . . . Thorsten Mahler, Marc Hermann, and Michael Weber
58
Understanding the Relationship between Requirements and Context Elements in Mobile Collaboration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sergio Ochoa, Rosa Alarcon, and Luis Guerrero Continuous User Interfaces for Seamless Task Migration . . . . . . . . . . . . . . Pardha S. Pyla, Manas Tungare, Jerome Holman, and Manuel A. P´erez-Qui˜ nones A Study of Information Retrieval of En Route Display of Fire Information on PDA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weina Qu, Xianghong Sun, Thomas Plocher, and Li Wang A Mobile and Desktop Application for Enhancing Group Awareness in Knowledge Work Teams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Timo Saari, Kari Kallinen, Mikko Salminen, Niklas Ravaja, and Marco Rapino A Study of Fire Information Detection on PDA Device . . . . . . . . . . . . . . . Xianghong Sun, Weina Qu, Thomas Plocher, and Li Wang
67 77
86
95
105
XVI
Table of Contents
Empirical Comparison of Task Completion Time between Mobile Phone Models with Matched Interaction Sequences . . . . . . . . . . . . . . . . . . . Shunsuke Suzuki, Yusuke Nakao, Toshiyuki Asahi, Victoria Bellotti, Nick Yee, and Shin’ichi Fukuzumi
114
Part II: In-Vehicle Interaction and Environment Navigation Nine Assistant Guiding Methods in Subway Design – A Research of Shanghai Subway Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Linong Dai
125
Pull and Push: Proximity-Aware User Interface for Navigating in 3D Space Using a Handheld Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mingming Fan and Yuanchun Shi
133
A Study on the Design of Voice Navigation of Car Navigation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chih-Fu Wu, Wan-Fu Huang, and Tung-Chen Wu
141
Front Environment Recognition of Personal Vehicle Using the Image Sensor and Acceleration Sensors for Everyday Computing . . . . . . . . . . . . . Takahiro Matsui, Takeshi Imanaka, and Yasuyuki Kono
151
Common Interaction Schemes for In-Vehicle User-Interfaces . . . . . . . . . . . Simon Nestler, Marcus T¨ onnis, and Gudrun Klinker
159
Dynamic Maps for Future Navigation Systems: Agile Design Exploration of User Interface Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Volker Paelke and Karsten Nebe
169
Flight Searching – A Comparison of Two User-Interface Design Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Antti Pirhonen and Niko Kotilainen
179
Agent-Based Driver Abnormality Estimation . . . . . . . . . . . . . . . . . . . . . . . . Tony Poitschke, Florian Laquai, and Gerhard Rigoll Enhancing the Accessibility of Maps with Personal Frames of Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Falko Schmid
189
199
Augmented Interaction and Visualization in the Automotive Domain . . . Roland Spies, Markus Ablaßmeier, Heiner Bubb, and Werner Hamberger
211
Proposal of a Direction Guidance System for Evacuation . . . . . . . . . . . . . . Chikamune Wada, Yu Yoneda, and Yukinobu Sugimura
221
Table of Contents
A Virtual Environment for Learning Aiport Emergency Management Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Telmo Zarraonandia, Mario Rafael Ruiz Vargas, Paloma D´ıaz, and Ignacio Aedo
XVII
228
Part III: Agents, Avatars and Personalisation User Profiling for Web Search Based on Biological Fluctuation . . . . . . . . . Yuki Arase, Takahiro Hara, and Shojiro Nishio Expression of Personality through Avatars: Analysis of Effects of Gender and Race on Perceptions of Personality . . . . . . . . . . . . . . . . . . . . . . Jennifer Cloud-Buckner, Michael Sellick, Bhanuteja Sainathuni, Betty Yang, and Jennie Gallimore User-Definable Rule Description Framework for Autonomous Actor Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Narichika Hamaguichi, Hiroyuki Kaneko, Mamoru Doke, and Seiki Inoue Cognitive and Emotional Characteristics of Communication in Human-Human/Human-Agent Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . Yugo Hayashi and Kazuhisa Miwa
239
248
257
267
Identification of the User by Analyzing Human Computer Interaction . . . R¨ udiger Heimg¨ artner
275
The Anticipation of Human Behavior Using “Parasitic Humanoid” . . . . . Hiroyuki Iizuka, Hideyuki Ando, and Taro Maeda
284
Modeling Personal Preferences on Commodities by Behavior Log Analysis with Ubiquitous Sensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Naoki Imamura, Akihiro Ogino, and Toshikazu Kato A System to Construct an Interest Model of User Based on Information in Browsed Web Page by User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kosuke Kawazu, Masakazu Murao, Takeru Ohta, Masayoshi Mase, and Takashi Maeno
294
304
Adaptive User Interfaces for the Clothing Retail . . . . . . . . . . . . . . . . . . . . . Karim Khakzar, Jonas George, and Rainer Blum
314
Implementing Affect Parameters in Personalized Web-Based Design . . . . Zacharias Lekkas, Nikos Tsianos, Panagiotis Germanakos, Constantinos Mourlas, and George Samaras
320
XVIII
Table of Contents
Modeling of User Interest Based on Its Interaction with a Collaborative Knowledge Management System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jaime Moreno-Llorena, Xavier Alam´ an Rold´ an, and Ruth Cobos Perez
330
Some Pitfalls for Developing Enculturated Conversational Agents . . . . . . Matthias Rehm, Elisabeth Andr´e, and Yukiko Nakano
340
Comparison of Different Talking Heads in Non-Interactive Settings . . . . . Benjamin Weiss, Christine K¨ uhnel, Ina Wechsung, Sebastian M¨ oller, and Sascha Fagel
349
Video Content Production Support System with Speech-Driven Embodied Entrainment Character by Speech and Hand Motion Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Michiya Yamamoto, Kouzi Osaki, and Tomio Watanabe Autonomous Turn-Taking Agent System Based on Behavior Model . . . . . Masahide Yuasa, Hiroko Tokunaga, and Naoki Mukawa
358
368
Part IV: Ambient Interaction An Interoperable Concept for Controlling Smart Homes – The ASK-IT Paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Evangelos Bekiaris, Kostas Kalogirou, Alexandros Mourouzis, and Mary Panou Towards Ambient Augmented Reality with Tangible Interfaces . . . . . . . . . Mark Billinghurst, Rapha¨el Grasset, Hartmut Seichter, and Andreas D¨ unser Rapid Prototyping of an AmI-Augmented Office Environment Demonstrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimitris Grammenos, Yannis Georgalis, Nikolaos Partarakis, Xenophon Zabulis, Thomas Sarmis, Sokratis Kartakis, Panagiotis Tourlakis, Antonis Argyros, and Constantine Stephanidis Challenges for User Centered Smart Environments . . . . . . . . . . . . . . . . . . . Fabian Hermann, Roland Blach, Doris Janssen, Thorsten Klein, Andreas Schuller, and Dieter Spath Point and Control: The Intuitive Method to Control Multi-device with Single Remote Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sung Soo Hong and Ju Il Eom New Integrated Framework for Video Based Moving Object Tracking . . . Md. Zahidul Islam, Chi-Min Oh, and Chil-Woo Lee
377
387
397
407
416
423
Table of Contents
XIX
Object Scanning Using a Sensor Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Soonmook Jeong, Taehoun Song, Gihoon Go, Keyho Kwon, and Jaewook Jeon
433
Mixed Realities – Virtual Object Lessons . . . . . . . . . . . . . . . . . . . . . . . . . . . Andreas Kratky
440
New Human-Computer Interactions Using Tangible Objects: Application on a Digital Tabletop with RFID Technology . . . . . . . . . . . . . S´ebastien Kubicki, Sophie Lepreux, Yoann Lebrun, Philippe Dos Santos, Christophe Kolski, and Jean Caelen
446
Context-Aware Cognitive Agent Architecture for Ambient User Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Youngho Lee, Choonsung Shin, and Woontack Woo
456
An Embodied Approach for Engaged Interaction in Ubiquitous Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mark O. Millard and Firat Soylu
464
Generic Framework for Transforming Everyday Objects into Interactive Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Elena Mugellini, Omar Abou Khaled, St´ephane Pierroz, Stefano Carrino, and Houda Chabbi Drissi mæve – An Interactive Tabletop Installation for Exploring Background Information in Exhibitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Till Nagel, Larissa Pschetz, Moritz Stefaner, Matina Halkia, and Boris M¨ uller
473
483
Relationality Design toward Enriched Communications . . . . . . . . . . . . . . . Yukiko Nakano, Masao Morizane, Ivan Tanev, and Katsunori Shimohara
492
Ultra Compact Laser Based Projectors and Imagers . . . . . . . . . . . . . . . . . . Harald Schenk, Thilo Sandner, Christian Drabe, Michael Scholles, Klaus Frommhagen, Christian Gerwig, and Hubert Lakner
501
Understanding the Older User of Ambient Technologies . . . . . . . . . . . . . . . Andrew Sixsmith
511
Multi-pointing Method Using a Desk Lamp and Single Camera for Effective Human-Computer Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Taehoun Song, Thien Cong Pham, Soonmook Jung, Jihwan Park, Keyho Kwon, and Jaewook Jeon Communication Grill/Salon: Hybrid Physical/Digital Artifacts for Stimulating Spontaneous Real World Communication . . . . . . . . . . . . . . . . . Koh Sueda, Koji Ishii, Takashi Miyaki, and Jun Rekimoto
520
526
XX
Table of Contents
Motion Capture System Using an Optical Resolver . . . . . . . . . . . . . . . . . . . Takuji Tokiwa, Masashi Yoshidzumi, Hideaki Nii, Maki Sugimoto, and Masahiko Inami
536
The Effects of an Anti-glare Sleeve Installed on Fluorescent Tube Lamps on Glare and Reading Comfort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shiaw-Tsyr Uang, Cheng-Li Liu, and Mali Chang
544
Electromyography Focused on Passiveness and Activeness in Embodied Interaction: Toward a Novel Interface for Co-creating Expressive Body Movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takabumi Watanabe, Norikazu Matsushima, Ryutaro Seto, Hiroko Nishi, and Yoshiyuki Miwa
554
Part V: Affect, Emotion and Engagement An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Panagiotis D. Bamidis, Christos A. Frantzidis, Evdokimos I. Konstantinidis, Andrej Luneski, Chrysa Lithari, Manousos A. Klados, Charalambos Bratsas, Christos L. Papadelis, and Costas Pappas Addressing the Interplay of Culture and Affect in HCI: An Ontological Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Emmanuel G. Blanchard, Riichiro Mizoguchi, and Susanne P. Lajoie Love at First Encounter – Start-Up of New Applications . . . . . . . . . . . . . . Henning Breuer, Marlene Kettner, Matthias Wagler, Nathalie Preuschen, and Fee Steinhoff
565
575
585
Responding to Learners’ Cognitive-Affective States with Supportive and Shakeup Dialogues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sidney D‘Mello, Scotty Craig, Karl Fike, and Arthur Graesser
595
Trust in Online Technology: Towards Practical Guidelines Based on Experimentally Verified Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Christian Detweiler and Joost Broekens
605
Influence of User Experience on Affectiveness . . . . . . . . . . . . . . . . . . . . . . . . Ryoko Fukuda
615
A Human-Centered Model for Detecting Technology Engagement . . . . . . James Glasnapp and Oliver Brdiczka
621
Relationship Learning Software: Design and Assessment . . . . . . . . . . . . . . Kyla A. McMullen and Gregory H. Wakefield
631
Table of Contents
XXI
Relationship Enhancer: Interactive Recipe in Kitchen Island . . . . . . . . . . . Tsai-Yun Mou, Tay-Sheng Jeng, and Chun-Heng Ho
641
ConvoCons: Encouraging Affinity on Multitouch Interfaces . . . . . . . . . . . . Michael A. Oren and Stephen B. Gilbert
651
Development of an Emotional Interface for Sustainable Water Consumption in the Home . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mehdi Ravandi, Jon Mok, and Mark Chignell Influences of Telops on Television Audiences’ Interpretation . . . . . . . . . . . Hidetsugu Suto, Hiroshi Kawakami, and Osamu Katai
660 670
Extracting High-Order Aesthetic and Affective Components from Composer’s Writings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Akifumi Tokosumi and Hajime Murai
679
Affective Technology, Affective Management, towards Affective Society . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hiroyuki Umemuro
683
Bio-sensing for Emotional Characterization without Word Labels . . . . . . Tessa Verhoef, Christine Lisetti, Armando Barreto, Francisco Ortega, Tijn van der Zant, and Fokie Cnossen An Affect-Sensitive Social Interaction Paradigm Utilizing Virtual Reality Environments for Autism Intervention . . . . . . . . . . . . . . . . . . . . . . . Karla Conn Welch, Uttama Lahiri, Changchun Liu, Rebecca Weller, Nilanjan Sarkar, and Zachary Warren Recognizing and Responding to Student Affect . . . . . . . . . . . . . . . . . . . . . . Beverly Woolf, Toby Dragon, Ivon Arroyo, David Cooper, Winslow Burleson, and Kasia Muldner
693
703
713
Part 6: Smart and Wearable Materials and Devices Usability Studies on Sensor Smart Clothing . . . . . . . . . . . . . . . . . . . . . . . . . Haeng Suk Chae, Woon Jung Cho, Soo Hyun Kim, and Kwang Hee Han Considering Personal Profiles for Comfortable and Efficient Interactions with Smart Clothes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S´ebastien Duval, Christian Hoareau, and Gilsoo Cho Interaction Wearable Computer with Networked Virtual Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jiung-yao Huang, Ming-Chih Tung, Huan-Chao Keh, Ji-jen Wu, Kun-Hang Lee, and Chung-Hsien Tsai
725
731
741
XXII
Table of Contents
The Impact of Different Visual Feedback Presentation Methods in a Wearable Computing Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hendrik Iben, Hendrik Witt, and Ernesto Morales Kluge
752
Gold Coating of a Plastic Optical Fiber Based on PMMA . . . . . . . . . . . . . Seok Min Kim, Sung Hun Kim, Eun Ju Park, Dong Lyun Cho, and Moo Sung Lee
760
Standardization for Smart Clothing Technology . . . . . . . . . . . . . . . . . . . . . . Kwangil Lee and Yong Gu Ji
768
Wearable ECG Monitoring System Using Conductive Fabrics and Active Electrodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Su Ho Lee, Seok Myung Jung, Chung Ki Lee, Kee Sam Jeong, Gilsoo Cho, and Sun K. Yoo Establishing a Measurement System for Human Motions Using a Textile-Based Motion Sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Moonsoo Sung, Keesam Jeong, and Gilsoo Cho
778
784
A Context-Aware AR Navigation System Using Wearable Sensors . . . . . . Daisuke Takada, Takefumi Ogawa, Kiyoshi Kiyokawa, and Haruo Takemura
793
Emotional Smart Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Akira Wakita, Midori Shibutani, and Kohei Tsuji
802
Novel Stretchable Textile-Based Transmission Bands: Electrical Performance and Appearance after Abrasion/Laundering, and Wearability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yoonjung Yang and Gilsoo Cho Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
806
815
BigKey: A Virtual Keyboard for Mobile Devices Khaldoun Al Faraj, Mustapha Mojahid, and Nadine Vigouroux University of Toulouse, IRIT, 31062 Toulouse, France {alfaraj,mojahid,vigourou}@irit.fr
Abstract. This paper describes BigKey virtual keyboard for mobile devices, designed to make the keys of virtual keyboard easier to acquire. The tiny size of keys makes efficient selection difficult. To overcome this drawback, we propose to expand key size that corresponds to next character entry. The proposed solution helps to facilitate the selection task by expanding the next entry. Moreover the prediction system reduces the visual scanning time to find letters that one is looking for. Users’ performance study showed that they were 25.14% faster and more accurate with BigKey virtual keyboard than with normal virtual keyboard. Keywords: Virtual keyboard, text input, PDAs, expanding targets, letter prediction.
1 Introduction An efficient text entry method for mobile devices is becoming one of the most prominent challenges in the world of mobile computing. The shrinking size of handheld devices has resulted in a keyboard that is not as convenient as that of desktop computer. Personnel Digital Assistants (PDAs) and smart phones equipped with a touch screen and a stylus, in general, tend to have alternative text input techniques like handwriting recognition technology and virtual keyboard on the screen. Handwriting recognition systems largely help to overcome the screen-space constraints of mobile computing products. However, user learning of making character strokes is required and that is not always easy, especially for novice users. Virtual keyboard (sometimes called on-screen keyboard) is a reproduction of a hardware keyboard on the screen of computing devices. It was originally designed for people with disabilities to access to computers and for some special needs as well. Also the mobile devices equipped with a touch screen and a stylus, have used it as another text entry solution. The virtual keyboard of handheld device has less number of keys (60) as compared to desktop keyboard (105). It was reduced by modes of labeled keys to enter numbers and special characters. However, the accurate selection of smaller keys still remains difficult. It requires a great amount of attention as user also has to focus on what he/she is writing. J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 3–10, 2009. © Springer-Verlag Berlin Heidelberg 2009
4
K. Al Faraj, M. Mojahid, and N. Vigouroux
A combination of gesture and virtual keyboard has been constructed like Quickwriting [1] and Shark [2]. The main goal of this concept is to allow word level entry with keyboard. Although many investigations have been devoted to find the most efficient text entry method for mobile devices by changing character arrangement and key shapes of virtual keyboard [3] [4]. Less attention has been paid to the key size that we consider as an essential element of efficient mobile text inputting. In this paper, we discuss our BigKey virtual keyboard principle that aims to optimize user’s performance. Subsequently, we will present our preliminary results obtained from a formal study of the proposal.
2 BigKey Virtual Keyboard Users of PDAs and smart phone feel difficulty in using the tiny size of keys while using virtual keyboard for small screen. Our primary design objective is to break this main obstacle that limits user’s performance. The principle of BigKey virtual keyboard is the expanding key size corresponding to the next character entry. This system is primarily designed for the mobile devices such as Ultra Mobile PC (UMPC), PDAs, smart phones and so forth, using their virtual keyboard. However this design is applicable to any computing device supporting the same input pattern. McGuffin and Balakrishnan have proposed an interface design of one-dimensional arrays of widgets consisting of button strip [5]. It is based on expanding a target when the pointer approaches it to facilitate its selection. To solve the drawback of a sideways motion resulting from expanding target, some overlapping between adjacent buttons is allowed. This principle is much more similar to that was proposed by Dasher [6]. Cirrin is a continuous stylus-based text entry technique [7]. The letters of English alphabet are arranged on the circumference of a circle by using the common sequences in English to minimize stylus travel. A word is entered by pressing and moving the stylus over the letters. Expanding targets idea explained above has been applied to Cirrin in order to improve user’s performance [8]. The experimental results indicate that Cirrin is more fast but less accurate. This is due to the occlusion and overlapping between neighboring buttons. Another application of expanding targets principle to virtual keyboard with QWERTY layout, called Fisheye, has been proposed [9]. The aim is always to make selection task easier for PDAs. The character is selected by lifting up the stylus when it is over the key. While analyzing the selection model, we noticed that the successive selection of characters on touch-screen with stylus is accomplished by using three-dimensional mode (3D mode). In other words, user has to lift the stylus between every selection of character (sequence of stylus up and stylus down); hence the stylus moves in threedimensional space. Whereas using the mouse as a pointing device makes selection task in two-dimensional mode (2D mode). Note that the pointing device used to evaluate the measure of performance of expanding targets design mentioned earlier was the mouse.
BigKey: A Virtual Keyboard for Mobile Devices
5
Considering the 3D mode, expanding targets when the stylus is close to them is rarely occurred because of the third dimension (lifting up the stylus). In other words, the expansion of target is mostly occurred when the stylus is over the target. In this case, there is no advantage to expand this key as the stylus is exactly over the key and user can select it if it is expanded or not. Moreover, with each animation of Fisheye keyboard, user has to re-compute the target’s position that makes it more difficult to acquire and more attention is required. According to Fitts’ law [10], the time “MT” to acquire a target of width “W” which lies at a distance or amplitude “A” is given by the relationship [11]:
M MT = a + b log 2 (
A + 1) W
(1)
Where a and b are constants determined through linear regression. The logarithmic term is called the index of difficulty (ID) and is measured in “bits”. As character selection is made from key-to-key, according to Fitt’s law, larger the size of the next key to be selected, shorter the time will be required to acquire it. Our BigKey system is based on two processes: first is to predict the next character entry based on the users previous input, second is to expand the corresponding predicted keys. The keys expand as a function of their letter probability of entry: the most probable the next letter, the largest the key is. To build our prediction system, we have employed tables of single letter and diagram frequency Counts proposed by [12]. Four expanded keys are maximum number used in our BigKey implementation as shown in figure 1.
Fig. 1. BigKey virtual keyboard while selecting the next entry for word “the”
The BigKey system offers two main advantages. On the one hand, it facilitates character selection according to Fitt’s law. On the other hand, it reduces the time of keyboard visual scanning to find the letters that one is looking for, especially for novice users.
6
K. Al Faraj, M. Mojahid, and N. Vigouroux
It could also be effective for the people with motor impairments who need to reduce the acquisition target time. Furthermore, this would help immensely persons with Alzheimer’s disease. As a result our design model makes the selection task easier through expanding the targets of next entry when the stylus is over the previous entry. The regular QWERTY keyboard layout is used as for the user’s familiarity with its layout. Moreover it was originally designed to keep commonly used letter combination farther from each other. In this way the overlapping between expanded keys is not occurred. Unlike hardware keyboard where one can type over by using several fingers. While using a virtual keyboard, one has to carry out all movements by using only a pointing device. In our BigKey proposed solution, the expanded keys can be considered as the most probable keys that user always puts his fingers over. Comparing the most probable next letter predictions for each letter with others, an intersection is mostly detected especially for vowels which are typically proposed as predictions. In hardware keyboard, the most accessible keys are those that are under the fingers. Similarly in our Bigkey system, the expanded keys are those that are the most accessible like the initial fingers configuration, even if the user does not type them when the prediction does not give the desired result.
3 Experiment The aim of this study is to verify our hypothesis: the proposed animation of virtual keyboard had significant effects on user performance for text entry task. 3.1 Subjects Nine volunteers (3 female) from our university campus participated in this study. Users average 27.11 years of age (ranging from 24 to 33 years). They were novice stylus-based text input users. All users had normal or corrected eyesight and all were right-handed to use the stylus as a pointing device. 3.2 Apparatus Users conducted the study on a Sony VAIO UMPC with a 1.2GHz Core Solo processor and 512MB of RAM. The display size was 4.5" SVGA TFT and ran at a resolution of 600 × 800. The pointing device used in our experiment was the stylus. The experiment included the three following virtual keyboards: The virtual keyboard without expanded keys (No-BigKey), the BigKey virtual keyboard with one expanded key (One-BigKey), and the BigKey virtual keyboard with four expanded keys (Four-BigKey). Table 1 shows the key size used in our study, the normal key had the same size of that is used for PDA virtual keyboard. Besides the key of the first most probable letter had the same size of that is used for the virtual keyboard of desktop computer. For each virtual keyboard, the program reads a series of 10 phrases ranging from 16 to 43 characters [13]. All virtual keyboards are built in .NET C#.
BigKey: A Virtual Keyboard for Mobile Devices
7
Table 1. The key size of BigKey virtual keyboard
Key
Size
Normal
18 × 16 pixels
The first most probable letter
26 × 24 pixels (+ 44.44 %)
The second most probable letter
24 × 22 pixels (+ 33.33 %)
The third most probable letter
22 × 20 pixels (+ 22.22 %)
The fourth most probable letter
20 × 18 pixels (+ 11.11 %)
3.3 Procedure The experiment consisted of two parts: training session followed by testing session. The first session consisted of entering the sentence “the quick brown fox jumps over the lazy dog” by using Four-BigKey virtual keyboard. Then, in the testing session each participant completed three sentence tasks using three virtual keyboards. Participants were divided into three-person groups to perform tasks in a different order. The first performed the experiment in following order: No-BigKey, OneBigKey and then Four-BigKey. While the second followed the order: One-BigKey, Four-BigKey and then No-BigKey. The succession of tasks for the third group was as following: Four-BigKey, One-BigKey and then No-BigKey. In this way, the task order had no impact on the results. The same phrases were used for all tasks but were in a different order for each one so that user cannot anticipate the phrase with the other tasks. Participants were instructed to enter the phrases “as quickly and accurately as possible”. They could make errors and corrections.
4 Results and Discussion In order to evaluate the efficiency of text entry technique, two essential metrics are available up till now. The text entry speed expressed in words per minute (wpm) or in characters per second (cps), and the accuracy during and after the text entry task. 4.1 Text Entry Speed The analysis of entry speed yields a significant result in favor of Four-BigKey. Comparing three virtual keyboards, the No-BigKey speed was the slowest 20.84 wpm, the OneBigKey was faster 23.66 wpm, and the Four-BigKey was the fastest 26.08 wpm, as shown in figure 2. The average improvement of speed is 25.14 % with Four-BigKey.
8
K. Al Faraj, M. Mojahid, and N. Vigouroux
This study shows that the fastest text entry speed was achieved with increasing the number of expanded keys. As letter prediction does not always give the intended result, increasing the number of expanded keys over one, was necessary. The question that remains still to be answered: What is the optimal number of expanded keys for each letter?
Fig. 2. Text entry speed for three virtual keyboards
4.2 Accuracy In our study, participants were allowed to enter phrases naturally, thus they may commit errors and make corrections. We measured errors made during text entry and the errors left in transcribed string using Corrected Error Rate and Not Corrected Error Rate metrics respectively [14]. Figures 3&4 show that participants made more corrections than leaving errors for presented text in all tasks. However, error rates were not significantly different between all tasks.
Fig. 3. Corrected error rate for three virtual keyboards
BigKey: A Virtual Keyboard for Mobile Devices
9
Fig. 4. Not corrected error rate for three virtual keyboards
Comparing the three virtual keyboards, the analysis yields the least error rate in favor of the Four-BigKey (see figures 3&4). These results suggest that keys were easier to acquire when expanding them through Four-BigKey.
5 Conclusion and Future Work We have shown that the expanding targets based on letter prediction can be an effective means of making targets easier to acquire for the virtual keyboard of handheld devices. This design offered 25.14% better speed during text entry for the BigKey virtual keyboard over the normal virtual keyboard and higher accuracy at the same time. On the basis of these preliminary results, we are conducting detailed experimentation to explore the optimal number and size of predicted keys. In the future, we are planning to study the efficiency of the BigKey system for people with motor impairment who need to reduce the fatigue of target acquisition. We also intend to explore the impact of expanding the next entry on the recall of the word completion for people with Alzheimer’s disease.
References 1. Perlin, K.: Quikwriting: Continuous Stylus-Based Text Entry. In: UIST 1998, pp. 251– 316. ACM Press, San Francisco (1998) 2. Zhai, S., Kristensson, P.O.: Shorthand Writing on Stylus Keyboard. In: CHI 2003, pp. 97– 104. ACM Press, Ft. Lauderdale (2003) 3. MacKenzie, I.S., Soukoreff, R.W.: Text Entry for Mobile Computing: Models and Methods Theory and Practice. In: Human-Computer Interaction, vol. 17(2), pp. 147–198. Lawrence Erlbaum, Mahwah (2002) 4. Zhai, S., Hunter, M., Smith, B.A.: The Metropolis keyboard: An exploration of quantitative for graphical keyboard design. In: UIST 2000, pp. 119–128. ACM Press, San Diego (2000)
10
K. Al Faraj, M. Mojahid, and N. Vigouroux
5. McGffin, M., Balakrishnan, R.: Acquisition of expanding targets. In: CHI 2002, pp. 57– 64. ACM Press, Minneapolis (2002) 6. Ward, D.J., Blackwell, A.F., Mackay, D.J.C.: Dasher: A Data Entry Interface Using Continuous Gestures and Language Models. In: UIST 2000, pp. 129–137. ACM Press, San Diego (2000) 7. Mankoff, J., Abowd, G.D.: Cirrin: A Word-Level Unistroke Keyboard for Pen Input. In: UIST 1998, pp. 213–214. ACM Press, San Francisco (1998) 8. Cechanowicz, J., Dawson, S., Victor, M., Subramanian, S.: Stylus based text input using expanding CIRRIN. In: Proceedings of the Working Conference on Advanced Visual Interface, AVI 2006, pp. 163–166. ACM Press, New York (2006) 9. Raynal, M., Truillet, P.: Fisheye keyboard: Whole keyboard displayed on PDA. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4551, pp. 452–459. Springer, Heidelberg (2007) 10. Fitts, P.M.: The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47(6), 381–391 (1954) 11. MacKenzie, I.S.: Fitts’ Law as a Research and Design Tool in Human-Computer Interaction. In: Human-Computer Interaction, vol. 7(1), pp. 91–139. Lawrence Erlbaum, Mahwah (1992) 12. Mayzner, M.S., Tresselt, M.E.: Tables of Single-Letter and Diagram Frequency Counts for Various Word-Length and Letter-Position Combinations. Psychonomic Monograph Supplements 1(2), 13–32 (1965) 13. MacKenzie, I.S., Soukoreff, R.W.: Phrase set for evaluating text entry techniques. In: CHI 2003, pp. 754–755. ACM Press, Ft. Lauderdale (2003) 14. Soukoreff, R.W., MacKenzie, I.S.: Metrics for text entry research: An evaluation of MSD and KSPC, and a new unified error metric. In: CHI 2003, pp. 113–120. ACM Press, Ft. Lauderdale (2003)
TringIt: Easy Triggering of Web Actions from a Phone Vinod Anupam Anexas Inc, 67 Shields Ln, Bridgewater NJ 08807, USA
[email protected]
Abstract. Much information that is of interest to mobile users is available on the Web, yet is difficult to access for most users. We introduce a novel method for users to interact with network-connected computers using their phones, and describe a system called TringIt that implements the method. TringIt enables users to trigger Web actions by simply dialing specific numbers – an action that we call a Phone Click. TringIt can be used out-of-the-box by any phone user. The Phone Click is most useful from mobile phones that can receive messages in response to the click. TringIt enables users to easily initiate interaction with businesses and content owners by simply dialing numbers discovered in offline media (e.g. print, TV, radio) as well as online media (e.g. Web, SMS, MMS.) It makes every mobile phone a more compelling information, interaction and participation device. Keywords: Phone Click, Tring, Dial-to-Click, Call Triggered Messaging, Userto-Application Signaling, SMS/MMS Click-through, Dial-able hyperlinks.
1 Introduction Dialing phone numbers is the easiest thing that users can do from a mobile phone. Users manually dial numbers, dial them out of address books, speed dial them, voicedial them, and even "click" on phone numbers in messages and Web pages to dial them. Phone-car integration systems (including e.g. Microsoft Sync) allow users to place calls hands-free in their automobiles. So far, users have been able to do one of two things by dialing a number: set up a voice call (to interact via voice with another user or with an Interactive Voice Response system) or set up a dial-up connection (to interact with a remote system via data modulated over the voice channel). In this paper, we describe a new way in which users can interact with networkconnected computers using their phones. We introduce the notion of a “Phone Click” – the triggering of a Web action in response to the dialing of a number. We describe a system called TringIt that enables any mobile phone user to interact with Web applications in this easy-to-use yet powerful way, and discuss how it is used to easily request information. The paper is organized as follows. In Section 2 we discuss several capabilities of mobile phones, some of which are leveraged by the new technique. In Section 3 we describe how phone signaling is used for user-to-application signaling, and describe some applications. Finally, we summarize our conclusions and discuss upcoming work. J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 11–20, 2009. © Springer-Verlag Berlin Heidelberg 2009
12
V. Anupam
2 Background and Motivation The mobile phone is the most widely used connected device. Over 3.9 billion (and growing) mobile phone users worldwide connect to the "network" - including the Public Switched Telephone Network (PSTN) and the Internet - via their mobile phones. Importantly, every mobile phone – even a relatively basic feature-phone – is a very sophisticated, network-connected computer. For most users, however, it is still difficult and/or expensive to request and receive information in their phone – the device is vastly under-utilized, compared to its potential! Most users use their mobile phone just to make voice calls and to send and receive messages. While the number of mobile users who access the Internet from their mobile phones is steadily increasing, adoption is still low in most markets (other than Japan and South Korea, which have seen significant adoption.) In the US, about 16% of subscribers use applications that access the Internet from their mobile phones [9]. Adoption is even lower in other mature markets - e.g., 10% in France, 7% in Germany – and especially in emerging markets - e.g., 2% in India, 7% in China. Many emerging mobile Web/Internet based applications have been unable to reach critical mass because limited adoption of mobile Internet leaves little room for network effects to kick in. Key barriers to adoption include: • Cost - User-incurred cost of data service is a key issue. • Usability - Mobile web browsing poses significant usability challenges. Mobileoptimized Web sites are still more the exception than the rule, and most mobileoptimized applications are not yet widely used. It is reasonable to posit that information solutions that work for all users, are easy to use and also are sensitive to user-incurred cost have a higher likelihood of adoption than those that are not. Mobile phone subscribers typically pay for calls that they place from their mobile phone, for messages that they send from their phone as well as for data connections initiated from their phone. In many parts of the world (e.g. Europe, Africa and Asia), incoming calls and incoming messages are free to the user, while in other parts of the world (e.g. USA) subscribers pay for incoming calls as well as incoming messages. Messaging uptake and usage has historically been high in markets where calls are/were significantly more expensive than messaging. This can be attributed to the subscriber's desire to minimize cost of communication, even at the cost of poor usability (e.g. triple-tapping to create messages.). Let us quickly look at the main communication channels available to users in their mobile phone in terms of their reach, their relative cost and their key deficiencies. 2.1 Phone Calls Voice calls dominate user-generated traffic in mobile phone networks, and continue to be the mainstay of mobile network operator revenue worldwide. Users call to interact with other users and businesses. Phones are optimized for voice calls, and provide capabilities like address books, speed dialing, voice dialing etc. to streamline communication via voice calls.
TringIt: Easy Triggering of Web Actions from a Phone
13
Businesses widely use phone numbers (often toll free numbers, like 1-800 numbers in the US) that users can call to speak with representatives, customer service agents etc. to receive information and to trigger interactions. While most voice calls are person-to-person calls, use of Interactive Voice Response (IVR) [7] systems is the second largest use case. Sophisticated IVR applications are widely used worldwide – reducing the need for expensive human agents. Technologies like VoiceXML [12] allow users to interact with these systems using voice and/or touch-tone input. IVR systems interconnected to the Web provide a powerful channel for information retrieval by all phone users. Universal reach (any phone user can make voice calls) is a key strength of the voice call. Additionally, phone numbers are familiar, and easily disseminated via both offline and online channels. However, there are some key deficiencies of voice as a channel for requesting and receiving information. Voice is the most expensive channel in most markets. Information received via voice does not persist in the phone for subsequent use (the user must either remember what he was told by the other party, or must transcribe it onto another medium - e.g., by writing something down on paper.) IVR systems are somewhat cumbersome to navigate. Finally, voice interfaces are not optimal for use in noisy environments – typical of mobile usage. 2.2 Messaging Messaging is the second most frequently used communication channel from mobile phones. Users transmit and receive billions of messages every day. Messaging is analogous to email – messages created by a sending party are transmitted using a store-and-forward metaphor towards the target party. SMS (Short Message Service) [1] is the most popular form of messaging. Since it is limited to a small payload of 160 7-bit characters, SMS is frequently used to send short messages and status updates. MMS (Multimedia Message Service) [10] is a high-capacity messaging channel that has support for larger messages possibly containing images and/or short video clips. While MMS message size limits are sometimes imposed by operators, typical limits are upwards of 100K bytes. Messaging initially offered an economical communication channel alternative to the much more expensive voice call. The benefit of lower cost outweighed poor usability - the fairly cumbersome process of multi-tapping messages via a phone's numeric keypad. However, messaging has some key strengths. It works in every mobile phone. It is less intrusive than voice calls. It is very easy to receive messages in phones. And much application infrastructure now exists allowing messaging systems to be interfaced to Web systems for a variety of applications. However, messaging has drawback: it is still somewhat cumbersome to create a message by multi-tapping. A typical call to action displayed in a magazine or TV ad, or transmitted via SMS/MMS is of the form "TEXT PIZZA to short code 123456 to receive a coupon.” It takes about 18 button presses to react to that call to action on a typical phone. And the vocabulary for interacting with different application servers varies greatly, imposing a cognitive load. Collectively, these present a usability barrier for users who want to respond to the call to action.
14
V. Anupam
2.3 USSD Unstructured Supplementary Services Data [2] is a communication channel available in GSM networks. Users can manually (or via an in-phone client application) create and send USSD messages, and can receive/review USSD messages via a notifier application. Unlike SMS, which uses transaction-based store-and-forward communication, USSD uses session based point-to-point communication and is therefore much faster. The key strengths of USSD are its wide deployment on GSM networks (every GSM phone is capable of sending and receiving USSD messages) and its low cost. The key drawbacks there are no standard vocabularies for interacting with application servers to request information, so messages are cumbersome to create. Additionally, USSD application server infrastructure is not as widely deployed as SMS application server infrastructure. And USSD does not work in CDMA networks. 2.4 Mobile Internet/Web Internet access from mobile phones is a powerful communication channel. Modern mobile networks provide high bandwidth and low latency Internet access. Most modern mobile phones have a mobile Web browser built in, and mobile-ready Web sites can provide a compelling interactive user experience. However, the mobile phone is a very constrained device, with a small screen and limited input capabilities. The browser in a typical mobile phone is significantly less sophisticated than a PC browser (but this is changing with high end phones like the Apple iPhone). Most Web sites are not mobile ready, leading to a very poor user experience when accessed from a mobile phone. These factors collectively make it a low-reach information channel - most mobile phone users do not use the mobile Web. To address usability concerns in mobile phones, a variety of in-phone client applications are being built and deployed. Clients that let users interact with email as well as instant messaging (IM) systems are available in many phones, and have seen some uptake. However, application infrastructure that enables interfacing these channels with Web servers is limited. Emerging mobile widget solutions (like Yahoo Go! [13]) provide the user a mobile-optimized information browsing experience, but have seen limited uptake. Such solutions, however, do provide infrastructure for interfacing with Web applications.
3 TringIt: Dial to Click Dialing a number is the easiest thing that a user can do from a mobile phone. And every mobile phone can receive messages. We exploit these attributes to create a solution that is both easy to use and usable by the entire mobile phone user population. The key idea is to use telephone signaling for application signaling, and to use messaging for information delivery. The PSTN (Public Switched Telephone Network) is the interconnection of POTS (Plain Old Telephony System) networks and PLMN (Public Land Mobile Network) networks operated by multiple network operators. To support VoIP (Voice Over IP), the PSTN interconnects to the Internet.
TringIt: Easy Triggering of Web Actions from a Phone
15
Modern phone networks use separate signaling and "data" channels and can be thought of as two parallel networks - the Common Channel Signaling network that uses protocols like SS7 (Signaling System 7) [6] to set up and tear down calls, and the "media" network that carries audio "data." Many networks now have all-IP cores. From the user perspective, however, the details are irrelevant. Users simply care about the fact that the call gets set up to the appropriate party when they place a call. A call in a telephone network proceeds in two stages: • A "signaling" stage in which the phone network locates and rings the called phone and provides feedback to the calling phone. • A "communication" stage involving voice transport between the calling party and the called party, possibly with data modulated over the voice channel. Importantly, the telephone network transmits two key pieces of information in the signaling stage of a call - the calling number and the called number. Dialed Number Identification Service (DNIS) [11] is a telephone network service that lets a subscriber of the service determine which telephone number was actually dialed by a caller. Automatic Number Identification (ANI) [4] is a telephone network service that permits subscribers of the service to capture the telephone number of a calling party. Via such services, call signaling automatically transmits a small amount of information from the calling party’s network to the called party’s network. 3.1 Similarity between Phone Signaling and Web Requests Interestingly, a user's interaction to place a call via the telephone network can be likened to Web request. We can think of a phone number as a URL, the phone as a browser, and the PSTN as an amalgamation of Internet infrastructure (name servers, routers etc.) and Web servers. A user uses his phone (the browser) to manually enter a number (a URL) or to speed-dial a number (a "browser bookmark"). The phone (the browser) transmits a request into the phone network. The request includes information about the called party (the URL) and the calling party (a Header in the request). The phone network attempts to route the call to the appropriate party (like the browser uses DNS to identify the IP address of the target server) and delivers signaling information to the called party (like a Web request being delivered to the server). If there is an error during this process (e.g., invalid number) appropriate feedback is provided to the calling party (like a “404 Not Found” error in HTTP). Otherwise, the phone starts the "voice client" that allows the user to listen to ringing tones. If the call is answered by the called party, all subsequent interaction happens via bi-directional streaming over a voice channel that is set up as a side effect of successful signaling. (This is equivalent to the browser launching a helper application that supports bidirectional voice.) Placing a phone call, therefore, can thus be thought of as a special kind of Web "click" from a phone. So far, this click has had very limited range of behavior - the click either results in the voice call being established or failing. (Setting up of a dialup connection is a special case, where a voice channel is first established, and the established channel is used to modulate/demodulate data over a carrier signal.).
16
V. Anupam
TringIt generalizes the use of telecom signaling, enabling it to drive any Internetconnected application. In particular, it enables rich phone + Internet converged applications [8] that work with any phone. TringIt enables mobile phone users to interact with Web applications in a powerful yet easy-to-use way by enabling Web actions to be triggered by simply dialing numbers - "Dial to Click!" TringIt is most useful for users who do not yet use the mobile Internet - it uses SMS/MMS to deliver information to users in response to the received phone call. However, TringIt is also useful for mobile phone users who currently use the mobile Internet. Numbers can be discovered via any media - online, TV, print etc. In a few button presses from an idle screen - the phone number + SEND, the user can "connect" with the real world entity associated with that number. A mobile phone with Internet access can be driven to the appropriate Web site. 3.2 Signaling Numbers: Phone Numbers as URLs As described in the preceding section, dialing a number to place a phone call can be likened to making an HTTP request. A phone number can therefore be thought of as a URL! So far, these URLs have simply been used to trigger the setting up of a voice call. We know that any action on the Web can be represented as a URL [5] - data needed for that action can either be encoded into the URL used for an HTTP GET request, or can be stored in a descriptor that is used to generate an HTTP POST request to the appropriate server. A phone number can be associated with such a URL by simply maintaining a lookup table mapping the phone number to the URL. If the system can detect when the phone number is called, it can use the lookup table to retrieve the associated URL and can trigger the appropriate Web action. This fundamentally changes what users can do by calling a number! The behavior of such a phone number is different from a regular phone number that is used to set up a voice call. We refer to this kind of number as a "Phone Signaling Number" (more concisely, "Signaling Number.” Any Web action can be associated with a Signaling Number, and thus be triggered by a simple phone call from any phone - wired or mobile. Importantly, this requires no new software in the phone, and thus works out-of-the-box with every phone in use on every network. If the associated Web action causes the generation and transmission of e.g. an SMS/MMS message to the phone, the calling user can dynamically receive "feedback" for the call – possibly containing requested information! The numeric form of this new kind of URL is ideal for easy input on the phone. Its compact and familiar form makes it easy to display and disseminate via any medium - books, magazines, newspapers, flyers, packaging, receipts, TV, radio, video and audio content, Web sites, email, SMS/MMS messages etc. And conventional letters-to-number mnemonics (1-777-4PIZZA to represent 1-777-474992) can make some subset of the numbers self-describing. These numeric URLs – “dial-able hyperlinks” – can easily be direct dialed from an idle phone screen. They can be stored in phone address books and can be speed-dialed and voice-dialed. They can be communicated from user to user. They can be embedded in SMS/MMS/email messages or in Web content and can be easily clicked by the receiving user.
TringIt: Easy Triggering of Web Actions from a Phone
17
3.3 Tring: A Phone Click As described earlier, when a call is being set up in the phone network, the calling party’s network transmits a small amount of information to the called party. This information can be used to create tremendous value. If the called party is associated with multiple phone numbers, the calling party can signal different things to the called party, based on the number it calls. If the calling party and called party can maintain state - i.e. remember information about previous transmissions, the interaction gets richer. E.g. if the calling party calls different numbers in some sequence it can convey additional information to the called party. Additionally, if the calling party and called party have shared context - i.e. some persistent (non-transient) data that both parties share (that is possibly transmitted and shared between the calling party and the called party using some separate offline mechanism) - the interaction gets even richer. The shared context allows specific meaning to be attributed to calls to and from different numbers, and the calling and called parties can be aware of that meaning. The ability to exchange user-to-application signals in the presence of state as well as shared context enables very compelling applications. TringIt enables the act of placing a call to a Signaling Number to be interpreted as the intent of the calling party to trigger the associated action registered by the called party. The associated Web action can equivalently be triggered by the called party or by any entity along the signaling path between the calling party and the called party. This makes the humble phone call much more versatile. A simple call can be used to trigger any Web action that has been associated with the number – a “Phone Click.”
TringIt Server Mobile Phone Network
1
3
Phone Network
Internet
2
Web Server Fig. 1. The Versatile “Phone Click”
Figure 1 depicts the flow of a typical Phone Click. In Step 1, signaling information travels over the mobile phone network and the PSTN to a TringIt Server when a call is placed to a Signaling Number. The TringIt Server looks up and triggers the corresponding Web action in Step 2, over the Internet. In Step 3, the TringIt Server transmits a message to the calling phone via the Internet and mobile phone network. The TringIt Server is a network-based server that interfaces to the phone network to receive Phone Clicks - calls that are used to trigger Web actions. Details of the TringIt Server are outside the scope of this paper, and are being published separately
18
V. Anupam
[3]. At a high level, the TringIt Server interconnects to the PSTN via physical and logical connections. It receives and terminates phone calls aimed at Signaling Numbers and triggers appropriate Web actions in response. Any phone number can potentially be used as a Signaling Number – the TringIt server just needs to be able to receive the call. Calls can be directed at the TringIt Server by routing or forwarding. For best usability, however, Signaling Numbers should come from a separate part of the phone number-space so that they are visually identifiable as numbers that trigger Web actions (as opposed to those that set up calls.) This can be achieved, e.g., by using numbers from a separate unused area code or country code, to create a concept like 1-800 numbers – users “know” that calls to 1-800 numbers are toll free. An appropriate prefix can be allocated and assigned by the appropriate phone number administration authority that controls and manages the phone numbering space. 3.4 Network-Based Context and Personalization One of the key challenges for non-voice use of mobile phones is usability. Portability and power constraints impose the requirement that the screen be small. A majority of phones only have a small set of buttons (typically 15-20), most representing digits for dialing numbers (and via techniques that use multiple taps per key, allow users to input alphabets and symbols as well). While mobile phones have been optimized for numeric and voice input, non-voice and non-numeric input is still cumbersome. Usability of interaction from such constrained devices can be greatly improved by maintaining as much information as possible in the network and allowing users and application servers to use that information easily as needed. We refer to this userrelevant information in the network as "Network Context" or simply context. TringIt stores commonly used user profile information like email addresses, postal addresses, preferences etc. in the network. The user maintains information in the context through a rich, non-phone mechanism – a Web portal accessed via an Internetconnected PC. The user can also maintain context information directly from the mobile phone - via a voice portal, SMS portal, phone client etc. if desired. The mobile phone differs from a Web browser in a PC in a very important way. It is the only device that most users carry with them almost all the time yet rarely share with other users - making it an ideal device to deliver a personalized interactive experience. As described earlier, signaling information that is transmitted across the phone network to set up a phone call typically includes the calling party number and the called party number. TringIt uses information derived from the calling party number as part of the data used to trigger the Web action. Awareness of the calling party's "identity" is used to provide personalized services triggered by the Phone Click – TringIt uses ANI information as a key to look up profile data in the context database. Importantly, ANI information is transmitted by default - and requires no manual intervention on the part of the calling user. The calling user not only indicates to TringIt what action he wants to trigger (based on the called number) but also simultaneously, and with no extra effort, provides information that can be used to personalize his request. This greatly improves usability for the calling user. TringIt uses well-known and understood techniques and best practices for maintaining context
TringIt: Easy Triggering of Web Actions from a Phone
19
information in the cloud, and to deal with issues of scalability, reliability, security, access control etc. Maintaining personalizable network context has a significant additional benefit – it enables multi-modal interaction. TringIt allows the user to dynamically intermix interaction via multiple input channels - e.g. via voice, by sending an SMS/MMS message, by clicking a link on a Web page or by using an application, and via Phone Clicks - the user can choose the channel that works best based on dynamic usability and price considerations. 3.5 Internet Access Not Required The mobile phone differs from a Web browser in a PC in some other key ways. By virtue of built-in technologies (e.g. a messaging client, address book), a mobile phone provides “data persistence” and is also “spontaneously reachable” from the network – making it an ideal portable receiver of information delivered by Web applications. TringIt leverages these features to make the mobile phone much more useful, even without Internet access. TringIt enables any mobile phone user to easily use the phone as an input device to trigger Web applications – like a programmable universal remote control – that then deliver information via messaging. The Phone Click is also useful for users with mobile Internet access. The incoming message can be a rich email message containing hyperlinks to Web sites, or a WAP Push that drives an in-phone Web browser to a particular mobile Web site to start a browsing session, or can wake up an in-phone application, triggering it to perform some action. A key contribution is the elimination of the need for end-to-end HTTP from the phone. Users can request information via a Phone Click. Application servers can retrieve Web documents on behalf of the user, transform the results appropriately to fit into an SMS or MMS message, and transmit the message to the user. Users can allow received information to persist in the mobile phone for as long as required, possibly eliminating subsequent need to re-fetch that information. TringIt makes it possible for a user to request news, weather and traffic updates, stock quotes etc. by simply dialing the corresponding number. The information request is personalized using profile data stored in the network. By simply adding numbers and content providers, TringIt makes it possible for users to easily request directions to a hotel, information about a product, daily specials at a restaurant, check order status etc. Like a Web click, the possibilities are limitless.
4 Conclusion The Phone Click uniquely enables a class of applications that work by simply calling numbers – without requiring any additional input over the telephony media channel. These applications automatically trigger Web actions and possibly send information to users in response to a call. By using information that is delivered over the signaling channel as well as any context information available in the network (relevant to the calling number, the called number and the application server) very compelling applications can be created - the caller simply needs to be aware of the action that will be triggered by the application server when he calls that number. The Phone Click can
20
V. Anupam
be employed to enable useful services for all phone users, not just users with mobile Internet access. Importantly, the Phone Click complements all existing channels for communication from a mobile phone - voice, messaging, USSD and mobile Web, making them all more powerful. This user-to-application signaling capability is a broadly useful enabler. It enables all mobile phone users to economically and easily initiate contact and interact with businesses, brands and information providers via their mobile phone. Users simply need to know the number to dial, and numbers can easily be discovered via traditional offline and online channels. TringIt eliminates key usability and cost “friction,” enabling mobile phone users to economically and easily initiate contact and interact with information providers via their mobile phone. It makes the mobile phone a more powerful tool and unlocks its potential as the ultimate information, interaction and participation device. Future Work. While the act of dialing numbers to initiate a voice call is very familiar to users, the notion of triggering Web actions by doing so is not. The authors plan to study the usability barriers associated with the Phone Click via experiments and focus groups. The authors are working on simplifying discovery of Signaling Numbers and on simplifying browsing/review of information delivered by applications – existing software in mobile phones offers significant scope for improvement – and are designing an in-phone client application that will further simplify the user experience. The authors are also developing cloud-based infrastructure that will enable businesses, brands and content providers to easily integrate the Phone Click into their applications.
References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
3GPP TS 03.40 – Technical Realization of the Short Message Service (SMS) 3GPP TS 03.90 – Unstructured Supplementary Service Data (USSD) Anupam, V.: Using Phone Signaling for Application Signaling (in preparation) Automatic Number Identification, http://en.wikipedia.org/wiki/Automatic_number_identification Berners-Lee, T., Cailliau, R., Luotonen, A., Nielsen, H.F., Secret, A.: The World-Wide Web. Comm. ACM 37(8), 76–82 (1994) Dryburgh, L., Hewitt, J.: Signaling System No.7 (SS7/C7): Protocol, Architecture and Services. Cisco Press (2004) Interactive Voice Response, http://en.wikipedia.org/wiki/IVR Kocan, K.F., Roome, W.D., Anupam, V.: A Novel Software Approach for Service Brokering in Advanced Service Architectures. Bell Labs Technical Journal 11(1), 5–20 (2006) Nielsen Mobile: Critical Mass - The Worldwide State of the Mobile Web (July 2008) OMA Multimedia Messaging Service V1.3 Dialed Number Information Service, http://en.wikipedia.org/wiki/DNIS W3C Voice Extensible Markup Language (VoiceXML) 2.1 Yahoo Go!, http://mobile.yahoo.com/go
Context Awareness and Perceived Interactivity in Multimedia Computing Xiao Dong1 and Pei-Luen Patrick Rau2 1
Industrial & System Engineering Department, University of Minneosta, Minnesota, USA
[email protected] 2 Industrial Engineering Department, Tsinghua University, Beijing, P.R. China
[email protected] H
H
H
H
Abstract. Context awareness and perceived interactivity are two factors that might benefit mobile multimedia computing. This research takes mobile TV advertisements as a scenario and verifies the impacts of perceived interactivity and its interaction with context awareness. Seventy-two participants were recruited and an experiment was conducted in order to identify those impacts. The main findings indicated the following: (1) the effect of high perceived interactivity advertisement is significantly better than the effect of low perceived interactivity advertisement; (2) the interaction of context awareness and perceived interactivity has a significant influence on the effect of mobile TV advertising. Keywords: Context awareness, perceived interactivity, mobile TV advertising.
1 Introduction Mobile multimedia has emerged as the hottest growth area in wireless services. It brings a host of new features and functions into the wireless market, providing advanced forms of communication, entertainment and productivity [1]. Mobile operators are investing considerably in broadcasting mobile TV with fully fledged services in various countries throughout Asia as well as other large scale trials around the world [2]. Although mobile advertising and TV advertising have been studied for many years, few researchers have extended their studies to include TV advertising on the platform of mobile devices. With the development of network and hardware capacity, mobile devices can be used to watch TV programs. The purpose of this study was to identify user perception and response to mobile multimedia services. The result can be a guidance of advertisement designers and a base for further studies.
2 Literature Review 2.1 Mobile Multimedia Mobile TV fulfills the growing need of entertaining and staying informed on the move. Mobile TV builds on established consumer behavior: end-users are familiar J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 21–29, 2009. © Springer-Verlag Berlin Heidelberg 2009
22
X. Dong and P.-L.P. Rau
with the concept of television and, with the continued need for mobility, the benefits of this new medium are clear. Mobile TV is enhanced by the element of interactivity, which adds value to the user experience and makes it a richer entertainment option. 2.2 Mobile Advertising Compared with traditional advertising media, mobile advertising can promote sales of goods and services, create brand images and product awareness (branding), disseminate information using a personally relevant and context-aware approach, support direct business transactions and encourage customer interaction [3,4]. In recent years, mobile advertising has been enhanced by the processing capability of handheld devices and the development of networks. Consequently, some innovative advertising methods are found in everyday life. For instance, mobile games and MMS advertising have emerged as creative advertisement venues. However, mobile advertisers must be very careful not to risk privacy issues and exhaust customer tolerance. We are of the opinion that it is the users and not the media designers or the market that are the ultimate determinants of effectiveness. 2.3 Context The context is frequently determined by user location [5,6]. Hence, depiction and association of user location is pivotal to a context-sensitive advertising system. Potential application fields can be found in areas such as travel information, shopping, entertainment, event information and different mobile professions [5]. Another reason for the importance of location is that it is easier to identify and measure compared with other context components. It can be measured with different positioning systems, such as embedded GPS modules, mobile phones which can be located by the telecom operator of a network, or service points utilizing WLAN, Bluetooth, or infrared technologies. 2.4 Perceived Interactivity Interactivity is generally believed to be a multi-dimensional construct [7,8,9,10], but there is no general consensus regarding the nature and the content of the dimensions. Based on constructs identified in internet studies and analysis of the characteristics of mobile communication, a model of interactivity for mobile advertisements was constructed by Gao et al. [11], comprising user control, direction of communication, synchronicity, connectedness, playfulness, and interpersonal communication. They also stated that different mobile advertising tools might differ in these dimensions due to the different communication style each tool has. For example, message push-ads might allow less user control, but an included reply option will give customers a convenient channel to respond; mobile banners are less intrusive compared with pushads, but they might be ignored or assumed to be only decorative images. User control is conceptualized as the degree of user intervention that is required to operate the system [12]. Dholakia et al. refer to user control as the extent to which an individual can choose the content, timing and sequence of a communication to change his/her viewing experience [13]. It is taken as the core component of interactivity by some researchers [10,14]. Two constructs identified by Steuer [10], range and
Context Awareness and Perceived Interactivity in Multimedia Computing
23
mapping, actually describe two aspects of control. The former refers to the number of options the environment provides the user to modify the task flow and the environment, and the latter refers to the extent to which the controls and manipulations in a computer-mediated environment are similar to controls and manipulations in the real word. This paper manipulates interactivity by adding different user controls to the advertisement.
3 Hypotheses and Methodology 3.1 Hypotheses Hypothesis 1: High perceived interactivity advertisements will have better advertising effectiveness(better memory, better attitudes towards the ads and brand, higher purchase intention) than low perceived interactivity advertisements. For message advertising, user control options are important. Control choice/range and mapping have been traditionally considered as fundamental constructs of interactivity [15,10]. Users can respond by replying to the message directly, call back with a provided telephone number, or visit another source linked in the message. Users also want the control and manipulation in computer-mediated environments to be similar to those in the real world. The more a user can control the options provided, the more interactive the customer perceives the advertisement to be. The more similar the mediated environment is to the real world, the more interactive the customer perceives the advertisement to be. Studies on Internet advertising interactivity have found that there is a strong correlation between perceived interactivity and advertising effectiveness in terms of attitude towards brand, attitude towards ads, and purchase intention [16,17,18]. Previous study also suggested that higher interactivity helps the customer experience “flow” during the interaction [19] and the consequences of the “flow” experience are increased learning and perceived behavioral control. Therefore it was hypothesized that high perceived interactivity has a positive influence on advertising effectiveness. Hypothesis 2: The interaction between interactivity and context-awareness will have an influence on mobile advertising effectiveness. While context-awareness ads give users higher involvement and make the ads more relevant, interactivity provides the customer a chance to communicate with the company, to search for further information or to disseminate information to others conveniently and quickly. As Kannan et al. [20] have already pointed out, it is critical to provide the customer a chance to respond at the point of purchase or usage immediately when sending context-aware advertisements. Immediately redeemable m-coupons, callback numbers, or simply a message requiring a reply from the customer are hypothesized as most likely to exert influence when sent in a contextaware manner compared to when sent in a context-irrelevant manner. Context awareness and interactivity are hypothesized to have an interaction effect on advertising effectiveness.
24
X. Dong and P.-L.P. Rau
3.2 Experiment Participants. Seventy-two participants (36 female and 36 male) from Universities in Beijing voluntarily took part in the experiment. They were randomly assigned to six groups with the combination of two-level context awareness and three-level perceived interactivity. The participants were all undergraduate and graduate students with no prior knowledge about the tasks to be performed during the experiment. The participants’ ages ranged from 20 to 36 years (mean=24, S.D. =2.26) and 55% of the participants had used mobile phones for more than five years. In addition, 98% of the participants had previously received SMS advertisements, and 32% of the participants had received MMS advertisements. 90% of the participants had more than five years of Internet experience, while 32% of the participants had experience connecting to the Internet via mobile devices. Experimental design and variables. Independent variables are perceived interactivity and its interaction with context awareness. Dependent variables were advertising effectiveness which consisting of memory of the advertisement, attitude towards the advertisement and brand, and purchase intention. Memory was measured by a free recall and recognition test. The ads attitude [8], brand attitude [21], user involvement [22] and purchase intention was measured using scales from other researches. Procedures. Each participant was tested individually. They were asked to complete a demographic and technology (internet, mobile services and TV advertisements) usage questionnaire. Then the participants were given an introduction to the experiment’s procedure and their tasks. A practice task was provided to let the participants view sample mobile TV advertisements and make sure they knew how to use the experiment devices (PDA-ASUS A620, KONKA TPC880). During the experiment, all participants visited five different scenarios (mall, bookstore, cell phone market, McDonald’s, and the IE building at Tsinghua University) with a predefined sequence. In each scenario, they did two information seeking tasks and viewed mobile TV advertisements on experiment devices after each task. According to different groups, context awareness and perceived interactivity features were embedded into mobile TV advertisements. Upon completion of all tasks, a free-recall test and an advertisement recognition test were given; the participants were not informed prior to the task that these tests would be applied. Then participants were asked to finish a post-test questionnaire, which measured the user’s attitude towards the ads, brand attitude, perceived interactivity, as well as purchase intention.
4 Results and Discussions In this section we present the reliability of each measure and the results of test hypotheses one and two. The internal consistencies for the questionnaire responses, using Cronbach’s a, were 0.95 for the advertisement attitude questionnaire, 0.95 for the brand attitude questionnaire and 0.87 for the involvement with the advertisement questionnaire, and 0.78 for the perceived interactivity questionnaire.
Context Awareness and Perceived Interactivity in Multimedia Computing
25
4.1 The Effect of Perceived Interactivity on Advertisement It was hypothesized in this study that high perceived interactivity advertisements would have better advertising effectiveness than low perceived interactivity advertisements. High range and high mapping are two different kinds of high level perceived interactivity. They are compared with low perceived interactivity separately to identify whether this hypothesis is true. After the experiment, the data showed that there were significant differences between exposure time to mobile TV advertisements in each group. This factor was used as a covariate in the ANCOVA process. The results can be found in Tables 1 and 2. Table 1. Data for Testing Hypothesis Three (High range×Low interactivity) Perceived interactivity High range
Low interactivity
Variable
Mean
SD
Mean
SD
P value
Ad free recall
2.93
1.73
1.77
1.05
Ad recognition
7.33
1.88
6.13
2.25
0.04*
Ad attitude
4.80
0.52
4.24
0.53
0.00*
0.003*
Brand attitude
4.87
0.53
4.28
0.41
0.00*
Purchase intention
4.69
0.69
4.08
0.49
0.00*
Table 2. Data for Testing Hypothesis Three (High mapping×Low interactivity) Perceived interactivity High range
Low interactivity
P value
Variable
Mean
SD
Mean
Ad free recall
2.82
1.46
1.77
1.05
0.006*
Ad recognition
7.46
2.43
6.13
2.25
0.02*
Ad attitude
4.60
0.65
4.24
0.53
0.02*
Brand attitude
4.62
0.57
4.28
0.41
0.01*
Purchase intention
4.35
0.70
4.08
0.49
0.10
SD
From these results we can see that hypothesis one is supported. High perceived interactivity advertisements will have better advertising effectiveness than low perceived interactivity advertisements. This was consistent with past studies. Cho and Leckenby [16] measured participants' intention to interact with a target (banner) ad and found positive relations (correlation coefficients ranging between .30 and .75) between intention to interact with the ad and attitudes toward the ad, attitudes toward the brand, and purchase intention. Yoo and Stout [18] also achieved similar results. McMillan and Hwang’s [17] study demonstrated that interactivity and involvement
26
X. Dong and P.-L.P. Rau
with the subject of a site were two possible predictors of positive attitude towards the Web site, and perceived interactivity accounted for more of the variance in attitude than did involvement. Analysis of relationships among the variables in the study suggested that the control sub-dimension of perceived interactivity had the strongest correlation with attitude toward the Web site. 4.2 Joint Effect of Context-Aware and Perceived Interactivity It was hypothesized in this study that the interaction between interactivity and context awareness influences mobile advertising effectiveness. As mentioned above, we also included exposure time to mobile TV advertisement in each group as the covariate in the ANCOVA test. The results showed that the interaction between perceived interactivity and context awareness significantly influenced the subject’s attitude towards mobile TV advertisements (F=4.183, p=0.019), their attitude towards brands (F=5.011, p=0.009), and their purchase intention (F=7.732, p=0.001). Although the interaction between context awareness and perceived interactivity have no significant influence on advertisement free recall and advertisement recognition, the p value is quite close to the significant level p=0.05 (F=2.22, p=0.117 for free recall; F=2.793, p=0.068 for recognition). Table 3. The Effect of Interactivity under the Context-aware Condition
Variable Ad free recall Ad recognition Ad attitude Brand attitude Purchase intention
Perceived interactivity High range High mapping Mean SD Mean SD 3.47 2.07 3.31 1.68 7.92 1.51 8.50 2.02 5.14 0.40 4.68 0.72 5.20 0.38 4.64 0.66 5.13 0.52 4.28 0.83
Low interactivity Mean SD 1.55 0.87 5.75 2.42 4.13 0.52 4.19 0.43 3.88 0.46
F value
P value
5.16 6.20 9.72 11.86 12.45
0.01 0.01 0.00 0.00 0.00
Table 4. The Effect of Interactivity under the Context-irrelevant Condition
Variable Ad free recall Ad recognition Ad attitude Brand attitude Purchase intention
Perceived interactivity High range High mapping Mean SD Mean SD 2.39 1.15 2.33 1.03 6.75 2.09 6.42 2.43 4.46 0.40 4.52 0.58 4.54 0.44 4.61 0.49 4.26 0.57 4.41 0.57
Low interactivity Mean SD 1.99 1.20 6.50 2.11 4.35 0.54 4.37 0.39 4.27 0.46
F value
P value
0.43 0.07 0.34 0.94 0.30
0.65 0.93 0.71 0.40 0.75
After examining the effect of interactivity in different context conditions (Tables 3 and 4), we found that when mobile advertisements are distributed in a context-aware manner, the interactivity of ads has a positive influence on the advertising effectiveness in terms of advertisement memory, advertisement attitude, brand attitude, and purchase intention. However, when mobile ads are distributed in a context-irrelevant way, the interactivity of advertisements has no significant influence on advertisement memory,
Context Awareness and Perceived Interactivity in Multimedia Computing
27
advertisement attitude, brand attitude and purchase intention. This finding is partially contradictory with previous literatures in Internet advertising, where it is generally agreed that perceived interactivity has positive influences on advertising effectiveness [16,17,18]. The main reason for the lack of significant effects of interactivity in the context-irrelevant situation may be the lower message involvement in the contextirrelevant condition. First, low involvement leads to a lower information processing level, which may cause the subject to neglect the interaction options of the advertisement or even the advertisement itself. Second, lower involvement with the advertisement content also results in lower motivation to respond to the ad [23]. The lower motivation to respond makes it less critical to provide interaction options, since they do not seek interaction options from the beginning. When the same advertisements were sent in a context-aware manner, however, the effects of the interactivity of mobile advertisements on advertising effectiveness became significant, as expected in hypotheses one. The results in Tables 5, 6 and 7 show that when the interactivity of mobile ads is low, to send them in a context-aware manner could result in even worse advertising Table 5. The Effect of Context Awareness to High Range Advertisements
Variable Ad free recall Ad recognition Ad attitude Brand attitude Purchase intention
Context awareness Location relevant Location irrelevant Mean SD Mean SD 3.47 2.07 2.39 1.15 7.92 1.51 6.75 2.09 5.14 0.40 4.46 0.40 5.20 0.38 4.54 0.44 5.13 0.52 4.26 0.57
F value
P value
2.46 2.46 17.55 15.50 15.13
0.13 0.13 0.00 0.00 0.00
Table 6. The Effect of Context Awareness to High Mapping Advertisements
Variable Ad free recall Ad recognition Ad attitude Brand attitude Purchase intention
Context awareness Location relevant Location irrelevant Mean SD Mean SD 3.31 1.68 2.33 1.03 8.50 2.02 6.42 2.43 4.68 0.72 4.52 0.58 4.64 0.66 4.61 0.49 4.28 0.83 4.41 0.57
F value
P value
2.97 5.21 0.37 0.02 0.19
0.10 0.03 0.55 0.90 0.67
Table 7. The Effect of Context Awareness to Low Interactivity Advertisements
Variable Ad free recall Ad recognition Ad attitude Brand attitude Purchase intention
Context awareness Location relevant Location irrelevant Mean SD Mean SD 1.55 0.87 1.99 1.20 5.75 2.42 6.50 2.11 4.13 0.52 4.35 0.54 4.19 0.43 4.37 0.39 3.88 0.46 4.27 0.46
F value
P value
1.06 0.66 0.98 1.05 4.14
0.31 0.43 0.33 0.32 0.05
28
X. Dong and P.-L.P. Rau
effectiveness than to send them in a context-irrelevant manner, in terms of memory, advertisement attitude, brand attitude and purchase intention. However, when the interactivity of mobile ads is high, it is better to send them in a context-aware manner so as to promote the advertising effectiveness in terms of memory, advertisement attitude, brand attitude and purchase intention. In this study, we also found that the high range effect was greater than the high mapping effect as the indicator of perceived interactivity. The finding that context awareness with low interactivity ads has a negative influence on advertisement effectiveness is surprising. We propose two reasons to address this. (1) Location-based services and advertisements should be provided with great care so as not to invade users’ privacy, since handsets are very personal devices. The low interactivity ads only “broadcast” information about products and services. It is possible that the feeling of being invaded by such ads might become even stronger when users detect that their cell phones are being spammed just because they happen to walk past a particular store. (2) Mobile TV advertisement is characterized by its rich media features. Context awareness may influence advertisement effectiveness through other factors, such as interactivity. But further study is required to identify their relationship.
5 Conclusion and Future Study The effect of perceived interactivity is clear in this study. High perceived interactivity advertisements have better advertising effectiveness than low perceived interactivity advertisements. This finding is consistent with former research on internet and mobile message advertisement. One guideline of perceived interactivity based on this research ought to be formed to give mobile TV advertisement business market instructions. The effects of context awareness on mobile TV advertising effectiveness take place according to different levels of interactivity. With high interactive advertisements, contextual advertising information does increase user response effectively and results in a more accepting attitude. Therefore, when the goal of a mobile advertising campaign is to generate responses, the context in which the response options are given to the users are of importance and must be taken into consideration by the mobile marketer. However, with “broadcasting” advertisements, the user’s attitude towards the brand and the consequent purchase intention were impaired by context aware advertisements rather than improved. This interaction style is needed to be thoroughly studied by future studies.
References 1. Robert, W.S.: Mobile multimedia goes Net-centric. Electronic Engineering Times. Manhasset: Mar 5(1156), 78–79 (2001) 2. Kenton, O., April, S.M., Alex, V.: Consuming Video on Mobile Devices. In: Proc. CHI 2007, pp. 857–866. ACM Press, New York (2007)
Context Awareness and Perceived Interactivity in Multimedia Computing
29
3. Dickinger, A., Haghirian, P., Murphy, J., Scharl, A.: An Investigation and Conceptual Model of SMS Marketing. In: Proc. 37th Hawaii International Conference on System Sciences 2004, p. 10031.2. IEEE Computer Society Press, Los Alamitos (2004) 4. Yunos, H., Gao, J.: Wireless Advertising’s Challenges and Opportunities. Computer 36, 30–37 (2003) 5. Kaasinen, E.: User needs for location-aware mobile services. Personal and Ubiquitous Computing 7, 70–79 (2003) 6. Younghee, J., Per, P., Jan, B.: DeDe: Design and Evaluation of a Context-Enhanced Mobile Messaging System. In: Proc. of CHI 2005. ACM Press, New York (2005) 7. Wu, G.: Perceived Interactivity and Attitude Toward Web Sites. In: Proc. Conference of the American Academy of Advertising 1999. American Academy of Advertising (1999) 8. Ha, L., James, E.: Interactivity Reexamined: An Analysis of Business Web Sites. In: Proc. Conference of the American Academy of Advertising (1998) 9. Liu, P., Shrum, L.: What is interactivity and is it always such a good thing? Implications of definition, person, and situation for the influence of interactivity on advertising effectiveness. Journal of Advertising 31, 53–64 (2002) 10. Steuer, J.: Defining Virtual Reality: Dimensions Determining Telepresence. Journal of Communication 42(4), 73–93 (1992) 11. Gao, Q., Rau, P.L.P., Salvendy, G.: Measuring perceived interactivity of mobile advertisements. Behavior & Information Technology (2006) 12. Van der Heijden, H.: Ubiquitous computing, user control, and user performance: conceptual model and preliminary experimental design. In: Proc. Tenth Research Symposium on Emerging Electronic Markets, pp. 107–112 (2003) 13. Dholakia, R., Zhao, M., Dholakia, N., Fortin, D.: Interactivity and Revisits to Websites: A Theoretical Framework, http://ritim.cba.uri.edu/wp/ 14. Bezjian-Avery, A., Calder, B., Lacobucci, D.: New Media Interactive Advertising vs.Traditional Advertising. Journal of Advertising Research 38(94), 23–32 (1998) 15. Coyle, J., Thorson, E.: The Effects of Progressive Levels of Interactivity and Vivideness in Web Marketing Sites. Journal of Advertising 30(3), 65–78 (2001) 16. Cho, C.-H., Leckenby, J.: Interactivity as a measure of advertising effectiveness: Antecedents and Consequences of Interactivity in Web Advertising. In: Proc. Conference of the American Academy of Advertising (1999) 17. McMillan, S., Hwang, J.S.: Measures of Perceived Interactivity: An Exploration of Communication, User Control, and Time in Shaping Perceptions of Interactivity. Journal of Advertising 31(3), 41–54 (2002) 18. Yoo, C.Y., Stout, P.: Factors Affecting User’s Interactivity with the Web site and the Consequences of User’s Interactivity. In: Proc. Conference of the American Academy of Advertising (2001) 19. Hoffman, D., Novak, T.: Marketing in Hypermedia Computer-Mediated Environments: Conceptual Foundations. Journal of Marketing 60(3), 50–68 (1995) 20. Kannan, P., Chang, A., Whinston, A.: Wireless commerce: marketing issues and possibilities. In: Proc. International Conference on System Science 2001. IEEE Computer Society Press, Los Alamitos (2001) 21. Li, H., Bukovac, J.L.: Cognitive Impact of Banner Ad Characteristics: An Experimental Stud. Journalism and Mass Communication Quarterly 76(2), 341–353 (1999) 22. Norris, C.E., Colman, A.M.: Context effects on recall and recognition of magazine advertisements. Journal of Advertising 21(3), 37–46 (1992) 23. Petty, R.E., Cacioppo, J.T.: Attitudes and Persuasion: Classic and Contemporary Approaches. Westview Press (1981)
Human Computer Interaction with a PIM Application: Merging Activity, Location and Social Setting into Context Tor-Morten Grønli and Gheorghita Ghinea The Norwegian School of Information Technology, Schweigaardsgt. 14, 0185 Oslo, Norway School of Information Systems, Computing and Mathematics, Brunel University, Uxbridge UB8 3PH, London, United Kingdom
[email protected],
[email protected]
Abstract. Personal Information Managers exploit the ubiquitous paradigm in mobile computing technology to integrate services and programs for business and leisure. Recognizing that every situation is constituted by information and events, this context will vary depending on the situation users are in, and the tasks they are about to commit. The value of context as a source of information is highly recognized and for individual dimensions context has been both conceptually described and prototypes implemented. The novelty in this paper is a new implementation of context by integrating three dimensions of context: social information, activity information and geographical position. Based on an application developed for Microsoft Window Mobile these three dimensions of context are explored and implemented in an application for mobile telephone users. Experiment conducted show the viability of tailoring contextual information in three dimensions to provide user with timely and relevant information. Keywords: PIM, context, context-aware, Microsoft pocket outlook, ubiquitous computing, HCI.
1 Introduction Personal Information Managers (PIM’S) exploit the ubiquitous paradigm in mobile computing technology to integrate services and programs for business and leisure. Activities performed with PIMs range from plotting appointments and tasks in a calendar to the automatic information exchange between mobile devices, different device synchronizations and base station communication. In every situation in our daily life, an independent context of information and events is defined. This context will vary depending on the situation users are in, and the tasks they are about to commit. Nonetheless, despite the fact that the challenge of defining context has already been addressed [1-3] and that its value as a source of information is recognised, the link between context and PIMs is still an immature and unexplored area of research. This paper addresses the use of context in an everyday mobile phone based PIM application which makes use of context based information to enhance the user experience. J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 30–38, 2009. © Springer-Verlag Berlin Heidelberg 2009
Human Computer Interaction with a PIM Application
31
The unique use of context in this PIM application combines activities, social information and geographical information.
2 Background Developers and researchers agree that context is an important factor when designing new applications. With PIM devices becoming increasingly widespread and in daily use by a large population, this opens interesting possibilities for development. Such applications would potentially be used daily by people in their homes or at their workplace, especially bearing in mind that people carry mobile devices and thereby the application with them almost 24 hours a day. Recent arguments state the possibility for business travelers and other mobile workers to leave the laptop at home and shift entirely to mobile devices because of their increased capacity and convenient size. Context in mobile applications has been looked at by more than a few researchers [1,10]. For example, Ludford et al. [8] looked at the use of context to provide useful information to the user on their mobile phone. In their work, context is based on the location and/or time of the day. This partly makes use of daily available context information, which, however, is not instantly fed back into the system as parameters for information retrieval. Efforts have also been made to make use of context as a tool for supporting business travelers in [8]. The definition of context here as the user’s planned activity in combination with the location is quite interesting. This, because it generates quite a lot of information about the user, but information is of reduced interest if we have no way of making use of it. Zhou et al. [11] have also demonstrated the use of context sensitive information for the real estate business. These are just two examples out of many and one could only imagine many other possible scenarios. On an overall basis, though, the use of context in applications is often missing or single dimensional. This focus should be changed, since automated PIM applications which take into account the total context of the user would possibly be able to not only support the everyday tasks of the user, but also improve efficiency and ease the work of the user by automatically tailoring information to the user’s needs and/or adapting the application to the user’s current setting. The CityFlocks application [1] is one step in this direction, however it falls short of offering a full solution to the problem. The widespread use of small mobile devices (PIMs) has, as shown, forced researchers and developers to consider context as an important criteria for highly mobile systems. The notion of context-aware computing is generally the ability for the devices to adapt their behavior to the surrounding environment, hence enhancing usability [7]. Towards this goal, Dey and Abowd [2] state that if we understand context fully in a given environment and setting, we would then be able to better choose what contextaware behaviours to sustain in applications. This could lead to more realistic applications and thereby applications more meaningful to users. This is also exemplified by Edwards [4] when he uses context information to build an application. In his application different layers represent different sources of information and they can be reused in later settings. Edwards argues that context is a major part of our daily life and that
32
T.-M. Grønli and G. Ghinea
computing with support for sharing and using contextual information (context-aware computing) would improve user interaction. Indeed when viewing people rather than systems as consumers of information, a new infrastructure is needed.
3 Design Our application interacts with the user by presenting information relevant to the users’ context. To be able to do this, system design and functionality are split into three main modules, each of which generate context-aware information and responds to different user interactions. This enables a precise definition of elements and thereby tailoring of information to be displayed according to the users’ actual needs. One of these main modules handles activity, one handles social context and the third handles the geographical location. The input from all three sources together provides the foundation for the user context computation. By this operationalization of context and contextaware concepts we are able to create a user context. This user context is then the foundation upon which the application takes action to present information. We now proceed to provide further details of each of the three modules involved in our PIM application. 3.1 Social Context This module computes the foundation upon which the social context is determined. Social context will naturally differ tremendously based upon each situation and for each individual. Still, it is possible to use one social context in common by several people by choosing concepts that are interpreted the same by most people. This is achieved through building a taxonomy of concept terms illustrated in Table 1 below. Table 1. Taxonomy of social context
Categories:
Sub categories:
Leisure
Work
Shopping Cinema Spare time Food Culture
Meeting Preparation Own time Travelling Phone meeting
Travel Train Car Tube Foot Transport
In our approach, building on the description of activities from Prekop and Burnett [9] information of the social context in the application are stored and meta-tagged as Pocket Outlook activities / appointments. These tags are based on the above taxonomy and are implemented for the user through extending the standard Pocket Outlook category interface (Figure 1). Thereby the user can enrich (tag) the activity with category tags through a familiar interface and thereby greatly increase the familiarity of the system.
Human Computer Interaction with a PIM Application
33
Fig. 1. Taxonomy interface
3.2 Geographical Location Module This module calculates location based on input from the internal Global Positioning System (GPS) in the device. When a valid connection through the GPS device to a GPS satellite occurs, it returns location information to the application. The input from the GPS is then parsed and the actual location retrieved by inspecting the longitude and latitude coordinates. These coordinates are then mapped to a one of 16 specific zones (in our case, Oslo, Norway). As the user is moving, information about the current zone is updated and stored in the application. This is done by letting the device interact with the running applications data sources and although the user not actually is feeding any information into the device, the physical moving around is sufficient for context-aware exploitation of user information. 3.3 Activity Module This module communicates with the Microsoft Pocket Outlook storage on the mobile device and retrieves appointments and activities. The module accesses the Pocket Outlook storage directly and also listens to system events generated in this storage. The user interacts with the activity module through the familiar Pocket Outlook interface, and attaches one or more of the category terms as described previously. In doing this, almost unknowingly, the user improves the quality of activity information and thus eases the use of the PIM application.
4 Implementation The application prototype is designed for and implemented on a Pocket PC device [6] (HTC 3600 phone) using the Microsoft Windows Mobile 6.0 operating system. The application is programmed in Microsoft .NET Compact Framework with C# as
34
T.-M. Grønli and G. Ghinea
implementation language. Geographical position is acquired through the use of GPS and activities and appointments are acquired through Microsoft Pocket Outlook. All data on the device are kept continuously up to date by computer synchronization with Microsoft Outlook 2007. This device is also chosen since it contains powerful enough hardware and has a decent enough storage area to be suitable for software development.
5 User Evaluation The PIM application was evaluated with a test group of 15 users who undertook a set of social and work-related activities whilst navigating a route through central Oslo (Figure 2).
Fig. 2. Suggested route through city
Users had to complete an evaluation questionnaire after the testing shown in Table 2 below. For all questions, users were asked to state whether they agreed or not to each statement. Measurement scale: Strongly Disagree (SD), Disagree (D), Mildly Disagree (MD), Mildly Agree (MA), Agree (A) and Strongly Agree (SA). Each possible answer from SD to SA was mapped to a number from 1 to 6, respectively, and the responses thus received were analyzed using a T-test (Table 3). We will in the following sections elaborate on the implications of our evaluation exercise.
Human Computer Interaction with a PIM Application
35
Table 2. Questionnaire
1 2 3
The information provided by the reminder system correctly matched my current location The information provided when I was “Sightseeing in old town” was incorrect. The summary of blocked events I received after appointment “DNB Nor Solli plass” was useful
4 5
The system provides duplicated information I liked the fact that the application is integrated with Outlook
6 7
The reminder system is useful I would use this application in my daily life
User responses to the evaluation questionnaire are summarized in Table 3 below. Table 3. T-Test results
Question 1
Mean Response 4.80
T-Value
P
8.088
0.000
2
2.47
-1.054
0.310
3
3.73
1.585
0.135
4
2.73
-0.654
0.524
5
5.47
12.854
0.000
6
5.20
12.602
0.000
7
3.93
3.287
0.005
From Table 3 we can see that all answers display a positive bias (Questions 2 and 4 are in fact negative statements about the application, therefore the negative bias here, reflects positive user statements). In question 1, 14 out of 15 answered that information displayed did match their current location and the responses are statistically significant. This would indicate a correct computed context and correct displayed information on an overall basis. For question number two, 11 out of 15 respondents were negative to that the information being displayed in one appointment was incorrect. This indicates that 11 got, at least partly, displayed correct information and four had incorrect or no information displayed. There is thus a strong bias towards negative answers, a few (4) positive answers and no middle values (MA / MD). This polarization of results however leads to the data for this question not to be statistically significant.
36
T.-M. Grønli and G. Ghinea
Fig. 3. Questionnaire results
As described, the application prototype is adaptable to different scenarios and user settings, but a context dependent application needs to be tailored to the users’ need when being deployed for a real life setting, e.g. the initial categories and their weight need to be configured in accordance with the findings of Prokop and Burnett [9]. When these issues are taken care of, the user experience might improve and increase even more the positive trend in answers to question six and seven. Moreover, as shown by Zhou et al. [11], information tailoring is an important task to help users interpret data. Our application focuses on tailoring by having minimal information displayed at the same time, when new messages are shown, thereby easing the users’ interpretation. Currently calendars and applications based on these do not take multi contextual information into account [7] as they often only reproduce the information already available there. Thereby, this can at worst lead to incorrect display of data and at the best a reproduction of data in a new interface. Our developed PIM application greatly differentiates from this by only displaying information based on the computed user context, given by the three factors social context, location and activity / appointment. Earlier approaches that have made use of calendar data from Pocket Outlook, often end in using Pocket Outlook data together with a simple timeline (i.e. [5]). In our approach, the use of Pocket Outlook data is extended to not only retrieve and display the data, but also to add extra meta-information to the appointments. Results from question five show that all respondents stated they liked the integration of the developed PIM application with the Outlook calendar. This is important because it shows they had no problems entering an appointment in one application, and having information displayed in another application (the prototype). In the evaluation exercise, generation of information was tightly connected with the actual task at hand and participants were asked to judge whether or not they found the application useful (question six). Our results show that all 15 users involved in the evaluation thought the
Human Computer Interaction with a PIM Application
37
application gave useful value. This would indicate that the reminder system is an application with practical use for the users. One other side of usefulness is the behaviour of the device and the application. Therefore each was asked to evaluate these parameters as well in question six. As a final question, after the test, the users were asked to state whether or not they would like to use this application in their daily life. As the t-test shows, the results for this question are statistically significant and one should note this indicates that the users found value in using the application in a daily life.
6 Concluding Remarks Context and context-awareness have long been acknowledged as important and have generated considerable research effort. However, integration into PIMs has so far been limited and the perspective has often been single-dimensional. In this article, the main aspects of the design, implementation and evaluation of an application prototype which integrates context / context-awareness into a PIM from a novel threedimensional perspective combining social-, geographical- and activity information have been presented. User evaluation of the proof of concept displayed a strong positive bias, highlighting its potential usefulness and applicability. Based on the developed prototype, we have shown the viability and usefulness of our approach and we do believe that tailoring information in the manner described in this paper takes the PIM concept one step further towards the ideal of providing tailored and timely information to mobile information users everywhere.
References 1. Bilandzic, M., Foth, M., Luca, A.: CityFlocks: Designing Social Navigation for Urban Mobile Information Systems. In: Proceedings ACM Designing Interactive Systems (2008) 2. Dey, A.K., Abowd, G.: Towards a Better Understanding of Context and ContextAwareness. In: Gellersen, H.-W. (ed.) HUC 1999. LNCS, vol. 1707, p. 304. Springer, Heidelberg (1999) 3. Dourish, P.: What we talk about when we talk about context. Journal of Personal and Ubiquitous Computing 8, 19–30 (2004) 4. Edwards, K.: Putting Computing in Context. ACM Transactions on Computer-Human Interaction 12(4), 446–474 (2005) 5. Hertzog, P., Torrens, M.: Context-aware Mobile Assistants for Optimal Interaction: a Prototype for Supporting the Business Traveler. In: Proceedings of the 9th international conference on Intelligent User Interfaces, pp. 256–258 (2004) 6. HTC, The HTC 3600 (2007), http://www.htc.com/product/03-product_p3600.htm (June 1, 2007) 7. Kaasinen, E.: User needs for location-aware mobile services. ACM Personal and Ubiquitous Computing 7(1), 70–79 (2003) 8. Ludford, P., Rankowski, D., Reily, K., Wilms, K., Terveen, L.: Because I carry my cell phone anyway: functional location-based reminder applications. In: Proceedings of Conference on Human Factors in Computing Systems, April 2006, pp. 889–898 (2006)
38
T.-M. Grønli and G. Ghinea
9. Prekop, P., And Burnett, M.: Activities, Context and Ubiquitous Computing. Journal of Computer Communications, Special Issue on Ubiquitous Computing 26, 1168–1176 (2003) 10. Rodden, T., Cheverest, K., Davies, K., Dix, A.: Exploiting context in HCI design for mobile systems. In: Workshop on Human Computer Interaction with Mobile Devices (1998), http://www.dcs.gla.ac.uk/~johnson/papers/mobile/HCIMD1.html 11. Zhou, M., Houck, K., Pan, S., Shaw, J., Aggarwal, V., Wen, Z.: Enabling ContextSensitive Information Seeking. In: Conference of Intelligent User Interfaces, January / February 2006, pp. 116–123 (2006)
CLURD: A New Character-Inputting System Using One 5-Way Key Module Hyunjin Ji1 and Taeyong Kim2 1 CLURD, 211-903, Hyundai Apt. Guui 3-dong, Kwangjin-gu, Seoul, Korea 2 School of Journalism & Communication, Kyunghee University, Hoegi-dong, Dongdaemun-gu, Seoul, Korea
[email protected],
[email protected]
Abstract. A character inputting system using one 5-way key module has been developed for use in mobile devices such as cell phones, MP3 players, navigation systems, and remote controllers. All Korean and English alphabet characters are assembled by two key clicks, and because the five keys are adjacent to each other and the user does not have to monitor his/her finger movements while typing, the speed of generating characters can be extremely high and its convenience is also remarkable. Keywords: Character Input, Typing, 5-way Key Module, Mobile Device, Keyboard, Wearable Computer.
1 Background People use cell phones, MP3 players, navigation systems, and remote controllers almost every day. Since these devices have to be small enough to hold and carry, only a limited number of keys can be installed in them. A challenge rises in that we have an increasing level of need to input text data using these devices. Therefore, with a few exceptions like Blackberry, which has 26 separate keys assigned for 26 English alphabet characters, device manufacturers has employed various methods that make possible for the users to input all characters conveniently with a small number of keys. The oldest and most popular method should be the one we have in our telephones. This is a sort of 'toggle' method, in which each numerical key from '2' to '9' is made to correspond to 3-4 character keys, so that a desired character is selected and inputted. Fig. 1 is the key layout of this method.
1 2 ABC 4 GHI 5 JKL 7 PQRS 8 TUV 0
3 DEF 6 MNO 9 WXYZ
Fig. 1. This may be the most widely applied character inputting system as it is installed in most of the traditional telephones J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 39–47, 2009. © Springer-Verlag Berlin Heidelberg 2009
40
H. Ji and T. Kim
A serious limitation of this method is experienced when a user tries to input a word like ‘feed.’ Since the characters ‘d,’ ‘e,’ ‘f’ are assigned to the same key, the user has to either wait awhile or input a splitter signal in order to input ‘e’ after ‘f.’ It is because the phone does not understand whether the user attempts to input a new character, which happens to be assigned to the same key, or change the inputted character to the next character assigned to the same key. In Korea, since the 'toggle' method requires too many times of manipulation when characters are inputted, to thus consume much time, other methods for inputting characters have been developed. The 'chun-ji-in' method and the 'naragul' method are two well-known examples. However, even with these widely-adopted methods in Korea, complicated and burdensome key manipulations are needed for inputting, and therefore characters cannot be speedily inputted. A source of the problem is that, like the old method of inputting English alphabet characters mentioned above, the two methods use 11 or 12 keys that are spread widely over the surface of the device. Another is that, in case of chun-ji-in, a splitter is required for the same reason that the old English inputting method is inconvenient, and in case of naragul, two modifier keys have to be used very frequently to change an inputted character into others that have the same root. Because of these limitations, a user has to move his/her fingers around the 11 or 12 keys busily and click as many as three keys located apart to input one character.
2 The CLURD System In order to overcome the limitations of the methods mentioned above, a new method of inputting characters using a 5-way key module has been developed and named “CLURD,” which stands for Center-Left-Up-Right-Down. 5-way key module comprises a center key disposed in the center, and an upper key, a lower key, a left key and a right key which are disposed in the top, bottom, left and right sides of the center key, respectively. Typical examples of the module are shown in Fig. 2.
Fig. 2. Typical 5-way key modules that can adopt the CLURD system
Even though there are only five keys in the key module, the CLURD system does not use the ‘toggle’ method. Instead, the system makes it a rule to assemble a character with two key clicks. Theoretically, a total of 25 combinations can be created with two clicks of five keys(5x5). If these 25 combinations are assigned to 24 Korean characters and a space datum respectively, it should be feasible to input Korean words
CLURD: A New Character-Inputting System Using One 5-Way Key Module
41
with no additional splitter or modifier keys. The combinations for the 24 Korean characters are illustrated in Figs. 3 & 4. The only unassigned combination is CenterCenter, and this is used to input a space datum. Incidentally, the data generator turns
ㄱ ㄷ ㅂ ㅅ
ㅈ
ㄲ ㄸ ㅃ ㅆ
ㅉ
‘ ’, ‘ ,’ ‘ ,’ ‘ ,’and ‘ ’ into ' ', ' ', ' ', ' ', and ' ,' respectively when the second key is pressed longer than a predetermined time. As shown in Figs. 3 & 4, the combinations are not randomly assigned to characters. Rather, the combination for each character is determined based on the geometric shapes of the keys. (See the first round-shape key module in Fig. 2.) The shapes of five keys are apparently different, and if the shapes of the two keys to be clicked are combined, they should resemble the shape of each of the 24 Korean characters fairly closely.
Fig. 3. CLURD Combinations for 10 Korean Vowels
Fig. 4. CLURD Combinations for 14 Korean Consonants
42
H. Ji and T. Kim
The CLURD system works for English alphabet characters in the same logic. As in the case of Korean, the combinations are thoughtfully assigned to characters so that they resemble the shapes of the characters as closely as possible. As shown in Fig. 5, the data generator assembles 'n', 'D', 'U', and 'C' among the English alphabetical characters if a signal generated by clicking each key combination of Up-Center, RightCenter, Down-Center, and Left-Center is input. The data generator assembles 'm', 'B', 'W', and 'E' if a signal generated by clicking each key combination of Up-Up, RightRight, Down-Down, and Left-Left is input.
Fig. 5. CLURD Combinations for English characters – Group 1
Fig. 6. CLURD Combinations for English characters – Group 2
CLURD: A New Character-Inputting System Using One 5-Way Key Module
43
Fig. 7. CLURD Combinations for English Characters – Group3
The Figs. 6 and 7 illustrate the key combinations of 'A', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'O', 'P', 'Q', 'R', 'S', 'T', 'X' and 'Z'. The data generator assembles 'V' and 'Y' as it does 'U' and 'T', respectively, however the second key should be pressed longer than a predetermined time. In the case that the language mode is set to be the Chinese Pinyin, ' ' is generated instead of 'V'.
Ü
3 Functionality and Usability of the CLURD System The CLURD system is compared with the existing methods, that is, the 'chun-ji-in' Korean character inputting method which is used in mobile phone terminals produced in Samsung Electronics Co., Ltd., the 'naragul' Korean character inputting method which is used in mobile phone terminals produced in LG Electronics Co., Ltd., and the Korean character inputting method which is used in mobile phone terminals produced in SK Telecom Co., Ltd., and illustrated in Table 1. In Table 1, the numeric values are theoretical input time expressed in units of milliseconds (ms; 1/1000 seconds). The numeric values in the parentheses means the number of the input typographical hits. These data in the specification have been Table 1. Theoretical comparison with the existing methods in light of typing speed and number of key clicks
Sentence to be input
사랑해 지금 전화해줘 늦을 것 같으니까 조금만 기다려
Samsung
LG
SK
CLURD
8975(13) 19800(28)
6740(9) 19070(27)
6805(10) 18935(24)
6480(16) 14580(36)
40115(54)
36435(46)
37350(50)
29970(74)
44
H. Ji and T. Kim
cited from the paper of Kim S., Kim K., Myung R. [1], and the data of CLURD are calculated in the same manner by the present authors. The CLURD system uses five keys that are adjacent to one another and located below a thumb of the user. Thus, a time consumed for moving fingers is greatly reduced compared with the existing methods, and the user’s gaze needs not to be shifted between the screen and keys because the user can easily locate five adjacent keys forming a ‘+’ shape located under his/her thumb. Thus, although the number of typographical hits increases by 50%, the total input time greatly decreases in comparison with the existing methods. Here, since the above-indicated times are theoretical input times calculated by a formula, and are presented for a purpose of comparison, an input can be performed within a shorter or longer time in all systems described. Also, the CLURD system has a merit in that the user is capable of inputting characters with the same hand holding his/her phone. The CLURD system can be used, though some level of caution is necessary for a safety reason, even when the user walks, drives a car, or puts the hand taking hold of the mobile device under table or in his/her pocket. The CLURD system does not have the kinds of problems that the existing methods have. That is, it does not need a splitter or a modifier key. The users only have to recall the combinations and input in the device using the key module. Also, the fact that the CLURD system has the space datum included in the key combinations improves its usability, according to the testers who experienced the system for a few days.
4 Limitation As the CLURD system works based on the predetermined key combinations for Korean and English characters, the users need to invest some time and efforts to Table 2. Results of memory test using an instructional animation (Korean)
CLURD: A New Character-Inputting System Using One 5-Way Key Module
45
memorize the combinations. Thanks to the fact that the combinations are not random, but associated with the shapes of the characters fairly closely, the memory burden seems not significant. Table 2 is the results of a memory test with junior high school students.
5 Augmentation of the CLURD System For the case in which the user of the CLURD system decides not to monitor the screen while typing for any reason, it should be better to add some sort of function notifying him/her in the case in which an error has occurred. Because the user’s vision is likely to have been allocated to another task in such a situation, it should be wise to utilize another sensory modality. A short vibration may be a good idea in case of a cell phone. Also, if the CLURD system is used as the main communicating device by speech-handicapped persons, it can be combined with a text to speech (TTS) module which converts texts into sounds.
6 Devices Using the CLURD System In the case of cell phone, PDA or wearable computer, the CLURD system can be used as a substitute for a regular keypad/keyboard. In the case of the remote controller for a digital TV set, the CLURD system is used for the purpose of searching program titles, inputting identification and password, or writing in the user’s address to order a product. Some examples of devices(cell phone, TV remote, door lock, typing mouse) are shown in Fig. 8. Other examples, though not shown, may include watch phones, wearable computers, and so on. For watch phone, the 5-way key module can be installed on the surface of the window in the form of transparent thermal touch pad. A watch phone can be used as an inputting device for
Fig. 8. Cell phone, remote controller for digital TV set, door lock, typing mouse with the CLURD system installed (Conceptual illustrations or prototype models)
46
H. Ji and T. Kim
Fig. 9. The on-screen keypad and the palm-size full-function keyboard with the CLURD system installed (Available in market); The palm-size keyboard was developed with a financial support from the Korea Agency for Digital Opportunity and Promotion
wearable computers with a wireless connection to the CPU. The on-screen keypad program and the palm-size keyboard shown in Fig. 9 are the first official products in market (introduced in April 2009) that adopt the CLURD system.
7 Conclusion As described above, CLURD is a character inputting system suitable for a mobile device where a sufficient number of keys cannot be arranged due to space restriction. Besides the fact that this character inputting system takes up little space on the surface of a device, the system has remarkable edges that other existing methods cannot materialize. First, it requires only one or two fingers for operation, and this allows the user to allocate the other hand and/or the rest of fingers to other concurrent tasks. This also means that the physically handicapped people who can use only one hand or one finger can type characters speedily. In fact, a Korean lady who is partly paralyzed and thus can use only one hand has passed an official computer literacy test that requires a fair speed of typing only with this palm-size keyboard. Another edge of the CLURD system is that the user does not have to look at the keys while typing since his/her finger knows where the five keys are located. This allows him/her to monitor whether he/she makes any typo by fixing his/her gaze to the screen. The CLURD system is also useful to the blind, especially when they take a note reading a book in Braille. They may read a book with left hand and type in a computer using the CLURD keyboard. (Taking a note on paper is meaningless for them because they cannot read it later.) A possible application for the mute is a hand-held device in which the CLURD system is installed together with a TTS system so that when the user presses the TTS button the device reads what he/she has just typed in the device using the CLURD system. The CLURD system can be nicely installed in the form of a virtual keypad in touch-screen interface. This has been confirmed with the on-screen keypad program designed for tablet computers and the prototype model of typing mouse. The time and efforts needed to get familiar with the key combinations may be the only limitation with the CLURD system. A good sign was that most of the testers (over 2,000 people) who have used add-on software designed for cell phone stated
CLURD: A New Character-Inputting System Using One 5-Way Key Module
47
that the time and efforts needed for learning the combinations were quite minimal and its functionality and usability were remarkable enough for them to willingly make such an investment.
Reference 1. Kim, S., Kim, K., Myung, R.: Hangul input system’s physical interface evaluation model for mobile phone. Journal of Korean Institute of Industrial Engineers 28(2), 193–200 (2002)
Menu Design in Cell Phones: Use of 3D Menus Kyungdoh Kim1, Robert W. Proctor2, and Gavriel Salvendy1,3 1
School of Industrial Engineering, Purdue University, 315 N. Grant St., West Lafayette, IN 47907 USA
[email protected] 2 Department of Psychological Sciences, Purdue University, 703 Third St., West Lafayette, IN 47907 USA
[email protected] 3 Department of Industrial Engineering, Tsinghua University, Beijing, P.R. China
[email protected]
Abstract. The number of mobile phone users has been steadily increasing due to the development of microtechnology and human needs for ubiquitous communication. Menu design features play a significant role in cell phone design from the perspective of customer satisfaction. Moreover, small screens of the type used on mobile phones are limited in the amount of available space. Therefore, it is important to obtain good menu design. Review of previous menu design studies for human-computer interaction suggests that design guidelines for mobile phones need to be reappraised, especially 3D display features. We propose a conceptual model for cell phone menu design with 3D displays. The three main factors included in the model are: the number of items, task complexity, and task type. Keywords: cell phones, menu design, 3D menu, task complexity, task type.
1 Introduction The number of mobile phone users has been steadily increasing due to the development of microtechnology and human needs for ubiquitous communication. People use mobile phones to communicate with their friends, family, and business partners, and also to obtain information through the mobile Internet. Moreover, people use embedded mobile phone features such as games, cameras and wireless Internet for various purposes of entertainment and shopping. Due to increasing features, mental workload of using cell phones has increased. Ling et al. [1] prioritized the design features and aspects of cell phones based on users’ feedback to optimize customers’ satisfaction. Although physical appearance and body color of cell phones had considerable influence on overall user satisfaction, menu design features also played a significant role. Therefore, obtaining a good menu design in cell phones is an important issue. There has been a lot of research about menu design for computers. When it comes to menu dimensions, many researchers have concluded that performance time and errors increase as the hierarchical levels of the menu structure increase [2, 3]. With J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 48–57, 2009. © Springer-Verlag Berlin Heidelberg 2009
Menu Design in Cell Phones: Use of 3D Menus
49
regard to menu type, hierarchical menus are more accurate and faster than fisheye menus [4]. Three-dimensional (3D) displays show many items of a menu at the same time, so they may give the same effect as a broader menu [5]. With regard to adaptability, computer menus that can be customized by users have been shown to be better than ones that adapt automatically [6]. Research on cell phone menu design is relatively recent. Geven, Sefelin, and Tscheligi [7] concluded that narrow hierarchies performed better than broader hierarchies in mobile devices, contrary to menu design in computers. With respect to menu type, Gutwin and Fedak [8] found that people were able to carry out a web navigation task better with the fisheye view than with alternatives. For adaptability, results have been similar to those for computer displays. Customized menus produced better performance and evaluation than the traditional static menu [9]. But, a lack of studies about 3D display for cell phones was found, and in this paper 3D design research is investigated in more detail. At this point, there are no standard interaction devices or interfaces used in 3D environments, and there is a lack of specific best practice guidelines to develop these 3D designs. 3D design is able to convey more information than text or twodimensional (2D) images, and it enhances the usability of the limited screen on a typical wireless device. Interactive 3D can therefore be used to remove some of the complexity and clutter present on menu systems of today’s handsets. 3D icons can be animated to show activity or changes in status, and the depth dimension can be utilized to show urgency or relative importance [10]. Therefore, new standards should be developed to allow personal digital assistants (PDAs) and mobile devices to render 3D applications. Review of previous menu design studies for human-computer interaction suggests that design guidelines for mobile phones need to be reappraised, especially 3D display features. To this end, the main objective of this paper is to propose an overall framework to develop mobile phone menu design guidelines regarding 3D displays. We review menu design components for computers in section 2 and investigate menu design factors for cell phones in section 3. Strengths and weaknesses of 3D design factors are considered in section 4. We compare menu design factors in section 5 and conclude after explaining a model of cell phone menu design in section 6.
2 Menu Design in Computers 2.1 Menu Dimension Many of the early studies of menu design for computers focused on the cognitive factors of a menu’s hierarchical structure and the structure’s impact on end users’ behaviors and performance in retrieving information. Out of this research, studies about whether it is better to have a broad or deep design have been conducted. Jacko et al. [2] suggested three components of hierarchical menu design: menu dimension, task complexity, and user knowledge structure. The results about the menu dimension supported that both performance time and errors increased as the levels of the menu structure increased. That is, depth in an information structure increases the likelihood of navigational errors and also increases performance time [11].
50
K. Kim, R.W. Proctor, and G. Salvendy
Seppala and Salvendy [3] also drew the conclusion that a broader mode of data presentation is more effective than a deeper one. Because searching back and forth through the menu system decreases the speed and accuracy of performance, the broader menu has better performance in the case of a personal computer. This is because increased depth involves additional visual search and decision-making, and greater uncertainty as to the location of target items due to the increased number of menu frames [12]. In other words, as the depth increases and the number of responses needed while going through a menu tree increases, more time for decision making and responding is required [3]. 2.2 Menu Type Menu structure can be classified as hierarchical and fisheye [4]. Fisheye is a menu display method that shows a region of the menu at high magnification, while items before and after that region are shown at gradually reduced sizes. Hornbaek and Hertzum [4] provided evidence that, for finding known items, conventional hierarchical menus were more accurate and faster than fisheye menus. Also, participants rated hierarchical menus as more satisfying than fisheye menus. For browsing tasks, the menus did not differ with respect to accuracy or selection time. Fisheye interfaces have an advantage in that they can accommodate many menu items in a limited amount of screen space by showing part of an information space at high magnification, while other parts are shown at low magnification to provide context. However, performance remained worse with fisheye menus than with hierarchical menus because the latter impose lower mental demands on users [4]. Within a hierarchical menu, cascading and indexed menus can be compared [13]. Participants searched three types of menu layouts: categorical index; horizontal cascading; vertical cascading. Search time differences between the three menu layouts were detected that strongly favored the index menu. One possible reason for this result is that the items in the index menus were in closer proximity. Another is that the index menus were centrally located on the screen, and thus would have been easier to see and acquire. 2.3 Adaptability Some commercial applications now have adaptable interfaces. For example, the Start Menu in Microsoft Windows XPTM has an adaptive function that provides automatically generated shortcuts to frequently used applications. Microsoft Office also provides Smart Menus, which are an adaptive mechanism where infrequently used menu items are hidden from view. Understanding these interfaces through strong empirical and theoretical studies is particularly important, because adaptive interfaces are now being introduced into productivity software and used by an increasing number of people [14]. Mitchell and Shneiderman [15] compared dynamic vs. static menus using a menu-driven computer program. Subjects who used adaptive dynamic menus for the first set of tasks were significantly slower than those who used static menus. Moreover, 81% of the subjects preferred working with static menus to working with dynamic menus. This preference
Menu Design in Cell Phones: Use of 3D Menus
51
likely is because dynamic menus can slow down first-time users, at least until they become accustomed to this interaction style. Findlater and McGrenere [6] compared the measured and perceived efficiency of three menu conditions: static, adaptable and adaptive. They found that users generally preferred the customizable version to the adaptive menus. In terms of performance, adaptive menus were not faster than either of the other conditions. User-driven customization is a more viable approach for personalizing user interfaces than systemdriven adaptation. The static menu was found to be significantly faster than the adaptive menu, and the adaptable menu was found to be significantly faster than adaptive menu under certain conditions. But, in terms of accuracy, there were no differences. However, the majority of users preferred the adaptable menu overall and ranked it first for perceived efficiency. Therefore, this study suggests that system-driven adaptation is not helpful.
3 Menu Design in Cell Phones 3.1 Menu Dimension As screens become smaller, the information they display changes more extensively with each scrolling action, making it more difficult to refocus on the page. In this way, screen size affects the navigation behavior and perceptions of mobile phone users [11]. Therefore, the breadth of information structures should be adapted to anticipated screen size. The advantage of depth is that it encourages funneling; the disadvantage is that it induces errors and increases the number of page transactions. On the other hand, the advantage of breadth is that it reduces navigation errors and the number of page transactions; the disadvantage is that it leads to crowding. Therefore, a user encountering greater depth has fewer options to process on a single page. Thus, the cognitive load on the user is reduced. Findings consistently have suggested an advantage of employing a deeper menu structure to achieve better user performance and accuracy. Geven et al. [7] showed that people perform better with narrow hierarchies than with broader hierarchies on small screens. Contrary to computers, where many options are usually presented at once, it is better to use a layered design in cell phones. Huang [16] showed that users prefer a less extensive menu structure on a smallscreen device. This result supports the recommendation of not having a broad menu structure on a small screen. With less space to display information, designers of cell phones tend to chunk menu items of a broader menu into several pages or screens. This chunking requires end-users to employ more scrolling operations, maintain more information in working memory, and engage in more searching and navigation behaviors. The consequence is to reduce the speed and accuracy in use of the menus. The following describes the two suggestions that Huang [16] developed: (1) Reduce both breadth and depth of the menu. (2) Instead of displaying only a limited number of items on one screen, include more menu items and options in one page.
52
K. Kim, R.W. Proctor, and G. Salvendy
Dawkins [9] also suggests that filling the screen as much as possible without requiring scrolling should be the ideal breadth of the menu. 3.2 Menu Type and Adaptability Many of the current visualization methods aimed at small screens rely on distorting the view. The viewpoint information is manipulated in a way that enables seeing important objects in detail, and the whole information space can be displayed at once with very low amount of detail [17]. The rubber sheet is one of view distortion techniques that allow the user to choose areas on the screen to be enlarged. Zooming and zoomable user interfaces (ZUI) are another way of presenting large information spaces even on a small screen. Combs and Bederson [18] studied image browsers and found that their system, based on a ZUI method (as well as 2D thumbnail grid), outperformed 3D browsers in terms of retrieval time and error rate. Displaying the overview and the detail at the same time is also more beneficial than the traditional linear format because the global context allows faster navigation [8]. Gutwin and Fedak [8] found that people were able to carry out a web navigation task better with the fisheye view. Some phones are already being designed with a fisheye display for selected items to be salient and clear. Therefore, a fisheye menu may be better than a 2D hierarchical menu. In computers, users can create folders, reorder the layout, and make shortcuts. But a mobile phone has limited screen size and a small input device. Moreover, telecommunication carriers want the buttons to be used for their wireless Internet service. They are therefore reluctant to offer many customization functions to users. In other words, mobile phones do not provide enough adaptation functions. Dawkins [9] evaluated personalized menus alongside a traditional static menu structure based on user preference and performance. He concluded that customized menus had better performance and evaluation than the traditional static menu. Therefore, customers seem to want more customization functions in their cell phones from the perspectives of performance and satisfaction.
4 3D Design 4.1 Benefits of 3D Design Human information-processing has evolved to recognize and interact with a 3D world. And the 3D design space is richer than the 2D design space, because a 2D space is part of 3D space. It is always possible to flatten out part of a 3D display and represent it in 2D [19]. Therefore, it is unsurprising that 2D interfaces have performed relatively poorly. For example, Ware and Franck [20] conducted an experiment that was designed to provide quantitative measurements of how much more (or less) can be understood in 3D than 2D. Results showed that the 2D interface was outperformed by 3D interfaces. These results provide strong reasons for using advanced 3D graphics for interacting with a large variety of information structures [20].
Menu Design in Cell Phones: Use of 3D Menus
53
The 3D interfaces make it possible to display more information without incurring additional cognitive load, because of pre-attentive processing of perspective views (e.g., smaller size indicates spatial relations at a distance). An ability to recognize spatial relations based on 3D depth cues makes it possible to place pages at a distance (thereby using less screen space) and understand their spatial relations without effort [21]. As described before, there are many 3D depth cues that can be provided to facilitate spatial cognition. The most obvious of these are perspective view and occlusion. Using these cues, the user gets the advantages of a 3D environment (better use of space, spatial relations perceived at low cognitive overhead, etc.). 3D allows larger menu items than the screen size. This would be a desirable feature for small screens that have a restricted screen resolution and size [22]. The effect of 3D is to increase the effective density of the screen space in the sense that the same amount of screen can hold more objects, which the user can zoom into or animate into view in a short time. It seems reasonable that 3D can be used to maximize effective use of screen space [23], especially in cell phones for which the screens are small screens. The use of 3D models on the Internet is gaining popularity, and the number of 3D model databases is increasing rapidly because 3D interfaces enable a more natural and intuitive style of interaction [24]. Since the use of 3D models is becoming more common on various cellular phone web sites, development of algorithms that retrieve similar information will be important in cell phone menu design [25]. 4.2 Weaknesses of 3D Design Creating a 3D visualization environment is considerably more difficult than creating a 2D system with similar capabilities. As the study of Cockburn and McKenzie [26] suggests, one should not assume that use of 3D provides more readily accessible information. In determining whether to implement a 3D display, designers should decide whether there are enough subtasks that would benefit from 3D representations. The complexity and the consistency of the user interface for the whole application must also be weighed in the decision. In the study of Ware [19], 3D navigation methods took considerably longer than 2D alternatives. Even if somewhat more information can be shown in 3D than in 2D, the rate of information access may be slower, and 3D applications may have greater visual complexity than 2D applications [27]. People often find it difficult to understand 3D spaces and to perform actions in them. It is clear that simply adapting traditional WIMP (windows, icons, menus, and pointers) interaction styles to 3D does not provide a complete solution to this problem. Rather, novel 3D user interfaces, based on interactions with the physical world, must be developed. Jones and Dumais [28] have suggested that little significant value is provided by adding physical location information to the storage and subsequent retrieval of a document over and above simply providing a semantic label for the same purposes. 4.3 Direct Comparison between 2D and 3D Few prior studies have directly compared 2D and 3D interactive systems. Also, there is a surprising lack of empirical research into the benefits (or costs) that are produced
54
K. Kim, R.W. Proctor, and G. Salvendy
by moving from 2D to 3D. Cockburn and McKenzie [29] compared subject’s efficiency in locating files when using Cone-Trees (a 3D technique for exploring hierarchical data structures) and when using a ‘normal’ folding tree interface similar to that used in Windows Explorer. Results showed that the subjects took longer to complete their tasks when using the cone interface. They rated the cone interface as poorer than the normal one for seeing and interacting with the data structure. Also, Cockburn and McKenzie [26] showed no significant difference between task performance in 2D and 3D, but a significant preference for the 3D interfaces. Recently there has been a growth of interest in 3D interactive systems for everyday ‘desktop’ computing applications, such as document and file management. However, the relative value of the third visual dimension in cell phone menu design has not previously been evaluated.
5 Models for Cell Phone Menu Design Jacko et al. [2] proposed modifications to an information-processing model developed by Salvendy and Knight [30]. In this model, three constructs of hierarchical menu retrieval were proposed: menu dimension, task complexity, and knowledge structure. Figure 1 illustrates a version of Jacko et al.’s [2] information-processing model extended to cell phone menu retrieval operation. The model takes advantage of the natural 3D human information-processing capabilities for cell phone menu interfaces, with distinctions similar to those identified by Jacko et al. The three main factors for cell phone menu design within 3D display included in the model are: the number of items, task complexity, and task type. Cell phones support more features such as broadcasting, mobile wallet and health condition sensor, etc. This is consistent with an issue raised by Norman [31], which is “a tendency to add to the number of features that a device can do, often extending the number beyond all reasons” (p. 173). With human cognitive limitations, a cell phone with too many features may overwhelm users due to its complexity [1]. Under these circumstances, it is important to investigate how the number of items can influence 3D menu design in cell phones. The number of items could influence menu dimensions, resulting in effects on perception, cognition, and motor response time. In this way, the number of items is an important characteristic of a virtual menu that will influence the item selection time. Moreover, inclusion of many menu items may decrease the usability of a 2D display solution. Therefore, deciding whether or not to use 3D design should depend on the number of items per menu screen. Task complexity can impact performance and satisfaction of 3D menu design because in a 3D environment the spatial relationships are perceived at low cognitive overhead [22]. Thus, performing a complex task may be better in a 3D environment than in a 2D environment. On the other hand, a 3D display sometimes has greater visual complexity. Therefore, direct comparisons between 2D and 3D menus for different levels of task complexity are needed. Task type influences the perceptual information required, the cognition operations involved in using that information, and necessary motor responses. Experiments need to be conducted to validate the proposed conceptual model.
Menu Design in Cell Phones: Use of 3D Menus
55
Fig. 1. Modified Information-processing Model for Cell Phone Menu Operation
6 Conclusion The widespread use of cell phones for a variety of purposes provides evidence that they are shifting from just a communication tool to being an integral part of people’s everyday life. It is important to study cell phone menu design because, though menu design plays a crucial role in cell phone usability, little work exists on developing cell phone menu design. Three factors were identified that may influence performance of menu retrieval tasks with 2D and 3D displays in cell phones: the number of items, task complexity, and the type of tasks. These three factors are included in the proposed conceptual model for cell phone menu design with 3D displays. Research designed to validate this model should provide insights into the human information-processing requirements of various cell phone menu interfaces.
References 1. Ling, C., Hwang, W., Salvendy, G.: A survey of what customers want in a cell phone design. Behaviour & Information Technology 26, 149–163 (2007) 2. Jacko, J.A., Salvendy, G., Koubek, R.J.: Modelling of menu design in computerized work. Interacting with Computers 7, 304–330 (1995) 3. Seppala, P., Salvendy, G.: Impact of depth of menu hierarchy on performance effectiveness in a supervisory task: computerized flexible manufacturing system. Human Factors 27, 713–722 (1985)
56
K. Kim, R.W. Proctor, and G. Salvendy
4. Hornbaek, K., Hertzum, M.: Untangling the usability of fisheye menus. ACM Trans. on Computer-Human Interaction, Article 6, 1–32 (2007) 5. Dachselt, R., Ebert, J.: Collapsible cylindrical trees: a fast hierarchical navigation technique. In: Information Visualization, INFOVIS 2001, pp. 79–86 (2001) 6. Findlater, L., McGrenere, J.: A comparison of static, adaptive, and adaptable menus. In: Proceedings of the 2004 conference on Human Factors in Computing Systems, pp. 89–96 (2004) 7. Geven, A., Sefelin, R., Tscheligi, M.: Depth and breadth away from the desktop: the optimal information hierarchy for mobile use. In: Proceedings of the 8th conference on Human-Computer Interaction with Mobile Devices and Services, pp. 157–164 (2006) 8. Gutwin, C., Fedak, C.: Interacting with big interfaces on small screens: a comparison of fisheye, zoom, and panning techniques. In: Proceedings of the 2004 conference on Graphics Interface, pp. 145–152 (2004) 9. Dawkins, A.L.: Personalized Hierarchical Menu Organization for Mobile Device Users. Vol. Master. North Carolina (2007) 10. Beardow, P.: Enabling Wireless Interactive 3D. article retrieved from the Superscape Plc (June 2004), http://www.superscape.comin 11. Chae, M., Kim, J.: Do size and structure matter to mobile users? An empirical study of the effects of screen size, information structure, and task complexity on user activities with standard web phones. Behaviour and Information Technology 23, 165–181 (2004) 12. Jacko, J.A., Salvendy, G.: Hierarchical menu design: Breadth, depth, and task complexity. Perceptual and Motor Skills 82, 1187–1201 (1996) 13. Bernard, M., Hamblin, C.: Cascading versus Indexed Menu Design. Usability News 5 (2003) 14. Gajos, K.Z., Czerwinski, M., Tan, D.S., Weld, D.S.: Exploring the design space for adaptive graphical user interfaces. In: Proceedings of the working conference on Advanced Visual Interfaces, pp. 201–208 (2006) 15. Mitchell, J., Shneiderman, B.: Dynamic versus static menus: an exploratory comparison. ACM SIGCHI Bulletin 20, 33–37 (1989) 16. Huang, S.C.: Empirical Evaluation of a Popular Cellular Phone’s Menu System: Theory Meets Practice. Journal of Usability Studies, 136–150 (2006) 17. Hakala, T., Lehikoinen, J., Aaltonen, A.: Spatial interactive visualization on small screen. In: Proceedings of the 7th international conference on Human Computer Interaction with Mobile Devices & Services, pp. 137–144 (2005) 18. Combs, T.T.A., Bederson, B.B.: Does Zooming Improve Image Browsing? In: Proceedings of the fourth ACM Conference on Digital Libraries, pp. 130–137 (1999) 19. Ware, C.: Information Visualization: Perception for Design. Morgan Kaufmann, San Francisco (2004) 20. Ware, C., Franck, G.: Evaluating stereo and motion cues for visualizing information nets in three dimensions. ACM Transactions on Graphics 15, 121–140 (1996) 21. Robertson, C.M., Larson, K., Robbins, D.C., Thiel, D., van Dantzich, M.: Data mountain: using spatial memory for document management. In: Proceedings of the 11th annual ACM Symposium on User Interface Software and Technology, pp. 153–162 (1998) 22. Rekimoto, J.: Tilting operations for small screen interfaces. In: Proceedings of the 9th annual ACM Symposium on User Interface Software and Technology, pp. 167–168 (1996) 23. Robertson, C.S.K., Mackinlay, J.D.: Information visualization using 3D interactive animation. Communications of the ACM 36, 57–71 (1993)
Menu Design in Cell Phones: Use of 3D Menus
57
24. Molina, J.P., Gonzalez, P., Lozano, M.D., Montero, F., Lopez-Jaquero, V.: Bridging the Gap: Developing 2D and 3D User Interfaces with the IDEAS Methodology. In: Jorge, J.A., Jardim Nunes, N., Falcão e Cunha, J. (eds.) DSV-IS 2003. LNCS, vol. 2844, pp. 303–315. Springer, Heidelberg (2003) 25. Suzuki, M.T., Yaginuma, Y., Sugimoto, Y.Y.: A 3D model retrieval system for cellular phones. In: IEEE International Conference on Systems, Man and Cybernetics, vol. 4, pp. 3846–3851 (2003) 26. Cockburn, A., McKenzie, B.: 3D or not 3D?: evaluating the effect of the third dimension in a document management system. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 434–441 (2001) 27. van Dam, A.: Post-WIMP user interfaces. Communications of the ACM 40, 63–67 (1997) 28. Jones, W.P., Dumais, S.T.: The spatial metaphor for user interfaces: experimental tests of reference by location versus name. ACM Transactions on Information Systems 4, 42–63 (1986) 29. Cockburn, A., McKenzie, B.: An Evaluation of Cone Trees. People and Computers XIVUsability Or Else! 425–434 (2000) 30. Salvendy, G., Knight, J.: Psychomotor work capabilities. In: Salvendy, G. (ed.) Handbook of Industrial Engineering, pp. 1–5 (1982) 31. Norman, D.A.: The Psychology of Everyday Things. Basic Books, New York (1988)
Mobile Interfaces in Tangible Mnemonics Interaction Thorsten Mahler, Marc Hermann, and Michael Weber Institute of Media Informatics University of Ulm, Ulm, Germany {thorsten.mahler,marc.hermann,michael.weber}@uni-ulm.de
Abstract. The Tangible Reminder Mobile brings together tangible mnemonics with ambient displays and mobile interaction. Based on the Tangible Reminder Project we present a new interface for mobile devices that is capable of viewing and editing data linked to real world objects. An intelligent piece of furniture equipped with RFID-sensors and digitally controlled lighting keeps track of appointments linked to real world objects that are placed in its trays. The mobile interface now allows the complete waiving of classic computer interaction for this ambient shelf. Instead, by implementing the toolglas metaphor, the mobile interface can be used to edit and view linked data to objects.
1
Introduction
Mark Weiser [1] formulated in 1991 his vision of ubiquitous computing stating the goal of bringing together the virtual world and the real world in a constant intertwining. The direct result for human computer interaction is that the interaction with real life artefacts can affect the virtual representations, thus making virtual objects easily graspable. According to Holmquist et al. [2] a lot of research is done in this domain focussing on different aspects depending on the primary goal they pursue: graspable interface, tangible interface, physical interface, embodied interface, to name just a few. Whatever the name, they are all unified by their common goal, to enrich the interaction with virtual objects by physicality. One of the first projects to describe this linkage between physical objects and virtual representations is Ishii’s and Ullmer’s paper ”Tangible Bits” [3]. Based on their observation with coupling everyday objects and virtual information, ”Bits and Atoms”, they tackle an interesting question: how can physical artefacts become interfaces for virtual objects and, from our point of view even more important, how can these interfaces be recognized? Their solution for this problem is the introduction of virtual light or virtual shadow. Both concepts show the information in a glow or shadow around the physical object in question. For example the software on the new Microsoft surface [4] table makes use of this concept. However, the visualization of information into the very vicinity of real objects requires a technically very well equipped environment. The Project Urp J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 58–66, 2009. c Springer-Verlag Berlin Heidelberg 2009
Mobile Interfaces in Tangible Mnemonics Interaction
59
[5] uses a large display underneath the objects in order to view the virtual shadows. The SenseBoard uses a projector to display detailed information on a ”View Details” command puck when it is placed over an item on the SenseBoard [6]. The AuraOrb provides newsticker information when a eyecontact sensor detects the focus of a user on it [7]. But if neither the environment nor the object itself is equipped with displaying technology, the augmentation of the objects remain invisible. The solution for this problem is the use of other metaphors, for example the see-through metaphor or the magic lens [8] [9]. The information linked to a real object is not projected into the real world. Rather a mobile device shows the information linked directly on its screen. A mobile device also compensates the non-ubiquity of intelligent environments through mobility [10]. In the Tangible Reminder Project we now make use of this technique. The next section gives an overview of the first prototype of the Tangible Reminder. After that we will focus on the concept of the mobile interface and finally present the whole system with this new interface.
2
The Tangible Reminder
The Tangible Reminder is a device combining the benefits of ambient displays and tangible interaction with personal objects. It is designed as a tool to keep track of appointments and deadlines and particularly to remind the user on upcoming events. As shown in Fig. 1. the ambient display subsystem in our prototype consist of three trays in which freely chosen RFID-tagged objects can be placed. According to the urgency of the appointment linked to every object the trays are colored green, yellow or red. Showing the appropriate color, the display remains calm, avoiding distraction from other tasks until the deadline is due. After that it begins flashing, grasping the user’s attention. [11] A main idea was to let the user choose personal objects with specific associations in the topic of the appointments. We use RFID technology to identify the objects in the trays. As RFID-Tags have gotten cheaper and smaller, a user can tag any object he wants to. Choosing a personal object makes it easier to mentally link it with appointments. A study of van den Hoven et al. showed that nearly all personal souvenirs bear a mass of information for the owner [12]. The memories associated with the object can give a clue when spotting it in the tray, a mnemonic to remind the topic of the appointment. If more information is needed, the user has to interact with the system. In our first prototype we established a non-ubiquitous way of allowing this ”easy transition to more in-depth information”, one of the heuristics both for graphical user interfaces [13] and ambient displays [14]. If additional information was desired, the object had to be removed from the tray and put on an interaction platform connected with an ordinary computer or laptop. The information was then displayed on the monitor. Also, before usage an object had to be technically linked to an appointment and additional information like addresses, images etc. The inspection system and the input system were the same in the first prototype,
60
T. Mahler, M. Hermann, and M. Weber
Fig. 1. The ambient display subsystem of the Tangible Reminder, with an object in each tray, showing the different colors for different states
so the user was able to change linked information when an object was placed on the platform. This form of inspection and input on traditional computers is not very ubiquitous, so we will present an alternative to that approach here.
3
A Concept for Mobile Interfaces for Tangible Mnemonics
One goal of tangible interaction is the masking of computer interaction by implicit actions with real objects. But this masking is also the problem for tangible interaction systems application in real life. This masking hinders the use of ready-mades, objects present in everyday life as it clouds the linkage between the real object and the virtual data. Neither the linkage itself, i.e. which data an object represents can be seen nor it is clear how this data can be edited. Both issues are solved in the original Tangible Reminder system by braking with the paradigm of implicit interaction by simply using a laptop as editing and viewing station. While this is true for objects linked to absolute appointments, the first steps to further pushing the computer in the background is taken. The introduction of relative appointments supersede the explicit computer interaction. Instead, the simple act of putting an object in the tray already triggers the
Mobile Interfaces in Tangible Mnemonics Interaction
61
appointment. For instance, the simple act of putting a special tea cup into the Tangible Reminder shelf results in an alert 3 minutes in the future. Clearly, this way of editing linked data is an improvement. Nevertheless, the visualization of the linkage and thus the display of the data still involves the computer. To overcome this drawback, Ishii and Ullmer [3] have proposed to use digital shadows or digital light. Whilst this metaphor is clearly interesting and integrates nicely into the real world, it is also very demanding as it needs a lot of special hardware and sensor integration which is only present in special rooms today. Instead we decided to tackle the challenges of displaying linkage and editing linked data via the use of mobile devices. Not only can they be easily carried to the real object in question, but they also perfectly support the see-through metaphor of Bier et al. [8] A portable device in this case acts as a magical toolglas, showing the link to a real object when placed over it. In the display the user can see the linked data and manipulate it if desired. This metaphor is easy to understand and to use. Nevertheless, the decision in favor of small and portable devices does not completely hide the computer. It rather shifts the interaction part to a small device. This device, though a complete computer as well, is perceived as being much simpler and easier to use. It does not cloud the computer as such but it clouds its complexity [15]. With the adoption of small computing devices, especially cell-phones, in time even smart-phones this approach is getting even more appealing.
4
The Tangible Reminder Mobile
The ambient display subsystem (see Fig. 1) of the original Tangible Reminder follows Weiser’s vision of implicit computer interaction and fits nicely into the surrounding. It brings together small personal objects acting as mnemonics, with a calm but also demanding interface if needed. Therefore, we decided to keep this part of the Tangible Reminder unchanged but to completely replace the input and inspection subsystem. 4.1
Interaction with the Tangible Reminder Mobile
To interact with an object in the Tangible Reminder Mobile system it is sufficient to simply put the PDA, containing the new mobile input and inspection subsystem, near an object enhanced with an RFID. Via the integrated RFIDReader the Tangible Reminder Mobile recognizes that the object nearby can be associated with virtual data. It queries the database and retrieves the stored data. This data is shown on the display pane. Fig. 2 depicts the scanning of a globe, which is in this case associated with a journey to San Diego. Besides the time additionally the reminder period is shown. The simple act of moving the Tangible Reminder Mobile near an object shows its linkage and capacities. The act of posing the Tangible Reminder Mobile over an object results in showing the virtual content of a real object as if watched through a magic lens for digital data.
62
T. Mahler, M. Hermann, and M. Weber
Fig. 2. Scanning the globe for the associated appointment
4.2
The Mobile Input and Inspection Subsystem
The subsystem on the PDA serves for two purposes: It shows the data linked to a certain object plus it is capable of editing this data or adding data if not yet stored with an object. Suitable objects contain a RFID-Chip simply sending an ID whenever a reader comes into close vicinity. With the readers we are using this reading distance is limited to about 5 cm which is very small. The reader therefor has to be posed directly over the object and its RFID to automatically raise the inspection screen. As we are dealing with appointments and reminders, the inspection screen shows the name of the appointment together with the exact date and the period after which a reminder should occur. Additionally a photograph of the associated real object can be shown. Fig. 3 shows this dialog in detail. In order to add or edit data stored with an object, the edit screen has to be opened. There all relevant data can be entered. To keep the interface usable we decided for special input fields like date-pickers to keep the pen interaction simple. The only text-field capable of free text is the appointment name. All other data can only be modified via controls. This better meets the accuracy of
Mobile Interfaces in Tangible Mnemonics Interaction
Fig. 3. The mobile device running the Tangible Reminder Mobile program, displaying the appointment view
63
Fig. 4. The appointment form allows for changing the associated appointment and the way of reminding
pen interaction on mobile devices as well as it reduces input errors. Fig. 4 shows an example for the input screen. Here the appointment linked with the globe is changed from an appointment on the 24th of December to a journey in June. 4.3
System Design and Changes Compared to the Original Tangible Reminder
To allow for the nice and simple magic-lens approach we decided to port only the input and inspection subsystem to the PDA while leaving the underlying concept unchanged to further support the display subsystem. The original Tangible Reminder makes use of RFID-Chips to recognize objects and to link them to appointments. There are different ways of making a small device capable of reading RFIDs. Instead of using an extra device, we equipped our PDA with a SD-Card-RFID-Reader. This solution integrated the Reader into the PDA and keeps the system small instead of using separated devices. The laptop in the original Tangible Reminder did not only work as an input device, it also kept the data stored for the linked objects. This data was retrieved by the display system to control the reminder functions of the shelf. The new Tangible Reminder system separates this earlier intertwining and divides the system according to their functions. The virtual data is now stored on a server
64
T. Mahler, M. Hermann, and M. Weber
that provides all domain related functions via a web-service. This service can be contacted and controlled by the display subsystem of the shelf as well as the mobile magic-lens subsystem. Thus, either subsystem has to deal only with the exact functions it has to fulfill. We rather make the virtual space underlying the real world accessible independent of environmental hardware by lending real objects a mobile interface, in our case rendering the Tangible Reminder system completely independent of traditional computer interaction.
5
Conclusion
With the Tangible Reminder Mobile system we presented a mobile interface for an ambient mnemonic system, the Tangible Reminder. This system brings together ambient displays with tangible interaction by reminding a user on appointments previously linked to everyday objects, functioning as mnemonics. To establish the linkage between real life objects and virtual data we needed a classic graphical computer interface in the original system. The Tangible Reminder Mobile System now provides an interface on a PDA making the Laptop as input device unnecessary. It thereby renders the Tangible Reminder system to an intelligent piece of furniture not recognized as computer interface. The mobile interface itself makes the usage of the Tangible Reminder more natural by implementing the magic lens metaphor. The smaller device not only makes the classic interface disappear, it also keeps the interface seemingly simpler. It combines the tools for information display and data manipulation and makes the interface mobile like the user chosen real life mnemonic objects. The interface therefor bridges the gap between the real and the virtual world and solves the problems of data manipulation and display in an elegant way. No classic computer interface is needed, just a PDA is used, a device that gets more and more common. The switch to this mobile interface is another step towards real implicit interaction with everyday objects with no computers visible, towards the vision of natural interaction with virtual data and ubiquitous computing alike.
6
Future Work
The Tangible Reminder Mobile system is just a step on the way to implicit and computer-less machine interaction. Further development is needed and planned in mobile interface improvement and in the field of implicit interaction. The integration of camera images to literally implement a see-through tool could make the interface more intuitive. However, from our point of view the benefit is marginal unless the system recognizes and marks the detected object which is a hard problem without marker usage. The next step therefor would be to recognize objects visually and superimpose digital data directly in the video image, implementing the digital shadow metaphor of Ishii and Ullmer [3].
Mobile Interfaces in Tangible Mnemonics Interaction
65
Another direction the Tangible Reminder can evolve is the use of real life artefacts that allow for programming by combination and handling of tool and mnemonic. This way, the computer could be made completely invisible and virtually superfluous. Yet, the problem of information display has to be solved. This could be done by switching to another media, sound and voice feedback for instance or attaching a display or projector to give visual feedback. Both extensions will make the Tangible Reminder an even more integrated intelligent piece of furniture not recognized as a common computer interface, lowering the inhibitions for computer usage.
Acknowledgments Many thanks to the programming team of the Tangible Reminder mobile project, Sung-Eun Kang and Michele Pinto, who worked on different aspects of mobile device portation and hardware integration as part of a practical course during their studies at the Institute of Media Informatics, University of Ulm.
References 1. Weiser, M.: The computer for the twenty-first century. Scientific American 265, 94–104 (1991) 2. Holmquist, L.E., Schmidt, A., Ullmer, B.: Tangible interfaces in perspective: Guest editors’ introduction. Personal Ubiquitous Comput. 8(5), 291–293 (2004) 3. Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits and atoms. In: CHI 1997: Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pp. 234–241. ACM Press, New York (1997) 4. Microsoft Corporation: Microsoft Surface (2008), http://www.microsoft.com/surface/ 5. Underkoffler, J., Ishii, H.: Urp: a luminous-tangible workbench for urban planning and design. In: CHI 1999: Proceedings of the SIGCHI conference on Human Factors in Computing Sstems, pp. 386–393. ACM Press, New York (1999) 6. Jacob, R.J.K., Ishii, H., Pangaro, G., Patten, J.: A tangible interface for organizing information using a grid. In: CHI 2002: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 339–346. ACM Press, New York (2002) 7. Altosaar, M., Vertegaal, R., Sohn, C., Cheng, D.: Auraorb: using social awareness cues in the design of progressive notification appliances. In: OZCHI 2006: Proceedings of the 20th Conference of the Computer-Human Interaction Special Interest Group (CHISIG) of Australia on Computer-Human Interaction: Design: Activities, Artefacts and Environments, pp. 159–166. ACM Press, New York (2006) 8. Bier, E.A., Stone, M.C., Pier, K., Buxton, W., DeRose, T.D.: Toolglass and magic lenses: the see-through interface. In: SIGGRAPH 1993: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, pp. 73–80. ACM, New York (1993) 9. Stone, M.C., Fishkin, K., Bier, E.A.: The movable filter as a user interface tool. In: CHI 1994: Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pp. 306–312. ACM Press, New York (1994)
66
T. Mahler, M. Hermann, and M. Weber
10. Mahler, T., Weber, M.: Mobile Device Interaction in Ubiquitous Computing. In: Advances in Human-Computer Interaction, pp. 311–330. In-Tech Education and Publishing (October 2008) ISBN 978-953-7619-15-2 11. Hermann, M., Mahler, T., de Melo, G., Weber, M.: The tangible reminder. Intelligent Environments. In: 3rd IET International Conference on Intelligent Environments, IE 2007, pp. 144–151 (September 2007) 12. van den Hoven, E., Eggen, B.: Personal souvenirs as ambient intelligent objects. In: sOc-EUSAI 2005: Proceedings of the 2005 Joint conference on Smart Objects and Ambient Intelligence, pp. 123–128. ACM Press, New York (2005) 13. Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: CHI 1990: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 249–256. ACM Press, New York (1990) 14. Mankoff, J., Dey, A.K., Hsieh, G., Kientz, J., Lederer, S., Ames, M.: Heuristic evaluation of ambient displays. In: CHI 2003: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 169–176. ACM Press, New York (2003) 15. Maeda, J.: The Laws of Simplicity (Simplicity: Design, Technology, Business, Life). MIT Press, Cambridge (2006)
Understanding the Relationship between Requirements and Context Elements in Mobile Collaboration Sergio Ochoa1, Rosa Alarcon2, and Luis Guerrero1 1
Universidad de Chile, Computer Science Department {sochoa,luguerre}@dcc.uchile.cl 2 Pontificia Universidad Catolica de Chile, Computer Science Department
[email protected]
Abstract. The development of mobile collaborative applications involves several challenges, and one of the most important is to deal with the always changing work context. This paper describes the relationship between these applications requirements and the typically context elements that are present in mobile collaborative work. This article presents a house of quality which illustrates this relationship and shows the trade-off involved in several design decisions. Keywords: Context Elements, Software Requirement, Mobile Collaboration, House of Quality.
1 Introduction One of the most challenging activities when developing software applications is to understand how user needs are satisfied by the application’s functionality. Groupware applications are not an exception, and although determining groupware users' needs in advance and creating the correspondent software may seem uncomplicated, the experience proves that it is not the case and that groupware users will refuse to use collaborative systems that do not support their needs in a wider context [Gru94; Luf00; Orl92; Suc83]. In addition, typical groupware applications have been criticized for being decontextualized, they do not consider the complex conditions (e.g., social, motivational, political or economic factors), where software will be executed causing users to reject the system [Gru94, Lju96, Bar97]. The problem is that most groupware developers usually focus almost exclusively on the analysis and specification of functional requirements neglecting the non-functional ones. Furthermore, users may be unaware of their needs in a wider context (e.g. social, physical, etc.), so that, by simply asking them to identify their requirements, software developers may not obtain the appropriate information [Ack00]. User’s needs are translated into functional requirements and some functions are implemented to satisfy such requirements. However, the appropriateness of such functionality is mostly evaluated later, when the software has been built. Misconceptions at that J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 67–76, 2009. © Springer-Verlag Berlin Heidelberg 2009
68
S. Ochoa, R. Alarcon, and L. Guerrero
point in the software life cycle are costly, which adds to the costs of groupware testing. Hence, new techniques are required in order to understand in advance the impact of the intended functionality on user’s needs in a wider context. One of such techniques is the Software Quality Function Deployment (QFD). QFD can be considered as a set of methods that allows contrasting the customer requirements with the product functionality and can be applied at each product development stage [Cha03, Haa96]. QFD aims to understand how users’ needs are intended to be satisfied; it is a consumer-driven process. Typically, QFD is applied in four stages: product planning, parts deployment, process and control planning and production planning. Each step produces a matrix that serves as an input to the subsequent steps. Due to its shape, the matrix is called a House of Quality (HOQ) [Kus99]. A HOQ relates customer requirements with technical descriptions of the product, including design activities. That is to say, the HOQ captures the essentially non-linear relationships between functions offered by the system and customers needs. Conflictive features become apparent and trade-offs are identified early in the design process. QFD has been used successfully for identifying customers and their needs [Mar98] considering user satisfaction as a measure of product quality. The use of formal tools that accompanies software development has been proven to be significant in various industries and some researches had accounted the usefulness of QFD for the design of groupware applications [Ant05, Gle04]. However, due to the complexity of groupware design, we believe that technical descriptions are not enough for analyzing whether user’s requirements are met or not. As stated by Grudin [Gru94] and Ackerman [Ack00], among others, groupware users needs go beyond technical functionality and involves various contexts of analysis, such as social, technological, and physical context. Based on a framework for contextual design derived by the authors [Ala06], the QFD technique and the authors’ experience, we present a correspondence matrix that shows the relationship between typical mobile groupware requirements and context elements inclusion. The analysis shows that trade-offs appear early during design, and some context have up to 9 relationships. Our aim is to provide groupware developers with formal software techniques that help them to reduce software costs while enriching its quality. We believe that such quality is strongly related with the contextualization degree of the application. Section 2 presents the context elements that typically are involved in mobile groupware applications. Section 3 describes the groupware requirements involved in the development of these tools. Section 4 presents the derived HOQ as well as an analysis of the relationships between user’s requirements and context elements. Finally, section 5 presents the conclusions and future work.
2 Context Elements The authors defined a set of context elements that are briefly explained below. Context elements should be considered during the development of collaborative mobile applications and are represented in the columns of the HOQ shown in figure 1.
Understanding the Relationship between Requirements and Context Elements
69
Readiness to use IT. This context element allows determining the group members’ preparation for using Information Technology tools. Users’ experience, readiness to use technology and learning will influence the kind of interaction dialogues, interfaces, protocol design options and even the project feasibility. Previous formal context (e.g., rules and regulations). This context element assists on characterizing users’ information needs, as well as the actions the group should perform in order to conform to such regulations. Previous informal context (e.g., social conventions and protocols). Unlike formal contexts, social conventions naturally emerge during everyday users’ interactions. They cannot be imposed and they constitute a frame for understanding each other behavior and purposes. Work practice tools. Every work practice community usually develops its own tools. These tools are not necessarily supported by technology. Provided they mediate social interactions, they can assist the analyst to understand the current underlying workflow. Group members interaction. This context element helps identify general interaction scenarios among group members in order to determine which of them require mobile support. Such interaction must consider users’ communication needs for data and/or voice transfer. Mobility Type: mobile coupled. This context element represents a type of mobility that can be present in a collaboration scenario. Group members performing mobile collaboration activities in a synchronous way are considered as carrying out mobile coupled activities. Mobility Type: mobile uncoupled. This context element represents to the asynchronous work carried out by mobile workers during a collaboration process. Communication requirements. Communication can be direct or mediated; public, private or a mixture; broadcast or multicast. This context element represents the communication requirements of a mobile collaboration activity. Communication strategies constrain the coordination strategies that can be applied. Coordination requirements. Coordination elements and policies are contextual element that needs to be identified. Some of these elements are: support for session management; floor control administration; user roles support; shared information handling. Activity criticality. It is important to determine the urgency of achieving the activity goals and the activity importance for the user. These criteria may influence the choice of communication and coordination strategies. Activity duration. Except for the case of mobile phones, activity duration in mobile collaboration based on PDAs, notebooks or Tablet PCs can be critical as it could be restricted by battery life. This context element identifies the activity duration and the requirement of power supply.
70
S. Ochoa, R. Alarcon, and L. Guerrero
Organizational structure (rigid/flexible). The organizational structure will influence the group needs for coordination and control policies. A rigid organization requires formal coordination with strict control, but flexible organizations must quickly react to environmental changes. This context element represents the type structure of the organization that host mobile workers. Collaboration policies/rules/norms/conventions. Every organization develops a series of social protocols, policies, rules and norms that regulate its workflow. It is important to identify which are the social rules that may be relevant for the intended collaborative application. Group size. Group size matters. Research in groupware has pointed out the importance of group size for the success of the coordination, communication and collaboration strategies. Most groupware design elements will be affected by the group size. Roles. An appropriate identification of roles will help developers to design useful applications. Otherwise, the collaborative mediation process could not be well supported. Clearly it may have a meaningful impact on the group performance. Group structure. The relationships among roles will define the group structure. An understanding of the group structure and the relationship between it and the organizational structure could be useful to design the interaction policies to support collaboration. Demographics. It is also important to take into account the users’ characteristics, e.g., their age, gender, race, and language may influence the application design. Usability of the application will probably be improved when considering this context element. Physical space. This element represents the available space for deploying and operating the collaborative mobile application. The smaller/less comfortable/less stable the physical available space is, the less likely is to use large or heavy computing devices. Adverse environmental conditions. This context element represents physical conditions such as noise, light, number of people around and distracting factors. These factors impose restrictions over the type of user interface to be used for interacting with the collaborative application. Safety and privacy. These are two important context elements to consider during the application design in case of mobile applications being used in public spaces. Handheld devices are especially appropriate for use in public spaces. User location (positioning). Traditionally in groupware, it refers to users’ location within the virtual environment and is known as location awareness. Current technology lets users locate the partners in the physical world. Power supply. The activity duration is in direct relation with this context element. The analysis of this element helps developers to identify if the power autonomy of the selected mobile device is enough to support each activity.
Understanding the Relationship between Requirements and Context Elements
71
Communication capability. This context element represents the availability of networking infrastructure in the work scenario. This element also includes the communication bandwidth that is possible to get in the physical scenario for supporting the mobile collaboration activity. Uptime effort. A mobile device may need short start-up time, e.g., when users have little time periods to carry out work or when quick reactions are required. This element represents the effort to leave available the mobile application. Transportability. It is important to identify those activities requiring mobility and to estimate the effort the users are able to spend while transporting the devices. Computing power. This element represents the processing and storage capacities required for a mobile computing device. Based on that, more than one device type can be selected to support activities with different requirements.
3 Computer Supported Mobile Collaboration Requirements Based on a literature review and the authors’ experience, this section describes general requirements of collaborative mobile solutions. These issues will be useful to help understand the type of applications and capabilities required to work in a specific scenario. Next, a brief explanation of each requirement is presented. Interoperability. The interoperability of a collaborative mobile application has two faces: communication capability and interaction services of the mobile units. Communication capability involves the threshold, protocols and infrastructure used to support the communication process among mobile units [Ald06, Kor01, Tsc03]. The structure and meaning of all information shared among the applications should be standardized in order to support any kind of interoperability. Multimedia support. If the application requires capturing, managing and/or transmitting heavyweight data types, such as image, video or audio, smaller the device size more limited will be the solution. The features of each device limit the quality and quantity of data that is able to capture, store and transmit. All road. Typically the nomadic work makes the work context changes periodically, therefore the groupware application has to be context-aware, and also it has to consider as much work scenarios as possible. Robustness. Nomadic work requires an important effort of the persons that use the computer-based applications. Several distracting events and interruptions are happening around them. Therefore, if the mobile groupware application is not robust and able to consider these distracting factors, then the users could not utilize the application to support the nomadic work. Autonomy. Typically, the nomadic workers carry out loosely-coupled work. It means they work autonomously and collaborate on-demand. Such autonomy must be provided by the software tool; therefore it must avoid using centralized resources.
72
S. Ochoa, R. Alarcon, and L. Guerrero
Usable or usefulness. The functionality provided by the tool, the design of the user’s interfaces, and mobile computing device utilized to perform a mobile collaborative activity, influence the usability of the solution in the work field. These three elements must be considered during the application design in order to improve the impact of the solution. Synchronous/asynchronous work. Mobile collaborative applications require synchronous/asynchronous communication capabilities depending on the type of activity to be supported (synchronous or asynchronous). If asynchronous communication is required, every mobile computing machine is able to provide such support based on minimal network availability. On the other hand, if synchronous communications is required, a permanent and stable communication service should be provided independently of the environment the user is located [Sar03]. Mobile phones supported by cellular networks are typically the best option for synchronous communication provided their large coverage range and good signal stability [Mal02]. However, these networks have a limited bandwidth. Another option is to provide synchronous communication capabilities to mobile applications using a Wi-Fi communication infrastructure [Rot01, Kor01]. Although the bandwidth is better than cellular networks, Wi-Fi signal stability depends on the physical environment where it is deployed [Ald06]. Furthermore, this type of networks has a limited coverage range [Mal02]. Portability (transportability). If the application requires to be used on the move, transportability is a strong requirement. Typically, the way to address this issue is through the mobile computing device chose to support the collaborative work. Smaller the device size the more transportable is the device. However, the device size reduction implies restrictions at least on the screen size and input capability [Kor01]. Privacy. If the privacy is an important requirement, mobile computing devices usually have small screens, and thus, they provide better privacy protection than notebooks and tablet PCs if data displayed on screen needs to be hidden from other people in public spaces. Furthermore, the physical distance between the user and the handheld device during the interactions is shorter than the distance between a user and his/her notebook or tablet PC. Another privacy consideration in mobile collaboration is the visibility of the users and users’ actions in MANETs or public networks [Kor01]. Ensuring accuracy of location information and users’ identities, and establishing private communication could be a critical issue in some cases [Che00]. Long time support (battery life). Activity duration in mobile collaboration provide a strong requirement on the type of device can be used to support it. Many researchers have identified the battery life as critical to support mobile collaboration [Kor01, Gue06]. However, the use of context-information provides a way to optimize the use of power supply resulting in a longer lasting battery life [Che00, Hak05]. On the other hand, it is always possible to carry extra batteries when PDAs, notebooks or Tablet PCs are used. Activity duration is not so critical in the case of mobile phones because these devices are able to work for many hours without being re-charged [Hak05].
Understanding the Relationship between Requirements and Context Elements
73
Capability to be deployed. Handheld devices are easy to deploy and carry, and also they require low user’s attention and have short start-up time. These features allow fast reaction from the users; such speed could be critically needed in these physical environments. Mobility. Users’ mobility on a physical environment depends on the features of the physical environment where the users are located and the current environmental conditions. A user equipped with a mobile computing device can be traveling, wandering and visiting [Kri00]. Traveling is defined as the process of going from one place to another in a vehicle. Wandering, in turn, refers to a form of extensive local mobility where an individual may spend considerable time walking around. Finally, visiting refers to stopping at some location and spending time there, before moving on to another location. Sarker and Wells report that “the optimal size of a device associated with wandering was necessarily lower than an acceptable device size when visiting or traveling” [Sar03]. Performance. The processing power needed for certain mobile applications can exceed what handheld devices can currently offer [Kor01, Gue06]. However, in case of PDAs, it is possible to find commercial devices with CPU speeds higher than 500 Mhz. The processing power limitation of these devices becomes visible, e.g., while processing multimedia information. Although every mobile computing device is able to address basic multimedia needs, just notebooks and tablet PCs are able to handle strong multimedia requirements, such as support for 3D games. Storage. Storage restrictions have been reported in the literature, especially related to handheld devices [Kor01]. However, these devices keep improving their storage and memory capacities. Last versions of these devices allow mobile applications to manage and store complex data types, even simple multimedia information. Data input. A possible requirement for a mobile collaborative application is the need for massive data entry. Typically, the mobile computing device used to support the solution will play a key role. PDAs and mobile phones use pen-based data input, which is slow, but also useful to support short annotations [Buy00, Sar03]. On the other hand, notebooks and tablet PCs are the most appropriate devices to support data intensive processes using the keyboard.
4 House of Quality The correspondence matrix, also called House of Quality (HOQ), has typically three parts (Fig. 1): customer requirements (leftmost rectangle), technical descriptions (upper rectangle), and relationships between customer requirements and technical descriptions (centered rectangle). In addition, the grey line shows the direction in which each relationship should be enhanced in order to improve the application’s capability to support the mobile work.
74
S. Ochoa, R. Alarcon, and L. Guerrero
Fig. 1. House of Quality
Analyzing the matrix it is possible to see that around a 30% of the intersections between rows and columns have some kind of relationship. It means each design decisions should be made carefully. The positive relationships must be increased and enhanced, and the negative ones should be minimized and neutralized. In addition, applications with high degree of interoperability among various software tools as well as coupled interaction pose the major challenges as they consume several resources (storage, bandwidth, battery) and may compromise application’s robustness, mobility, and performance. Authors expect this matrix helps developers to make fast and accurate decisions during the development process. When a design decision has to be made, the designer will evaluate their alternatives, against the HOQ, in order to determine which is the most appropriate. Therefore, the proposed tool not only systematizes and facilitates the decision making process, but also it makes it cheaper and expedite.
5 Conclusions and Future Work The article presents the typical users’ requirements and work contexts that are present in the development of mobile groupware applications. The paper also presents and analyzes the relationship among these two set of components.
Understanding the Relationship between Requirements and Context Elements
75
The analysis shows that trade-offs appear early during the application design. In addition, such analysis allows designers to easily identify the context variables that should be monitored in order to detect a work context change, or improve the users’ interaction paradigm. Our aim was to provide a tool (the HOQ) that allows mobile groupware developers to improve software quality, in term of usability and effectiveness, through the improvement of the decision making process at the design time. We believe that product’s quality is strongly related with the contextualization degree of the mobile application. As a next step, we are analyzing in detail three mobile groupware applications, in order to show how the HOQ can be used to support particular design decisions, and also to show the impact these decisions have in the products’ usefulness. If the authors assumptions becomes true, this proposal could have an important impact in the development of mobile groupware applications.
Acknowledgements This work was partially supported by Fondecyt (Chile), grant Nº 11060467, and LACCIR grants No. R0308LAC001 and No. R0308LAC005.
References 1. Ackerman, M.S.: The Intellectual Challenge of CSCW: The Gap Between Social Requirements and Technical Feasibility. Human Computer Interaction 15(2/3), 179–204 (2000) 2. Alarcón, R., Guerrero, L., Ochoa, S., Pino, J.: Analysis And Design of Mobile Collaborative Applications Using Contextual Elements. Computers and Informatics 25(6), 469–496 (2006) 3. Aldunate, R., Ochoa, S., Pena-Mora, F., Nussbaum, M.: Robust Mobile Ad-hoc Space for Collaboration to Support Disaster Relief Efforts Involving Critical Physical Infrastructure. ASCE Journal of Computing in Civil Engineering. American Society of Civil Engineers (ASCE) 20(1), 13–27 (2006) 4. Ramirez, J., Antunes, P., Respício, A.: Software Requirements Negotiation Using the Software Quality Function Deployment. In: Fukś, H., Lukosch, S., Salgado, A.C. (eds.) CRIWG 2005. LNCS, vol. 3706, pp. 308–324. Springer, Heidelberg (2005) 5. Bardram, J.: I Love the System -I just don’t use it! In: Proc. of International ACM SIGGROUP Conf. on Supporting Group Work, Phoenix, US, pp. 251–260 (1997) 6. Buyukkokten, O., Garcia-Molina, H., Paepcke, A.: Focused Web Searching with PDAs. Computer Networks. International Journal of Computer and Telecommunications Networking 33(1-6), 213–230 (2000) 7. Chan, L.K.V., Wu, M.L.V.: Quality Function Deployment: A Comprehensive Review of Its Concepts and Methods. Quality Engineering 15(1), 23–36 (2003) 8. Chen, G., Kotz, D.: A Survey of Context-aware Mobile Computing Research. Dept. of Computer Science, Dartmouth College, Tech. Rep. TR2000-381 (2000), ftp://ftp.cs.dartmouth.edu/TR/TR2000-381.ps.Z
76
S. Ochoa, R. Alarcon, and L. Guerrero
9. Glew, P., Vavoula, G.N., Baber, C., Sharples, M.: A ‘learning space’ Model to Examine the Suitability for Learning of Mobile Technologies. In: Attewell, J., Savill-Smith, C. (eds.) Learning with Mobile Devices: Research and Development, London, pp. 21–25. Learning and Skills Development Agency (2004) 10. Grudin, J.: Groupware and social dynamics: Eight challenges for developers. Communications of the ACM 37(1), 92–105 (1994) 11. Guerrero, L., Ochoa, S., Pino, J., Collazos, C.: Selecting Devices to Support Mobile Collaboration. Group Decision and Negotiation 15(3), 243–271 (2006) 12. Haag, S., Raja, M.K., Schkade, L.L.: Quality Function Deployment: Usage in Software Development. Communications of the ACM 39(1), 41–49 (1996) 13. Hakkila, J., Mantyjarvi, J.: Collaboration in Context-Aware Mobile Phone Applications. In: Proc. of HICSS 2005. IEEE Computer Society Press, Los Alamitos (2005) 14. Kortuem, G., Schneider, J., Preuitt, D., Thompson, T., Fickas, S., Segall, Z.: When peerto-peer comes face-to-face: collaborative peer-to-peer computing in mobile ad-hoc networks. In: Proc. of First Int. Conf. on Peer-to-Peer Computing, pp. 75–91 (2001) 15. Kristoffersen, S., Ljungberg, F.: Mobility: From stationary to mobile work. In: Braa, K., Sorensen, C., Dahlbom, B. (eds.), Planet Internet, Lund, Sweden, pp. 137–156 (2000) 16. Kusiak, A.: Engineering Design: Products, Processes, and Systems. Academic Press, San Diego (1999) 17. Ljungberg, J., Holm, P.: Speech acts on trial. Scandinavian Journal of Information Systems 8(1), 29–52 (1996) 18. Luff, P., Hindmarsh, J., Heath, C.: Workplace studies: Recovering work practice and informing system design. Cambridge University Press, Cambridge (2000) 19. Malladi, R., Agrawal, D.: Current and future applications of mobile and wireless networks. Communications of the ACM 45(10), 144–146 (2002) 20. Martin, M.V., Kmenta, S., Ishii, K.: QFD and the Designer: Lessons from 200+ Houses of Quality. In: Proc. of World Innovation and Strategy Conference (WISC 1998), Sydney, Australia (1998) 21. Orlikowski, W.: Learning from notes: Organizational issues in groupware implementation. In: Proceedings of the ACM Conference on Computer-Supported Cooperative Work, CSCW 1992, pp. 362–369. ACM Press, New York (1992) 22. Roth, J., Claus Unger, C.: Using Handheld Devices in Synchronous Collaborative Scenarios. Personal and Ubiquitous Computing 5(4), 243–252 (2001) 23. Sarker, S., Wells, J.: Understanding Mobile Handheld Device Use and Adoption. Communications of the ACM 46(12), 35–40 (2003) 24. Suchman, L.A.: Office Procedures as Practical Action: Models of Work and System Design. ACM Transactions on Office Information Systems 1(4), 320–328 (1983) 25. Tschudin, C., Lundgren, H., Nordström, E.: Embedding MANETs in the Real World. In: Conti, M., Giordano, S., Gregori, E., Olariu, S. (eds.) PWC 2003. LNCS, vol. 2775, pp. 578–589. Springer, Heidelberg (2003)
Continuous User Interfaces for Seamless Task Migration Pardha S. Pyla, Manas Tungare, Jerome Holman, and Manuel A. P´erez-Qui˜nones Department of Computer Science Virginia Tech Blacksburg, VA, 24060, USA
[email protected],
[email protected],
[email protected],
[email protected]
Abstract. In this paper, we propose the Task Migration framework that provides a vocabulary and constructs to decompose a task into its components, and to examine issues that arise when it is performed using multiple devices. In a world of mobile devices and multiple computing devices, users are often forced to interrupt their tasks, move their data and information back and forth among the various devices manually, recreate the interaction context, and then resume the task on another device. We refer to this break from the task at hand as a task disconnect. Our objective is to study how software can bridge this task disconnect, enabling users to seamlessly transition a task across devices using continuous user interface. The framework is intended to help designers of interactive systems understand where breaks in task continuity may occur, and to proactively incorporate features and capabilities to mitigate their impact or avoid such Task Disconnects altogether.
1 Introduction and Motivation Today, with the advent of mobile devices and the deployment of computing in various form factors, the paradigm of a single user interacting with a single computer on a desk is losing its dominance. Even though the massive storage and computational power of a desktop computer has helped it to continue to be a central part of our daily work, most people interact with more than one device for their everyday tasks. From a recent survey of knowledge workers in a huge software development company and a major university [15], almost all participants reported that they use at least two computational devices for their day-to-day activities. The desktop computer and the notebook computer are still two primary devices that people use to accomplish their daily work. This proliferation of multiple computing devices is burdening the user with overheads for transferring information among different devices. Often, users are forced to copy files back and forth, open and close applications, and repeatedly recreate task context as they move from one device to another. In this paper we provide a theoretical foundation to describe the extraneous actions users need to perform as they switch from one device to another and propose the idea of Continuous User Interfaces (CUIs) to facilitate seamless task migration across multiple devices. We describe the development and evaluation of a proof-of-concept continuous user interface and observations from a preliminary usability study. J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 77–85, 2009. c Springer-Verlag Berlin Heidelberg 2009
78
P.S. Pyla et al.
1.1 Tasks, Activities, Units and Cost Before formally defining a task disconnect, we first describe the vocabulary and terminology for tasks and the various associated parameters. Tasks
User Action
User Action
User Action
Procedure: Execution Sequence of Units
cost = x
cost = z
cost = y
Unit 1
Unit 2
Unit 3
Instructions
Instructions
Instructions
Fig. 1. Tasks, User actions, Units and Instructions
A task can be defined as a goal to be attained in given conditions [11]. Leplat expresses these conditions using three points of view: the states to be covered, the permitted operations, and the procedure [12]. At a lower level, we define tasks to be user actions. A user action is what the subject puts into operation (cognitive operations, behavior) in order to meet task demands. We also make use of Leplat’s definition of elementary units to be the elementary tasks, and elementary states or operations. Leplat uses these definitions to describe task complexity. However, we use the term units to further subdivide user actions to their lowest granularity. For non-trivial tasks (i.e., tasks that involve multiple activities), we define a procedure to be an operation execution sequence of multiple units. We also associate with each unit a parameter required for the successful execution: an instruction. Instructions are knowledge directions necessary to execute units, that can exist in the user’s understanding of the world or it can exist in the aids and artifacts in the task environment.The cost of a unit is a multidimensional attribute set that is incurred during the execution of a unit [12]. These dimensions could be cognitive, physical, memory-intensive, resource-intensive or a combination depending on the nature of the unit and the expertise of the user. Another important parameter of a task is time. In the words of Leplat, every task takes place in time and may be described by the temporal dimensions of its organization [12]. Out of the few temporal dimensions that Leplat describes, temporal ruptures is of particular importance to our work. We adapt and modify Leplat’s definition of temporal ruptures to mean interruptions by activities that do not directly contribute to the successful execution of the task at hand.
Continuous User Interfaces for Seamless Task Migration
79
1.2 Task Disconnects With the increasing proliferation of mobile and ubiquitous computing devices, people are forced to adapt their daily workflows to try and keep their work products and other data available to themselves at all times. E.g., Jones et al. [10] found that people often emailed to themselves the URLs of the websites they visited or their favorites list when they needed to access it from a different location. In scenarios such as this, where a user attempts to work with two devices, the need to transfer task and information back and forth between them is a burden. Users are forced to use workarounds like USB key drives, remote desktop software, e-mail, network file storage, and other means. These attempts to orchestrate a migration of data back and forth between the two devices create a gap in task execution. When a task is to be migrated from one device to another, the process involves more than just breaking it into two chunks along a selected boundary. It also involves the insertion of extra units into the task procedure that are entirely absent when the same task is executed on a single device. It is the inclusion of these extra units that hinders the seamlessness of task migration. Depending upon the exact nature of the task and the two devices in question, this process may involve simply adding more instructions to the procedure (low additional cost), or may involve an entirely new set of user actions (high additional cost). A task disconnect is a temporal task rupture arising due to extraneous user actions required when performing a task using multiple devices. Such extraneous user actions are required to accomplish the task across multiple devices, but do not directly aid in the completion of the task at hand. This raises issues such as how to help the user make a switch from one task condition to another in such a way that the need for a user’s attentional resources, cognitive workload, reaction time, and memory demands are minimized.
2 Related Work Bellotti and Bly [2] observed information workers to be mobile within the confines of their office; this local mobility existed mainly to enable the use of shared resources and for communication with other staff members. While mobile, users were seen to use secondary computational devices in addition to traditional devices such as desktops. A few strands of research have tried to address the problem of migrating tasks or applications over multiple devices. However, most of these studies have focused primarily on the technological aspects of this problem: Chu et al. [6] take the approach of migrating an entire application to support seamless task roaming, but with considerable latency (thus interrupting the user’s tasks sequence.) They do not discuss the implications on the user’s tasks and goals. Bandelloni et al. [1] describe user interaction with an application while moving from one device to another, in three levels of migration: total, partial and mixed. Chhatpar and P´erez-Qui˜nones [5] call this migration dialogue mobility and propose a requirement for the application data and logic to be separate from the user interface. Neither one of these projects take the task perspective we propose in this paper. Florins et al. [9] describe rules and transformations that attempt to provide graceful degradation of user interfaces while an application is migrated from one device to
80
P.S. Pyla et al.
another; even though their work is based on the same principle of continuity, their focus is on user interface generation and not on task migration. ARIS [4] is a window management framework that relocates running applications from one display to another; TERESA [14] helps design and develop model-based nomadic applications. Toolkits and tools such as TERESA have utility in rapidly deploying applications that can be migrated over multiple devices, but do not address the task semantics that users wrestle with while trying to interact with a multi-device interface. Denis and Karsenty [7] discuss a conceptual framework for the inter-usability of multiple devices. They provide an analysis of different cognitive processes in inter-device transitions and postulate two dimensions required for seamless interaction: knowledge continuity and task continuity. We base our work and the definition of Continuous User Interfaces on this requirement of seamlessness. We take this task-centered approach to solving the problem and we provide a definition, description, parameters, requirements, and a prototype to demonstrate a seamless interaction over multiple devices without task disconnects. Interruptions are events that break the user’s attention on a particular task to cater to another task that is in need of attention, and are the focus of a whole area of study by themselves [13]. Task disconnects can also be envisioned as analogous to interruptions, but occurring over multiple devices, locations, contexts, and most importantly, over a much longer time interval. Interruptions happen often in normal conversations [3,8]. However, in the field of linguistics [8,3], not all interruptions are disruptive or need repair. Even in cases where the interruptions are disruptive, the costs associated with repair are low, because humans have an inherent ability to repair, recover and proceed with most of the conversations using their ingrained social and cultural aids.
3 Study Design We targeted a specific application domain with sufficient complexity to allow us to observe clearly the different parameters responsible for task disconnects. Software development is a domain that involves work on several tasks of different affinities for devices. Software engineers perform requirements gathering and early design at client locations, away from their own offices, with laptop computers. They bring the artifacts of this stage to their office and continue to create a complete design using their desktop computers. The software that is designed must finally run on clients’ machines. We chose this application domain because of the need to use several tools such as text editors, drawing packages, scheduling programs, etc. when accomplishing a task, and because the nature of the task requires the use of multiple devices. We built a prototype to support the preliminary design phase of software engineering where developers collect client requirements and generate initial design prototypes, diagrams, and models. 3.1 Prototype We built a prototype that incorporated knowledge continuity and task continuity to provide a seamless multi-device user experience. The Task Explorer (Figure 2(a)) allowed users to create and demarcate a task, track the activities they performed in a to-do list tool (included), and provided constant visual feedback on the status of the connected devices in range. Opening a task in the Task Explorer launched the Task Viewer
Continuous User Interfaces for Seamless Task Migration
(a) Task Explorer
81
(b) Task Viewer
Fig. 2. Task Explorer and Viewer
(Figure 2(b)), a place to view documents and files related to a task. In our prototype application domain, these are requirements documents, diagrams and prototypes, e-mail addresses, and people related to the project in a unified view. Each task is uniquely color-coded to establish a visual identity with the task. Opening a document such as a requirements specification launched that file in an external editor. This was implemented as an application on a tablet interface. The interface leveraged spatial organization, shape, color, partitioning of data and function, recovery of state of data and recovery of activity context on its user interface. Tasks were migrated from the desktop computer to the tablet interface either automatically (because the task was open) or manually (by dragging and dropping). For each task, we displayed the task parameters in the same color gradient on the tablet and the desktop. The last drawing that was being accessed on the desktop computer was automatically loaded to maintain task continuity and activity context. If the drawing was cleared and replaced by another, the new diagram was synchronized automatically with the desktop. This obviated the need to open and save documents, making the interface more like paper. As artifacts were being generated, they were populated into the task tree on the right side of the screen. The task tree on the tablet brought together artifacts related to the task on the desktop computer, e.g. requirements documents, people, to-do list, and email messages. 3.2 Evaluation Interviews and user surveys were conducted to gather insights into the example task of prototyping and the existence of disconnects when using multiple devices to prototype. Six professional software developers were asked open-ended questions targeting the technologies and devices they used to prototype and any insights into disconnects arising due to the mediation by these technologies. In addition, we received N=32 responses to a survey that targeted software developers, graduate students with software development experience, and researchers in HCI who were familiar with computing and did prototyping tasks.
82
P.S. Pyla et al.
The prototype was evaluated with a group of graduate students with a software engineering background. A total of six participants participated in the evaluation. Three of the six participants constituted a control group where they were given tasks that required switching between a tablet and a desktop computer. The other three participants comprised our test group and were asked to perform the same tasks using our prototype. Each participant was assigned a total of seven tasks. Each task required drawing simple low-fidelity user interface prototypes using our custom drawing tool, updating requirements specifications using a text editor, or a combination of these two tasks. Participants were provided a background scenario explaining the context of a software development project for a fictitious client and the need to transfer documents between the tablet and the desktop. The participants were asked to use a tablet when meeting the client. Their interaction with the client was scripted in the scenario provided. The participants were asked to think aloud while they worked and the evaluator prompted the users when they stopped talking during a task. 3.3 Tasks The first task required the participant to make changes to an existing requirements document based on a fictitious client’s new insights into the project at the client’s location (i.e. using a tablet). The second task required the participant to prepare a low-fidelity prototype for the new requirements specification on the desktop. The third task asked the participant to visit the client to demo the prototype that was created on the desktop at the participant’s office. The fourth task required the participant to work on the desktop and to add more description to some requirements based on the client’s feedback. The participants were asked to assume they were at home for the fifth task (i.e. they were to use a tablet.) When they thought of a design feature, they were to create a new prototype with that insight to demo to the client the next day. The sixth task asked the participant to visit the client and demo the new prototype and get feedback. Based on the feedback, they were required to change the prototype and the requirement specification. The last task was set at the participant’s office where they were asked to update their desktop files with the latest prototype and requirements specifications. These tasks were designed with the goal of making the participants transfer information between the two devices as they progressed through the tasks. In the test group, this transfer was automatic because the participants used our prototype. In the control group, the participants had to move the files themselves using their choice of a USB pen drive, email, or other server-based technologies. The control group participants were provided the tablet and the desktop that were both connected to the Internet. They were given one task at a time with the associated scenario to provide the context of the interaction. At the end of the session, all participants were asked to fill out a questionnaire.
4 Results and Discussion In this study, we found several aspects of multi-device interaction that current systems do not adequately support. Specifically, several users reported dropping the use of multiple devices in favor of a single computer, to avoid the costs of task migration. They
Continuous User Interfaces for Seamless Task Migration
83
also reported that migrating files across devices taxes their short-term memory, is often frustrating, and likely to result in errors. We examine each of these in turn, based on our observations and responses from our study participants. 4.1 Use of a Single Computer Another interesting observation that one participant made was: “this [migrating data] almost makes me use the tablet alone for all the tasks and forget about my desktop if I had the choice”. When asked if she would do that even if the task at hand required more processing power (such as that available in a desktop), she responded affirmatively. Several survey respondents in another study [15] also confirmed that they chose to use only a single computer for convenience rather than have to copy/move files among several computers. This illustrates that the high costs of user actions associated with a task switch from one device to another prompt users to forgo computational power in favor of eliminating the switch entirely. 4.2 Consistency (or the Lack Thereof) of File Locations One common complaint from participants was that they needed to remember file locations and the state of a file on each device. As one participant put it, “this version control is getting irritating”. Remembering such extraneous information increases the short-term memory costs of this activity tremendously. Given that short-term memory is unreliable, it is difficult for the user to remember which device had the latest version of the data if temporal ruptures in a task take place over a longer period of time. This is another observation that directly supports our hypothesis that transferring activity context is important. The experimental group, who performed the same tasks using our prototype, were instantly able to locate documents and the information they needed. When switching from one device to the other, they reported being able to restart their task immediately and to be productive because the environment was already automatically loaded with the appropriate task context. Because information was re-displayed using less screen real estate, users were immediately able to focus on their work while keeping related information in their peripheral vision. The only limitation of the system was that users spent time moving and resizing the requirements window to enable them to easily see both and work between them. The act of copying files manually involves two steps: copying the data over, and placing it in the correct location on disk. Most current data copying tools and media (e.g. USB drives, email to self, etc.) assist in performing the first task, but not in the second. Thus, to perform the second step, users are forced to rely on their short-term memory, increasing cognitive workload and scope for error. Automatic system support for the second step therefore was viewed as a distinct advantage of our prototype. The related issues of version control and conflict management were also automatically handled.
84
P.S. Pyla et al.
4.3 Fear, Frustration In the questionnaire, all three control group participants reported a fear of making errors due to the overheads associated with the migration of information across devices. They also reported difficulty in keeping track of document version information. One participant commented that if a scenario such as the one in the evaluation were to occur in real life, the costs would be higher because of longer temporal ruptures. One of the participants forgot to copy the files from the desktop to the tablet before visiting the client. When she realized this, she remarked, “Wow! In real life, this would mean I’d have to go back to my office to get my updated files or redo the prototype that I did in the last task.”. On the questionnaire, members of the experimental group reported that they were less likely to make errors accomplishing the tasks. Also, because file state and application state were transferred automatically, the experimental group only had to worry about finding the appropriate location in the UI to begin work again. There were comments by some users that it would be nice to have a better view of all the files related to a project, but creating a new file system view was not the purpose of our prototype. Overall, participants of the experimental group responded that the application was more satisfying and easier to use (per Likert scale ratings.) This means that as the task procedure lengthens (in light of the extraneous actions required for task switching), and associated increase in costs, there is a corresponding rise in the likelihood of user error. In continuous user interfaces, such costs are reduced because of in-built system support for task migration. 4.4 Use of Mobile Computers as Secondary Displays For the second task, where the participants were required to create prototypes based on the requirements specification document, all the three participants in the control group preferred using the tablet as an information display. They opened the specification document on the tablet and referred to it as they sketched the prototype on the desktop. When asked about this, they said that having the information on a secondary display was good as it did not make them switch between different windows on one device. This might mean that CUIs should leverage the capabilities of the various devices even when they are co-located.
5 Discussion and Summary We explored the issues that arise when users use multiple devices to execute a single task. We found that current systems lend inadequate support for several user actions that must be performed during a task migration between/among devices. Among the problems reported were: dropping the use of multiple devices in favor of a single computer; increased short-term memory costs while migrating files across devices; higher frustration; and a higher likelihood of errors. We proposed and designed a prototype Continuous User Interface that ensured a seamless task migration for users attempting to perform a requirements specification
Continuous User Interfaces for Seamless Task Migration
85
and gathering task, using a tablet computer and a desktop computer. This system provided support for automatic migration of task context (e.g. the applications that were in use; pages and objects such as diagrams that were selected and active; etc.) between the two devices. In an evaluation conducted, participants reported that it helped mitigate the disruptive effects of task disconnects to a high degree. An interesting observation was that users expected to be able to annex existing collocated devices when performing their tasks (i.e., using the tablet computer at their desk along with their primary desktop computer.) They also reported that the automatic availability of necessary data on mobile computers directly contributed to higher perceived reliability and lower likelihood of error.
References 1. Bandelloni, R., Patern`o, F.: Flexible Interface Migration. In: IUI 2004: Proceedings of the 9th International Conference on Intelligent User Interface, pp. 148–155. ACM Press, New York (2004) 2. Bellotti, V., Bly, S.: Walking away from the desktop computer: distributed collaboration and mobility in a product design team. In: CSCW 1996: Proceedings of the 1996 ACM Conference on Computer Supported Cooperative Work, pp. 209–218. ACM Press, New York (1996) 3. Bennett, A.: Interruptions and the interpretation of conversation. Discourse Processes 4(2), 171–188 (1981) 4. Biehl, J., Bailey, B.: Aris: An interface for application relocation in an interactive space. In: Proc. 2004 Conference on Graphics Interface, pp. 107–116 (2004) 5. Chhatpar, C., P´erez-Qui˜nones, M.: Dialogue mobility across devices. In: ACM Southeast Conference (ACMSE), Savannah, Georgia (2003) 6. Chu, H.-h., Song, H., Wong, C., Kurakake, S., Katagiri, M.: Roam, a seamless application framework. Journal of Systems and Software 69(3), 209–226 (2004) 7. Denis, C., Karsenty, L.: Inter-usability of multi-device systems - a conceptual framework. In: Seffah, A., Javahery, H. (eds.) Multiple User Interfaces: Cross-Platform Applications and Context-Aware Interfaces, pp. 373–384. John Wiley and Sons, Chichester (2004) 8. Drummond, K.: A backward glance at interruptions. Western Journal of Speech Communication 53(2), 150–166 (1989) 9. Florins, M., Vanderdonckt, J.: Graceful degradation of user interfaces as a design method for multiplatform systems. In: IUI 2004: Proceedings of the 9th International Conference on Intelligent User Interface, pp. 140–147. ACM Press, New York (2004) 10. Jones, W., Bruce, H., Dumais, S.: Keeping found things found on the web. In: CIKM 2001: Proceedings of the tenth international Conference on Information and Knowledge Management, pp. 119–126. ACM Press, New York (2001) 11. Leontiev, A.: Le Developpement du Psychisme. Editions Sociales, Paris, France (1972) 12. Leplat, J.: Task complexity in work situations. In: Tasks, Errors and Mental Models, pp. 105–115. Taylor & Francis, Inc., Philadelphia (1988) 13. McFarlane, D.C.: Interruption of people in human-computer interaction. Doctoral dissertation, The George Washington University (1998) 14. Mori, G., Patern`o, F., Santoro, C.: Tool support for designing nomadic applications. In: IUI 2003: Proceedings of the 8th international Conference on Intelligent User Interfaces, pp. 141–148. ACM Press, New York (2003) 15. Tungare, M., P´erez-Qui˜nones, M.: It’s not what you have, but how you use it: Compromises in mobile device use. Technical report, Computing Research Repository, CoRR (2008)
A Study of Information Retrieval of En Route Display of Fire Information on PDA Weina Qu1, Xianghong Sun1, Thomas Plocher2, and Li Wang1 1
State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China {quwn,sunxh,wangli}@psych.ac.cn 2 Honeywell ACS Labs, Minneapolis MN 55418, USA
[email protected]
Abstract. This study was concentrated on which way is the most convenient for firefighter to get information, comparing among audio display, text display, and combined multi-modal display. Can fire commanders effectively obtain key fire information while they are en route to the fire, especially when they sitting in a moving and bumpy car? The task includes free-browse, free-recall and searching information. The result showed that: (1) Audio only always made firefighter taking the longest time to browse and search, but the introduction of audio display made the two combined displays more quickly to access information, and more easy to remember. (2) Searching in a moving environment took a little longer than searching in lab. (3) Comparing in the lab and in moving car, it was found that searching in a moving environment took a little longer than in lab. (4) It was proved that text display was still a necessary and indispensable way to display information. Keywords: Information retrieval, Display, PDA, Free-browse, Free-recall, Search.
1 Introduction The rapid growth of the IT industry during the last few decades has increased demands on mobile devices such as PDAs, cellular phones, and GPS navigation systems. With emerging concepts of context-aware computing, the mobile devices can provide mobile users with timely information by using not only common knowledge but also environmental context such as current time and location [1]. PDA has applied many systems. For example, alerts in healthcare applications [2], navigation system [3]. Auditory system is another important sensory system for getting information, which is the major complement to visual system. Moreover, human responds to auditory stimulus is faster than visual stimulus [4]. For Firefighting, time means life saving. En route display system is a kind of handheld device with mobile communication, which aimed to help fire commander to access the most current fire information as quickly as possible. Purpose of the experiment is to answer the following questions: can fire commanders effectively obtain J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 86–94, 2009. © Springer-Verlag Berlin Heidelberg 2009
A Study of Information Retrieval of En Route Display of Fire Information on PDA
87
key fire information while they are en route to the fire, especially when they sitting in a moving and bumpy car? Comparing among audio display, text display, and combined multi-modal display, which way is the most convenient for firefighter to get information?
2 Method 2.1 Experimental Environment Test equipment. An en route display prototype (installed in a handheld PDA device) has been developed as the experimental platform, which could show the current fire related information to fire commanders, such as the address of the building got a fire alarm, the location of the first alarm in the building, and so on. The size of words is based on previous experiment[5]. Based on the result from card sorting experiment, all the fire information has been clustered into three-level menu structure. The first level and the second level are total information (see Fig. 1). The third level is specific information.
Fig. 1. Content of the first and the second level menu
Fire scenarios. Totally 16 fire scenarios drawn from the previous 3D fire information display prototype[6] were pre-installed in the prototype, supposed to happen in 3 buildings: Camden building, a 45-floor simple structured tower building, and an 8-floor
88
W. Qu et al.
complex structured hospital building with unregulated shape. 2 of them were Camden building fires (single fire). 7 of them were tower building fires (including 2 SS fires (Single fire seed spread on Single floor), 3 SM fires (Single fire seed spread on Multiple floors)), and 2 MM fires (Multi fire seeds spread on Multiple floors). The other 7 of them were hospital fires (including 2 SS fires, 2 MM fires, and 3 SM fires) were prepared as the scenario pool. Experimental places. The experimental tasks were completed both in a lab (indoor test part) and in a moving car (outdoor test part). A set of standard usability test room was used as the indoor test place. A Volkswagen sagitar car was used as test car. Driving speed is about 45 kilos/hour. In the test room, firefighters were asked to fulfill two kinds of test tasks (free recall, information searching) by using en route display prototype. In the moving car, one firefighter sitting in co-driver’s seat was asked to do a dual-task: the main task is a counting how many street lamps passing by on right side along the road and speak out the number loudly; the secondary task is to finish information searching by using en route display prototype. 2.2 Participants 12 firefighters participated in our experiment. 11 of them aged from 21 to 29, and 1 subject is 35 years old. 6 of them have bachelor degree. 9 of them have more than 5-year firefighting experience. All participants had accomplished all the experimental tasks.
3 Procedure Each test was conducted by one firefighter and two experimenters. During the test, one experimenter worked as a moderator, the other one was in charge of video recording and note taking. The whole test consisted of three parts: training, test, and interview. Training experimenter took a couple of minutes to explain 5 key’s function on the PDA device so that subjects can know clearly how to use them (up: previous item, down: next item, left: previous level menu, right: next level menu and middle key: updated information) to browse the system and get information. Test. This part was divided into two stages: test in lab, test in moving car. Test in lab 1. Task 1 free browse and free recall: firefighter was asked to explore en route system for a few minutes (3-5 minutes) then stopped when he thought he was already familiar with the prototype, and was asked to recall whatever he can remember in his mind. Each firefighter could only try one of the four display ways to explore the en route system. 2. Task 2 Information searching: moderator show four items to firefighter one by one, and ask him using the en route prototype to find the items as quickly as possible. In order to assure firefighter had not only found the location of the item but also remembered the content, after finding the item, he was asked to repeat the content about the item without seeing the information.
A Study of Information Retrieval of En Route Display of Fire Information on PDA
89
Test in moving car 3. Task 2 Information searching: same as the test in lab. Interview. After the test part, firefighters were asked to evaluate the menu structure, to tell their preference to the four ways of fire information display. Then experimenter asked firefighter several questions. 3.1 Data Analysis of Free-Browse and Free-Recall Task Free-recall task was to test what firefighters can really get from the en-route display system and what they have memorized. Experimental design. One way between subjects design was used in this task. There were four ways of information display on en route system, which were audio only, text only, combined text + audio and text + the third level auditory display. 12 subjects were randomly divided into four groups. Each group has 3 firefighters. Each group tried only one kind of four information display ways. They were asked to browse and operate the PDA device freely till they felt familiar with the system and knew the fire related information. Then, they were asked to recall key information from they had browsed. Browsing time, the item number they browsed, and the items they recalled were recorded. All subjects used the same scenario (Camden building fire). Experimental result. Table 1 showed the numbers of browsed items and average time of free browsing. The first level menu had 6 items, so there are total 6 scores. If subject browsed only 1 item, he would get 1 score. The second level and third level menu had 24 items respectively. It’s found that “text only” and “combined text + audio” display are better than other two kinds of display. Subjects spent least time on “combined” display style. Table2 showed free-recall item numbers under the four different kinds of display conditions. If subject could recall one item correctly, he would get 1 score. It’s found that subjects could get more scores using “test only” and “combined text + audio” display than the other two display styles. Table 1. Free-browse item number and average time
audio only
1st level menu (6 scores including updated inf.) 5.3
2nd level menu (24 scores)
3rd level menu (24 scores)
average time for exploring
12.3
6
0:05:35
text only
5.3
17
13.3
0:04:40
combined text + audio
5.3
16
13.3
0:03:17
text + the 3rd level audio
5.3
12
10
0:03:41
5.3
14.3
10.67
0:04:18
mean
90
W. Qu et al. Table 2. Free-recall item numbers
audio only text only combined text + audio text + the 3rd level audio
1st level menu (6 scores including updated inf.) 0.7 1
2nd level menu (24 scores)
3rd level menu (24 scores)
total number
2.7 2.3
2 5
5.4 8.3
3
4.7
1
8.7
1
2.3
3
6.3
From the data on table 1 and table 2, we can say firefighters have browsed all the items at the first level, most items at the second level, and about half of the items at the third level. The more items they browsed, the more items they can correctly recalled. But the amount of their memory was still around the limitation of short memory: 7+/-2. 3.2 Data Analysis of Searching Task Experimental design. Searching task was to test the efficiency of the en route system for firefighter’s operations. 2*4 between subjects design was used in this task. Two factors were the places where the en route system was used, and the ways of fire information display. Lab and moving car were used as the two experimental places. Four different display ways (same as mentioned above) were tested to find which way is easier and more convenient to search information. 12 subjects were randomly divided into four groups. Each group has 3 firefighters. Each group used the same way of display as in the browse and recall task to search information. In the lab situation, each subject was asked to search four items: facility manager, hose, security passage and name and address of the building got a fire alarm. All subjects used the same scenario (hospital fire). In moving car situation, each subject searched four different items: Power Company, control room, road information and building structure. All subjects used the same scenario (Camden fire). In this task, subjects were asked to find the item as quickly as possible and repeat detailed content without watching back the PDA screen. The searching time and the percentage of correctly repeated content were recorded. Experimental result. Table 3 and table 4 showed percentage of correct repetition and searching time in the lab and in moving car. Searching time didn’t include subjects’ repetition time. Percentage of correct answers means accuracy of subject repeating detailed content. About the percentage of correct answer, 0 meant that subjects could not repeat the detailed content that they found; 1 meant that subjects could repeat part of detailed content; 2 meant that subjects could repeat all detailed content. It’s found that audio display took the longest time to search, and “text + 3rd level auditory display” took the shortest wherever in the lab and in moving car. The results
A Study of Information Retrieval of En Route Display of Fire Information on PDA
91
Table 3. Percentage of correct repetition and searching time in the lab Percentage of correct answers 0 1 2 Searching time
combined text + audio
Text + 3rd level audio
audio only
text only
16.70% 33.30% 50%
0 25% 75%
0 16.70% 83.30%
0 0 100%
0:00:55
0:00:17
0:00:25
0:00:13
Table 4. Percentage of correct repetition and searching time in moving car Percentage of correct answers 0 1 2 Searching time
combined text + audio
Text + 3rd level audio
audio only
text only
33.30% 33.30% 33.30%
8.30% 8.30% 83.30%
8.30% 0.00% 91.70%
8.30% 16.70% 75%
0:01:03
0:00:31
0:00:31
0:00:23
Fig. 2. Comparison of searching time between lab situation and car situation
showed that combined text + audio display were better repeated by firefighters than the other two ways. Among the four displays, there was no significant difference existed. Fig. 2 showed the comparison of searching time between the lab situation and the moving car situation. It’s found there was the same trend among the four ways of
92
W. Qu et al.
information display, and searching in moving car took about 8 seconds longer than searching in the lab.
4 Interview The short interview was trying to answer the following questions: 1. What do you think the menu structure? At each level of the menu, which item do you think the most important? Could you sort the items at the same level by their significance? Is there any suggestion to improve the current menu structure? 2. Which way of showing fire information do you prefer to choose when you are using the en route prototype on the way to the fire scene? Why? 3. Do you think the information shown in the system is easy to remember? How much information can you remember each time? 4. Do you think the fire information is easy to find? 5. For the button of updated information, what do you think it, useful, or not? 6. How do you think the en route information display, is it useful, or not? Why? 7. Is there any other suggestion to the prototype? 4.1 Subjective Ratings to the Four Display Styles From Table 5, we can say most of users prefer combined visual and audio display. (answer to the question 2) Table 5. Preference to the four display styles
audio only in the lab in moving car
combined text + audio
text only
0
16.70%
66.70%
8.33%
33.30%
41.70%
Text + the third level auditory display 16.70% 16.70%
4.2 Subjective Ratings to Menu Sequence 1 means the most important, 5 means the least important. Some firefighters said the most important item should be put on the first line, but at the first level menu the alarms was put on the bottom line although it was ranked the most important by everyone (see Table 6). Table 6. Preference to the first level menus sequence
mean
1 Facility phone contact 3.5
2 Firefighting equipment 3.2
3 Site information 3.8
4 General building information 2.6
5 Alarms 1.9
A Study of Information Retrieval of En Route Display of Fire Information on PDA
93
For the sorting of 2nd level and 3rd level menu items, it was found that at each category firefighters’ rating was consistent with the current sorting in Table 7. Combining the data in Table 6 and 7, to answer the first question about the menu structure, we think the current one is good enough except two things: 1) alarms should be put on the first line at the first level. 2) Security passage should be moved from “4 General building information” to “3 site information”. Table 7. Preference to the second level menus sequence
mean
mean
mean
mean
mean
1.3 1.2 hazard facility manager coordinator 2.6 3.2 2.2 2.1 fire equipment 2.3 hose equipment shutoff 1.9 3.3 3.4 3.1 secu3.2 road 3.3 road rity passage limit information 1.4 2.3 2.3 4.3inf.of 4.2gener surround4.1 ocal inf. of ing buildcupants building ings 2.8 2.4 4.6 5.1 1st 5.2 5.3 alarm alarm list hazard 1.4 2.2 2.4 1.1 building owner 1.8
1.4 power company 3.9 2.4 outdoor standpipe 3.4
4.4 building structures 3.5
1.6 water department 4.3 5.3 2.6 2.5contr power ol room room 3.9 5
1.5 gas company
4.5 keybox location 4.5
4.6name and address 2.4
5 Conclusion In order to answer how useful the en route information display system for firefighters’ information accessing, current situation understanding, and decision making, we did a series of tests to investigate the efficiency of the system, to compare different display ways including audio, text, and their combinations to find the most appropriate one. Based on the data, and the subjective ratings, we can summarize our findings as follows: 1. En route information display system was useful to help firefighter get the critical fire information and make decision more quickly and accurately. 2. Comparing the four information display ways: audio only, text only, audio + text, and text + the 3rd level audio, audio only always made firefighter taking the longest time to browse, search, but the introduction of audio display made the two combined displays (text + audio, and text + 3rd level audio) more quickly to access information, and more easy to remember.
94
W. Qu et al.
The reasons making audio display the worst could be: a. the voice message was not clear enough for firefighter to hear, especially in moving car; b. 3-level information structure is difficult for people to understand just by listening. But the data also showed that, after training, if firefighter has already got the information structure in mind, the convenience of operating with audio way will be the same as operating the system with other way. 1. Comparing the two situations of using the en route system: in lab, and in moving car, it was found that searching in a moving environment took a little longer than searching in lab. 2. It was proved that text display was still a necessary and indispensable way to show information. The reason was that, even there was a voice message playing, people still need to look at the text to make sure what they heard and understand was correct, especially for the building address, alarm location, and the contactors’ name, and so on. Because those information is really critical for firefighting and life saving.
References 1. Kim, N., Lee, H.S., Oh, K.J., Choi, J.Y.: Context-aware mobile service for routing the fastest subway path. Expert Systems with Applications 36, 3319–3326 (2009) 2. Chiu Dickson, K.W., Kwok Benny, W.-C., Kafeza, M., Cheung, S.C., Eleanna, K., Hung Patrick, C.K.: Alerts in healthcare applications: process and data integration. International Journal of Healthcare Information Systems and Informatics 2, 36–56 (2009) 3. Lee, W.C., Cheng, B.W.: Effects of using a portable navigation system and paper map in real derving. Accident analysis and Prevention 40, 303–308 (2008) 4. Quan, P.: Design of application Interface based on the human cognition. Computer Engineer and Application 19, 148–150 (2001) 5. Sun, X.H., Plocher, T., Qu, W.N.: An empirical study on the smallest comfortable button/icon size on touch screen. In: Aykin, N. (ed.) HCII 2007. LNCS, vol. 4559, pp. 446– 454. Springer, Heidelberg (2007) 6. Qu, W., Sun, X.H.: Interactive Style of 3D Display of Buildings on Touch Screen. In: Harris, D. (ed.) HCII 2007 and EPCE 2007. LNCS (LNAI), vol. 4562, pp. 157–163. Springer, Heidelberg (2007)
A Mobile and Desktop Application for Enhancing Group Awareness in Knowledge Work Teams Timo Saari1, Kari Kallinen2, Mikko Salminen2, Niklas Ravaja2, and Marco Rapino2 1
Temple University, 1801 N. Broad Street, Philadelphia, PA, USA, and Center for Knowledge and Innovation Research (CKIR), Helsinki School of Economics, Finland, and Helsinki Institute for Information Technology (HIIT), Finland
[email protected] 2 Center for Knowledge and Innovation Research (CKIR), Helsinki School of Economics, Finland
[email protected],
[email protected],
[email protected],
[email protected]
Abstract. In this paper we present a first prototype for a mobile and desktop system and application for enhancing group awareness in knowledge work teams. The prototype gathers information from the interactions of the group within the application and analyses it. Results are displayed to members of the group as key indexes describing the activity of the group as a whole and the individual members of the group. The advantages of using the prototype are expected to be increased awareness within group possibly leading to positive effects on group performance. Keywords: Group awareness, emotional awareness, knowledge work, mobile application, desktop application.
1 Introduction We see knowledge work consisting of the capacity to act in intelligent ways in one’s context and environment. Senge [1] suggests that while information implies knowing “about” things, and is received and passed on, knowledge implies knowing “how”, thereby giving people the capacity for effective action. Davenport et al. [2] define knowledge work as “the acquisition, creation, packaging, or application of knowledge. Characterized by variety and exception rather than routine, it is performed by professional or technical workers with a high level of skill and expertise.” Consequently, knowledge work includes the creation of knowledge, the application of knowledge, the transmission of knowledge, and the acquisition of knowledge. McGrath and Hollingshead [3] have proposed that technologies, as they have been applied to groups, can be placed along a dimension of increasing and decreasing richness of social cues. Face-to-face groups have access to a rich variety of social cues that they can then use to determine the preferences and positions of other group members. On the other hand, computer-mediated groups, do not have access to nonverbal cues, and must rely simply on the written word. That is, computer mediated groups and hence computer mediated group work are low in the richness of social cues. J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 95–104, 2009. © Springer-Verlag Berlin Heidelberg 2009
96
T. Saari et al.
There are different types of group awareness some of which are relevant to worklike tasks. According to Greenberg [4], there are several types of group awareness needed for effective collaboration: • Workspace awareness is ``the up-to-the minute knowledge a person requires about another group member's interaction with a shared workspace if they are to collaborate effectively''. • ``Group-structural awareness involves knowledge about such things as people's roles and responsibilities, their positions on an issue, their status, and group processes.'' • ``Informal awareness of a work community is basic knowledge about who is around in general (but perhaps out of site), who is physically in a room with you, and where people are located relative to you.'' • ``Social awareness is the information that a person maintains about others in a social or conversational context: things like whether another person is paying attention, their emotional state, or their level of interest.'' Other information can be the special skills a co-worker has. Also, emotional awareness within a group has been discussed in relation to knowledge work groups [see 5]. Within this article we see emotional awareness, i.e. awareness of emotion and mood states of the members of the group as part of social awareness. Often in knowledge work situations, awareness of others provides information that is essential for frictionless and effective collaboration. Even though group awareness is taken for granted in face-to-face work, it is rather difficult to maintain in distributed settings. Hence, there is a considerable challenge in designing technology to support the types of group awareness that actually may lead to increased performance or other beneficial effects at work. We propose an application to increase awareness in knowledge work groups. Our application enhances group awareness by making explicit the implicit interaction patterns in a group. We expect our application to have beneficial effects on performance in knowledge processes and tasks.
2 Use Scenario and System Design Supporting general knowledge work processes Knowledge work tasks can roughly be classified to job specific tasks and general processes [see 6]. Job-specific tasks differ greatly as a function of the type of work. Examples are preparing a budget, analyzing results in terms of estimated and actual costs, planning and scheduling a project, eliciting and documenting system requirements, and writing applications software. There are also many general processes when working. General processes can be such as goal setting, communication, updates, group cohesion and synchrony maintenance and informal group communication and coordination. “Mobile” knowledge work differs mostly from “normal” knowledge work in that it takes place in distributed settings with the use of mobile technologies. Mobile knowledge work can be mostly mobile such as when a person is communicating with others
A Mobile and Desktop Application for Enhancing Group Awareness
97
and accessing files on the field while conducting work tasks. Mobile knowledge work is naturally intertwined with non-mobile knowledge work as people move in and out of their offices. Mobile technologies for knowledge work also mix with desktop computing environments as workers also carry their mobile phones to the office. Our use scenario is enhancing general communication processes of knowledge work teams. We feel that general communication and coordination processes of knowledge work vs. task-focused processes are not well supported by current technologies. In short we propose a system in which various data is collected from the status of single users and transmitted to other users to an easy-to-use mobile and desktop application. The application supports better group awareness in terms of workspace awareness, group-structural awareness, informal awareness and social awareness. Group performance and effectiveness We hypothesize that our system and applications will influence group performance. However, there is more to groups than performance on task only. For instance Andriessen [7] has divided the key aspects of group interactions as i) performance related to activities and specific tasks aimed at reaching a common goal for the group and ii) group maintenance, i.e. activities aimed at enhancing and building cohesion and trust in the group or to gain status and power in the group. Regards group awareness discussed above we expect that “better” group awareness (workspace awareness, group-structural awareness, informal awareness and social awareness) will then lead to positive outcomes in group interactions, in both performance and group maintenance. In addition to group performance another concept, group effectiveness has been proposed to consist of three components: 1. production function: effectiveness is here defined as “the degree to which the productive output meets or exceeds the performance standards set by the clients”. Criteria used to define such dimension are product quality, product quantity, efficiency and innovativeness; 2. group well-being function: is defined as “the degree to which the attractiveness and vitality of the group is strengthened”; 3. member support function: “the degree to which participating in the group results in rewards for the individual group members” [6, pp 100]. We propose that different types of group awareness are linked to group performance and group maintenance and are prerequisites for group effectiveness. Previous studies have shown that there are several possibly central concepts that are related to social interaction in our focus area of group performance. We use the examples of group cohesion, cooperativeness or reciprocity and convergence below. Group cohesion has referred to as “the extent to which group members perceive and feel attracted to the group” [6, pp 118]. Cohesion in a group has also been described as “a tendency to stick together or to be united either physically or logically” [8, pp 130]. Another important dimension in defining the quality of a social interaction is cooperativeness, a kind of reciprocity of communication. It is described as the behavior of people towards others in the group with which they share common interests and tasks. The behavior is also characterized by the fact that each person strives towards their goals within the group and this progress is facilitated by the other person’s actions, leading each to expect reciprocation [9].
98
T. Saari et al.
There is also another interesting approach to seeing how groups communicate: conveyance and convergence. Conveyance can be seen as the exchange of information among participants of the group in which the interpretation of the messages is done by the receiving individual [10]. This refers to the individual act of receiving and interpreting messages. Convergence, on the other hand, is about producing and facilitating shared social meanings among participants, rather than individual interpretations. In previous studies it has been suggested that these two different dimensions of meaning-making in a group may emerge differently based on the different mediating technologies used. People in face-to-face interactions tend to reach consensus (high level of convergence) faster than groups using online chatrooms [11]. It has been proposed that synchronous media are better in facilitating convergence whereas asynchronous media are better in supporting conveyance [12]. It then seems that it may be sensible to think of solutions to enhancing cohesion, cooperativeness, the efficient emergence of shared meanings and convergence as part of group awareness in the group in the context of technological support for group work. We have tried to address these issues in our prototypes with several ways of creating and sharing group awareness. Activity indexes for a working group For our prototypes we have preliminarily selected a number of key activity indexes that describe the state of the group or its activities. The indexes are formed by analyzing the user’s activities through our system. The indexes are displayed to users via our applications in visual representations. We have preliminarily chosen among many potential indexes which would be useful in group work. It should be noted that we have not yet tested our chosen indexes in field tests within the context of our applications and that the indexes after our first field tests could be altered, rejected or changed. Our chosen indexes for group activity are: group reciprocity, group centralization, my participation, my reciprocity, my popularity and my influence power. These indexes are hypothesized to be related to different types of group awareness. We expect that making such indexes available in a group work situation therefore enhances certain types of group awareness. In more detail the indexes and their meanings are as follows divided into group and individual levels:
A. Group level indexes: 1. Cooperation level Description: Describes the collaboration level in the group. Based on the tendency to contact others within the group in work tasks. A group could at certain times be very highly collaborative and at some times very little. This could be used to give an understanding the activity level and “tightness” of the group. Types of group awareness supported: Workspace awareness: what is the activity level of the group, what is the context into which I am sending my messages or requests? Social awareness: What is the level of “attention” of the group, how intensively the group works together?
A Mobile and Desktop Application for Enhancing Group Awareness
99
2. Communication hierarchy Description: Describes whether the group activity reflects a tendency to be hierarchical (top-down) around some individuals or whether it is democratic (many-tomany). In other words it is like the “power structure” of the group. Types of group awareness supported: Group-structural awareness: Who is the intellectual leader of the group? Who provide the best ideas? Is the group really collaborative and democratic or really top-down led by a single individual, perhaps corporate supervisor of the group? What are the different roles of people in the group?
B. Individual indexes: 1. My influence power Description: Describes how much a person is involved in the actions of the group. Can be based on how many contacts or messages the user receives. This reflects the importance of the user in the group. Types of group awareness supported: Group-structural awareness: How central am I as a person in the group? How central are others in the group? Do I have a lot of influence over the group? Who has most and least influence over the group? 2. My popularity Description: Describes the user’s popularity based on the idea that a user is more popular the more messages he/she receives. Types of group awareness supported: Group-structural awareness and social awareness: Am I popular? Who is the most popular person in the group? How popular is the person I am sending my message to? How should I craft my negative feedback message to this person as he is so popular in the group? 3. My participation Description: Describes how actively the user participates in the activity of the group. Is assessed on how much activity the user inputs into the system. Types of group awareness supported: Group-structural awareness: How actively do I participate in the groups activities? How active is my boss or co-worker?Social awareness: What is the level of interest of a person in the task we are doing based on his participation index? 4. My reciprocity Description: Describes how much the user is mutually contacted when the user contacts others. It reflects a kind of symmetry or mutuality of communication as seen from the point of view of a single user. For instance, a user could be communicating a lot but not receiving a lot of feedback, or the user’s sent and received messages reflect a balance or communication activities. Types of group awareness supported: Social awareness: What is the attention level of other users I am contacting regards my messages? Do they reply to my messages actively or not at all? Is there someone I am ignoring? Am I isolated in this group and if so, why? Should I talk more to this person as I have not really communicated with him?
100
T. Saari et al.
3 Prototype System and Application Mobile Application We constructed two prototypes based on our approach to designing the system: a mobile application running on mobile phones and a desktop application running on top of Microsoft Outlook. These two applications are the first ones constructed and reflect the status of the project at the time they were built. First we will discuss the mobile application. Technical description: client The client is based on Flash and Python technologies. The Flash part of the application is basically for handling vector graphics and showing results that come from the server. Python handles the hardware devices like Bluetooth and camera. We used Py60 for the Symbian platform and PyCE for WM. The Flash/Python application is able to run in many platforms such as Symbian phones (S60 2nd 3rd editions), WM, desktops (Linux/Windows/Mac OSX) and PocketPC. Technical description: server Server is based on Python/PHP. The main computations for social network analysis (SNA) indexes are done by the PSE, a special software that analyses and generates the social indexes discussed earlier. The interaction with PSE happens through a special protocol - SOAP - and the SOAP client is PHP based. Interface description When starting, the application updates data and this may take some time. The application opens to an orbit view (see Figure 1, column A). In this view the members of the group are placed to a three concentric circles with the user in the middle. Other group members are placed to the orbits around you either by “My Participation” or by “My Reciprocity” indexes. A person on the innermost orbit has similar index values with the user, whereas a person on the outermost orbit has very dissimilar values with the user. The user is given explanations of all the icons shown in Figure 1. For example MyParticipation is explained as ollows My participation refers to how extensively involved a person is in the communication among the groups members. Similar explanations and icons are used for other indexes. Left and right buttons in the mobile phone are used for rolling the circles to select a user, for the selected user the user name is highlighted. A firing button press opens a details window for the group member that has been selected. In the details window following data are shown (Figure 1, panel D): user name, availability and time tag, activity: a free text field for the current activity, last used: a time tag for when the user last time launched the mobile application, nearby: a list of nearby group members, based on mobile phone’s Bluetooth proximity, also, a user taken photo, with a time tag, is shown. The details window can be shut by pressing again the firing button. The Actions menu under the right soft key (Figure 1, panel E) offers sending a private message to the selected user, calling to the selected user, or sending him/her a sound message. From the actions menu the application can also be put to the background or closed.
A Mobile and Desktop Application for Enhancing Group Awareness
101
For some phone models it is not possible to put applications to the background, in these cases the only way is to shut the application, if other phone functionality is needed, and then again restart it.
A - Orbit
D - Details pop-up
B - Main
E - Details pop-up
C - List
F - My Details
G - Chat
Fig. 1. Different views of the first version of the mobile application. A) The orbit view. B) Main menu. C) List view. D) Details pop-up window for a single user data. E) Details pop-up window with the actions menu opened. F) My details view for inputting availability information etc. G) Chat view.
Pressing the Main with the left soft key opens the Main menu (Figure 1, panel B). Here the user can choose between the previously described orbit view, the List view, Input menu, and the Chat area. In the list view the list of group members is sorted with the same criteria as in the orbit view. Pressing the firing button opens the details pop-up window for a single user. In the My Details the user can input his/her status, mood, reason, action and launch the camera for taking pictures (Figure 1, panel F). On some phone models it is not possible to take pictures through the application. In the chat area the messages are preceded with a time tag (for example 1h = the message was sent 1 hour ago) and a user name (Figure 1, panel G). Desktop Application The desktop application works through an Outlook plug-in with Microsoft Outlook 2003 (and Outlook 2007 to a limited degree) and provides augmentation to the mail client on the basis of emails sent within the application. Additionally, Internet Explorer 7 or Mozilla Firefox 2.x is required for the Outlook tab to work properly.
102
T. Saari et al.
After the installation the user should be able to see the Pasion button (i.e. the button for our application to be launched) below the standard command bar (where there are icons to create a new document, open, save etc.) in Outlook. The Outlook plug-in collects data about e-mail traffic between trial group members and sends it to the Pasion server (i.e. our application server). The collected data are sender, recipient, time and date. It should be noted that no information on the content of e-mails is collected The data collected by tour server is used in calculation of the chosen indexes. These indexes are visualized both in the Outlook plug-in and in mobile application. The indexes are calculated from pooled e-mail and chat messages. In the upper left panel of the Outlook tab is a list of group members (Figure 2). Pressing Sort list in the upper left panel opens the sorting criteria menu. The user list can be sorted by Username, My Participation (automatically calculated index, described above), My Reciprocity, My Popularity, My Influence Power, Mood (manually set variable), and Availability (manually set variable).
Fig. 2. First version of desktop application interface in Outlook
Placing the mouse pointer over the user icon in the list view opens a pop-up window. From this window it is possible to send the user a private instant message, or view the user’s detailed information. Below the group panel is the MyStatus panel. Where the user can set his/her mood, reason, activity, or download a picture which is then shown in the pop-up details window. It is also possible to ask other group members to update their mood, reason, activity, or picture by placing the cursor over the Ask button. The request to update will be sent to the chat area. After editing the values the Set button must be pressed for the values to update. A logout button is also located there. Pressing the details button opens a view where the user can change password, language (English, Finnish, Italian, and Spanish), phone number, Bluetooth address, and e-mail.
A Mobile and Desktop Application for Enhancing Group Awareness
103
The orbit view operates similarly as with the mobile application, described above. Bringing a cursor over a user shows the details window on the right. A private instant message can be sent to user by clicking him/her. A separate tab is opened for private messaging to each group member. These can be closed from the chat areas title bar. In the details pop-up window there are shown: the user name, user taken picture, reason for mood, activity, last used time tag and nearby users determined by the Bluetooth proximity. The shown indexes are: My Participation, My Popularity, My Influence Power, and My Reciprocity.
4 Discussion One of the key future challenges for our work is to identify tasks that most benefit from the use of our system. After identifying the tasks we can redesign our system to support them and optimize the use of increased group awareness to fit the purpose. We hope that our initial emphasis on general communication processes across several types of tasks in knowledge work teams will produce results that help to focus our work further. There may also be challenges in the area of privacy when using our system. For instance, we hypothesize that no single user would wish to publicly transmit their social information. Similarly, problems of social comparison may arise. This has at least two sides. First, if a user is always in the periphery of the group as indexed by our visualizations in different knowledge work tasks, it may indicate unpopularity, not being liked, or not being valued. Second, as this is a system for work, there is always the question of one’s value to the employer. If the social network analysis visualizations show to one’s boss that one is constantly not producing much input and is at the periphery of interactions at a task in which one should perhaps not be at the periphery, this may create doubts about the role and value of such a person to the employer. On the other hand, both problems described above regards social comparison also cut the other way. For instance, if someone is very central and active as indexed by our system in some knowledge work task, it informs the person as well as the boss of his popularity and value to employer. In this way one could perhaps have “hardcore data” of one’s performance within the group and in the eyes of the employer. At the group level, it could be hypothesized that people tend to pump-up their performance relative to others and this would have positive outcomes regards group performance in total. However, the main problem may be the social mirror- effect. The indexes used create an individual mirror image in terms of: “Who am I in the context of this group? What is my worth? Am I popular and liked? Am I effective at my tasks?” Our system can also create an image of the others in the system, such as: “Who is this popular person? Why is this person the center of our communication? Why is this person at the periphery of our discussions when I expected more?” Our system creates a mirror image on some dimensions of the group using the system. This image can be either flattering or not. Despite the obvious challenges we feel that users will benefit from the use of our system by gaining a more holistic view of the group they are working with in addition to gaining insights about themselves in
104
T. Saari et al.
relation to the group. Our rationale is to enable users to transmit and receive enriched social cues to enhance their communication processes while working. The intrusiveness, resolution and accuracy of gathering the information as well as the understandability of visual representations of various indexes are naturally critical issues. The next stage of our work is to run field tests of our application in real-life working environments. Based on this research a new, refined version of the system will be built with tested and selected functionalities and improved visualization schemes.
References 1. Senge, P.: Sharing knowledge. Executive Excellence 14(11), 17–18 (1997) 2. Davenport, T., Jarvenpaa, S., Beers, M.: Improving knowledge work processes. Sloan Management Review 37(4), 53–65 (1996) 3. McGrath, J.E., Hollingshead, A.B.: Putting the “group” back in group support systems: Some theoretical issues about dynamic processes in groups with technological enhancements. In: Jessup, L.M., Valacich, J.S. (eds.) Group Support Systems: New Perspectives, pp. 78–96. Macmillan, New York (1993) 4. Greenberg, S., Gutwin, C., Cockburn, A.: A using distortion-oriented displays to support workspace awareness. Technical report, Dept. of Comp. Science, Univ. of Galgary, Canada (January 1996) 5. Saari, T., Kallinen, K., Salminen, M., Ravaja, N.: A System for Facilitating Emotional Awareness in Mobile Knowledge Work Teams. In: 41st Hawaii International International Conference on Systems Science (HICSS-41 2008), Proceedings, Waikoloa, Big Island, HI, USA, January 7-10, 2008. IEEE Computer Society, Los Alamitos (2008) 6. Woodman, R.W., Sawyer, J.E., Griffin, R.W.: Towards a theory of organizational creativity. Academy of Management Review 18(1), 293–321 (1993) 7. Andriessen, J.H.E.: Working with groupware. Understanding and Evaluating Collaboration Technology. Springer, London (2003) 8. Reber, A.S.: The penguin dictionary of psychology. Penguin, London (1985) 9. Raven, B.H., Rubin, J.Z.: Social psychology: People in groups. Wiley, New York (1976) 10. Wheeler, B.C., Dennis, A.R., Press, L.I.: Groupware comes to the Internet: charting a new world. ACM Sigmis Database 30(3-4), 8–21 (1999) 11. Dennis, A.R., Valacich, J.S.: Beyond media richness: an empirical test of media synchronicity theory. In: Proceedings of the Thirty-Second Hawaii International Conference on System Sciences, vol. 1 (1999) 12. Hung, Y.T.C., Kong, W.C., Chua, A.L., Hull, C.E.: Reexamining media capacity theories using workplace instant messaging. In: Proceedings of the 39th Annual Hawaiii International Conference on System Sciences, vol. 1, 19.2 (2006)
A Study of Fire Information Detection on PDA Device Xianghong Sun1, Weina Qu1, Thomas Plocher2, and Li Wang1 1
State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China {quwn,sunxh,wangli}@psych.ac.cn 2 Honeywell ACS Labs, Minneapolis MN 55418, USA
[email protected]
Abstract. This study was concentrated on how useful the en route information display system for firefighters’ information accessing, current situation understanding, and decision making, we did a series of tests to investigate the efficiency of the system, to compare different display ways including audio, text, and their combinations to find the most appropriate one. The result showed that: (1) Audio only always made firefighter taking the longest time to information detection, but the introduction of audio display made the two combined displays (text + audio, and text + 3rd level audio) more quickly to access information, and more easy to remember. (2) It should be clarified that en route system could be used very well either in quiet and static environment, or in a moving and a little bumping environment if user could get some training before using it. Keywords: Information detection, PDA, fire.
1 Introduction Fire incidents came into being with the discovery and utilization of fire and are closely linked to the advancement of human civilization [1]. Firefighter will know about details after arriving firing spots today. For saving time, a handheld PDA device has been developed, which could show the current fire related information to fire commanders. Fire alarm system is an essential part of high buildings in modern times, which helps firefighters’ detection more efficiently and reduces the casualty [2]. Ko’s research focused on proposed robust fire-detection algorithm that is installed in home network server [3]. How useful the en route information display system for firefighters’ information accessing, current situation understanding, and decision making, we did a series of tests to investigate the efficiency of the system, to compare different display ways including audio, text, and their combinations to find the most appropriate one. Purpose of the experiment is to answer the following questions: is the information they can get from the en route display system really helpful for firefighting decision making? In other words, comparing with the situation without en route display system, is the en route information really help the incident commander assess the situation and make decisions any faster or more accurate? Could firefighters be “primed” by en route information? J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 105–113, 2009. © Springer-Verlag Berlin Heidelberg 2009
106
X. Sun et al.
2 Method 2.1 Experimental Environment A handheld PDA device has been developed as the experimental platform, which could show the current fire related information to fire commanders. The size of words is based on previous experiment [4] .Totally 16 fire scenarios drawn from the previous 3D fire information display prototype [5] were pre-installed in the prototype. The experimental tasks were completed both in a lab and in a moving car. Since the PDA device was not dedicated for the en route display system, all the keys and buttons used in the experiment were four arrow keys for going up/down to the upper/lower level menu, and one middle key for getting updated information, the other keys became to be distracters. In order to avoid interrupting the application of en route prototype during the test, all the other functions were closed, and all the other keys and buttons were masked by plaster (see Fig. 1).
Fig. 1. PDA device as the experimental platform
Experimental places divided two parts: in a lab and in a moving car. In the test room, firefighters were asked to finish fire information detection task by using PDA prototype and 3D fire information display prototype. In the moving car, firefighters were asked to do a dual-task: the main task is to counting how many street lamps passing by on right side the road and speak out the number of lamps loudly; the secondary task is to finish fire information detection task using PDA prototype. 2.2 Participants 12 firefighters aged from 21 to 35 years participated in our experiment. 9 of them have more than 5-year firefighting experience. 6 of them have bachelor degree.
3 Procedure Test part was divided into three stages: test in lab, test in moving car, and test in lab with 3D prototype.
A Study of Fire Information Detection on PDA Device
107
Test in Lab 1. First alarm and fire spread finding (first on PDA then on PC touch screen): firefighter was exposed to four different scenarios (using hospital fires: SS fire, and SM fire) one by one. Each scenario was corresponding to one kind of displays: audio only, text only, combined text +audio, and partly combined text + audio. The display order was counterbalanced. For each scenario, firefighter was asked to find the location of the first fire alarm, and the fire spread by using en-route display as quickly as possible, task completion time was recorded as the performance of en route system. Then he was asked to go to the 3D fire information display system, which was used to compare the efficiency in the experiment of 2006[5], to find the location of the first alarm and the fire spread again. Task completion time was also recorded as the performance of 3D fire information display prototype. Test in Moving Car 2. First alarm and fire spread finding on PDA: procedure and the task requirement is the same as the task 3 in the lab. Also four scenarios (tower building fires: SS fire, and SM fire) was exposed to firefighter one by one. But firefighter was only asked to find the first alarm and the fire spread on PDA device. Task completion time was recorded as the performance. Test in Lab again: 3. First alarm and fire spread finding on touch screen: Just after the test in car and going back to the lab, only one scenario (the last one of the four scenarios used at the fifth step) was exposed to firefighter. The time of finding the location of first alarm and the fire spread was recorded as the performance. 4. First alarm and fire spread finding: A MM fire (multi fire seeds spreading on multiple floors) was exposed to firefighter, and record the task completion time both with en route system and the 3D information display prototype. 3.1 Data Analysis of Fire Detection Task This task was to test whether en route system help the incident commander assess the situation and make decisions any faster or more accurately and which way of information display was more helpful. Experimental design. 2*4 within subjects design was used in this task. Two factors were the places where the en route system was used, and the ways of fire information display. Lab and moving car were used as the two experimental places. Four different display ways were the same as mentioned above. Each firefighter was asked to complete 8 scenario fire detections (including locating the first fire alarm and judge the fire spread). 4 of them (hospital fires: SS or SM) were detected in the lab, and the other 4 (tower building fires: SS or SM) were detected in a moving car. Each scenario was corresponding to one kind of display style. The display order of the four scenarios and the matching pair between scenarios and the ways of information display were counterbalanced. For each subject, the test in the lab went first, and then followed by the test in moving car.
108
X. Sun et al.
During the test in car, as described in the experimental environment part, firefighter needs to fulfill a dual-task: counting as a distracted task was used as main task, and fire detection is the secondary task. Experimental Result Test in Lab Fire detection with en route system. Table 1 showed mean and std. deviation of task completion time in finding first alarm when using en route prototype. Task completion time was recorded from choosing the scenario by moderator to find the first alarm item. It’s found that “audio only” display was the worst way. It took the longest time. (F (3) =25, p=0.00). Among the other three display styles, there was no significant difference existed. Table 1. Task completion time in finding first alarm task with PDA
Mean of task completion time Std. Dev. N
combined text + audio
text + 3rd level audio
audio only
text only
0:00:50
0:00:11
0:00:09
0:00:06
0:00:26 12
0:00:11 12
0:00:07 12
0:00:02 12
Table 2 showed mean and std. deviation of task completion time in finding fire spread when using en route prototype. Task completion time was recorded from just finding the first alarm to finding the fire spread item. Among the four display styles, Both table 1 and table 2 showed the audio display took the longest to fulfill the fire detection task, but there was no significant difference existed. Table 2. Task completion time in finding fire spread task with PDA
Mean of task completion time Std. Dev. N
combined text + audio
text + 3rd level audio
audio only
text only
0:00:15
0:00:03
0:00:04
0:00:09
0:00:24 12
0:00:01 12
0:00:04 12
0:00:18 12
Fire detection with 3D information display prototype. Table 3 showed mean and std. deviation of task completion time in finding first alarm when using 3D information display prototype. Among the four display styles, there was no significant difference existed. Table 4 showed mean and std. deviation of task completion time in finding fire spread when using 3D information display prototype. Among the four display styles, there was no significant difference existed.
A Study of Fire Information Detection on PDA Device
109
Usefulness of the en route information display system. In our previous experiment conducted in 2006 [5], it’s found that, without the help of en route system participants didn’t finish fire detection task within 30s. In this experiment, combining the data in table 3 and table 4, we found all subjects could finish all the tasks no more than 30s when using “Text + the third level auditory” display for SS or SM fires. Table 3. Task completion time in finding first alarm task using 3D prototype
Mean of task completion time Std. Dev. N
combined text + audio
text + 3rd level audio
audio only
text only
0:00:24
0:00:17
0:00:24
0:00:11
0:00:25 12
0:00:18 12
0:00:25 12
0:00:05 12
Table 4. Task completion time in finding fire spread task using 3D prototype
Mean of task completion time Std. Dev. N
combined text + audio
text + 3rd level audio
audio only
text only
0:00:45
0:00:22
0:00:24
0:00:19
0:00:50 12
0:00:14 12
0:00:21 12
0:00:16 12
Test in Moving Car Sitting in a moving car, firefighter was asked to do a dual-task: counting street lamps loudly + fire detection. In this situation, moving and bumping made text on screen difficult to read, and dual-task made attention resource very limited to fire detection task. One hypothesis was that audio display would help firefighter to access information and understand it easily and quickly. Especially for the situation that user’s hands were not available for operating the en route system, which happened rarely in the lab situation. Therefore, except for the four kinds of displays we used in lab, an auto-play way was added as the fifth level of the information display ways. Here the auto-play meant that the fire information could be automatically play by voice message and any key/button operation was not necessary, which ensure firefighter could get all the information even if they had no hand, or had no time to activate the en route system. So, for each firefighter in this test part, he would do 5 fire scenarios and each scenario was corresponding to one kind of display styles. There are total 5 display styles, including auto-play audio way. The order of the 5scnarios was counterbalanced. Table 5 showed mean and std. deviation of task completion time in finding first alarm when using en route system. The time was recorded from choosing the scenario by moderator to the time point of having found the first alarm item. It’s found that finding first alarm using “audio only” display was the worst way. It took the longest time (F (3) =4.4, p=0.01). The bottom half in table 5 showed mean and std. deviation
110
X. Sun et al. Table 5. Task completion time in fire detection using en route system in car
task completion time Mean the first fire alarm
Std. Dev. N Mean
the fire spread
combined text + audio
text + 3rd level audio
audio only
text only
0:00:40 **a
0:00:13
0:00:05
0:00:05
0:00:54
0:00:10
0:00:02
0:00:03
12 0:00:08 **b
12
12
12
0:00:03
0:00:03
0:00:03
0:00:02
0:00:01
0:00:01
12
12
12
Std. 0:00:05 Dev. N 12 a: F (3)=4.4 p=.01; b: F (3)=9.2 p=.00.
of task completion time in finding fire spread when using en route system. It’s found that using “audio only” display to find fire spread was the worst way too. It took the longest time also (F (3) =9.2, p=0.00). Counting Task Performance This task aims to simulate an attention distracter, which can interrupt how the subject finished detection task. All the firefighters were required to count how many street lamps were passing by on right side along the road and to speak out the number loudly. Participants began to counting task when the recorder asked the participants to do it. After the experiment, the experimenter and the recorder count the number of street lamps in all together. From the table 6, we can see that S1 and S10 have the worst counting task performance, 67% and 69%, respectively. Most of the participants have good counting task performance. Someone even counted the number absolutely correct! The mean of correct percentage is 89%. Table 6. Percentage of correct counting street lamps in moving car Subjects
S1
S2
S3
S4
S5
S6
S7
S8
Percentage of correct 67% 90% 97% 96% 89% 85% 92% 100 answer
S9 S10
S11
S12
99% 69% 88% 91%
Comparison of the two test situations: in test room vs. in moving car. Fig 2 and Fig 3 showed the comparison of task completion time of finding first alarm and fire spread task between lab situation and car situation. They showed the same trend wherever in the lab or in moving car. It seemed that subjects could take shorter time in car to fulfill the fire detection task than to do it in the lab. There could be two reasons for the result: 1) The en route system was not so complicated to operate and understand, so even in a moving and bumping situation, it was
A Study of Fire Information Detection on PDA Device
111
0:01:00 0:00:52 audio only 0:00:43
text only Text + the third level auditory display combined text + audio
0:00:35 0:00:26 0:00:17 0:00:09 0:00:00 in the lab
in driving car
Fig. 2. Task completion time of finding first alarm
0:00:17 0:00:16
audio only text only
0:00:14 0:00:12
text + the third level auditory display
0:00:10 0:00:09
combined text + audio
0:00:07 0:00:05 0:00:03 0:00:02 0:00:00 in the lab
in driving car
Fig. 3. Task completion time of finding fire spread
still easy for firefighter to use. 2) There perhaps existed a training effect. In our experiment firefighter always completed the tasks in the lab first and then went through the rest tasks in car, which probably made them more and more familiar with the PDA device.
112
X. Sun et al.
Comparison of the two auditory displays: auto-play vs. play by manual control. For the auto-play display, mean of the task completion time is 2 minutes 52 second. 8 subjects thought automatic style wasted much time to find useful information. Test in Lab Again (Complicated Fire Test) Most scenarios used in lab test and in car test were SS fire, or SM fire, which were easy for firefighters to understand the current fire situation. In order to find the advantages and disadvantages of en route system, and to find the appropriate conditions that en route system could be made the best use of, after completing the 8 fire scenarios and firefighter’s going back to the lab firefighter was asked to fulfill fire detection task to a complicated fire (hospital or Camden fires: MM fire). The task completion time was recorded. 12 subjects were divided into three groups, and each group did one scenario using one of the four kinds of display styles. After they finished finding first alarm and fire spread by en route system, they would be asked to use the 3D FirstVision graphical display prototype to do the fire detection task. Table 7 showed mean and std. deviation of task completion time of fire location when using en route system. Result was similar as above. Audio display took the longest time (F (3) =71.03, p=0.00). Among the other three display styles, there was no significant difference existed. For the performance of fire spread, among the four display styles, there was no significant difference existed. Table 8 showed the task completion time of fire detection when using 3D prototype. Among the four display styles, there was no significant difference existed. Table 7. Task completion time in fire detection using en route system
task completion time
the first fire alarm
the fire spread
Mean Std. Dev. N Mean Std. Dev. N
audio only 0:00:12 ** 0:00:1 3 0:00:4 0:00:4 3
text + 3rd level audio
text only
combined text + audio
0:00:03
0:00:04
0:00:02
0:00:0 3 0:00:2 0:00:1 3
0:00:01 3 0:00:2 0:00:0 3
0:00:0 3 0:00:3 0:00:1 3
**F (3) =71.03, p = .00. Table 8. Task completion time in fire detection using 3D prototype
task completion time the first fire alarm the fire spread
Mean Std. Dev. N Mean Std. Dev. N
audio only
text only
combined text + audio
0:00:16 0:00:12 3 0:00:19 0:00:07 3
0:00:12 0:00:10 3 0:00:53 0:00:09 3
0:00:26 0:00:16 3 0:01:01 0:00:43 3
text + 3rd level audio 0:00:14 0:00:4 3 0:00:31 0:00:16 3
A Study of Fire Information Detection on PDA Device
113
For complicated fires, subjects couldn’t finish all the tasks within 30s. And few of the 12 firefighters could find there were 2 fire seeds in the test scenario.
4 Conclusion In order to answer how useful the en route information display system for firefighters’ decision making, we did a series of tests to investigate the efficiency of the system, to compare different display ways including audio, text, and their combinations to find the most appropriate one. Based on the data, we can summarize our findings as follows: 1. En route information display system was useful to help firefighter get the critical fire information and make decision more quickly and accurately. 2. Comparing the four information display ways: audio only, text only, audio + text, and text + the 3rd level audio, audio only always made firefighter taking the longest time to understand the information, but the introduction of audio display made the two combined displays (text + audio, and text + 3rd level audio) more quickly to access information, and more easy to remember. 3. Comparing the two situations of using the en route system: in lab, and in moving car, it was found that for the task of fire information detection, it took shorter when it was in moving car than in lab, but there was no statistical difference. This result didn’t mean people could do a better job in car than in lab. It should be clarified that en route system could be used very well either in quiet and static environment, or in a moving and a little bumping environment if user could get some training before using it.
References 1. Guo, T.N., Fu, Z.M.: The fire situation and progress in fire safety science and technology in China. Fire Safety Journal 42, 171–182 (2007) 2. Fang, Z.J.: Development of human-machine interaction: multimedia and multisensory. Human Factors 2, 34–38 (1998) 3. Ko, B.C., Cheong, K.H., Nam, J.Y.: Fire detection based on vision sensor and support vector machines. Fire Safety Journal 44, 322–329 (2009) 4. Sun, X.H., Plocher, T., Qu, W.N.: An empirical study on the smallest comfortable button/icon size on touch screen. In: Aykin, N. (ed.) HCII 2007. LNCS, vol. 4559, pp. 446– 454. Springer, Heidelberg (2007) 5. Qu, W., Sun, X.H.: Interactive Style of 3D Display of Buildings on Touch Screen. In: Harris, D. (ed.) HCII 2007 and EPCE 2007. LNCS (LNAI), vol. 4562, pp. 157–163. Springer, Heidelberg (2007)
Empirical Comparison of Task Completion Time between Mobile Phone Models with Matched Interaction Sequences Shunsuke Suzuki1, Yusuke Nakao2, Toshiyuki Asahi1, Victoria Bellotti3, Nick Yee3, and Shin'ichi Fukuzumi2 1
NEC Corporation, Common Platform Software Research Laboratories, 8916-47, Takayama-Cho, Ikoma, Nara 630-0101, Japan {s-suzuki@cb,t-asahi@bx}.jp.nec.com 2 NEC Corporation, Common Platform Software Research Laboratories, 2-11-5, Shibaura, Minato-ku, Tokyo 108-8557, Japan {y-nakao@bp,s-fukuzumi@aj}.jp.nec.com 3 Palo Alto Research Center, 3333 Coyote Hill Road, Palo Alto, CA 94304, USA {bellotti,nyee}@parc.com
Abstract. CogTool is a predictive evaluation tool for user interfaces. We wanted to apply CogTool to an evaluation of two mobile phones, but, at the time of writing, CogTool lacks the necessary (modeling baseline) observed human performance data to allow it to make accurate predictions about mobile phone use. To address this problem, we needed to collect performance data from both novice users’ and expert users’ interactions to plug into CogTool. Whilst novice users for a phone are easy to recruit, in order to obtain observed data on expert users’ performance, we had to recruit owners of our two target mobile phone models as participants. Unfortunately, it proved to be hard to find enough owners of each target phone model. Therefore we asked if multiple similar models that had matched interaction sequences could be treated as the same model from the point of view of expert performance characteristics. In this paper, we report an empirical experimental exercise to answer this question. We compared identical target task completion time for experts across two groups of similar models. Because we found significant differences in some of the task completion times within one group of models, we would argue that it is not generally advisable to consider multiple phone models as equivalent for the purpose of obtaining observed data for predictive modeling. Keywords: Cognitive Model, CogTool, Evaluation, Human Centered Design, Human Interface, Mobile Phone, Systematization, Usability Test.
1 Introduction Usability evaluation should be performed in the early phase of a product development process [1]. In addition, commercial enterprises demand that evaluations do not incur high costs. To satisfy these requirements, some of the authors of this paper have been J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 114–122, 2009. © Springer-Verlag Berlin Heidelberg 2009
Empirical Comparison of Task Completion Time between Mobile Phone Models
115
developing systematized evaluation methods and tools that can be applied early and economically [2]. CogTool [3] is a user interface evaluation tool for predicting task execution process and task completion time, using a given interface. In CogTool, a user model based on ACT-R cognitive architecture [4] mimics execution of a task by using graphical specification data extracted from frames of a storyboard for the task, which is input into CogTool in advance. CogTool offers a low-cost evaluation approach for the early part of a product development process. Just a sketch as the storyboard, which need not be functionally implemented, is enough for evaluation with CogTool. This small requirement allows us to evaluate the user interface early, cutting the cost of developing the system in which the user interface works. As a computational user model, not an actual human, executes tasks in CogTool, costs such as recruiting, organizing and paying participants and use of a usability lab are avoided. To apply CogTool to an evaluation for a new system, it is necessary to refine the user model to improve the accuracy of its predictions, using observed data of actual experts’ and novice’s task execution (observed performance data). In this refinement, we planned to incorporate observed performance data into CogTool’s user model and then compare its predictions with additional observed data [5]. The user model in CogTool can represent a novice who explores how they should interact with the target system, or an expert who can quickly execute the most efficient interaction sequences. In order to collect enough observed data to both incorporate in CogTool and to compare with its predictions, we needed to recruit a considerable number (approximately twenty) of experts, who had owned the specific model of the product to be evaluated for longer than two months. This research was part of an effort that also included comparing CogTool’s predictions in mobile phone evaluation to subjective user impressions in order to see if there were any correlations between these two different evaluation approaches.
2 Challenge of Recruiting Owners of Specific Models Recruiting owners of specific mobile phone models is very hard because the numbers of owners of a given model are low, due to the fact that new models are released frequently. Also, recruiting is expensive because “Owner of a specific mobile phone model” is a stricter qualifying condition for a recruiter than general conditions such as age and gender. Even if cost is not an obstacle, the recruiting may take a long time. Of course, an alternative way to make the recruiting easier would have been to reduce the number of participants. However we wanted to keep the required number (twenty) because we would like to modify the user model in high accuracy and to analyze the correlation between completion time and subjective impression statistically. Thus, it was clear that it would be quicker and cheaper to find owners of several similar related models that have matched interaction sequences for target tasks than only owners of one specific model. We defined a “matched interaction sequence” as the same sequence of key presses required to complete a given task.
116
S. Suzuki et al.
As mentioned above, our main objective in this research was a planned comparison between CogTool model predictions and observed user performance data. Our planned method was to capture the duration of each interaction event on the mobile phone by analyzing video frames and recording real user key presses [5] as they perform task execution steps in the same order as is specified for the CogTool model (thus excluding idiosyncratic user performance). This protocol was what drove the demand that all mobile phone models for the observed data have to have matched interaction sequences. In this paper, we report on a preliminary procedure that was conducted prior to our main observed user performance data collection effort. This was an empirical validation to clarify whether we could treat multiple mobile phone models with matched interaction sequences as equivalent for the purposes of predictive modeling.
3 Experiment This section describes our experiment in which we collected user task completion times (not individual key presses as is planned for our future study) across phone models in two groups (A and B), each defined by its members being a target phone that we wished to evaluate with CogTool or having matched interaction sequences to the target phone. This meant that within each group, the user interfaces were similar and tasks could be executed using exactly the same steps across models. The main difference between the models was simply their physical form factor (they had equivalent but differently sized and spaced keys). Participants executed a set of the same tasks with the same key press sequences across all the models in the group. After collecting data on participant performance, we compared the mean of the completion time for each task between the phone models. We explain this method in more detail below. 3.1 Mobile Phone Models We defined 2 mobile phone model groups. In Group A, there was the N905i, which was a target model for CogTool, and the N905iμ, which has the same interaction sequences as the N905i (Fig.1). In Group B, there was W61CA, another target model for CogTool, and the W61H and W53H, which had same interaction sequences as the W61CA (Fig.2). The models in each group have a matched key layout. For example, both N905i and N905iμ have a mail key above a main menu key. However, size, form, depth to press, and distance of keys varies by model. Because Fitts’ law [6] [7], which was used in CogTool, was the logarithm of key distance / key size, we supposed that a small gap of the distance or the size between the models would not affect the time to reach from a key to another. In this experiment, we selected only tasks with matched interaction sequences on the phone model within the group. For instance, in the case where a user has to select a target item to go to the next frame in a task, if the number of items above the target is different between the models, the number of key presses is also different with them. This difference means that their interaction sequences are not matched. Therefore, we
Empirical Comparison of Task Completion Time between Mobile Phone Models
117
Fig. 1. N905i (left) and N905iμ (right) in Group A
Fig. 2. W61CA (left), W61H (center), and W53H (right) in Group B
did not use either these models or tasks in the experiment. If the difference in number of items did not affect the interaction sequence (e.g., the items which are below the target item), the models and the task were usable in this experiment. Also, in cases where displayed labels of the items except the target one differed between the models, we used the models and the tasks in this experiment, because the participants trained to be experts, who already knew the interaction sequences for the task, could find which item they should select without comparing its label with other items’. 3.2 Participants 20 participants (16 males and 4 females, age: 20-40s) for Group A and 24 participants (18 males and 6 females, age: 20-40s) for Group B took part in this experiment. We did not select participants based on prior experience with specific mobile phone models. Instead, we provided all participants with extensive time to learn specific target tasks on specific phone models as described in 3.6 Learning.
118
S. Suzuki et al. Table 1. Task list for Group A
Content
Task 1
Task 2 Task 3
After inputting 000-3454-1111, store the number in a phonebook. Enter the name “Nihondenki” in Kana letter. Set a schedule named “Nomikai” in Kana letter from 19:00 to 23:00 tomorrow. Turn on/off “the auto keypad lock after folded”
The number of key presses 39
43 24
Task 4
Check the newest mail sent to Aoki-san in the “Friend” folder.
8
Task 5
Take a picture soon after launching a camera. Then save the picture in the “Camera” folder in the “My Picture” folder.
11
Table 2. Task list for Group B
Content
Task 1
Task 2 Task 3 Task 4
After inputting 000-3454-1111, store the number in a phonebook. Enter the name “Nihondenki” in Kana letter. Set a schedule named “Nomikai” in Kana letter from 19:00 to 23:00 tomorrow. Check the newest mail sent to Aoki-san in the “Friend” folder. Take a picture soon after launching a camera. Then save the picture as an idle screen, with a sub menu.
The number of key presses 39
40 6 13
3.3 Tasks We used 5 tasks for Group A and 4 tasks for Group B. The tasks are listed in Table 1 and Table 2. They are common mobile phone functions. At the same time, we selected tasks with various numbers of key presses. Although there were alternative interaction sequences for each task, we instructed all participants to use the same sequence for each task in this experiment. 3.4 Task Assignment to the Participants For Group A, we assigned two of the five tasks (see Table 1) to each participant. Thus, for each task in Group A, we had data from eight participants. For Group B, we
Empirical Comparison of Task Completion Time between Mobile Phone Models
119
again assigned two of the four tasks (see Table 2) to each participant. Thus, for each task in Group B, we had data from twelve participants. Each participant completed these two tasks across the phone models within the group. 3.5 The Number of Trials for Data Collection During the study, each participant repeated each task five times. We will refer to this portion of each task as the “main trial”. In addition, when participants switched from one phone model to the next, they performed two practice trials before the main trials for each task. These practice trials are not included in the data analysis. 3.6 Learning We set aside time for participants to learn assigned interaction sequences. In the learning phase, they executed the tasks assigned to each of them, with the assigned interaction sequences. In this experiment, we had to compare data generated by experts, because the observed data required for refinement of CogTool also needed to come from experts (as well as novice users). Another purpose of the practice was to reduce variance in completion time, because the more trials a person does, the smaller the gap of the completion time between the previous trial and the next one, along a general learning curve [8]. In these purposes, we set as many practice trials as possible so that participants were able to learn the interaction and develop as much expertise as possible. The practice for both models took place before the main trial part, making the participant’s learning level for each model more similar since we would expect transfer effects. If the order of the experiment had been “1. practice with N905i, 2. main trial with N905i, 3. practice with N905iμ, 4. main trial with N905iμ”, the participant’s learning levels for N905iμ could have been higher than for N905i because the participants benefit from experience with the phone they use first so that they would have far more experience during 4. main trial with N905iμ than when they did 2. main trial with N905i. Therefore, we set the order as, “1. practice with N905i, 2. practice with N905iμ, 3. main trial with N905i, 4. main trial with N905iμ”. By setting the order alternately for each task, we avoided a large gap between the learning levels for each model. The numbers of practice for each task were 46 times (23 times per a phone model) in Group A and 36 times (13 times per a phone model) in Group B. The numbers were dictated by a practical concern that the entire session for each participant should be completed in 90 minutes to avoid participant’s fatigue. For example, in Group A, there are all 120 trials; 20 main trials for 2 tasks (5 trials × 2 models × 2 tasks), 8 trials to get used to the model when the model switching (2 trials × 2 models × 2 tasks), and 92 practice trials (46 trials × 2 models × 2 tasks). If it takes maximum 45 seconds to execute 1 task, it takes 90 minutes to execute all 120 trials (45 seconds × 120 trials = 5600 seconds).
4 Results We conducted a one-way repeated-measures ANOVA for each task with phone model as the factor. In Group A, there was no effect of phone model in any of the tasks (p’s > .16), see Figure 3. In Group B, we found two significant differences. There was
120
S. Suzuki et al.
a significant effect of phone model in task 2 (F[2,22] = 5.86, p < .01) and task 4 (F[2,22] = 18.72, p < .01), see Figure 4. The other two tasks in Group B were not significant (p’s > .11). Post-hoc comparisons showed that in task 2, W53H was significantly different from the other two models (p’s < .05). And in task 4, W61CA was significantly different from the other two models (p’s < .05).
Fig. 3. Average completion time for each task with each model in Group A
Fig. 4. Average completion time for each task with each model in Group B
Empirical Comparison of Task Completion Time between Mobile Phone Models
121
5 Discussion Based on the results discussed above, for our planned data collection exercise to gather mobile phone interaction performance data to incorporate into CogTool, we will use only one model as the target model. Thus, even though it is likely to be more time consuming and difficult, we should recruit only owners of, and do our evaluations on, only the specific target phone model that we plan to model with CogTool, even though the recruiting cost is more expensive. One possible concern with the study design was that we had 12 participants for each task in Group B, but only 8 participants for each task in Group A. Thus, it may be the case that we only found significant differences in Group B because we had more statistical power from the larger sample size. To examine this concern, we reanalyzed the data from Group B with 4 participants randomly removed from each task. We found that both tasks were still significant at p < .05. This suggests that the difference in sample size alone is not why we found significant differences in Group B but not Group A. One of possible reasons of the significant difference between phones is the physical characteristics of the keys because many of the participants commented that these characteristics had affected their subjective performance. For example, some of the participants commented that flat keys had been difficult to distinguish from adjacent keys because of the lack of tactile cues. Others commented that keys with deeper key press feel made it easier to distinguish multiple repeated key presses using the tactile sense. Actually, W61H and W53H have flatter and shallower keys than W61CA has.
6 Conclusion The study suggests that we should not consider multiple mobile phone models with matched interaction sequences as equivalent to the same model, because we found significant differences in the mean task completion time between the models in Group B. Even though we found no significant differences between the two models in Group A, the findings from Group B suggest that a more conservative approach overall in using only one model may be warranted for developing cognitive models to minimize potential noise from usage variations across phone models. In Group B, there were one or two tasks with a significant difference in completion time between the models even though only four tasks out of a total of 10 target tasks were executed in this experiment. Based on the differences found in this preliminary study, we expected that it would be hard to find 10 tasks that did have matched interaction sequences but did not exhibit significant differences in completion time that would be suitable for our planned main objective to collect valid observed data on which to base modeling of mobile phone interaction. With more tasks, more participants and more trials in the main study, we would expect the number of significant differences between models to increase and make our observed data less reliable. As mentioned in the Discussion section based on the participants’ comments, we expect one of possible reasons of the significant difference between phones is the
122
S. Suzuki et al.
difference in tactile key press sensation due to hardware differences between different phone models.
References 1. Nielsen, J.: The Usability Engineering Life Cycle. Computer 25(3), 12–22 (1992) 2. Bellotti, V., Fukuzumi, S., Asahi, T., Suzuki, S.: User-Centered Design and Evaluation The Big Picture. In: Proceedings of Human Computer Interaction International. Springer, Heidelberg (to appear, 2009) 3. John, B.E., Prevas, K., Salvucci, D.D., Koedinger, K.: Predictive Human Performance Modeling Made Easy. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2004, pp. 455–462. ACM, New York (2004) 4. Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y.: An integrated theory of the mind. Psychological Review 111(4), 1036–1060 (2004) 5. Teo, L., John, B.E.: Comparisons of Keystroke-Level Model predictions to observed data. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2006, pp. 1421–1426. ACM, New York (2006) 6. Fitts, P.M.: The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47, 381–391 (1954) 7. Fitts, P.M., Peterson, J.R.: Information capacity of discrete motor responses. Journal of Experimental Psychology 67, 103–112 (1964) 8. Newell, A., Rosenbloom, P.S.: Mechanisms of skill acquisition and the law of practice. In: Rosenbloom, P.S., Laird, J.E., Newell, A. (eds.) The Soar papers. Research on integrated intelligence, vol. 1, pp. 81–135. MIT Press, Cambridge (1993)
Nine Assistant Guiding Methods in Subway Design – A Research of Shanghai Subway Users Linong Dai School of Media and Design, No. 500 Dongchuan Road, Shanghai Jiaotong University, Shanghai, China
[email protected]
Abstract. In big cities, it often occurs that passengers (users) have great difficulties to recognize subway stations. Except improving the signs of subway stations, based on large amounts of field researches, we find 9 practical and effective methods to help passengers to identify subway stations. These 9 methods include visual design, aural design, and tactual design etc. This paper also tries to apply some theories of cognitive psychology about human memory in the research of subways. These methods are also applicable to other space design in subway and even general underground space design. Keywords: User Research, Subway Station, Quick-Identification.
1 Introduction The following photos of the 4 stations were taken in Shanghai Subway Line 1 from the same point of view from the train.
Fig. 1. 4 different stations in Shanghai subway
From both photos taken on the site and large amount to interview, we find that most stations in Shanghai Subway are similar. If we omit the signs which reads name J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 125–132, 2009. © Springer-Verlag Berlin Heidelberg 2009
126
L. Dai
of a station, users can barely identify stations quickly through watching the platforms. In fact, the crowd condition of the train and the different user perspective all result in missing the sight of signs. In addition, from the research on familiarity of subway users and their behaviors, we find something very interesting: the more familiar a user with subways, the less he reliant on signs. Furthermore, the flood of advertisements also disturbed users’ sight. Thus we propose to develop some assistant methods to help users obtaining guiding information form the environment. Biological cognitive theories emphasize on the offers of the environment, and the offers about human surviving can be obtained instinctively, or can be learned without much effort. This enlightened us to make good use of human instinct of identifying environment. Thus, through proper design, we can make it more energy saving and more convenient for users in guiding themselves in underground environment. The way people guiding themselves on streets enlightened us that people seldom depend on signs in identifying familiar environment, such as the way home. People tend to build their cognition right from information provided by the environment. In order to transfer the good experience on the ground to underground environment, we need to add more characteristic information for users to memorize. This may be a new way in subway guiding field. The researches on behavior and way of cognition of subway users will enlighten designers to build humane subway environment. The techniques of research include abundant observation and interviews, eyetrack experiences and psychological experiences, questionnaires, and some literature reviews. We tried to find out new fields of guiding methods in subways through developing all kinds of senses of users, such as vision, hearing, and touching, and combine them with researches about human memories and human experiences.
2 9 Assistant Guiding Methods for Subways 2.1 Visual Series – Spatial Design Aids Guiding in Subways Voice from a Beijing subway user: I absolutely won’t miss Yonghe Palace Station! Even the passengers only have been there once, they also feel the same. Compared to those featureless stations, Yonghe Palace station is easy for users to recognize and memorize.
Fig. 2. Yonghe Palace Station, Beijing (left) VS the featureless Gulou Street Station (right)
Nine Assistant Guiding Methods in Subway Design – A Research
127
In subway construction, we can add some features to subway space, thus effectively help passengers to recognize stations and guide themselves. Common method such as, setting several layers of ground or ceiling, making inter-junction of horizontal position, comparing different spatial areas, making different arrangement of the columns along corridors, etc. Even adding features in a part of the space is also effective, e.g. a clearstory in the ceiling or a raised plant container on the ground. Furthermore, we can imitate the way people recognizing surroundings above the ground, i.e. to build characteristic landmarks in underground spaces, which is also a good guiding method for users. Spatial design can effectively leave strong memories in users’ minds, because the ability of recognizing three-dimension spaces is people’s instinct, and is continually developing in the process of human evolution. One person can store huge amounts of spatial memories in his mind. If we make good use of human brains, the results will be remarkable. 2.2 Visual Series – Color Design Aids Guiding in Subways Applying au unique color for each station works well in the design of Hongkong subway lines. As long as the passenger recognizes the color of the destination station, he will take off at the right station. The result of our research shows that we should pay attention to three aspects in color design for subways: 1. Use colors with high saturation, avoid using compound colors. Make it easy for users to identify the hue of the color. 2. The area of the color should be large enough for users to identify through the window in every corner of the train. 3. Do not use similar color in two neighboring stations.
Fig. 3. Hongkong Subway: Zhonghuan Station is red (see the left picture), while Jiulongtang Station is blue
128
L. Dai
Chicago subways are named by color. Stations of the same line all continuously bear the same colored ribbon in and out of the stations. It is easy for a user to know which line he is taking, thus strengthen the guiding effect of color. Surely enough, when we use colors to mark different subway lines, large areas of high saturated color is needless; otherwise, with all the stations of one similar color, memories of the users will be disturbed. 2.3 Visual Series – Decoration Design Aids Guiding in Subways The result of eye-track experiments shows that testees are able to identify a station by decoration. Another interesting finding is that memories of decoration is not accumulating with time, but appears to be a salutatory process, i.e. once or twice a user is attracted by some decorations, he will soon form a vivid memory of it. If he sees the same decorations again, he will recognize them immediately. Considering the limited space for users to pass, when we apply this method for guiding, we’d better use big and global decoration in the station, or set the decorations along the only path for users. Furthermore, as users need to identify a station from a train window quickly, the decorations in the platform should be distinctive and eye-catching; otherwise it will reduce the resolving power of it. As for the big decorations in other places, even not so eye-catching, users will identify them when passing by. 2.4 Visual Series – Lighting Design Aids Guiding in Subways Lighting is inevitable in underground spaces. Use lighting system to guide users is a good idea. Especially in the transfer stations, sometimes, it is too crowded for users to find the features of ground and wall. Then, using lights on the air for guiding is another effective way. There are two kinds of lighting design to aid guiding in subways: the shape of the lights and the color of the lights. As most lights in subways are fluorescent lamps, we advice that we should make different arrangement of these lamps, thus take advantage of the shape of the lamps. Fig. 4 shows with low cost, the guiding effect will rise. Condition permitted, the lighting system design in Japan is better in effect. As for using the color of lights for guiding, colorful lights such as LED is needed.
Fig. 4. The different arrangement of fluorescent lamps in Beijing subways
Nine Assistant Guiding Methods in Subway Design – A Research
129
Fig. 5. Lidabashi Station, Japan, designed by Makata Sei
2.5 Visual Series – Product Design Aids Guiding in Subways In our interviews, some sensitive passengers said that they use seats, billboards, newspaper boards, lamps, and advertisements to identify destination stations. Products which can aid guiding in subways are not only unitary but also diverse. They can help the users who pay great attention to details identifying stations quickly. Fig. 6 shows two subway stations with different multi-functional billboards in Vienna, which also serve as a symbol for station identification.
Fig. 6. Different multi-functional billboards in Vienna subways
2.6 Visual Series – Material Design Aids Guiding in Subways In the interview to Hongkong subway users, the effect of material in guiding is confirmed. Different materials are used in different lines of Hongkong subways. Thus the inter-changeable station Lijing Station between Quanwan Line and Dongyong Line use mosaic on one wall and plastic-aluminum board on another.
130
L. Dai
Fig. 7. Lijing Station of Hongkong subway
In the research of Shanghai subway users, some users said: Shanghai South Railway Station is relatively new. Thus proves that some passengers do use material to help identifying stations. But most passengers haven’t perceived the different materials used in Shanghai subways. Experiment shows that the effect of material in guiding is relatively poor. Only when the feature of material is enlarged and form a global impression, the effect will be obvious. 2.7 Aural Design Aids Guiding in Subways Some Shanghai subway users identify stations by sound. We got following comments in our interviews: I know which station is People’s Square even with my eyes closed, because almost all the people on the train will take off at this station with a big noise. Although I miss the broadcast of the name of a station, it will be fine, because the following broadcast will introduce some tourist sites for passengers. Once I heard a familiar name, I know it’s time for taking off. What’s more interesting is the musical broadcast in Pusan subway in Korea. If the train will soon stop in a station with sea nearby, you will hear the sound of ocean wave and sea-gulls. If the station is near mountains, you will hear birds twittering in the woods. Commuters will easily identify stations with those sounds of nature. Subway users take advantage of all kinds of sound to help identify stations. Based on memory theory, music is easier for people to memorize than other sounds. Using background music to guide passengers is feasible. Especially for those people who with poor eye-sight, a familiar tune is more friendly than any other guiding method. 2.8 Scent Design Aids Guiding in Subways We have not find any example about how scent help Shanghai subway users identifying stations in our research, but some passengers did mentioned that sometimes a
Nine Assistant Guiding Methods in Subway Design – A Research
131
special scent will remind them of a place. For example, a user who frequently takes off at Xujiahui Station mentioned that he can even smell the path to Pacific Bazzar because there is a W.C. on the way. These interesting reports excite us to explore how scent memories aid guiding in subways. As we all know, the nose of human being can help to store large amounts of information in human memory. Compared to other sense organs, nose is more reliant on intuition. Once a memory of scent is formed, it is hard to forget. 2.9 Experimental Memory Aids Guiding in Subways From some interesting interviews we find that some passengers’ guide themselves by a special experience in a certain station in the past, i.e. they memorize the station because they did something special in such place. Yes, I can identify Shanghai Railway Station because there is a W.C. on the platform. That’s true. I have been there, too. Haha~ Our interview proved that if we provide some additional functions in subways properly, e.g. commerce, entertainment, and exhibition, those colorful experiences will also help users memorizing and recognizing subway space. Subways in Paris and Japan, which built many years before, also have lots of commercial areas. Underground commerce can not only bring great profits, but also brings more stories about subways, and makes the dull trip a colorful experience. 2.10 Other Aspects of Design Aids Guiding in Subways There are many more interesting methods about how users guide themselves in subways, which is beyond our list. Only when we merge ourselves into real sites, can we hear them, see them, and feel them. For example, one passenger said he identify stations by watching the side of the opening door. Another passenger said that, we only need to remember one featured station, and then count the stops for destination. Other passengers remember the inter-change station by noticing other passengers’ behavior. We also meet a passenger who marks the number of the door to get on the train in the morning. Some passengers are so smart that even they have fallen asleep on the train; they will wake up automatically on the destination station.
3 Conclusion The 9 methods mentioned above are rooted in clues from different human senses. We hope that through proper design, underground spaces can provide adequate methods for users to memorize, thus let users identify and get to their destination quickly and conveniently. Make it easy for passengers forming cognizing-map in their minds, improve the efficiency of underground traffic. But we should notice that many users have their unique methods to identify underground environment, the construction conditions in different cities are also various, it is not possible for designers to consider every details. So how these methods make effect depends on the development of subway design and the purpose of constructors.
132
L. Dai
What’s more, the reorganization to environment of human beings is a global system. Based on theories of Gestalt psychology, perception of human beings is an integrated Gestalt, which is inseparable. Thus the 9 methods mentioned above are guide lines for designers in their exploration. It was the comprehensive effect of all the methods that make users identify underground environment quickly. A really good designer is who can guide users to use facilities conveniently and efficiently, thus bring pleasant and humane experiences.
Reference Li, J., Tang, Y., Qu, L., Zhang, D., Chen, R., Yao, Z., Xi, W., Zhou, X., Ren, H., Yu, L., Zhao, Z.: All the data in this paper is from the PRP Creative Project of Shanghai Jiaotong University (Serial Number: PRP-C10067) and National Creative Experience Project of College Students (Serial Number: ITP040). Project team members include the graduate and undergraduate students as follows
Pull and Push: Proximity-Aware User Interface for Navigating in 3D Space Using a Handheld Camera Mingming Fan and Yuanchun Shi Department of Computer Science &Technology, Tsinghua University, P.R. China
[email protected],
[email protected]
Abstract. In the 3D object controlling or virtual space wandering tasks, it is necessary to provide the efficient zoom operation. The common method is using the combination of the mouse and keyboard. This method requires users familiar with the operation which needs much time to practice. This paper presents two methods to recognize the zoom operation by sensing users’ pull and push movement. People only need to hold a camera in hand and when they pull or push hands, our approach will sense the proximity and translate it into the zoom operation in the tasks. By user studies, we have compared different methods’ correct rate and analyzed the factors which will affect the approach’s performance. The results show that our methods are real-time and high accurate.
1 Introduction Many 3D interaction tasks need the zoom operation. Suppose that if we want to wander in the 3D campus, we may need to go ahead to watch the landscapes. In order to satisfy this requirement, we can use the mouse to control the moving direction and the up arrow key to move ahead. The disadvantages of the method are as following. First, the operation needs relatively complex combinations of keyboard shortcuts with mouse movement and clicks. This usually operates with two hands. Second, it gives a low level of naturalness and is not a good choice for the children or people who are not familiar with the keyboard and mouse operations. In order to crack the above two disadvantages, we propose a method that users could simply pull or push their hands to move in or out by holding the camera. When they want to go ahead, they just need push their hands forward. When they would like to go back, they just need pull their hands back. Our approach needs only people’s natural movement and almost need no study. Besides the naturalness, the operation only needs one hand and people may use the other hand for other operations. Some researches [2, 5, 6] have done the familiar studies, such as the Harrison and Dey [2] try to recognize the people proximity by the camera in the computer. However, during their mode, the camera is still and the approach is not proper for the 2D or 3D interaction tasks such as the object control or the virtual space navigation. IsseU [5] is similar to our approach. IseeU tries to calculate the change in the standard deviation of the positions of feature points, which are selected in the image captured by the camera, and transform it into a zooming message. However, we analyze that it is not enough to give a high accuracy. J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 133–140, 2009. © Springer-Verlag Berlin Heidelberg 2009
134
M. Fan and Y. Shi
After having studied the previous works, we first give two methods to recognize the zooming message. After that we test the accuracy rates of them and analyze the factors which have an effect on the accuracy. Then by taking more factors into consideration, such as how to support large distance zoom, we modify the methods to make them more efficient.
2 Framework of the Algorithm The handheld camera is just a tool for interaction. Because the camera is hold steadily in user’s hand, the camera’s movement will reflect the hand’s movement. In order to detect the camera’s movement, first, we detect some corner points in the image frames captured by the handheld camera, then by analyzing the geometric characters of the corner points’ positions, we try to decide whether the movement is zooming or not. During the following part, we propose two methods to detect the zoom and then compare them with each other to see which one is better.
Corner points detecting
Corner points tracking
Zooming detected
Controlling Application
Fig. 1. The whole framework of processing
3 Corner Points Detecting and Tracking Corner-like points [4] which are corners, with big eigenvalues in the image, are easy to find on incoming frames and are relatively stable while being tracked. Tracking the points means finding the new positions of the corner points, which appeared in the last frame, in the frames. Our approach tracks the feature points by implementing sparse iterative version of Lucas-Kanade optical flow in pyramids [1] .
Fig. 2. The green points are the corner points
Pull and Push: Proximity-Aware User Interface for Navigating
135
4 Zoom Detecting Algorithms In this part, we will discuss two algorithms to detect the zoom in detail and then compare their performance. 4.1 Algorithm One: Sensing the Distance As the figure3 shows, A, B, C, D are positions of the corner points in the last frame, the A’, B’, C’ D’ are the positions in the frame. The average distance between the corner points and their centers becomes farther when the camera zooms in, since the distance between the camera and background are shortened. According the above analysis, first, we calculate the positions of the corner points in the last and now frames. Then, calculate the average distances among them and their centers. Finally, calculate the rate of the new distance and old distance. If the rate is over 1.0, it means that the camera pulls back. If the rate is less 1.0, it means that the camera pushes forward. In the real experiment, due to the hand’s jitter, the camera may have the slight movement. In order to reduce the jitter’s interference, we set a threshold to instead of the above number 1.0.
Fig. 3. Corner points’ positions before and after the camera zoom in. o and o’ are the centers of the old corner points in the last frame and new corner points in now frame.
4.2 Algorithm Two: Sensing the Change of the Area As the figure 3 shows, the corner points form a polygon ABCDE. After the zoom in
,
operation, the polygon becomes the A' B 'C ' D ' E ' the area of the polygon ABCDE changes to be larger. Through sensing the change of the area, we can decide whether a zoom in or out happens. 4.3 The Accuracy of the Two Algorithms Participants Seven participants, six male and one female, take part in the test. They use a webcamera with frame rate 30 fps(frames per second) and a pentium 4 PC with the
136
M. Fan and Y. Shi
main frequency 3.2GHz. Each of them takes the experiment for about five minutes. Before the test, they are given no more than five minutes to be familiar with the camera. Experiment We have rendered a 3D virtual space with DirectX 3D(see figure4). People are asked to use the camera to go ahead or back in the virtual space. We count the total decisions and the right decisions, then calculate the accuracy rate. (We ask the testers to do zoom in operation, then we count the total judgment and the actural zoom in times).
Fig. 4. The left one is the last image. When users push the camera forward, our view goes forward and the house becomes bigger than ever.
Accuracy Rate Participants are asked to do the zoom in and out movements to test two algorithms’ accuracy rates. According to the Fitts law [3], the distance between the camera and positions of the corner points in the real world will have an effect on our algorithms. So in order to test how the distance factor affects our algorithms’ performances, we do the experiments at different distances, such as 0.6m, 2~2.5m, 5m. We calculate the seven participants’ results and give the average accuracy rates at different distances in figure 5. Discussion From the figure5, we can conclude that: • The average accuracy rate of the algorithm2, which senses the proximity by calculating the changes of areas, is higher than the algorithm1, which detects the zoom by calculating the changes of distances. • As the distance between the camera and the corner points’ real world positions increase, the accuracy rate declines rapidly. At the distance about five meters, the accuracy rate of algorithm one has been below 50% and the accuracy rate of algorithm two is almost equal 50%. • The zoom detecting algorithms are totally sensitive about the distance. Within the five meters, the algorithm can keep the accuracy rate above 50%. Within the
Pull and Push: Proximity-Aware User Interface for Navigating
137
Fig. 5. The average accuracy rates of seven participants’ results
distance 1~2m, the two algorithms can keep the accuracy rate over 80%. This results guide us that our hands had better push or pull the camera in a direction in which there are some objects with in 1~2m. • The experiments are taken by seven participants who only use the camera to operate less than five minutes. The results show that they can operate the zooming easily and need less time to practice. 4.4 Large Distance Zooming Support Using a Finite State Machine From the questionnaire, they reflect that the approach is not suitable for moving a long distance at one time. If they want to go ahead in the virtual space for a long time, they must keep pushing or pulling the camera for a long time. This is impossible due to users’ moving space is limited. In order to crack this hard nut, we give the strategy that we first detect the movement. If we continuous detect the zoom in movement for two times, then we simply think that users want to zoom in. In this situation, we output the zoom in. if users want to stop zooming in, they can pull back the camera. If our approach detects the zoom out movement for two times, we think that users want to pull back. The whole procedure can be described as a finite state machine (FSM)(Figure 6). During the zoom in / out state, our approach will output the “zoom in/out” decision. Suppose the current state is “zoom in”, and current judgment of the algorithm is “zoom out”, then the counter Count1 will add by one, then we examine whether Count1 is two or not. If the count1 is two, then the state will change to “zoom out” and the output is “zoom out”. But if the count1 is not two, then the state will still be “zoom in” and the output is “zoom in”. if the current state is “zoom in” and the current judgment is “zoom in”, our approach will output “zoom in” and set the count1 as zero. The reason why we use the counters when the state is changing is to make our algorithm stable. Since the camera is held in people’s hand and the hand will be
138
M. Fan and Y. Shi
Zoom in Count1 = 0
Zoom out Count2 = 0 Count2= 2
Zoom in Count2 = 0 Zoom out Count1 ++ && Count1 !=2
Count1= 2
Zoom out Count1 = 0 Zoom in Count2 ++ && Count2 != 2
Fig. 6. The finite state machine of zoom in and zoom out
naturally jitter while suspending in the air. And this jitter maybe causes some zoom in or out motion. If the approach does not use the state machine or two counters, then only a slight noise will cause false decisions. Experiment for Testing the Effect of Finite State Machine Participants and the hardware conditions are the same as the above experiment. We have done two zoom detecting algorithms, one uses the finite state machine and the other does not. All participants have required to do zoom in and out movements alternatively for about five minutes. Each have done the experiments twice, one time is without the finite machine and the other time with the finite machine. The average distance between the corner points’ positions and users’ hands’ positions is about one meter which is good for our algorithm to work. The average accuracy rates are calculated.
Fig. 7. The accuracy rates of the algorithms. One is 0.93, the other is 0.95. The result shows that the FSM improves the algorithm’s performance.
After the experiments, participants give us some valuable feedbacks, based on which we conclude the following • With the finite state machine, they can continuously zoom in or out. While they use for the object controlling, they can magnify or reduce the size of objects. While
Pull and Push: Proximity-Aware User Interface for Navigating
139
using in the virtual space wandering, they can continuously go ahead or back in the scene. • They can switch between the zoom in and zoom out movement with higher accuracy rate. • Before the hand’s motion state changes to the other one, the counter must count to two. Since the frame rate is 30fps, then the delay time is 6.6 milliseconds which is almost real-time to our eyes. By using the finite state machine and the counter, the accuracy rate is improved.
5 Applications As we have claimed that the given proximity-aware algorithm can be used in the object controlling and the virtual space navigation. In the object controlling task, people can magnify or reduce the virtual object by pushing or pulling the camera. The application is shown in Figure8. In the virtual space navigating tasks, the algorithm can be used for going forward or back in the scene which is especially useful for the games(Figure4).
Fig. 8. The left one is the former image of a cube, when the user pushes the camera forward, the cube’s size will increase as is shown in the right picture
6 Conclusions In this paper, we have proposed and compared two proximity-aware algorithms. From the experiment’s results, we conclude that the algorithm two has a better performance. In order to support the large distance zoom and improve the accuracy rate, we take in a finite state machine. Comparing to the traditional mouse and keyboard operation, our methods are much more natural and easier to learn. Our approaches are real-time and have high accuracy rate. Our methods can be used in the object control and virtual space navigating tasks to fulfill the zoom function.
Acknowledgements Specialized Research Fund for the Doctorial Program of Higher Education, China, No.20050003048 and this research is also supported by the Nokia Research Center.
140
M. Fan and Y. Shi
References 1. Bouguet, J.V.: Pyramidal implementation of the Lucas Kanade Feature Tracker Description of the algorithm. Intel. Corporation Microprocessor Research Labs (1999) 2. Harrison, C., Anind, K.D.: Lean and Zoom: Proximity-Aware UserInterface and Content Magnification. In: Proc. CHI, pp. 507–510 (2008) 3. ISO. Ergonomic requirements for office work with visual display terminals (VDTs) Requirements for nonkeyboard input devices. ISO 9241-9 (2000) 4. Shi, J., Tomasi, C.: Good features to track. In: Proc. IEEE Comput. Soc. Conf. Comput. Vision and Pattern Recogn., pp. 593–600 (1994) 5. Sohn, M., Lee, G.: ISeeU: camera-based User interface for a handheld computer. In: ACM MobilCHI 2005, pp. 299–302 (2005) 6. Wang, J., Canny, J.: TinyMotion: Camera Phone Based Interaction Methods. In: Proc. CHI 2006, pp. 339–344 (2006)
A Study on the Design of Voice Navigation of Car Navigation System Chih-Fu Wu, Wan-Fu Huang, and Tung-Chen Wu Graduate School of Industrial Design, Tatung University 40 JhongShan North Road, 3rd Section Taipei 10452, Taiwan
[email protected]
Abstract. This study tries to find the designing blind spots of the voice prompt function in the current car navigation systems and make improvement suggestions. The experimental plan was implemented through videotape analysis of the voice-prompt mode, referring to Urban Road Classification Regulations and the questionnaire survey results. Driving simulation tests were conducted with 15 experimental subjects, 13 road combinations, and 3 running speeds, and different prompt modes which were run synchronously were also included. Compared with the present mode (prompt time is determined by distance.), the newly-designed mode (prompt time is determined by running speed.) significantly improved driving performance and reduced mental workload. When driving on a main artery with fast lanes and slow lanes, adding a changing-lane prompt with a clear sound to the system can help increasing the driving accuracy rate. Keywords: navigation systems, voice prompt function, driving accuracy rate.
1 Introduction Following the development of science and technology, car navigation systems have increasingly been used by a lot of drivers. Car navigation systems provides drivers with information about how to get from one place to another in a turn-by-turn format such as distance to the turn, the name of the street to turn onto and turn direction [1]. However, whether a car navigation system is pre-installed on the dashboard or setup on the windshield of the driver, the driver inevitably needs to move the eyesight from the road ahead to a 3.5 to 8-inch LCD display of the automobile navigation system. Such distraction from the road is one of the main causes of traffic danger [2,3]. Nowadays, car navigation systems provide information through not only monitor displays but also voice prompt messages, so as to reduce the time drivers spend on monitors and reduce the danger of driving. Moreover, certain drawbacks remain in voice prompt function of car navigation systems, which prevent drivers from using them as the only way to receive navigation information. However, noises on the road, chattering voices of the passengers or sounds made by other vehicles, as well as music, radio and other factors may all interfere with navigation information. If voice playbacks are overlapping, it may confuse drivers and cause accidents. Unclear J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 141–150, 2009. © Springer-Verlag Berlin Heidelberg 2009
142
C.-F. Wu, W.-F. Huang, and T.-C. Wu
navigation information may cause accidents, especially when drivers need to switch lanes from one multi-lane road section to another, or to enter special terrains such as tunnels or overpasses. Due to the limitation of information processing, attention needs to be allocated when a person is multitasking, and his/her mental workload is increased as a result. Attention resources of drivers can be categorized into visual resources, operative resources, mental-workload resources, acoustic resources and information ranking for analytical purposes [4]. Drivers depend largely on the visual modality for drivingrelated information [5]. While mental workload varies along with changes in acoustics and operation, several studies claim that if a message is instant, simple and to be reacted immediately after being received, vocal prompt is suggested; if a message is complicated, lengthy and not to be reacted immediately after being received, visual prompt is preferable [6]. Some studies suggest that names of road are not necessary to be provided by voice prompt in navigation system, because these are not easily comprehended by drivers immediately [7]. Given that driving is metal-requisite, the timing of information to be provided should be limited: if the information is provided too early, drivers may forget such information; if the information is provided too late, drivers may not have enough time to execute relevant tasks. Although mobile navigation systems have been increasingly popular, issues with regard to navigation have appeared as well. Foreign or domestic studies in this area, however, are rather scarce. This study, as a result, aims first at pointing out the issues regarding comprehension of displayed information and voice prompt information from currently available car navigation systems through questionnaires to drivers. Further, with driving simulation tests, driving patterns of drivers passing intersections of different kinds of roads are recorded and compared with the results of questionnaires. Finally, through the combination of variables such as monitor displays, driving reminders and timing of voice prompt, further designs and verification are conducted for the study to be applied in the design of voice prompt systems.
2 A Survey of Currently Car Navigation Systems The study started from investigating products on the market. Three types of car navigation system from four large manufacturers were selected, and installed to proper positions above the wind outlet of air-conditioner in a real car. After setting the route, the car was driven on the same way in Taipei city and the playing mode was recorded in video taping for three runs. From the content of the video, the syntax of driving reminders and broadcasting timing with matching road classification principle were analyzed for designing questionnaire. Questionnaires are used in the early analysis so as to understand issues regarding the provision of information to drivers from currently available automobile navigation systems. They are described below. • Sample Setting: Three most representational car navigation systems (Mio CT720,
PaPaGo R12, TomTom ONE ) were selected from the products available in the market. Two CCDs were used to simultaneously record the status on the road ahead and the graphic displayed and voice played by the navigation systems. In total, three rounds of tests were conducted. Grammar analysis is employed based on
A Study on the Design of Voice Navigation of Car Navigation System
143
the abovementioned samples. After the analysis, Mio-Tech’s product is the most complicated one, with driving information provided before a decision point in a pattern of a intersection in front + b please drive close to X + c please turn + d enter Y + e leave Y + f drive Z, in which “a” and “f” are broadcasted before every intersection, “b” is broadcasted when driving straight is required, “c” is broadcasted when a turn needs to be made at an intersection, “d” and “e” mean entering or leaving special roads, such as overpasses or tunnels, are required, “X” means turning left or right, “Y” means special terrains, such as overpasses and tunnels, and “Z” means the name of the name to drive on. The information provided by PaPaGo is less complicated than that by Mio-Tech but in a similar pattern. The difference is that it does not provide names of the roads to drive on (namely, without the item, f drive Z). TomTom uses the simplest way to broadcast, which only presents one simple sentence such as “PLEASE TURN LEFT” or “PLEASE DRIVE STRAIGHT”. • Categorization of and Regulation on Downtown Roads: Based on the analysis of voice prompt functions of the abovementioned mobile navigation systems, navigation scripts do not adapt in accordance with different navigation contents for different roads. Though additional contents are made for special roads such as overpasses and tunnels, they are too limited. In order to fully understand the issues regarding the provision of driving information with mobile navigation systems, roads need to be fully understood. In accordance with Urban Road Classification Regulations relevant regulations in Taiwan [8], according to the property of the road service such as the speed limit, traffic lane number, lane width, traffic control devices, etc. condition, downtown roads may be categorized into expressway, major road, sub road and service road. Combination of roads and questionnaires are used to understand issues regarding comprehensibility of voice prompt of mobile navigation systems. • A survey of questionnaires interview: The information from the navigation system was presented on paper in the questionnaires with voice prompts being played. As for the graphic information, 13 representative graphic images were captured during the actual uses of the current navigation systems. Through cluster analysis with Urban Road Classification Regulations, with road names changed to prevent the effect from subjects’ memories, they were then re-made into the testing images for the questionnaires with CoreDraw. There are 12 questions in 3 categories including awareness, UI preference, and habit. Only those who had used a navigation system before need to answer the habit-related questions. The investigation was carried out for forty-three subjects (20-55 years old, M=28.4) who owned driving licenses and had driving experiences. Fifteen of forty-three subjects had driven an automobile with a car navigation system. From the above analysis, we can summarize the finding as follows: (1) Products of Mio and PaPaGo have better comprehensibility compare to that of TomTom for those who haven’t used car navigation systems to get more detail the drive information. (2)On the roads with fast lanes and slow lanes, starts at about 20~30 meters from intersections, there might be double solid white line as “keep in lane” lane marking to
144
C.-F. Wu, W.-F. Huang, and T.-C. Wu
divide the car flows toward the same direction for the sake of safety. If a right-turn needs to be made at this kind of intersections, drivers must switch to the slow lane on the outside before passing where the double solid white line starts. (3) The voice prompts in the current products are usually played 500 meters before critical intersections. The same content is played four times. The last time begins about 20~30 meters before intersections are also present graphically in Fig. 1. But drivers usually care only about the last prompt and ignore the first three. Therefore there would be actions like braking, slowing down, etc. right after the last prompt.
Fig. 1. Some voice prompt models of the currently available products (on expressway)
3 Driving Simulation Test The key point in the design of human-machine interface is to discuss the interaction relationship among practical users, objects and environment. As to this study, the real experience in car driving was the best in the sense of reality. However, due to lots of factors interfering the road driving and experimental variables not easy to control, simulation driving was carried out to assure the consistency of test conditions in the experiment. Virtual reality scenes were constructed by thirteen combinations of turns in the road that created from the roads classification and regulation on downtown roads. 3.1 Participants A total of 15 graduate students of Tatung University at the age of 20 or more (20-46 years, M=26, SD=3.2) and were paid to participate in the experiment. All participants
A Study on the Design of Voice Navigation of Car Navigation System
145
have valid driving licenses and driving experience. All participants have normal or corrected visual acuity. 3.2 Apparatus In the present study, the simulated vehicle cab, a 1990 FORD TELSTAR car, included all the normal automotive displays and controls found in an automatic vehicle. The simulator used two Pentium IV PCs to control the simulator, scenario respectively and one Notebook to show visual in-car navigation system. The scenario graphics were projected onto a projector screen located about 2 m in front of the driver’s seat to produce 60 *40 degree field of view. The steering, throttle and brake inputs were connected to the PC that controlled the simulator software. The car navigation system is set up on in right-front of the driver, the height and location of which is around the ventilator of the air-conditioning system as the visual display. For a driver of normal height, the in-vehicle display was 15degrees below his/her straight-ahead plane and 20 degrees to the right. A speaker in front of the passenger seat provided auditory information in the form of a digitized human female voice with a speech rate of ~150 words/min and sound effects.
Fig. 2. One scene of driving simulation ware/hardware
&
Structure of virtualization control soft-
GW InstruNet Data Retrieving System is used for transformation and retrieval of data. Data is retrieved and input into Excel and SPSS for further determination. For analyses of the broadcasting timing of navigation systems, speakers are connected with the GW Instruments data retrieving box at Vin+ and Vin-, and hence voltage values vary as sounds are played. 3DSTATE World Editors developed by Morfit was used to generate the virtual environment. Scene data can be recalled by using Visual Basic together with the animation engine of 3D Developer Studio for Visual. Structure of Virtualization control software/hardware is depicted in Fig. 2. 3.3 Experimental Designs Two different tests were assessed. One was driving simulation tests of currently available car navigation system with modifying switch-lane reminder (hereafter referred to
146
C.-F. Wu, W.-F. Huang, and T.-C. Wu
as Test1), and the other was driving simulation tests for newly-design evaluation car navigation system (hereafter referred to as Test2). Test1 was a 3 *2*2*2 mixed-factors model that compared results by speed on the driven road (three levels: 70 km/hr in expressway , 50 km/hr in major road, as well as 20 km/hr in service road ), way of voice prompt (two levels: complicated broadcasting patterns: Mio-Tech, simple broadcasting patterns: TomTom), monitor display(two levels: on, off), reminders of switching lanes (two levels: on, off).When the last factor is set to be “off”, the mode being tested is the currently available voiceprompt mode; when it’s set to be “on”, the mode being tested is a new trial. The way it works is that a clear and short sound will be played before a voice prompt to remind divers paying attention to the voice content which is about to be played. The effect is like that of the broadcasts to look for somebody in the railroad stations. The evaluation on the newly-designed mode (prompt time is determined by running speed instead of prompt time is determined by distance.) was performed in Test2. In this voice-prompt mode, the reaction distance should be [speed (km/h)*2.5second+30] meters. The number of tests was reduced because of the experience from Test1. Only the factors “speed on the driven road” was kept the same as planned. The numbers of levels were reduced to 1 for the two factors “monitor display (one level: on)” and “way of voice prompt (one level: simple broadcasting patterns: TomTom)”. But the number of levels for the factor “reminders of switching lanes” was increased to 3 (three levels: display, sound, voice). The display mode shows a red arrow icon as a guide to switching lanes (to the left or right lane) on the navigation screen. The sound mode is the same as the “on” mode in Test1. The voice mode has voice prompts for switching lanes (to the left or right lane). Dependent variables were based on both objective and subjective measures. Whether it would be successful or not while turning into scheduled road would be regarded as the object evaluation criterion. Subjective measures were obtained using a modified five-point scale NASA TLX workload assessment. 3.4 Procedure First, a ten-minute explanation is provided with regard to the test purpose and procedure and to instruct the subjects on how to manipulate in the driving simulation environments with the wheel, accelerator and brake pedal (those were connected to the serial port of the personal computers). Then the subject entered the test phase where he will receive the 36 navigation scenarios in pre-planned different sequences but were counterbalanced to prevent any learning/order effect. A short break was taken if necessary after each scenario trial. For each navigation test, all subjects were required to drive through the designated routes to reach an identical destination. All participants were told to arrive the destination in accordance with the driving information provided by available mobile navigation models, to drive along the route as accurate as possible, and to maintain the car moving in a controllable speed. Chart of changes in voltage are drawn with Excel based on data retrieved, which are analyzed to determine whether subjects have driven in accordance with the rules of the test. Finally, subject filled out a NASA TLX subjective evaluation questionnaire to describe his/her subjective feelings toward different experimental conditions. It took ~120 min for participants to complete the present study.
A Study on the Design of Voice Navigation of Car Navigation System
147
4 Results Data collect for present study were analyzed by inferential statistics of ANOVAs. The LSD treatment contrast tests were used for post-hoc comparison. Table 1. Test1 Resuls -1 ( Currently Available Products )
Driving Speeds 70km/h Expressway 50km/h Major Road 20km/h Service Road
Broadcasting Patterns Mio TomTom Mio TomTom Mio TomTom
Driving Correctness(%) Monitor Display On Monitor Display Off 13.3% 0.0% 26.7% 0% 40.0% 0.0% 93.3% 26.7% 96.7% 36.7% 100% 46.7% 100% 93.3% 100% 90% 100% 86.7%
Table 1 shows that only 26.7% of drivers can correctly leave the expressway by only listening to the final navigation information 20~30 meters before the event decision point. On major roads, 96.7% of drivers can make turns correctly with the monitor on, and 36.7% with it off. On service roads, whether monitors are on or off, the correctness rates are more than 90%. As TomTom’s navigation contents are shorter and require less time for broadcasting within the same distance, they tend to be fully listened to by drivers compared with those of Mio-Tech’s. Table 2. Test1 Resuls -2 ( Switch-Lane Reminding with sound effect)
Driving Speeds 70km/h Expressway
50km/h Major Road
20km/h Service Road
Switch-Lane Reminding Sound On Off On Off On Off
Driving Correctness 78.3% 68.3% 83.3% 58.3% 80.0% 85.0%
Chi-Square Inspection Value DOF Sig. 1.534
1
0.215
9.076
1
0.003**
0.519
1
0.472
From Table 2, it shows that when driving on major roads with median strips, driving correctness rate is 83.3% with switch-lane reminders and 58.3% without such, while “p” is less than 0.05 under the chi-square test. Table 2 shows descriptive statistics and a chi-square inspection sheet of driving with different broadcasting patterns. There is neither significant difference between these two broadcasting patterns, nor between whether the monitor display is on or not. Two major disadvantage of currently available car navigation systems based on the conclusion from early analysis: (1) The timing of final voice prompt on function
148
C.-F. Wu, W.-F. Huang, and T.-C. Wu
cannot be changed based on the car speed. As a result, when the driving speed is rather high, navigation information may not be fully broadcasted, and driving action may not be made in time by drivers. (2) It would result in the same problem if drivers don’t fully understand the switch-lane prompt from the navigation system when driving. But when using reminding sounds on some sections of the roads, as they may be misunderstood as “PLEASE MARK A TURN”, when no switch-lane reminder is provided, driving correctness is higher, nevertheless. Table 3. Test2 Results (Switch-Lane Reminding with Different Ways)
Driving Speeds 70km/h Expressway 50km/h Major Road 20km/h Service Road
Switch-Lane Reminding Sound Voice Display Sound Voice Display Sound Voice Display
Driving Correctness 93.3% 100% 73.3% 86.7% 100% 53.3% 73.3% 80.0% 93.3%
Chi-Square Inspection Value DOF Sig. 5.850
2
0.05*
10.833
2
0.004**
2.218
2
0.345
Table 3 shows descriptive statistic data of correctness on three kinds of tested roads. Correctness rates on the three kinds of roads are generally the same, while different driving reminders contribute to changes in correctness rate. Based on the significance derived with chi-square tests, on expressways and major roads, three different kinds of reminders affect driving correctness significantly, while there is no significant difference on service roads. Subjective measure Subjective grading items include quality of information provision, suitability of broadcasting timing, necessity of monitor display and comprehensibility of information. Through single-variable analyses of variance, it shows that broadcasting patterns designed by this study are better than those of the currently available products and can reduce drivers’ mental workload.
5 Discussion and Conclusions During driving and navigating, drivers have to monitor the car by searching the environmental information and shifting attention from one information source to another [10,11], and drivers depend largely on the visual modality for driving-related information [5]. According to the multiple resource theory, in a heavily loaded visual display environment, an auditory display will improve time-sharing performance [12]. When driving conditions and information are complicated, drivers may have more difficulty in filtering and remembering useful information presented by an auditory display because of the memory interference problem [13]. Similar results were found in the present study.
A Study on the Design of Voice Navigation of Car Navigation System
149
Although voice information aims at reducing the occasions in which drivers move their eyesight from the road, deficiencies in voice navigation functions may nevertheless become a safety concern. However, drivers may have difficulty in paying attention to the auditory display all the time. The timing of the appearance of prompt messages is a keypoint. Four of these findings are worth summarizing: • Prompting Issues of Voice Navigation Systems: Insufficient prompted information and unsuitable timing of prompting result in misunderstanding of voice navigation information. Unclear information, such as not reminding drivers to switch lanes, leads to driving actions not being accomplished when event decision points, such as turning and switching lanes, are passed. In terms of prompting timing, drivers usually neglect the first three prompted contents before the decision, while the final information is not prompted based on driving speeds and may not be heard by drivers in time. • Switch-Lane Reminder : With regard to reminding sounds for switching lanes: (1) On major roads, such sounds can effectively improve driving correctness (p<0.05), with voice reminders being the best one (100%), sound reminders the second (93.3%) and display reminders the third (73.3%); (2) On express ways, although there is no significant difference, driving correctness is still improved (from 68.3% to 78.3%), with voice reminders being the best one (100%), sound reminders the second (86.7%) and display reminders the third (53.3%); (3) On service roads, there is no significant difference, with display reminders being the best one (93.3%), voice reminders the second (80.0%) and sound reminders the third (73.3%). • Timing of prompt: According to the voice prompt pattern in the tests conducted, the distance before event decision point should be [speed × 2.5 seconds] meters for the broadcasting to be finished. According to the study [9,14], 2.5 seconds may be further reduced to 1.93 seconds due to reaction time in acoustic transmission is shorter than that in visual one. As switching lines is prohibited within 30 meters from intersections in downtown areas, for major roads and expressways, the reaction distance should be [speed × 2.5 seconds + 30] meters. • Further Suggestions: A proper amount of navigation information and its optimization have not been discussed in this study. In terms of tests conducted, interfering events and other vehicles are not simulated for the driving environment.
Acknowledgements This research was supported by the National Science Council under contract number NSC 97-2221-E-036-032-MY2, Taiwan(R.O.C).
References 1. Perez, W.A., Mast, T.M.: Human factors and advanced traveller information systems (ATIS). In: Proceedings of the Human Factors Society 36th Annual Meeting, pp. 1073– 1077. Human Factors Society, Santa Monica (1992) 2. Wierwille, W.W.: Development of an initial model relating driving in-vehicle visual demands to accident rate. In: Third Annual Mid-Atlantic Human Factors Conference Proceedings. Virginia Polytechnic Institute and State University, Blacksburg (1995)
150
C.-F. Wu, W.-F. Huang, and T.-C. Wu
3. French, R.L.: In-vehicle navigation-status and safety impacts. Technical Papers from ITE’s 1990,1989, and 1988 conference (1990) 4. Wierwille, W.W., Hulse, M.C., Fischer, T.I., Dingus, T.A.: Visual adaptation of the driver to high-demand driving situation while navigating with an in-car navigation system. In: Vision in Vehicles III, pp. 79–87. Elsevier Press, Amsterdam (1991) 5. Lansdown, T.C.: Visual allocation and the availability of driver information. In: Gothengater, T., Carbonell, E. (eds.) Traffic & Transport Psychology: Theory and Application, pp. 215–223. Pergamon Press, Amsterdam (1997) 6. Sanders, M.S., McCormick, E.J.: Human Factors In Engineering and Design, 7th edn. McGraw Hill Press, Singapore (1993) 7. Rasker, P.C., Post, W.M., Schraagen, J.M.C.: Effects of two types of intra-team feedback on developing a shared mental model in command & control teams. Ergonomics 43, 1167– 1189 (2000) 8. Construction and planning Adency Ministry of the Interior(CPAMI) Information, http://www.cpami.gov.tw/web/index.php 9. McGehee, D.V., Mazzae, E.N., Scoot, G.H.: Baldin: Driver Reaction Time in Crash Avoidance Research:Validation of a Driving Simulator Study on a Test Track. In: Proceedings of the IEA 2000/HFES 2000 Conference, vol. 3 (2002) 10. Dewar, R.E.: In-vehicle information and driveroverload. International Journal of Vehicle Design 9, 557–564 (1988) 11. Dingus, T.A., Hulse, M.C.: Some human factors design issues and recommendations for automobile navigation information systems. TransportationResearch 1, 119–131 (1993) 12. Wickens, C.D., Sandry, D., Vidulich, M.: Compatibility and resource competition between modalities of input, central processing, and output. Human Factors 25, 227–248 (1983) 13. Liu, Y.-C.: Comparative study of the eVects of auditory, visual and multimodality displays on drivers’ performance in advanced traveller information systems. Ergonomics 44(4), 425–442 (2001) 14. Oglesby, C.H., Hicks, R.G.: Highway Engineering. John Wiley & Sons, Chichester (1982)
Front Environment Recognition of Personal Vehicle Using the Image Sensor and Acceleration Sensors for Everyday Computing Takahiro Matsui, Takeshi Imanaka, and Yasuyuki Kono Graduate School of Science and Technology, Kwansei Gakuin University, 2-1 Gakuen, Sanda, Hyogo, Japan {acl27598,bbv84420,kono}@kwansei.ac.jp
Abstract. In this research, we propose the method for detecting moving objects in front of the Segway by detecting running state for the Segway. Running state of the personal vehicle Segway is detected with both an image sensor and an acceleration sensor mounted on the Segway. When objects are moving in front of the Segway, the image sensor can capture the motion while the acceleration sensor shows a different result. By analyzing the difference our method successfully recognizes moving objects from environment. Keywords: Segway, Image Sensor, Acceleration Sensor, Optical Flow.
1 Introduction This research examines the safety and comfort of the personal vehicle Segway [1] which uses electrical energy instead of gasoline in everyday environment. Unlike other vehicles such as cars and bicycle, the Segway can be used to run not only on the road in outdoors, but also indoors such as inside the buildings or factories shown in Fig. 1 right. Safety must be especially considered for the Segway when driving in such place, since they have high pedestrian traffic. To improve the safety of the Segway, a system to notify the danger to the rider and avoid the incident automatically is necessary. This paper proposes the method to detect moving objects in front of the Segway. Rider can easily recognize static obstacles and avoid them, but it is difficult to avoid dynamic obstacles such as unexpected pedestrian and other personal vehicles. Thus, if a system can recognize these obstacles and notify the riders or avoid them automatically, safety of the Segway can be assured. Most of the researches which use vehicle video system to recognize the front environments are instead for use of cars, and are not applicable for slow vehicle like the Segway, which requires to recognize dynamic obstacles in close range. Moreover, the operation of the Segway is different from conventional vehicles in the way that has no acceleration pedal and no brakes. A rider stands on the footplate between two wheels to adjust speed, movement, and direction of the Segway. Vehicles other than the Segway do not have these features. J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 151–158, 2009. © Springer-Verlag Berlin Heidelberg 2009
152
T. Matsui, T. Imanaka, and Y. Kono
Fig. 1. A difference between a car and the Segway
Our proposed method detects the running state of Segway by employing both an image sensor(camera) aimed forward and an acceleration sensor. By integrating sensor readings, our method detects moving objects in the frontal direction. In this research, we focus on the tilting of the handle bar while driving the Segway. From the image sensor mounted on the handle bar, the system analyzes the video captured from it and detects the optical flow [2,3] to obtain the running state among the following six states: forward, backwards, right turn, left turn, acceleration and deceleration. When a moving object such as a walking person is within the range of the image sensor in front of the Segway, it affects the trend of the optical flow, although the readings from acceleration sensors are not affected by the object. Using these data obtained from both sensors, we analyze whether the moving object actually exists within the certain range which the incompatible information has been detected. This method will contribute for safety in the future, notifying the riders to avoid the moving obstacles in front of the Segway.
2 Research Background 2.1 Related Works Many of related works usually mount the image sensor on the car. Although the posture of the camera is overall stable in these works, that on the Segway changes depending on the running state, i.e., the rider’s center of gravity. For example, when the rider wants to turn right, he tilts the handle bar to the right, and the image sensor also tilts to the right. The rider leans forward to gain speed, so the handle bar together with the camera also leans forward. It is necessary to analyze the running state of Segway by detecting the tilt of the image sensor which is affected by the change of the Segway posture.
Front Environment Recognition of Personal Vehicle
153
A method for detecting three-dimensional position of moving objects with an invehicle camera is proposed [4]. The method in [4] is not suitable to the Segway because the posture of the camera is not stable while driving. Another approach is to detect approaching vehicles by classifying horizontal edges in an image into two classes; ones on the ground and one above the ground [5]. The method in [5] is also difficult to apply to complicated everyday scenes. 2.2 Preparation The image sensor is mounted on the handle bar as indicated in Fig. 2(A). The acceleration sensor is mounted on the footplate. The image sensor is Glasshopper of Point Grey Research Inc shown in Fig. 3. The acceleration sensor is WAA-001 of Wireless Technologies Inc shown in Fig. 4. The specification of the sensors is shown in Table 1 and 2 [6] [7].
Fig. 2. Placement of sensors on the Segway
Fig. 3. Grasshopper
Fig. 4. WAA-001
154
T. Matsui, T. Imanaka, and Y. Kono Table 1. Spec of the Grasshopper
Table 2. Spec of the WAA-001
3 Moving Object Detection by Sensor Fusion In this research, the image sensor is mounted on the handle bar of the Segway and the acceleration sensor is mounted on the footplate between two wheels. Running states of the Segway are detected from these sensors, and moving objects are recognized from the discrepancy of the detection results.
Fig. 5. Optical flows change when a moving object is recognized
The image sensor detects and analyzes optical flow patterns of each running states while the acceleration sensor measures the acceleration of the Segway to determine the running states. The optical flow shows the movement of feature points between
Front Environment Recognition of Personal Vehicle
155
video frames expressed as vector. Considering the running state of the Segway as moving forward, when the image sensor detects no moving objects, the optical flow displays as shown in Fig. 5 left. When a moving object is within the field of view, the optical flow is disrupted as shown in Fig. 5 right. On the other hand, the moving object has no influence on the detection results of the acceleration sensor, which measures the acceleration of the Segway for detection of the running states.
4 Running State Detection Using Each Sensor 4.1 Posture Change and Running State of the Segway The method to detect the running state is described here, namely; forwards, backwards, right turn, left turn, acceleration and deceleration.
Fig. 6. Forward
Fig. 9. Eceleration
Fig. 7. Backwards
Fig. 10. Right turn
Fig. 8. Acceleration
Fig. 11. Left turn
Details of optical flow pattern for each running state is described as follows. In forward running state, a vanishing point and optical flows appear as shown in Fig. 6. In backwards running state, a vanishing point and optical flows appear as shown in Fig. 7. In accelerating running state, optical flows appear as shown in Fig. 8. The flows rise from the bottom to the top as the handle bar leans forward. The camera is faced downwards and a range of the image sensor moves to bottom. In deceleration running state, optical flows appear as shown in Fig. 9. The flows descend from top to bottom as the handle bar is pulled back, and the camera is faced upward. In addition, after the Segway has decelerated, note that the posture of the Segway returns to the upright position. In turning right running state, optical flows appear as shown in Fig. 10. In turning left running state, optical flows appear as shown in Fig. 11.
156
T. Matsui, T. Imanaka, and Y. Kono
4.2 Running States Detection Using the Acceleration Sensors When running state of the Segway changes, the acceleration sensor data change characteristically. Fig. 13 shows one example of the acceleration data. In addition, the line X shows the acceleration that is traveling direction of the Segway, and the line Y shows the acceleration that is horizontal direction and vertical to X axis against traveling direction of the Segway shown in Fig. 12. The acceleration data change remarkably when the running state changes. For example, when the vehicle is running forward, the line X does not show visible changes, but, large values are obvious along the positive direction when moving the vehicle from an idling state and along the negative direction during acceleration. Also, in line Y, large values are obvious along the positive direction when turning right, and along the negative direction when turning left. Likewise, by detecting the significant changes in the acceleration data, we can distinguish the running states from the image sensor. Fig. 13 shows readings from the acceleration sensor starting from the idling state, turning right, moving forward for a while, and then turning left.
Fig. 12. Three axial directions
Fig. 13. An example of an acceleration sensors data
Front Environment Recognition of Personal Vehicle
157
4.3 Integrating to Sensors Information This section compares image sensor data and acceleration sensor data obtained simultaneously while riding the Segway. Fig. 14 shows the readings while the Segway is moving forward. When no moving objects are in the sight, the image sensor simply detects the optical flow for “moving forward” as explained in section 4.1. No remarkable changes can be verified from the acceleration sensor data. In contrast, Fig. 15 shows the case when the moving object exists in front of the Segway. In this case, the optical flow shows different trends from that in Fig. 14, although the acceleration sensor obtains similar data with Fig. 14. The moving object which affects the optical flows can be detected by examining the moment when such discrepancy occurs. Fig. 16 and 17 show examples while the vehicle is turning right. In this case, acceleration sensor data shows the same trend, although optical flow shows different trends.
Fig. 14. Two sensors data when a moving object does not appear (moving forward)
Fig. 15. Two sensors data when a moving object appears (moving forward)
Fig. 16. Two sensors data when a moving object does not reflect (turning right)
158
T. Matsui, T. Imanaka, and Y. Kono
Fig. 17. Two sensors data when a moving object does not reflect (turning right)
5 Conclusion This paper described our method to detect front moving objects by using the incompatibility between readings from an image sensor and an acceleration sensor. Moreover, the detection method of running state using two sensors to detect a moving object is described. As future work, it is necessary to examine whether the safety of the personal vehicle can be achieved with our method.
References 1. Segway, http://www.segway.com 2. Horn, B.K.P., Schunck, B.G.: Determining Optical Flow. Artifical Intelligence 17, 185–204 (1981) 3. Bouguet, J.Y.: Pyramidal Implementation of the Lucas Kanade Feature Tracker, Intel. Corporation, Microprocessor Research Labs (2000) 4. Onoguchi, K.: Shadow Elimination Method for Moving Object Detection. In: Proceedings of International Conference on Pattern Recognition (ICPR 1998), pp. 583–587 (1998) 5. Okada, R., et al.: Obstacle Detection using Progressive Invariant and Vanishing Lines. In: ICCV, pp. 330–337 (2003) 6. Point Grey Research, http://www.ptgrey.com/products/grasshopper/index.asp 7. ATR-Promotions, http://www.atr-p.com/sensor01.html
Common Interaction Schemes for In-Vehicle User-Interfaces Simon Nestler, Marcus Tönnis, and Gudrun Klinker Fachgebiet Augmented Reality Technische Universität München Fakultät für Informatik Boltzmannstraße 3, 85748 Garching bei München, Germany {nestler,toennis,klinker}@in.tum.de
Abstract. In this paper different interaction schemes which are currently implemented by major automotive manufacturers have been identified and analyzed. Complete overviews on all in-vehicle user-interface concepts are rarely spread. This paper gives a deeper insight in interaction schemes and userinterface concepts which are implemented in current cars. Additionally an expert review with 7 experts was performed to get a first impression which userinterface interaction schemes work well in the in-vehicle context. In order to get an impression of the suitability of the interaction schemes for the development of usable in-vehicle user-interfaces we performed different tests. The results are reported in text and tables. Keywords: User Interface Design, In-vehicle information systems, IVIS.
1 Introduction The variety of driver information systems (IVIS) continuously increases. Integration of IVIS into a consistent and integrated human centered interaction concept gets more and more important. We have identified and analyzed different human computer interaction interaction schemes which are currently implemented by major automotive manufacturers. The identification of these HCI interaction schemes and the determination of their suitability for the use in cars is the basis for future design and development of intuitive and easily learnable user-interfaces in cars. The identification of interaction schemes and the evaluation of their suitability for the use in cars were performed in a two-stage expert review: Whereas in the first step the general usability was tested (with SUS / SEA), in the second step the usability was tested more specifically (with HMI). This paper starts with an overview on related work in the field of in-vehicle userinterface concepts. Afterwards it gives an overview on different interaction schemes which are implemented in current automobiles. Finally the evaluation and the evaluation results are presented. J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2009, LNCS 5612, pp. 159–168, 2009. © Springer-Verlag Berlin Heidelberg 2009
160
S. Nestler, M. Tönnis, and G. Klinker
2 Related Work Ablassmeier et al. proposed new search techniques for in-car interfaces [1]. Their proposed search agent has a high potential to increase the concentration on the primary driving task. Burnett et al. focus on the usability of car navigation systems and give a comprehensive overview on the issues concerning human-machine interfaces [2]. Research on in-vehicle information systems quite often is limited to the presentation of navigation information and warnings [5]. Burnett et al. identified a rapid growth of interest in the development and utilization of tactile interfaces in cars [3]. In their opinion the human skin surface offers an important means for presenting information to the users, even if their other senses may already be overloaded. Finally they summarized the arguments for and against allowing drivers to enter a destination with a vehicle system while driving. In their opinion the inhibition of this functionality whilst being onthe-move is not an ideal solution. Consequently the research in user interfaces and human factors has to investigate the potential of novel in-car user-interfaces [4]. Another field of in-vehicle human computer interaction research are considerations of driver distraction. The risk caused by usage of mobile devices is commonly taken into account [6]. Although this study describes different types of driving distractions caused by mobile devices, the distraction by in-vehicle information systems is analyzed rather limited. Cell-phone dialling tasks in cars have been analyzed by Salvucci as well [16]. He generated a model for a priori predictions of total times for different tasks. Stevens et al. published a checklist for the assessment of in-vehicle systems, which contains a questionnaire, instructions and additional supporting information [17, 18]. Nevertheless it remains unclear whether following the proposed procedures leads to systems with better design and supports the identification of design errors. Additionally the European Commission published a statement on the user-interface design principles for in-vehicle information and communication systems [7]. When focussing on the in-vehicle user interface design additional related work has to be considered as well. Green et al. proposed design guidelines for driver information systems by establishing the resumption lag as a factor in predicting an IVIS-style task time [8, 9]. Particular design handbooks such as the European HARDIE report from Ross et al. contain guidelines how information should be to the car driver [13]. Ito et al. analyzed eyes-off-the-road times which are caused by the manipulation of IVIS [11]. Maximum eyes-off times measured in the evaluation were between 4s and 5s. Libuda presents an example of the potential of multimedia user interfaces [10]. He describes the development of an in-vehicle user interface with different input options: language, manual mode and signs. The in-vehicle presentation of navigational information was analyzed by Narzt et al. They found, that typically either a flat arrow or a virtual bird eye view is used best for visualizing the current position [12]. Presenting information on Head-Up displays (HUD) leads to reduced access costs and increased time with eyes on the road [19]. This way of information presentation can improve the detection of objects in the outside world, lane tracking and velocity control [20].
Common Interaction Schemes for In-Vehicle User-Interfaces
161
3 Interaction Schemes The overview on related work shows that a lot of research has been performed in the field of driver distraction, display technologies and in-vehicle applications. Publications which give a deeper insight into interaction schemes are rarely spread. It is difficult to get a broader overview on user-interface concepts which are implemented in current cars. The underlying interaction schemes play an important role in the estimation of the usability and performance of the in-vehicle user-interface. We identified different interaction schemes when taking a closer look at seven different in-vehicle user-interfaces: A, B, C, F, L, M and T. As a result of this analysis, several interaction schemes were identified. These are: integrated interaction, logical connections, information distribution, information presentation in the HUD, menu manipulation, short cuts, independent state transitions. 3.1 Integrated Interaction Concept Two different interaction schemes regarding the general interaction concept exist in current cars. In most cars interaction is based on a central multi-functional controller with equal functionality (rotating, shifting and pressing) as shown in Figure 1. In some cars, however, interaction bases on touch screen devices. Both of these interaction concepts show advantages and disadvantages when used in a car. Due to considerations of driver distraction the functionality of touch screen devices is reduced during motion, while the controller based concepts offer the full functionality even during motion. Furthermore the hard-keys beside the central controllers differ significantly between the different manufacturers. Whereas the central controller concept at userinterface B bases on a controller which can be pushed pressed and rotated, the userinterface A includes many additional hard keys. A composition of these into controller concepts can be found at user-interface M as shown in Figure 1.
Fig. 1. The integrated interaction concepts at user-interface B (left), user-interface M (middle) and user-interface A (right) base on a multi-functional controller and one or more additional buttons
3.2 Logical Connections The existence or absence of a logical connection between the central information display (CID) and the digital instrument panel (DIP) are two further interaction schemes which are implemented in current user-interfaces. Some concepts connect the CID and DIP logically as shown in Figure 2a, the modification of the CID's system state
162
S. Nestler, M. Tönnis, and G. Klinker
(a)
Fig. 2. (a).The digital instrument panel (left) and the central information display (right) are connected logically at user-interface F. (b).The digital instrument panel (left) and the central information display (right) are not connected logically at user-interface M.
changes the DIP's system state and vice versa. Most current cars, however, use two completely independent system states as shown in Figure 2b. 3.3 Information Distribution The information distribution across the different display areas in the cockpit follows two different user-interface interaction schemes. The first interaction scheme includes the equal distribution of all information on all available displays, typically CID and DIP. In the second interaction scheme one display is the dominant while the other display only provides sparse information as shown in Figure 3. The CID as well as the DIP is used as the central display in current user-interface concepts for cars.
Fig. 3. In the digital instrument panel (left) very sparse information is presented whereas in the central information display (right) extensive information is available at user-interface T
Common Interaction Schemes for In-Vehicle User-Interfaces
163
3.4 Information Presentation in the HUD The introduction of HUDs for cars leads to new interaction schemes for the information distribution between the DIP and HUD. One interaction scheme is the redundant visualization of the most relevant information in the DIP and in the HUD. The relevant information is not distributed between these two displays; it is duplicated as shown in Figure 4. The other interaction scheme contains the consistent distribution of information which leads to the removal of information from the DIP.
Fig. 4. In the head-up display at user-interface B (left) more information is presented than in the head-up display at user-interface C (right)
The concept of user-interface B contains the presentation of important detailed information; the concept at user-interface C is limited to speed information and a rather schematic navigation hint. Due to the fact that the information at user-interface B is quite detailed the driver has to look on the DIP less frequently. 3.5 Menu Manipulation Two different interaction schemes for the manipulation of menus exist. In some cars the main menu is realized completely in software. No hard keys for the direct access of menu items are available. In the contrary interaction scheme all items in the main menu are represented by hard keys, the main menu is built in hardware.
Fig. 5. In user-interface A the main menu is represented by hard keys (left), in user-interface B (middle) the main menu is completely realized in software, and in user-interface T (right) the main menu is represented by hard keys at the side of the screen (dark buttons) as well as on the screen (light buttons)
164
S. Nestler, M. Tönnis, and G. Klinker
In some current cars a combination of these two interaction schemes exists, the menu items can be selected by a soft menu as well as by a hard menu. Figure 5 shows the different menu types; User-interface A uses a menu with hard keys, user-interface B uses a soft menu and user-interface T uses a combination of both principles. 3.6 Short Cuts Short cuts are a common interaction scheme for accessing frequently used functionality more easily. Two different interaction schemes are implemented in current cars. Some cars are equipped with a large number of controls for the direct access of frequently used functionalities. Other cars, however, use short cut controls which can be defined freely by the user as shown in Figure 6.
Fig. 6. In the shortcut concept at user-interface C a large number of controls for direct access is provided (left), whereas in the dynamic shortcut concept at user-interface B (right) only a limited amount of dynamic shortcuts is available
3.7 Independent State Transitions Distraction of the driver's attention is always an issue when discussing different interactive schemes for user-interfaces in cars. Especially common interaction schemes such as returning to a neutral system state have an influence on the distraction of the driver's attention. In some cars the system returns after certain timeout to the initial state if no further interaction has been performed, as shown in Figure 7. Consequently this concept
Fig. 7. The user-interface F automatically switches from the menu (left) to the initial state after a certain timeout (right)
Common Interaction Schemes for In-Vehicle User-Interfaces
165
forces the driver to perform his task quite fast – if he wants to continue his task at the same point at which he was interrupted. Besides an increase of the interaction times the distraction of the driver might increase as well.
4 Evaluation Ross et al. give recommendations for the evaluation of IVIS. They state that smallscale expert evaluations in a task-based context lead to good results [14]. Since a rigorous and comprehensive evaluation of technology is quite expensive, the proposed method was used in our evaluation as well. In our expert-review a group of 7 experts evaluated seven different user-interfaces: A, B, C, F, L, M and T. First of all we tried to get an impression of the usability of the different in-vehicle user-interface with the SUS (system usability scale) test [21]. The results of the SUS are shown in Figure 8 / left. The small sample size made it impossible to interpret the result statistically. We used this SUS test to get an initial estimation of the usability without explicitly distinguishing between usable and less usable in-vehicle interfaces. The test revealed, however, that there is still room for improvement in all user-interfaces, because none of the user-interface is clearly above 50 points (which means that the users could not decide between the two antipoles userinterface is highly usable and user-interface is not usable at all). The workload was measured by the SEA test [22]. The users performed two tasks: manipulation of the radio (Figure 8 / middle) and of the navigation system (Figure 8 / right). Again these tests reveal room for improvement; whereas the workload of the radio manipulation (selecting a radio station and changing the volume) was not very high, the manipulation of the navigation system (entering a destination) was rather high.
Fig. 8. The usability of the different user-interfaces (left) was evaluated by the SUS-Test, the cognitive workload of the radio manipulation (middle) and the manipulation of the navigation system (right) were evaluated by the SEA-Test
To get an impression of the suitability of the different interaction schemes in the in-vehicle user-interface context, we performed a more specific HMI test from [15]. Whereas SUS and SEA are general tests for the evaluation user-interfaces, the HMItest focuses on user-interfaces in vehicles. This HMI test is suitable to evaluate
166
S. Nestler, M. Tönnis, and G. Klinker
numerous aspects of in-vehicle user-interfaces. We selected the aspects, which are connected with the identified interaction schemes (regarding form and content): occlusion, visiblity, grouping, shortcuts, overview, layout, design, consistency and cancelling. Some of these aspects were evaluated separately for each display (CID, DIP, HUD). The user had to rate each attribute of the user-interface from 1 (very poor) to 5 (very good). Table 1. In the HMI test we focused on occlusion, visiblity, grouping, shortcuts, overview, layout, design, consistency and cancelling. The users rated from 1 (very poor) to 5 (very good). The first value shows the mean, the second value in brackets shows the standard deviation.
5 Results and Discussion When taking a closer look at the lowest and highest values in each category in Table 1 (these values are written bold), we were able to connect these results with the interaction schemes identified above. The occlusions occurred most often at user-interface T (touch screen concept) whereas they occurred quite seldom at user-interface C (central controller concept). A large number of hard keys made it quite difficult for the driver to see the right key and reach it easily. Consequently the visibility and accessibility of the hard keys in the user-interface C (static shortcut concept) was rated rather poor, the visibility and accessibility of user-interface B (dynamic shortcut concept) was rated best. The spatial grouping of the two displays and the logical connection of DIP and CID was rated poor for user-interface F. The cars with no logical connection of the displays (userinterface A and M) were rated best regarding the logical and spatial connection. The shortcut concept at user-interface C (a hard key for every function) was rated quite poor, whereas the shortcut concept at user-interface A (a hard key for each menu) was rated best. The overview was best at user-interface M, which uses a balanced
Common Interaction Schemes for In-Vehicle User-Interfaces
167
information distribution between CID and DIP. With user-interface T the users lost the overview, caused by the combination of a rather sparse DIP and an overloaded CID. Deducing functionality and way of interaction from the design worked best at user-interface B (single controller) and worst at user-interface C (many different controllers with different functionality and way of interaction). The information presentation in the HUD was rated best at user-interface B (detailed information in the HUD) as opposed to the HUD at user-interface C which offers only limited information (speed and navigation hints). The user-interface T was rated poor regarding the similar screen layouts, which is a consequence of the redundant menu structure (soft menu and hard key menu). The menu manipulations on basis of hard keys (user-interface A) and on basis of a soft menu (user-interface B) were both rated equally well. Cancelling settings was most difficult with user-interface F where independent state transitions are performed after a certain timeout. User-interface A contained the best concept for aborting interactions by using a hard key for the cancelling of interactions. The analysis of the questionnaire revealed the differences between the HCI models in the different cars. The questionnaire then qualified the different models by showing advantages and disadvantages in interaction. Through the analysis we have the opportunity to identify how different HCI models fit together and can get integrated into one common concept to reduce diversity and complexity of HCI in cars. These results enable research to gain insight in opportunities for highly combined and integrated IVIS systems for optimized driver workload and preference.
Acknowledgements This work was performed in collaboration with BMW Forschung & Technik GmbH. The authors would like to thank Dr. Klaus-Josef Bengler and Mariana Rakic for their support in the identification of the interaction schemes. Furthermore we would like to thank Kerstin Sommer for her support in the design of the different questionnaires. Finally we thank Tony Poitschke for his support in the examination of the evaluation results.
References 1. Ablassmeier, M., Poitschke, T., Rigoll, G.: A new approach of a context-adaptive search agent for automotive environments. In: CHI 2006 extended abstracts on Human factors in computing systems, pp. 1613–1618 (2006) 2. Burnett, G.E.: Usable vehicle navigation systems: are we there yet? In: Proceedings of Vehicle Electronics Systems 2000, European Conference and Exhibition, Leatherhead, UK (2000) 3. Burnett, G.E., Porter, J.M.: Ubiquitous computing within cars: designing controls for nonvisual use. International Journal Human-Computer Studies 55, 521–531 (2001) 4. Burnett, G.E., Summerskill, S.J., Porter, J.M.: On-the-move destination entry for vehicle navigation systems: Unsafe by any means? Behaviour & Information Technology 23(4), 265–272 (2004) 5. Campbell, J.L., Richman, J.B., Carney, C., Lee, J.D.: In-vehicle display icons and other information elements. Guidelines, Federal Highway Administration, vol. I (2004)
168
S. Nestler, M. Tönnis, and G. Klinker
6. Chittaro, L., De Marco, L.: Driver Distraction Caused by Mobile Devices: Studying and Reducing Safety Risks. In: Proceedings 1st International Workshop Mobile Technologies and Health: Benefits and Risks (2004) 7. Godthelp, H., Haller, R., Hartemann, F., Hallen, A., Pfafferott, I., Stevens, A.: European Statement of Principles on Human Machine Interface for In-Vehicle Information and Communication Systems (May 1998) 8. Green, P.: Estimating Compliance with the 15-Second Rule for Driver-Interface Usability and Safety. In: Human Factors and Ergonomics Society Annual Meeting Proceedings, Surface Transportation, pp. 987–991 (1999) 9. Green, P., Levison, W., Paelke, G., Serafin, C.: Preliminary human factors design guidelines for driver information systems, Tech report: FHWA-RD-94-087. US Government Printing Office, Washington, DC (1995) 10. Libuda, L.: Improving Clarification Dialogs in Speech Command Systems with the Help of User Modeling: A Conceptualization for an In-Car User Interface. In: GI-Workshop ABIS-Adaptivität und Benutzermodellierung in interaktiven Softwaresystemen (2001) 11. Ito, T., Miki, Y.: Japan’s safety guideline on in-vehicle display systems. In: Proceedings of the Fourth ITS World Congress, Brussels, Belgium, VERTIS (1997) 12. Narzt, W., Pomberger, G., Ferscha, A., Kolb, D., Müller, R., Wieghardt, J., Hörtner, H., Lindinger, C.: A New Visualization Concept for Navigation Systems. In: Stary, C., Stephanidis, C. (eds.) UI4ALL 2004. LNCS, vol. 3196, pp. 440–451. Springer, Heidelberg (2004) 13. Ross, T., Vaughan, G., Engert, A., Peters, H., Burnett, G., May, A.: Human factors design guidelines for information presentation by route guidance and navigation systems. CEC DRIVE II Project V2008 HARDIE, Deliverable 19, May 1995, 79 p. (1995) 14. Ross, T., Burnett, G.: Evaluating the human-machine interface to vehicle navigation systems as an example of ubiquitous computing. International Journal of Human-Computer Studies 55(4), 661–674 (2001) 15. Rottner, R.: Entwicklung einer praxistauglichen Methode zur Relativbewertung von modernen Anzeige- und Bedienkonzepten im Kraftfahrzeug, Diploma Thesis, Technische Universität München (2002) 16. Salvucci, D.D.: Predicting the effects of in-car interface use on driver performance: an integrated model approach. International Journal of Human-Computer Studies 55(1), 85–107 (2001) 17. Stevens, A., Board, P.A., Quimby, A.: A Safety Checklist for the Assessment of inVehicle Information Systems: Scoring Proforma, Project Report PA3536-A/99, Crowthorne, UK, Transport Research Laboratory (1999) 18. Stevens, A., Quimby, A., Board, A., Kersloot, T., Burns, P.: Design Guidelines for Safety of In-Vehicle Information Systems, TRL Limited (2004) 19. Sojourmer, R.J., Antin, J.F.: The effects of a simulated head-up display speedometer on perceptual task performance. Human Factors 32(3), 329–339 (1990) 20. Weintraub, D.J.: Human Factors Issues in Head-Up Display Design: The Book of HUD Descriptive, State-of-the-art report, Dayton University Ohio Research Institute (1992) 21. Brooke, J.: SUS: A quick and dirty usability scale. In: Jordan, P., Thomas, B., Weerdmeester, B., McClelland, I. (eds.) Usability evaluation in industry, pp. 189–194. Taylor & Francis, London (1996) 22. Eilers, K., Nachreiner, F., Hänecke, K.: Entwicklung und Überprüfung einer Skala zur Erfassung subjektiv erlebter Anstrengung. Zeitschrift für Arbeitswissenschaft 40(4), 215– 224 (1986)
Dynamic Maps for Future Navigation Systems: Agile Design Exploration of User Interface Concepts Volker Paelke1 and Karsten Nebe2 1
Leibniz Universität Hannover, IKG Appelstrasse 9a, 30167 Hannover Germany +49-511-7622672
[email protected] 2 University of Paderborn, C-LAB Fürstenallee 11, 33100 Paderborn Germany +49-5251-606132
[email protected]
Abstract. Maps have traditionally been used to support orientation and navigation. Navigation systems shift the focus from printed maps to interactive systems. The key goal of navigation systems is to simplify specific tasks, e.g. route planning or route following. While users of navigation systems need less skills in navigation specific activities, e.g. reading maps or manual route planning, they must now interact with the user interface of the navigation device, which requires a different set of skills. Current navigation systems aim to simplify the interaction by providing interfaces that use basic interaction mechanisms (e.g. button based interfaces on a touch-screen), exploiting the fact that many users are already familiar with such techniques. In the presentation of the information most navigation systems employ map-like displays, possibly combined with additional information, again to exploit familiarity. While such an approach can help with early adoption, it can also limit usefulness and usability. There is, however, a large opportunity to improve input, output and functionality of navigation systems. In this paper we expand a model of classical map based communication to identify possibilities where “dynamic maps” can enhance map based communication in navigation systems. We report on how an agile design exploration process was applied to examine the design space spanned by the new model and to develop system probes. We discuss the user feedback and its implications for future interface concepts for navigation systems.