HCI International 2011 – Posters’ Extended Abstracts, Part II

Communications in Computer and Information Science

174

Constantine Stephanidis (Ed.)

HCI International 2011 – Posters’ Extended Abstracts International Conference, HCI International 2011 Orlando, FL, USA, July 9-14, 2011 Proceedings, Part II

13

Volume Editor Constantine Stephanidis Foundation for Research and Technology - Hellas (FORTH) Institute of Computer Science (ICS) N. Plastira 100, Vassilika Vouton 70013, Heraklion, Crete, Greece and University of Crete Department of Computer Science, Crete, Greece E-mail: [email protected]

ISSN 1865-0929 e-ISSN 1865-0937 ISBN 978-3-642-22094-4 e-ISBN 978-3-642-22095-1 DOI 10.1007/978-3-642-22095-1 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2011930138 CR Subject Classification (1998): H.4, H.5, I.2, H.3, C.2, D.2

© Springer-Verlag Berlin Heidelberg 2011 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Foreword

The 14th International Conference on Human–Computer Interaction, HCI International 2011, was held in Orlando, Florida, USA, July 9–14, 2011, jointly with the Symposium on Human Interface (Japan) 2011, the 9th International Conference on Engineering Psychology and Cognitive Ergonomics, the 6th International Conference on Universal Access in Human–Computer Interaction, the 4th International Conference on Virtual and Mixed Reality, the 4th International Conference on Internationalization, Design and Global Development, the 4th International Conference on Online Communities and Social Computing, the 6th International Conference on Augmented Cognition, the Third International Conference on Digital Human Modeling, the Second International Conference on Human-Centered Design, and the First International Conference on Design, User Experience, and Usability. A total of 4,039 individuals from academia, research institutes, industry and governmental agencies from 67 countries submitted contributions, and 1,318 papers that were judged to be of high scientific quality were included in the program. These papers address the latest research and development efforts and highlight the human aspects of design and use of computing systems. The papers accepted for presentation thoroughly cover the entire field of human–computer interaction, addressing major advances in knowledge and effective use of computers in a variety of application areas. This volume, edited by Constantine Stephanidis, contains posters’ extended abstracts addressing the following major topics: • • • • • • • •

Novel interaction environments Virtual and augmented environments Gestures, gaze and multimodality in HCI Touch-based and table-top interaction Brain-Computer interfaces and brain monitoring Ergonomics and human modelling issues Health and wellbeing Learning, education and cultural heritage

The remaining volumes of the HCI International 2011 Proceedings are: • Volume 1, LNCS 6761, Human–Computer Interaction—Design and Development Approaches (Part I), edited by Julie A. Jacko • Volume 2, LNCS 6762, Human–Computer Interaction—Interaction Techniques and Environments (Part II), edited by Julie A. Jacko • Volume 3, LNCS 6763, Human–Computer Interaction—Towards Mobile and Intelligent Interaction Environments (Part III), edited by Julie A. Jacko • Volume 4, LNCS 6764, Human–Computer Interaction—Users and Applications (Part IV), edited by Julie A. Jacko

VI

Foreword

• Volume 5, LNCS 6765, Universal Access in Human–Computer Interaction— Design for All and eInclusion (Part I), edited by Constantine Stephanidis • Volume 6, LNCS 6766, Universal Access in Human–Computer Interaction— Users Diversity (Part II), edited by Constantine Stephanidis • Volume 7, LNCS 6767, Universal Access in Human–Computer Interaction— Context Diversity (Part III), edited by Constantine Stephanidis • Volume 8, LNCS 6768, Universal Access in Human–Computer Interaction— Applications and Services (Part IV), edited by Constantine Stephanidis • Volume 9, LNCS 6769, Design, User Experience, and Usability—Theory, Methods, Tools and Practice (Part I), edited by Aaron Marcus • Volume 10, LNCS 6770, Design, User Experience, and Usability— Understanding the User Experience (Part II), edited by Aaron Marcus • Volume 11, LNCS 6771, Human Interface and the Management of Information—Design and Interaction (Part I), edited by Michael J. Smith and Gavriel Salvendy • Volume 12, LNCS 6772, Human Interface and the Management of Information—Interacting with Information (Part II), edited by Gavriel Salvendy and Michael J. Smith • Volume 13, LNCS 6773, Virtual and Mixed Reality—New Trends (Part I), edited by Randall Shumaker • Volume 14, LNCS 6774, Virtual and Mixed Reality—Systems and Applications (Part II), edited by Randall Shumaker • Volume 15, LNCS 6775, Internationalization, Design and Global Development, edited by P.L. Patrick Rau • Volume 16, LNCS 6776, Human-Centered Design, edited by Masaaki Kurosu • Volume 17, LNCS 6777, Digital Human Modeling, edited by Vincent G. Duffy • Volume 18, LNCS 6778, Online Communities and Social Computing, edited by A. Ant Ozok and Panayiotis Zaphiris • Volume 19, LNCS 6779, Ergonomics and Health Aspects of Work with Computers, edited by Michelle M. Robertson • Volume 20, LNAI 6780, Foundations of Augmented Cognition: Directing the Future of Adaptive Systems, edited by Dylan D. Schmorrow and Cali M. Fidopiastis • Volume 21, LNAI 6781, Engineering Psychology and Cognitive Ergonomics, edited by Don Harris • Volume 22, CCIS 173, HCI International 2011 Posters Proceedings (Part I), edited by Constantine Stephanidis I would like to thank the Program Chairs and the members of the Program Boards of all Thematic Areas, listed herein, for their contribution to the highest scientific quality and the overall success of the HCI International 2011 Conference. In addition to the members of the Program Boards, I also wish to thank the following volunteer external reviewers: Roman Vilimek from Germany, Ramalingam Ponnusamy from India, Si Jung “Jun” Kim from the USA, and Ilia Adami, Iosif Klironomos, Vassilis Kouroumalis, George Margetis, and Stavroula Ntoa from Greece.

Foreword

VII

This conference would not have been possible without the continuous support and advice of the Conference Scientific Advisor, Gavriel Salvendy, as well as the dedicated work and outstanding efforts of the Communications and Exhibition Chair and Editor of HCI International News, Abbas Moallem. I would also like to thank for their contribution toward the organization of the HCI International 2011 Conference the members of the Human–Computer Interaction Laboratory of ICS-FORTH, and in particular Margherita Antona, George Paparoulis, Maria Pitsoulaki, Stavroula Ntoa, Maria Bouhli and George Kapnas. July 2011

Constantine Stephanidis

Organization

Ergonomics and Health Aspects of Work with Computers Program Chair: Michelle M. Robertson Arne Aar˚ as, Norway Pascale Carayon, USA Jason Devereux, UK Wolfgang Friesdorf, Germany Martin Helander, Singapore Ed Israelski, USA Ben-Tzion Karsh, USA Waldemar Karwowski, USA Peter Kern, Germany Danuta Koradecka, Poland Nancy Larson, USA Kari Lindstr¨om, Finland

Brenda Lobb, New Zealand Holger Luczak, Germany William S. Marras, USA Aura C. Matias, Philippines Matthias R¨ otting, Germany Michelle L. Rogers, USA Dominique L. Scapin, France Lawrence M. Schleifer, USA Michael J. Smith, USA Naomi Swanson, USA Peter Vink, The Netherlands John Wilson, UK

Human Interface and the Management of Information Program Chair: Michael J. Smith Hans-J¨ org Bullinger, Germany Alan Chan, Hong Kong Shin’ichi Fukuzumi, Japan Jon R. Gunderson, USA Michitaka Hirose, Japan Jhilmil Jain, USA Yasufumi Kume, Japan Mark Lehto, USA Hirohiko Mori, Japan Fiona Fui-Hoon Nah, USA Shogo Nishida, Japan Robert Proctor, USA

Youngho Rhee, Korea Anxo Cereijo Roib´ as, UK Katsunori Shimohara, Japan Dieter Spath, Germany Tsutomu Tabe, Japan Alvaro D. Taveira, USA Kim-Phuong L. Vu, USA Tomio Watanabe, Japan Sakae Yamamoto, Japan Hidekazu Yoshikawa, Japan Li Zheng, P.R. China

X

Organization

Human–Computer Interaction Program Chair: Julie A. Jacko Sebastiano Bagnara, Italy Sherry Y. Chen, UK Marvin J. Dainoff, USA Jianming Dong, USA John Eklund, Australia Xiaowen Fang, USA Ayse Gurses, USA Vicki L. Hanson, UK Sheue-Ling Hwang, Taiwan Wonil Hwang, Korea Yong Gu Ji, Korea Steven A. Landry, USA

Gitte Lindgaard, Canada Chen Ling, USA Yan Liu, USA Chang S. Nam, USA Celestine A. Ntuen, USA Philippe Palanque, France P.L. Patrick Rau, P.R. China Ling Rothrock, USA Guangfeng Song, USA Steffen Staab, Germany Wan Chul Yoon, Korea Wenli Zhu, P.R. China

Engineering Psychology and Cognitive Ergonomics Program Chair: Don Harris Guy A. Boy, USA Pietro Carlo Cacciabue, Italy John Huddlestone, UK Kenji Itoh, Japan Hung-Sying Jing, Taiwan Wen-Chin Li, Taiwan James T. Luxhøj, USA Nicolas Marmaras, Greece Sundaram Narayanan, USA Mark A. Neerincx, The Netherlands

Jan M. Noyes, UK Kjell Ohlsson, Sweden Axel Schulte, Germany Sarah C. Sharples, UK Neville A. Stanton, UK Xianghong Sun, P.R. China Andrew Thatcher, South Africa Matthew J.W. Thomas, Australia Mark Young, UK Rolf Zon, The Netherlands

Universal Access in Human–Computer Interaction Program Chair: Constantine Stephanidis Julio Abascal, Spain Ray Adams, UK Elisabeth Andr´e, Germany Margherita Antona, Greece Chieko Asakawa, Japan Christian B¨ uhler, Germany Jerzy Charytonowicz, Poland Pier Luigi Emiliani, Italy

Michael Fairhurst, UK Dimitris Grammenos, Greece Andreas Holzinger, Austria Simeon Keates, Denmark Georgios Kouroupetroglou, Greece Sri Kurniawan, USA Patrick M. Langdon, UK Seongil Lee, Korea

Organization

Zhengjie Liu, P.R. China Klaus Miesenberger, Austria Helen Petrie, UK Michael Pieper, Germany Anthony Savidis, Greece Andrew Sears, USA Christian Stary, Austria

Hirotada Ueda, Japan Jean Vanderdonckt, Belgium Gregg C. Vanderheiden, USA Gerhard Weber, Germany Harald Weber, Germany Panayiotis Zaphiris, Cyprus

Virtual and Mixed Reality Program Chair: Randall Shumaker Pat Banerjee, USA Mark Billinghurst, New Zealand Charles E. Hughes, USA Simon Julier, UK David Kaber, USA Hirokazu Kato, Japan Robert S. Kennedy, USA Young J. Kim, Korea Ben Lawson, USA Gordon McK Mair, UK

David Pratt, UK Albert “Skip” Rizzo, USA Lawrence Rosenblum, USA Jose San Martin, Spain Dieter Schmalstieg, Austria Dylan Schmorrow, USA Kay Stanney, USA Janet Weisenford, USA Mark Wiederhold, USA

Internationalization, Design and Global Development Program Chair: P.L. Patrick Rau Michael L. Best, USA Alan Chan, Hong Kong Lin-Lin Chen, Taiwan Andy M. Dearden, UK Susan M. Dray, USA Henry Been-Lirn Duh, Singapore Vanessa Evers, The Netherlands Paul Fu, USA Emilie Gould, USA Sung H. Han, Korea Veikko Ikonen, Finland Toshikazu Kato, Japan Esin Kiris, USA Apala Lahiri Chavan, India

James R. Lewis, USA James J.W. Lin, USA Rungtai Lin, Taiwan Zhengjie Liu, P.R. China Aaron Marcus, USA Allen E. Milewski, USA Katsuhiko Ogawa, Japan Oguzhan Ozcan, Turkey Girish Prabhu, India Kerstin R¨ ose, Germany Supriya Singh, Australia Alvin W. Yeo, Malaysia Hsiu-Ping Yueh, Taiwan

XI

XII

Organization

Online Communities and Social Computing Program Chairs: A. Ant Ozok, Panayiotis Zaphiris Chadia N. Abras, USA Chee Siang Ang, UK Peter Day, UK Fiorella De Cindio, Italy Heidi Feng, USA Anita Komlodi, USA Piet A.M. Kommers, The Netherlands Andrew Laghos, Cyprus Stefanie Lindstaedt, Austria Gabriele Meiselwitz, USA Hideyuki Nakanishi, Japan

Anthony F. Norcio, USA Ulrike Pfeil, UK Elaine M. Raybourn, USA Douglas Schuler, USA Gilson Schwartz, Brazil Laura Slaughter, Norway Sergei Stafeev, Russia Asimina Vasalou, UK June Wei, USA Haibin Zhu, Canada

Augmented Cognition Program Chairs: Dylan D. Schmorrow, Cali M. Fidopiastis Monique Beaudoin, USA Chris Berka, USA Joseph Cohn, USA Martha E. Crosby, USA Julie Drexler, USA Ivy Estabrooke, USA Chris Forsythe, USA Wai Tat Fu, USA Marc Grootjen, The Netherlands Jefferson Grubb, USA Santosh Mathan, USA

Rob Matthews, Australia Dennis McBride, USA Eric Muth, USA Mark A. Neerincx, The Netherlands Denise Nicholson, USA Banu Onaral, USA Kay Stanney, USA Roy Stripling, USA Rob Taylor, UK Karl van Orden, USA

Digital Human Modeling Program Chair: Vincent G. Duffy Karim Abdel-Malek, USA Giuseppe Andreoni, Italy Thomas J. Armstrong, USA Norman I. Badler, USA Fethi Calisir, Turkey Daniel Carruth, USA Keith Case, UK Julie Charland, Canada

Yaobin Chen, USA Kathryn Cormican, Ireland Daniel A. DeLaurentis, USA Yingzi Du, USA Okan Ersoy, USA Enda Fallon, Ireland Yan Fu, P.R. China Afzal Godil, USA

Organization

Ravindra Goonetilleke, Hong Kong Anand Gramopadhye, USA Lars Hanson, Sweden Pheng Ann Heng, Hong Kong Bo Hoege, Germany Hongwei Hsiao, USA Tianzi Jiang, P.R. China Nan Kong, USA Steven A. Landry, USA Kang Li, USA Zhizhong Li, P.R. China Tim Marler, USA

XIII

Ahmet F. Ozok, Turkey Srinivas Peeta, USA Sudhakar Rajulu, USA Matthias R¨ otting, Germany Matthew Reed, USA Johan Stahre, Sweden Mao-Jiun Wang, Taiwan Xuguang Wang, France Jingzhou (James) Yang, USA Gulcin Yucel, Turkey Tingshao Zhu, P.R. China

Human-Centered Design Program Chair: Masaaki Kurosu Julio Abascal, Spain Simone Barbosa, Brazil Tomas Berns, Sweden Nigel Bevan, UK Torkil Clemmensen, Denmark Susan M. Dray, USA Vanessa Evers, The Netherlands Xiaolan Fu, P.R. China Yasuhiro Horibe, Japan Jason Huang, P.R. China Minna Isomursu, Finland Timo Jokela, Finland Mitsuhiko Karashima, Japan Tadashi Kobayashi, Japan Seongil Lee, Korea Kee Yong Lim, Singapore

Zhengjie Liu, P.R. China Lo¨ıc Mart´ınez-Normand, Spain Monique Noirhomme-Fraiture, Belgium Philippe Palanque, France Annelise Mark Pejtersen, Denmark Kerstin R¨ ose, Germany Dominique L. Scapin, France Haruhiko Urokohara, Japan Gerrit C. van der Veer, The Netherlands Janet Wesson, South Africa Toshiki Yamaoka, Japan Kazuhiko Yamazaki, Japan Silvia Zimmermann, Switzerland

Design, User Experience, and Usability Program Chair: Aaron Marcus Ronald Baecker, Canada Barbara Ballard, USA Konrad Baumann, Austria Arne Berger, Germany Randolph Bias, USA Jamie Blustein, Canada

Ana Boa-Ventura, USA Lorenzo Cantoni, Switzerland Sameer Chavan, Korea Wei Ding, USA Maximilian Eibl, Germany Zelda Harrison, USA

XIV

Organization

R¨ udiger Heimg¨artner, Germany Brigitte Herrmann, Germany Sabine Kabel-Eckes, USA Kaleem Khan, Canada Jonathan Kies, USA Jon Kolko, USA Helga Letowt-Vorbek, South Africa James Lin, USA Frazer McKimm, Ireland Michael Renner, Switzerland

Christine Ronnewinkel, Germany Elizabeth Rosenzweig, USA Paul Sherman, USA Ben Shneiderman, USA Christian Sturm, Germany Brian Sullivan, USA Jaakko Villa, Finland Michele Visciola, Italy Susan Weinschenk, USA

HCI International 2013

The 15th International Conference on Human–Computer Interaction, HCI International 2013, will be held jointly with the affiliated conferences in the summer of 2013. It will cover a broad spectrum of themes related to human–computer interaction (HCI), including theoretical issues, methods, tools, processes and case studies in HCI design, as well as novel interaction techniques, interfaces and applications. The proceedings will be published by Springer. More information about the topics, as well as the venue and dates of the conference, will be announced through the HCI International Conference series website: http://www.hci-international.org/ General Chair Professor Constantine Stephanidis University of Crete and ICS-FORTH Heraklion, Crete, Greece Email: [email protected]

Table of Contents – Part II

Part I: Novel Interaction Environments Measuring Human Interaction in Digital Television Using Profiles and Geolocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Valdecir Becker and Marcelo Kn¨ orich Zuffo

3

PC-Based Warning Mechanism System of Fall Risk in Elderly . . . . . . . . . Chih-Sheng Chang, Cherng-Yee Leung, and Jeih-Jang Liou

8

System of Systems for Sensor and Actuator Networks . . . . . . . . . . . . . . . . . Tiffany Elise Chua, Mark Merlo, and Mark Bachman

13

Smart Clothes Are New Interactive Devices . . . . . . . . . . . . . . . . . . . . . . . . . Gi-Soo Chung and Hee-Cheol Kim

18

Ebook Readers: An iPod for Your Books in the Cloud . . . . . . . . . . . . . . . . Ann-Marie Horcher and Maxine Cohen

22

The Ambient from the Young Passengers’ Perception in the Carriage of Taiwan High Speed Rail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jeichen Hsieh and Chan Yo Shan

28

An Effective Disaster Evacuation Assist System Utilized by an Ad-Hoc Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yasuki Iizuka, Kyoko Yoshida, and Kayo Iizuka

31

Locating Projectors Using Intensity of Reflected Beams Based on Phong Shading Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yukio Ishihara and Makio Ishihara

36

Embodied Communication Support Using a Presence Sharing System under Teleworking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yutaka Ishii and Tomio Watanabe

41

Visibility Experiment and Evaluation of 3D Character Representation on Mobile Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hiromu Ishio, Shunta Sano, Tomoki Shiomi, Tetsuya Kanda, Hiroki Hori, Keita Uemoto, Asei Sugiyama, Minami Niwa, Akira Hasegawa, Shohei Matsunuma, and Masaru Miyao Composite Context Information Model for Adaptive Human Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sukyoung Kim, Eungha Kim, and Youngil Choi

46

52

XVIII

Table of Contents – Part II

A Framework for a User Friendly Wireless Sensor Network Configuration System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Julia C. Lee and Lawrence J. Henschen FlexRemote: Exploring the Effectiveness of Deformable User Interface as an Input Device for TV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sang-Su Lee, Seungwoo Maeng, Daeeop Kim, Kun-Pyo Lee, Wonkyum Lee, Sangsik Kim, and Sungkwan Jung A Study of User Needs for the ‘Techno Kitchen’ . . . . . . . . . . . . . . . . . . . . . Martin Maguire, Colette Nicolle, Russell Marshall, Ruth Sims, Clare Lawton, Sheila Peace, and John Percival Consideration of the Human-Computer Interface in the Operation Room in the Era of Computer Aided Surgery . . . . . . . . . . . . . . . . . . . . . . . . Kazuhiko Shinohara ‘STISIM-Drive’ Meets ‘MotorcycleSim’: Using Driving Simulation Software to Develop a Unique Motorcycle Simulator for Rider Behavior Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alex W. Stedmon, David Crundall, Elizabeth Crundall, Rose Saikayasit, Editha van Loon, Alex Irune, Patrick Ward, and Neil Greig

57

62

66

72

76

AirportLogic: Usability Testing, Prototyping, and Analysis of an Airport Wayfinding Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bennett Stone and Yun Wang

81

Using on-Bicycle Rider Assistant Device While Cycling: A Hazard Perception Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chao-Yang Yang and Cheng-Tse Wu

85

Part II: Virtual and Augmented Environments Human–Robot Collaboration with Augmented Reality . . . . . . . . . . . . . . . . Siam Charoenseang and Tarinee Tonggoed Making Pixel Patterns Automatically for Camouflage - Using Color Information from Their Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Woon Jung Cho, Wonmi Ahn, Myung Shik Kim, Jeeyea Park, Seungduk Kim, and Kwang-Hee Han Virtual Bridge: AR-Based Mobile Interaction for Easy Multimedia Control of Remote Home Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DongJin Eun, Taik Heon Rhee, Seonghoon Kang, Minsuk Choi, Sangil Lee, and Hark-Joon Kim

93

98

102

Table of Contents – Part II

XIX

Design and Implementation of a Low-Cost Projected Virtual Reality System to Support Learning Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rodrigo G´ omez and Helmuth Trefftz

107

Interface Design to Support Situation Awareness in Virtual Puppetry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Keisha Harthoorn and Stephen Hughes

112

Immersive Video Game Based on Exercise Prescription . . . . . . . . . . . . . . . Daegun Kim and Changhoon Park

116

Assessing the Use of Cognitive Resources in Virtual Reality . . . . . . . . . . . William E. Marsh, Jonathan W. Kelly, Veronica J. Dark, and James H. Oliver

120

Augmented Reality Approach to Domestic Maintenance Tasks . . . . . . . . . Jorge Mart´ın-Guti´errez and Irene In´es Santos P´erez

125

Development of AR Display System for Dental Surgical Simulator . . . . . . Katsuhiko Onishi, Shota Ito, Yusuke Kawamura, and Hiroshi Noborio

130

Earthquake Disaster Prevention Support Tool – Visualization of Prevention Effectiveness by Utilizing Augmented Reality . . . . . . . . . . . . . . Kyoko Yoshida, Masahiro Urabe, Hayato Tsuchiya, Yasuki Iizuka, and Kayo Iizuka

134

Part III: Gestures, Gaze and Multimodality in HCI A Three-Dimensional Fingertip Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yangkeun Ahn, Kwangmo Jung, and Jiman Hong

141

Rule Based Trajectory Segmentation Applied to an HMM-Based Isolated Hand Gesture Recognizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jounghoon Beh, David Han, and Hanseok Ko

146

Head-Free, Remote Eye-Gaze Detection System with Easy Calibration Using Stereo-Calibrated Two Video Cameras . . . . . . . . . . . . . . . . . . . . . . . . Yoshinobu Ebisawa, Kazuki Abo, and Kiyotaka Fukumoto

151

Eye Position Effect on Audio-Visual Fusion Involves New Proposals for Multimodal Interface Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David Hartnagel, Alain Bichot, Patrick Sandor, and Corinne Roumes

156

A Virtual Mouse System Using Finger-Gestures of Twisting-in . . . . . . . . . Takashi Kihara and Makio Ishihara

161

Control of Five Finger of Computer Graphics Hand Using Electromyographic Signal Measured with Multi-channeled Small Laplacian Electrodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takuya Kimura and Akinori Ueno

166

XX

Table of Contents – Part II

Kinematic Analysis of Remote Target Pointing Hand Movements in a 3D Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yung-Hui Lee and Shu-Kai Wu Design and Implementation of Deformation Based Gesture Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wonkyum Lee, Sungkwan Jung, Sangsik Kim, Woojin Ahn, and Sang-su Lee

171

176

The Expansibility of User Interfaces Using Peripheral Multisensory Stimulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ju-Hwan Lee

180

Use of Hands-Free Mouse for Game Control . . . . . . . . . . . . . . . . . . . . . . . . . Moyen Mohammad Mustaquim

184

An Armband-Type Touch-Free Space Input Device for HCI . . . . . . . . . . . Dongwan Ryoo and Junseok Park

188

Modeling of Purchasing Behavior for Application on Merchandise Display and Package Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kotaro Suzuki, Nobuyuki Nishiuchi, and Mi Kyong Park

193

Decoding of Hand Shapes Based on ElectroMyoGraphic Signals during Playing Guitar Chords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hideaki Touyama and Masafumi Mizuguchi

197

Exploring Whole-Hand Gestures in a Tabletop Environment for Urban Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peter Vandoren, Karel Frederix, Karin Coninx, and Frank Van Reeth

201

Input Interface Using Fingertip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ryo Wada and Tomohiro Hase

206

Recognition Method for Foot Written Characters . . . . . . . . . . . . . . . . . . . . Masahiro Yonezawa, Takako Nonaka, and Tomohiro Hase

210

Part IV: Touch-Based and Table-Top Interaction Sounds in Space: 3D Audio Experiences through Tangible Navigation . . . Andrew Blakney and Sudhir Mudur

217

Multi-touch Surface Table with Multi-point Tactile Feedback . . . . . . . . . . Siam Charoenseang and Navakun Sribang

222

Suggested Considerations on the Design of Multi-touch Interfaces for Commercial Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ting-Han Chen

227

Table of Contents – Part II

XXI

A Study on the C/R Ratio of Direct-Operation Multi-touch Interface . . . Kuan-Hung Chen, Chun-Wen Chen, and Wenzhi Chen

232

Multi-touch Table as Conventional Input Device . . . . . . . . . . . . . . . . . . . . . Andreas Dippon, Florian Echtler, and Gudrun Klinker

237

Properties of Shadow-Cursor for Calibrating Screen Coordinates of Tabletop Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Makio Ishihara and Yukio Ishihara

242

Emotional Expression by a Person’s Grip on a Tactual Communication Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yasuhiro Matsuda and Tsuneshi Isomura

247

Effect of Target Size and Duration of Visual Feedback on Touch Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jeeyea Park and Kwang-Hee Han

252

Development of an Economical Haptic Stimulus Device . . . . . . . . . . . . . . . Greg Placencia, Mansour Rahimi, and Behrokh Khoshnevis

257

Feeling Home – Tangible Information Visualization in Smart Home Environments in Relation to the Concept of Transhumanism . . . . . . . . . . Florian Weingarten and Sahin Albayrak

262

Part V: Brain-Computer Interfaces and Brain Monitoring Calibration Time Reduction through Source Imaging in Brain Computer Interface (BCI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Minkyu Ahn, Hohyun Cho, and Sung Chan Jun

269

How Much Features in Brain-Computer Interface Are Discriminative? – Quantitative Measure by Relative Entropy . . . . . . . . . . . . . . . . . . . . . . . . Sangtae Ahn, Sungwook Kang, and Sung Chan Jun

274

EEG-Based Measurement of Subjective Parameters in Evaluations . . . . . Daniel Cernea, Peter-Scott Olech, Achim Ebert, and Andreas Kerren

279

Fundamental Study of the Pictogram-Scanning-BCI . . . . . . . . . . . . . . . . . . Hiroyuki Inada and Hisaya Tanaka

284

EEG Based Comparative Measurement of Visual Fatigue Caused by 2D and 3D Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Young-Joo Kim and Eui Chul Lee

289

A New Design of the Multi-channels Mobile and Wireless EEG System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chin-Teng Lin, Wan-Ru Wang, I.-Jan Wang, Lun-De Liao, Sheng-Fu Chen, Kevin Tseng, and Li-Wei Ko

293

XXII

Table of Contents – Part II

An Experimental Comparison of Brain Activity in Professional and Non-professional Sewers during the Use of Sewing Needles . . . . . . . . . . . . . Masako Omori, Yukari Morishita, and Asuka Kawakita

299

EEG-Based Measure of Cognitive Workload during a Mental Arithmetic Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Brice Rebsamen, Kenneth Kwok, and Trevor B. Penney

304

EEG Measurements towards Brain Life-Log System in Outdoor Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hideaki Touyama and Kazuya Maeda

308

Part VI: Ergonomics and Human Modelling Issues On the Applicability of Digital Human Models for Personal Equipment Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Thomas Alexander and Jessica Conradi Discussing Validation of 3D Character Animation Demonstrating Ushiro-Ukemi Pedagogical Progression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mauro Cesar Gurgel de Alencar Carvalho, Bruno Martins Carvalho, Felipe Leal de Paiva Carvalho, Heidi Dias Oliveira Junior, Gerson Gomes Cunha, Luiz Landau, and Est´elio Henrique Martin Dantas

315

320

The Provision of Digital Information in the Seat Comfort of the Seat Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kuen-Meau Chen, Siu-Tsen Shen, and Stephen D. Prior

325

The Effect of Damping in an Input Device on Human Positioning Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Koen Crommentuijn and Dik J. Hermes

330

Performance and Comfort When Using Motion-Controlled Tools in Complex Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ines Ann Heber, Michael Oehl, and Christine Sutter

335

Pen Tip Position Estimation Using Least Square Sphere Fitting for Customized Attachments of Haptic Device . . . . . . . . . . . . . . . . . . . . . . . . . . Masanao Koeda and Masahiko Kato

340

Corrected Human Vision System and the McGurk Effect . . . . . . . . . . . . . . Ladislav Kunc and Pavel Slav´ık

345

Facial Landmark Extraction for Lip Tracking of Patients with Cleft Lip Using Active Appearance Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nayoung Lee, Chuck Heaston, Ada Rey, Terry Hartman, and Carroll-Ann Trotman

350

Table of Contents – Part II

XXIII

Kansei Evaluation of the Projection for the Approach to Universal Design: Computerization of Tactile Sensibility . . . . . . . . . . . . . . . . . . . . . . . Miyong Lee, Kazuhiro Nishida, and Yoshihiro Narita

355

A Framework of Motion Capture System Based Human Behaviours Simulation for Ergonomic Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ruina Ma, Damien Chablat, Fouad Bennis, and Liang Ma

360

Visual Perception Model for Sense of Materials . . . . . . . . . . . . . . . . . . . . . . Wenhao Wang and Toshikazu Kato

365

The Effects of Stereoscopic Display Luminance and Ambient Illumination on Visual Comfort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pei-Chia Wang, Sheue-Ling Hwang, Kuan-Yu Chen, Jinn-Sen Chen, Jinn-Cherng Yang, and Hung-Lu Chang Preferred Setting of Keyboard and Mouse for Using a Supine Computer Workstation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hsin-Chieh Wu and Ho-Rong Chu

369

374

Part VII: Health and Wellbeing An Interactive Multimedia System for Monitoring the Progressive Decline of Memory in Alzheimer’s Patients . . . . . . . . . . . . . . . . . . . . . . . . . . Hala Al-Muhanna, Rawan Al-Wabil, Hailah Al-Mazrua, Noura Al-Fadhel, and Areej Al-Wabil Personal Smart Spaces for Diabetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manal AlBahlal and Jalal AlMuhtadi Quality and Usability Assessment for Health Information Websites: Can Commonly Used Evaluation Criteria Be Appropriately Applied to Assess Chinese-Language Websites? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chang Fang-Fang, Ku Chia-Hua, Wang Kung-Jeng, and Wu Wei-Li

381

386

391

Computer Interaction and the Benefits of Social Networking for People with Borderline Personality Disorder: Enlightening Mental Health Professionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alice Good, Arunasalam Sambhanthan, Vahid Panjganj, and Samuel Spettigue

395

Design Improvement Requirements for the Upper Extremity Rehabilitation Devices in Taiwan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lan-Ling Huang, Chang-Franw Lee, and Mei-Hsiang Chen

400

Observation Research of Consumer Behavior for Marketing Decision Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hideyuki Imai, Noriko Hara, and Toshiki Yamaoka

405

XXIV

Table of Contents – Part II

Exercise Reminder Software for Office Workers . . . . . . . . . . . . . . . . . . . . . . Ahsen Irmak, Rafet Irmak, and Gonca Bumin

410

Games for Health: Design Cognition-Focused Interventions to Enhance Mental Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hyungsin Kim, Viraj Sapre, and Ellen Yi-Luen Do

415

Promoting Positive Employee Health Behavior with Mobile Technology Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hyungsin Kim, Hakkyun Kim, and Ellen Yi-Luen Do

420

Believable Agents, Engagement, and Health Interventions . . . . . . . . . . . . . Christine L. Lisetti

425

Le-ADS: Early Learning Disability Detection System for Autism and Dyslexia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nor’ain Mohd Yusoff, Nor Syarafina Rusli, and Ruhaiza Ishak

433

Interaction Design of Encouraging Daily Healthcare Habit with Communication Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jun’ichi Osada, Tomoharu Yamaguchi, Ryohei Sasama, and Keiji Yamada Can Digital Signage Help Consumers Eat Healthier? . . . . . . . . . . . . . . . . . . Anicia Peters and Brian Mennecke Constraint-Based Nurse Rostering for the Valpara´ıso Clinic Center in Chile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Renzo Pizarro, Gianni Rivera, Ricardo Soto, Broderick Crawford, Carlos Castro, and Eric Monfroy

438

443

448

Connecting with Dysphonia: Human-Computer Interface for Amyotrophic Lateral Sclerosis Patients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chun-Yang Su and Ju-Joan Wong

453

Assessing Health Information Websites for Inclusion of Web 2.0 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adam Townes and Michelle Rogers

458

Encouraging Daily Healthcare Habit with Communication Robots . . . . . . Tomoharu Yamaguchi, Ryohei Sasama, Jun’ichi Osada, and Keiji Yamada

463

Part VIII: Learning, Education and Cultural Heritage The Evaluation of the Applicability of Distance Education in Vocational Colleges by the Students of Erzurum Vocational College, Computer Technologies Department, Erzurum, Turkey . . . . . . . . . . . . . . . . . . . . . . . . . Yusuf Ziya Ayik

469

Table of Contents – Part II

XXV

An Evaluation of SignBright: A Storytelling Application for Sign Language Acquisition and Interpersonal Bonding amongst Deaf and Hard of Hearing Youth and Caregivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Melissa M. Burton, Chad Harbig, Mariam Melkumyan, Lei Zhang, and Jiyoung Choi

474

Collaborative Analysis and Communities of Practice in Health Sciences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Juan Alberto Castillo M.

479

The Application of Interactive Media Display Technology in Environmental Science Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chun-Ching Chen and Chien-Ming Chen

484

Applying User-Centered Techniques to Develop a Radiology Teaching File System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marcelo dos Santos and Asa Fujino

489

The Study of the Interaction of Public Art with Digital Technology . . . . Shih Yin Huang and Ming-Shean Wang

494

Seven Wonders: An Interactive Game for Learning English as a Foreign Language in Junior High-School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . George Kapnas, Stavroula Ntoa, George Margetis, Margherita Antona, and Constantine Stephanidis Study-Buddy: Improving the Learning Process through TechnologyAugmented Studying Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . George Margetis, Stavroula Ntoa, Maria Bouhli, and Constantine Stephanidis

499

504

Improving Academic Performance and Motivation in Engineering Education with Augmented Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jorge Mart´ın-Guti´errez and Manuel Contero

509

Evaluation of Robot Based Embedded System Study Environment in Technical High School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yosuke Nishino and Eiichi Hayakawa

514

Extending Authoring for Adaptive Learning to Collaborative Authoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dade Nurjanah, Hugh Davis, and Thanassis Tiropanis

519

A Collaborative Tool for Communities of Practice to Share Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Justus N. Nyagwencha, Sheryl Seals, and Tony Cook

524

Classic Art for Modern People . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nikolaos Partarakis, Sokratis Kartakis, Margherita Antona, George Paparoulis, and Constantine Stephanidis

529

XXVI

Table of Contents – Part II

Information and Communication Technology (ICT) and Special Education System in the Kingdom of Saudi Arabia: A Case Study . . . . . Mukhtar M. Rana, Mohammad Fakrudeen, Mahdi H. Miraz, Sufian Yousef, and Alshammari Abderrahman Torqi ICT Training of Maestros of Primary Schools Located in Barrios Carenciados in Argentina. A Twofold Challenge: How They Can Master New ICT Technologies and Transform the Way They Teach . . . . . . . . . . . C. Osvaldo Rodriguez The Design of the Satellite Spaces for Informal Learning and Its Validity Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Syoko Shimora, Kazuyoshi Yamauch, and Natsuko Ohtake Window Control Interface to Attract Teacher’s Gaze Area for Watching a Reaction of Remote Learners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takumi Yamaguchi, Haruya Shiba, Naohisa Matsuuchi, Yusuke Nishiuchi, Kazunori Shimamura, and Takahiko Mendori Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

534

539

544

549

555

Table of Contents – Part I

Part I: Design Methods, Techniques and Knowledge Professional Graphic Designers Approaching Visual Interface Design . . . . Joanne Elizabeth Beriswill

3

Co-discovery Method and Its Application with Children as Research Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alessandra Carusi and Cl´ audia Mont‘Alv˜ ao

8

Get Your Mobile App Out the Door . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Heather Cottingham and Michele Snyder

13

Activity-Centered Design: An Appropriation Issue . . . . . . . . . . . . . . . . . . . Yvon Haradji, Germain Poizat, and Florence Mott´e

18

Conjoint Analysis Method That Minimizes the Number of Profile Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hiroyuki Ikemoto and Toshiki Yamaoka

23

Research on the Role of the Sketch in Design Idea Generation . . . . . . . . . Yuichi Izu, Koichirou Sato, and Yoshiyuki Matsuoka

29

Initial Perspectives from Preferences Expressed through Comparisons . . . Nicolas Jones, Armelle Brun, and Anne Boyer

33

Reducing Uncertainty in a Human-Centered Design Approach: Using Actor-Network Theory Analysis to Establish Fluid Design Guidelines . . . Ryan Kirk and Anna Prisacari

38

Verification of Centrality to Extract Proper Factors in Model Construction Process by Using Creativity Technique . . . . . . . . . . . . . . . . . . Kodai Kitami, Ryosuke Saga, and Kazunori Matsumoto

43

User-Centered Approach for NEC Product Development . . . . . . . . . . . . . . Izumi Kohno and Hiromi Fujii

48

Idea Creation Method Based on Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nozomi Koyatsu and Kazuhiko Yamazaki

53

Designing Interfaces for Home Energy Users: A Preference Study . . . . . . . Janelle LaMarche and Olga Sachs

58

Exploring the Relationship between Thinking Style and Collaborative Design Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chiung-Cheng Liao, Wenzhi Chen, and Hsien-Hui Tang

63

XXVIII

Table of Contents – Part I

Identifying Product Opportunity Based on Interactivity . . . . . . . . . . . . . . . Seungwoo Maeng, Daeeop Kim, Sang-Su Lee, and Kun-Pyo Lee

67

Idea Creative Method Based on Metaphor for Product Design . . . . . . . . . Takuya Mitsumaru

72

Persona-Storyboard Fusion: A Hybrid Approach to Improving Design Artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Michael Stewart, Jennifer Francois, Hongbo Zhang, and D. Scott McCrickard Studying Analysis Method for the Design Innovation . . . . . . . . . . . . . . . . . Takashi Inaba and Kazuhiko Yamazaki Balancing Trust and Automation Needs for Effective Home Energy Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hari Thiruvengada, Pallavi Dharwada, Anand Tharanathan, Wendy Foslien, Sriharsha Putrevu, and John Beane

77

82

86

Defining a Process for Cross-Product User Interface Consistency . . . . . . . Leslie Tudor and Cheryl L. Coyle

91

Tweaking HCI Methods for m-Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Al´ıcia Valls Saez, Muriel Garreta-Domingo, Gemma Aguado, and Marta Lopez Reyes

96

Part II: Usability and User Experience Contextual Awareness as Measure of Human-Information Interaction in Usability and Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Michael J. Albers

103

A Usability Model for Government Web Sites . . . . . . . . . . . . . . . . . . . . . . . . Deborah S. Carstens and Annie Becker

108

Usability Analysis of Website with Unconventional Layout . . . . . . . . . . . . Ro-Han Chang and Ying-Ya Su

113

Methodologies for Evaluating Player Experience in Game Play . . . . . . . . . Kimberly Chu, Chui Yin Wong, and Chee Weng Khong

118

How Does This Look? Desirability Methods for Evaluating Visual Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Edward S. De Guzman and Julie Schiller

123

An Analysis of Usage Patterns in Utilization of Interaction Styles . . . . . . Martin Dost´ al

128

On the Differences in Usage of Word Processing Applications . . . . . . . . . . Martin Dost´ al

133

Table of Contents – Part I

XXIX

Usability Study of TEL Recommender System and e-Assessment Tools United . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Beatriz Florian and Ram´ on Fabregat

138

Perceived Multimedia Quality: The Impact of Device Characteristics . . . Gheorghita Ghinea and Kyle J. Patterson

143

Usability Testing with Children: What We Have Overlooked . . . . . . . . . . . Hanayanti Hafit, Fariza Hanis Abdul Razak, and Haryani Haron

147

The Usability Assessment of Web-Based Learning Systems . . . . . . . . . . . . Chen-Wei Hsieh, Hong-Xon Chen, Yung-Chi Hsu, and Sherry Y. Chen

151

Using Pre-session Homework to Elicit More Insights during Web Usability Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Christopher Jewell and Franco Salvetti

156

Personalized ATMs: Improve ATMs Usability . . . . . . . . . . . . . . . . . . . . . . . . Armin Kamfiroozie and Marzieh Ahmadzadeh

161

The Effect of Induced Priming on Product Perceived Usability . . . . . . . . . Jihyun Kim, Myung Shik Kim, and Kwang-Hee Han

167

Who Are the People That Experience Soft Usability Problems? . . . . . . . . Chajoong Kim and Henri Christiaans

171

Gaze Analysis Tool for Web Usability Evaluation . . . . . . . . . . . . . . . . . . . . Takuo Matsunobe

176

Usability Evaluation for Software Keyboard on High-Performance Mobile Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takao Nakagawa and Hidetake Uwano

181

Usability Evaluation Method Employing Elements of “Thinking” and “Seeking” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nobuyuki Nishiuchi, Takehiro Ando, and Mi Kyong Park

186

Software Testing Method Considering the Importance of Factor Combinations in Pair-Wise Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ruoan Xu, Yoshimitsu Nagai, and Syohei Ishizu

191

Part III: Cultural, Cross-Cultural and Aesthetic Issues in HCI The Problematic of Beauty Performed in the Collaborative Action of Technology and Human . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HyunKyoung Cho

199

XXX

Table of Contents – Part I

The Politics of Collaborative Action of Technology and Human . . . . . . . . HyunKyoung Cho

204

What Makes Difference in Mobile Application Sales in Countries? . . . . . . Hyeyoung Eun, Hyunsuk Kim, and Sungmin Hong

209

hvMuseum: A Participatory Online Museum of Everyday Artifacts for Cultural Awareness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Young-ae Hahn

214

The Cross-Cultural Adaptation of the Work Role Functioning Questionnaire to Turkish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ahsen Irmak, Gonca Bumin, and Rafet Irmak

218

WARAI PRODUCT: Proposal to the Design Approach Designing the Product That Causes Laughter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takaaki Kamei and Kazuhiko Yamazaki

223

The Beauty Formation of Digital Media Projects via Sticking Jewels Techniques Applied to the Research and Education of Culture Creative Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jhih-Wei Lee and Chun-Ming Huang Leveraging Card-Based Collaborative Activities as Culturally Situated Design Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D. Scott McCrickard, DeMarcus Townsend, Woodrow W. Winchester, and Tiffany Barnes

227

232

Designing for Cultural Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anicia Peters, Britta Mennecke, Jos´e Camou, Kiraz Candan Herdem, and Lei Zhang

237

Can Culture Translate to the Virtual World? . . . . . . . . . . . . . . . . . . . . . . . . Raghavi Sakpal and Dale-Marie Wilson

242

Product Pleasure Enhancement: Cultural Elements Make Significant Difference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tyan-Yu Wu

247

Part IV: Cognitive and Psychological Issues in HCI An Optimal Human Adaptive Algorithm to Find Action-Reaction Word-Pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Arpit Agarwal, Rahul Banerjee, Varun Pandey, and Riya Charaya

255

Modeling Users in Web Transactional Tasks with Behavioral and Visual Exploration Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Areej Al-Wabil and Mashael Al-Saleh

260

Table of Contents – Part I

Evaluating Information Visualizations with Working Memory Metrics . . . Alisa Bandlow, Laura E. Matzen, Kerstan S. Cole, Courtney C. Dornburg, Charles J. Geiseler, John A. Greenfield, Laura A. McNamara, and Susan M. Stevens-Adams A Study on Human Error in the Interaction with the Computer Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Luiz Carlos Begosso, Maria Alice Siqueira Mendes Silva, and Thiago Henrique Cortez Psycognition: Cognitive Architectures for Augmented Cognition Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Karmen Guevara A Study on the Cognitive Differences between Beginners and Experts Regarding Cooking Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Keisuke Ishihara, Toshiki Yamaoka, Kazumi Tateyama, and Chinatsu Kasamatsu

XXXI

265

270

275

280

Red for Romance, Blue for Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ilyung Jung, Myung Shik Kim, and Kwang-Hee Han

284

Time: A Premise of Virtual Life . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hee-Cheol Kim

289

Extracts Cognitive Artifacts from Text through Combining Human and Machine Learning in an Iterative Fashion . . . . . . . . . . . . . . . . . . . . . . . . . . . Ryan Kirk

293

Modeling Human Behavior for Energy-Usage Prediction . . . . . . . . . . . . . . Anand S. Kulkarni, Karla Conn Welch, and Cindy K. Harnett

298

The Effect of a Visual Element on Musical Sensitivity . . . . . . . . . . . . . . . . Jieun Lee, Mitsuko Hayashi, and Masashi Nosaka

303

A Scrutinized Analysis Method of the Human Error Potential Due to the Introduction of New Digital Devices to Nuclear Power Plants . . . . . . Yong Hee Lee

308

Understanding Users by Their D.I.S.C. Personality through Interactive Gaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qin En Looi, Swee Lan See, Chi Shien Tay, and Gin Kee Ng

312

Modeling Attention Allocation in a Complex Dual Task with and without Auditory Cues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Brian McClimens and Derek Brock

317

XXXII

Table of Contents – Part I

Relationship between Emotional State and Physiological and Psychological Measurements Using Various Types of Video Content during TV Viewing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kiyomi Sakamoto, Shigeo Asahara, Kuniko Yamashita, and Akira Okada

322

Physiological Measurement Applied in Maritime Situations: A Newly Developed Method to Measure Workload on Board of Ships . . . . . . . . . . . Wendie Uitterhoeve, Marcella Croes-Schalken, and Dick Ten Hove

327

Physiological Correlates of Emotional State . . . . . . . . . . . . . . . . . . . . . . . . . Andrea K. Webb, Meredith G. Cunha, S.R. Prakash, and John M. Irvine A Study on the Operator’s Erroneous Responses to the New Human Interface of a Digital Device to be Introduced to Nuclear Power Plants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yeon Ju Oh, Yong Hee Lee, and Jong Hun Yun To Substitute Fast-Forward/Backward Keys for Numeric Keypad of TV Remote Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Horng-Yi Yu, Jui-Ping Ma, and T.K. Philip Hwang

332

337

342

Part V: Inclusive Design and Accessibility Digital Inclusion Index (DII) – Measuring ICT Supply and Usage to Support DI Implementation Planning Policies . . . . . . . . . . . . . . . . . . . . . . . Graziella Cardoso Bonadia, Nyvea Maria da Silva, and Cristiane Midori Ogushi Serious Game for Cognitive Testing of Elderly . . . . . . . . . . . . . . . . . . . . . . . Sangwoo Byun and Changhoon Park Leisure Activities for the Elderly–The Influence of Visual Working Memory on Mahjong and Its Video Game Version . . . . . . . . . . . . . . . . . . . . Chih-Lin Chang, Tai-Yen Hsu, Fang-Ling Lin, Chuen-Der Huang, and I.-Ting Huang

349

354

358

An Empathic Approach in Assistive Technology to Provide Job Accommodations for Disabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chien-Bang Chen

363

A Study on Interface Design Guidelines of Web Maps for Elder Users . . . Chun-Wen Chen and Kevin Tseng

368

Impact of Prior Knowledge and Computer Interface Organization in Information Searching Performances: A Study Comparing Younger and Older Web Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Aline Chevalier, Paulette Rozencwajg, and Benjamin Desjours

373

Table of Contents – Part I

XXXIII

Sketching Haptic System Based on Point-Based Approach for Assisting People with Down Syndrome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mario Covarrubias, Monica Bordegoni, and Umberto Cugini Helping Hands versus ERSP Vision: Comparing Object Recognition Technologies for the Visually Impaired . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marc A. Lawson, Ellen Yi-Luen Do, James R. Marston, and David A. Ross

378

383

Examining the Current State of Group Support Accessibility: An Expanded Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . John G. Schoeberlein and Yuanqiong Wang

389

Verbalizing Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lisa Tang and Jim Carter

394

Experiencing Accessibility Issues and Options . . . . . . . . . . . . . . . . . . . . . . . Lisa Tang, David Fourney, and Jim Carter

399

Adopting User-Centered Design for the Translating of Barrier-Free Design Codes/Regulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tsai-Hsuan Tsai, Wen-Ko Chiou, Huey-Yann Liao, and Tai-Xian Tseng

404

User Research for Senior Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kaori Ueda and Kazuhiko Yamazaki

409

Audio-Haptic Description in Movies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lakshmie Narayan Viswanathan, Troy McDaniel, and Sethuraman Panchanathan

414

Part VI: Social Interaction and On-line Communities Becoming Friends on Online Social Networking Services . . . . . . . . . . . . . . . Wonmi Ahn, Borum Kim, and Kwang-Hee Han

421

On-line Communication as a Part of the “Symbolic Politics” . . . . . . . . . . Evgeniy Ishmenev

426

A Displaying Method of Food Photos to Know Child’s Dietary Life for Parents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kenta Iwasaki, Kazuyoshi Murata, and Yu Shibuya

431

Memory Makers – The Experience of Camera Usage by Women . . . . . . . . Yu-Lung Kao and Ju-Joan Wong

435

Unique Motivation for Using Global Social Network Site in Korea . . . . . . Hyosun Kim and Kwang-Hee Han

440

XXXIV

Table of Contents – Part I

Color Image Effect of Online Community on Age: Focusing on Self-expression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jihyun Kim, Hyeryeong Kim, and Kwang-Hee Han

445

Gender in the Digital Age: Women’s Participation in Designing Social Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tˆ ania Cristina Lima and J´ ulio Cesar dos Reis

449

Study of Communication Aid Which Supports Conversation Held at Railway Station . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kaoru Nakazono, Mari Kakuta, and Yuji Nagashima

454

Sociable Tabletop Companions at “Dinner Party” . . . . . . . . . . . . . . . . . . . . Hye Yeon Nam and Ellen Yi-Luen Do

459

Quality of Community in Social Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kohei Otake, Tadakazu Fukutomi, and Tomofumi Uetake

464

Taiwanese Facebook Users’ Motivation and the Access of Information Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chun-Ming Tsai, Yu-Ting Huang, and Ji-Lung Hsieh

469

Connecting Generations: Preserving Memories with Thanatosensitive Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cyndi Wiley, Yun Wang, Ryan Musselman, and Beverly Krumm

474

Part VII: Work and Collaboration Introducing CAPER, a Collaborative Platform for Open and Closed Information Acquisition, Processing and Linking . . . . . . . . . . . . . . . . . . . . . Carlo Aliprandi and Andrea Marchetti

481

Secure Transmission of Medical Images by SSH Tunneling . . . . . . . . . . . . . Felipe Rodrigues Martinˆez Basile and Fl´ avio Cezar Amate

486

Service Components for Unified Communication and Collaboration of an SOA-Based Converged Service Platform . . . . . . . . . . . . . . . . . . . . . . . . . . Ki-Sook Chung and Young-Mee Shin

491

Fine-Grained Adaptive User Interface for Personalization of a Word Processor: Principles and a Preliminary Study . . . . . . . . . . . . . . . . . . . . . . . Martin Dost´ al and Zdenek Eichler

496

Development of Learning Achievement Index for Project Human Resource Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yusuke Emori, Takuya Furusawa, and Tsutomu Konosu

501

Table of Contents – Part I

Design and Development of Information Display Systems for Monitoring Overboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tadasuke Furuya, Atsushi Suzuki, Atsushi Shimamura, Takeshi Sakurada, Yoichi Hagiwara, and Takafumi Saito Fault Diagnosis of Induction Motors Using Discrete Wavelet Transform and Artificial Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . In-Soo Lee Study on Providing Multi-faceted Information on Technology Intelligence Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mikyoung Lee, Seungwoo Lee, Pyung Kim, Hanmin Jung, and Won-Kyung Sung

XXXV

506

510

515

Simulating Additional Area on Tele-Board’s Large Shared Display . . . . . Peter LoBue, Raja Gumienny, and Christoph Meinel

519

Components Based Integrated Management Platform for Flexible Service Deployment in Plant Factory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Aekyung Moon, Song Li, and Kyuhyung Kim

524

Development of the Many Nodes Connected and Simple Operated HD Remote Lecture System by Automatic Control . . . . . . . . . . . . . . . . . . . . . . . Takeshi Sakurada, Yoichi Hagiwara, and Tadasuke Furuya

529

Enhancing Flexibility of Production Systems by Self-optimization . . . . . . Robert Schmitt, Carsten Wagels, Mario Isermann, and Marcel Mayer

534

Do They Use Different Set of Non-verbal Language in Turn-Taking in Distributed Conferences? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hidekazu Tamaki, Suguru Higashino, Minoru Kobayashi, and Masayuki Ihara Floating 3D Video Conference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kun-Lung Tseng, Wen-Chao Chen, Tung-Fa Liou, and Kang-Chou Lin

539

544

Part VIII: Access to Information and Knowledge User Interface Design for the Interactive Use of Online Spoken German Journalistic Texts for the International Public . . . . . . . . . . . . . . . . . . . . . . . Christina Alexandris

551

How the Shapes of School Emblems for Colleges Convey Imagery . . . . . . Mu-Chien Chou

556

Extensible CP-Based Autonomous Search . . . . . . . . . . . . . . . . . . . . . . . . . . . Broderick Crawford, Ricardo Soto, Carlos Castro, and Eric Monfroy

561

XXXVI

Table of Contents – Part I

A Hybrid Approach to User Activity Instrumentation in Software Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Martin Dost´ al and Zdenek Eichler

566

Web Resource Selection for Dialogue System Generating Natural Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Masashi Inoue, Takuya Matsuda, and Shoichi Yokoyama

571

R&D Information System to Support Knowledge Creation . . . . . . . . . . . . Hyojeong Jin, Il Yeon Yeo, Youn-Gyou Kook, Byung-Hee Lee, and Jaesoo Kim

576

A New Method for Designing a Sitemap . . . . . . . . . . . . . . . . . . . . . . . . . . . . Soheila Khodaparasti and Marzieh Ahmadzadeh

580

On-line Handwritten Signature Verification Using Hidden Semi-Markov Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Daw-Tung Lin and Yu-Chia Liao

584

Accessing Previously Shared Interaction States through Natural Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Arthi Murugesan, Derek Brock, Wende K. Frost, and Dennis Perzanowski

590

Japanese Sentence Input Method Using Acceleration Sensor . . . . . . . . . . . Masaki Sugimoto, Kazufumi Nakai, Nobuo Ezaki, and Kimiyasu Kiyota

595

Where to Put the Search Concepts in the Search Result Page? . . . . . . . . . K.T. Tong and Robert W.P. Luk

600

Kansei Modeling on Visual Impression from Small Datasets . . . . . . . . . . . Shunsuke Uesaka, Kazuki Yasukawa, and Toshikazu Kato

605

A Movie Recommendation Mechanism Based on User Ratings in the Mobile Peer-to-Peer Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chian Wang and Dai-Yang Lin

610

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

615

Part I

Novel Interaction Environments

Measuring Human Interaction in Digital Television Using Profiles and Geolocation Valdecir Becker and Marcelo Knörich Zuffo Laboratório de Sistemas Integráveis, Departamento de Engenharia de Sistemas Eletrônicos, Escola Politécnica da USP, Av. Prof. Luciano Gualberto Trav. 3 n. 158, 05508-900 SP São Paulo Brasil [email protected], [email protected]

Abstract. This paper presents an audience measurement system for Brazilian digital free-to-air television, with profiles and geolocation introduction. The work uses activity theory framework for analyzing the context in which the audience and the interaction occur. There are nine user and behavior profiles proposed. The geographic distribution of the audience is shown by maps. Keywords: Audience measurement, profiles, geolocation.

1 Introduction This paper presents an audience measurement system for digital free-to-air TV using profiles and geolocation. Nine user and behavior profiles were developed from the activity theory analysis used in human-computer interaction for analyzing the context in which the interaction occurs, This allowed for a more detailed understanding of the viewers as compared with the traditional audience measurement. In addition, the system provides maps with the behavior and use in front of television. This information is useful in developing the content and precise advertising campaigns. Traditionally, the audience measurement is based on tuned channels, generating data about time before television and most watched channels [1]. The measurement is done by companies like Ibope [2], in Brazil, and Nielsen [3], in United States, which connect audimeter on the receivers [11]. The audimeters get the tuned channel and send the information to a database. This process is limited in scope (total ratings, share, gross rating points and some profiles, such as age and social class) and causes discomfort to the viewer because of the need of an audimeter [1, 4, 10]. The audience measurement system reported in this paper eliminates the need for such audimeters.

2 User Identification With the TV digitalization, one can use software that collects more detailed and precise data, without audimeter use [5,6]. The receiver [7] itself collects and sends C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 3–7, 2011. © Springer-Verlag Berlin Heidelberg 2011

4

V. Becker and M.K. Zuffo

data. In the system developed here, the receiver connects to a database, informing the current channel and the number of people watching television. This article describes the evolution of the system, incorporating behavior and television use profiles in addition to the information related to user's location. For this purpose, the users at the measurement point are registered (Fig. 1). The definition about which households and user should be in the sample is obtained statistically, as typical audience measurement panels. For demonstration purpose, we defined five measurement points with 20 users, distributed among nine profiles, described below.

Fig. 1. Identification screen, which allows the association between the user and the profiles

When switched on, the receiver authenticates on the database using an IP connection, and receives information about the registered users. Then the user identifies itself. The receiver sends the information to the database with the location of it. The database does the association between identification and the profiles stored on it.

3 Geolocation After the user identification, the receiver sends to a database the information with a Postcode number of the locality. The user registers the Postcode with the measurement software at the first time of turning on the TV with the measurement software. With this information, maps are prepared for the audience behavior, relating the location and with the profiles. The maps are generated using a methodology called Geofence [8], which designs a virtual perimeter with smaller maps in geographic areas pre-established by the Postcode number. Thus, it is possible to cross information between the user profiles and the audience, and locate them geographically.

Measuring Human Interaction in Digital Television Using Profiles and Geolocation

5

4 Audience Profiles The profiles were defined based on activity theory, which is employed to interpret the use of technology and to develop systems for human computer interaction. Activity theory is mainly used in the activity-focused design, which starts the analysis not with the user, but with the activities required to complete the tasks. The analysis starts in the context of system use and the task actions, considering the activities involved in the process and the artifacts used in mediation [9]. This is the point where the activity theory becomes relevant to the development of user profiles and behavior with regard to television. The viewer uses technology (artifacts and tools) to establish a relationship with the content within a social context. Artifacts mediate this communication. From this context and such mediation, nine user and behavior profiles were defined, which can be analyzed individually or together. 1. 2. 3. 4. 5. 6.

Sex – male or female composition of the audience Age – how old is the audience on the average Disability profile - the need for accessibility On line profile - internet access and frequency of use TV audience profile – the average time watching TV Commercial break profile - engagement in programming, change the channel during commercial breaks. 7. Crowd profile - how is the TV being watched, alone or accompanied. 8. Incoming profile – social class and average incoming 9. Technological profile - using technologies such as PC, cable TV, DVD player / Blu-ray / VCR, number of TVs The profiles one, two and eight are already used by the measurement institutes, like Nielsen and Ibope. The profiles one, two, three, eight and nine are previously recorded with information provided by users. The profiles four to seven are updated monthly, based on the stored data. These profiles can be applied to both fixed and mobile reception. The main difference between these two modes is the technology of collecting information, which should be transparent to those who access the information. Besides the division into two lines of analysis, based on people who see TV (profiles 1-7) and the environment they occupy (profiles 8 and 9), one can also add profiles to the composition (profiles 1, 2 , 3, 8 and 9) or audience behavior ( profiles 4, 5, 6 and 7). The information provided by the system can be accessed through an IP address. The user can select information by applying filters on the audience, share, on a particular profile or compose maps with profiles and behavior over the time. The intersection of information about audience and the different profiles allows detailed analysis about the composition and behavior of people watching TV. Fig 2 shows an example of the obtained data.

6

V. Becker and M.K. Zuffo

Fig. 2. Example screen with information about profile composition of the audience. In this case, with the Technological profile.

5 Conclusion This work proposes significant reformulation of the way the audience is traditionally analyzed. Currently the most important audience information is only related to the time that the channel is tuned on and the number of people watching it. To this information we have added details of behavior and location, allowing the broadcasters to better plan the TV schedule and providing advertisers better investment targets. The combination of profiles and the geographic area allows us to understand the viewers' tastes and motivations. This understanding is essential to improving the television content.

References 1. Webster, J.G., Phalen, P.F., Lichty, L.W.: Rating Analysis: The Theory and Practice af Audience Research, 3rd edn. Lawrence Erlbaum Associates, Inc., Londres (2006) 2. IBOPE. Instituto Brasileiro de Opinião Pública e Estatística, São Paulo (2010), http://www.ibope.com.br (accessed 01/10/2010) 3. Nielsen: São Paulo (2010), http://en-us.nielsen.com (accessed 01/10/2010) 4. AIMC. El uso de la tecnología en la medición de las audiencias, Spain (2008) 5. Becker, V., Zuffo, M.: Audience measurement system for digital television. In: Adjunct Proceedings - EuroITV 2010, pp. 111–114. Tampere University of Technology, Tampere (2010) 6. Becker, V., Zuffo, M.: Medição de audiência em ambientes de TV digital. In: Proceedings of the XXXIII Congresso Brasileiro de Ciências da Comunicação. Caxias do Sul, RS (2010)

Measuring Human Interaction in Digital Television Using Profiles and Geolocation

7

7. Costa, L.C.P., Herrero, R.A., Nunes, R.P., De Biase, M.G., Zuffo, M.K.: Over the Air Download for Digital Television Receivers Upgrade. IEEE Transactions on Consumer Electronics 56(1) (2010) 8. Munson, J.P., Gupta, V.K.: Location-based notification as a general-purpose service. In: Proceedings of the 2nd International Workshop on Mobile Commerce, Atlanta, Georgia, USA, pp. 40–44 (September 28, 2002) 9. Kuutti, K.: Activity Theory as a potential framework for human-computer interaction research. In: Nardi, B. (ed.) Context and Consciousness: Activity Theory an Human Computer Interaction, pp. 14–44. MIT Press, Cambridge (1995) 10. Balnaves, M., O’Regan, T.: Survey wars: the explosion of measurement in audience ratings. In: Papandrea, F., Armstrong, M. (Org.) Record of the Communications Policy & Research Forum 2009, Sydney, pp. 256–263 (2009) 11. Nielsen: Anytime Anywhere Media Measurement: A Progress Report, USA (June 14, 2007)

PC-Based Warning Mechanism System of Fall Risk in Elderly Chih-Sheng Chang1, Cherng-Yee Leung2, and Jeih-Jang Liou1 1

The Department of Product and Media Design, Fo Gung University, Taiwan 2 The Department of Industrial Design, Tatung University, Taiwan [email protected]

Abstract. It is difficult to recognize and classify movement patterns correctly. We have developed and evaluated a method to classify movement using contact forces during sit-to-stand (STS) movement in elderly people. We used the key points of the ground reaction force (GRF) to identify the sequence of important time points in the STS movement of elderly people. The data showed that the reaction forces had three distinctive patterns: typical-modal (t-modal), multimodal (m-modal), and incompetent-modal (i-modal). This is the first study to identify STS patterns based on objective force data and key points. The STS patterns can be used to supplement the subjective Berg Balance Scale (BBS) score to provide a more precise analysis of STS movement in the elderly. Keywords: BBS, GRF.

1 Introduction The population is aging in all developed countries, and as a result, the number of people requiring additional strategies to remain independent is likely to increase. Performance on STS tests has been found to predict subsequent disability, illness, falls, and hip fractures [4]. Currently, STS movement is investigated using observational performance tests to assess the risk of falling or performance measurements [7]. The observational performance tests, which tend to be subjective, use several assessment tools that combine measures of balance with measures of gait and mobility to determine a person’s risk of falling, e.g., the Berg Balance Scale [2] and the Tinetti Gait and Balance Assessment [8]. The BBS relies on interviews conducted by rehabilitation professionals and was developed as a performance-oriented measure of balance in elderly individuals [1]. Although the BBS has excellent test–retest reliability for the assessment of standing balance, Newton [5] reported a wide range of ability among elderly subjects who had the same mode score on the BBS. This finding suggests that the BBS score does not provide a precise measurement of performance ability. The present study describes the key points of STS movement in elderly people and introduces a PC-based approach to the assessment of STS patterns, with a view to supplementing the subjective BBS to provide a more precise analysis of STS movement in elderly people.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 8–12, 2011. © Springer-Verlag Berlin Heidelberg 2011

PC-Based Warning Mechanism System of Fall Risk in Elderly

9

2 Methods 2.1 Participants Twenty healthy elderly subjects (age, 68.95 ± 4.59) participated in the study; their mean height, weight, and body mass index (BMI) were 158.50 ± 5.85 cm, 59.39 ± 10.24 kg, and 23.55 ± 3.91, respectively. The selection criteria were (1) over 65 years old; (2) no acute medical illness in the past 3 months; (3) no orthopedic diagnosis; (4) no muscular disease; (5) BBS score ≥41 (6) BI score ≥60 (7) MMSE score >17; and (8) IADL score of ≥7. 2.2 Apparatus In the present study, the GRF was defined as the force applied to the ground by the buttocks and/or the feet. Vertical GRFs were recorded from two separate force platforms (size, 500 × 500 mm; accuracy, 0.20 N) composed of eight load cells. A personal computer simultaneously saved the two sets of GRF data at 1000 Hz using a 16-bit analog-to-digital converter acquisition board (NI PCI-6220; National Instruments Inc., Austin, TX, USA) and dynamic amplifiers. 2.3 Procedure The subjects sat on an armless chair of standard height (40 cm) on a buttock-force platform with their arms folded across the chest. A back support on the chair was used to ensure that the subjects’ trunks were leaning back in a standard position, and their bare feet were positioned on the leg-force platform. No other restrictions were imposed on the initial position. Each subject performed the task in a comfortable and natural manner and at a self-selected speed. 2.4 Measurements Figure 1 shows selected GRF parameters. Two curves (B, buttock weight; L, leg weight) describing the vertical GRF as a function of time were obtained from the force platforms. Curve T was the sum of forces B and L. The time taken to stand up was calculated from these curves. Bs, Ls, Ts, the onset of the B, L, and T curves: Tp, Lp, maximal GRF at curves T and L; B0, B at zero. Prior to analysis, the data were smoothed using the moving-average method. The force data indicated that the different sequences of the STS task exhibited specific characteristics that formed distinct patterns. Bs, Ls, and Ts occurred immediately after initiation of the action cue when the difference between the GRF at that time point and the previous value did not equal zero. Seat-off was defined as the time at which the thighs lost contact with the chair, when the instant of curve B was at B0. Tp and Lp occurred when the GRF for curves T and L, respectively, were maximal. The GRF oscillated following seat-off, and the STS phase ended when the GRF reached body weight.

10

C.-S. Chang, C.-Y. Leung, and J.-J. Liou

Fig. 1. The GRF parameters (N15)

2.5 Statistical Analysis The GRF time recordings were transformed logarithmically (Box & Anderson, 1955). A normal distribution test was performed, followed by a one-way ANOVA with repeated measures to evaluate the effect of Bs, Ls, and Ts. Significant results were followed up using the least significant difference (LSD) method as a post hoc test. The mean times of Tp, Lp, and seat-off were compared in a similar manner. A P-value <0.05 was deemed statistically significant.

3 Results The mean time ± the standard deviation (SD) for Bs, Ls, and Ts were 6.40 ± 0.24, 6.52 ± 0.27, and 6.60 ± 0.35, respectively. Different aspects of onset significantly affected the time of the activating moment (F = 5.67, P < 0.05). The LSD indicated that Bs occurred significantly earlier than Ls and Ts, and that Ls and Ts did not significantly affect the activating moment. The mean times for Tp, Lp, and seat-off were 7.53 ± 0.20, 7.59 ± 0.24, and 7.62 ± 0.21, respectively. These different time points significantly affected the time of the activating moment (F = 14.62, P < 0.001). Tp, Lp, and seat-off appeared in sequence; the LSD analyses indicated that the time of Tp, Lp, and seat-off were all significant. Based on the characteristics indicated by the force data, the STS movement sequences were classified into three patterns (Figure 2). The mean BBS score of the t-modal, m-modal, and i-modal patterns were 49.88 ± 2.64, 46.29 ± 4.54, and 43 ± 2, respectively. Figure 3 shows the relationship between the force pattern and the BBS score.

PC-Based Warning Mechanism System of Fall Risk in Elderly

11

Fig. 2. The three characteristic force patterns observed during the STS movement in elderly participants. From left to right: typical-modal (t-modal), multi-modal (m-modal), and incompetent-modal (i-modal).

Fig. 3. The relationship between the force pattern and the BBS score

4 Discussion Our data indicate that Bs is the onset of the STS movement in elderly people and that Ls and Ts are less accurate indices in comparison with Bs. Thus, using Ls or Ts to identify STS onset may underestimate the duration of the STS movement in elderly people. However, Chang and colleagues [3] reported that GRF component Bs and Ls, which were not significant in the present study, could both be used to mark the onset of the STS movement in healthy adults. The age of the participants may explain the difference between our findings and those of Chang. Tp and Lp are the maximum force points of the body and legs respectively. Their physical significance is different from that of seat-off, which represents the moment when the buttock leaves the seat. Although disagreement exists over the order of these three points [6], Chang and colleagues [3] reported the order of Tp, Lp, and seat-off in healthy adults using two high-frequency force platforms. Our findings in elderly subjects are consistent with those of Chang. The force data indicated that the STS movement task could be divided into three distinct patterns. In general, the curves of t-modal pattern are smooth, regardless of the B, L, or T curve. The curves of the m-modal pattern have multi-peak characteristics in curves B, L, and T. The curves of the i-modal pattern oscillate

12

C.-S. Chang, C.-Y. Leung, and J.-J. Liou

widely, particularly the B and L curves. The large amplitude swings in the i-modal pattern were caused by multiple outputs from one subject who tried, but was unable to complete the STS movements. In our study, the relationship between the BBS and STS patterns derived from the objective instrument was linear (Figure 5). Moreover, the SD of the t-modal and i-modal patterns on the BBS score was smaller than that of the m-modal pattern. Thus, the same BBS score may be associated with different STS patterns. For example, a subject with a BBS score of 48 may have shown either a tmodal (smooth movement) or m-modal (unstable movement) pattern, whereas another subject with a BBS score of 44 may have been able to stand up with no assistance (mmodal) or been unable to perform the STS task (i-modal) on the first try. Thus, the objective STS patterns provide a more precise assessment of balance in the elderly than does the more subjective BBS. Aknowledgments. This research was supported by a grant from the National Science Council (NSC99-2221-E-431-002) and (NSC99-2221-E-036-032).

References 1. Berg, K.O., Wood-Dauphinee, S.L., Williams, J.I., Gayton, D.: Measuring balance in the elderly: preliminary development of an Instrument. Physiotherapy Canada 41, 304–311 (1989) 2. Berg, K.O., Maki, B.E., Williams, J.I., Holliday, P.J., Wood-Dauphinee, S.L.: Clinical and laboratory measures of postural balance in an elderly population. Archives of Physical Medicine and Rehabilitation 73, 1073–1080 (1992) 3. Chang, C.S., Leung, C.Y., Liou, J.J., Tsai, W.W.: Evaluation of key points in the sit-tostand movement using two force platforms. Perceptual and Motor Skills 111(2), 496–502 (2010) 4. Lipsitz, L.A., Jonsson, P.V., Kelley, M.M., Koestner, J.S.: Causes and correlates of recurrent falls in ambulatory frail elderly. Journal of Gerontology 46, M114-M122 (1991) 5. Newton, R.A.: Balance screening of an inner city older adult population. Archives of Physical Medicine and Rehabilitation 78, 587–591 (1997) 6. O’meara, D.M., Smith, R.M.: The effects of unilateral grab rail assistance on the sit-to-stand performance of older aged adults. Human Movement Science 25, 257–274 (2006) 7. Rogers, M.E., Rogers, N.L., Takeshima, N., Islam, M.M.: Methods to assess and improve the physical parameters associated with fall risk in older adults. Preventive Medicine 36(3), 255–264 (2003) 8. Tinetti, M.E., Williams, T.F., Mayewski, R.: Fall risk index for elderly patients based on number of chronic disabilities. The American Journal of Medicine 80, 429–434 (1986)

System of Systems for Sensor and Actuator Networks Tiffany Elise Chua, Mark Merlo, and Mark Bachman University of California, Irvine, 3312 Engineering Gateway, Irvine, CA 92697 {tchua,mmerlo,mbachman}@uci.edu

Abstract. Sensor and actuator networks are often the backbone of control applications. They are used in many different fields, such as health care, home automation, and industrial control. Despite the prevalence of sensor and actuator networks, there is a lack of tools to support the rapid prototyping of sensing and control applications. We have developed a system to support a generic platform for rapid prototyping of sensing and control applications in which different sensors and actuators are brought together to perform specific functions. At the heart of the system-of-systems is the Control INterface to Devices and Instruments (CINDI), a small Linux-based box with a number of standard interfaces. The system was built with simplicity in mind and to support component reuse. Keywords: system of systems, sensor and actuator networks, rapid prototyping, pervasive computing, ubiquitous computing.

1 Introduction Many applications involve utilizing a controller to process inputs from various sensors and send outputs to various actuators. Such systems can be viewed as a “systemof-systems” of sensors and actuators, which on their own may be operational, but are networked together to perform a common specific function. For example, Hata et al [1] built systems of systems for elderly health care to prevent potential patient falls by producing an alert whenever the patient is about to get out of the bed. An ultrasonic transducer detected the presence or absence of a patient in bed, and by an air pressure sensor with air tube embedded in the mattress monitored the heart rate of the patient. Used together, the sensors provided a fairly accurate method of detecting when a patient is about to get out of bed using a fuzzy logic controller with two inputs. In industrial automation, systems of systems are commonly used for data acquisition, equipment monitoring, advanced process control and complex machine control. Although a simple programmable logic controller may do the job when the humanmachine interface only includes simple buttons and lights, an advanced system of systems for industrial control may be able to provide higher level data handling and higher processing speeds, with more extensive communications capabilities and better human machine interfaces. The development of custom-built or dedicated system of systems involving sensor and actuators usually requires microprocessor programming and communication C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 13–17, 2011. © Springer-Verlag Berlin Heidelberg 2011

14

T.E. Chua, M. Merlo, and M. Bachman

protocol development. In addition, custom-built systems typically consume a considerable amount of time and effort in maintenance. A number of platforms have been proposed to shorten the development cycle by providing modular and reusable components. Phidgets, the hardware equivalent of widgets, have been developed to provide prebuilt programmable modules such as simple I/O controllers, servo controllers, and proximity sensors for PC-based control [2,3]. Various phidgets can be combined to create a system of systems. Another example is the Atlas Sensor Platform, which consists of both hardware and middleware. It provides a hardware node built with a stack of interchangeable layers – the device layer, the processing layer and the communication layer. This allows the various types of devices to be connected and provides various options for connecting to the controller that resides in the middleware. The systems have been applied to various fields such as home automation [4] as well as industrial applications [5]. Both systems provide modularity and extensibility, and allow the user to setup a custom controller for a networked system of sensors and actuators through an application programming interface and specify the algorithms for processing inputs and producing outputs. In control applications, several types of controllers are commonly used. For example, aside from simple logic or sequential controllers, the proportional-integralderivative (PID) controller is one of the most common. With a few parameters, namely the variable set point, proportional, integral and derivative constants, a PID controller can be configured to bring an input variable to its set point by adjusting a manipulated variable based on the error difference between the input(s) and set point in a feedback loop. The PID controller can be cascaded in an application with multiple inputs, or built in parallel for multiple outputs. Fuzzy logic controllers are also commonly used to combine logic with linear control. Other frequently used components include data loggers, indicators, and alarms.

2 System Design and Implementation Here we discuss the development of a system to support a system of systems of sensor and actuators. The system is designed to be easy to use, with no programming required on the part of the user to get running. The system provides pre-built controllers for common tasks. To use, users simply specify the inputs and outputs of the controllers. In the following sections, we describe the system design and architecture of a Control INterface to Devices and Instruments (CINDI). 2.1 Block Diagram The block diagram is shown in Fig. 1. The system consists of CINDI middleware running in a Linux-based operating system. The system has a number of USB ports through which motes can be directly connected. The system is designed for hierarchical control, with CINDI at the top of the hierarchy, motes passing data to and from CINDI, and sensors, actuators and other devices connected to motes. Currently, only CINDI motes are supported, but future versions may allow the use of custom devices.

System of Systems for Sensor and Actuator Networks

15

CINDI

Mote 1

Mote n Mote 2

Sensor

Actuator Sensor

Actuator

Fig. 1. Block diagram of a system of systems for sensors and actuators

2.2 Architecture The architecture of CINDI is shown in Fig. 2. The top box represents CINDI itself, and the bottom box represents a CINDI mote connected to sensors and/or actuators. In the top box, the interface layer shows the various types of connections possible. At present, only the USB-Serial interface is implemented. The OS driver layer is provided by the Linux operating system. The device layer contains the input and output handlers. Information on device capabilities and how to receive and send data from and to the device resides in the configuration database. The controller layer contains controllers for input and output devices from the device layer. Controller configurations are also stored in the configuration database. A configuration tool is provided to configure devices and controllers in the system. CINDI Controller Layer Controller 1

Controller 2

Controller n

Config Database

Device Layer Dev 1

Dev 2

Dev 3

Dev n

OS Driver Layer USB Driver

Bluetooth Driver

USB

Bluetooth

Zigbee Driver

...

Interface Layer Zigbee

...

Config Tool TCP/IP

USB Application Layer ADCs / DACs

Mote

Sensors Actuators

Fig. 2. CINDI Architecture

16

T.E. Chua, M. Merlo, and M. Bachman

The bottom box shows the architecture of a CINDI mote. The mote has a USB interface, configured as a serial communication port. A PIC-based microcontroller on the mote receives data from and sends data to built-in analog-to-digital converters on the chip. Sensors and actuators connected to the CINDI mote. 2.3 Use Cases Use cases are shown in Fig. 3 to describe the operation of the system. To configure the CINDI box, an administrator selects the controller to be used for the CINDI mote. Controller setup involves creating a controller from one of the pre-built controller templates, and setting up the input, output, and timing. The controller can be configured to begin to operate at startup or to operate only when manually started. Once configured, the system is ready to use. A user may manually start a controller that is not configured to begin at startup.

Fig. 3. CINDI Use Cases

3 Summary and Future Work Designing a control system for a wide variety of system of systems is a challenging problem. The type of controller needed is different for each application domain. Simple controllers were built in this example, but the system can be extended in the future to provide different types of single-input-single-output (SISO), single-input-multipleoutput (SIMO), multiple-input-single-output (MISO), and multiple-input-multipleoutput (MIMO) controllers. The architecture of CINDI makes it suitable for hierarchical control, in which CINDI is at the top of the hierarchy and sensors and actuators communicate to each other through CINDI. Although the system is simple and does not support real-time applications, it is easy to use and setup for rapid prototyping. Currently the system only supports sensors and actuators connected through USB-based CINDI motes. Future versions may allow the user to configure the system to support custom devices or motes with a defined I/O protocol for reading/writing data through other types of interfaces such as Bluetooth and Zigbee.

System of Systems for Sensor and Actuator Networks

17

References 1. Hata, Y., Yamaguchi, H., Kobashi, S., Taniguchi, K., Nakajima, H.: A Human Health Monitoring System of Systems in Bed. In: IEEE International Conference on System of Systems Engineering, pp. 1–6 (2008) 2. Greenberg, S., Fitchett, C., Tn, C.: Phidgets: Incorporating Physical Devices into the Interface. In: CHI (2001) 3. Marquardt, N., Greenberg, S.: Distributed physical interfaces with shared phidgets. In: Proceedings of the 1st international conference on Tangible and embedded interaction - TEI 2007, ACM Press, New York (2007) 4. Bose, R., King, J., Pickles, S., Elzabadani, H., Helal, A.: Building Plug-and-Play Smart Homes Using the Atlas Platform. In: Proceedings of the 4th International Conference on Smart Homes and Health Telematic (ICOST), Belfast, the Northern Islands (2006) 5. Sollacher, R., Niedermeier, C., Vicari, N., Osipov, M.: Towards a Service Oriented Architecture for Wireless Sensor Networks in Industrial Applications (2007)

Smart Clothes Are New Interactive Devices Gi-Soo Chung1 and Hee-Cheol Kim2,* 1

Textile Fusion R&D Department, Korea Institute of Industrial Technology (KITECH), Ansan, Gyeonggi-Do, Korea 2 School of Computer Engineering/UHRC, Inje University, Obang-Dong 607, Gimhae, Gyeong-Nam, Korea [email protected], [email protected]

Abstract. Clothes are more than wears. They are interactive wearable devices. Recently, smart clothes have gained interest among researchers and practitioners, and shown us the potential as comfortable mobile computers in a ubiquitous world. This paper presents the two cases where the digital garment that we developed have been tested and applied to: communication and video display, and healthcare. While recognizing that the garment is still insufficient for ordinary users, we hope that its problems will be gradually resolved with the technological development. Keywords: Digital yarn, interactive device, smart clothes, wearable computing.

1 Introduction Worn on the user’s body, wearable computers should be equipped with four elements: context awareness, minimal cognitive overload, no user interaction with certain tasks, and integration with outfit [1]. Smart clothes are natural devices with which we wear for 24 hours, and best fitted to fulfill the elements. In this paper, we describe our digital garment and the two cases where it has been tested. Even if its development is at an experimental stage and tests are small scaled, we suppose that the cases can provide some hints and directions for further development.

2 Digital Garment The garment consists of both ordinary yarns and digital yarns enabling data transmission, as well as different electronic modules including sensors. Therefore, the development of the digital garment demands fusion technology of fiber engineering and information technology. Figure 1 shows an overview of the digital garment that we developed. *

Corresponding author of the paper.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 18–21, 2011. © Springer-Verlag Berlin Heidelberg 2011

Smart Clothes Are New Interactive Devices

19

Fig. 1. An overview of the digital garment

Among many technologies required, the production of digital yarns is most crucial. The first author of the paper has developed the digital yarn [2]. Its major material is copper alloy, which means it has conductivity. Therefore, the data can be transmitted through it. The yarn in the clothes transfers both analog and digital signals, and is divided into two parts (see Figure 2), core and outer parts. Its core part consists of 7 micro wires and a special resin. Its outer part is covered with dyed normal yarn. In particular, electric resistance of the yarn is much lower, 7.5 Ω/m, than previous conductive yarns [3-5].

Fig. 2. Digital yarn developed by Dr. Chung

20

G.-S. Chung and H.-C. Kim

3 Two Cases 3.1 Communication and Video Display The digital garment is made by ordinary yarns on the one hand, and digital bands on the other hand for the data transmission. Here, a digital band is a set of 10-30 digital yarns that are used for a communication line. Figure 3 shows us a testing environment. A CCD camera to record scenes is connected with a digital band by which recorded videos are transmitted real-time to an LCD monitor. This confirms that digital yarns transfer the video data properly without transmission delay. In fact, the digital yarn’s transmission speed is very high. It is about 80 Mbps. It implies that an 800MB movie file can be transmitted within one and a half minute approximately. However, we also have to admit of inconvenient aspects in the garment. The monitor in the garment is too heavy to carry all the time. The connectors between digital bands and ordinary yarns are a bit big so that users can feel uncomfortable when they wear the garment. However, we learned that this environment could be used for special tasks where inconvenience might be endured, rather than for ordinary lives.

Fig. 3. Video display with the digital garment

3.2 Healthcare The digital garment with biosensors to measure ECG (Electrocardiography) was tested for the purpose of healthcare. Here, the ECG signals acquired from the sensors are transmitted to the monitor through the digital yarns so that the user could see them in the monitor. The result was promising that the signals were accurate enough when compared with the data acquired without the garment. Thus, we found the huge potential for health management with the digital garment. However, we still need a technological advance to develop more comfortable and non-invasive sensors when the clothes are fitted with them. We are working in this direction [6].

Smart Clothes Are New Interactive Devices

21

Fig. 4. Measuring ECGs with the digital garment

4 Conclusion This paper gave a brief introduction to the digital garment, and presented the two cases in terms of its usage. The purpose for testing it in the cases was not to gain nor evaluate usability, but to ascertain the potential of new interactive clothes. Whereas technological advancement is requested, we believe that the digital garment will become a wearable computer in practice in a near future. Acknowledgments. This work is funded by the Korean Ministry of Knowledge Economy (#10033321).

References 1. Pezzlo, R., Pasher, E., Lawo, M.: Intelligent Clothing. IOS Press, Amsterdam (2009) 2. Chung, G.S., An, J.S., Lee, D.H., Hwang, C.S.: A Study on the Digital Yarn for the High Speed Data Communication. In: The 2nd International Conference on Clothing and Textiles, pp. 207–210 (2006) 3. Textile Wire version 03.01-e, Elektro-Feindrhat-Ag, Switzerland 4. Bekaert (a firm in Belgium), Bekinox VN 12/2x275/175S/316 L/HT, http://www.swicofil.com/bekintex.html 5. Linz, T., Kallmayer, C., Aschenbrenner, R., Reichl, H.: Embroidering Electrical Interconnects with Conductive Yarn for the Integration of Flexible Electronic Modules into Fabric. In: IEEE International Symposium on Wearable Computing, Osaka, Japan, (October 19–21, 2005) 6. Kim, H.C., Chung, G.S., Kim, T.W.: A Framework for Health management services in nanofiber technique-based wellness wear systems. In: IEEE Healthcom 2009, Sydney, Australia, pp. 70–73 (December 16–18, 2009)

Ebook Readers: An iPod for Your Books in the Cloud Ann-Marie Horcher and Maxine Cohen Nova Southeastern University, USA [email protected]

Abstract. The new way to carry books or literature is to not carry it at all. Not only have computing resources and data moved to the cloud, so has the latest novel or self-help book. The key to the acceptance of this technology is usable design that anticipates a reader’s needs while still invoking a comfortable form factor – the book [1]. Though e-book readers have been available for a decade, the latest wave of e-book readers has achieved a new level of technology acceptance. This trend and potential acceptance of future models can be understood by applying task-technology fit theory [2] to a usability study of the Barnes and Noble Nook and Amazon Kindle ebook readers. Keywords: ebook, digital libraries, usability, task-technology fit, gender bias.

1 Introduction Computing resources are moving to the cloud [3], along with digital repositories of information [4]. Beyond simply replicating the content of books, the ebook evolves that repository to a new form that retains the value of the book to deliver organized units of information, but has the flexibility for new paths for access [5]. The ebook has also transformed the previous form factor of books with variable weight and size to a totally mobile, but predictable shape [6]. Beyond the predictable size of the device, the content on the device has the advantage of flowing between a series of interoperable devices [7]. The typical mobile user is manipulating a group of on-body devices [8]. Initially the development of digital libraries focused on technology at the expense of usability [9]. Similar to the problems experienced in the design of secure systems [10], usability, like security, cannot be tacked on at the end. The greater problem was not whether digital repositories could be built, but whether a user community would accept online reading. Even younger segments of the population, such as university students [11], have traditionally preferred print. The technological limitations of computer screens created a usability barrier to the acceptance of online reading [5]. Even though the superiority of the ebook for sharing and delivery was acknowledged, the eye fatigue from prolonged on-screen reading, as in reading an entire book, made the format undesirable. The recognition of the impact of typography upon the readability of text preceded the evolution to the ebook [12]. The presentation of text information on a screen with less resolution than paper exacerbated the issue [13], resulting in decades of studies recommending guidelines for formatting screen text [14, 15]. Still the gold standard for clarity remained the black on white configuration of printed text [16]. The screen size limitations of small-screen C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 22–27, 2011. © Springer-Verlag Berlin Heidelberg 2011

Ebook Readers: An iPod for Your Books in the Cloud

23

devices like PDAs made clarity of display even more critical to usability [17]. Because of portability similar to the printed book, the small-screen devices became a natural target for the ebook. The navigation of material was second barrier. Moving through the electronic version of information was slower, and the users could not easily find certain phrases, or flip to other sections as they could using a printed version [5]. Once again, the form factors of small screen devices were a critical factor to ebook acceptance, particularly for the recreational reader. Using alternative navigation techniques such as grab-and-drag on a touch screen instead of a keyboard improved the user experience for interaction with large tables and maps [18]. The navigation experience paralleled the user experience with a physical book. Similarly, navigation of an ebook needed to equal the convenience of flipping through a physical book, or jumping to a section based on the table of contents.

2

Planning

The usability testing focused on three major task workflows commonly performed by a reader interacting with text material regardless of the storage media. These major areas are shown in Figure 1.

Fig. 1. Typical workflows for ebook reader usage

The user must progress through the previous capabilities to succeed at the tasks at the higher levels. The first two sections have close equivalents in the physical book. The enhanced capabilities address areas where the ebook has a design advantage over the printed counterpart. The test goals are to achieve feedback in the following areas that have been documented in the literature as usability barriers to ebook adoption. 2.1 Test Methods A comparison of usability between the ebook readers was chosen because this most directly parallels the user experience in field when selecting this capability. Experimental design in usability studies is as critical as the design of the interface being evaluated [19]. Small errors in design, such as confirmation bias in the task workflow [20] can overshadow the results. The replication of the normal environment of the user provides context for the usability and influences the results [21, 22]; therefore the usability testing was done in the field at various locations on a college campus with Internet availability.

24

A.-M. Horcher and M. Cohen

2.2 Participants Participants were required to have a computer and use it regularly. This established a minimal expertise level. The study participants were selected to represent a range of demographic groups in terms of age, sex, education, and computer expertise. The subjects were also selected for their interest and willingness to try out ebook readers. Table 1. Ebook reader usability barriers Usability Barrier Screen readability Navigation

Portability/Physical

Network connection

Literature Reference Kang, Y.-Y., Wang, M.-J. J., & Lin, R. (2009). Usability evaluation of E-books. Chen, N., Guimbretiere, F., Dixon, M., Lewis, C., & Agrawala, M. (2008). Navigation techniques for dual-display e-book readers. Kang, Y.-Y., Wang, M.-J. J., & Lin, R. (2009). Usability evaluation of E-books. Churchill, D., & Hedberg, J. (2008). Learning object design considerations for small-screen handheld devices. Walker, G. H., Stanton, N. A., Jenkins, D. P., & Salmon, P. M. (2009). From telephones to iPhones: Applying systems thinking to networked, interoperable products. Serif, T., & Ghinea, G. (2008). Mobile information access in the real world: A story of three wireless devices.

2.3 Procedure The participants performed a list of tasks with each ebook reader. Each task tested an aspect of ebook reader usability as described by the usability barriers in Table 1. The study compared the Barnes & Noble Nook and the Amazon Kindle 3. 2.4 Measurements As part of the entry questionnaire the subject was asked to assess their attitude toward ebook readers in a three point Likert scale. The same question was asked at the conclusion in an exit questionnaire. The overall time to complete all tasks was recorded, as well as number of times the user requested assistance. The observer noted the comments made by user while thinking aloud for later analysis. The tasks for which the subject requested assistance were also noted. The time and date of session was recorded. At the conclusion of the session, the observer interviewed the subject to get a usability rating for every task, and for each ebook reader. In addition to giving a rating on a 5 point Likert scale for 16 tasks on each ereader, the subject was also asked for an overall preference of Barnes &Noble ereader or Amazon Kindle ereader.

3 Results The selection of preferred software was not consistent with the software receiving the higher composite score of usability. In cases where it was inconsistent, the subjects

Ebook Readers: An iPod for Your Books in the Cloud

25

stated that one usability characteristic was so overwhelming that it out-weighed the other considerations. In some cases this overwhelming reason resulted in the selection of the Amazon Kindle. In the others, it resulted in the selection of Barnes &Noble. The subjects mentioned they liked how the ebook looked on the screen, and used that criteria alone to make their choice. This is an indication that the screen usability may have greater weight than the other usability barriers. Of the five usability barriers, the tasks related to navigation received the lowest scores. No subject felt navigation deserved the highest score. In contrast, the portability and screen readability tasks received the highest ratings for both ebook readers. This suggests that the use of e-ink has resolved the readability problem, even where the layout of the screens might create an accessibility issues. The ability of users to move around the book and manipulate the information needs further refinement to reach achieve the level of usability desired by the typical subject.

4 Discussion and Conclusions In addition to the results from the questionnaires, observations of each subject also yielded some interesting points. Neither age, nor computer expertise was a predictor or how the subjects perceived usability. Across the board, though, women reacted differently to the technology. There was greater anxiety among the female subjects about whether they were performing the test correctly. Furthermore, due to the design of the ebook readers, female subjects did not get a success experience when using the device correctly. The buttons were too stiff to respond to the typical feminine pressure, touch screens did not perceive the stroke of the women’s fingers. This suggests the devices are not calibrated to a norm that includes both genders. The impact of the device not responding is two-fold. The usability becomes suspect, and the subject loses confidence in their ability to control the device [23]. When linked to research that notes the impact of social-cultural issues on early adoption [24], this lack of usability to one gender points to a bias in design. Since men dominate the technology fields [25], and the early adopters [26], it would follow that first generations of device design reflect a male aesthetic. This is an area for further examination. Women also reacted differently to the size and shape of the device. In each case before beginning the task list, the female subjects examined the size, shape and texture of the device. Though not measured specifically in the results, this reaction to texture and color suggests there may some usability issues related to whether the subject feels they can hold the device comfortably.

References 1. Tsakonas, G., Papatheodorou, C.: Exploring usefulness and usability in the evaluation of open access digital libraries. Information Processing & Management 44(3), 1234–1250 (2008) 2. Goodhue, D.L., Thompson, R.L.: Task-Technology Fit and Individual Performance. MIS Quarterly 19(2), 213–236 (1995)

26

A.-M. Horcher and M. Cohen

3. Khmelevsky, Y., Voytenko, V.: Cloud computing infrastructure prototype for university education and research. In: Proceedings of the 15th Western Canadian Conference on Computing Education, pp. 1–5 (2010) 4. Kratky, A.: Re-thinking reading in the context of a new wave of electronic reading devices. In: Proceedings of the First international conference on Human-computer interaction, tourism and cultural heritage, pp. 1–11 (2011) 5. Kang, Y.-Y., Wang, M.-J.J., Lin, R.: Usability evaluation of e-books. Displays 30(2), 49– 52 (2009) 6. Golovchinsky, G.: Reading in the office. In: Proceeding of the 2008 ACM workshop on Research advances in large digital book repositories, pp. 21–24 (2008) 7. Walker, G.H., et al.: From telephones to iPhones: Applying systems thinking to networked, interoperable products. Applied Ergonomics 40(2), 206–215 (2009) 8. Ashbrook, D., Lyons, K.: Ensembles of on-body devices. In: Proceedings of the 12th international conference on Human computer interaction with mobile devices and services, pp. 503–504 (2010) 9. Blandford, A., et al.: Disrupting digital library development with scenario informed design. Interacting with Computers 19(1), 70–82 (2007) 10. Baskerville, R.: Information systems security design methods: implications for information systems development. ACM Comput. Surv. 25(4), 375–414 (1993) 11. Moyer, J.E.: Teens today don’t read books anymore: a study of differences in interest and comprehension in multiple modalities. In: Proceedings of the 2011 iConference, pp. 815– 816 (2011) 12. Tinker, M.A., Paterson, D.G.: Studies of typographical factors influencing speed of reading. VII. Variations in color of print and background. Journal of Applied Psychology 15(5), 471–479 (1931) 13. Hassan, T., Hu, C., Hersch, R.D.: Next generation typeface representations: revisiting parametric fonts. In: Proceedings of the 10th ACM symposium on Document engineering, pp. 181–184 (2010) 14. Garcia, M.L., Caldera, C.I.: The effect of color and typeface on the readability of on-line text. Computers & Industrial Engineering 31(1-2), 519–524 (1996) 15. Humar, I., Gradisar, M., Turk, T.: The impact of color combinations on the legibility of a Web page text presented on CRT displays. International Journal of Industrial Ergonomics 38(11-12), 885–899 (2008) 16. Scharff, L.V., Ahumada, A.J.: Why is light text harder to read than dark text? Journal of Vision 5(8), 812 (2005) 17. Churchill, D., Hedberg, J.: Learning object design considerations for small-screen handheld devices. Computers & Education 50(3), 881–893 (2008) 18. Burigat, S., Chittaro, L., Gabrielli, S.: Navigation techniques for small-screen devices: An evaluation on maps and web pages. International Journal of Human-Computer Studies 66(2), 78–97 (2008) 19. Gray, W.D., Salzman, M.C.: Damaged merchandise? a review of experiments that compare usability evaluation methods. Hum.-Comput. Interact. 13(3), 203–261 (1998) 20. Brown, S.A., et al.: Expectation confirmation: An examination of three competing models. Organizational Behavior and Human Decision Processes 105(1), 52–66 (2008) 21. Maguire, M.: Context of Use within usability activities. International Journal of HumanComputer Studies 55(4), 453–483 (2001) 22. Kay, J.: A test-first view of usability. Interacting with Computers 21(5-6), 347–349 (2009)

Ebook Readers: An iPod for Your Books in the Cloud

27

23. Hurtienne, J., et al.: Physical gestures for abstract concepts: Inclusive design with primary metaphors. Interacting with Computers 22(6), 475–484 (2010) 24. Park, S., Yoon, S.-H.: Separating early-adopters from the majority: The case of Broadband Internet access in Korea. Technological Forecasting and Social Change 72(3), 301–325 (2005) 25. Todd, K., Mardis, L., Wyatt, P.: We’ve come a long way, baby!: but where women and technology are concerned, have we really? In: Proceedings of the 33rd annual ACM SIGUCCS fall conference, pp. 380–387 (2005) 26. Chau, P.Y.K., Lung Hui, K.: Identifying early adopters of new IT products: A case of Windows 95. Information & Management 33(5), 225–230 (1998)

The Ambient from the Young Passengers' Perception in the Carriage of Taiwan High Speed Rail Jeichen Hsieh and Chan Yo Shan Tung Hai University, Dept. of Industrial Design, Taiwan [email protected], [email protected]

To explore the Taiwan High Speed Rail passenger experience of environmental attitudes, the concept car as a potential environmental factors (Table 1) to Hershberg & Cass environmental significance of the semantic scale (Table 2) (Xu Leiqing, 2004) as Measurement tools of environmental factors in the compartment of the bipolar adjective semantic differential method is usually the number must be used about 10 to 30 is more suitable, it will be adjusted to 2to 3 group to obtain more stable reference. Online questionnaire with Taiwan High Speed Rail train running video will be a reference tool to assist measurement. The video contents directly play to compare the experience feedback of fifty passengers.

Table 1. Potential environmental of high-speed rail travel

Potential environmental factors Passenger vehicles

乘乘

Changes in acceleration and deceleration Stationary traffic Speed Air Quality Temperature and humidity Path Noise Light Seating range Color

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 28–30, 2011. © Springer-Verlag Berlin Heidelberg 2011

The Ambient from the Young Passengers' Perception

29

Table 2. Scale selection

Concept Overall rating Evaluate the effectiveness Evaluation of the unique aesthetic Activities Space Effect Neat and clean Organization Temperatures Bright

Main scale

－the bad of useful － useless unique－ plain positive － negative small － spacious ugly － fine clean － dirty casual －mess warm － cold light － dark good

Alternative scale

－unpleasant friendly － hostile interesting － boring complex － simple private － public rough － smooth well organized － messy formal －free hot － cool cheerful － boring pleasant

The Carriage and Use of Environmental Objects According to the Taiwan High Speed Rail High Speed Rail website Inside information on the object for investigation of high-speed rail passengers traveling within the scope of physical activity when environmental objects, finishing as the standard range of cars and commercial cars (Table 2.3), does not contain a barrier-free space, toilets, vending machines, public telephones. Environment provides an object inside a passenger seat on the journey to provide the needs of Taiwan High Speed Rail interior finishing environmental objects such as table 3. Research show that although many high-speed rail seat space to facilitate the design and intimate personal use universal design considerations, there are still inadequacies in the actual use of passengers when the environment was due to restrictions that could result in the occurrence of passenger interference between the activities , are summarized as follows: 1. Table design in the seat behind the current seat back angle adjustment, back seat passengers to use the table immediately affected, and the appropriate table can not be smoothly adjusted. 2. For standard car seats configured as 3 +2, crowded, passengers will feel sitting in the middle than the narrow space available. 3. Standard Cabin narrow the distance between each seat, so passengers to use the table, almost the desktop can be temporary seat at a glance. 4. Seat can be rotated 180 degrees relative to facilitate understanding of the passengers sit, also happens to be equal to all the people are no table can be used.

30

J. Hsieh and C.Y. Shan Table 3. Environmental Investigation Picture

Passenger Information System

Standard class seats

Business class seats

Function LED electronic display boards to provide passengers the train station and meteorological information. Standard car seats configured as 2 +3 seating, seat back angle can adjust, and the seat can be rotated 180 degrees, seat length of 54cm, seat width 43cm, 55cm. Business Class seats for the 2 +2 seating configuration, seat back angle adjustment to fit passenger demand, the seat can be rotated 180 degrees, seat length 54cm, seat width 47cm, seat 57cm. Seats are provided for each seat dining table, maximum weight 10kg

Picture

Function Hidden hangers for objects, maximum weight 2kg

Hidden hangers Seat reading light at the top, providing visitors to read as the auxiliary light Reading light

Pedal

Pedal line with ergonomic design to provide passenger leg room for stretching and support

Personal music listening system allows travelers to choose five music stations Music system

Power outlets

Note: From the Taiwan High Speed Rail, http://www.thsrc.com.tw

Power outlet facilities enable travelers to use all kinds of electronic products such as notebook computers, PDA, etc.

An Effective Disaster Evacuation Assist System Utilized by an Ad-Hoc Network Yasuki Iizuka1, Kyoko Yoshida2, and Kayo Iizuka2 1

School of Science, Tokai University, Kanagawa, Japan [email protected] 2 School of Network and Information, Senshu University, Kanagawa, Japan {k-yoshida,iizuka}@isc.senshu-u.ac.jp

Abstract. This paper proposes a system that supports effective evacuation from danger using an original Distributed Constraint Optimization Problem (DCOP) algorithm DiSTaS-Anne (Distributed Tabu Search with simulated Annealing). The use of DCOP facilitates the assisted optimization of people’s evacuation timing, by estimating the location of evacuees. This system enables assistance in terms of evacuation guidance to be given to relieve congestion, by calculating evacuation routes via an ad-hoc network of evacuees’ mobile devices (phones, PCs, etc.), intercommunication function and location information. Keywords: optimization, Distributed Constraint Optimization Problem, disaster evacuation.

1 Introduction At times of disaster, or immediately prior to such periods, smooth evacuation is one of the key issues. However, it is also rather difficult to achieve, because people tend to rush forward when faced with disaster. Crowded evacuation passageways of buildings in the event of fire, or roads congested with cars containing people fleeing from predicted hurricanes are examples. Conversely, sometimes people do not take evasive action and do nothing based on the effect of mass psychology. Traffic jams occur when many people or vehicles attempt to traverse a limited route simultaneously and moreover, result in an evacuation time that is longer than would otherwise be required. By making each individual wait for a while to ensure an acceptable flow, avoiding traffic jams becomes feasible. The system the authors are proposing in this paper facilitates the assisted optimization of people’s evacuation timing, by estimating the location of evacuees. This system enables assistance to be given in the form of evacuation guidance to relieve congestion, by calculating evacuation routes via an ad-hoc network of evacuees’ mobile devices (phones, PCs, etc.), intercommunication function and location information. The requirements for the mobile devices are to be equipped with wireless LAN (Wi-Fi). Locations of evacuees are estimated by mobile devices using the positional relationship between the device and the wireless base station. The evacuation

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 31–35, 2011. © Springer-Verlag Berlin Heidelberg 2011

32

Y. Iizuka, K. Yoshida, and K. Iizuka

route and timing of each evacuee are calculated by distributed processing using evacuees’ devices connected by an ad-hoc network. Some related works concerning disaster evacuation simulation systems exist using multi agent, which tend to be focused on pre-existing simulation. The focus of this paper is on evacuation guidance at the time of disaster.

2 Issues about Disaster Evacuation When disaster occurs, to facilitate evacuation, information on the current situation of each site is useful [1], hence the authors’ proposal for a real-time disaster situation mapping (RDSM) system. Knowing “which evacuation route is safe” (by using this system or not), the next issue is “how to evacuate safely and effectively.” However, smooth and effective evacuation is not always easy. People tend to rush forward to passageways which are perceived to be safe, which results in congestion. The more people rush, the greater the congestion. The provision of additional appropriate information concerning evacuation guidance may make it feasible to evade congestion, and successfully shorten evacuation times. The requirement and process for the additional appropriate information concerning evacuation guidance is as follows: • • •

Determining people’s individual locations. Plan regional and short-term range evacuation. Provide information about appropriate evacuation routes and appropriate evacuation timing.

Usually, disaster countermeasures offices will be set up in organizations, in order to determine and provide appropriate evacuation routes. However, planning and providing appropriate information rapidly is not always easy. In addition, such countermeasures offices have limited resources. An effective disaster evacuation assistance system must cope with these issues.

3 Disaster Evacuation Assist System 3.1 Major Function The mobile devices in the system estimate their location via the global positioning system (GPS) or their relative position from Wi-Fi base stations, and communicate with devices in their vicinity via an ad-hoc network. Using the latter, devices autonomously bring up evacuation plans using preset knowledge inputs of evacuation and the current situation. Plans would be devised considering traffic of evacuation routes of places such as buildings, related constraints, and the need to avoid congestion. Evacuation routes and timing are abstracted from the plans and will be displayed on users’ (evacuees’) devices.

An Effective Disaster Evacuation Assist System Utilized by an Ad-Hoc Network

33

Receive Wi-Fi signals

Estimate location

Solve distibuted constraint optimization problem

Fig. 1. Usage Image of the Disaster Evacuation Assist System

3.2 System Features Features of the disaster evacuation assist system are listed below: •

•

•

Makes it feasible to swiftly calculate optimized solutions and use fewer computer resources: By using distributed constraint optimization problems (DCOP), which involve problem-solving logic used under a decentralized environment, it becomes feasible to solve large scale problems though the calculation amount of each agent (devices, in this case) is relatively small. For disaster evacuation, optimized solutions must be swiftly obtained, hence the authors propose the application of DiSTaS-Anne (Distributed Tabu Search with simulated Annealing) for computation logic. The details of DiSTaS-Anne are described in the next chapter. Makes it feasible to estimate location automatically: As for outdoors, estimating the location (degree of latitude and longitude) is feasible by using GPS. Mobile devices which are GPS-equipped are common these days. As for indoors, location information can be specified based on the relative position to Wi-Fi base stations, and enabled by matching the radio field intensity of Wi-Fi signals from at least two base stations. This scheme is in practical use such as PlaceEngine [6]. Usable in everyday situations (not just during disasters): This system is realized as an example of application software of mobile devices. By installing it on devices that users carry around every day, it can be used in emergencies.

4 Scheme for Calculating Appropriate Evacuation Route and Timing (DiSTaS-Anne) To ensure an appropriate evacuation route and timing, the authors propose the application of DiSTaS-Anne (Distributed Tabu Search with simulated Annealing) for a disaster evacuation assist system.

34

Y. Iizuka, K. Yoshida, and K. Iizuka

4.1 About DiSTaS-Anne DiSTaS-Anne is one of the methods of Distributed Constraint Optimization Problems (DCOP). DCOP has recently attracted attention as a problem solving framework under a decent decentralized environment in the field of artificial intelligence (AI)[2][3] DCOP is a framework that attempts to solve various constraint satisfaction problems (CSP), namely a classical AI problem solving framework, with an enhanced problem area in a decentralized environment. In DCOP, knowledge is expressed in the form of constraints. Formally㸪DCOP are defined as follows. A set of variables , , ⋯ , , in which each variable is assigned a value taken from a finite and discrete domain , , ⋯ , , and each variable is also assigned to multiple agents , , ⋯ , . Constraints : × → { , } are defined between and and a cost function : × → ℝ exists for each constraint.

．

,

=

,

=

ℎ

Where, < . The agents only have information about which is assigned to , which is constraint of , and cost function ∗ . At this time, the purpose of DCOP is to obtain an assignment for variable A that minimizes the summation ( )=∑ (A) of the cost function. In DCOP, the problem is solved while the values of the variables are exchanged between the agents tied together by constraints through message transmission. Well-known algorithms used to solve DCOP include ADOPT [4], and distributed stochastic search algorithms (DSA) [5]. When using DCOP for real-life problems, problems must be solved in distributed environments with minimal calculation resources and promptly . Under such circumstances, a fast and efficient approximate solution is desired. The authors had proposed Distributed Tabu Search with simulated Annealing (DiSTaS-Anne) in order to solve DCOP. DiSTaS-Anne is a method which is based on distributed search, with added features of Tabu Search and simulated annealing. Its redeeming feature is the ability to swiftly obtain high-quality approximate solutions. The evaluation result of this algorithm is shown in Figure 2.

4.2 Evacuation Planning Process Using DiSTaS-Anne The evacuation planning process using DiSTaS-Anne in the disaster evacuation assist system as proposed by the authors is described as follows: Knowledge such as evacuation routes or their capacity is preset in the form of a constrained condition. Devices can obtain information on numbers and distribution of neighboring evacuees by ad-hoc communication between devices. Inputting this information, situations are solved as DCOP. The solutions are shown to each evacuee on their devices. The solution are then recalculated and updated periodically to ensure optimized solutions.

An Effective Disaster Evacuation Assist System Utilized by an Ad-Hoc Network

35

Fig. 2. Effectiveness Evaluation of DiSTaS-Anne

5 Conclusion and Future Works In this paper, the authors proposed a disaster evacuation assist system using DiSTaSAnne. Using this system, effective evacuation derivation would become possible with limited system resources via ad-hoc communication and distributed processing. Flexibility of system resources is important when disaster occurs. Future work is planned, including the detailed formulation of constraint conditions and estimation of calculating time of DiSTaS-Anne applied to disaster evacuation assist system. The effectiveness will be verified by detailed simulation.

References 1. Iizuka, K., Iizuka, Y., Yoshida, K.: A Real-time Disaster Situation Mapping System for University Campuses. In: The 14th International Conference on Human Computer Interaction (2011) 2. Iizuka, Y., Takeuchi, I.: A Multiplexed Method for Distributed Constraint Optimization Problems. IPSJ Journal, Information Processing Society of Japan 50(12), 3136–3149 (2009) 3. Iizuka, Y., Suzuki, H., Takeuchi, I.: Multi-agent Tabu Search Method for Distributed Constraint Satisfaction Problems. The IEICE Transactions on Information and Systems (Japanese Edition) J90-D(9), 2302–2313 (2007) 4. Modi, P.J., Shen, W.-M., Tambe, M., Yokoo, M.: Adopt: Asynchronous distributed constraint optimization with quality guarantees. Artif. Intell. (161), 149–180 (2005) 5. Zhang, W., Wang, G., Xing, Z., Wittenburg, L.: Distributed stochastic search and distributed breakout: Properties, comparison and applications to constraint optimization problems in sensor networks. Artif. Intell. (161), 55–87 (2005) 6. PlaceEngine, http://www.placeengine.com/PlaceEngine

Locating Projectors Using Intensity of Reflected Beams Based on Phong Shading Model Yukio Ishihara1 and Makio Ishihara2 1

Innovation Center for Medical Redox Navigation, Kyushu University, 3-1-1, Maidashi, Higashi-ku, Fukuoka 812-8582, Japan [email protected] 2 Faculty of Information Engineering, Fukuoka Institute of Technology, 3-30-1, Wajiro-Higashi, Higashi-ku, Fukuoka 811-0295, Japan [email protected]

Abstract. In this work we focus on handheld projector-camera systems and discuss how to find out where and in which orientation those projectors are. We propose a way to find out them by measuring the intensities of the beams reflected on the screen. The point is that our system does not rely on fiducial markers which are usually stuck on the screen to help to locate the projector. Additionally, in our system, a laser-based projector is used, and the intensity of beams being projected and reflected on the screen is modeled based on Phong shading model. Through the evaluation, we show the potential of our system in HCI areas. Keywords: beam intensity, Phong shading model, laser-based projector, real-time calibration, projector-camera system.

1 Introduction Because of the downsizing and portability, projectors are easily installed anywhere and widely used in various ways as well as in common use such as presentations. Usually projectors are fixed on-centered to the screen. If projectors are put angled, the projected images will be distorted. Sukthankar et al. [1] built a projector-camera system where the projector is allowed to be off-centered to the screen. In the system the projector projects a calibration pattern and the camera captures it on the screen. From the correspondences between the calibration pattern and observed calibration pattern, the mapping between the camera and projector coordinates is obtained. And also the mapping between the camera and screen coordinates is obtained by observing the outline of the screen. These two mappings give the desired mapping between the projector and screen coordinates. By using the desired mapping, the projected image is corrected to appear not-distorted. In contrast to using one projector, other researches focus on multiple projectors to display a single tiled image on a large screen. Chen et al. [2] aligned 24 projectors to project a large seamless image. Likewise this system needs to be calibrated by obtaining the mappings mentioned above. On the other hand there is another way of use of a projector, which is allowed to move while projecting images. In this case, real-time calibration, or quick calibration, C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 36–40, 2011. © Springer-Verlag Berlin Heidelberg 2011

Locating Projectors Using Intensity of Reflected Beams Based on Phong Shading Model

37

is required whenever the projector moves. If the mapping between the projector and screen coordinates is obtained every time the projector moves, the calibration pattern will interrupt the projection. So, the calibration pattern cannot be used. Instead, the outline of the quadrilateral illuminated by the projector is used. Beardsley et al. [3] and Rehg et al. [4] built a projector-camera system capable of real-time calibration. Four fiducial markers are stuck on the screen, which show the four corners of a rectangle where the projected image is supposed to appear. The camera captures both the outline of the quadrilateral illuminated by the projector and the four fiducial markers. Before images are projected, they are pre-warped first, and then projected on the screen. This pre-warping is always performed. As a result, those images appear not-distorted within the four fiducial markers even while the projector is moving. In addition, projector-camera systems capable of real-time calibration are useful for HCI because the current orientation and position of those projectors are available all the time when in use. The work of Beardsley et al. [3] is a good example of projector-camera systems used in HCI. In their system, a mouse cursor is placed at the center of the illuminated quadrilateral while projecting the desktop image of MS-Windows within the four fiducial markers. The cursor moves when the projector is panned. At the time, as mentioned above, the desktop image is kept within the four fiducial markers. So the projector can be used as a mouse. However fiducial markers make the interaction area be limited around the markers. In this work, we aim to extend the interaction area by no use of fiducial markers. Previously we proposed a way to find out the orientation and position of a projector by measuring the intensities of the beams reflected on the screen [5]. As that technique needs no fiducial markers, the screen can be extended as far as the reflected beams are observed. In that work, a DLP projector was used and the intensities of the beams were modeled by their own defined equations. Recently, handheld laser-based projectors were released by AAXA Technologies Inc. Those projectors benefit from use of laser, so the projected images are always in focus. This feature could solve a problem posed in the previous work, that the projected images become blurred badly when the DLP projector is paned in either direction. Therefore, in this work the DLP projector is replaced with the laser-based projector. This will also simplify how the intensities of the reflected beams are calculated because of no need to take into account blurring caused by the projector being out of focus. That is, the intensities depend only on the positions of the projector and camera. In computer graphics area, Phong shading model [6] plays an important role in providing a way to give the intensities of the beams reflected on objects with respect to the positions of light sources and an observer. Thus, in this work, we take an advantage of Phong shading model to create the intensity curves of the beams reflected on the screen. And then, the orientation and position of the projector is obtained based on the intensity curves. The rest of this manuscript is organized as follows. In Section 2, how projectors are located is explained. And the intensities of the beams reflected on the screen are modeled in Section 3. In Section 4, our system is evaluated, and the concluding remarks are given in Section 5.

38

Y. Ishihara and M. Ishihara

Fig. 1. Our projector-camera system. A camera is attached on a projector, and captures the images projected by the projector.

2 How to Locate Projectors First of all it is assumed that projectors project beams of the same intensity in every direction when a one-colored image, such as a white-filled image, is projected. Fig. 1 illustrates our projector-camera system, where a camera is attached on a projector. ߠ௛ and ߠ௩ are the angles at which the projector is angled horizontally and vertically toward the screen, respectively. ‫ ݐ݊ܥ‬is the center of the projector's perspective on the screen, and ‫ ܦ‬is the distance between the projector and ‫ݐ݊ܥ‬. ܲ is a position on the screen. ߠଵ and ߠଶ are the horizontal and vertical angles to ܲ from ‫ݐ݊ܥ‬. ܱ is the nearest position on the screen from the projector. The angle at the projector between ܱ and ܲ is obtained as follows. ݂ఏ (ߠ௛ ,ߠ௩ ,ߠଵ ,ߠଶ ) = acos(…‘• ߠ௛ …‘• ߠ௩ …‘• ߠଵ …‘• ߠଶ െ …‘• ߠ௛ •‹ ߠ௩ •‹ ߠଶ െ •‹ ߠ௛ •‹ ߠଵ …‘• ߠଶ )

(1)

When intensities are observed at ܲ௧ , ܲ௟ and ܲ௥ as ‫ܫ‬௧ , ‫ܫ‬௟ and ‫ܫ‬௥ , respectively, then the projector can be located, or ߠ௛ , ߠ௩ and ‫ ܦ‬can be derived from the following simultaneous equations. ‫ܫ‬௧ = ݂ூ ቀ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡͲǡ ‫ܫ‬௟ = ݂ூ ቀ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡ െ ‫ܫ‬௥ = ݂ூ ቀ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡ

ఏ೛೓ ଶ ఏ೛೓ ଶ

ఏ೛ೡ ଶ

ሻǡ ‫݂ܦ‬ఏ ሺߠ௛ ǡ ߠ௩ ǡ Ͳǡ Ͳሻൗ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡͲǡ

ǡͲሻǡ ‫݂ܦ‬ఏ ሺߠ௛ ǡ ߠ௩ ǡ Ͳǡ Ͳሻൗ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡ െ ǡͲሻǡ ‫݂ܦ‬ఏ ሺߠ௛ ǡ ߠ௩ ǡ Ͳǡ Ͳሻൗ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡ

ఏ೛೓ ଶ ఏ೛೓ ଶ

ఏ೛ೡ

ሻቁ,

(2)

ǡͲሻቁ,

(3)

ǡͲሻቁ,

(4)

ଶ

where ݂ூ (ߠ, ‫ )ܦ‬gives the intensity of the beam reflected at ‫ ݐ݊ܥ‬when the projector is angled by ߠ in either direction and is placed ‫ ܦ‬away from ‫ݐ݊ܥ‬. ߠ௣௛ and ߠ௣௩ are the horizontal and vertical angles of the projector’s perspective.

Locating Projectors Using Intensity of Reflected Beams Based on Phong Shading Model

39

3 Intensity Curves It is already explained in the previous section that the projector can be located if ݂ூ (ߠ, ‫ )ܦ‬is known. In this section, ݂ூ (ߠ, ‫ )ܦ‬is modeled by observing the intensities of beams by the camera. A white-filled image was projected and the reflected beams from ‫ݐ݊ܥ‬ were observed by the camera at 5ൈ6 = 30 positions: ‫ = ܦ‬50, 75, 100, 125, 150 cm and ߠ ൌ Ͳ௢ , ͳͲ௢ , ʹͲ௢ , ͵Ͳ௢ , ͶͲ௢ , ͷͲ௢ . Fig. 2 shows the intensities of the observed beams. The intensities range from 0 to 255. Eqn. (5) of Phong shading model was best fit to those intensities as shown in Fig. 2. ଵ

݂ூ (ߠ, ‫ܫ = )ܦ‬௖ = ௔ା௕ሺଶ஽ሻାሺଶ஽ሻమ ൛‫ܭ‬ௗ ‫ܫ‬௣ …‘• ߠ ൅ ‫ܭ‬௦ ‫ܫ‬௣ …‘• ఈ ሺʹߠሻൟ + ‫ܭ‬௔ ,

(5)

where ‫ܫ‬௖ and ‫ܫ‬௣ are the intensity of the beams observed by the camera and the one projected by the projector, respectively. ܽ, ܾ, ‫ܭ‬ௗ , ‫ܭ‬௦ , ߙ and ‫ܭ‬௔ define the intensity curves. ‫ܫ‬௣ was set to 255 because a white-filled image was projected. The set of optimal values was obtained by the steepest descent method as follows. ܽ ൌ9582.668, ܾ ൌ18.613, ‫ܭ‬ௗ ൌ14781.993, ‫ܭ‬௦ ൌ െ2148.044, ߙ ൌ1.026, ‫ܭ‬௔ ൌ 10.388.

Intensity

200 Observed Modeled

150 100 50 0 50

75

D (cm)

100

125

150

0

10

20

30

40

50

θ (deg.)

Fig. 2. Intensity curves

4 Evaluation Finally we evaluated our system. The projector and camera were put at each of the same 30 positions but ߠ௩ was kept to 0 in order to check the accuracy of ߠ௛ . The orientation of the projector and the distance from the screen, or ߠ௛ and ‫ ܦ‬were estimated by Eqn. (3) and (4). Fig. 3 shows the result. There are about ͸௢ error on ߠ௛ and 5 to 10 cm error on ‫ܦ‬. The estimation errors could be caused by fluctuation in the intensities of the observed beams.

40

Y. Ishihara and M. Ishihara

Fig. 3. Accuracy of estimated values of ߠ௛ and ‫ܦ‬

5 Conclusions In this work, we proposed a way to find out the orientation and position of a projector by measuring the intensities of the beams reflected on the screen. Specifically, there were two main changes from our previous projector-camera system. That is, a laserbased projector was used and intensity curves were modeled based on Phong shading model. From the evaluation, it showed the successful first step towards use of handheld projector-camera system without fiducial markers, especially for HCI purposes. In the future, we will discuss the effect of the estimation errors on usability in application use, and also improve the estimation process for higher accuracy.

References 1. Sukthankar, R., Stockton, R.G., Mullin, M.D.: Automatic keystone correction for cameraassisted presentation interfaces. In: Tan, T., Shi, Y., Gao, W. (eds.) ICMI 2000. LNCS, vol. 1948, pp. 607–614. Springer, Heidelberg (2000) 2. Chen, H., Sukthankar, R., Wallace, G., Li, K.: Scalable alignment of large-format multiprojector displays using camera homography trees. In: Proc. VIS 2002 IEEE Computer Society, pp. 339–346 (2002) 3. Beardsley, P., Baar, J.V., Raskar, R., Forlines, C.: Interaction Using a Handheld Projector. IEEE Computer Graphics and Applications 25(1), 39–43 (2005) 4. Rehg, J.M., Flagg, M., Tat-Jen, C., Sukthankar, R., Sukthankar, G.: Projected light displays using visual feedback. In: Proc. ICARCV 2002, vol. 2, pp. 926–932 (2002) 5. Ishihara, Y., Ishihara, M.: Locating a projector using the strength of beams reflected on a screen. In: Proc. AUIC 2007, vol. 64, pp. 47–50. Australian Computer Society, Inc. (2007) 6. Phong, B.T.: Illumination for computer generated pictures. Communications of the ACM 18(6), 311–317 (1975)

Embodied Communication Support Using a Presence Sharing System under Teleworking Yutaka Ishii1 and Tomio Watanabe2, 3 1

Information Science and Technology Center, Kobe University, 1-1 Rokkodai, Nada, Kobe, Hyogo 657-8501, Japan 2 Faculty of Computer Science and System Engineering, Okayama Prefectural University, 111 Kuboki, Soja, Okayama 719-1197, Japan 3 CREST of Japan Science and Technology Agency [email protected], [email protected]

Abstract. We have proposed the concept of a presence sharing system Ghatcha [GHost Avatar on a Telework CHAir] in which the users’ embodiment is not indicated by the avatars but by the chairs that suggest the presence of avatars. This system provides the same communication space for the users’ embodiment, thus creating a feeling of working alongside remote workers. In this paper, we develop prototype systems using a virtual or a real environment. And the effectiveness of the virtual prototype system is confirmed in the experiment. Keywords: Embodied Interaction, Avatar, Remote Communication, Telework, Remote Operating Chair.

1 Introduction A telework increases productivity and operational efficiency by offering employees the flexibility to work from their home offices. It would gain popularity as it can be utilized in different ways. However, the quality or efficiency of work might deteriorate as a result of a telework as it leads to a sense of isolation or a lack of concentration. Thus, it is important to examine remote collaboration support in detail. Remote collaboration has various purposes and applications, and it is expected to support for each situation. The subjects of this research are not remote users performing a group task but individual users performing their own specific tasks wherein all their co-workers also perform tasks with the same aim such as a job of a home-based worker or individually pursuing online distance learning. In the case of tasks that are not synchronized, however, the video image might contradict our expectations. In order to solve the problem, Honda et al. proposed a virtual office system “Valentine” using an awareness space and provided a work support environment for home-based workers [1]. Moreover, various media communication methods have been proposed for practical use, such as the design of a communication environment, which aims at maintaining and fostering human relations for family members living apart, or a communication system wherein the furniture or daily necessities, which are separated in different rooms, can be linked [2],[3]. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 41–45, 2011. © Springer-Verlag Berlin Heidelberg 2011

42

Y. Ishii and T. Watanabe

However, when a user’s own avatar is used as a communication media for an embodied interaction, many issues arise with regard to the relation between the users and their avatars. For example, if a human-type avatar is used, the correspondence of the user’s motion and that of the avatar’s would be hindered by input devices. Otherwise, the appearance of an avatar cannot appropriately represent a user’s embodiment. The embodiment is not always indicative of the avatar information in the input/output of the smooth interaction system. It would be useful to integrate the embodiment with the environment information for the development of an effective interaction support system. Hence, this paper proposes a new presence sharing system called Ghatcha: GHost Avatar on a Telework CHAir [4]. The Ghatcha system is based on the embodiment of the environment information of the chair motion rather than that of the avatar. The present study aims to recreate the environment in which remote users interact with their co-workers in the same virtual office, and enhance their motivation in performing their tasks. In this paper, a prototype of the system using a virtual environment is developed, and the effectiveness of this system is demonstrated by an evaluation experiment.

2 Development of the Prototype System Using CG The prototype system using a CG avatar was developed. In this system design, the chair motions are measured by various sensors such as a gyroscope, an accelerometer, or a magnetic sensor. The virtual chair motions are represented based on the measurements, and are shared on the network. The mutual motions of each user are transmitted to the office model from the shared communication space. This collaborative system determines the third interaction space with the chairs for each remote coworker. The CG prototype system is generated by an HP workstation xw4200 (CPU: Pentium4 3.6GHz, RAM: 512MB, NVIDIA Quadro FX3400), OS: Windows XP Professional SP1, and DirectX9.0b. The frame rate is 30 fps. The chair motions are measured by a laser sensor mouse (Logicool MX Air) attached under the chair. The communication scene using the system is shown in Fig. 1. This example displays only the user’s human type avatar.

Fig. 1. Communication scene using the prototype system

Embodied Communication Support Using a Presence Sharing System under Teleworking

43

3 Development of the System in the Real Environment Not only a virtual environment, the prototype system in a real environment was developed as shown in Fig. 2. The system was constructed by the 4 servo motors through the USB cable. User's chair motions were measured by the same way as the virtual system. User can put any objects for the substitute of his/her partners on the chairs, including the one of himself/herself. This miniature system provide interaction awareness between users using their embodiments.

Fig. 2. Prototype system in the real environment

4 Evaluation of the Experiment Using the Virtual System 4.1 Experimental Setup The system evaluation experiment was performed by the prototype system using CG in the previous chapter 2. The subjects consisted of 10 pairs, and they worked on a simple task wherein they made paper cranes by folding pieces of paper. The task was repeated twice using two scenes: one where the chair system was connected with the motions of the user and another where they were not connected. The subjects were ordered to fold the papers as much as they could. After the task was finished in each scene, the user’s behavior was observed during a waiting period of 3 minutes. The only information that was shared through the system was the motion of the chair. Only the user’s human type avatar was displayed in addition to both the users’ chairs. Thus, the user makes his/her presence felt not as the chair but as the avatar. The partner’s avatar was not represented in the virtual space. The subjects answered the questionnaire after the task in each scene. They were provided an explanation of the conditions and the setting of the experiment, and they agreed to the experiment before the experiment started. The time taken to conduct the experiment was about 40 minutes on average including the waiting time and the time taken to answer the questionnaire. The example of the evaluation experiment scene is shown in Fig. 3.

44

Y. Ishii and T. Watanabe

Fig. 3. Example of an evaluation experiment

4.2 Sensory Evaluation The two scenes were evaluated on a seven-point bipolar rating scale ranging from –3 (lowest) to 3 (highest), in which 0 denotes a moderate score. For the sake of convenience, the results of the means and the standard deviations are shown in Fig. 4. The questionnaire consisted of eight categories: four categories on the impression of the work and the other four categories on the evaluation of media communication. In most of the categories, the significant difference between the two scenes was obtained by administering the Wilcoxon’s rank sum test; a significance level of 0.1% for the items of “Do you feel like sharing the same space with a partner?”, and “Do you feel like working together with a partner?” A significance level of 1% was obtained for the items “Do you enjoy your task?” “Do you believe that you could associate yourself with the character?” and “Do you recognize a partner’s motion?” The effectiveness of the prototype system is evinced by the positive evaluation of each category in the scene that the chair motions were connected. The scene that the chair motions weren’t connected has a negative evaluation that is significant at the 5% level for the item “Are you bored by your task?”

Fig. 4. Results of the questionnaire

Embodied Communication Support Using a Presence Sharing System under Teleworking

45

In addition, comments such as “I didn’t move too much while folding pieces of paper into the figure of a crane” and “I was pleasant to manipulate my chair after working the task” are obtained as responses of a free description of the respondents’ opinions in the questionnaire. The effectiveness of the system can be observed in the waiting time after working rather than during the working time.

5 Conclusion In this paper, the prototype systems using a virtual or a real environments were developed. And, the evaluation experiment was performed using the virtual prototype of the system, and the effectiveness of the system was demonstrated by a sensory evaluation.

References 1. Honda, S., Tomioka, H., Kimura, T., Oosawa, T., Okada, K., Matsushita, Y.: A companyoffice system “Valentine” providing informal communication and personal space based on 3D virtual space and avatars. Information and Software Technology 41, 383–397 (1999) 2. Miyajima, A., Itoh, Y., Itoh, M., Watanabe, T.: TSUNAGARI Communication: The design of the communication environment which aimed at keeping and fostering human relations and a field test for family members living apart. The Transactions of Human Interface Society 5(2), 171–180 (2003) (in Japanese) 3. Tsujita, H., Tsukada, K., Siio, I.: SyncDecor: Appliances for Sharing Mutual Awareness between Lovers Separated by Distance. In: CHI 2007 Conference Proceedings and Extended Abstracts, Conference on Human Factors in Computing Systems, pp. 2699–2704 (2007) 4. Ishii, Y., Osaki, K., Watanabe, T.: Ghatcha: GHost Avatar on a Telework CHAir. In: Proc. of HCI International 2009, pp. 216–225 (2009)

Visibility Experiment and Evaluation of 3D Character Representation on Mobile Displays Hiromu Ishio1, Shunta Sano1, Tomoki Shiomi1, Tetsuya Kanda1, Hiroki Hori1, Keita Uemoto1, Asei Sugiyama2, Minami Niwa1, Akira Hasegawa1, Shohei Matsunuma3, and Masaru Miyao1 1

Graduate School of Information Science, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8601, Japan 2 School of Engineering, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8601, Japan 3 Nagoya Industrial Science Research Institute, Noa Yutsuya Bldg 2F, 1-13, Yotsuya-dori, Chikisa-ku, Nagoya 464-0819, Japan [email protected]

Abstract. In the experimental study, we measured visibility and readability of text characters presented on a small 3D liquid crystal display (LCD) and evaluated features of the 3D character representation for application to mobile devices. For the visibility evaluation, we focused on time lag for first recognition, time required for full reading, and maximum distance of 3D objects popping out from the fixed display. We also report the dependency of the results on the age of subjects in comparison with 2D and 3D representations. Keywords: 3D character representation, liquid crystal display (LCD), mobile display.

1 Introduction Recent advances in stereoscopic vision technology have brought us wide range of display applications ranging from large TVs and public screens to small devices such as mobile phones and portable games. In this trend, it is most probable that 3D representations not only of scenery images but of texts will become widely used on such devices in the near future. A useful application may be “augmented reality” (AR) [1], where computer generated additional information such as text characters, images, etc. are superimposed on a real visual environment in order to complement human sensory information. In AR applied to mobile devices, text information may need to be presented more effectively in a stereoscopic way depending on the location of a real object to be augmented in 3D. Nevertheless, there is no index available in such a case to identify visibility and readability of the text information. Therefore, it is important to investigate visibility and readability of text characters displayed on mobile 3D devices [2, 3]. In the experimental study, we measured visibility and readability of characters presented on a small 3D liquid crystal display (LCD) and evaluated features of the 3D C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 46–51, 2011. © Springer-Verlag Berlin Heidelberg 2011

Visibility Experiment and Evaluation of 3D Character Representation on Mobile Displays

47

character representation for application to mobile devices. For the visibility evaluation, we focused on time lag for first recognition, time required for full reading, and maximum distance of 3D objects popping out from the fixed display. We also report the dependency of the results on the age of subjects in comparison with 2D and 3D representations.

2 Method The subjects in this study were 109 healthy males and females (age 20-78). The experiment consisted of three parts (Exp. I, II and III) which were carried out in sequence to each of them. We obtained informed consent from all subjects, and approval for the study from Ethical Review Board in the Graduate School of Information Science at the Nagoya University. Experiment I. Visibility Evaluation in Pop-out Distance Measurement of 3D Images. The aim was to evaluate visibility of 3D images for each subject and, for those who succeeded in stereoscopic view, to measure maximum distance of a 3D object virtually popping out from the fixed display. The details of experimental setup were as follows: We first measured pupil distance (PD) of each subject. Then we set a small (3.8 inch) LCD 40 cm in from of subjects under illumination condition of 500 lx. It is a naked-eye 3D display (a test model of SHARP LYNX 3D SH-03C) with resolution of 800 x 480 pixels and can present 3D images by using parallax barrier system that separates two different images to present to the right and left eyes with parallax barrier on the display (Fig.1, Fig. 2.).

Fig. 1. SHARP LYNX 3D SH-03CTM

Fig. 2. Parallax barrier display

Fig. 3. 3D images with a spherical object moved forward and back continuously with a cycle of 10 seconds

48

H. Ishio et al.

There we presented 3D images where a spherical object moved forward and back continuously with a cycle of 10 seconds (see Fig. 3). The spherical object movies called Power 3D™ were kindly supplied by Olympus Visual Communications, Corp. In principle, the spherical object appears as a 3D image at 40 cm (i.e., the location of LCD) and moves toward the subjects in front of them. We asked them to gaze at the center of the spherical object and to answer whether they could view the object in 3D during the motion (Fig.4). We further asked those who succeeded in stereoscopic view to point out the nearest location of the object with their finger, and measured the maximum distance of the object popping out from the fixed display. Experiment II. Visibility Evaluation in Time Lag Measurement for First Recognition of 3D Text Characters. The aim was to evaluate visibility of 3D text characters for each subject and, for those who succeeded in stereoscopic view, to measure time lag for first recognition of the 3D text characters. The details of experimental setup were as follows: The subjects first watched the small 3D display which showed nothing (i.e., black background only). At some point, we presented text characters (lowercase and uppercase alphabet letters and numbers aligned still at random, see Fig. 5) in 3D on signal to the subjects. Then we measured the time required for first recognition of the 3D characters as they are. For comparison, we chose large (H 5.0 mm x W 1.3 mm each) and small (H 2.5 mm x W 0.63 mm each) size of characters in the experiment.

Fig. 4. Succeeded subjects in stereoscopic view were asked to point out the nearest location of the object with their finger, and measured the maximum distance of the object popping out from the fixed display

Experiment III. Readability Evaluation for Full Reading of 3D Text Characters. The aim was to evaluate readability of 3D text characters for subjects who succeeded in recognition of those characters, by measuring time required for full reading. The details of experimental setup were as follows: The subjects were 49 males and females (age 20-78) who could recognize 3D text characters. We measured time required to finish counting the number of capital character “A” which was involved in a random sequence of characters in 3D. For comparison, we used the same sizes of large and small characters in this experiment as were adopted in Exp. II. We also carried out the same kind of counting number experiments for the same sizes of characters by using 2D (i.e., ordinary nonstereoscopic) representation.

Visibility Experiment and Evaluation of 3D Character Representation on Mobile Displays

49

3 Results In Exp. I, 87.2% of the subjects could and 12.8% of those could not recognize 3D images where a spherical object moved forward and back continuously. We show the number of subjects who could and could not recognize the 3D images in each age group in Fig. 6. We could not, however, find any specific tendency between visibility of 3D images and age group of subjects. In the case of subjects who could recognize 3D images, the maximum distance of the object popping out from the fixed display was measured as 10.2 cm (mean) ± 1.8 cm (SD). In Fig. 7, we show the relation between pupil distance of the subjects and maximum pop-out distance (mean value) of the 3D object. We find that our experimental results in general agree well with a theoretical prediction calculated by using PD.

Fig. 5. Text characters used for 3D representation. Arrows in the right picture indicate instructed reading directions.

In Exp. II, 78.3% and 77.4% of the subjects could recognize the large and small characters in 3D, respectively. For the subjects who could recognize 3D characters, the time required for first recognition of those characters in large and small size was 9.2 sec (mean) ± 5.3 sec (SD) and 9.0 sec (mean) ± 10.1 sec (SD), respectively. In Fig. 8, we show the relation between the mean time required for first recognition of 3D characters and age group of the subjects. In the case that the represented characters in 3D are smaller, it tends to require more time especially for subjects of over 60 years old to recognize those characters.

Fig. 6. Number of subjects who could and could not recognize the 3D images in each age group

50

H. Ishio et al.

In Exp. III, we measured time required for full reading of 3D text characters. The results are 9.2 sec (mean) ± 5.3 sec (SD) for large 3D characters, 9.0 sec (mean) ± 10.1 sec (SD) for small 3D characters, 12.6 sec (mean) ± 6.0 sec (SD) for large 2D characters, and 16.0 sec (mean) ± 13.5 sec (SD) for small 2D characters. In Fig. 9, we show the relation between the mean time required for full reading of 3D text characters and age group of the subjects. There, depending on the character size, we find significant difference in the mean time required for full reading. That is, in the case that characters are larger, the mean time required for full reading decreases in both 2D and 3D representations. In such a case, there is almost no difference in the mean time in 3D and 2D representations. On the other hand, in the case that characters are smaller, there is almost no difference in the mean time in 3D and 2D representations for subjects under 40 years old while it requires more time for subjects over 40 years old to read characters in 3D representation, compared to the case of 2D representation.

Fig. 7. Pupil distance of the subjects and maximum pop-out distance (mean value) of the 3D object

Fig. 8. Mean time required for first recognition of 3D characters in each age group of the subjects

Fig. 9. Mean time required for full reading of 3D text characters in each age group of the subjects

4 Conclusions In conclusion, whether we can recognize 3D characters or not does not so much depend on our age. The time required for first recognition of 3D characters increases in the case of smaller characters especially for people over 60 years old. There is not so

Visibility Experiment and Evaluation of 3D Character Representation on Mobile Displays

51

much difference in time required for full reading of 2D and 3D characters in larger size while the time significantly increases in 3D representation of smaller characters for people over 40 years old. Therefore, we should carefully choose suitable size of characters in 3D representation not to lose their readability especially for elderly people.

References 1. Cawood, S., Fiala, M.: Augmented Reality: A Practical Guide, Pragmatic Bookshelf, Raleigh, North Carolina (2008) 2. Hasegawa, S., et al.: Aging Effects on the Visibility of Graphic Text on Mobile Phones. Gerontechnology 4, 200–208 (2006) 3. ISO 13406-2: Ergonomic Requirements for Work with Visual Displays Based on Flat Panels –Part 2 Ergonomic Requirements for Flat Panel Displays (2001)

Composite Context Information Model for Adaptive Human Computing Sukyoung Kim, Eungha Kim, and Youngil Choi Service Cognitive Control Team, Electronics and Telecommunications Research Institute, 138 Gajeongno, Yuseong-gu, Daejeon, 305-700, Korea {ksk0314,ehkim,yichoi}@etri.re.kr

Abstract. This study is to propose the composite context information model that provides an appropriate model depending on the situation around user. In our research, describe the definition of composite context information, propose the reuse method of the preexisted context information and design the composite context information process structure to improve performance. For developing the user-centric context-aware computing, above all important thing is, provide the high-level concept like human beings in knowledge framework and many application layers use it, don’t care the inner structure for processing complex context information. Keywords: composite context information, context-aware computing.

1 Introduction Nowadays, many researchers or institute is to predict the context-aware computing one of the future internet’s technologies. Context-aware computing can be used in various fields by grafting and very close a complex reality to the human being. That is, to success the context-aware computing, is important to model from the human being to computer being. But it is not easy, because all of humans being do make the model to process by computer, beside has become more complex according to develop human world. Moreover, rapidly evolving the web’s paradigm needs the like thing with human beings, and supporting the standardized infra of context-aware can be reduce the development cost of application. Despite the many context-aware system proposed, since the concept of a meaningful context similar in many ways represented a different vocabulary, it is not easy to reuse the context information by most early context-aware system. The important thing is the modeling method to extract from status to context and select the appropriate service. According to the modeling method, the context-aware system can be support the concept of high-level context or can be share the context information. Development of the technologies like that the service delivery platform or the network generation service overlay network depend on to select or to composite the appropriate service by the context. So, like this, to structure the service system with the context process architecture shall be subject to the user situation oriented and provide the mechanism from context to context information. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 52–56, 2011. © Springer-Verlag Berlin Heidelberg 2011

Composite Context Information Model for Adaptive Human Computing

53

Beside, Many researchers have been defined for the context and context information, each definition are limited that applies to services and application. For increasing reuse and share the high-level context information, for reducing the complexity in application and for enhancing the standardization of context information, we decide to define the CCI(composite context information). As define the CCI, we suggest the structure that construct of multi-layered representation architecture which is consist of the unit context information.

2 Propose the CCI Model and Process This chapter describes the overview of our research method. And we descried briefly the related study in terms of the context information modeling. 2.1 Definition of CCI We define the CCI which is “the enhanced high-level context information by connecting or compositing of composite information related with multi-entity to decide and provide for service of system”, in here, entity can be the things that is generate the situation or associate with human. Fig.1 shows the concept of above definition. Many entities generate the context information and the CCI made by composite with each the context information. The right side of fig.1, describe the main criteria for constructing the CCI, and we can find the CCI’s contents consist of DL and SWRL.

Fig. 1. According to define the CCI, describes the concept of CCI with service and address to the main criteria of CCI

2.2 Analysis of Modeling Method for CCI We search and analyze the related study of context-aware system, and more detail analyzes the modeling method of each system and main element to satisfy that compose the CI. As result of that, we can draw the fig.2 which describes the main elements and main contents to enhance the performance of context-aware middleware [2, 3]. We can know the important thing which is select the model to apply the above features when construct the CCI model and framework.

54

S. Kim, E. Kim, and Y. Choi

Fig. 2. As the result of analysis, the main elements and main contents to enhance the performance of context-aware middleware

Besides, to decide the modeling method, compare with the modeling and processing method of early context-aware system. Result to this, we compare with the architecture, context model, context process and historical context data through the existing R&D systems consist of CASS, CoBrA, Context Management Framework, CORTEX, Gaia, Hydrogen and SOCAM. We decide the whole structure our test system, architecture is centralized middle in service network and context model is OWL DL+SWRL and context processing is context reasoning engine based on Pellet. 2.3 Design the Structure of CCI Process and Data Architecture In this section, we proposed the design of CCI structure to process the context information and data repository architecture to enhance the CCI decision. Fig.3 shows the structure of CCI process with CCI Repository and reusing process to reuse the existed context-aware context information. As we can see the Fig.4, our

Fig. 3. Design the structure of CCI processing and the CCI or the Context Information repository

Composite Context Information Model for Adaptive Human Computing

55

study focus the CCI interpreter and Rule processing role of decision the CCI List and inference the rule of CCI. Two part of our structure, in order to decide the processing of CCI, first matching process with required context to CCI List in the repository, second decision the function type to process for dynamic or changeable context from outside and third, rule processing that is CCI and rule extraction to translate for transmission to reasoning engine. Besides, we design the data structure of CCI in repository to enhance the speed of searching, reasoning and reusing the CCI or CI. Fig.4 is show the layered structure of CCI to repository, and as explain to our strategies in repository and sequence of processing CCI, and recombine with CI rule and CCI rule Op list to express CCI.

Fig. 4. Design the multi-layered structure to CCI processing by combine with CI rule index and CCI rule Op list

Fig. 5. Example of the modeling for expression the “Availability” one of CCI

56

S. Kim, E. Kim, and Y. Choi

3 Modeling the CCI by OWL DL Finally, as we can see fig.5, we modeling the CCI with defined vocabulary “Availability” that is consist of context entities connected with device, network, location, service and user. The expression of “Availability” is composite with unit expression of each entity, and we can find the high-level CCI like as “ServiceEnable”, “ResponseStatus”. All of CCI is subclass of “CompositeContext” class and “CompositeContext” is subclass of “Context” class. First, this expression is store to the CCI schema repository and it is used for matching and reasoning when composite with the CCI rule index and CCI rule operator.

4 Conclusion For developing the user-centric context-aware computing, above all important thing is, provide the high-level concept like human beings in knowledge framework and many application layers use it, don’t care the inner structure for processing complex context information. So do this, we suggest the whole structure and model for CCI. To develop the study, we have to build the repository and casual system architecture and test our proposed method. And we have a plan to construct the user-defined CCI repository that is to enhance the volume and quality of the CI and accumulate the knowledge about the variety of CCI. Acknowledgments. The project related to this paper is performed by the Ministry of Knowledge Economy and the Korea Evaluation Institute of Industrial Technology [Research and development of composite context based adaptive service path configuration technology (10035146)].

References 1. Dey, A.K.: Providing Architectural Support for Building Context-Aware Applications. PhD Thesis, Georgia Institute of Technology (November 2000) 2. Biegel, G., Cahill, V.: A framework for developing mobile, context-aware applications. In: Proceedings of the 2nd IEEE Conference on Pervasive Computing and Communication, pp. 361–365 (2004) 3. Chen, H., Finin, T., Joshi, N.: An Ontology for Context-Aware Pervasive Computing Environments. The Knowledge Engineering Review 18(3), 197–207 (2003) 4. Patkos, T., Bikakis, A., Antoniou, G., Papadopouli, M., Plexousakis, D.: Design and challenges of a semantics-based framework for context-aware services. Int. J. Reasoning-based Intelligent Systems 1(1/2) (2009) 5. Preuveneers, D., Berbers, Y.: Adaptive context management using a component-based approach. In: Kutvonen, L., Alonistioti, N. (eds.) DAIS 2005. LNCS, vol. 3543, pp. 14–26. Springer, Heidelberg (2005) 6. Baldauf, M., Dustdar, S., Rosenberg, F.: A survey on contextaware systems. Int. J. Ad Hoc and Ubiquitous Computing 2(4), 263–277 (2007)

A Framework for a User Friendly Wireless Sensor Network Configuration System Julia C. Lee and Lawrence J. Henschen Northwestern University, Evanston, IL, 60208 USA [email protected], [email protected]

Abstract. Wireless sensor networks (WSNs) are being embedded into the fabric of our everyday lives. But there design still requires expertise in computer science and engineering in addition to the expertise for the application to which the WSNs are directed, and this slows the progress towards WSN solutions to important modern-day problems. We outline a system which may ultimately allow designers with little or no computer background to design WSNs. The system is based on a well-crafted user interface combined with a knowledge base about WSN hardware and software.

1 Introduction In 1990 HCI research pioneer Mark Weiser remarked that “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it”[1]. (Wireless) sensor network applications are leading the way to “fabric” the modern technologies into our “everyday life”. Gehrke and Liu [2] note “ … the growing pervasiveness of small-scale computing devices throughout the physical world, and the wireless sensor network community is a driving force behind this”. Currently, designing a wireless sensor network (WSN) requires expertise in many different areas from hardware to software, from system to application, from tools to abstract modeling, etc. Our work focuses on the development of a system that will assist developers of WSN applications who have little or no computer hardware/software background. The system will prompt the user with questions related to the application, provide choices for answers, and then combine those answers with its knowledge base and models to automatically create configuration and software for the application. This approach illustrates that one important solution to the application of new technology is to combine knowledge with a good user interface.

2 Review of Issues Related to Wireless Sensor Networks We briefly review some of the important issues related to WSN design and development so that the dimension of the complexity of the problem can be recognized by the reader. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 57–61, 2011. © Springer-Verlag Berlin Heidelberg 2011

58

J.C. Lee and L.J. Henschen

• Different application type or area: Application areas range widely, each with its own peculiar requirements [3][4]. Examples include: military (e.g. surveillance), environmental monitoring, health (e.g. patient monitoring), engineering, home/office, robotics, smart parking lots, interactive toys, interactive museums, detecting car thefts, vehicle tracking and detection, and many more. • Network configuration and routing: Network configuration focuses on the number and distribution topology of the nodes [5][6]. Routing focuses on how to get the information from the nodes to a base station. There are many choices, and the pros and cons of each depend on the application, the number of nodes involved, the sensor nodes chosen, etc. A user with little expertise in these topics would find it impossible to make good choices. • Node configuration: Sensor nodes available for application vary greatly in terms of features and capabilities [7], and users not familiar with computer engineering concepts would find it difficult to understand the differences. These details can be hidden from the users if a system has the intelligence to connect the user’s requirements to the information /knowledge bases. • Software and firmware: The most popular operating system for sensor nodes is TinyOS [8]. Applications on TinyOS based systems are written mostly in nesC [9]. However, mastering TinyOS or nesC is not an easy job for users who do not have a computer science background. • Other critical issues include [3]: WSNs often have large numbers of nodes; they are often densely deployed; nodes are prone to failures; the topologies can change frequently; nodes are limited in power, computational capacity, and memory; WSNs use broadcast communications instead of point-to-point; etc. From the above we can see the broad spectrum of knowledge required for WSN development, and this knowledge is separate from the issues specific to the application (e.g. environmental monitoring) [3][4][5].

3 An Automatic Sensor Network Configuration System Figure 1 shows an overview of the kind of system we are studying to help noncomputer engineers design WSNs. The system includes the following components: 1. A reference knowledge database: contains up to date information about wireless sensor networks including but not limited to types of sensors, types of motes, drivers, software libraries, distribution topologies, routing algorithms, etc. 2. A question and answer (QA) system: A hierarchy of carefully designed questions will guide the user through the “interview” process. The questions are mostly multiple choice menus; the user can easily choose from a set of pre-designed answers. When the user chooses one answer from a higher level, a lower level of relevant questions and answers will be made available to the user. Answers will be mapped into parameters that will be used by the other components of the system. 3. A static model: This model deals with the static aspects of the WSN application system, e.g. node/mote type, number, distribution topology, static routing scheme,

A Framework for a User Friendly Wireless Sensor Network Configuration System

59

etc. This component uses the parameters generated by the QA system and the information from the reference knowledge database to generate a configuration file for the chosen hardware components. 4. A dynamic model: This model deals with the dynamic aspects of the WSN application system, e.g. sensor data collection, processing and communication, etc. This component also uses parameters generated by the QA system, information from the reference knowledge database, and the configuration data from the static model to generate the software for the nodes of the application system. With such a system the users (i.e., application experts) will concentrate on the description of the application, not on computer engineering issues such as programming and hardware configuration.

Fig. 1. An Automatic Configuration System for Wireless Sensor Network Applications

4 Some of Our Major Approaches Due to space limitations we can only briefly introduce some of our approaches to constructing the system sketched in the previous section. 4.1 Smart Questions and Answers The interface between the user and our system is the QA system. Questions are designed hierarchically. The top level questions categorize the application that the user is developing, for example “military”, “environmental”, etc. Sub-categories allow the system to get into more details of the application; for example in the “environmental” category, one could choose from: “air pollution monitoring”, “animal migration monitoring”, etc., and then specify other information such as the monitoring area, measurement types, measurement frequency, etc. Because the questions are all related to the user’s application and the data entry is by clicking, a non-computer engineer will find it very easy to describe the application of interest. 4.2 Taxonomy Taxonomy will play a crucial role in the design of the QA sub-system, and there are several taxonomies described in the literature. In [10], WSNs are categorized as

60

J.C. Lee and L.J. Henschen

“Event Detection and Reporting”, “Data Gathering and Periodical Reporting”, “Sink Initiated Querying”, and “Tracking Based Applications”. In [11], the authors classify the communications within a sensor network into “application” and “infrastructure” and further classify the “application” into “cooperative” and “non-cooperative”. They classify data delivery into four categories: “continuous”, “event driven”, “observer (sink)-initiated”, and “hybrid”. Lan Bai, et. al. [12] categorize sensor network applications into “archetypes” based on 8 properties of sensor network applications: “mobility”, “initiation of sampling”, “initiation of data transmission”, “actuation”, “interactivity”, “data interpretation”, “data aggregation”, and “homogeneity”. Taxonomy will be used in many ways in our system, but particularly in the user interface (i.e., the QA system). For example, if the user indicates the WSN is of type “Data gathering and periodic reporting”, the system will prompt for the kinds of values to be measured and the sampling and reporting periods. 4.3 Knowledge and Deductive Rules Rules will be used to generate application specific parameters based on the answers chosen by the user. For example, a rule like “transportation” & “parking” & “existence of car” & “hourly” Æ (infrared&ultrasonic&light&webcom) & (((SBT80)&(Tmote Sky)) || Mica2) would indicate that a transportation application dealing with parking, in which detection of cars is the information sensed and reported hourly, should use four sensors per node (infrared, ultrasonic, light, and webcam) and one of two commercially available nodes (Tmote Sky or Mica2, [7][13]), etc. Additional QA could further refine these recommendations. 4.4 Models and Parameters Modeling (static and dynamic) plays an important role in our system. Our current work focuses on generating the programs that run on a node, and this is done by the dynamic model of our system. We think that in a WSN all the activities performed by nodes can be modeled as events; this includes sensing data, transmitting data, and system activity (e.g. put some part to sleep) among others. We model each event as a 6-tuple E=(P, C, A, R, L, D). C is a set of condition expressions (one of which can be time). A is a function or action, and R is a set of result values for the actions. L is the location id of the event. P is the set of events that this event depends on, and D is the set of events that depend on this event. For example, in the smart parking lot application, the arrival of a car is an event. The condition of the event will be that the measurements of the four sensors exceed the thresholds for those measurements. The actions may include recording the occupancy of this parking position, starting to count the time for this car, etc. This event triggers another event, namely sending a message to the control (sink) node with the node's identity L.

A Framework for a User Friendly Wireless Sensor Network Configuration System

61

5 Conclusion A tool like we have described, with a well-crafted user interface and an intelligent system behind it, will allow users with little or no WSN training to design WSNs in their application field.

References 1. Dix, A., Finlay, J., Abowd, G., Beale, R.: Human-Computer Interaction, 3rd edn., p. 181. Prentice Hall, Edinburgh Gate (2004) 2. Gehrke, J., Liu, L.: Sensor Network Application. Internet Computing (2006) 3. Akyildiz, I., Su, W., Sankarasubramaniam, Y., Cayirci, E.: Wireless Sensor Networks: A survey. Computer Networks 38, 393–422 (2002) 4. Arampatzis, T., Lygeros, J., Manesis, S.: A survey of Applications of Wireless Sensors and Wireless Sensor Networks. In: Proceedings of the 13th Mediterranean Conference on Control and Automation, pp. 719–724. IEEE Press, New York (2005) 5. Lewis, F.: Wireless Sensor Networks. In: Cook, D., Das, S. (eds.) Smart Environments: Technologies, Protocols, and Applications. John Wiley, New York (2004) 6. Tan, J.: A Scalable Graph Model and Coordination Algorithms for Mobile Sensor Networks. In: Li, Y., Thai, M., Wu, W. (eds.) Wireless Sensor Networks and Applications, pp. 65–83. Springer, Heidelberg (2008) 7. Hempstead, M., Lyons, M., Brooks, D., Wei, G.: Survey of Hardware Systems for Wireless Sensor Networks. Journal of Low Power Electronics 4, 1–10 (2008) 8. Levis, P., Madden, S., Polastre, J., Szewczyk, R., Whitehouse, K., Woo, A., Gay, D., Hill, J., Welsh, M., Brewer, E., Culler, D.: TinyOS: An Operating System for Sensor Networks., http://www.cs.berkeley.edu/~culler/AIIT/papers/TinyOS/levis0 6tinyos.pdf. 9. Gay, D., Levis, P., Culler, D., Brewer, E.: nesC 1.1 Language Reference Manual, http://nescc.sourceforge.net/papers/nesc-ref.pdf 10. Iyer, A., Kulkarni, S., Mhatre, V., Rosenberg, C.: A Taxonomy-based Approach to Design Large-scale Sensor Networks. In: Li, Y., Thai, M., Wu, W. (eds.) Wireless Sensor Networks and Applications, pp. 3–33. Springer, Heidelberg (2008) 11. Tilak, S., Abu-Ghazaleh, N., Heinzelman, W.: A Taxonomy of Micro-Sensor Network Models. Mobile Computing and Communications Review, 28–36 12. Bai, L., Dick, R., Dinda, P.: Archetype-Based Design: Sensor Network Programming for Application Experts, Not Just Programming Experts. In: Proceedings of the International Conference on Information Processing in Sensor Networks, pp. 85–96 (2009) 13. Lee, S., Yoon, D., Ghosh, A.: Intelligent Parking Lot Application Using Wireless Sensor Networks. In: International Symposium on Collaborative Technologies and Systems CTS (2008)

FlexRemote: Exploring the Effectiveness of Deformable User Interface as an Input Device for TV Sang-Su Lee1, Seungwoo Maeng1, Daeeop Kim1, Kun-Pyo Lee1, Wonkyum Lee2, Sangsik Kim2, and Sungkwan Jung2 1

Dept. of Industrial Design, KAIST. 335 Gwahangno Yusung-Gu, Daejeon, Republic of Korea 2 KAIST Institute, KAIST. 335 Gwahangno Yusung-Gu, Daejeon, Republic of Korea {sangsu.lee,maengoon,up4201,kplee}@kaist.ac.kr, {wklee,sskim98,skjung}@itc.kaist.ac.kr

Abstract. We tried to explore the possibility of using deformation-based interaction as a new input method to remote control. We selected eight basic commands to control TV and design deformation-based gestures for each command based on prior studies regarding organic user interface. A prototype, the FlexRemote, was made and tested by informal user study. The study showed that the users have higher preferences to the FlexRemote in comparison with the conventional input devices and the users can make intuitive interactions through physical deforms. From this positive explorative study, we are expecting that this study will present a new direction in intuitive and fun interaction method with TV in living room environments, which is gaining greater importance. Keywords: interaction design, deformation-based interaction, organic user interface.

1 Introduction The conventional TV is mainly composed of relatively simple interactions such channel or volume control. However, recently studies have actively been made on the use of TV as a home display which supports various media activities in the living room environment. Thus, methods to interact with TV in a remote distance in the living room are getting greater importance. Meanwhile, many researchers have been seeking new interface opportunities to move beyond traditional input methods, such as WIMP [8]. Various studies have been made to find out the possibility of deformation as a new interaction method [1,2,6,7]. This study sought the possibility of using organic user interface (OUI) [3], which is a new method different from the conventional ones in the remote distance interaction for TV. OUI is not a fixed form, but can change its form organically according to functions making intuitional interactions possible. We expected that the study will make more intuitional and interesting interactions possible. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 62–65, 2011. © Springer-Verlag Berlin Heidelberg 2011

FlexRemote: Exploring the Effectiveness of Deformable User Interface

63

This study tried to design and verify what effect the interactions through deforms have on the actual users as input devices for remote TVs. A prototype was made based on previous studies for understanding user towards deformable interface as an input method, and informal user tests were conducted. For the prototype we made, deformable gestures which were obtained through our prior user reference were mapped concerning commands which are applicable to information devices. FlexRemote, which is OUI device and can implement a total of 8 general commands through deformable gestures, was made. This device of 2-dimentional flexible plastic material can be easily flexed in any direction. This study looked into the effectiveness of deformable interactions as a remote input device which is remote from the display. It is expected that this study will present a new direction in comfortable interactions with TV in living room environments, which is gaining greater importance.

2 Designing Deformation-Based Interface We designed the FlexRemote, a deformation-based input device for smart TV, based on prior study regarding deformation-based interaction. For the commands for smart TV, we selected basic commands of general information device which is also applicable to the smart TV. We categorized commands into four: commands for device status, for navigation, for content manipulation, and for action. Two commands were selected for each category, total eight basic commands were selected from existing Category Status

Commands and Gestures

(Folding right to left)

(Folding left to right)

Power on

Power off

(Bending right side upward)

(Bending left side upward)

Next Channel/Next

Previous Channel/Next

Navigation

Content manipulation (Bending upward)

(Bending downward)

Volume up/Zoom-in

Volume down/Zoom-out

Action

(Bending the upper right corner)

(Swing)

Favorite channel/Scrap content

Delete content/Back

Fig. 1. Eight basic commands and deformation based gestures we designed for FlexRemote

64

S.-S. Lee et al.

studies [1,2,4,5,7]. More complex commands, such as typing texts, were excluded due to the difficulty of performing them with a deformation-based gesture at this exploratory study. For eight basic commands we have chosen, eight different deformationbased gestures was assigned according to our prior study of understanding how users manipulate deformable display as an input device [4]. In this prior work, we tried to understand deformation-based user gestures by observing users interacting with artificial deformable displays with various levels of flexibility, and we could gain userdefined gestures for 11 commands. We could map our 8 TV commands with those prior user-defined gestures. Figure1 shows how our 8 commands mapped with userdefined gestures.

3 Implementation and User Study We made our FlexRemote prototype based on our design. It consists of 16 flex sensors so that it can recognize eight deformation gestures we selected in prior section. (Figure2, left) We setup the virtual TV environment based on Windows PC system. User can control 8 commands we designed by manipulating our FlexRemote prototype. With this prototype we made, an informal user study was conducted with the students in our departments. We gave each participant a use case scenario which involves all 8 commands we designed. First we show them how those 8 commands work with 8 different deformation-based gestures, and then we asked participants to control TV by themselves (Figure2, right). Also we asked participants to do the same task using keyboard and mouse combination. From an in-depth interview with each participant, most users told us their higher preferences to the FlexRemote in comparison with the conventional input devices such as traditional remote controller or keyboard/mouse. Unlike the conventional method where users have to pay attention to the input device and the screen alternatively, the users can make intuitive interactions through physical deforms without moving their vision from TV. Most of the users became easily familiar with the commands. Regarding workload, the interviews showed that physical demand was rather high, but their joy from the new manipulation was greater than the physical demand. This may mean that physical demand can be an issue as the new effect in a long term, studies should be made on long-term effects of deformable interactions, and also comparative studies should be made with other methods such as body gestures which are recently studied.

Fig. 2. FlexRemote prototype (left) and user study (right)

FlexRemote: Exploring the Effectiveness of Deformable User Interface

65

4 Conclusion and Future Works In this study, we tried to explore the possibility of using deformation-based interaction as a new input method to remote control. We selected basic commands to control smart TV and design deformation-based gestures for each command based on prior studies regarding organic user interface. A prototype, the FlexRemote, was made and tested by informal user study. The study showed that the users have higher preferences to the FlexRemote in comparison with the conventional input devices and the users can make intuitive interactions through physical deforms. From this positive explorative study, we are expecting that this study will present a new direction in intuitive and fun interaction method with TV in living room environments, which is gaining greater importance. For the future works, we are planning to conduct user study within the real context; in the real living room environment when people actually watching TV programs. From this further study, we could gain understanding of social acceptability [6] of new interaction method which is also very important as a practical solution. Acknowledgment. This research was supported by WCU(World Class University) program through the National Research Foundation of Korea funded by the Ministry of Education, Science and Technology (R33-2008-000-10033-0). This work was also partially supported by the Industrial Strategic technology development program, KI002156 funded by the Ministry of Knowledge Economy(MKE, Korea).

References 1. Gallant, D.T., Seniuk, A.G., Vertegaal, R.: Towards more paper-like input: fexible input devices for foldable interaction styles. In: Proc. UIST 2008, pp. 283–286. ACM Press, New York (2008) 2. Holman, D., Vertegaal, R., Altosaar, M.: PaperWindows: Interaction Techniques for Digital Paper. In: Proc. CHI 2005, pp. 591–599. ACM Press, New York (2005) 3. Holman, D., Vertegaal, R.: Organic user interfaces: designing computers in any way, shape, or form. Communications of the ACM 51(6), 48–55 (2008) 4. Lee, S., Kim, S., Jin, B., Choi, E., Kim, B., Jia, X., Kim, D., Lee, K.: How users manipulate deformable displays as input devices. In: Proc. of CHI 2010, pp. 1647–1656. ACM, New York (2010) 5. McNeill, D.: Hand and Mind: What Gestures Reveal about Thought. University of Chicago Press, Chicago (1992) 6. Rico, J., Brewster, S.: Usable gestures for mobile interfaces: evaluating social acceptability. In: Proc. of CHI 2010, pp. 887–896 (2010) 7. Schwesig, C., Poupyrev, I., Mori, E.: Gummi: User Interface for Deformable Computers. In: Proc. CHI 2003, pp. 954–955. ACM Press, New York (2003) 8. Van Dam, A.: Post-WIMP user interfaces. Commun. ACM 40(2), 63–67 (1997)

A Study of User Needs for the ‘Techno Kitchen’ Martin Maguire1, Colette Nicolle1, Russell Marshall1, Ruth Sims1, Clare Lawton1, Sheila Peace2, and John Percival2 1

Loughborough Design School, Loughborough University, Ashby Road, Loughborough, Leics, LE11 3TU, UK [email protected] 2 Faculty of Health & Social Care, Horlock Building, 110 The Open University, Walton Hall, Milton Keynes, MK7 6AA, UK

Abstract. A project, Transitions in Kitchen Living (TiKL) has been conducted to study kitchen histories and current use by older people. A multidisciplinary team developed a common interview methodology to collect people’s personal kitchen histories and their views on their current kitchen. Now and in the future ambient assistance and applied electronic technology might provide additional functionality to make kitchens easier to use for all ages. This paper relates some of these technical developments to user needs identified in the TiKL project and reports on the results of a survey to obtain older people’s views on them. Keywords: kitchen, advanced technology, older people, people with disabilities.

1 Introduction The kitchen is an important space in the home. In order to support easier use and independent living for older people, it is desirable to provide, a kitchen environment that is comfortable so that tasks can be performed with a minimum amount of unnecessary effort and strain. Good design and physical ergonomics can help to achieve this. However to expand the possibilities to support people as they age, computer technology, if developed in a subtle and sensitive way can help people to conduct tasks in the kitchen. They can also help to monitor a person’s well being to detect if someone has fallen and needs help. There are many issues related to ethics, privacy, intuitiveness and ease of use that need to be considered before such systems are implemented [1]. It is well known that such technological developments are no substitute for an older person at home having human contact and support from friends and family. This paper develops the work of a project (Transitions in Kitchen Living - TiKL) conducted by social gerontologists at the Open University and ergonomists at Loughborough University in England and funded by the UK’s New Dynamics of Ageing Programme, to study people’s lives in relation to the kitchen [2, 3]. Forty eight older people, living in a range of accommodation in Bristol and Loughborough, took part. Each participant was interviewed about their ‘kitchen history’ – all the different kitchens they have experienced in their lives, and also their abilities, needs, likes and dislikes in relation to their current kitchen. Many of the participants had C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 66–71, 2011. © Springer-Verlag Berlin Heidelberg 2011

A Study of User Needs for the ‘Techno Kitchen’

67

developed simple and practical solutions to overcome the problems they faced. These included adapting cupboards for easier access, installing task lighting in dark corners, erecting lower work surfaces for use while seated (possibly in a wheelchair) and repositioning plugs for easier access. Smaller appliances such as table top dishwashers and microwave/cooker combinations were also found suitable when there was little space available. All these ideas were assembled and incorporated into a ‘Kitchen Living Guide’ to help other people adapt their kitchens to better meet their changing needs as they grow older. A natural extension of this study was to review the possibilities of using more advanced technology to make kitchen life easier. A small survey was also conducted with a group of older group of people to obtain their reactions to some of the ideas and concepts behind the ‘techo kitchen’.

2 General Problems and Requirements in the Kitchen The TiKL kitchen study reviewed the current kitchen environment and identified a range of general problems that people faced. Participants were asked about their capabilities with respect to kitchen activities. The following chart (Figure 1) shows the number of participants who did have specific capability problems. 30 25 6 20 15

3

2

Some probs. 4

10

16

15 5

Problems

20

9

0 Sight

Hearing

Reaching

Dexterity

Fig. 1. Numbers of participants experiencing physical limitations out of total of 48 (‘Some problems’ indicates those saying they had no problems but then described problems or coping strategies)

It can be seen that the physical limitation of reaching (up, down or in front) affected 26 people (54% of the sample), while problems of lack of dexterity and strength, poor sight and hearing also affected significant proportions of the sample. The interview study identified specific task related problems in the kitchen resulting from these physical limitations as described by the project participants. It also recorded solutions that people had adopted and provided a stimulus to think about new possible solutions.

68

M. Maguire et al.

Table 1 below lists 16 common problems that people might face in the kitchen (both older and younger people) alongside solutions that could be adopted to address them. The second column shows traditional kitchen design and ergonomic solutions that arose during the study. The third column presents more advanced technological solutions that could be adopted or developed. Table 1. Comparison of possible ‘low tech’ and ‘hi tech’ solutions to problems identified in the kitchen study Problem

Possible ‘low tech’ solution

Possible ‘hi tech’ solution

1. Poor lighting in parts of kitchen for doing tasks 2. Tiredness and needing to sit down to carry out tasks 3. Difficulty in reaching window or blind over sink

Install under cupboard lights or table lamps with switches Table or lower work surface for sitting or perching stool Reposition catches to be at bottom of window frame

4. Bending down to low or reaching up to high shelves. 5. Reading small instructions on packaged food. 6. Bending down to check cooking in oven/take out pan 7. Transporting food from oven to work surface or table. 8. Drudgery of washing up

Relocate shelving height. Pull out or carousel shelving. Improved lighting to read instructions or magnifier Oven placed at waist height. Light- check without opening Trolley to help transfer items between kitchen-dining room Wash up in stages

Activity is sensed and light is turned on automatically Work surface motorized so can be raised or lowered Remote control opening and closing of windows/blinds Motorised cupboards that can be moved up and down Gadget to scan and read instructions aloud.

9. Concern that iron, oven or kettle left on when out

Return to house or contact neighbour to check

10. Hearing door bell or phone with kettle on 11. Needing help with shopping. 12. Lack of space for ironing board in the kitchen 13. Lack of plug points. 14. Kettle too heavy to lift to tap to fill. 15. Wash basket too heavy to take outside. 16. General cleaning (floor, on cupboards, behind units)

‘Robot’ to transfer items between kitchen-dining room Dishwasher installed (perhaps small unit) Auto shut off of these items if left on too long or if go out

Neighbour, friend or relative helps with transport. Additional space (e.g. utility room) for board to be set up. Additional electricity points installed

Visual alert if door bell pressed or phone rings Online shopping service. Fridge notifies of items need. Pull out ironing surface built into kitchen units. Future gadgets may be powered wirelessly

Smaller kettle that can be lifted more easily. Lighter washing basket. Use with smaller loads. Easy clean floor and work surfaces.

Hot water unit to provide hot water ‘on tap’ ‘Robot’ to carry clothes to outside ‘Robot’ kitchen cleaner with access behind units

A Study of User Needs for the ‘Techno Kitchen’

69

3 Survey to Explore Requirements for Technological Support Drawing upon the list of possible innovations in Table 1, a similar group of older people (21 in total, 16 females, 5 males, from 50 to 85 and an average age of 71) were surveyed to think about a range of possible technological enhancements that might support them in the kitchen. Each respondent was asked to state whether they would want each innovation or not. A summary of the survey results are shown in Table 2. Table 2. User reactions to ‘hi-tech’ innovations in the kitchen (N=21) Innovation 1. Press button on fridge to read out food items present or run out of.

% Approval 33%

Comments Would help with shopping. Good idea for freezer where more food stored. Opening door less saves energy. Unsure of technology behind it. Would you trust it? Would also need to know if items in date. Would need to log food items in and out.

2. Sensor alert if smoke, CO, fire or gas leak when you are out.

76%

Would help with safety. Good idea. Who would come to deal with it? If return to house and has been leak, putting on light could cause fire.

3. Quick cooling oven hob.

76%

Would be safer especially for grand children. Good idea – have sometimes been caught out by residual heat on ceramic hob.

4. Shut off electrical equipment when you leave the house.

90%

Yes it is easy to go out and forget that things still on. Would be safer. Save money on electric not being used. Peace of mind.

5. Washing machine has only 6 buttons for easy use.

75%

Only use 2 or 3 programmes most of time. Less confusing. Too many programmes never used. Am happy with dial and choice.

6. Open/close windows and blinds with button press.

76%

Would be handy as window situated over sink. Would make life easier often have to stretch to close them. Would be safer. Blind may hit items on window sill. Would make us lazy.

7. Water sensor welfare alert if taps not run for some time.

43%

Daughter regularly checks I’m OK. Useful for emergencies e.g. if someone has fallen. You would have to be able to cancel the system if went on holiday.

70

M. Maguire et al. Table 2. (Continued)

Innovation 8. Automatic task light when working in dark corner.

% Approval 71%

Comments Very useful. Could see better and avoid eye strain. Could save energy. Safety and emergencies in the dark. I wouldn’t need to stay in the corner. Need good light all the time.

9. You or others alerted if water on floor indicates possible flood.

75%

Useful – water can run a long time before noticed. Would be safer and avoid waste. Too many alarms may make you worry about too many minor things.

10. Simple touch screen to order shopping and have delivered.

24%

Save carrying heavy shopping. Like the choice or to check quality in store. Feel too complicated for the people intended to help. If became disabled friends would assist.

11. Recipe display on wall so easier to follow when cooking.

52%

May make it more efficient and save writing recipe’s down. Don’t follow recipes. Prefer to print with large font. Interested to see prototype.

12. Instructions read out about food preparation and cooking actions.

24%

Cooking an effort so could help if tired. Would be useful if can’t read cooking instructions label. I know how to cook things. Prefer to cook at my pace.

There was also some interest in adjustable wall cupboards that could move up and down (42% approval) electrically on runners for easier access to higher cupboard shelves. One person stated that “As a 4 foot 11 inch person this would be wonderful and would suit households with people of different heights and reach capability”. Another said that this “Would bring the cupboard down to their eye line and reach”. There was keen interest in a worktop that can be lowered temporarily for use while seated (67%). It was thought that this “Would keep people’s interest in cooking and independence later in life”. However it was also said that this could reduce cupboard space potential in the kitchen. Another idea was a facility in the kitchen to read out the small print (instructions, ingredients) on food packaging or other products (50%). Participants were also asked to propose their own innovations. There was a requirement from two people for more help to be energy or water efficient in the kitchen, for example, a plug to cut down on energy use in the kitchen by avoiding standby or a better mixer tap so that hot water was delivered quickly rather than running it until it warmed up. Three people suggested a facility (perhaps in the fridge) to help assess food quality e.g. “Has egg or cheese gone off?” Some people wanted to hand over complete tasks to technology: “Something to do the ironing” and “A robot that could cook, clean, serve food, do the shopping and the washing and drying up!” The results of the survey show that there is enthusiasm for certain types of technological support in the kitchen provided it is well thought through and designed

A Study of User Needs for the ‘Techno Kitchen’

71

to meet people’s needs. An understanding of older people’s viewpoints is important to ensure that such developments are useful and practical to kitchen users in the future.

References 1. Gill, J.: Ambient intelligence - paving the way - how integrated services can deliver a more inclusive society. In: EU RTD Framework project COST 219 (2008) 2. Maguire, M., Nicolle, C., Marshall, R., Sims, C., Lawton, L., Peace, S., Percival, J.: A study of the age-friendliness of kitchens. In: INCLUDE Conference, Helen Hamlyn Centre for Design, Royal College of Art, London (April 18-20 , 2011) 3. Sims, R.E., Marshall, R., Maguire, M., Nicolle, C., Lawton, C., Peace, S., Percival, J.: Older people’s experiences of their kitchens: Dishes and wishes, Contemporary and Human Factors 2011. In: Anderson, M. (ed.) Proceedings of Ergonomics & Human Factor 2011, Stoke Rochford, UK, April 12-14, pp. 387–393. Taylor & Francis Group, London (2011)

Consideration of the Human-Computer Interface in the Operation Room in the Era of Computer Aided Surgery Kazuhiko Shinohara School of Health Science, Tokyo University of Technology, Nishikamata, Ohta, Tokyo,144-8535, Japan [email protected]

Abstract. Human-computer interactions in the operation room (OR) are rapidly increasing as a result of advances in endoscopic surgery and the implementation of computerized physicians order entry systems (CPOE). Not only computerized medical devices and robotics, but also various image guidance systems with augmented reality to support endoscopic surgery, are being developed and some are already widely used in the OR. While CPOE provide the surgical team with quick and wide-ranging access to individual patients’ medical records and images while in the OR, essentially the procedures undertaken by surgeons and scrub nurses cannot be changed. In this study, ergonomic problems involving the human-computer interface in the operation room were investigated for the smooth uses of computers in this era of computer-aided surgery. The amount of cords, cables and medical tubing is increased between 3- and 5-fold over that used in the conventional OR, which carries latent risks for fault connections between the equipment and slipping and tripping accidents among the OR team. The use of a CPOE requires additional movement to the computer terminal during surgery. If fingerprint recognition is used as the CPOE biometrics system, surgeons and scrub nurses cannot access the CPOE themselves during operations. As countermeasures against such problems, the integration of cables and wires, improvement of the WIFI environment and electromagnetic compatibility with medical devices, and implementation of tablet PCs with voice recognition or touch screens are required. Moreover, it is essential for the human-computer interface in the OR to be further improved by accounting for OR-staff’s work flow. Keywords: computer aided surgery, operation room, human-computer interface.

1 Introduction Human-computer interactions in the operating room (OR) are rapidly increasing as a result of advances in endoscopic surgery and the implementation of computerized physician order entry systems (CPOE). Endoscopic surgery is a typical example of minimally invasive surgery, in which internal organs are operated on via small-caliber channels under endoscopic view. Much of the endoscopic procedure depends on computerized medical electronic devices, including surgical robots and image-guidance C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 72–75, 2011. © Springer-Verlag Berlin Heidelberg 2011

Consideration of the Human-Computer Interface in the Operation Room

73

systems. Thus endoscopic surgery leads the way in computer-aided surgery. Furthermore, implementation of CPOE in the OR is rapidly progressing in Japan. These systems started as an order and reservation entry system, however their scope is becoming broader and is leading to the realization of a “paperless and filmless hospital”. The aim of this study was to investigate the problems associated with the implementation of the human-computer interface during the era of computer-aided surgery.(Fig.1)

2 Materials and Methods We compared the workflow and ergonomic problems of the OR staff during a typical operation for colon cancer using computer-aided endoscopic surgery with a CPOE versus conventional open surgery with paper-based medical records. In this study, the CPOE refers not only to an order and reservation entry system but also to the system for entering and viewing laboratory data and clinical images, such as ultrasonography, computed tomography, and X-ray images.

3 Results 3.1 Additional Devices Required for the Endoscopic Surgery and CPOE in OR For endoscopic surgery, additional medical electronic apparatus such as an electronic endoscopy system (including endoscope, light source, image controller, and video recorder), an automatic insufflator (pneumoperitoneum apparatus), and an ultrasonic cutting and coagulation system are required. In the case of robotic surgery, a surgical manipulator such as the da VinciTM system is also required. All this apparatus is computer controlled. By contrast, the only medical electronic apparatus required for conventional open surgery is the electrocautery device. Thus, human-computer interaction during endoscopic surgery has increased compared with conventional surgery. The number of cords, cables, and medical tubes between the operative field and the medical apparatus has also increased between 3- and 5-fold compared with conventional open surgery. 3.2 Additional Work and Problems Associated with CPOE in the OR To access a patient’s medical records and imaging data (such as computed tomography, ultrasonography, and X-ray images) during surgery, surgeons and anesthesiologists must move to the CPOE terminals each time. Also, difficulties have been encountered in accessing data on the CPOE terminals. Similarly, nurses and clinical engineering technicians who operate various medical devices must also move to and access the CPOE terminals for personal authentication of the patient, reading the patient’s medical records and laboratory data, and ordering laboratory tests. Nurses must also access the system during the operation to record events, such as the infusion of intravenous fluids or administration of drugs. The clinical engineering technicians must input data from off-line medical devices. In the case of a filmless and paperless OR, nurses and technicians must additionally record separately the lot number of blood and implants that are used during surgery.

74

K. Shinohara

a

b Fig. 1. A scene of endoscopic surgery(a) and CPOE terminal in the OR(b)

4 Discussion Human-computer interactions in the OR are rapidly increasing in this era of endoscopic surgery and CPOE compared with in the era of conventional open surgery and paper- and film-based medical records. Since modern endoscopic surgery began in the 1990s, indications for its use have greatly increased. The main merit of endoscopic surgery for the patient is minimal invasiveness, but it brings the disadvantage that surgeons have to operate on the internal organs via small-caliber channels with restricted tactile sense and visual information. To compensate for this, many types of medical electronic apparatus have been developed. Not only computerized medical devices and robotics, but also various image-guidance systems with augmented reality to support endoscopic surgery are already widely used in the OR. All these types of apparatus are controlled by computers. In addition, the implementation of CPOE is progressing rapidly in Japan and may eventually lead to the realization of a paperless and filmless hospital. The system covers e-prescriptions, medical records, hospital accounts, and logistic systems among others. It includes not only verbal and numerical medical records but also medical images, such as X-rays, computed tomography scans, magnetic resonance images, endoscopy, and photographs of surgical specimens. The system allows for secure and rapid transmission of medical information as well as improves patient safety by providing computerized references and check systems for prescribing medication. CPOE systems also provide the surgical team with quick and wideranging access to patients’ medical records and images while in the OR. However there is also resistance in the OR to adapting to using CPOE due to perceptions regarding the impact of the human-computer interaction on the busy and stressful workflow in the OR. From this study of the ergonomic issues involved in the human-computer interface in the OR, several problems are revealed. The number of cords, cables, and medical tubes is increased compared with conventional open surgery. This increase provides latent risks for faulty connections between the equipment and also for slipping or

Consideration of the Human-Computer Interface in the Operation Room

75

tripping accidents among the OR team. The use of a CPOE requires additional movements to the computer terminal during surgery. Such movement from staff members’ positions to the CPOE terminal not only complicates workflow, but also increases the risk that individuals may neglect their duties; this is especially true for anesthesiologists, clinical engineering technicians, and nurses who must closely monitor machines. These movements provide an additional workload for the OR staff and potential risks for slipping/tripping accidents within the team. For surgeons and scrub nurses, there are difficulties in operating the keyboard and mouse of the computer terminal during surgery. If fingerprint recognition is used as the CPOE biometrics system, surgeons and scrub nurses cannot access the CPOE themselves during operations without interrupting the surgery to access the system. As a countermeasure against these problems, cables and wires must be integrated, and improvements are required in the WiFi environment and electromagnetic compatibility with medical devices, as well as in the use of tablet PCs with voice recognition or touch screens. Application of a mobile terminal with eye-movement control and/or voice control may be one solution. However, care must be taken in the design of the human-computer interface to avoid disturbing the intuitive and finely coordinated activities of surgeons during surgery. Workflow analysis of the OR staff during surgery is a fundamental requirement for the development of an ergonomically acceptable human-computer interface in the OR in this era of computer-aided surgery.

5 Conclusion For the smooth and safe implementation of medical electronic devices and CPOE in computer-aided surgery, it is essential for the human-computer interface to be further improved to accommodate OR staff characteristics and workflow. Basic education on computer information technology is also necessary as part of medical education and training.

References 1. Shinohara, K.: Implemantation problems of computerized physician order entry system in the operation room. In: Proceeding of Human computer interaction 2009 (CDR) (2009) 2. Shinohara, K.: Safety and ergonomic problems on the clinical practice of gastrointestinal endoscopy. In: Proceeding of 2nd International conference on Applied Ergonomics (CDR) (2008) 3. Benedict, A.J.: Electronic Prescribing Adoption. In: Proceeding of 2nd International conference on Applied Ergonomics barriers and influencing factors (CDR) (2008)

‘STISIM-Drive’ Meets ‘MotorcycleSim’: Using Driving Simulation Software to Develop a Unique Motorcycle Simulator for Rider Behavior Research Alex W. Stedmon1, David Crundall2, Elizabeth Crundall2, Rose Saikayasit1, Editha van Loon2, Alex Irune3, Patrick Ward2, and Neil Greig4 1

Centre for Motorcycle Ergonomics & Rider Human Factors, University of Nottingham, UK [email protected] 2 School of Psychology, University of Nottingham, UK 3 Dimax Technology Ltd, Nottingham, UK 4 The Institute of Advanced Motorists, IAM House, London, UK

Abstract. In order to compare groups of road users who have fundamentally different skills, attitudes and behaviours, one of the first in-depth motorcycle simulation studies of its kind was conducted. The project was developed using the innovative ‘MotorcycleSim’ simulator designed and built at the University of Nottingham. The simulator is a research tool to investigate aspects of motorcycle ergonomics and rider human factors and is the first of its kind in the world to incorporate ‘STI-SIM Drive’ software that allows motorcyclists to ride a full size motorcycle and interact with a virtual riding environment (VRE). To build a simulator that was both fit for the purpose of research and provided the desired levels of fidelity associated with real world riding, a user-centred design process was adopted from the outset (based in principles of ISO:13407). Keywords: transport simulation, motorcycles, rider behavior.

1 Introduction Capturing road user experiences in the real world is extremely valuable and underpins the applied nature of ergonomics within human-machine interaction [1]. However, due to practical, safety and ethical issues surrounding real world research, simulators offer the potential to develop alternative research tools [2]. Simulators offer a level of abstraction from the real world by providing an artificial environment in which users can experience characteristics of a real system [2]. Whilst there are limits to the degree of realism that can be achieved in simulators, if they are designed well, they offer high levels of ecological validity [1]. Underpinning this is a key human factors question regarding the level of realism required for a simulator to serve its intended purpose (e.g. training, research, product development, etc) based on a fundamental understanding of user-centred design principles [3]. From the outset, a detailed user-centred design methodology was adopted based on principles of ISO:13407 [4]. Without such an approach there is always a risk of specifying a solution that fails to support the users’ understanding of the target C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 76–80, 2011. © Springer-Verlag Berlin Heidelberg 2011

‘STISIM-Drive’ Meets ‘MotorcycleSim’: Using Driving Simulation Software

77

application [5]. A questionnaire survey incorporated 13 characteristics that participants rated on a 5-point scale [1]. The results indicated that the riders required the simulator to lean like a real motorcycle, have acceleration and braking effects and for the controls to work in a realistic manner. From additional comments that were made, riders highlighted the level of realism the simulation software might offer and how other traffic and road conditions could be incorporated into the virtual riding environment (VRE). Furthermore, a number of riders felt the simulator should be capable of replicating vibration and allow for riding with a passenger. The user requirements formed the basis of the development of MotorcycleSim as well as highlighting development routes for the future [1].

2 MotorcycleSim: An Overview MotorcycleSim has been built using a full size and fully equipped Triumph Daytona 675 motorcycle (kindly supplied by Triumph Motorcycles). The existing motorcycle controls (e.g. throttle, brake lever, brake pedal, gear selector, clutch lever) were modified or adapted to work with the simulation software in a realistic manner. MotorcycleSim uses commercially available ‘STISIM-Drive’ software which is an industry leader and has a worldwide support community. Although primarily a driving simulation package, the software can be used to build interactive VREs that allow for strictly controlled experimental repeatability in a laboratory setting as riders experience identical scenarios which are not possible on the road [3]. Within the ‘STISIM-Drive’ software it is possible to change vehicle characteristics to alter the simulated handling and performance profile of the motorcycle. Given focus of the software on driving applications, the vehicle characteristics have been fine-tuned to mimic a motorcycle and to a large extent this has been achieved by ‘fooling’ the software that the vehicle is a small and lightweight car with the acceleration and braking characteristics of a motorcycle. Alongside this, bespoke software and engineering solutions were developed to integrate the motorcycle hardware with the simulation software [2]. The visual representation (projected onto a large screen in front of the rider) can be manipulated in a number of ways: • enhanced acceleration and braking effects are presented by altering the degree of dynamic pitch in the visual scene as the motorcycle accelerates and slows down; • the scenery tilts as the rider steers in a particular direction to enhance the perception into leaning into a corner • the rider’s field of view can be increased to take account of peripheral visual cues. MotorcycleSim can be used in static or dynamic modes. In the dynamic mode the motorcycle tilts as the bespoke software takes data for speed and the steering angle in order to calculate an angle of lean (Figure 1).

78

A.W. Stedmon et al.

Fig. 1. The unique titling action of MotorcycleSim

MotorcycleSim, has been designed to offer four different simulation modes: static simulation with ‘STISIM-Drive’; cornering/leaning training; cornering/leaning training with ‘STISIM-Drive’; and an advanced dynamic model with ‘STISIM-Drive’ that aims to offer a realistic riding experience by utilising a comprehensive dynamics model to control the leaning of the motorcycle as the rider navigates the VRE.

3 Integrating MotorcycleSim with ‘STISIM-Drive’ In order to interact with the ‘STI-SIM Drive’ software, the mechanical movement of the throttle, braking, gear selection and steering angle of the motorcycle was interfaced with a standard automotive games controller unit (Figure 2). This solution was used as no precedent for a motorcycle controller exists with ‘STISIM-Drive’ and as such, a novel solution had to be explored: • throttle – the throttle on the motorcycle was connected to the games controller accelerator pedal via the standard throttle cable. When the throttle on the motorcycle is applied the throttle cable pulls the spring loaded foot pedal down. The degree of movement can be calibrated to different ranges to represent different motorcycle throttle characteristics. • braking – a limitation of the games controller is that the standard car brake pedal interface only offers a single braking input. On motorcycles using both the front and rear brake together or independently is a key riding skill and so an innovative solution was developed. Integrated and distributed ratios were developed so that the functional fidelity of the brakes is therefore preserved as the user is required to use both brakes to get the maximum braking effort (as would be expected on a real motorcycle). • gear selection – the motorcycle gears are interfaced with the ‘STISIM-Drive’ software using two ‘push to make’ switches wired to auxiliary buttons on the gear selector unit. The device was designed and manufactured to operate as a real motorbike, and not interfere with the rider’s natural riding position and gear changing action, and to have a minimal impact on the aesthetics of the motorcycle.

‘STISIM-Drive’ Meets ‘MotorcycleSim’: Using Driving Simulation Software

79

• steering angle - the steering wheel was removed from games controller so that the steering column could be connected to the steering unit of the motorcycle. A bespoke design allows for an initial free rotation of the column to allow for the ‘STISIM-Drive’ ±720o steering calibration.

Fig. 2. Integrated motorcycle controls with an automotive games controller

4 Using and Evaluating MotorcycleSim MotorcycleSim provided the basis for an in depth investigation of rider behaviour given its unique ability to test rider behaviour in a controlled environment [6]. Within a main riding scenario various sub-scenarios were developed and additional tasks were designed within the integrated experiment approach in order to explore a wide range of skills, attitudes and behaviours across three rider experience groups. The study was designed to compare Novice, Experienced and Advanced trained riders and the process adopted allowed for comprehensive data collection and analysis with minimal disruption to participants or biases creeping into the data [6]. By combining ‘STISIM-Drive’ and ‘MotorcycleSim’ this approach demonstrated the power and innovation of exploring a variety of rider attributes in a single investigation. Furthermore, and in line with the iterative principles of ISO:13407, an evaluation of MotorcycleSim has been conducted which have provided evidence to support the effectiveness of static simulation as well as ideas for future development [3].

5 Conclusions MotorcycleSim is at the forefront of motorcycle simulation. No other simulator exists like it, that provides a full size, interactive, realistically controlled, accurate performance measurement platform which can be used to investigate motorcycle design, rider equipment, rider behaviour, road safety and rider training and competence. A major strength in its development has been the underlying user-centred approach in capturing

80

A.W. Stedmon et al.

user requirements at the earliest opportunity which have provided a basis for the initial design and onward development of MotorcycleSim. The potential of transport ergonomics in all its guises is vast and novel methods and procedures are likely to emerge in order to better understand the fine detail of road user behaviour. Simulated trials will continue to provide a valuable research tool for investigators and given the investment in technologies and equipment, it is often necessary to design integrated approaches that offer the greatest potential to capture data on many levels. Acknowledgement. The authors would like to thank The Institute of Advanced Motorists, Phoenix Distribution (Arai helmets) and ‘STISIM-Drive’ for their support.

References 1. Stedmon, A.W., Brickell, E., Hancox, M., Noble, J., Rice, D.: MotorcycleSim: a usercentred approach for an innovative motorcycle simulator. In: Golightly, D., Rose, T., Bonner, J., Boyd Davis, S. (eds.) Creative Inventions & Innovations for Everyday HCI. The Ergonomics Society, Leicester (2008) 2. Stedmon, A.W., Young, M., Hasseldine, B.: Keeping it real or faking it: The trials and tribulations or real road studies and simulator research in transport research. In: Bust, P. (ed.) Contemporary Ergonomics 2009. Taylor & Francis, London (2009) 3. Stedmon, A.W., Hasseldine, B., Rice, D., Young, M., Markham, S., Hancox, M., Brickell, E., Noble, J.: MotorcycleSim: an evaluation of rider interaction with an innovative motorcycle simulator. The Computer Journal (2009), doi:10.1093/comjnl/bxp071 4. Earthy, J., Sherwood Jones, B., Bevan, N.: The improvement of human-centred processes: Facing the challenge and reaping the benefit of ISO:13407. International Journal of Human Computer Studies 55(4), 553–585 (2001) 5. Stone, R.J.: Virtual Reality. Virtual and Synthetic Environments: Technologies and Applications. In: Karkowski, W. (ed.) International Encyclopaedia of Ergonomics and Human Factors. Taylor & Francis, London (2001) 6. Stedmon, A.W., Crundall, D., Crundall, E., Saikayasit, R., Ward, P., van Loon, E., Irune, A.A.: Advanced Training & Rider Performance. University of Nottingham (2010)

AirportLogic: Usability Testing, Prototyping, and Analysis of an Airport Wayfinding Application Bennett Stone¹ and Yun Wang² 1

Department of Human Computer Interaction, Iowa State University, USA 2 Department of Graphic Design, Iowa State University, USA [email protected], [email protected]

Abstract. As air travel continues to be prevalent in our lives, and as the use of mobile devices designed to assist us become more common, the creation of wayfinding applications for mobile devices continues to be a challenge for both developers and interface designers. Unfortunately, many airport specific wayfinding applications do not live up to user expectations. In preliminary project phases, usability testing and evaluation of existing applications was performed, followed by prototyping and evaluation of a proposed application. Task-based user testing was performed to address two primary questions: 1) are current and proposed applications efficient for users?, and 2) is the information architecture and navigation suitable for average users? A prototype application and corresponding website were developed, and tested, with user testing confirming enhancements over previous airport way finding applications. Keywords: user interface design, mobile application, wayfinding.

1 Introduction Responsible for predicting user needs and technological capacity, most airport specific way finding applications ultimately do not live up to user expectations. After investigating a several airport navigational tools and applications for the iPhone, it was discovered that there is no single application that provides both basic and specific functionality necessary for travelers to effectively locate essential airport services and locations. The primary problem is that existing airport wayfinding and navigational assistance applications, which are designed for use on mobile devices have failed to provide both effective user interfaces, and useful way finding tools and information, essentially rendering them useless to travelers. Many existing applications contained unnecessarily large amounts of information in visual interfaces that were nearly impossible for individuals to fully use. In addition, several other applications were specific to only certain airport attractions such as pubs, airlines, or shopping, leaving all other potential points of interest up to the traveler to locate without the assistance of the application. In this project, the tasks included creating a simplified and intuitive mobile application for the iPhone in order to assist with common wayfinding tasks in both United States and global airports, as well as a developing corresponding websites that C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 81–84, 2011. © Springer-Verlag Berlin Heidelberg 2011

82

B. Stone and Y. Wang

would provide valuable pre-travel assistance with tasks such as booking airline, hotel, and rental car reservations; and creating an effective linking system between the mobile application and website. Tasks within the mobile application are assigned a rank order based on preliminary user feedback and evaluation of existing mobile applications in order to increase the speed and efficiency of wayfinding and location determination through a more intuitive user interface. Using task-based user testing with eight participants for the full testing, each participant was asked to complete several tasks based on a researcher provided scenario. Each participant was asked to complete basic operations within the AirportLogic application such as navigating to a gate within a selected airport, locating power outlets, and locating public transportation to the nearest city.

2 Method Preliminary user testing was performed using six participants selected from a design course, using existing airport wayfinding applications in order to effectively determine both necessary and unnecessary functionality, as well as isolating problem areas within each application. After preliminary user testing, a prototype application was developed and produced as a task-specific-functional1 prototype. Having designed the application to meet only essential users needs while traveling, the navigational structure was simplified through several stages of refinement prior to user testing, which was shown to be successful (100% of participants were able to perform the requested tasks). This task based user testing was designed to address three primary questions: 1) is the application efficient for the user? Does it allow a user to locate primary facilities, functions, and directions within an airport? 2) is the information architecture and navigation suitable for average users? and 3) are average users able to expediently find what they are looking for within the application?

Fig. 1. Left to right: Identification of problem areas during user testing, and solutions

1

Only pages, and information relevant to assigned tasks airport wayfinding application were made available. Participants were presented with an error screen when attempting to navigate to areas of the application that were not part of the user testing.

AirportLogic: Usability Testing, Prototyping, and Analysis

83

2.1 Participants Eight participants were randomly selected from a pool of participants for this research, each travelling by airplane at least 1 time per year (round trip). Table 1. Participant Information

2.2 Procedure Participants were asked to complete a pre-testing user profile indicating gender, native language, university status, computer and mobile application usage frequency, specific mobile travel application usage, and number of annual trips by airplane, after which they were directed to a task list. Participants were instructed to proceed through each task while thinking aloud. The list of tasks they were asked to complete are shown in Table 2. AirportLogic testing was performed using an iPhone simulator on a 15” MacBook Pro, using an external two-button scroll mouse while user application interactions were recorded internally using QuickTime’s screen recording feature. A researcher was present throughout the user testing to answer questions, take notes, and when necessary, redirect participants. After completion of user testing, participants were asked to complete a post use questionnaire that requested feedback on application usability and functionality, organization, overall impressions, and freeform comments. 2.3 Measures Data collected included user information, total number of user clicks per task, total time per task, number of errors per task, number of incorrect menu items selected, and number of incorrect icons selected2, and information regarding perceived usability. Data was analyzed and grouped by demographic variables, task, and user average as compared to ideal3 number of clicks or time spent on task. 2

3

Incorrect icon selection was determined by participant selection of a navigational icon that had already been selected, or was available with fewer steps. Ideal refers to the maximum number of selections, or time as determined by researchers through preliminary testing of the AirportLogic application.

84

B. Stone and Y. Wang Table 2. Task list and description

Locate and initialize AirportLogic Locate an international airport Navigate to a gate Locate a restroom near the gate Locate a power outlet near the gate Locate favorites within the application Locate a Starbucks near current location Locate public transportation to the city

Used to determine icon legibility No analysis: single option only Locate and identify terminal 1, concourse C, gate 20 Locating the nearest restroom to current user location Locating the nearest power outlet to current location Locate frequently used options within AirportLogic Locate Starbucks, and add to favorites menu. Multiple-step task Locate a train from the airport to downtown.

3 Results User testing showed positive results for the AirportLogic prototype; 100% of participants were able to successfully complete the user testing, 87.5% of participants were satisfied with the application, and there were no tasks rated as “difficult” or “problematic” by users. While a number of participants selected areas on screen that were not attached to menu items, resulting in an increased number of attempted item selections for locating tasks located within map display items, the majority of tasks were completed using minimal item selection, and were rated as being successful as compared to the researcher ideal number of selections and time periods.

4 Discussions and Conclusion This analysis of existing airport wayfinding applications, and design of a more focused and usable wayfinding application proved to be successful. Though there were problems identified within the interface used for testing and analysis, user ability to navigate through primary functionality was dramatically improved, while the final implementation of the AirportLogic application addresses all user-testing located issues. This ultimately led to the generation and implementation of innovative new ways of allowing individuals to navigate to primary locations of interest within airports. Future directions for this research include the implementation of iPhone-based prototype for user testing, as well as performing user testing within an airport.

Using on-Bicycle Rider Assistant Device While Cycling: A Hazard Perception Assessment Chao-Yang Yang and Cheng-Tse Wu Department of Industrial Design Chang Gung University Kweishan, Taoyuan, 333, Taiwan [email protected]

Abstract. Technologies have been developed to assist the cyclist for precise training exercise such as monitoring their body and bicycle state. Due to the limited position placing on-bicycle rider assistant device and riding posture, it enlarges the essential view range of the chance the cyclist distracted from the road. The risks using the system have been evaluated through an interactive video-based hazard perception/reaction test in which response times were measured. 5 participants, 3 male and 2 female participated a 2-phase section including 7 traffic scenes in a 5 minutes cycling exercise with and without using on-bicycle rider assistant device. As the results, average reaction times tended to increase 0.33 seconds and average mis-maintained pushing cadence time was 31.4 seconds when cycling with the device assistance. It is concluded that cadence control assisted by the device can interfere the efficiency of hazard perception, an important safety factor of cycling. Keywords: on-bicycle rider assistant device, hazard perception, response time.

1 Introduction Cycling becomes a popular leisure activity due to green issues and increasing request of changing lifestyle. For different purpose, bicycle has been designed into different functions such as road bike is for speeding and mountain bike is for crossing multi terrain. Cycling is seen as substitutions of fashion, taste and professional sport in which various add on accessories have been innovated e.g. riding information device becomes necessary for cyclist to monitor the distance, speed, heart rate, Revolution Per Minute (RPM), road slope, elevation, and GPS map. Road bicycle in nowadays can reach the speed of 40 kilometer per hour with ease and the risks to involve into an accident are high and it is possible to be fatal. Apart from riding, using these devices has created additional tasks that may increase the cyclist’s mental load and distract their attention to the road conditions. Recent researches of transportation safety mainly focus on car driving. Brown, Tickner and Simmonds (1969) have pointed out that two types of resource interference are generated from using mobile phone in car driving: interference between visual and hand controlling requirement; interference between conversation and driving behavior. This behavior could increase risk of accident by weakening C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 85–89, 2011. © Springer-Verlag Berlin Heidelberg 2011

86

C.-Y. Yang and C.-T. Wu

driver’s performance in speed control, straightness maintenance, sudden hazard response and ignorance of road signs (McKenna & Crick, 1997; Sagberg & Bjørnskau, 2006). Using telephone at the time of the crash was 2.4 times more than the crash of being at fault (Sagberg, 2001), and risk of accident was decreased with response time of hazards perception (Sagberg & Bjørnskau, 2006). However, these researches may not be enough to indicate the harness using latest On-Bicycle Rider Assistant Device (OBRAD) during riding a bicycle, as cycling activity is more dynamic and physical exercise intensive. For example, to maintain the speed, legs are pushing the paddles according to the road conditions. This could stress cyclist’s mental load and weaken his/her judgment of hazards. Therefore, we aimed to investigate cyclist’s safety risks through assessing his/her response time in detecting the hazard on the road in using OBRAD.

2 Methods To prevent uncontrolled factors such as weather or traffic, as shown in Fig 1, the experiment environment was setup in the lab with a size 48 (suitable for 158~170cm tall cyclist) steel tube road bicycle, which was reported as high crash risk type (Rodgers, 1997), Tacx™ Fortius Multiplayer T1930 Virtual Reality Cycle Trainer and Tacx™ Trainer software version 2.0 (T1990.02). Epson EB-410We short throw projector was used to project a 4:3, 230cm diagonal length, screen. The screen was positioned 170cm in front of the participant’s eyes. Garmin® Edge705 OBRAD were employed to display information with the size 36pt Display font; the display of the device was 40cm to the participant’s eyes. 5 Chang Gung university students, 3 male and 2 female, age between 18 and 22 years old, have more than 10 years bicycle riding experience, were selected to participate this experiment. Before the experiment took place, the bicycle were adjusted to the participant’s comfort i.e. the seat post were adjusted by the participant’s inseam length. To reduce unnecessary mental load, the gears were fixed in 50 (chainwheel) by 14(freewheel) and the break was deactivated. Human response on surrounding stimulus is discussed in a three-steps process: stimulus identification, response selection and response programming. Stimulus is detected through sensory organs e.g. eye, ear or skin and recognized through comparison of previous memory. Second, the type of physical reaction is selected and then human body will generate a series of muscle movement to perform the reaction. The duration of this mental processing and physical movement is counted as reaction time (Sagberg & Bjørnskau, 2006; Schmidt & Lee, 2005). Response time is the duration from a human receiving a stimulating signal to the first response (Sagberg & Bjørnskau, 2006). To test the response time, we selected obvious stimulus such as oncoming and overtaking cars to avoid unnecessary association. A button on the control bar for thumb clicking was designed to minimize individual physical difference in performing response. A red dot signal was transmitted to the screen and recorded with the main screen where response time were counted as the duration between the first appearance of hazard to the red dot signal showed up in the screen.

Using on-Bicycle Rider Assistant Device While Cycling

87

Fig. 1. The experiment appartus

Fig. 2. Examples of hazards in the video (up-left: oncoming car in white; up-right: oncoming car in the shade; lower-left: car overtaking; bottom-right: oncoming car in dark color)

The participants were asked to ride a 5 minutes section of rural area. The resistances controlled by the cycle trainer changed according the road conditions, in which the range of slope was controlled within ±3%, to prompt the participant monitoring the device and adjust their RPM. A practice section was first commenced for participants to warm up and get used to control their RPM assisted by the device. In task 1 (controlled group), participants clicked the button once they percept a developing hazard that could endanger cycling e.g. oncoming car or car overtaking (as shown in Fig 2) (Sagberg & Bjørnskau, 2006). There were 7 situations that could

88

C.-Y. Yang and C.-T. Wu

possibly develop into hazards occurred in the section and the duration of the critical intervals varied between situations. After 15 minutes break, the participants commenced task 2 in which the participants were additionally asked to perform a secondary task: maintaining leg cadence in 50±5 revolutions per minute (RPM). The times and duration that the cadence was not maintained within the tolerance were counted.

3 Results As shown in Table 1, without using the device, participants generally reacted the hazards within 0.6~1 seconds. However, participant’s performance was decreased in respects of speed control and hazard perception when they used the device. The average response time (standardised for each situation and averaged across situations) tended to be slightly longer in the experiment group (task 2). In the 5 minutes ride the participants averagely 0.33 seconds more in responding the hazards when use the device than without using it. They mis-controlled the RPM range for 31.4 seconds in which most of them associated with hazard situation. For example, the participants A failed to maintain their RPM 6 times in which 4 of them happened during the critical of hazards. Additionally, a false alarm happened on the time spot 04:13, the participant mistaken a white guardrail as a developing hazard. Table 1. The response time to the hazards situations (RT: response time)

Situation 1. In the town, a van appeared from shade 2. Outside the town, a white car overtook from left side. 3. On the straight road, a black van appeared from the far end of the road. 4. In a sharp right turn, a white car appeared. 5. In a sharp left turn, a white car appeared. 6. In a sharp right turn, a black car appeared from left and was willing to join the main road. 7. On the straight road, a white car appeared from the far end.

Time spot

Task 1 RT (sec)

Task 2 RT (sec)

Hazard missed

RPM missed

00:21

0.6

1.6

0

3

00:31

1

1.4

0

3

01:20

0.6

2.1

0

4

01:39

0.6

0

1

1

01:53

0.8

1.2

0

0

02:12

0.8

0.4

0

0

02:31

1

1

0

2

Using on-Bicycle Rider Assistant Device While Cycling

89

4 Discussion The results have suggested that using the information device while riding bicycle has increased the cyclist’s mental load and affect cycling performance. The complete set of the device should be designed in consideration of the possible risks it brings up. For example, the idea that having auditory stimulation could enhance human reaction performance e.g. speed and accuracy may be implemented in assisting cyclist to percept hazards around. It is possible that the relationship between individual experience and response time differs between situations (Sagberg & Bjørnskau, 2006). The participants were sensitive on the situation that hazards appear around them but far end of the road. Riding distance, reported as an important crash factor (Rodgers, 1997), will be considered in the future research. The results have implied the future research to look at the issues of the interface design, observation behavior, body posture, possible neck tiredness caused by repetitive glancing the device and different cycling conditions such as physical tired or uphill road. Acknowledgements. This research was supported by grants (NSC 99-2221-E-182049) from the National Science Council of Taiwan, and we thank them for financially supporting this research.

References Brown, I.D., Tickner, A.H., Simmonds, D.C.V.: Interference between concurrent tasks of driving and telephoning. Journal of Applied Psychology 53(5), 419–424 (1969) McKenna, F.P., Crick, J.: Developments in hazard perception. Transport Research Laboratory, Crowthorne (1997) Rodgers, G.B.: Factors associated with the crash risk of adult bicyclists. Journal of Safety Research 28(4), 233–241 (1997), doi:10.1016/S0022-4375(97)00009-1 Sagberg, F.: Accident risk of car drivers during mobile telephone use. Internation journal Vehicle Design 26(1), 57–69 (2001) Sagberg, F., Bjørnskau, T.: Hazard perception and driving experience among novice drivers. Accident Analysis & Prevention 38(2), 407–414 (2006), doi:10.1016/j.aap.2005.10.014 Schmidt, R., Lee, T.: Motor control and learning: a behavioral emphasis, 2nd edn. Human Kinetics, Champaign (2005)

Part II

Virtual and Augmented Environments

Human–Robot Collaboration with Augmented Reality Siam Charoenseang and Tarinee Tonggoed Institute of Field Robotics, King Mongkut's University of Technology Thonburi, Thailand [email protected], [email protected]

Abstract. This paper presents an implementation of human-robot collaboration by using augmented reality technique for providing necessary information to human operator. In this research, human operator and a robot arm share the same workspace in virtual object assembly task. The virtual objects are created in the form of 3D computer graphics that is superimposed on the real video image. While working in assembly task, the robot will assist the human operator by loading the virtual objects. Furthermore, a task planner controls all robots’ operations accordingly to human actions. Using augmented reality, human operator will receive robot’s task plan in the form of computer graphics during assembly task. The computer-generated information will support human operator’s decision for a suitable next step action. Keywords: Human - Robot Collaboration, Augmented Reality, Task Planning.

1 Introduction During recent years, augmented reality technology, which presents computer graphics information superimposed on real time video image, gains more interests. Previous research works implemented augmented reality to provide necessary information in the form of computer graphics to the operator during working with real objects [1],[2],[3]. Augmented reality is also used in human-robot collaboration task. Human operator will receive the augmented task information while working with robot [4],[5]. However, human and robot do not share the same workspace in previous related research works. Hence, this paper presents the development of human-robot collaboration with augmented reality system in the common workspace. The task presented in the research is a virtual object assembly. During human- robot operation, human will obtain virtual objects, graphics instruction, and robot’s action information through augmented reality technique. Moreover, robot will receive control commands from human operator to assist in transferring virtual objects.

2 System Overview This research allows the collaboration between human and robot in virtual assembly task. The virtual objects which are used in this task are in the form of 3D computer C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 93–97, 2011. © Springer-Verlag Berlin Heidelberg 2011

94

S. Charoenseang and T. Tonggoed

graphics and rendered on the detected marker plates. During assembly process, the operator will receive all graphics information which is 2D guidance texts, 3D symbols, and 3D virtual objects through LCD display. In addition, the operator can allow or decline the assistance from robot in transferring marker on which virtual objects are superimposed. 2.1 System Configuration Figure 1 shows the system configuration developed in this research. It consists of an operator, 5 DOF robot arm, computer, LCD screen display, video camera, and maker plates. During virtual object assembly, the operator and robot will move virtual objects on marker plates as system guidance. USB video camera is used to capture image of the workspace. The computer is responsible for image processing, task planning management, rendering all graphics on LCD screen, and controlling robot operation. The robot performs as an operator’s assistance. Its operations depend on task plan and operator’s decision commands.

Fig. 1. System Configuration

2.2 System Data Flow From data flow diagram in Figure 2, this system consists of 4 main components which are vision manager, graphics manager, task manager, and robot manager. First, ARToolKit [6] software library is used in the vision manager to process captured video images and obtain positions and orientations of targeted marker plates. Then, the vision manager sends targeted objects’ positions and orientations to the task manager. The task manager is responsible for generating action plan using STRIPS planning algorithm in assembly task. Next, the robot manager computes forward/inverse kinematics, reads robot’s joint angles, and sends commands to control the robot’s movement. The last component is the graphics manager which generates 2D and 3D computer graphics to present all guidance information and virtual objects on LCD display to the operator.

Human–Robot Collaboration with Augmented Reality

95

Fig. 2. System Data Flow Diagram

2.3 System Operation When system starts, it presents graphics instructions so that the operator can select the assembly model as shown in Figure 3(a). After selecting the assembly model, several virtual objects are rendered and superimposed on the markers. The system then generates a task plan of selected assembly model. That plan is interpreted to manage system states, present graphics information, and control the robot. The 2D graphics shows suggestion texts and information related to robot action while 3D graphics presents assembly guidance arrows such as suitable positions and orientations of virtual objects over the real video image on the LCD screen as shown in Figure 3(b) and 3(c). During the operation, robot will assist the operator to transfer targeted object after “Yes” virtual object is selected as shown in Figure 3(d). The operator can decline any robot action by selecting “No” virtual object. If the operator does not follow the suggested task plan, the system will generate a new task plan for that assembly task automatically.

(a) Model Selection

(b) Rotation Guide

(c) Robot Selection

(d) Robot Operation

Fig. 3. System Operation

3 Experimental Results There are 3 experimental sets to evaluate the proposed system in the areas of system performance, usability, and values for specific task.

96

S. Charoenseang and T. Tonggoed

3.1 System Performance Robot’s repeatability is one of the system performances. At least 6 targeted positions in the robot workspace are tested for obtaining the robot’s repeatability in both x and y axes. This experiment results indicate that the repeatability errors in x and y axes are 0.52 and 0.35 millimeters, respectively. Next, the graphics update rate is about 17 fps with the image resolution of 640 x 480 pixels. This system has ability of tracking the targeted markers which are moved with velocity less than 10 centimeters/second. 3.2 Usability After using the proposed system to do virtual assembly task, questionnaire is used to collect user’s satisfaction on the system’s usability. The results indicate that most of users are satisfied with the system’s usability. Using graphic arrow helps the users to understand the virtual object’s direction more easily. Hence, the usability of guidance arrows indicating the virtual object’s orientation got the highest satisfaction score with 92 percents. However, most users did not pay much attention at the suggestion text during operation. As the result, the usability of suggestion texts got the lowest satisfaction score with 74 percents. 3.3 Values for Specific Task This proposed system is applied in a virtual assembly task. In this task, the users were asked to assemble the virtual object as a given picture on the instruction sheet. This assembly task is set in three conditions as assembly with/without any system's information and assembly with system's information along with robot's assistance. The goal of this experiment is to reduce the operation’s time when the users work with this system’s assistances. Since the 3D virtual objects are mostly complicated, this task is difficult for the user who does not have much skill in 3D graphics assembly. Therefore, assembly without any system’s information took the most averaged assembly time at 182 seconds. With graphics guidance information, it can reduce averaged assembly time into 64 seconds or 64.84 percents of assembly time obtained from system without graphics guidance information. The averaged assembly time with system’s information and the robot is 144 seconds or 20.88 percents of assembly time obtained from system without graphics guidance information. The speed of this operation depends on the robot’s speed. After the experiments, the questionnaire is used to collect the user’s satisfaction. The results show that most of the users are pleased with this proposed system for the virtual assembly task. Because of using guidance graphics overlaid on real time video during assembly task can reduce the operating time. Therefore, the ability of system’s suggestion assisting user to assembly faster got the highest satisfaction score with 87 percents. Since the robot must work with human operator, it is set to operate at the low speed. Some users are not satisfied with this operation speed so the satisfaction of robot’s assistance got the lowest score with 81.33 percents.

Human–Robot Collaboration with Augmented Reality

97

4 Conclusions and Future Works The human- robot collaboration with augmented reality for virtual assembly task was proposed. The system’s assistance using augmented reality and the robot can reduce the operation times of the virtual assembly task. Furthermore, the use of augmented reality in the assembly task can save cost and training time. The virtual objects in assembly task can be changed into various object’s models by importing from the CAD applications. Moreover, the graphics guidance can help the user to understand the assembly task more easily so the assembly time is reduced. Based on the experimental results, there are many issues that can be developed to improve the system performance and usability. Since the operator does not pay much attention to read graphic texts during operation, the augmented guidance information should be graphic symbols instead of texts. The assembly time of operating with the robot can be reduced by increasing the robot’s speed. However, the change of robot’s speed should be considered along with human’s safety. For more natural communication, the interaction between robot and human can be improved by using gesture or speech recognition. Acknowledgments. This research work is financially supported by the National Science and Technology Development Agency, Thailand.

References 1. Triebfürst, W.W.G.: ARVIKA: Augmented Reality for development, production and service. In: Proceedings of the 1st International Symposium on Mixed and Augmented Reality. IEEE Computer Society, Washington, DC, USA (2002) 2. Pathomaree, N., Charoenseang, S.: Augmented reality for skill transfer in assembly task. In: IEEE International Workshop on Robot and Human Interactive Communication ROMAN 2005, pp. 500–504 (2005) 3. Yuan, M.L., Ong, S.K., Nee, A.Y.C.: Assembly Guidance in Augmented Reality Environments Using a Virtual Interactive Tool. Singapore-MIT Alliance Symposium, Singapore (2005) 4. Otmane, S., Mallem, M., Kheddar, A., Chavand, F.: Active Virtual Guides as an Apparatus for Augmented Reality Based Telemanipulation System on the Internet. In: Proceedings of 33rd Annual Simulation Symposium (SS 2000), Washington, DC, USA, pp. 185–191 (2000) 5. Green, S.A., Chase, J.G., XiaoQi, C., Billinghurst, M.: Evaluating the Augmented Reality Human-Robot Collaboration System. In: 15th International Conference on Mechatronics and Machine Vision in Practice M2VIP 2008, pp. 521–526 (2008) 6. Human Interface Technology Laboratory, ARToolKit, http://www.hitl.washington.edu/artoolkit/documentation

Making Pixel Patterns Automatically for Camouflage – Using Color Information from Their Background Woon Jung Cho1, Wonmi Ahn1, Myung Shik Kim1, Jeeyea Park1, Seungduk Kim2, and Kwang-Hee Han1 1

Cognitive Engineering Lab. Department of Psychology, Yonsei University, Japan 2 Department of Computer Science, Yonsei University, Japan [email protected], [email protected]

Abstract. Camouflage is an attempt to obscure a target’s outline and match its background for hiding the target. Detectability of a target depends on the features of the background and changes in the surrounding. Thus, we have to consider visual information from the background and apply them to the development of an effective camouflage pattern. As dynamic environments are essential for designing an adaptive pattern, controlling digital images and making patterns automatically by use of computer can be efficient alternatives through the designing process of development and evaluation. In this study, we made the automatic pixel patterns program for the camouflage assessment. For making patterns, color information derived from background was used and pixel-dot type of pattern was considered in view of the digital pattern. Automatic stimuli-pattern maker developed in this study follows several steps; load an image, select an area, extract colors, and create a pattern. This automatic implementation has some benefits for design processes for camouflage pattern development. Further studies are needed not only for program-upgrade but also for improvement of color strategy through analyzing camouflage features. Keywords: camouflage pattern, camouflage assessment, photosimulation, adaptive pattern.

1 Introduction Recently, we heard of the breaking news that British a Defense Company is developing the invisible tank. Using attached electric sensors on the surface of the tank it could process color, line and shape from surroundings, and then project them on a tanks’ exterior. This invisible tank technology, so-called ‘e-camouflage system’ aims to merge target and background. In the context of military, these demands for finding the adaptive camouflage strategy in dynamic environment conditions will grow as battle conditions vary. Detectability of a target depends on the features of the background and changes in the surrounding. Thus, we have to consider visual features like colors or patterns from C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 98–101, 2011. © Springer-Verlag Berlin Heidelberg 2011

Making Pixel Patterns Automatically

99

environments and apply them to the design process for an effective camouflage. In addition, empirical detectability data of developed patterns on dynamic surrounding conditions are required for effective camouflage pattern rather than designers’ subjective point of view. In this study, we developed the automatic pixel pattern program for photosimulated camouflage assessment aimed to investigate how visual features from background could affect a target’s ability of camouflage.

2 Camouflage Pattern Design and Assessment 2.1 Camouflage Pattern Design Camouflage is an attempt to obscure the target’s signature and match its background[1,2,6] in order to hide or disguise the identity of the target. In the same context, the purpose of camouflage pattern is to minimize the noticeability and maintain the visibility of pattern at the same time[2,5]. To satisfying the two contradicting purposes for camouflage, features of camouflage pattern have to be similar with the visual features of the targeted background as much as possible. Some features like color, shape, and size among others can be critical factors for designing patterns. Designers have to deal with these factors sufficiently and increase similarity with the targeted region for effective camouflage design. So far, most camouflage patterns were generated by the subjective view of expertdesigners who could deal carefully with characteristics of background conditions like color and pattern, and so on. But now, these traditional approaches can incur relatively high cost and time to create adaptive camouflage patterns as the military contexts are changed[3,5]. If dynamic environments are essential for designing adaptive patterns like the ‘e-camouflage’ system, controlling digital images and making patterns automatically by computer can be efficient alternatives through the design process of development and evaluation. 2.2 Camouflage Assessment by Photosimulation Evaluation of the camouflage effectiveness is also a fundamental step for optimal camouflage pattern. There are two camouflage assessment methods; one is field observation, the other is photosimulation[4]. The former performs the camouflage assessment on natural field conditions. While this field observation is able to obtain most practical evaluation data, it has some potential risk factors disturbing detection performance of camouflaged targets. The latter, in contrast, photosimulation performs on the controlled experimental situation, so it is made possible to produce rich statistical data from different groups of observers with the same image sets. Moreover, it has some strengths in designing patterns in respect that we are able to collect images continuously under the identical conditions and spare the additional expense [4]. Photosimulation is used to design or evaluate the developed pattern for dynamic and various military contexts because it is implemented on digital environments [3,4]. Actually, it is usual to apply color information as quantitatively computable data to experimental settings. These

100

W.J. Cho et al.

simulated results are applicable to real military environments, but potential influence factors in natural situations are not considered. In this study, we designed the visual target search experiments applying photosimulation and developed the automatic program for making pattern-stimuli under a dynamic background for the purpose of comparing camouflage effectiveness of patterns easier.

3 Automatic Program for Making Camouflage Pattern Stimuli Making patterns automatically from digital images is beneficial to camouflage assessment experiments using photo simulation. The reason is that generation of patterns is easier for various backgrounds and quantitative descriptions of background features. 3.1 Overall Flow of Automatic Program The automatic pattern-stimuli maker developed in this study follows several steps for generating patterns as the outcome. (1) At first, load a photo used as a background image and (2) assign target area. After target area is selected, (3) this program extracts colors based on color information of the selected area, and (4) lastly creates a pixel pattern using extracted colors which are based on color features of the background.

Fig. 1. Overall flow of developed program

3.2 Use of Color Information and Generation of Pattern Stimuli If the colors in the camouflage pattern are well extracted from the surroundings, they are used to allow the target to blend into the background and reduce any visual cues[2,3,5]. Several countries investigated color characteristics of different environments and applied different color schemes to development of patterns for dynamic surroundings[2]. In many ways, the use of color information derived from

Making Pixel Patterns Automatically

101

background images plays an important role in detectability of camouflaged targets. Further, it is critical to extract the limited numbers of appropriate colors. In this program, we used color information as a best feature for effective camouflage and pixel-dot type of pattern was considered in view of the digital pattern. This automatic pattern program derives selected numbers of colors based on color frequency information of the selected area and then replaces all pixels of the targeted area with extracted frequent-colors according to the similarity measure.

4 Discussions In this study, we developed the automatic pixel pattern program for the camouflage pattern stimuli, which is aimed to investigate how visual features from background could affect detectability of camouflaged targets through photo-simulated camouflage assessment. As a result of the developing and testing the automatic pattern stimuli program, it was determined that strategy used for color processing was less effective for considering color characteristics of a background in some conditions. Nevertheless, this implementation deserves to be considered continuously through the design process for camouflage pattern development. For better effectiveness, further studies are needed not only to upgrade the program but also to improve color strategy through analysis of camouflage features. Acknowledgement. This work has been supported by the Low Observable Technology Research Center program of Defense Acquisition Administration and Agency for Defense Development.

References 1. Copeland, A., Trivedi, M.: Models and metrics for signature strength evaluation of camouflaged targets. Algorithms for synthetic aperture radar imagery IV, 194–199 (1997) 2. Friskovec, M., Gabrijelcic, H., Simoncic, B.: Design and Evaluation of a Camouflage pattern for the Slovenian Urban Environment. Journal of Imaging Science and Technology 54(2), 020507-(11) (2010) 3. Mitchell, K., Staples, C.: Methods for deriving optimum colours for camouflage patterns. In: RTO SCI Workshop on Search and Target Acquisition in Netherlands (1999) 4. Peak, J.: Guidelines for Camouflage Assessment Using Observers. Nato Research & Technology Organization Neuilly-Sur-Seine RTO-AG-SCI-095 (2006) 5. Ramsley, A.: Camouflage patterns-effects of size and color, technical report released on DTIC online (1976) 6. Riley, T.: Testing visual Camouflage, innovations report, National centre for computer animation in Bournemouthh University (2004)

Virtual Bridge: AR-Based Mobile Interaction for Easy Multimedia Control of Remote Home Devices DongJin Eun, Taik Heon Rhee, Seonghoon Kang, Minsuk Choi, Sangil Lee, and Hark-Joon Kim Samsung Electronics Co., Ltd. 416 Maetan3, Yeongtong, Suwon, Gyeonggi 443-742, Republic of Korea {dj.eun,taikheon.rhee,shoon.kang,minsuk1.choi, lignas.lee,harkjoon.kim}@samsung.com

Abstract. This paper proposes an interaction for controlling multimedia contents of remote devices using a mobile device based on AR technology. Using a real-time object recognition method, home devices detected by the camera of a mobile device are displayed on the camera preview screen along with the thumbnails of their own multimedia contents around the recognized positions. A user may drag a multimedia content which he or she wants to play, and drop it onto another target home device which he or she wants to play the content through. The user study showed that the proposed interaction expects higher usability since once a home device is registered with its device name when registering its image shown on the mobile camera for the object recognition, this matching process is no longer necessary when a user controls the device through the mobile device. Keywords: Mobile Interaction, Augmented Reality, Mobile AR, DLNA, Contents Sharing, Real-Time Object Recognition.

1 Introduction Recently, most of homes have several home devices holding multimedia contents, such as televisions, digital video recorders, photo frames or PCs, and the interaction issues between home devices have taken on great important. Although standardized technologies for transferring multimedia contents among home devices, including DLNA (Digital Living Network Alliance) [1], are spreading rapidly, the user interfaces to find other home devices or multimedia contents on them are still complex so that users not only need many steps to control the home devices. Moreover, users feel a lot of trouble in matching a device itself with its name or identifier displayed on the other device which is to connect with it. Most of the commercial DLNA solutions use a list of home devices with text and icon, when a user needs to connect a server device with a display device. Figure 1 shows the example screenshot of selecting remote devices in commercial solutions, Samsung AllShare [2] and Braintransfer DLNA remote [3]. As shown in Figure 1, a user should always memorize the correct name of the target device whenever he or she wants to control remote devices. In addition, such a series of selection steps is C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 102–106, 2011. © Springer-Verlag Berlin Heidelberg 2011

Virtual Bridge: AR-Based Mobile Interaction for Easy Multimedia Control

103

Fig. 1. Example screenshots of selecting remote devices in commercial solutions

required in the commercial solutions and it makes the user interface look complex and difficult to use. This paper proposes an interaction for controlling multimedia contents of remote devices using a mobile device based on AR technology. When a user focuses on one or more home devices through the camera of a mobile device, they are detected by a real-time object recognition method on the camera preview screen of the mobile device. The position of each recognized device is displayed on the touch-enabled camera preview screen, along with the thumbnails of its own multimedia contents around the recognized position. Then the user drags the thumbnail of a multimedia content which he or she wants to play, and drops it onto the recognized position of another target device which he or she wants to play the content through.

2 Virtual Bridge: Interaction Design Since Tani et al. [4] proposed the concept of using live video images to make users interact with a real-world object located at a distance, a number of studies have been utilized the metaphor in various usage scenarios. Boring et al. [5] mentioned a number of related works which adopted the “interaction through video” metaphor, while they proposed Touch Projector, a mobile interaction through video. In their work, the Touch Projector allows users to manipulate content shown on displays at a distance by touching the image of the content taken by a mobile camera. Our proposed interaction called the Virtual Bridge focuses on virtually bridging two real-world home devices by a mobile controller. In other words, our interaction connects a Digital Media Server (DMS), which stores content, with a Digital Media Renderer (DMR), which play content from a DMS, by the touch interaction on live video images taken by the camera of a mobile Digital Media Controller (DMC), which finds the content on DMS and plays it on DMR. The definitions of DMS, DMR and DMC are introduced on the DLNA specification [1]. The name of the “Virtual Bridge” reflects the concept that a user’s bridging interaction of two devices in the real world through the virtual space on a live video image. Figure 2 shows the walkthrough of the concept of the Virtual Bridge. Note that the mobile controller (DMC) itself may take on a role of content server (DMS) and directly connect its content to the remote device (DMR) for rendering.

104

D. Eun et al.

Fig. 2. User scenario flow. (a) A user focuses on one or more home devices through the camera of a mobile device. (b) The position of each recognized device is displayed on the touchenabled camera preview screen, along with the thumbnails of its own multimedia contents around the recognized position. (c) The user drags the thumbnail and drops it onto the position of another target device. (d) The content corresponding to the thumbnail is played through the target device.

The Virtual Bridge interaction needs pre-registration procedure of the home devices as shown in Figure 3. Once a home device is registered with its device name when registering its image shown on the mobile camera for the object recognition,

Fig. 3. Pre-registration procedures. (a) A user takes a snapshot of the target device to register and extract features of the image. (b) The snapshot with the extract features is matched with the device name for the real-time object recognition afterwards.

Virtual Bridge: AR-Based Mobile Interaction for Easy Multimedia Control

105

this matching process is no longer necessary when a user controls the device through the mobile device.

3 Implementation and Evaluation To compare usability of the proposed interaction with Samsung AllShare [2], a conventional approach applied in commercial products, we implemented a prototype of Virtual Bridge interaction controller. The controller (DMC) prototype was implemented as an android application on a Samsung Galaxy Tab (SHW-M180S). A variant of SIFT algorithm [6] has been implemented for the real-time recognition of home devices on live video images. The DLNA functions in Windows Media Player 7 application on a notebook PC and Samsung Galaxy-S, a smartphone with Samsung AllShare were used for the content server (DMS). Also, a television supporting Samsung AllShare was used for the content renderer (DMR). All the devices were assumed to be connected to a Wi-Fi network in advance. Figure 4 shows examples of the implemented prototype in use.

Fig. 4. Prototype implementation of the Virtual Bridge. (a) Searching content through a server (smartphone). (b) Moving content to a renderer (TV).

To measure user-satisfaction with the proposed interaction compared with the commercial solution, a questionnaire based on the IBM computer usability satisfaction questionnaire [7] was given to 10 participants, who were aged between 25 and 36 with a mean of 29.7 years. The questionnaire consisted of 7 items in total, using a 5-point Likert scale from strongly agree (5) to strongly disagree (1). Figure 5 presents a summary of the results. In general, the proposed interaction gained better results than the conventional approach. Especially, the participants felt that it was more enjoyable, more intuitive, and easier to learn than the conventional approach. From the questionnaire results with comments of the participants, we may infer that the results are due to the direct matching between the device name listed in a controller and its image shown in the live video image.

106

D. Eun et al.

Fig. 5. Average user feedback regarding the interaction techniques

4 Conclusion and Future Work This paper proposes an interaction for controlling multimedia contents of remote devices using a mobile device based on AR technology. While the conventional approach always requires matching a remote device and its name, the proposed interaction makes a user connect content from a server to a render intuitively on the camera preview screen of a mobile controller. The proposed interaction relies on performance of the real-time object recognition algorithm to detect home devices on live video images. The currently implemented algorithm only detects a home device when its image is taken in the similar direction with the one taken at the registration time. In the future, we plan to overcome such a limitation by improving the recognition algorithm robust on direction variations, as well as introducing proximity sensors or indoor position determination technologies. Meanwhile, we also plan to find various user scenarios beyond the multimedia control, based on interaction among home devices.

References 1. Digital Living Network Alliance, http://www.dlna.org 2. SAMSUNG AllShare, http://www.samsung.com/au/tv/internet-tv.html 3. DLNA Remote, Braintransfer Inc., http://braintransfer.de/en/services/mobilesolutions/ dlnaremote 4. Tani, M., Yamaashi, K., Tanikoshi, K., Futakawa, M., Tanifuji, S.: Object-oriented video: interaction with real-world objects through live video. In: Proc. CHI 1992, Monterey, CA, USA, pp. 2287–2295 (1992) 5. Boring, S., Baur, D., Butz, A., Gustafson, S., Baudisch, P.: Touch Projector: Mobile Interaction through Video. In: Proc. CHI 2010, Atlanta, GA, USA, pp. 2287–2295 (2010) 6. Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. Int. J. Computer Vision 60(2), 91–110 (2004) 7. Lewis, J.R.: IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use. Int. J. Hum.-Comput. Interact. 7(2), 57–78 (1995)

Design and Implementation of a Low-Cost Projected Virtual Reality System to Support Learning Processes Rodrigo Gómez and Helmuth Trefftz Virtual Reality Lab , EAFIT University Medellín, Colombia [email protected]

Abstract. Virtual Reality technologies have been successfully incorporated into the learning processes and potential new applications in education are explored continuously. We found that one of the difficulties to popularize its use in the educational context, in countries with emerging and developing economies, is the cost of hardware required to generate satisfactory immersive experience. In this work we considered virtual reality from the perspective of human-computer interaction to support learning processes. The characteristics of low-cost projected virtual reality system (PVR) proposed, combines study and integration of available technology solutions, the development of an image synchronization routine that enables the use of a single video projector and the design of a printable pattern that preserve the state of polarization on the projection screen. Keywords: Virtual Reality, projection surface, polarization.

1 Introduction The Virtual Reality (VR) is a strong tool for the learning support process, including tangible and abstract concepts, increase critical performance in activities such as surgical procedures and technical transfer skills [1,2]. Immersion in the VR experience is determined as a key element in the cognitive variables associated with the processes of learning [3]. The technology used in VR is the main limitation for immersive experience. The computer graphics is the spearhead for generation of three-dimensional digital worlds and so far is limited by the processing power, not by lack of algorithms [4]. Something alike occurs with visualization hardware, even though today we have massive HD stereoscopic screens, the first 3D optical perception still requires the use of passive and active goggles. The autosteroscopic display are still far away from popularized as a tool for daily technology VR that support learning processes [5]. In the case of most emerging & developing countries, the costs associated with IT equipment for virtual reality, and their rapid obsolescence, have made quite difficult the incorporation of VR as a tool to support the learning processes [6]. Notwithstanding the foregoing, the software development and 3D digital content have been identified as a potential factor to increase the economic competitiveness of emerging economies, where there was a latent need for hardware related to VR technology [7,8]. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 107–111, 2011. © Springer-Verlag Berlin Heidelberg 2011

108

R. Gómez and H. Trefftz

This paper considers the critical points for the implementation of a virtual reality system design that includes: a single video projector, a polarize rotary disk filter like a beam shutter type, a printable pattern that preserves the state of polarization on the projection screen.

2 Related Work The PVR system was treated as a set of devices and software which combined allow stereo viewing digital content and the interaction with them [9]. We identified three critical subsystems for the operation of a PVR system Fig.1(a), the display, data transmission and interaction subsystem.

Fig. 1. PVR system scheme based on three subsystems (a). Schematic display subsystem (b).

2.1 PVR Subsystems Display subsystem, Responsible for viewing digital content and devices for the whole image makers (video projector),image filtering (polarizers), alignment, and optical image formation (projection surface). As shown in Fig. 1(b), unlike traditional configurations, requiring the use of two video projectors [10] required for generating the stereoscopic selective filtering of each image perspective to the user corresponding eye. Other works [11] have shown how the selective control of each image at a frequency of 30 Hz for each eye has been sufficient to generate an acceptable stereoscopic. Guided by [11] and considering new technologies for the selection of images [12] PVR system was designed with a single video installation projector and a rotary disk-shaped polarizer. In order to obtain the perspective perception in the passive PVR, conventionally projected two images with different points view simultaneously [10], so the display surface (Projection screen), must retain the polarization of each image. Projection surfaces are called "Silver" screen, in the commercial use, and have higher associated costs against the standard projection surfaces. Other studies have put forward proposals alternatives to preserve the polarization with good results performance [13]. Send data subsystem, includes hardware and software necessary to receive and send information subsystems according to the characteristics and requirements

Design and Implementation of a Low-Cost Projected Virtual Reality System

109

thereof. At this time, the stereoscopic driver that have better performance uses GPU data processing [14]. It supports the majority of branded graphics cards, and is designed to work with most 3D hardware solutions.

Fig. 2. Schematic send data Subsystem

Subsystem interaction is defined as the set of peripherals which is effected by interaction with the digital scene. In the market offers a wide range of devices that meet this function, standing between haptic and kinetic devices [15]. The Wii controller has been tested as a successful tool human-computer interaction technologies related to VR. A wide range of applications and libraries available in different programming languages under the GNU (General Public Licences); on the other hand, it was considered a low cost hardware (40 USD per unit) [16].

3 Experimental Procedures and Results The display subsystem design was set with reference to the optical axis of projection, the polarized filter was placed into the lens front of the video projector (Figure 3), in order that the light that comes from the video projector, is polarized when goes through the disk.

Fig. 3. Rotary polarizer for display subsystem: (a) Schematic representation, (b) montage implemented

For the rotation synchronization of the polarizer with the sequence of projected images, we developed a routine called "digital image synchronization stereoscopic (DSIS)" (figure 4), synchronization sequence image. DSIS routine maintains the cross-polarization between the projected images in sequence and compensates the natural delay generated by the rotation of the motor at a constant revolution 60 Hz.

110

R. Gómez and H. Trefftz

Fig. 4. Timing diagram of the DSIS routine

In order to maintain the standard of usability in the system application proposed, a mono images output application from BINO software was implemented on the VGA output signal to DSIS input routine, establishing itself as the reference signal for synchronizing video and disk rotation frequency polarizer, thus has an open loop to maintain state sequential polarization between the projected images. For the projection surface, we designed a printable pattern that successfully retained polarization stage (Figure 5); cost implementation was minimal compared to the Silver screen available on market.

Fig. 5. Gray scale for printables patrons. The projection surface had better perfomace whit printable patron number 6.

4 Conclusions and Future Work One of the main differential PVR system designed over conventional systems was the use of a single video beam. Which avoided the need for optical axis alignment of the images, it requires additional study to compare the level of immersion of this design compared to conventional. The timing sequence of image for the cross polarization, could consider an alternative technique for stereoscopic display and its performance depends on the delta of change in the state of images polarization. The designed printable patron can be used as a projection surface PVR system with similar benefits to the “silver” screen. However it requires a holographic quality printer resolution in order to optimize its functionality.

References 1. Ausburn, L.J., Ausburn, F.B.: Effects of desktop virtual reality on learner performance and confidence. Opening a line of inquiry, Journal of Industrial Teacher Education 45(1), 54– 87 (2008)

Design and Implementation of a Low-Cost Projected Virtual Reality System

111

2. Lapointe, J.-F., Robert, J.-M.: Using VR for Efficient Training of Forestry Machine Operators. Education and Information Technologies 5(4), 237–250 (2000) 3. Psotka, J.: Immersive training systems: Virtual reality and education and training. Instructional Science 23(5-6), 405–443 (1995) 4. Goodnight, N., Wang, R., Humphreys, G.: Computation on Programmable Graphics Hardware. IEEE Computer Graphics and Applications, 12–15 (2005) 5. Javidi, B.: Three-Dimensional Television, Video and Display Technology. Springer, New York (2002) 6. Pickles, J.: The social implications of geographic information systems. Guilford Press, New York (1995) 7. Commission for Communications Regulation COLOMBIA government.: Analysis of the ICT sector in Colombia: Evolution and Challenges. Document Analysis and Relation of Infrastructure Center Industry Knowledge (2010) 8. Dutta, S., Mia, I.: The Global Information Technology Report 2008–2009 Mobility in a Networked World. World Economic Forum Editors (2009) 9. Churchill, E.F., Snowdon, D.N.: Collaborative Virtual Environments. In: Munro (ed.) Springer, Heidelberg (2001) 10. Gutiérrez, M., Vexo, F., Thalmann, D.: Stepping into Virtual Reality. Springer, Heidelberg (2008) 11. Lee, K.-H., Jang, T.-J.: A study on a stereoscopic display system using a rotary disk type beam shutter. In: Smith, M.J., Salvendy, G. (eds.) HCII 2007, Part I. LNCS, vol. 4557, pp. 868–875. Springer, Heidelberg (2007) 12. Masat, M., Minami, A.: High-speed polarizing device and high-speed birefringence measuring apparatus and stereoscopic image display apparatus utilizing the polarizing device. US Patent No 7,742,169 B2. (June 22, 2010) 13. Zelle, J.M., Figura, C.: Simple, low-cost stereographics: VR for everyone. In: SIGCSE Bul., vol. 36, pp. 348–352. ACM, New York (2004) 14. Open inventor architecture group.: The Oficial Reference Document for Open Systems. Addison-Wesley, Reading, MA, USA (1994) 15. DeFanti, T.: VR: Past, present, and future. In: International Conference on Image Processing (ICIP 1998) vol. 3, p. 1 (1998) 16. Lee, J.C.: Hacking the Nintendo Wii Remote. IEEE Pervasive Computing, 39–45 (JulySeptember 2008)

Interface Design to Support Situation Awareness in Virtual Puppetry Keisha Harthoorn and Stephen Hughes Department of Computer Science University of Northern Iowa {harthook,sthughes}@uni.edu

Abstract. We propose virtual puppetry as a potential mechanism for enhancing students’ presence in a virtual learning environment. To develop this style of interaction will require substantial attention to the user interface in order to promote the operator’s situation awareness of both the real and virtual environments. This presentation describes the development of an initial prototype and some of the ongoing concerns for controlling virtual puppets. Keywords: Virtual Puppetry, Interface Design, Situation Awareness, Virtual Environments, Presence, Virtual Heritage.

1 Motivation Immersive theaters (Domes, Caves, etc) and other interactive large-screen displays are becoming increasingly popular installments in museums to allow patrons to experience “Virtual Heritage” exhibits. Likewise, portable versions of these exhibits are finding their way into classrooms. Experiencing ancient structures first-hand can give patrons or students a powerful familiarity with these environments that they would not otherwise have [1]. The central element of these displays is usually a threedimensional computer model of an ancient space. One of the fundamental concerns for many of these virtual learning environments is how to cultivate “presence”, the subjective feeling of being part of the virtual environment. This is based on the long held belief that the increased student presence will yield opportunities for deeper learning [2]. In hopes of increasing presence, considerable effort goes into modeling and rendering environments to make them visually accurate and believable; however, the illusion of presence can be easily disrupted by other external factors. We propose that, in addition to visual fidelity, the communication channel between teachers and students also plays a significant role in students’ presence. Many educational VR systems involve teachers and students collaboratively viewing a virtual model. In this scenario, all of the participants are on the outside of the virtual world, looking in. Therefore, any interaction between teacher and student occurs outside of the virtual environment and necessarily disrupts the sense of presence. This disruption can be avoided by moving all participants into the virtual space, allowing them to be represented in the environment through personal avatars, providing an interaction style is reminiscent of Second Life [3]. While this ensures that all interaction takes place in the virtual world, students must divert some of their C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 112–115, 2011. © Springer-Verlag Berlin Heidelberg 2011

Interface Design to Support Situation Awareness in Virtual Puppetry

113

attention to manipulation of their character. This may come at the expense of focusing on the material to be learned. An alternative model allows students to interact directly with a teacher represented virtually in the environment. While technology is readily available to automate virtual characters, their actions often appear transparently scripted, especially when the character needs to interact with a real person [4]. To allow robust interaction between the audience and virtual characters probably still demands human interpretation of audience questions, reactions and gestures. We therefore suggest that to students’ presence in educational VR systems could be enhanced by interacting directly with a virtual character that is controlled by an unseen human operator; a puppeteer. To develop this style of interaction will require substantial attention to the user interface for the human operator. People have the advantage of being able to tap into a wealth of deictic behaviors and gestures that are practically involuntary. Moreover, there are social constraints for interacting with the audience (making eye contact, for example) which are second nature for human presenters, but need to be directly activated by a puppeteer. Not only must the virtual character support various gestures and behaviors, but they must also be easily and effectively toggled by the operator. Situation awareness is the ability of an operator to perceive, comprehend and act in a highly dynamic environment [5]. The ability to establish and maintain situation awareness is heavily influenced by the design of the interface. In this application, the operator to be aware of events that are taking place in both the environment and in the audience and be able to respond to each appropriately. For this reason, it is critical for the interface to aggressively support the situation awareness of the puppeteer.

2 Design Principles and Prototype The development of our virtual puppeteering interface was driven by principles that are consistent with the interface design guidelines proposed by Endsley [5]. Specifically, we attempt to present a view of the overall environment, focus the viewer on aspects of the environment that are relevant, take into account both topdown and bottom-up processing and simplify interaction demands on cognitive load. 2.1 Multiple Views Our first principle recognizes that the framework must be capable of generating and displaying multiple distinct views to communicate information about both the virtual environment and audience. For our prototype, we worked with the Unity game engine [6] in order to generate the virtual puppet and environment. Three distinct views of the virtual information were designed: 1.

A first-person perspective of the environment allows the puppeteer to see the environment from the eyes of the puppet. 2. A third-person view shows the puppet in the context of the environment as seen by the audience. 3. A map view provides the context of the character within the environment from a bird’s eye view.

114

K. Harthoorn and S. Hughes

In addition to the views of the virtual environment, a live view of the audience can be maintained using a video feed from a webcam. 2.2 Focus+Peripheral Display Since the puppeteer will be required to monitor and interact with both the audience and the virtual environment, it is critical that the puppeteer is able to quickly assess the state of a particular view and understand what intervention is needed. The second principle that we attempted to design for recognizes that all information related to the operator’s goals should be available, but information relevant to the current task should be more prominently displayed. To achieve this, we designed a multi-screen Focus+Peripheral display (see Figure 1). This array of physical display devices consists of four standard-size monitors with larger display projected above them. The projected screen serves as the primary screen Fig. 1. Focus+Peripheral Display for interaction, and is known as the “Focus” screen. The four smaller screens hold peripheral information and allow the operator to monitor other views of the environment. When information contained on a peripheral display requires the attention of the operator, he/she must swap that peripheral display to the focus display. Note that the physical location of each peripheral view remains constant so that the operator can always find the desired view. In our initial development, we compared two techniques for bringing the display into focus: 1) Warp - A button press corresponding to the desired peripheral screen causes it to take focus; the location of the mouse pointer remains in the same relative screen location. 2) Point-to-Screen - The user gestures toward the peripheral screen that they wish to activate; the mouse pointer is repositioned to the relative entry/exit point of the focus screen. An informal comparison suggests that the point-to-screen seemed intuitive, but the warp technique was faster and more natural. 2.3 Puppet Capabilities: Marotte Interaction The term puppet covers a wide range of manipulative devices. Puppets vary from simple finger puppets up to complex marionettes, which are controlled by any number of bars, levers and strings. While it is compelling to have control of a fully articulated puppet, we sought to create a system that minimizes the control options. Granting complete, direct control of the virtual character’s posture and gestures would potentially overload the operator’s working memory at the expense of situation awareness. Moreover, this level of control may not be practical for instructors without an extensive knowledge of advanced puppetry. To this end, we attempted to distill the puppet interface down to its minimum. To capture the essence of an environmental

Interface Design to Support Situation Awareness in Virtual Puppetry

115

guide, we adopted the model of a Marotte, which is a modified rod puppet [7]. With a physical Marotte, the puppeteer is capable of manipulating the puppet’s head and body using a central rod. The puppeteer uses a free hand to act directly as the puppet’s hand. This allows the puppeteer to control the location and direction of gaze of the puppet, as well as perform basic hand gestures to point or manipulate small objects. In the virtual version of our puppet, we focused on head and hand gestures. To further simplify controls of the puppet, we limited the interaction with each of the views to a simple context-driven selection. The puppeteer controls the puppet exclusively by selecting an element of the display; the puppet responds with the appropriate head and/or hand actions given the context of the display. For example, when the audience view is active, the puppeteer may click on the face of an audience member; the puppet responds by acknowledging that individual with a hand gesture and making eye contact. Likewise, clicking on a location in the map view causes the puppet to navigate (move its body) to that location. As the prototype is evaluated, the capabilities of the puppet may need to be enhanced, but this basic operation forms a foundation for exploring puppet mediated instruction For example, the puppeteer can direct the puppet interact with the environment by clicking on various objects from the puppet’s 1st person perspective. Affordances for object types can be embedded in the environment model, still minimizing the demand on the puppeteer to issue complex commands.

3 Evaluation Pending ongoing evaluation and revision of this prototype, we intend to conduct a more a formal study on the effects of this design on the puppeteers situation awareness and puppet mediated instruction impacts student presence.

References 1. Jacobson, J.: Ancient Architecture in Virtual Reality; does Visual Immersion Really aid Learning? Dissertation, School of Information Sciences. University of Pittsburgh (2008) 2. Devine, K.: Place and Time: Creating Contextualized Presence for Virtual Heritage. In: Proc. 13th Intl Conference on Virtual Systems and Multimedia, pp. 65–72 (2007) 3. Second Life, http://secondlife.com/ 4. Mateas, M.: Expressive AI. Leonardo. Journal of the International Society for Arts 34(2), 147–153 (2001) 5. Endsley, M.: Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors: The Journal of the Human Factors and Ergonomics Society 37(1), 32–64 (1995) 6. Unity Game Engine, http://unity3d.com/ 7. Puppet Theater, http://www.puppettheater.com

Immersive Video Game Based on Exercise Prescription Daegun Kim and Changhoon Park Dept. of Game Engineering, Hoseo University, 165 Sechul-ri, Baebang-myun, Asan, Chungnam 336-795, Korea [email protected], [email protected]

Abstract. It is increasingly desirable to have good health. And, regular exercise is the most effective way to improve fitness. But, most people who repeat the exercise easily feel the boredom and tend to give up. This paper propose an immersive video game based on exercise prescription. The game is designed to ensure safety and effectiveness exercise. And, exercise intensity and time can be adjusted automatically with monitoring the player’s physical status. We expect this kind of game will help people increase their interest and motivation. Keywords: Health, Video Game, Exercise Prescription, Safety, Effectiveness.

1 Introduction Advances in medicine have resulted in increase of life expectancy and it is increasingly desirable to have good health. Physical fitness is considered a measure of the body’s ability to function efficiently and effectively in work and leisure activities, to be healthy, to improve the immune system, and to prevent disease. And, regular exercise is the most effective way to improve fitness. But, most people who repeat the exercise easily feel the boredom and tend to give up[1]. This paper introduces an overview of an immersive video game based on exercise prescription which provides not only entertainment but also a program of exercise[2]. We expect this game will help people increase their interest and motivation, so exercise can be a part of their daily routine.

Fig. 1. Game concept design C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 116–119, 2011. © Springer-Verlag Berlin Heidelberg 2011

Immersive Video Game Based on Exercise Prescription

117

This game is intended to support aerobic exercise to improve heart and lung function have a strong organization, has the effect of the blood vessels. And we believe that it is important to ensure exercise safety and effectiveness. So, we apply exercise prescription to game mechanics. Then, game can adjust exercise intensity and time for a player based on heart rate.

2 Immersive Game for Exercise This game provide a modified exercise bike and wireless heart rate monitor. We have modified exercise bike as a game controller. This bike has rotation sensors connected to the handlebar and pedals. This make it possible to exercise and play a video game at the same time with pedaling and steering. Especially, the level of resistance in the pedal can be controlled by PC connected via USB. And, wireless hear rate monitor is used to estimate calorie expenditure and exercise stress. These interface for exercise for exercise prescription

Fig. 2. Exercise bike and wireless hear rate sensor

Using exercise bike, player can control a game character’s speed and direction in the track. This means that we want to use fun to encourage physical activity. This game will give exercise results along with the enjoyment of game competition. And the resistance of the pedal is adjusted dynamically depending on the slope of the track in the game. The higher slope of virtual world is, the more the pedal resistance increase. At this moment, player can experience virtually uphill and downhill while watching the game screen. Thus, the design of the track is closely related a exercise program.

3 Design and Implementation Using exercise bike, player can control a game character’s speed and direction in the track. This means that we want to use fun to encourage physical activity. This game will give exercise results along with the enjoyment of game competition. And the resistance of the pedal is adjusted dynamically depending on the slope of the track in the game. The higher slope of virtual world is, the more the pedal resistance increase. At this moment, player can experience virtually uphill and downhill while watching the game screen. Thus, the design of the track is closely related a exercise program.

118

D. Kim and C. Park

In this paper, we aim to propose an effective and safe exercise program by using game technology. To achieve this goal, the racing track is designed by applying the following prescription principals. First, it would be efficient to perform aerobic exercise at least 20 minutes per session. Because, it takes at least that long for you to start burning fat. Second, warm-up and cool-down add a few minutes to exercise routine in order to reduce risk of injury and improve athletic performance. Warming up gradually revs up your cardiovascular system, increases blood flow to your muscles and raises your body temperature. And, cooling down after your workout may help gradually reduce the temperature of your muscles and regulate blood flow. Third, to get the most health benefits from aerobic activity, we should exercise at a level strenuous enough to raise heart rate to target zone. And, target heart rate zone is 50 to 75 percent of maximum heart rate (HRmax). The HRmax is the highest heart rate an individual can safely achieve through exercise stress, and depends on age. The most accurate way of measuring HRmax is via a cardiac stress test. For general purposes, people instead typically use a formula to estimate their individual maximum heart rate. The most common formula encountered, with no indication of standard deviation, is HRmax = 220 − age.

Fig. 3. Racing track based on exercise prescription principals

This game is developed based on an object oriented framework for virtual reality applications, NAVER[3, 4]. And, we have two different versions depending on 3d rendering engine, OpenScneGraph and Gamebryo. OpenSceneGraph is an open source high performance graphic toolkit. And, Gamebryo is a cross-platform 3D graphics engine and targeted at game development. This project has been executed with 5 students for a period of 9 months.

Fig. 4. Screenshots from the game

Immersive Video Game Based on Exercise Prescription

119

Fig. 5. User test

4 Conclusion and Future Work In this paper, we proposed an immersive video game to provide a program of exercise as well as entertainment. This game aims to support exercise safety and effectiveness. To achieve this goal, we apply exercise prescription to game mechanics. Then, the game can adjust exercise intensity and time for a player based on his or her heart rate. We expect this game will help people increase their interest and motivation, so exercise can be a part of their daily routine. Acknowledgement. This research project was supported by the Sports Promotion Fund of Seoul Olympic Sports Promotion Foundation from Ministry of Culture, Sports and Tourism.

References 1. Mokka, S., Vaatanen, A., Heinila, J., Valkkynen, P.: Fitness computer game with a bodily user interface. In: Proc. Of the 2th Intl. Conference on Entertainment Computing, vol. 38 (2003) 2. Van Dan, N., Kameyama, M.: Bayesian-networks-based motion estimation for a highly-safe intelligent vehicle. In: SICE-ICASE International Joint Conference, pp. 6023–6026 (2006) 3. Park, C., Ko, H., Kim, T.: NAVER: Design and Implementation of VR Framework for Gyeongju VR Theater. Computer and Graphics 27(2), 223–236 (2003) 4. Park, C., Hirose, M., Ko, H.: A platform for interactive VR storytelling. In: Pan, Z., Aylett, R.S., Diener, H., Jin, X., Göbel, S., Li, L. (eds.) Edutainment 2006. LNCS, vol. 3942, pp. 193–202. Springer, Heidelberg (2006)

Assessing the Use of Cognitive Resources in Virtual Reality William E. Marsh1, Jonathan W. Kelly1,2, Veronica J. Dark1,2, and James H. Oliver1 1

Human Computer Interaction Graduate Program, Iowa State University, 1620 Howe Hall, Ames, Iowa, USA 2 Department of Psychology, Iowa State University, W112 Lagomarcino Hall, Ames, Iowa, USA {marsh,jonkelly,vjdark,oliver}@iastate.edu

Abstract. Due to system limitations, interactions in virtual environments are often unnatural and this may impact performance. During learning, unnatural interactions draw from a finite pool of cognitive resources, meaning that those resources cannot be used for a concurrent, possibly more important task. Because users typically have primary objectives to accomplish in the virtual world, we argue that interaction techniques and other system design choices should account for task compatibility. We use a dual-task paradigm to study resource usage during locomotion tasks varying in their similarity to real-world locomotion. In one experiment, unnatural locomotion interfaces required additional spatial resources compared to natural movements. Some participants used unique strategies unlikely in traditional dual-task studies, possibly due to the high level of immersion. Keywords: Virtual reality, dual-task paradigm, working memory, locomotion, cognitive resources, user interfaces.

1 Introduction Due to system limitations, interactions in virtual reality (VR) often require responses that are unlike those in the natural world and that are based on limited sensory feedback. These unnatural responses may affect a user’s performance at interface manipulations and attempts to interact with the system are likely to interfere with performance on other tasks as well. During the interface-learning phase, unnatural interactions draw from a pool of cognitive resources [1], meaning that those resources cannot be used for a concurrent, possibly important task. Users often have primary objectives to accomplish in the virtual world, aside from interface manipulation, and understanding how interface-learning affects performance on these objectives may enable better interface design. 1.1 Limitations of Virtual Reality Hardware limitations in VR often lead to unnatural input techniques to perform actions in the virtual world. Locomotion is one noteworthy example because a C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 120–124, 2011. © Springer-Verlag Berlin Heidelberg 2011

Assessing the Use of Cognitive Resources in Virtual Reality

121

majority of virtual environments require that the user have the ability to move about while accomplishing tasks [2]. VR systems cannot support completely natural locomotion through an infinite virtual world due to boundaries such as tracking ranges or walls. Many techniques have been proposed to address these problems and to make aspects of locomotion more natural. Examples include treadmills [3], giant “hamster balls” [4], and methods of tricking the user with modified sensory feedback [5]. System limitations affect not only the input techniques, but also the feedback provided to the user by the system. Systems often have low-resolution graphics and a reduced field-of-view, for example. We are interested in understanding what the actual impact of using these systems is on concurrent tasks performed in the environment. 1.2 Dual-Task Paradigm Psychology researchers have distinguished between different pools of cognitive resources [6]. Most multi-component models of memory distinguish between verbal and visuo-spatial resources, and some research supports further partitioning the visuospatial pool into separate visual and spatial components. Verbal resources are needed by tasks requiring storage or manipulation of verbal items, while visuo-spatial resources are used for tasks involving storage or manipulation of visual or spatial information. Baddeley's [7] often cited model of working memory makes the distinction between verbal and visuo-spatial resources and further describes a “central executive” that serves to direct attention, mediating access to the other two components. A dual-task selective-interference paradigm is used to determine which set of resources a given task requires. In this paradigm, a task of interest is administered concurrently with other simultaneous tasks of known resource demands. If there is selective interference, above a general dual-task deficit, between the task of interest and a simultaneous task, it can be concluded that the task of interest requires some of the same resources as the simultaneous task. A commonly used simultaneous cognitive task is the span task in which participants are asked to remember a span of items. The nature of the items, either verbal (numbers, letters, or words) or spatial (locations), determines the type of resource that is used. For example, if a concurrent spatial span task disrupts performance at a task with unknown cognitive requirements more than a verbal span task does, then the unknown task is presumed to require spatial resources [7].

2 Using the Dual-Task Paradigm in Virtual Reality Due to the limitations of VR, interactions are likely to compete for resources from the pools described above. The dual-task paradigm has not been widely used in VR, but it can be applied to examine the cognitive resources required for different types of VR interfaces. Cognitive resource usage in VR is interesting for several reasons: • When interacting with a VR system, a user typically has other more important, resource-demanding tasks to complete in the environment than just using the

122

W.E. Marsh et al.

interface. VR is often used in domains with high cognitive demands, such as teleoperation of unmanned vehicles [8]. • Locomotion through virtual environments provides a particular challenge beyond what might normally be encountered in desktop computer applications. These environments are often large or even infinite and it is impossible to provide a fully natural interface within the physical bounds of the VR hardware. • The immersive nature of VR means that system designers have a high degree of control over all aspects of a user’s experience, including cognitive task demands.

Fig. 1. Examples of verbal and spatial memory task flow

We conducted a study applying the dual-task selective-interference paradigm to investigate cognitive resource usage during virtual locomotion tasks using interfaces varying in naturalness. Each participant was assigned one of three interfaces (between subject): gamepad, body-based, and real walking (in increasing order of assumed naturalness). Each participant completed basic movement tasks (sidestep, walk forward, rotate, duck) concurrently with different working memory conditions (within subject): spatial span, verbal span, and no task. In these scenarios, a user was presented with memory items (see Fig. 1), asked to complete movements, and then tested on the memory items. The verbal span task required a participant to remember a sequence of numbers and the spatial span task required memory of a sequence of boxes presented in different spatial locations within a grid. We predicted that the spatial task would interfere with locomotion the most and that the verbal task would interfere very little in comparison to the no-task condition. We customized the span task difficulty to each participant’s individual ability level by assigning sequences of five items if the participant could successfully complete two practice tasks at the fiveitem level. Otherwise the difficulty was set to four items. Although performance is often slower in a dual-task situation (a general dual-task deficit), results on many measures showed faster performance when participants had a concurrent task than when they did not. Many participants indicated during the postexperiment interview that they found it difficult to perform the movement tasks while simultaneously remembering a verbal or spatial sequence. We believe that users hurried to complete the movements in order to get back to devoting all of their resources to the working memory tasks. Unfortunately, because of the nature of the movements used, we had no measure of movement accuracy to assess the possibility of a speed-accuracy tradeoff. Memory task results showed marginal significance of interface group and of the interaction between group and memory task. As predicted, the locomotion tasks interfered more with the spatial span tasks than the verbal tasks. Spatial memory items

Assessing the Use of Cognitive Resources in Virtual Reality

123

were forgotten more than verbal items and stopping performance in the body-based group was worse when combined with a spatial task than a verbal task. The post-experiment interviews also revealed other relevant aspects of interacting with the VR interface. Users reported memory problems caused by being startled by the ducking task, in which an I-beam came floating overhead. A few users in the bodybased and gamepad groups believed that rotation tasks might have interfered with the memory tasks. Finally, there is the possibility that participants may have adopted strategies to remember the spatial span task that were specific to full-scale VR. For example, one user reported that in order to accomplish the spatial span task, she imagined placing her hands and feet on the squares, as in the game Twister. We note that the imagined motor actions may have interfered with the locomotion movements.

3 Conclusions and Future Work The experiment described above illustrates the kind of information that can be obtained using the dual-task selective interference paradigm to assess cognitive issues relating to VR. While this study focused on an input technique, the limited perceptual fidelity provided by VR systems may also lead to cognitively demanding strategies. We plan next to investigate the cognitive implications of restricting a user’s field-ofview. The design will be similar to the one described above except that the betweensubjects variable will be field-of-view (wide, narrow) instead of locomotion interface. If we understand the types of task conflicts that may exist in a given domain, we can strategically make system design choices and possibly even create systems that adapt according to the user’s current task load. The unexpected result that participants performed some locomotion tasks better when there was a memory task than with no task should be further explored. While we were able to measure accuracy in the span tasks, the locomotion tasks that we used involved only small, fast, direct movements that did not allow us to get a good measure of accuracy. In future research we need tasks that also allow us to measure accuracy, so that we can determine if there might be a speed/accuracy tradeoff, whereby improved performance on the memory task comes at a cost to the locomotion task. We also would like to ensure that users understand that completing movements faster will not get them to the memory recall faster. Finally, several users believed that they were capable of more difficult memory tasks and generally performance was high, so in future studies we may increase the task difficulty to more fully tax users. Acknowledgments. This research was funded by a grant from the Air Force Research Laboratory.

References 1. Anderson, J.R.: Acquisition of cognitive skill. Psychological Review 89(4), 369–406 (1982) 2. Bowman, D.A., Kruijff, E., LaViola, J.J., Poupyrev, I.: 3D User Interfaces: Theory and Practice. Addison-Wesley Professional, Boston (2005)

124

W.E. Marsh et al.

3. Darken, R.P., Cockayne, W.R., Carmein, D.: The Omni-Directional Treadmill: A Locomotion Device for Virtual Worlds. In: 10th Annual ACM Symposium on User Software and Technology, pp. 213–221. ACM, New York (1997) 4. Medina, E., Fruland, R., Weghorst, S.: Virtusphere: Walking in a human size VR ”hamster ball”. In: Human Factors and Ergonomics Society 52nd Annual Meeting, pp. 2102–2106. Human Factors and Ergonomics Society, Santa Monica, CA (2008) 5. Razzaque, S., Kohn, Z., Whitton, M.C.: Redirected walking. In: Eurographics, pp. 289–294. European Association for Computer Graphics, Geneve, Switzerland (2001) 6. Navon, D.: Resources - A theoretical soup stone? Psychological Review 91(2), 216–234 (1984) 7. Baddeley, A.: Is working memory working? The fifteenth Bartlett lecture. The Quarterly Journal of Experimental Psychology: Section A 44(1), 1–31 (1992) 8. Foo, J.L., Knutzon, J., Kalivarapu, V., Oliver, J.H., Winer, E.: Three-Dimensional Path Planning of UAVs in a Virtual Battlespace using B-Splines and Particle Swarm Optimization. Journal of Aerospace Computing, Information, and Communication 6, 271– 290 (2009)

Augmented Reality Approach to Domestic Maintenance Tasks Jorge Martín-Gutiérrez and Irene Inés Santos Pérez Universidad de La Laguna, Dpto. Expresión Gráfica en Arquitectura e Ingeniería, Avda. Astrofísico Francisco Sánchez s/n 38206 La Laguna, Spain [email protected], [email protected]

Abstract. Augmented reality technology is becoming more familiar and utilized at user level thanks to new generation of smartphones provided of webcam, big format tactile screens and internet connection. This allows creating applications which make certain human tasks nicer and more comfortable. AR technology has been applied to mechanical engineering field, concretely for assistance on repairing and maintenance tasks of complex and specific systems using expensive systems which are not very useful nor accessible for performing simple or domestic tasks. In this work we intend making this technology approachable for applications and tasks which are more common to everybody. For example, two AR technology based tools, one belonging to a smartphone platform and another using a head mounted display (HMD) glasses connected to a PC. Both allow helping user while performing installation, maintenance and adjustment of a mountain bike brakes system.

1 Introduction In next decade we will access real world through augmented reality; this process will reach global audience in 2021 according to the “Realidad Aumentada: una nueva lente para ver el mundo”- report [1]. Augmented reality (AR) is a technology which allows real-time superimposition of virtually generated images and information over real world images meaning adding a synthetic virtual part to the real one. AR is a technology helping enrichment of our perception of reality making possible real world information is complemented by digital one. AR experiences have become easy to use and especially mobile. Progress made in mobile devices, particularly smartphones with cameras but also in different technologies which combine real world with virtual information allowing the chance to enjoy this applications making possible AR accessing broad consumer sector through actual devices such as smartphones, game consoles and computers with webcams. Gartner1 consultant states AR will achieve broad public reach gradually in a 5 to 10 years horizon. Although AR has already begun to introduce in the market, its use is not quite extended yet but a great growth is expected within the next few years thanks 1

http://www.gartner.com/it/page.jsp?id=1447613

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 125–129, 2011. © Springer-Verlag Berlin Heidelberg 2011

126

J. Martín-Gutiérrez and I.I. Santos Pérez

to mobile devices (especially smartphones). Trend in virtual reality systems is becoming more transportable, comfortable and user friendly each time. Actually, computers and smartphones are both devices chosen although is expected that development of display technologies will allow creation of glasses or lens allowing showing virtual information right in front of human eye.

2 Augmented Reality in Maintenance Tasks Industrials There are few contributions where augmented reality technology is applied to machines’ mounting and maintenance and most of them are developed through research work. In less accessible fields like aerospace, applications have been developed for supporting maintenance staff performing their tasks [2, 3, 4]. In the same application field Schwald [5], introduces an AR system for training & assistance in maintenance of complex industrial equipment using an optical see-through head mounted display. ARMAR project, has developed, designed, implemented and tested on users a beta version of an AR application for support of army mechanics during routine maintenance tasks inside an armored vehicle turret [6, 7]. Actually, mechanics from the army and manufacturers like Boeing use AR glasses when staff works on vehicles, glasses show repairs step by step, target necessary tools including textual instructions. This kind of experience supports learning as well training of specific tasks. I+D department of automotive giant BMW2 (through BMW research project), include among their working lines development of an AR application supporting mechanics while performing maintenance, diagnose and repair of any fault. A recently published study has also applied AR technology for maintaining photovoltaic solar installations equipment [8]. According to these references, it could be thought that use of AR technology in repair and maintenance fields is subject to complex systems although that’s not the case because it’s actually available to anyone. A few years ago, AR apps demanded specialized equipment which wasn’t portable. Nowadays applications for laptops and smartphones merge digital information into the real world quick and easily. Design of easy apps makes communication between people and computers more natural each time allowing those who are not used to computers can interact with the system. So, the use of augmented reality may be extrapolated to nearer and common environments for everybody.

3 Project Target In this work we have developed an application based on AR use for support, repair and maintenance of a standard mountain bike brakes system. Also, it focuses on making the AR system useful as training for all bicycle users targeting substitution of the troublesome maintenance manuals and assembly instructions. Augmented manual (AR_Brakes-V manual) is developed to visualize in two platforms: smartphone or computer (through HMD). The first of both has been developed for being accessible 2

http://www.bmw.com/com/en/owners/service/augmented_reality_workshop_1.html

Augmented Reality Approach to Domestic Maintenance Tasks

127

to everybody meanwhile the second is the most suitable platform from machines requiring more complex maintenance tasks. It’s intended performing a usability study in the very near future for comparing both platforms. This work is a pilot study, which may be extensible to all electronic devices, vehicles, home appliances, etc, as they are all provided of user manuals for configuration or installation and usually understanding their instructions becomes a tough task. The possibility of creating those manuals in augmented reality as assistance during training, configuration and installation of those equipment could make learning how to use them much easier.

4 Generic AR-Manual for Devices Project consisted in developing an augmented manual with instructions of steps for installing the brake lever, V-brakes and replacement of the cartridge shoe providing also assistance for adjusting the cable’s tension. Augmented manual is presented in spiral binding so it can be leant over the mountain bike’s handlebar. Each page has a marker that belongs in AR to every task that must be accomplished. The application requires accurate position and orientation tracking in order to register virtual elements in the real world and so we have used a marker-based method. Therefore, the system requires a webcam for capturing the real world. The captured image recognizes virtual objects on the visible markers. Program used for developing scenes in PC has been Build_AR [9] while AndAR [10] software was used for creating mobile devices scenes both being GNU software – General Public license). 3D modeling of the parts manipulated during tasks was performed using Autodesk Inventor 3D software. Besides, textual information is added about tools used in every step. 3D graphics are animated so user can understand gestures that should be carried out. User provided with a mobile phone including camera will lean the augmented book over the bike’s handlebar where an included adapter should be set. Through the phone’s screen he can see virtual parts superimposed over the real ones (displaced respecting marker) (see Fig.1). Besides seeing animated components on the real environment, user will also be able to listen to a recorded voice which will guide him through every task following instructions accordingly.

Fig. 1. Augmented scenes with HMD and PC platform

Fig. 2. Augmented Smartphone

scenes

with

128

J. Martín-Gutiérrez and I.I. Santos Pérez

Most accessible and common device suggested is mobile phone although an AR glasses with earphones included could also be used (Fig.2). Traditional user manual will be substituted by several markers and AR application would be available for download from the product’s website.

5 Conclusion This study introduces a simple augmented reality system seeking incorporation of this technology to home appliances and usual vehicles for providing help in their configuration or maintenance. An application is introduced allowing user visualizing in a smartphone the augmented scenes for performing installation, substitution and maintenance tasks on a mountain bike brakes system. The AR system can substantially improve quality of maintenance services while saving costs as the same AR system can be used for different versions of several products (for example, Vbrakes is standard for many bikes models). This kind of apps can be extended for setup and installation of electrical appliances (TV recorder and programmer, Hi-fi systems, home cinemas, game consoles, etc.). We propose substitution of the usual user manual for setup and maintenance of home appliances or devices for AR based portable applications. User may access product’s website from his mobile device (iPhone, smartphone) for download and installation of the AR application. As a future work we propose performing a usability study for knowing the efficacy, efficiency and satisfaction of created AR apps drawing conclusions about viability of introducing these apps for user’s performance of common tasks. Acknowledgements. The Spanish Ministry of Innovation and Science, through the “Not oriented Fundamental Investigation Projects” (“Improvement for spatial reasoning and visualization through technologically developed tools” Project ref. TIN2010-21296-C02-02) partially supported this work.

References 1. Ariel, Fundación Telefónica: Realidad Aumentada: Una nueva lente para ver el mundo, Planeta edn. Madrid, Spain (2011) 2. Rahardja, S., Wu, E., Thalmann, D., Huang, Z.: Visualization of intelligent maintenance systems through mixed reality. In: Proceedings of the 7th International Conference on Virtual Reality Continuum and its Applications in Industry, VRCAI 2008 (2008) 3. Haritos, T., Macchiarella, N.D.: Augmented Reality (AR) for Aircraft Maintenance Technician’s Training. In: Proceedings of Society of Applied Learning Technologies: New Technologies (2007) 4. Macchiarella, N.D.: Augmenting Reality as a Medium for Job Task Training. Journal of Instruction Delivery Systems 19(1), 21–24 (2005) 5. Schwald, B., Laval, B.: An augmented reality system for training and assistance to maintenance in the industrial context. In: 11th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, pp. 425–432 (2003)

Augmented Reality Approach to Domestic Maintenance Tasks

129

6. Henderson, S., Feiner, S.: Augmented Reality for Maintenance and Repair (ARMAR). Technical Report AFRL-RH-WP-TR-2007-0112, United States Air Force Research Lab (2007) 7. Henderson, S., Feiner, S.: Evaluating the Benefits of Augmented Reality Documentation for Maintenance and Repair. IEEE Transactions on Visualization and Computer Graphics (2011) 8. Benbelkacem, S., Zenati-Henda, N., Belhocine, M., Bellarbi, A., Tadjine, M.: Augmented Reality Platform for Solar Systems Maintenance Assistance. In: Proceeding of the International Symposium on Environment Friendly Energies in Electrical Applications. In: EFEEA 2010 (2010) 9. Build_AR. Human Interface Technology Laboratory New Zealand (HIT Lab NZ) (2010.03.07), http://www.buildar.co.nz/ 10. AndAR project. Android Augmented Reality (2010.03.07), http://code.google.com/p/andar/

Development of AR Display System for Dental Surgical Simulator Katsuhiko Onishi, Shota Ito, Yusuke Kawamura, and Hiroshi Noborio Osaka Electro-Communication University, 1130-70 Kiyotaki, Shijonawate, Osaka, Japan [email protected]

Abstract. In this paper, we describe about a display system for dental surgical simulator. The design concept of our system is that the system allows the user to learning the dental surgical methods with the real hand and body posture. Therefore our system set a display, which shows virtual teeth model and real teeth and gums, close to user’s hand position and allows the user to manipulating object directly. Keywords: dental surgical simulator, direct manipulation, augmented reality.

1 Introduction In dental surgical region, the most of surgical training methods are used plastic teeth or live patients. These methods are good for training surgical skills, for example the usage of surgical tools, the surgical procedure and so on. But it is difficult to execute tasks many times because it needs the new plastic teeth at each task or live patients. Therefore many kinds of dental surgical simulator have been proposed [1], [2], [3], [4]. They allow users to learning dental surgical methods by using the unique interface. But almost of them has just only typical computing display system. The user cannot have an experience about real hand position and the body posture in the dental surgery. In this paper, we describe about a display system for dental surgical simulator. The system allows the user to learning the dental surgical methods with the real hand and body posture. Our system set a display close to user’s hand position and allows the user to operate surgical tasks directly. And it shows combined image with virtual teeth model as a surgical target and a real teeth model as other parts of the patient dental model. For adapting any view images of user’s head position, the system measures the head position and a real teeth model position by using 3DOF magnetic sensor.

2 AR Display System In dental surgery, the surgical environment is almost common that a patient lay down in front of a dentist and the dentist operates surgical tasks to him/her. In such situation, the dentist hands position and body posture is different from that of operating traditional dental surgical simulation. And in the dental surgical procedure, C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 130–133, 2011. © Springer-Verlag Berlin Heidelberg 2011

Development of AR Display System for Dental Surgical Simulator

131

it is needed to hold the surgical tools and restrict the motion for accuracy tasks, for example, drilling teeth and so on. In such situation, the dentist usually put his/her hand on the other teeth as restricting the hand motion for detailed operation. To achieve the manipulation like dental surgical operation with such kind of user’s hands position and body posture, our system set the half mirror horizontally between head position and hands position as shown in Figure 1. And the system shows combined image with virtual teeth and teeth model to user. Furthermore to adapt user’s head motion for probing and confirming surgical target, it measures user’s head motion and teeth model. And it shows the virtual teeth on LCD display with adequate view volume from the user’s head position. And it needs accuracy for operating tasks to such kind of small teeth that a dentist has to use a surgical tool with hold it on a parts of around target teeth. To achieve such kind of operating environment, our system uses real teeth model around surgical target teeth and the user is able to hold the hands on it while they execute surgical tasks and they enable to operate surgical simulation tasks precisely. User with head tracker LCD display Virtual teeth Half mirror Combined image

Surgical tool Teeth model

Fig. 1. System image

3 Prototype Display System The overview of our prototype display system is showed Figure 2(a). The application is implemented on Windows PC with OpenGL. And it measures the user’s head position and the teeth model position by using 3DOF magnetic sensor. Our system shows users a teeth model and virtual teeth projected on display by reflecting on the half mirror. Figure 2(b) and Figure 2(c) are examples of view images that the user is able to see on the half mirror. It measures a user’s view position and show the adequate view volume from the position. The system allows the user to display virtual and real teeth model simultaneously from any user’s view positions.

132

K. Onishi et al.

(a) System overview

(b) User’s view 1

(c) User’s view 2

Fig. 2. Prototype display system

4 Conclusion and Future Work In this paper, we proposed a display system of dental surgical simulator. Our system enables users to show the combined image of virtual teeth and teeth model. In our display methods, user can see the combined image from any point of view by tracking their head position and model position. We made prototype system and confirmed of our method. As future work, we are planning to implement our method to one of the other dental surgical simulator which can generate realistic virtual teeth and surgical tools and the user can manipulate it by using the haptic interface.

Development of AR Display System for Dental Surgical Simulator

133

Acknowledgments. This is supported in part by 2010 Grants-in-aid for Scientific Research(No.22360109) from the Ministry of Education, Culture, Sports, Science and Technology, Japan.

References 1. Noborio, H., Kawamoto, Y.: Digital Collision Checking and Scraping Tooth by Dental Bar. In: The 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 227–234 (2010) 2. Rhienmora, P., Gajananan, K., Haddawy, P., Suebnukarn, S., Dailey, M., Supataratarn, E., Shrestha, P.: Haptic augmented reality dental trainer with automatic performance assessment. In: The 15th international conference on Intelligent user interfaces (IUI 2010), pp. 425–426 (2010) 3. Kim, L., ParkAn, S.: Efficient Virtual Teeth Modeling for Dental Training System. International Journal of CAD/CAM 8(1), 41–44 (2008) 4. Yau, H., Tsou, L., Tsai, M.: Octree-based virtual dental training system with a haptic device. Computer-Aided Design & Applications 3, 415–424 (2006)

Earthquake Disaster Prevention Support Tool －Visualization of Prevention Effectiveness by Utilizing Augmented Reality Kyoko Yoshida1, Masahiro Urabe1, Hayato Tsuchiya1, Yasuki Iizuka2, and Kayo Iizuka1 1

School of Network and Information, Senshu University, Kanagawa, Japan {k-yoshida,iizuka}@isc.senshu-u.ac.jp 2 School of Science, Tokai University, Kanagawa, Japan [email protected]

Abstract. Japan has faced major problems with large-scale earthquake countermeasures due to its location in an earthquake zone. The Building Standards Act of Japan has been amended since previous major earthquake damage and new buildings have been built with a safer design than ever before. However, measures against falling or overturning of furniture, which could injure people inside buildings, are not sufficient. In this paper, the authors propose an earthquake disaster prevention support tool with which one can experience the shaking of furniture with or without earthquake disaster countermeasures using a 3DCG animation function of Augmented Reality (AR) and three applications of this tool. Keywords: earthquake disaster prevention, Augmented Reality (AR), disaster simulated experience.

1 Introduction Japan has faced major problems with large-scale earthquake countermeasures due to its location in an earthquake zone. The probability of the metropolitan area of Japan being struck by an inland earthquake within thirty years is 70% and the number of dead and injured people when it occurs is estimated to be about 250,000 at the maximum [1] [2]. The Building Standards Act of Japan has been amended since previous major earthquake damage and new buildings have been built with a safer design than ever before. However, measures against falling or overturning of furniture, which could injure people inside buildings, are not sufficient. In the Great HanshinAwaji Earthquake in 1995, furniture fell over and scattered in about 60% of the houses that were not fully or partially destroyed. Nearly half of the indoor injuries were due to falling or overturning of furniture [3] [4]. To solve these problems, we developed an earthquake disaster prevention support tool (AR Earthquake Disaster Prevention kit), with which one can experience the shaking of furniture with or without earthquake disaster countermeasures using a 3DCG animation function of Augmented Reality (AR). In this paper we also propose three applications of this tool. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 134–138, 2011. © Springer-Verlag Berlin Heidelberg 2011

Earthquake Disaster Prevention Support Tool

135

2 Background It is considered that preventive measures against the predicted inland earthquake in the metropolitan area would minimize damages. In the questionnaire survey that we conducted targeting the students of Senshu University, however, it was found that many students took no earthquake countermeasures because they thought that such earthquake countermeasures would cost a lot or they did not know about such countermeasures. One of the reasons for the slow progress of earthquake disaster countermeasures is that people do not know the cost of the countermeasures. In consideration of this current situation, we propose in this paper the AR Earthquake Disaster Prevention kit to visualize the cost effectiveness of earthquake disaster countermeasures using AR. AR is the technology that shows pictures of an actual environment (or part of it) taken with a Web camera and composites additional virtual information with the pictures.

3 AR Earthquake Disaster Prevention Kit The AR Earthquake Disaster Prevention kit shows a 3DCG animation of an estimated disaster caused by an earthquake of Marker differing scales, using the basic tool of the AR Furniture Card (Figure 1). The AR Furniture Card Image of contains a marker that shows furniture Name of furniture furniture, expected seismic Countermeasure: intensity scale, description about Taken/Not taken whether an earthquake disaster countermeasure is taken or not (two cards are prepared for the Expected seismic intensity same conditions with and without countermeasures being taken), and name and price of the products Product name and price of earthquake used for the countermeasure (if the countermeasure card is for furniture with the countermeasure being taken). The marker is read by the Web camera Fig. 1. AR Furniture Card and a 3DCG animation of furniture, such as bookshelf and desk, is shown on a PC monitor to simulate the shaking of the furniture during an earthquake of the expected seismic intensity. The method of shaking of the furniture, which depends on the seismic intensity scale and the earthquake disaster countermeasures, is simulated according to existing research results, movies and other data in order to reproduce actual shaking during an earthquake of the expected scale [5]. In the following three applications of the AR Furniture Card are presented.

136

K. Yoshida et al.

3.1 AR Earthquake Disaster Countermeasure Effect The AR Earthquake Disaster Countermeasure Effect reads two AR Furniture Cards: One is for furniture with the earthquake disaster countermeasure taken and the other is for the same furniture without the countermeasure. The AR Earthquake Disaster Countermeasure Effect realizes the shaking of furniture during an earthquake with or without the earthquake countermeasure being taken. With this tool one can see how the earthquake disaster countermeasure makes a difference when the furniture falls over during an earthquake. It visualizes in a 3DCG animation how furniture shakes and objects inside the furniture fall out (Figure 2). On the AR Furniture Card of the furniture for which an earthquake countermeasure is taken, the name and price of the items used for the countermeasure are described, which enables us to see immediately the effect and cost of the earthquake disaster countermeasure. With this tool one can recognize the danger that arises if no earthquake disaster countermeasure is taken. This tool thus aims to promote the countermeasure to be taken to prevent furniture from falling over. This tool is used Fig. 2. Earthquake Disaster Countermeasure Effect mostly on desks. 3.2 AR Earthquake Disaster Floor Layout The AR Earthquake Disaster Floor Layout is a tool by which one can place the AR Furniture Cards of bookshelf, bed, table, etc. on a room planning drawing, similar to considering one’s own room layout, to simulate the shaking of furniture during an earthquake(Figure 3). With this one can see dangerous and safe areas inside the room during an earthquake and design the furniture layout of each room in consideration of safety during an earthquake disaster. This tool can also be used for the layout design for office rooms of companies. The tool is used mostly on desks. 3.3 AR Earthquake Disaster Simulated Experience With the AR Earthquake Disaster Simulated Experience, one can place AR Furniture Cards in a room and see in full scale on a large screen what the room looks like during an earthquake (Figure 4). With the AR Earthquake Disaster Simulated Experience, one can simulate furniture for which no earthquake countermeasure is taken and virtually experience in full scale the situation whereby the furniture shakes and falls over during an earthquake of JMA seismic intensity scale 7 or lower. One can thus acknowledge the importance of earthquake disaster countermeasures with

Earthquake Disaster Prevention Support Tool

Fig. 3. AR Earthquake Disaster Floor Layout (Left)

137

Fig. 4. AR Earthquake Disaster Simulated Experience (Right)

this tool. This function is realized by creating 3DCG objects on the same scale as actual furniture. This tool is expected to be used in a 3D space, e.g. private room.

4 Evaluation A questionnaire survey was conducted targeting 85 students of Senshu University to measure the change in their consciousness between before and after their experience of the AR Earthquake Disaster Countermeasure Effect. The result showed that the number of students who had concern in earthquake countermeasures increased by 11.8%, the number of students who were more scared of earthquakes increased (with the number of students who were not scared of earthquakes decreased by 7.1%), and the number of students who did not take earthquake countermeasures but want to take measures increased by 3.7% (Table 1). We therefore consider that the AR Earthquake Disaster Countermeasure Effect raises people’s awareness of disaster and their interest in earthquake disaster countermeasures. Table 1. Result of questionnaire survey Are you concerned in earthquake countermeasures?

Yes Neither yes or no. No

Are you scared of earthquakes?

Before(%) 32(37.6) 26(30.6)

After(%) 42(49.4) 21(24.7)

Before(%) 45(52.9) 7(8.2)

After(%) 47(55.3) 11(12.9)

27(31.8)

22(25.9)

33(38.8)

27(31.8)

Do you want to implement an earthquake countermeasure? (Question to those who do not have earthquake countermeasure.) Before(%) After(%) 22(40.7) 24(44.4) 14(25.9) 14(25.9) 18(33.3)

16(29.6)

We received at the exhibition comments such as “I became aware of earthquake disaster in visible ways,” “I think that knowing about earthquake disaster countermeasures is not enough. The countermeasures would be more promoted if sales companies of countermeasure products against earthquake disasters utilize them.”

138

K. Yoshida et al.

5 Conclusion In this paper we proposed the AR Earthquake Disaster Prevention kit which utilizes AR technology and showed the evaluation result. Future problems include resolution enhancement of the model to improve the reality sensation, increase of diversity of earthquake countermeasure products for furniture, application of the special effect to objects other than furniture by combining specific models, and translating the model scenarios into reality.

References 1. An estimation of ground motion map of Japan, http://www.jishin.go.jp/main/chousa/09_yosokuchizu/index.htm 2. A report on the anticipated damage caused by Tokyo Metropolitan Earthquakes, The committees for technical investigation on countermeasures for Tokyo Metropolitan Earthquakes, the Central Disaster Management Council, Cabinet Office, Government of Japan (2005), http://www.bousai.go.jp/jishin/chubou/taisaku_syuto/pdf/ higaisoutei/gaiyou.pdf 3. A report on investigation of house damage in the Great Hanshin-Awaji Earthquake, Architectural Institute of Japan (1996), http://www.fdma.go.jp/html/life/kagu9.html 4. Promotion of Measures to Secure Furniture, Tokyo Fire Department (2004), http://www.tfd.metro.tokyo.jp/inf/h16/i024_02.htm 5. Fukuwa Laboratory, Nagoya University, http://www.sharaku.nuac.nagoyau.ac.jp/laboFT/fall_furniture/gaiyou.html

Part III

Gestures, Gaze and Multimodality in HCI

A Three-Dimensional Fingertip Interface Yangkeun Ahn1,2, Kwangmo Jung2, and Jiman Hong1 1

2

Soongsil University, Seoul, South Korea Korea Electronics Technology Institute, Seoul, South Korea [email protected]

Abstract. This paper proposes a method to recognize the fingertip threedimensionally using infrared stereo cameras. The proposed method can be used for human-computer interactions with a three-dimensional display and is designed to provide robust performance against finger trembling, kinematic errors, and sensor noise. This paper describes the proposed method in detail and also presents implementation results. Keywords: Fingertip Recognition, Stereo Camera, Spatial Touch.

1 Introduction This paper proposes a three-dimensional interface using a fingertip with no separate equipment to ensure interactions between the three-dimensional image projected onto the space and the user. The image projection system discussed here is a device that projects a two-dimensional image onto the concave mirror, resulting in binocular parallax and enabling the user to gain a sense of visual depth. This system provides the user with an immersive interface. The use of the existing touch screen technique to ensure interaction between the visualized image and the user entails the following problems: First, the input device is exposed to the user when a physical touch screen is installed, thus undermining the user’s sense of immersion. Second, the touch screen equipment should recognize the depth of the finger, as the image perceived by the user is not two-dimensional, but conventional touch screens are incapable of recognizing depth. In order to provide the user with an immersive interface, therefore, an optical method should be chosen such that the existence of the touch screen is unknown to the user. Also, the location of the user’s finger should be identified in a threedimensional space so as to facilitate interaction with the user. This paper proposes a method using two infrared cameras to identify the location of the finger in threedimensional space (see Fig. 1). In order to implement the proposed method, this paper discusses detection of the location of the fingertip within images, alignment of the fingertip points of two images to calculate the three-dimensional location, and the design of a linear Kalman filter to deal with image noises.

2 Methodology and Test Details The method proposed in this paper can be summarized as follows: First, infrared light is projected onto the finger using the infrared emitter. Second, the images of the C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 141–145, 2011. © Springer-Verlag Berlin Heidelberg 2011

142

Y. Ahn, K. Jung, and J. Hong

reflected finger are recorded with infrared stereo cameras. Lastly, the pixel coordinates of the finger extracted from the two images and the cameras’ intrinsic parameters are used to calculate the three-dimensional location of the finger. The overall process is described in Fig. 2. This section introduces details of each method.

Stereo Camera

Screen

Infrared LED Array

Stereo

Camer a Jig

Hand

Fig. 1. Conceptual diagram of proposed device and configuration of testing equipment Input Image

+

-

Source Image

Thresholding

Background Image

Input Image

+

+

-

Source Image

Thresholding

Background Image

Distance Disparity Calculation

Stereo Matching

Fig. 2. Conceptual diagram of processing process

2.1 Configuration of Testing Equipment The testing equipment used in this paper is shown in Fig. 1. The infrared stereo cameras are used here to minimize errors caused by the subject(s) other than the finger. The impact of noise can be reduced, as infrared wavelengths alone are accepted from near distance. Two Chameleon (a model from Point Grey) cameras are

A Three-Dimensional Fingertip Interface

143

installed on the stereo jig, as shown in Fig. 1. An infrared radiation device is used to project infrared light onto the finger and extract the finger’s image. 2.2 Extraction of Fingertip Image This section explains how to extract the image of the fingertip from the fed images. This method consists of hand area extraction, binarization, fingertip area extraction, and Kalman filter-based stabilization. When the system operates, the background image of the environment, where the equipment is located, is obtained by showing the hand in the image. The obtained background image is used in the difference operation with a subsequently entered image to remove the background from the latter image, and the resulting image consequently contains the hand area reflected by infrared light only. The extracted image of the hand is a gray-scale image with each pixel having a value from 0 through 255. Binarization is carried out to clearly distinguish the finger area from the remaining area. 2.3 Extraction of Fingertip Area To identify the coordinates of the fingertip from the binarized hand area, a filter bank is created based on human observations. The filter bank is a method to connect filters with different conditions in series and obtain desired results in each phase to ultimately gain the fingertip’s image. This method breaks the geometrical forms of the fingertip into three phases in total, producing powerful results from a combination of simple rules. The geometrical characteristics of the fingertip can be summarized as follows [1]: − The finger area remains white within the binarized image; − The whole of the small circle S is included in the white area in Fig. 3; and − The large circle L cuts the fingertip area in two points only (i.e., points a and b) in Fig. 3. The method used here applies the aforementioned three filters consecutively to take the ultimately remaining area as the fingertip area candidate. The algorithm for this method can be expressed as follows:

Fig. 3. Geometric analysis of fingertip and optic axes of two non-transversal cameras

144

Y. Ahn, K. Jung, and J. Hong

For each pixel (x,y): For each filter: If i-th rule at pixel (x,y) is false Skip pixel; Else Mark (x,y) as fingertip End if; End for; End for; Return fingertip candidates; 2.4 Calculation of Three-Dimensional Location of Fingertip This paper assumes that the cameras are calibrated in advance. The intrinsic parameter matrices of the two cameras, obtained through calibration, are denoted as here; all locations are specified with the left camera as the basis. Fig. 3 describes how the optic axes of the two cameras do not exactly intersect with the fingertip points extracted from the stereo vision. Thus, the minimum distance crossing the two rays orthogonally should be derived [2]. The extrinsic parameters of the calibrated stereo cameras are referred to as , which represent the left rotating matrix (=I), the right camera’s rotating matrix in relation to the left side, the location of the left camera (=[0,0,0]), and the right camera’s location with the left camera as the basis, respectively. The new parameters of the stereo vision can be written as below: (1) To identify the three-dimensional location of the finger using new parameters, : the following linear system should be solved to derive the parameters

, (2)

To solve the above equation, the matrix linear system:

is derived as given below to set up the (3)

, can be derived from The parameter set, fingertip can be determined on this basis using .

, and the location of the

2.5 Test Results The system is used to test the user’s inputs. When the user enters rectangular, triangular, and circular movements, the results also resemble those movements. Performing computation 20 times per second, the system can be reliably used as an interface.

A Three-Dimensional Fingertip Interface

145

Fig. 4. Test results

3 Conclusion This paper has proposed an infrared stereo camera-based method for threedimensional fingertip recognition. Using the proposed method, we identified the location of the fingertip with the series filter bank, stabilized it using the Kalman filter, and obtained the three-dimensional coordinates of the fingertip with the left camera as the basis. To assess the level of accuracy in the future, it will be essential to videotape already known movements and compare the two results. Also, research is needed on a correction algorithm designed for 1:1 correspondence with the designated image.

References 1. Letessier, J., Berard, F.: Visual Tracking of Bare Fingers for Interactive Surfaces. In: UIST 2004, ACM, New York (2004) 2. Trucco, E., Verry, A.: Introductory Techniques for 3-D Computer Vision. Prentice Hall, Englewood Cliffs (1998-2006)

Rule Based Trajectory Segmentation Applied to an HMM-Based Isolated Hand Gesture Recognizer Jounghoon Beh1, David Han2, and Hanseok Ko1,3 1

Institute for Advanced Computer Studies, University of Maryland, College Park, USA {jhbeh,hsko}@umiacs.umd.edu 2 Office of Naval Research, USA [email protected] 3 School of EE, Korea University, Seoul, Korea [email protected]

Abstract. In this paper, we propose a simple but effective method of modeling hand drawn gestures based on their angles and curvature of the trajectories. Each gesture trajectory is composed of a unique series of straight and curved segments. In our Hidden Markov Model (HMM) implementation, these gestures are modeled as connected series of states analogous to series of phonemes in speech recognition. The novelty of the work presented here is the automated process we developed in segmenting gesture trajectories based on a simple set of threshold values in curvature and accumulated curvature angle. In order to represent its angular distribution of each separated states, the von Mises distribution is used. Likelihood based state segmentation was implemented in addition to the threshold based method to ensure that gesture sets are segmented consistently. The proposed method can separate each angular state of training data at the initialization step, thus providing a solution to mitigate ambiguity on initializing HMM. For comparative studies, the proposed automated state segmentation based HMM initialization was considered over the conventional method. Effectiveness of the proposed method is shown as it achieved higher recognition rates in experiments over conventional methods. Keywords: Trajectory segmentation, hand gesture recognition, hidden Markov model, HMM initialization.

1 Introduction The proper presentation of hand gesture motion is essential for the performance of an automatic hand gesture recognition system. The trajectory of motion is widely used as a distinct feature to represent not only hand gesture recognition [1],[2] but also action recognition [3] and hand-written character recognition [4],[5],[6]. Hand gesture in this work is represented by a trajectory composed of spatiotemporal sequence of hand positions in 2-D. In addition gesture trajectory can be decomposed into several strokes and curves that maintain a consistent angular tendency. Hand signal modeling in most of the related research has been based on classifying each of such angular tendencies, and connecting them to a Markov chain C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 146–150, 2011. © Springer-Verlag Berlin Heidelberg 2011

Rule Based Trajectory Segmentation

147

to add temporal property such as Hidden Markov Model (HMM) [7]. In HMM each stroke or curve is modeled as a state with statistical parameters such as mean, mixture weight, and variance that are optimized by Baum-Welch re-estimation formulae [8] with training data. However, in practice, some of the parameters such as a number of states comprising a gesture vocabulary are determined manually. The number of states is an important parameter because an excessive number of states can result an over-fitting problem if the number of training samples is insufficient compared to the number of model parameters. When there are insufficient number of states, HMM’s ability to discriminate correctly in turn is reduced. A common way of determining the optimal number of states is by trial and error. Gunter and Bunke [4] optimized the number of state by iterative refinement of the recognizer performance. Starner, Weaver, and Pentland [1] determined the number of state as a constant regardless of gesture pattern. Lee, Kim, and Kim [5] used the skeleton pattern of handwritten character to determine the number of state for on-line handwriting recognition. Elmezain et al. [2] empirically assigned the number of state based on the hand motion trajectory. In this paper, we propose a Figure-based Trajectory Segmentation (FTS) algorithm to automatically determine the number of states based on a simple set of rules in terms of trajectory angles.

2 Proposed Method 2.1 Figure-Based Trajectory Segmentation Gesture trajectories are acquired by sampling centroid locations of the hand. The proposed method of the FTS segments the trajectory into a series of states by dividing them as either a straight line or a curve. For a straight line segment, curvature of the segment is expected to be negligibly small. For a curved section, its curvature value is finite and is expected to remain below some value. By a simple thresholding, these two types of trajectory segments can be easily separated. If, on the other hand, the curvature is significantly large, it can be inferred that the trajectory exhibited an abrupt heading change such as a vertex. Additional segmentation may be performed on curves by considering their cumulative value of the curvatures by another threshold. Upon exceeding the threshold, the curve section is segmented, and the trajectory is considered to have transitioned into another segment. Our implementation of the FTS is illustrated in Fig. 1. Curvature is computed [3] by

k (t ) =

|| v(t ) × a(t ) || || v(t ) ||3

(1)

where v(t) and a(t) denote velocity and acceleration of hand motion at time t respectively. The cumulative curvature angle is defined by k c (t ) = where

τ

t

∑ k (i) || v(i) ||

i =t −τ

denotes the time at previous segmentation point.

(2)

148

J. Beh, D. Han, and H. Ko START

Read data Calculate curvature Y

curvature > thc

Segment

N Accumulate curvature angle Y

cumulated angle > tha

Initialize cumulation

N N

End of data Y

thc: threshold for line segmention tha: threshold for curve segmention

END

Fig. 1. Flowchart of the proposed FTS

The number of segment is adjustable by tuning the angular change threshold. In this work its value is set to 80˚. In Fig. 2 the examples of segmentation for hand gesture trajectory “A” and “D” are depicted.

(a) Hand motion trajectory for letter “A”

(b) Hand motion trajectory for letter “D”

Fig. 2. Example hand motion trajectories “O” mark denote the detected turning point of segmentation

2.2 Determining Number of States The proposed FTS method was applied to a training database of a set of alphanumeric hand gesture symbols. Due to variability of gesture motion from different individuals, it has been observed that a same gesture symbol may be divided into different number of segments. Among various segmentation lengths of these symbols, we selected the number of states shown to be the most frequent in the training set as the number used for the construction of the HMM models. 2.3 HMM State Parameter Initialization Since the angular heading of hand motion is considered as an input feature of our recognizer, we employ the von Mises probability density function (PDF) [9] to represent the state distribution of the angular heading, x. The von Mises PDF is given by

Rule Based Trajectory Segmentation

f ( x | μ,κ ) =

eκ cos( x − μ ) 2πI 0 (κ )

149

(3)

where I0(·) is the modified Bessel function of order zero. μ and κ are, respectively, the circular mean and concentration that are analogous to the mean and variance in the normal distribution. For each state in each gesture HMM, the segmented trajectory data are used to estimate the parameter μ and κ initially. Then, we refine the segment boundary (turning point) by aligning input feature to corresponding state using (3). This procedure is repeated iteratively until there is no change in the segment boundary position. After this refinement the final update of μ and κ are to be the initialized parameters.

3 Performance Evaluation 3.1 Experimental Settings The following are the set of alpha-numeric symbols we have chosen for our initial study: 1. Numbers: 1, 2, 3, 4, 5, 6, 7, 8, 9 2. Alphabet: A, B, C, D, X, Y, Z 3. Symbols: ↑, ↓, ←, →, @ The hand gesture database consists of samples from 22 people. Three separate hand gesture sets were collected from each individual. For evaluation, 11 people’s data are used for training the gesture recognition HMM, and the remaining data were used for testing. Video frame rate in the database was 40 Hz and the image resolution was 160 x 120.

3.2 Results We compared the performance of the proposed method to conventional approaches [5, 6]. In Table 1, the fixed number method sets the number of state of each HMM to a constant, and the Bakis method sets the number of state of each HMM to the average length of the corresponding sequence of feature multiplied by a constant f. In our implementation of the fixed number method, the best result was obtained with 4 states, and for the Bakis method we obtained the best result with setting f to 0.08. Table 1. Performance comparison

Recognition error rate (%)

Fixed number 9.53

Bakis 12.99

Proposed method 7.50

150

J. Beh, D. Han, and H. Ko

From the result the proposed method achieves the lowest error rate. Since training the HMM is based on the maximum likelihood estimation principle, a proper initial estimation is essential for obtaining globally maximized likelihood function for good recognizer performance. The proposed segmentation inherently clusters angular tendencies in a gesture symbol trajectory. It determines the number of states in HMM consistently, therefore it would also result proper initialization the parameters of each state. Another advantage of the proposed method is that it automatically determines the number of states, unlike other method that require manual trial-and-error procedure, thus it simplifies the training part of the algorithm.

4 Conclusions A rule based trajectory segmentation method to initialize hand gesture HMM is proposed in this paper. The number of state in HMM is automatically determined by the number of segments in the hand gesture by a simple rule based algorithm. Those segments are used to initialize statistical parameters of state of each HMM. From the experimental result, the proposed method reduced the error rate by an average of 31.8% over the conventional methods. Advantages of the proposed segmentation method are that the training phase is simple and yet consistent in making good initial estimates for the HMM.

References 1. Starner, T., Weaver, J., Pentland, A.: Real-time American sign language recognition using desk and wearable computer based video. IEEE Trans. PAMI 20(12), 1371–1375 (1998) 2. Garg, P., Aggarwal, N., Sofat, S.: Vision based hand gesture recognition. World Academy of Science, Engineering and Technology 49 (2009) 3. Rao, G., Yilmaz, A., Shah, M.: View-inveriant representation and recognition of actions. International Journal of Computer Vision 50(2), 203–226 (2002) 4. Gunter, S., Bunke, H.: HMM-based hand written word recognition: on the optimization of the number of states, training iterations and Gaussian components. Pattern Recognition 37, 2069–2079 (2004) 5. Lee, J.J., Kim, J., Kim, J.H.: Data-driven design of HMM topology for on-line handwriting recognition. In: The Proceedings of 7th International Workshop on Frontiers in Handwriting Recognition, pp. 239–248 (2000) 6. Gunter, S., Bunke, H.: Optimizing the number of states, training iterations and Gaussian s in an HMM-based Handwritten Word Recognizer. In: The Proceedings of the 7th International Conference on Document Analysis and Recognition (2003) 7. Juang, B.-H., Rabiner, L.R.: A probabilistic distance measure for hidden Markov models. AT&T Technical Journal 64(2), 391–408 (1985) 8. Baum, L.E.: An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes. Inequalities 3(9), 1–8 (1972) 9. Fisher, N.I.: Statistical Analysis of Circular Data, Ch. 3.3.6 The von Mises distiribution, pp. 48–56. Cambridge University Press, Cambridge (1993)

Head-Free, Remote Eye-Gaze Detection System with Easy Calibration Using Stereo-Calibrated Two Video Cameras Yoshinobu Ebisawa, Kazuki Abo, and Kiyotaka Fukumoto Faculty of Engineering, Shizuoka University, Johoku 3-5-7, Hamamatsu, 432-8561 Japan [email protected]

Abstract. The video-based, head-free, remote eye-gaze detection system based on detection of the pupil and the corneal reflection was developed using stereocalibrated two cameras. The gaze detection theory assumed the linear relationship; θ=k|r’|. Here, θ is the angle between the line of sight and the line connecting between the pupil and the camera, and |r’| indicates the size of the corneal reflection - pupil center vector. Three novel easy calibration methods were proposed; ‘automatic’, ‘one-point’, and ‘two-point’. In the ‘automatic’, the user does not have to fixate the specific location in the PC screen. In the ‘one-point’, the angular difference between the optical and visual axes of the eye was determined and used for compensation. The ‘two-point’ was proposed to compensate the nonlinear relationship between |r’| and θ, which occurs when θ is large. The precision of gaze detection was compared among the three methods using the developed gaze detection system. Keywords: eye-gaze detection, pupil, corneal reflection, user calibration, head-free, infant.

1 Introduction So far, several non-contact, remote, eye-gaze detection systems allowing head displacements have been developed for human monitoring or human interface [1][2]. However, the current systems require some effort by the user that he or she must fixate several points on a PC screen for gaze calibration [3][4]. Moreover, in many of the other systems, a movable range of the user’s head is narrow. We have already developed a precise head-free eye-gaze detection system with three cameras [5]. To detect pupil easily and to generate the corneal reflection, which are necessary to estimate the eye-gaze point, the near-infrared LEDs were installed in each camera. Two of the cameras are stereo-calibrated wide view cameras, which detect 3-D positions of the pupils. Another one is a narrow view camera with a pantilt drive unit. The latter camera tracks one of the pupils and detects the angle of the line of sight by high resolution. The angle was determined using the corneal reflection - pupil center vector. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 151–155, 2011. © Springer-Verlag Berlin Heidelberg 2011

152

Y. Ebisawa, K. Abo, and K. Fukumoto

Although the resolution of eye-gaze detection of this system was high, the system can detect only one eye. Moreover, the pan-tilt unit may obstruct the progress of development of the system by its high cost and dynamic mechanism, and so on. So, in the present study, we develop two camera systems in which the function of the narrow camera is installed in those of the stereo-calibrated two cameras. In addition, in the three camera system, we proposed two easy calibration methods: the moving target and two-point calibration methods [5]. Both methods used only the inclination of the corneal reflection - pupil center vector for calibration. Accordingly, by using the size as well as the inclination of the vector, the improvement of the precision is expected. Moreover, in the two-camera system, four points of gaze on the screen can be detected. Averaging them may increase the resolution of gaze detection. Recently, the recent research showed that the gaze distribution of the infant with autism is peculiar [5], and the application of the eye-gaze detection system to the autism diagnosis support system is expected. In addition, the medical screening at the stage of the infant is hoped for. However, it is difficult to make infants fixate several calibration points. So we propose three easy calibration methods and compare them in the developed two-camera system experimentally.

2 System Configuration and Eye-Gaze Detection Theory Each optical system of the developed two-camera system consists of a NTSC B/W video camera having a near-infrared sensitivity, a 25 mm lens, an infrared pass filter (IR80), and light sources having many infrared LEDs (Fig. 1(a)). The two cameras are driven with slightly shifting synchronization. The LEDs are lit synchronized with the exposure period (500µs). A PC captures the images of the user’s face via an image grabbing board and then detects the centers of a pupil and a corneal reflection for each are detected. The two cameras are separately set under a 19 inch display. By camera calibration, the camera position, O, comes to be known in the world coordinate system (Fig. 1(b)). By stereo matching, the 3-D pupil center position, P, is detected. Let a gaze point, Q, be the intersection between the line of sight and a visual object plane (PC screen, known), the line of sight and the position of Q are determined by the two angles (θ and φ) and the position of P. To understand this theory easily, we define a virtual gaze plane, H, which is perpendicular to line OP and passes the camera position, O. The plane rotates along with the pupil position. The intersection of plane H and line PQ is defined as point T. The position of T in the coordinates system of H is expressed by θ and φ. In the gaze detection theory, the angels, θ and φ, are related to the corneal reflection - pupil center vector, r, (Fig.1 (b)). Here, actually the vector is calculated as the corresponding 3-D vector, r’, and is used. The direction of r’, φ’, is defined as the angle from the horizontal axis in the camera image. Actually φ’ is compensated by the pose of the camera. First, the theory assumes φ’=φ. Here, φ is angle between line OT and the horizontal axis on the plane H. This horizontal axis is defined as the nodal line between plane H and the horizontal plane in the world coordinate system. Second, the theory assumes a linear relationship between |r’| and θ, as follows.

Head-Free, Remote Eye-Gaze Detection System with Easy Calibration

θ = k |r’|

153

(1)

Therefore, obtaining the value of k in eq. (1) means the gaze calibration. However, in general, there is the several-degree difference between the visual axis and the optical axis of the eye. This difference produces the errors in gaze point detection. To determine the point precisely, the user must fixate one calibration point at least. In addition, when θ is large, the relationship between |r’| and θ may become nonlinear. It depends on the user individual. Since the system in this study has the two cameras, the gaze detection theory is expanded as shown in Fig. 2.

(a)

(b)

Fig. 1. (a) System appearance and (b) gaze detection theory. (a) The light source arranging the infrared LEDs having two different wavelength in two concentric rings (inner: 850 nm, outer: 940 nm) is attached to each camera. When the inner LEDs turn on, the pupils become bright. When the outer LEDs turn on, they become dark. These two groups of LEDs are alternately turned on synchronously with the field of the camera. By differentiating the consecutive bright and dark pupil images, the pupils become conspicuous in the difference image due to cancellation of the background images except for the pupils. The pupils are detected in the difference image. (b) See in the text.

Fig. 2. The expanded gaze detection theory. P indicates one of the two eyes. OL and OR indicate the positions of the two cameras. If the calibration value, k, is determined, the two lines of sight can be calculated for one eye. For another eye, the two lines of sight can be also determined.

154

Y. Ebisawa, K. Abo, and K. Fukumoto

3 Eye-Gaze Calibration Methods We tried the following three calibration methods. (a) ‘Automatic’ calibration method: The user does not need to fixate any known point. In Fig. 2, the vectors OLTL and ORT R are determined by k, |r’L|, |r’R|, φL’ and φR’. TL and TR are projected onto the gaze projection plane, G. To minimize the distance between the two projected points (T L’ and T R’) on plane G, the value of k was determined. This method assumes to detect the optical axis of the eyeball. (b) ‘One-point’ calibration method: The user fixates one point presented at the center of the PC screen. The horizontal and vertical angular differences between the optical and visual axes (ΔQx, ΔQy) are recorded. The value of k is also determined by the same method as method (a). They are used for gaze detection. (c) ‘Two-point’ calibration method: The user fixates not only the calibration point presented at the center but also another point presented at the top of the screen. This method is proposed to compensate the nonlinear relationship between θ and |r|; this may occur when θ is large. The following equations are used.

⎧k |r ' | θ=⎨ 1 ⎩θ B + k 2 (|r ' |-|r ' B |)

k 1|r ' | ≤ θ B k 1|r ' | > θ B ,

(2)

where θB and |r’B| indicate the averages of θ and |r’| when the user fixated the center calibration point. The values of k and (ΔQx, ΔQy) obtained by method (b) are used for gaze detection. The value of k is used as k1.

4 Experiments Three university students participated in the experiments. In the calibration procedure, the subject fixated the two calibration points; the center and top of the screen. After calibration procedure, the subject fixated nine targets evenly arranged on the PC screen one by one. 30 gaze points (1 sec) were acquired for each target. In both procedures, the distance between the subject’s face and the PC screen was approximately 80 cm. Fig. 3 (a)-(c) compared the gaze point distributions among the three calibration methods for subject KY. Each dot shows the average of both eyes’ gaze points. Fig. 3 (d) compares the average gaze detection errors in visual angle among the three calibration methods. The three subjects showed the average errors of 2.16 deg in the ‘automatic’, 1.37 deg in the ‘one-point’, and 0.94 deg in the ‘twopoint’ calibration method. In another experiment, the same three subjects moved back and forth by ±10 cm and right and left by ±5 cm. At seven positions within the range, they fixated the nine targets. Here, either was selected from among the one-point and two-point calibration methods depending on subjects. The average and SD of the gaze error for the three subjects was 0.92±0.40 deg.

Head-Free, Remote Eye-Gaze Detection System with Easy Calibration

155

Fig. 3. Comparison of results among three easy calibration methods

5 Conclusions The eye-gaze detection system using the stereo-calibrated two cameras was developed. The three novel easy calibration methods were proposed. The automatic and one-point calibration methods may be useful for the infant. If calibrated well, the developed gaze detection system experimentally showed the average gaze error less than 0.92±0.40 deg for 20 cm-back and forth, and 10 cm-right and left head movements.

References 1. Guestrin, E.D., Eizenman, M.: General theory of Remove Gaze Estimation Using the Pupil Center and Corneal Reflections. IEEE Trans. on Bioml. Eng. 53(6), 1124–1133 (2006) 2. Zhu, Z., Ji, Q.: Novel Eye Gaze Tracking Techniques under Natural Head Movement. IEEE Trans. on Bioml. Eng. 54(12), 2246–2260 (2007) 3. Tobii Technology, http://www.tobii.com/ 4. Eye Tech Digital Systems, http://www.eyetechds.com/ 5. Kondou, Y., Ebisawa, Y.: Easy Eye-Gaze Calibration using a Moving Visual Target in the Head-Free Remote Eye-Gaze Detection System. In: Proc. of VECIMS 2008, pp. 145–150 (2008) 6. Jones, W., Carr, K., Klin, A.: Absence of Preferential Looking to the Eyes of Approaching Adults Predicts level of Social Disability in 2-Year-Olds Toddlers with Autism Spectrum Disorder. Archives of General Psychiatry 65(8), 946–954 (2008)

Eye Position Effect on Audio-Visual Fusion Involves New Proposals for Multimodal Interface Design David Hartnage, Alain Bichot, Patrick Sandor, and Corinne Roumes Département Action & Cognition en Situation Opérationnelle, Institut de Recherche Biomédicale des Armées, BP 73, 91223 Brétigny-sur-Orge, France [email protected]

Abstract. Combination of audio and visual information is expected to ensure an efficient interface design for spatial information. Then, we focus on Audiovisual (AV) fusion referred to the perception of unity of audio and visual information despite there spatial disparity [1]. Previous experiment showed that AV fusion varied over space mainly with horizontal eccentricity [2]. As audio spatial information is coded in relation to head position and visual information is coded relative to eye position, question arises on eye position effect. The current psychophysical experiment investigates the effect of horizontal eye position shift on the variation of AV fusion over the 2D frontal space. Results showed that eye position affects AV fusion. Current data support the need for including eye position inputs when displaying redundant visual and auditory information in integrated multimodal interfaces. Results are discussed considering the probable effect of visual display structure.

1 Introduction A wide variety of working situations such as teleoperation, simulation for training and air operations, require the use of a virtual 3D restitution of the world. The so-called Virtual Reality can remain partial in the case of Augmented Reality when computed data are displayed overlaying the natural perception of the environment. The HumanMachine Interface (HMI) necessarily provides the operator with information in a non natural way. The human has to deal with a new kind of reality characterized by new physical rules. In order to design more ecologically the interface and to ensure the expected level of efficiency, it is worth understanding how the human assumes these new realities. A present HMI design challenge concerns multisensory integration. Combining different sensory information provides faster reaction time [3] and ensures a redundant information processing. These benefits are decisive in time determinant situations and may enhance situation awareness. This defy implies to study the way human perceive auditory and visual information as coming from the same location in space. This phenomenon is referred to as audio-visual (AV) fusion. The present paper concerns the rules of AV fusion in space and their involvement for HMI design. Perception of the world is basically multimodal; the world is not spread over sensory cues, it is perceived as unified [4]. Experimental results have shown an enhancement of cell responses for multimodal stimuli in various locations of the C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 156–160, 2011. © Springer-Verlag Berlin Heidelberg 2011

Eye Position Effect on Audio-Visual Fusion Involves New Proposals

157

nervous system [5][6]. Association of sensory cues provides many advantages: it is ecological, accurate, efficient and reliable; information is redundant; and responses are faster. Information coming from different modalities may be perceived as one. This phenomenon of perceptual fusion is usually compared to the ventriloquism effect [7]: the perception of the spatial location of a sound is biased in the direction of a visual stimulus (visual capture). Audio-visual fusion (AV fusion) has been investigated by Godfroy et al. [2]. They showed that AV fusion capability varied with the eccentricity of the bimodal stimulation in the participant’s perceptive field, the smallest spatial windows laying in the median sagittal plane. AV spatial windows were found to be symmetrical in relation to that plane. So, AV fusion space was considered as head-centered. These variations over space followed closely the spatial resolution of audition. There are quite a large number of reference frames depending on the sensory system involved, the context, and the goal [8]. Each of them can be egocentric (relative to the subject), allocentric (relative to the environment) or even geocentric (relative to the earth). What kind of frame of reference supports the AV fusion in space? Actually, primary unimodal egocentric reference frames are different whether information involves the visual sensory system or the auditory system. As vision information is initially coded relative to the retina, which depends on the position of the eye, the reference frame for vision is considered as eye-centered. For audition, spatial information depends mainly on inter-aural time differences (ITD), inter-aural level differences (ILD) in azimuth, and spectral cues in elevation [9]. These cues depend on the position of the head; so, the auditory reference frame is considered as head-centered. The aim is to define the limits of human AV fusion perception when the eyes are shifted in order to display trustworthy bimodal information whenever the operator does not look straight ahead, which may be the most common situation. Results will be discussed in the frame of the suggested applications.

2 Methods 2.1 Subjects Seven volunteers participated in this study, 4 women and 3 men aged, from 25 to 45. They all had normal or corrected to normal vision and no auditory defects. 2.2 Experimental Setup Stimuli Control The subject was located at the center of an acoustically-transparent, hemi-cylindrical screen, 120 cm in radius and 145 cm in height. The subject’s head was maintained by a custom bite-board with the eyes at a 110-cm height. The head was oriented 10° leftward of the center of the screen to increase the space of investigation when a fixation spot was presented 20° to the right. The orientation of the gaze was monitored with an ASL 504 (50Hz) eye-tracker placed 45 cm in front of the subject at a level lower than the field of investigation to prevent from any visual masking. To

158

D. Hartnage et al.

avoid any effect of allocentric cues, the experimental room was in total darkness, and noise level was reduced as much as possible (< 39 dBA). Stimuli could only be presented if the subject was looking at the red fixation spot with an angular error less than 1.66° for a mean duration of 500 ms (300-700 ms). Such a feedback between the eye-tracker sampling and the experimental software was used guarantee the dissociation of the two references frames at the bimodal stimulus onset. The bite-board permitted the control of the head position (i.e. the auditory reference frame) and the eye-tracker permitted the control of the eye position (i.e. the visual reference frame). The visual part of the bimodal 500ms stimulus was a 1° spot of light, it was provided by a green laser beam (MELLES-GRIOT, 532 nm, 5 mW) attenuated with optical filters in order to reduce luminance at 3 cd.m-2 (<5µW). The auditory part of the bimodal stimulus was a 49 dBA broad-band pink noise delivered by one of the 19 loudspeakers (LS) located behind the screen, oriented toward the subject’s head. Disparity For each of the 7 loudspeakers at eye level, 61 disparities between the spot and the sound source were tested. The spot could be displayed 0° to 20° horizontally apart from the center of the associated loudspeaker. At 0° of azimuth the spot could be displayed 0° to 30° apart with a 5° step in elevation. For all other loudspeakers only 9 disparities were tested. 2.3 Task The subject had to judge the perception of unity emerging from the bimodal stimulus using a joystick. When the spot and the pink noise were perceived as coming from a unique and common location in space, “fusion” response was selected by pulling the joystick. When the two unimodal stimuli were perceived as issuing from two distinct locations, the “non fusion” response was selected by pushing the joystick.

3 Results For most of the loudspeakers, fusion limits in elevation could not be established. Subjects still responded “fusion” for more than 50% of the trials for disparity up to 30°. AV fusion limits in azimuth varied along the horizontal dimension in the perceptive field (F(6,1243)= 21.783; p<.001). This variation appeared laterally deviated in the direction of eye shift in the reference frames dissociated condition. No significant difference was obtained between the 2 conditions (F(1,1243)= 0.973; ns) but the interaction between conditions and eccentricity was significant (F(6,1243)= 4.93; p<.05). The signs of the differences were analyzed. Differences between the two conditions were significant for 5 paired eccentricities out of the 7 tested. Moreover, the sign of the differences changed as if the curve of the dissociated condition would have been shifted 10° to the right (toward eye shift, 50%).

Eye Position Effect on Audio-Visual Fusion Involves New Proposals

159

4 Discussion The present basic experiment was conducted in order to study the effect of gaze shift on AV fusion. Results show that lateral gaze shift influences AV fusion perception in azimuth but not in elevation. Data collected in this experiment must be taken into account when considering HMI design. Designers from both virtual environments and enhanced perception used head tracker, and the displayed data changes in relation to head movement. As a consequence, gaze is assumed to be in line with head position when displaying visual and auditory information. It can be suggested currently that the lack of eye monitoring may bias the expected perception of visual and auditory redundant information. This question is particularly important for aircraft Head Mounted Display’s (HMD), where time is determinant. In such a 3D dynamic environment where the relative position of the various actors may dramatically change, it may be fruitful for the pilot to take advantage of the high mobility of the eyes compared to the dynamic of the head equipped with helmet. Indeed, the spherical shape of the eye, it’s location in the hemispherical socket, make any change in the eye orientation effortless. So it can be inferred that pilots usually favor eye movement instead of head movement. In many situations, especially in combat, there are high Gz accelerations, head movement will be limited and pilots will use more eye movement. Even though, it can be argued that this situation is very complex compared with psychophysics tasks, the present results have to be considered to avoid error in multimodal display. The experiment reported in this paper deliberately involved non informative visual and auditory stimuli. No allocentric cues were available. The bimodal stimulus was the only sensory input to process. Such conditions are in crucial opposition with HMIs where a high flow of data is concerned. Nevertheless, such basic data are not available in the literature. The ideal technological solution would take into account head and eye position to change the display in relation to the position of the eye and head to match with the perceptual space of the operator. To overpass this technical difficulty, it may be easier to exchange a hardware refinement for a software manipulation. If AV fusion naturally varies according to head and eye orientation, can an additional item in the information displayed provide a counter-measure to the eye shift effect? The present experiment was conducted in total darkness, with no additional visual cue provided. However, previous experiments have shown that visual display can influence AV fusion perception as well. Perhaps visual cues like linear symbols enhancing the edges of the display or a peculiar spatial arrangement of the features structuring visual space can give an allocentric reference frame to the subject. This allocentric reference frame would compensate for dissociation between auditory and visual reference frames. These data suggested further investigations to determine whether visual or auditory additional markers in the perceptive field may counteract eye inputs effect on audio-visual fusion. Moreover, such selected visual and auditory stimuli may also counterbalance the reported tendency of an upward offset of the perceived location of auditory sources in darkness [10].

160

D. Hartnage et al.

References 1. Bertelson, P., Radeau, M.: Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Perception & psychophysics 29(6), 578–584 (1981) 2. Godfroy, M., Roumes, C., Dauchy, P.: Spatial variations of visual-auditory fusion areas. Perception 32(10), 1233–1245 (2003) 3. Diederich, A.: Intersensory facilitation, vol. 369. Peter Lang, Frankfurt (1992) 4. Gibson, J.J.: The senses considered as perceptual systems. Greenwood Press, Westport (1966) 5. Stein, B.E., Meredith, A.M.: The merging of the senses. The MIT press, Cambridge (1993) 6. Giard, M.-H., Peronnet, F.: Visual-auditory Integration during Multimodal Object recognition in Humans: A Behavioral and Electrophysiological Study. Journal of Cognitive Neuroscience 11(5), 473–490 (1999) 7. Jack, C.E., Thurlow, W.R.: Effect of degree of visual association and angle of displacement on the ventriloquism effect. Perceptual and Motor Skills 37, 967–979 (1973) 8. Paillard, J.: Motor and representational framing of space. In: Paillard, J. (ed.) Brain and space, pp. 163–182. Oxford University Press, Oxford (1991) 9. Blauert, J.: Spatial Hearing. In: The psychophysics of human sound localization, The MIT press, London (1983) 10. Pedersen, J.A., Jorgensen, T.: Localization performance of real and virtual sound sources. In: New Directions For Improving Audio Effectiveness, pp. 29.1–29.30. RTO, France (2005) 11. Perrot, D.R., Saberi, K.: Minimum audible angle thresholds for sources varying in both elevation and azimuth. Journal of the Acoustical Society of America 87, 1728–1731 (1990) 12. Witkin, H.A., Asch, S.E.: Studies in space orientation. IV. Further experiments on perception of the upright with displaced visual fields. Journal of Experimental Psychology 38, 762–782 (1948)

A Virtual Mouse System Using Finger-Gestures of Twisting-in Takashi Kihara1 and Makio Ishihara2 1

Graduate School of Computer Science and Engineering, Fukuoka Institute of Technology, 3-30-1, Wajiro-higashi, Higashiku, Fukuoka, 811-0295 Japan [email protected] 2 Faculty of Information Engineering, Fukuoka Institute of Technology, 3-30-1, Wajiro-Higashi, Higashi-ku, Fukuoka, 811-0295 Japan [email protected]

Abstract. In this manuscript, we introduce the finger gesture of twisting-in to make an input device and conduct an experiment on performance. The finger gesture of twisting-in is a series of movements of a finger twisting and pushing. It gives us the feeling of twisting and pushing a button. The experiment shows that our input device has the potential to be practical and usable. Keywords: twisting-in, finger gesture, virtual mouse, input device.

1 Introduction A keyboard and mouse is a common input device. In recent years, various portable input devices have been manufactured. For example, NIPPON STYLE [1] manufactures a portable input device called a virtual keyboard using a semiconductor laser (see Fig. 1, left) and Apple manufactures iPad with a touch screen. In research, the trend is to use multiple off-the-shelf cameras in order to track the user’s fingertip or his/her hand in three dimensional coordinates in order to assemble an input device. The user moves his/her fingertip or hand before the cameras to be interpreted to computer’s commands. In this manuscript, we use a single camera to track the fingertip. When using two or more cameras, we have to adjust the position of every camera. However, when there is one camera, it does not take the time and effort into the setting. As for single camera-based input devices, Y. Hirobe et al. [2] built an input device which recognizes the fingertip from a picture taken by the attached single camera and enables the user to perform a series of movements of the fingertip to organize the corresponding computer’s command (see Fig. 1, right). This manuscript focuses on a user-friendly gesture-based input device so that the user can manipulate it in a more natural and easier way.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 161–165, 2011. © Springer-Verlag Berlin Heidelberg 2011

162

T. Kihara and M. Ishihara

Fig. 1. Virtual key board (left) and In-air typing interface for mobile devices using a single camera (right)

2 Finger Gesture of Twisting-in Fig. 2 shows an example of the finger gesture of twisting-in. The finger gesture of twisting-in is a series of movements of a finger twisting and pushing. For example, we do this when thrusting the cork of wine. Since this finger gesture is familiar with our everyday activities, it gives us the feeling of twisting and pushing a button, and provides us with a more natural and intuitive interface to computers. In this manuscript, we use the finger gesture of twisting-in to generate a couple of mouse events.

Fig. 2. Finger gesture of twisting-in

3 Design of Virtual Mouse System Fig. 3 outlines the system. This system consists of a PC and web camera. The system tracks an index finger with the camera and recognizes the index fingertip as a mouse cursor. If we twist our index finger before the camera, we can click on and drag an item depending upon the fingertip’s position and orientation. To perform this, the system analyzes the finger gesture of twisting-in from a sequence of pictures taken by the camera and transmits the corresponding mouse event to the operating system. At first, the system extracts skin color from each picture taken by the camera to make a binary image that represents a boundary of the

A Virtual Mouse System Using Finger-Gestures of Twisting-in

163

index finger. Next, the system detects the position and orientation of the index fingertip from the binary image. Finally the system tracks this information to generate mouse events. Fig. 4 shows how the system generates mouse events. There are three sorts of mouse events: move, clickin and clickout. Move event represents movement of the mouse cursor and clickin event represents the mouse button pressed, and the clickout event represents the mouse button released. For example, clickin event is transmitted when the index finger is leaned to more than 30 degrees counterclockwise in order to drag.

Fig. 3. Outline of our system

Fig. 4. Timing of mouse events’ generation

4 Experiment and Result To evaluate the system, we conducted an experiment on performance. We used the one-direction tapping test described by ISO(9241-9). Two rectangles of width w and with a center-to-center distance d were presented to the subject (see Fig. 5). The task was to point and click, along one axis, within each rectangle 25 times. There were 5 subjects between the age of 23 and 24. Each subject was asked to perform the task 6 trials for each of our system, a common mouse and touchpad. The index of difficulty varied over the trials. The index of difficulty is given by the following equation. Index of Difficulty =

d w

(1)

The elapsed time and the number of mis-clicks outside rectangles during each trial were measured.

164

T. Kihara and M. Ishihara

Fig. 5. One-direction tapping test

Fig. 6 and Table 1 show the results. In Fig. 6, the horizontal axis shows the elapsed time in second and the vertical one shows the effective index of difficulty. The effective index of difficulty is given by the following equation. Effective Index of Difficulty =

d we

we = 4,133 sx

(2) (3)

where sx is the standard deviation of the selection coordinates in a horizontal axis. Table 1 shows the average of the number of mis-clicks outside rectangles at each index of difficulty for our system, a mouse and touchpad.

Fig. 6. Index of difficulty as a function of elapsed time

Fig. 6 implies our system requires more time to complete the task than the others. This is because the image processing uses CPU power much enough to cause a certain delay in refresh rate. However, Table 1 implies the result seems fine if the index of difficulty is four or less. We expect our virtual mouse system to work well if the image processing performance is improved.

A Virtual Mouse System Using Finger-Gestures of Twisting-in

165

Table 1. Number of mis-clicks outside rectangles Index of difficulty Mouse TouchPad Our system

1

2

3

4

5

6

0.4 0.6 1.6

1.2 1 1.4

1 0.8 0.6

2.2 1.4 2

1.8 1 6.6

2 2 4.4

5 Conclusion In this manuscript, we introduced the finger gesture of twisting-in to make an input device and conducted an experiment on performance. The finger gesture of twistingin is a series of movements of a finger twisting and pushing. It gives us the feeling of twisting and pushing a button. The experiment showed that our input device seemed not practical but usable. In our future work, we are going to improve image processing speed with CUDA technology to reduce the delay in refresh rate.

References 1. http://www.tanomi.com/limited/html/00034.html NIPPON STYLE Virtual Key Board (VKB) 2. Hirobe, Y., Funahashi, K., Niikura, T., Watanabe, Y., Komuro, T., Ishikawa, M.: In-air Typing Interface for Mobile Devices using a single Camera. ITE Technical Report 33(23), 66–68 (2009), IST2009-32, ME2009-108

Control of Five Finger of Computer Graphics Hand Using Electromyographic Signal Measured with Multi-channeled Small Laplacian Electrodes Takuya Kimura and Akinori Ueno Department of Electrical and Electronic Engineering, School of Engineering, Tokyo Denki University, 2-2 Kanda-Nishiki-cho, Chiyoda-ku, Tokyo 101-8457, Japan [email protected], [email protected]

Abstract. The authors propose a new simple method for controlling five fingers of computer graphics hand respectively using electromyograms signal. We manufactured five double-polar concentric ring electrodes, for measuring surface Laplacian electromyograms(Laplacian EMGs) from flexor muscle corresponding to each finger flexing action. Laplacian EMGs can detect the current flowing into the component that is proportional to the vertical surface current density. The experimental results shown the each manufactured electrode could measure the Laplacian EMG corresponding to each finger flexion without any interference and that the developed interface could control the CG hand in real time. These results infer that the proposed approach is promising as an EMG-based human-computer interface or human-machine interface. The next challenges to be addressed are to conduct the evaluation test for many subjects and to facilitate electrodes attachment. Keywords: EMG, Human interface, Laplacian electrode.

1 Introduction In recent years, devices using EMG such as EMG prosthetic hand has been developed to control the computer and human interface. And, the Laplacian electrode can detect signals along the axis into the origin perpendicular to the surface. A pilot measuring circuit and a dedicated interface circuit were also assembled by reference to previous our approach [1]-[2]. Therefore, multiple Laplacian electrodes is expected to be detected independently of each finger on each EMG. The purpose of this study are Laplacian multi-channel EMG signals detected using multiple electrodes, and development of device of control of five finger of CG hand using the EMG signal.

2 Theoretical Basis of Body Surface Laplacian The use of the body surface Laplacian has been first proposed by Hjorth in electroencephalogram in 1975 [3] and by He and Cohen in electrocardiogram in 1992 [4], but not in EMG. Considering a local orthogonal coordinate system (x, y, z) with C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 166–170, 2011. © Springer-Verlag Berlin Heidelberg 2011

Control of Five Finger of Computer Graphics Hand

167

origin at a point on the body surface where the z axis is orthogonal to the body surface, the Laplacian EMG, LS, is defined by applying a Laplacian operator to the body surface potential φ as follows: ⎛ ∂ 2φ ∂ 2φ ⎞ ⎛ 1 ⎞ ∂J LS = −∇ 2 xyφ ≡ −∇ xy ⋅ ∇ xyφ ≡ −⎜ 2 + 2 ⎟ = −⎜ ⎟ z ⎜ ∂x ⎟ ∂ x ⎝ σ ⎠ ∂z ⎝ ⎠

(1)

Thus the Laplacian EMG signal is negatively proportional to the normal derivative of the normal component of the current density at the body. Therefore, Laplacian EMG is supposed to be sensitive to the firing of the measurement. Accordingly, the signal is potentially useful as an input for human interfaces, because it is less susceptible to the interferences caused by the activity of approximal muscles than the conventional EMG signal.

3 System Configuration 3.1 Laplacian Electrode In this study, fabricated Laplacian electrode is bipolar concentric ring electrode. Bipolar concentric ring electrode can be realized by disk electrode and forming the electrodes on the outer ring of concentric circles. And taking the difference between the two electrodes can detect the vertical component proportional to the current flowing into the origin. Therefore, the Laplacian electrodes can be to measure potentials from a localized area smaller diameter. This fabricated electrode, the two electrodes area is 50mm2, the distance between the electrodes is 1.0mm, diameter electrodes is 14mm. 3.2 Detection of EMG Fig.1 shows the block diagram of the Laplacian EMG measurement circuit. This circuit is composed of high input impedance instrumentation amplifier, notch filter, high pass filter, low pass filter, inverting amplifier, additional type Driven-RightLeg(D.R.L.)circuit. Laplacian electrode skin contact impedance is high because it is small, therefore inst.amp. with high input impedance first-stage IC was used. The circuit constants in the H.P.F. and L.P.F. were set to obtain a cutoff frequency of 10 and 1000Hz, respectively. Notch filters were used to reduce 50Hz interference. Amplification factor of 50,000 times the entire circuit. Additional type D.R.L. circuit was tried for S/N improvements by feedback signal to the Body-Earth, the signal adding the residual signal of five channels. 3.3 Signal Converter Fig. 2 shows the block diagram of the signal converter circuit. This circuit is composed of H.P.F., Absolute value circuit, Integrating circuit, Inv.Amp., Comparator, Buffer, Relay circuit and USB Gamepad. The first stage is H.P.F. with a cutoff frequency of 53Hz, which cuts the low frequency component of EMG in order

168

T. Kimura and A. Ueno

Laplacian Electrode Inst. Amp.

-

Notch Filter 50Hz

H.P.F. 10Hz

L.P.F Inv. 1kHz Amp.

Buffer

×5

Electrode + (Body-Earth) D.R.L.

Fig. 1. A block diagram of the Laplacian EMG measurement circuit/

to reduce baseline variation. In the second stage, full-wave rectified signal to the Absolute value circuit In the third stage, the time constant of 0.1s to smooth at the integration circuit. Then, the comparator input signal is amplified 10 times. A pulse outputs that exceeds the threshold of the comparator. The comparator has hysteresis to prevent chattering. Shorts the gamepad button and send to PC when the input signal of high the at relay circuit.

EMG Amp.

PC

H.P.F.

Absolute value circuit

USB Gamepad

Relay circuit

Integrating circuit

Buffer

Inv. Amp.

Comparator

Fig. 2. Block diagram of the signal converter circuit

4 Evaluation of Developed System Three male volunteer aged 22 were instructed to sit down on a chair. Laplacian electrode was fixed by surgical tape and rubber bands to each subject's right forearm. Arrangement of electrodes was fixed to the best part out of the fingers being in EMG at pilot experiment. We control for EMG than CG hand when flexion of the five finger. In addition, similar experiments were performed with disposable electrodes fabricated same diameter attached to the same site. And it is compared the discrimination probability. In addition, the discrimination probability if we could control without interfering with the other fingers, finger obtained at 5 × 20 sets of 100 trials. Show the discrimination probability of disposable electrodes in Table 1, discrimination probability of Laplacian electrodes in Table 2.

Control of Five Finger of Computer Graphics Hand

1ch Thumb 2ch Index

169

1.77Vp-p 14.6Vp-p

3ch Middle 4ch Ring

10.7Vp-p 11.2Vp-p

5ch Little

4.58Vp-p 2s Fig. 3. Recordings of Lalacian EMG during individual finger flexions Table 1. Discrimination Probability of disposable electrodes

Subject A Subject B Subject C

1ch Thumb

2ch Index

3ch Middle

4ch Ring

5ch Little

Total

0%

0%

0%

55%

0%

11%

65%

15%

10%

5%

50%

29%

35%

0%

90%

5%

95%

45%

Table 2. Discrimination Probability of Laplacian electrodes

Subject A Subject B Subject C

1ch Thumb

2ch Index

3ch Middle

4ch Ring

5ch Little

Total

95%

75%

65%

100%

100%

87%

100%

85%

95%

80%

90%

90%

100%

85%

70%

95%

100%

90%

5 Conclusion The result is a high probability Laplacian EMG without interfering with other finger flexion can be measured in response to each finger, five fingers can be controlled in real time showed that CG hand. The next challenges to be addressed are to conduct the evaluation test for many subjects and to facilitate electrodes attachment. In addition, on-off only control to the

170

T. Kimura and A. Ueno

CG, and it is expected to be high discrimination probability is obtained by performing signal processing such as pattern recognition.

References 1. Miyazawa, K., Ueno, A., Mori, H., Hoshino, H., Noshiro, M.: Development and Evaluation of a Wireless Interface for Inputting Characters using Laplacian EMG. In: 28th Ann. Int. Conf. IEEE EMBS, pp. 2518–2521 (September 2006) 2. Ueno, A., Uchikawa, Y., Noshiro, M.: A Capacitive Sensor System for Measuring Laplacian Electromyogram through Cloth –A Pilot Study-. In: 29th Ann. Int. Conf. IEEE EMBS, pp. 2518–2521 (September 2006) 3. Hjorth, B.: An on-line Transformation of EEG Scalp Potentials into Orthogonal Source Derivations. Electroenceph. Clin. Neurophysiol. 39, 526–530 (1975) 4. He, B., Cohen, R.J.: Body: Body Surface Laplacian ECG Mapping. IEEE Trans. Biomed. Eng. 39(11), 1179–1191 (1992)

Kinematic Analysis of Remote Target Pointing Hand Movements in a 3D Environment Yung-Hui Lee and Shu-Kai Wu Department of Industrial Management, National Taiwan University of Science and Technology, No. 43 Kee-lung Road, Section 4, Taipei, Taiwan, ROC [email protected]

Abstract. The study investigates kinematics of target pointing hand movements in a free-hand, touchless, 3D environment. Consistent with Fitts’ original information processing model, target pointing hand movement time in a 3D space increased with an increase of distance, and increased with a decrease of target size. The relative primary submovement movement time (% of total movement time) for target with depth (67.24%) was higher than that for targets without depth (64.49%). Pointing targets in the upper half of the reference framework required a lifting up of the arm to work against the gravity, and thus cost more than those in the lower half. For targets with depth, they required an even elevated upper-arm, thus result a longer “primary submovement time”, a decreased “peak velocity”, and a decreased relative time to peak velocity. Keyword: Target pointing hand movement; 3D, depth perception, hand/arm kinematics.

1 Introduction As an alternative to traditional input devices (mouse, trackball, joysticks, and etc.), “3D target acquisition hand movement” was explored as an input function to various appliances that require hand-free and no–touch interaction with computer. In comparing with wearable input devices, hand movement seem to be a more natural alternative, can be considered as a part of the environment, and be operates remotely (Nakamura, et al., 2008). Lee et al. (2010) investigates and models the performance of target pointing hand movements in a free-hand, touchless 3D environment. Their results indicate that these movements are constrained in several ways. As the distance to a target is increased, or the target size is decreased, movement time is increased. Locations of the targets affected the performance. Moreover, the pointing target “Forward” position in the Z-axis (3.88 s) required more time than any other position. Recent analyses of the kinematics of pointing movements reveal an invariant shape characterized by an initial high velocity peak, which is sometimes followed by one or more low velocity, secondary peaks (Thompson et al., 2007). The goal of the current work is to clarify the nature of the above mentioned constraints on the performance of target pointing hand movements in 3D environment. In particular, we will investigate the combined effect of and interaction among target size, distance, and location on the C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 171–175, 2011. © Springer-Verlag Berlin Heidelberg 2011

172

Y.-H. Lee and S.-K. Wu

kinematics of pointing movements. We believe that the effects on movement kinematics from multiple sources will help to reveal the nature of the speed/accuracy tradeoff, and consequently, how these constrained movements are organized based on task relevant information.

2 Method 2.1 Subjects Ten volunteers, aged 22–26 years (23.4 ± 1.5 years old), self-declared right-handed, participated in this study. They were paid voluntary for the study. 2.2 The 3D Control A 3D computer vision based systems was used to track free hand, on-touch, control movement in real time. Figure 1 illustrated the hand/head position tracking and the target acquisition hand movement. The system is capable of recognizing gestures at 20 Hz. It was then the velocity and the accelerations of the hand movements were calculated. Details of system validations can be found in Lee et al. (2011).

Fig. 1.Two cameras tracked hand/head positions and target acquisition hand movements. Subjects sat 3 meters in front of a display screen.

2.3 The 3D Perspective Display Targets, cursor, and the visual database were computer generated and were presented on a 42-inch LCD screen (BENQ S-Series). Since targets were presented in a 2D perspective display, shadow of the target on the mesh with linear perspective and a forward-looking schematic was used to provide visual information in 3D. 2.4 Tasks There are a total of 17 targets on a reference framework (17 positions). Eight targets are located in the middle frame (with z=0, without depth). They are in the X-axis, Yaxis, and XY plan (a 2D plan). Other 9 targets were located, with depths, in the z-axis. Spherical targets are one of three sizes (1.8, 2.8 and 3.8 cm in radius) and are located in one of the three distances (8, 12, and 16 cm). Please see Lee et al. (2011) for illustration of the design of the tasks.

Kinematic Analysis of Remote Target Pointing Hand Movements in a 3D Environment

173

2.5 Performance Measurements Movements for each trial were divided into primary submovement and secondary submovmeent followed Thompson, et al. (2007). A reaction time was determined as the time from the target appearing on the screen and the velocity of the hand exceeding 8% of peak velocity. The remaining time constituted the total movement time used in the analysis. This time was further parsed into a primary phase and a cumulative secondary phase. In addition, measures of interest in the analysis were the mean velocity and the peak velocity of the primary submovment, relative time to peak velocity (% of total movement time), and relative primary submovement movement time (% of total movement time). Followed Fitts (1954), we deduced Fitt’s regression models based on moving distances (D) and target sizes (S). A calculation of the task difficulty index (ID=log2 2D/S) shows that ID ranged from 2.07 to 4.15. 2.6 Experimental Procedure To avoid mixing the change of target sizes with the change of target sizes as they recede in distance from an observer in 3D spaces, a split plot design was adopted, where the target size was selected as a whole plot. Target acquisition tasks were repeated in three different days for each subject. The experiment involved 3 days of experiment for each subject, each day consisted of three sessions and a total of 153 (3 sizes × 3 distances × 17 positions) targets were acquired. 2.7 Data Analysis A split-plot design was used for the data analysis, with target size as a whole plot effect. Factors of target distance and position were randomized within the plot. Tukey Test was used for post hoc comparison. An alpha level of 0.05 was selected as the minimum level of significance.

3 Result and Discussion 3.1 Primary, Secondary Submovement Time Primary submovement time: Target pointing hand movement time in a 3D space increased with an increase of distance, and increased with a decrease of target size. Post hoc analysis indicates that the time for a 3.8 cm target (1.59±0.96s) was significantly shorter than that for 2.8 cm (1.77±1.02s), followed by 1.8 cm (2.47±1.69s) (p<0.05). The time for a distance of 8 cm was 1.56±1.24s, which was significantly shorter than that for 12 cm (1.93±1.18s), followed by 16 cm (2.34±1.43s) (p<0.05). There was also a significant two-way interaction effect between “depth” and “position” (F(7, 63)= 7.07, p<0.01). This interaction suggests that the time for targets without depth and in the lower part of the spherical frame was shorter than that for targets with depth. However, the time for targets with depth and in the upper part of the spherical frame was shorter than that for targets without depth. Secondary submovement time: A total of 72% tasks that had “secondary submovement(s).” Only “Target size” significantly affected secondary submovement

174

Y.-H. Lee and S.-K. Wu

time (F(2, 18)= 195.12, p<0.01). Post hoc analysis indicates that the mean time for a 3.8 cm target (0.81±1.32s) was significantly shorter than that for a target of 2.8 cm (1.06±1.53s), followed by that for 1.8 cm (2.34±2.49s) (p<0.05). Relative primary submovement movement time (% of total movement time): ANOVA results showed that “size” and “distance” had a significant effect on the relative time to peak velocity. Figure 2 illustrated the relative time to the total movement time, mean velocities, peak velocities, and relative time to peak velocity as a function of index of task difficulty and effect of “depth”. Post hoc analysis indicated a larger percentage for targets with size of 3.8 cm (70.90±27.27%) than that of 1.8 cm (57.12±26.38 %). There was a larger percentage for targets with a distance of 16 cm (70.05±23.02 %) than that of 8 cm (58.89±30.38 %). Data in Figure 2 (a) showed that the percentage for target with depth (67.24%) was higher than that for targets without depth (64.49%). The primary, or ballistic, portion of a rapid aiming task is believed to be under feedforward control, with secondary submovements reflecting the ‘‘homing-in’’ or corrective phase.

(a)

(b)

(c)

(d)

Fig. 2. The (a) relative time to the total movement time, (b) mean velocities, (c) peak velocities, and (d) relative time to peak velocity as a function of index of task difficulty and effect of “depth”.

3.2 Velocity and Related Kinematic Variables Mean velocity: Mean velocity was significantly affected by “target size” and “distance”. In addition, ANOVA results reveal a significant two-way interaction effect between “depth” and “distance” (F(2, 18)= 12.49, p<0.01). This interaction suggests that targets “with depth” have significantly lower mean velocities when they were with longer “distances”.

Kinematic Analysis of Remote Target Pointing Hand Movements in a 3D Environment

175

Peak Velocity: Peak velocity was significantly affected by “distance” and “position”. ANOVA results indicate that the “depth” of the targets also affected peak velocity (F(1, 9)= 171, p<0.01). The peak velocity for targets without depth was 14.31±4.61 cm/s, which was significantly faster than that for targets with depth (12.98±3.70 cm/s) (p<0.05). ANOVA results also reveal a significant two-way interaction effect between “depth” and “distance” (F(2, 18)= 15.44, p<0.01). This interaction suggests that targets with larger index of task difficulty have a higher peak velocity as Fig. 2 (c) illustrated. This situation required participants to execute the largest movements of the hand along with large forces (Kuhtz-Busckbeck et al., 1998). For targets with depth in the upper half, they required an even elevated upper-arm, thus result a longer “primary submovement time” and a lower “peak velocity”. Relative time to peak velocity: Both “size” and “distance” had a significant effect on the relative time to peak velocity (% of total movement time). There was also a significant two-way interaction effect between “depth” and “position” (F(7, 63)= 6.48, p<0.01). Please see Fig. 2 (d) for illustration. The percentage for targets decreased linearly with the increased of index of task difficulty, indicating participants spent an increased amount of time in the deceleration phase of the movement. The increased mass for movements in depth reduced the peak velocity, resulting in more corrective, secondary submovements (i.e., smaller relative time to peak velocity), consistent with the role of task constraints in the speed/accuracy tradeoff.

References 1. Fitts, P.M.: The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47, 381–391 (1954) 2. Kuhtz-Buschbeck, J.P., Stolze, H., Boczek-Funcke, A., Johnk, K., Heinrichs, H., Illert, M.: Kinematic analysis of prehension movements in children. Behavioural Brain Research 93(12), 131–141 (1998a) 3. Lee, Y.H., Wu, S.K., Liu, Y.P.: Performance of Remote Target Acquisition Hand Movements in a 3D environment, Human Movement Sciences (2010) (in printing) 4. Nakamura, T., Takahashi, S., Tanaka, J.: Double-crossing: A new interaction technique for hand gesture interfaces. Computer Human Interaction 8th Asia Pacific Conference, 292–300 (2008) 5. Thompson, S.G., McConnell, D.S., Slocum, J.S., Bohan, M.: Kinematic analysis of multiple constraints on a pointing task. Hum. Movement Science 26, 11–26 (2007)

Design and Implementation of Deformation Based Gesture Interaction Wonkyum Lee1, Sungkwan Jung1, Sangsik Kim1, Woojin Ahn1, and Sang-su Lee2 1

KAIST Institute for IT Convergence, 335 Gwahangno, Yuseong-Gu, Daejeon, South Korea 2 Department of Industrial Design, KAIST, 335 Gwahangno, Yuseong-Gu, Daejeon, South Korea {wklee,skjung,sskim98,woojinahn}@itc.kaist.ac.kr, [email protected]

Abstract. We present an approach of designing and implementing a novel interaction in a future flexible display. By featuring flexibility of flexible displays, deformation-based gestures are employed as input techniques for interacting with digital contents. We designed gestures for the selected tasks, commonly found in commercial device, based on our prior study investigating how users prefer to manipulate flexible displays. We devised a concept of device which use the deformation based gesture interaction and implemented a prototype based on currently available technology. Sensors mounted on the prototype enable us to recognize deformation without visual sensors which are conventional in gestural recognition. The prototype is evaluated with applications that support our interaction by participants. Evaluation results show that deformation based gesture interaction can increase intuitiveness and fun. Keywords: interaction design, deformation-based interaction, flexible display, organic user interface.

interaction,

gesture

1 Introduction While recently various researches about user interface are studied by many researchers, the new concept of user interface, Organic User Interface (OUI), has drawn much attention in the HCI community. OUI represents that deformation of shape or form can be means of interacting with computers [1]. Various interaction methods employing deformation were proposed by many researchers and these studies showed the possibilities of deformation-based gesture interaction in flexible display. Schweisg et al. proposed a bendable computer prototype, in which zoomingin and zooming-out are implemented with bend sensor [2]. Twend was a prototype that uses bending gestures as input for a mobile device based on optical bend sensor [4]. In Paperwindows [3], paper-like interaction has been simulated with projector. Lee et al. investigate the user-preferred manipulation in flexible display based on user observation, and it gives a guideline of designing deformation-based interaction [5]. Our work is extension of these researches with user-centered interaction design and practical implementation of deformation-based gesture recognition. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 176–179, 2011. © Springer-Verlag Berlin Heidelberg 2011

Design and Implementation of Deformation Based Gesture Interaction

177

2 Designing Interaction We conducted user-centered approach to design deformation-based gesture interaction. By considering that most frequently used contents of information devices are music, videos, texts, and images, we extracted the common commands in interacting with the contents. The common commands are anticipated to be essential commands in flexible display targeting mobile information device. We categorized the commands with depths that distinguish the processes from beginning to end of interacting with the contents (Fig. 1).

Fig. 1. Set of commands and deformation-based gestures designed for flexible display

The deformation-based gestures were defined and assigned into the commands based on prior study [5]. In this study, 31 participants were asked to deform the given 3 types of A4-sized sheets as means of manipulating imaginary display, in which each type has the distinct elasticity. 343 gestures observed when users deform thin plastic were considered in our interaction scenarios, and the most preferred gesture for each command was selected in terms of average preference, average planning time and agreement score [6]. Selected set of gestures consists of bending, flipping, translating and swaying. The obtained set of gestures gains totally more than 75% preference from the evaluation results and the gestures were mapped into commands for completion of designing interaction (Fig. 1).

3 Implementation We made our prototype based on our designed interaction. Since there were, to our knowledge, few commercial flexible displays that provide full flexibility, mounting flexible display was ruled out and graphic output was displayed on rigid screen. When user manipulates our prototype with deformation, the screen showed the interaction result corresponding to the gesture user made. The prototype (Fig. 2) that we implemented consists of two boards; one is flex board implemented on flexible printed circuit board (FPCB) and another is MCU board implemented on rigid printed

178

W. Lee et al.

circuit board (PCB). The dimension of flex board is 25.5cm x 26cm, and the thickness is 3mm at most, enough to be integrated into possible future flexible display. 16 bend sensors at 16 edges both sides (at 8 edges each side), 5 accelerometer sensors at 4 corners and at center and 1 gyroscope sensor are mounted on flex board to measure the deformation of flex board. The dimension of the MCU board is 6 cm x 9 cm, and the thickness is 11mm at most. The MCU board has TI MSP430 MCU and the MCU reads the sensor data at a frequency of 200 Hz and the sends the sensor data to computing machine, Windows PC in this paper, via USB interface. The concept of recognizing deformation was use of developable surface model [7]. This model enabled us to estimate the deformation of planar sheet in limitations of measurement resolution. The bend sensor read only the degree of total bending without details ranging actual bend sensor, however guiding rule, along which flat sheet is bent, were obtained from the bending angle data of edges. All of our deformation based gestures were categorized by guiding rule, and mapping into gestures were conducted by matching between guiding rules from measurement data and each reference data corresponding to each gesture. Additional data of accelerometer sensors and gyroscope sensors was employed not only to recognizing other non-deformation based gestures such as flipping, swaying, and translating but to help recognize deformation with orientation status.

Fig. 2. Prototype for deformation-based gesture interaction

4 Evaluation We conducted evaluation of proposed interaction with user study. Participants, 25-32 years old university students, were asked to interacting with Google Earth application, E-book applications and media player application implemented in Windows PC environment. Without any instruction, they were asked to manipulate the prototype making arbitrary gestures. Participants were also asked to use mouse or keyboard with interacting same application. From user observation and in-depth interview, we figured out that deformation based gesture interaction helped user pay attention more than conventional input

Design and Implementation of Deformation Based Gesture Interaction

179

device and that they adapted to the new manipulation in a short time without instruction. Proposed interaction features analog interaction, for example, when user bends the prototype more, user feels turning page faster. The analog interaction is difficult to experience in mouse/keyboard, while the analog interaction provided great satisfaction. It was also found that although burden coming from physical manipulation to make appropriate gestures weakened efficiency, it increased joy in feeling varying shape of device according to deformation.

5 Conclusion and Future Works In this study, we proposed deformation based gesture interaction as a means of interacting flexible display device with digital contents. User-preferred interaction was designed based on prior research. We implemented a prototype with practicability and evaluated proposed interaction with user study. It is concluded that proposed interaction increases intuitiveness, joy and concentration and shows the possibilities of means of interacting future flexible display devices. However, new application for proposed interaction is demanded due to the limitations of traditional GUI based application, in which the deformation based gestures were not full of efficiency. High accuracy in recognizing the gestures, integration with real flexible display and an exploratory study on relation between deformation based gesture input and possible additional input are also our future works. Acknowledgments. This work was supported by the Industrial Strategic technology development program, KI002156 funded by the Ministry of Knowledge Economy(MKE, Korea).

References 1. Holman, D., Vertegaal, R.: Organic user interfaces: designing computers in any way, shape, or form. Commun. ACM 51, 48–55 (2008) 2. Schwesig, C., Poupyrev, I., Mori, E.: Gummi: User Interface for Deformable Computers. In: Proc. CHI 2003, pp. 954–955. ACM Press, New York (2003) 3. Holman, D., Vertegaal, R., Altossar, M., Troje, N., Johns, D.: Paperwindows: interaction techniques for digital paper. In: Proceedings of the CHI 2005, pp. 591–599. ACM Press, New York (2005) 4. Herkenrath, G., Karrer, T., Borchers, J.: Twend: twisting and bending as new interaction gesture in mobile devices. In: CHI 2008, pp. 3819–3824. ACM Press, New York (2008) 5. Lee, S., Kim, S., Jin, B., Choi, E., Kim, B., Jia, X., Kim, D., Lee, K.: How Users Manipulate Deformable Displays as Input Devices. In: CHI 2010, pp. 1647–1656. ACM Press, New York (2010) 6. Wobbrock, J.O., Aung, H.H., Rothrock, B., Myers, B.A.: Maximizing the Guessability of Symbolic Input. In: CHI 2005, pp. 1869–1872. ACM Press, New York (2005) 7. Perriollat, M., Bartoli, A.: A Quasi-Minimal Model for Paper-Like Surfaces. In: IEEE Computer Vision and Pattern Recognition, pp. 1–7. IEEE Press, Los Alamitos (2007)

The Expansibility of User Interfaces Using Peripheral Multisensory Stimulation Ju-Hwan Lee Department of Newmedia, Korean German Institute of Technology, Seoul, Korea [email protected]

Abstract. The present study has explored the expansibility of using everyday objects for game interface to provide additional multisensory stimuli. The perceived expansibility of game interfaces was investigated by measuring user various responses such as the degree of presence, immersion, and enjoyment. Offering actual sensations assumed in the virtual environment of game could enhance user experience by effectively accessing their emotions. Participants were provided with one of four output modes: basic condition, room light stimulation, chair vibration stimulation, & both light and vibration. Consequently, tactile stimulation has resulted in higher presence and immersion than visual did. The result also suggests that a gamer could experience the expanded interfaces with additional multisensory output, bringing more enjoyment. These tendencies are especially recognized for high immersive tendency group. These findings can be applied to various fields of game interface design. Keywords: Multisensory, Multimodal, Computer Game, User Interface, Expanded Game Interface.

1 Introduction A growing trend in game interface design is using multimodal input and output user interfaces. Previously, much of efforts to improve the game interface are centered on high fidelity graphics and sound, but the success of console game platforms has suggested other alternatives, for example an input device affording natural gesture and movement. It is presumed that a player feels more engaged and immersed in the game by evoking their various sensations beyond merely processing visual information. Presence is defined as the subjective experience of being in one place or environment even when one is physically situated in another [1]. We empirically investigated the expansibility of perceived user interfaces by adopting presence that defines epistemological dimension regarding spatial perception. Immersion, often synonymous to presence, refers to the sensation of being surrounded by a completely other reality. According to previous game studies, immersion is measured by the absence of awareness of real environment along with complete immersion in virtual game environment [2, 3]. Therefore, there are always possibilities that additional stimuli from peripheral space exterior game interface might work as a hindrance to gamer immersion in the game. However, research applying multisensory stimulation C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 180–183, 2011. © Springer-Verlag Berlin Heidelberg 2011

The Expansibility of User Interfaces Using Peripheral Multisensory Stimulation

181

to game interface has displayed that a social interaction activated by input devices supporting natural movement has enhanced immersion [4]. The result indicates that virtual and real environments are not an exclusive dimension, offering a gamer an expanded experience by a specific environmental connection. There are two common methods to improve gamers’ experience by displaying multi-sensory information in game interface. The first is to offer sensory stimuli similar to those simulated in the virtual environment (i.e. the scent of a flower given in the scene of full bloom flowers in 4D movie theatre) [5]. Displaying useful information such as remaining energy level, ammunition is second approach in which gamers benefit in processing information valuable to their mission more efficiently, increasing their motivation and immersion to the game [6]. These approaches could improve gamers’ enjoyment and satisfaction by accessing their emotion along with multisensory experience. Based on theoretical backgrounds, we have explored the possibility of using everyday objects for game interface to provide additional visual and tactile stimuli. The perceived expansibility of game interface was emphatically investigated by measuring user’s subjective responses such as presence, immersion, and enjoyment. We also measured the task performance since the potential relationship between presence and task performance is arguable whether performance is benefited or deteriorated by the enhanced presence. Offering actual sensations assumed in the virtual environment of game could enhance users’ experience by effectively accessing their emotions. We measured participants’ immersive tendency with using Immersive Tendency Questionnaire [1], since individual differences also play a critical role in experiencing presence along with technological components of media.

2 Experiment 2.1

Methods

Participants. 12 Participants (male: 8; female: 4, average age = 24.4) took part in this experiment, after administrating Immersive Tendency Questionnaire to measure their immersive tendency in virtual environment or game. Materials. A simple shooting game was developed and connected to Arduino board in order to synchronize events in game with external visual and tactile stimuli additionally displayed. In the event of blast of an enemy, tactile feedback was provided by vibrator installed on their seats for tactile stimuli and a flicker lasting a second by a light bulb for visual stimuli in the event of blast in the game. Procedure. Participants played the game after instruction of game. Shooting down enemies and avoiding crush with them simultaneously was the key of the game. Performance was measured by game score which a gamer gain from bringing down enemies during playing time. After the session, the questionnaire regarding immersion, presence, and enjoyment was administered for their subjective evaluation. 2.2 Results The participants’ subjective rating data were analyzed by a series of repeated measured analyses of variance (ANOVAs) with four levels of game output interface

182

J.-H. Lee

modes as an independent variable: basic, visual stimuli (light bulb), tactile stimuli (vibration of chair), and both tactile & visual stimuli(vibration plus light), with comparing low immersive tendency group with high immersive tendency group. The main effect of four output interface modes in the score of performance, immersion, presence, and enjoyment were not significant, but immersion [performance: F(3,11)=1.11, p>.05; immersion: F(3,11)=3.48, p<.05; presence: F(3,11)=1.51, p>.05; enjoyment: F(3,11)=1.63, p>.05]. However, on the inside of these results, the discrepancy between basic and tactile condition and the one between basic and simultaneous tactile and visual condition in immersion are the most significant [basic and tactile stimuli: F(1,11)=7.61, p<.05; basic and both visual & tactile stimuli: F(1,11)=5.04, p<.05]. Tactile stimuli have resulted in higher presence and immersion than visual stimuli did. The strong response toward tactile stimuli rather than basic condition was consistent with the result of the one in presence [tactile stimuli: F(1,11)=7.74, p<.05], supporting the existing research that immersion and presence are positively related. High immersive tendency group was likely to feel more immersed, presence, enjoyment to different levels of output interface modes, while

Fig. 1. A) Four levels of game output interfaces, B) An illustration of the experimental set-up used in this study

Fig. 2. The mean scores of participants’ immersion, presence, and enjoyment responses as a function of four output interface modes and immersive tendency groups

The Expansibility of User Interfaces Using Peripheral Multisensory Stimulation

183

low immersive tendency group showing less difference among different levels [immersion: F(1,10)=9.39, p<.05; presence: F(1,10)=2.00, p=1.87; enjoyment: F(1,10)=5.00, p<.05]. Especially, high immersive tendency group displayed the increase of enjoyment for visual condition and both visual & tactile stimuli condition than low immersive tendency group.

3 Discussion The result suggests that a gamer could experience the expanded interface with additional multisensory output, bringing more enjoyment. These tendencies are specially recognized for the higher immersive tendency group. The finding can be applied to various game interface designs. Additional sensory cues, for example, can be used to assist players or even distract them to increase difficulty level. Future studies are expected in relation to different game interface designs and genres.

References 1. Witmer, B.G., Singer, M.J.: Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence 7(3), 225–240 (1998) 2. Reeves, B., Nass, C.: The Media Equation. CSLI Publications/Cambridge University Press, Stanford (1996) 3. Roskos-Ewoldsen, D.R., Roskos-Ewoldsen, B., Dillman Carpentier, F.: Media Priming: An updated synthesis. In: Media Effects: Advances in theory and research, pp. 74–93. Routledge, London (2008) 4. Lindley, S.E., Le Couteur, J., Berthouze, N.L.: Stirring up Experience through Movement in Game Play: Effects on Engagement and Social Behavior. In: CHI 2008, Florence, Italy, pp. 511–514 (2008) 5. Riva, G., Davide, F., IJsselsteijn, W.: Being There: Concepts, effects and measurements of user presence in synthetic environments. IOS Press, Amsterdam (2003) 6. Nesbitt, K.V., Hoskens, I.: Multi-Sensory Game Interface improves Player Satisfaction but not Performance. In: 9th AUIC 2008, Wollongong, Australia, pp. 13–18 (2008)

Use of Hands-Free Mouse for Game Control Moyen Mohammad Mustaquim Department of Informatics and Media Uppsala Universiy, Uppsala, Sweden [email protected] Abstract. Interaction using gaze is not a new concept. It has been ready to deliver the promises to many people who suffer from severe disability and the only means of communication for them is eye or some muscle movement. The ideas proposed here construct a system that can collect gaze data, studies the oculomotor functions and capture natural human behavior. The hands-free mouse concept here is based on this idea which will provide the user to move the cursor and select object using eye, or any other muscular movement for the operation of game control. The concept of game in this study is, a tool that can help users with special needs doing exercise in an informal way. This can be helpful for patients who are under physiotherapy. A game interface can be used to show progresses of user’s muscle movement using scanning technique challenging the users for advancing certain level while exercising parallel. Keywords: Gaze, Eye Tracking, Inclusive Design, Special Need, Game.

1 Introduction Design for all or universal design is a focused topic in the world of designer, engineer and industry. Computer games which are not considered anymore as an entertainment issue for children or young adults are also not beyond the thought of universal design. Achieving inclusivity in computer games design is a difficult job. The principal thought that comes to general people’s mind when talking about inclusive game is that a game that gives accessibility to people with special needs. Although this is or will not be the primary focus of inclusive games design, it is and will be one of the major issue of all time- how can we design game that is accessible for everyone regardless of their age or physical disabilities. Using human modalities, the idea of game control for special need people is a vital topic of research going on in human computer interaction field. As the problem sometimes is that not everyone is able to or not comfortable enough to use their hand for using traditional input device in computer such as keyboard and mouse, the use of other inputs for controlling action came to the discussion in researchers mind. Computer games is not an easy thing to control either for people with such limitations and special needs. However to include them in the service like games it is important to consider multiple input controlling methods therefore. 1.1 Alternative Input in Computer Games A game controller is a device used for controlling the playable character or objects or otherwise providing input in a computer game. The controller is usually connected to C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 184–187, 2011. © Springer-Verlag Berlin Heidelberg 2011

Use of Hands-Free Mouse for Game Control

185

a game console or a computer by means of a wire, chord and also by means of wireless communication. Typical game controllers are keyboards, mice, joysticks, game pads etc. The principal function of a game controller is to control the movement or actions of a playable body-object or otherwise influence the events in a video- or computer game. However, certain groups of people have limited accessibility or no accessibility at all to such game controllers because of their physical limitations. Alternative ideas for game control are therefore important to consider for such groups of people. Speech is a common approach of controlling game. Using user emotion to control game action is dependent on detecting emotion in user’s speech command which is vital for disable people. Another approach is gaze based input. As mentioned earlier it is quite expensive as this system uses computer vision and cameras. Use of bio-signal interfaces such as brain computer interface which is based on EEG signals is an alternative but it is sensitive to noise hence not a very good idea to use for supporting people with disabilities for controlling game which is quite receptive issue for them. 1.2 Theoretical Concept of Hands-Free Mouse The demonstrated system called HaMCoS in [1] and the evaluation of the system in a case study in [2] is the motivation of this paper’s proposal of a hand free mouse system for controlling game. The system proposed in [1] uses Electromyogram or EMG signals that is originating from the muscle of the user. Intentional muscle contractions are detected depending on whether or not the intensity of the input signal exceeds a certain threshold. It then decides on the instant the mouse cursor starts moving, changes direction or stops[1]. The system responds by detecting a contraction event and the muscular activity in our case for controlling game is the brow muscle, although the muscle of user’s choice can be used for detecting contraction. A sensor can be placed on top of forehead of the user using a temporary adhesive for instance. One sensor is enough for the purpose as the sensor is used to detect one muscular movement only. However the idea of small muscle contraction is considered to be a single click of mouse whereas the double contraction is comparable to double mouse clicks. This is how the sensor in the proposed system detects the signal from the user sensor. 1.3 Game-As a Tool for People with Special Needs Using gaze technology people with special needs can be highly benefited if customized system can be fashioned for them to use the gaze technology as an input method. Game as mentioned earlier is not only for entertainment anymore, but the concept of game is viewed in a different point now a days. Using the concept of HaMCoS [1] if the sensor placed on someone’s muscle can detect muscular movement and send a signal to the computer then this input signal can be used to control a graphical user interface in an interactive way. While designed properly, this can be a motivational tool for special need people to accomplish different tasks; for

186

M.M. Mustaquim

instance ‘physiotherapy in an entertaining way’ without the help of a physiotherapist at certain cases. This leads less physical meeting between patient and physiotherapist which can save money for the patient. The following section describes the method and proposed model for this idea.

2 Method The method for creating such facility for ‘special needs’ group of people has two segments. Creating an entertaining game where user activities are stored as data is number one. While the use of sensor in a proper way in subject’s area of interest in body part where they want to do exercise, is number two. An EOG, Electrooculography (depending on eye movements), sensor can be placed on forehead or near eyebrow or some specific area of the face where the user want to train themselves to recover from paralysis situation due to stroke or other neurological issues. EOG is actually a hybrid approach of EMG. For exercising hands there can be different options. Use of Electromyogram (EMG) sensor on top of hand can be a simple approach. However the Gyroscope and Accelerometer can also be used in this case. Depending on the level of accuracy the sensor for hand can be chosen. However, EOG and EMG is the focused discussion in this research as they both detect muscular movement, hence the proposed model talks about EMG method to imply the sensor for receiving data from user’s muscle movement from upper or lower limb and triggers the GUI of the game. 2.1 Proposed Model For facial movement detection the EOG sensor is placed on the forehead as shown on figure1. To make a feasible use of the sensor on the facial and head area, two things were considered. Gaze accuracy describes the angular average distance from the actual gaze point to the one measured by the eye tracker. Gaze precision describes the spatial variation between individual gaze samples. Gaze accuracy and gaze precision are typically measured in degrees of visual angle. While Gyroscope is the best method to detect motion from upper or lower limb (hand for example), EMG sensor is still chosen for few reasons. EMG is not influenced by gravity and it’s sensitive to remote muscle activity and interference. Intimate contact with subject is needed for EMG sensors and small sensing elements are available with no influence on motion. Figure 2 shows an EMG sensor placed on subject’s hand while Figure 3 shows the block diagram of the complete system.

Fig. 1. A sample sensor placed on user’s forehead

Fig. 2. An EMG sensor placed on user’s hand

Use of Hands-Free Mouse for Game Control EOG SIGNALS

USER USER

187

EMG AND/OR EOG SENSOR

GUI Of GAMES EMG SIGNALS

Fig. 3. Block diagram of the proposed system

3 Result and Discussion The type of proposed hands-free system for specific purpose, for controlling game should be fast, efficient, and reliable. Gaze interaction was found to be slower than mouse/hand interaction and more erroneous than head interaction during some usability tests. Usability evaluations of gaze systems should not focus only on efficiency, however. High availability of system components and multimodal input options are also important factors contributing to effectiveness use of the systems. Proper design of an interactive user interface that gives the user a kind of feelings of competing on a game environment by rising psychological challenges to achieve a certain success level, can thus be an useful idea for many people who are in need of physiotherapy. Further research, usability testing by real users can thus lead this research to be an interesting topic of interest.

References 1. Torsten Felzer, R.N.: How to operate a PC without using the hands. In: Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 198– 199 (2005) 2. Felzer, T., Nordmann, R.: Evaluating the Hands-Free Mouse Control System: An Initial Case Study. In: Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I., et al. (eds.) ICCHP 2008. LNCS, vol. 5105, pp. 1188–1195. Springer, Heidelberg (2008)

An Armband-Type Touch-Free Space Input Device for HCI Dongwan Ryoo and Junseok Park Electronics and Telecommunications Research Institute., 161 Gajeong Dong, Yuseong Gu, Daejeon, Korea {dwryoo,parkjs}@etri.re.kr Abstract. In this paper, the design of an armband-type non-touch space input mouse for human computer interface is presented. Demand for a non-touch type input system has been increasing with the development of large-display technology. One of the non-touch type input system is a system based on gesture recognition. There is a lot of research on gesture recognition systems based on vision, but the presented device has advantages in terms of accuracy. The globe type input system is already developed, but the users do not want restrictions on finger actions. Unlike a glove type device, an armband device type does not constrain finger movements. We designed an armband- type space input mouse by using arm movement and wrist gesture actions. Keywords: Armband, Non-touch, HCI, 3D space.

1 Introduction Interaction technologies between a computer and a user are changing not only from interactions based on a traditional keyboard and mouse in the desktop environment, to the interactions based on a user's natural behavior in a three-dimensional input space, but also from interface based on audio-visual displays to realistic interfaces by using human touch senses. A variety of research for input device is being conducted [1]-[4]. Demand for a non-touch type input system has been increasing with the development of large-display technology. One of the non-touch type input system is a system based on gesture recognition. There is a lot of research on gesture recognition systems based on vision, but the presented device has advantages in terms of accuracy. Users would like to control devices by using small motions. The globe type input system is already developed, but the users do not want restrictions on finger actions. Unlike a glove type device, an armband device type does not constrain finger movements. In this paper, the design of an armband-type non-contact space input for human computer interface is presented. An armband-type non-touch 3D space input mouse is I/O interface technology for manipulating various devices using a user's arm/wrist gestures. The user wears the device on the wrist in contactless 3D space environment, such as a large smart interactive TV or HMD. We designed an armband-type space input mouse by using arm movement and wrist gestures. The practical considerations of an armband-type non-touch space input system are the contactless interface based on multi-sensor; the natural behavior of users such as arm, and wrist motion recognition; and convenience, such as small size unrestricted motions. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 188–192, 2011. © Springer-Verlag Berlin Heidelberg 2011

An Armband-Type Touch-Free Space Input Device for HCI

189

2 Design of an Arm Band-Type Space Input Mouse We designed an armband-type space input mouse that uses arm movements and wrist gestures. Unlike a glove type device, the armband device type does not constraint on the finger movements. Therefore, wearing the armband, users can type on a keyboard and wear gloves. This type is a non-touch space input mouse that uses arm movements and wrist gestures(bending actions), as consisting of gyro sensors, accelerometers and proximity sensor array. The point of the mouse can be moved by recognizing arm movements from data processing of the IMU sensors, such as gyro sensors, and accelerometers. From data processing the proximity sensors array, a user can right/left-click input with wrist bending gestures.

Fig. 1. Non-touch 3D space input mouse: Armband-type

This space input system can also be used by a surgeon wearing surgical gloves to see MRI images etc. In other words, this device can also be used as a space input mouse in areas with special environmental constraints. It can be used by disabled people with impaired fingers, as well as for game, and portable handheld information devices. Like a PC mouse, this armband type space Input mouse also has six kinds of basic functions such as up, down, left, and right move and left click, and right click. Figure 1 shows how to click by using the non-touch 3D space input mouse(armbandtype). This armband-type space input system consists of IMU sensors to recognize arm gestures, and proximity sensors array to recognize wrist bending gesture.

3 Implementation and Experimental Results 3.1 Non-touch 3D Space Input Mouse Prototype The armband-type non-touch 3D space input mouse is I/O interface technology for manipulating various devices using a user's arm and wrist gestures with wearing on

190

D. Ryoo and J. Park

the wrist in contactless 3D space environment, such as large displays or HMDs. The focus of the second type of mouse prototype is on improving accuracy. And Haptic feedback was also considered. In addition, by using the wrist gestures, this system is available to some disabled people with impaired finger. The non-touch 3D space input mouse board(armband-type) and the USB type receiver dongle for wireless communication are shown in figure 2. The USB type receiver dongle is for IEEE 802.15.4 wireless communication and support of HID. The system consists of an MCU with WPAN, IMU sensors, a proximity sensor array, and a vibration motor. The two gyro sensors and accelerometer are used to move the pointer. The proximity sensors array is used to recognize wrist gestures. In this system, infrared-type and capacitance-type proximity sensors are used respectively. The accuracy of left/right clicking using wrist gesture was very good. The capacitance type and infrared type sensors had almost the same accuracy. The mouse can be used for mapping between instructions and various wrist gestures.

(a)

(b)

Fig. 2. (a)Non-touch 3D space input mouse board: Armband-type. (b) The USB type receiver dongle for wireless communication.

We adapt IEEE 802.15.4 as the wireless network part of this system that provides a low powered wireless data transmission. This wireless network communication based on IEEE 802.15.4 can be useful in a multi-pointer. And the vibration motor is used for realistic vibration feedback. After recognizing of wrist gestures, this system provides vibration feedback for users. Figure 3 shows the non-touch 3D space input mouse prototype for armband-type. The figure also shows the location of the proximity sensor array. To introduce the hands-free concept, the tilt information of the accelerometer sensor was used. Depending on the slope(roll) of the wrist, we divide pointer states into two classes: a mouse point active state and a halt state. In an active state, the mouse pointer can be moved freely. On the other hand, in a halt state, the mouse pointer is halted for hands-free of users. This is to prevent the unintended input as a result of the user's hand gestures. This method provides more convenience to users.

An Armband-Type Touch-Free Space Input Device for HCI

191

Fig. 3. Non-touch 3D space input mouse prototype: Armband-type

This function has the concept of calibration of the pointer and hands-free. Table 1 shows the main feature and specifications of the non-touch 3D space input mouse prototype for armband-type. Table 1. Main features and specifications of the Non-touch 3D space input mouse prototype: Armband-Type

Main features and Specifications - IMU(gyro sensors and accelerometers)-based pointer manipulation : data range:-300 ~ + 300 deg/sec, resolution: 0.1 deg/sec, support of Projected X, Y - Improve accuracy : the wrist gesture recognition using a proximity sensor array(left/right click, scroll, etc) - Accuracy (left / right click) : 99 % - Vibration for realistic feedback - Low power wireless communication based on WPAN(802.15.4) - Compatibility : support of human interface device spec. (HID standard protocol)

3.2 Signal Processing for the Armband Space Input Mouse The signal processing for the armband-type space input mouse are as follows. − FIR filtering, Normalize , Feature extraction, Power extraction − Moving average Windowing Processing, Adaptive threshold − Silence period for robust, Rule based reasoning.

4 Applications of the Armband-Type Space Input Mouse The armbands system can be used in various applications. The applications of the armband space input mouse are as follows. − − − − −

An entertainment system(Games/Simulation/Smart TV) A virtual control in VR, AR(3D UI control) A space mouse An HCI for a surgeon wearing surgical gloves and HMDs. An input device for disabled people with impaired finger

192

D. Ryoo and J. Park

The applications of the armband space input mouse are shown in figure 4 . We expect this input system to be used as one of the HCI systems. The applications of the armband space input mouse are shown in figure 4 . We expect this input system to be used as one of the HCI systems.

Fig. 4. Applications of The armband-type space input mouse

The armband-type space input mouse type is used in entertainments(games). It can also be used for smart TV or IPTV. The armband-type space input mouse was tested for effectiveness by applying for many applications.

5 Conclusion In this paper, the design of an armband-type non-touch space input for humancomputer interface is presented. The armband-type space input system was implemented for a feasibility test. And the system was tested for effectiveness. The applications of the armband space input system are presented. The extended gesture functions are tasks for the feature. Acknowledgments. This work was supported by IT R&D Program of MKE/KEIT [KI002096, Contact-free Multipoint Realistic Interaction Technology Development].

References 1. Nattapong, T., Natthapol, W.: A Low-Cost data Glove for Human Computer Interaction Based on Ink-jet Printed Sensors and ZigBee Networks. In: ISWC (2010) 2. Ryoo, D., Park, J.: Design of an Armband Type Contact-Free Space Input Device for Human-Machine Interface. In: International Conference on Consumer Electronics, ICCE (2011) 3. Chen, X., Zhang, et al.: Multiple Hand Gesture Recognition based on Surface EMG Signal. In: Proceedings of the International Conference on Bioinformatics and Biomedical Engineering, pp. 516–519 (2007) 4. Na, J., Choi, W., Lee, D.: Design and Implementation of a Multimodal Input Device Using a Web Camera. ETRI Journal 30(4) (August 2008)

Modeling of Purchasing Behavior for Application on Merchandise Display and Package Design Kotaro Suzuki1, Nobuyuki Nishiuchi1, and Mi Kyong Park2 1

Department of Management Systems Engineering, Tokyo Metropolitan University 2 Division of Management Systems Engineering, Tokyo Metropolitan University 6-6 Asahigaoka, Hino, Tokyo, 191-0065, Japan

Abstract. In this study, we modeled customer purchasing behavior in a retail store using a Bayesian network-based probabilistic modeling tool to apply towards improving merchandise display and package design. The following four aspects of purchasing behavior were examined: purchasing gestures, glances made during the selection of merchandise, customer psychology concerning purchasing, and body information, including age, height, gender, eyesight, and dominant hand. To monitor purchasing behavior, three college students fitted with an eye tracker were asked to select the most appealing product among twenty-five snack food products that were displayed on five-tier shelving. Factors affecting customers’ merchandise selection in the experimental retail store were then computed using a Bayesian network, and a purchasing behavior model was constructed from the results. Using the constructed model, a method of effective merchandise display based on the characteristics of retail purchasing behavior was proposed. Keywords: Purchasing Behavior Model, Bayesian Networks, Retail Store, Eye Tracking.

1 Introduction Service industries now account for approximately 70% of the gross domestic product (GDP) in Japan [1]. However, decreased sales in the retail industry have recently been acknowledged as a major problem in Japan and have prompted various approaches for stimulating economic recovery. Among them, proactive attempts to introduce engineering techniques into service industries have attracted attention because these industries have typically relied on intuitive approaches without exact analysis. It is known that approximately 70% of retail store customers make purchasing decisions after they enter the store. This means that effective merchandise display and package design can encourage customers to purchase merchandise; however, the concrete details of the purchasing decision process are not well understood. Therefore, to gain insight into this process and allow for more effective merchandise display, it is necessary to model purchasing behavior based on a series of actions, such as gazing at merchandise, handling merchandise, and making a final purchasing decision. Here, we modeled customer purchasing behavior in a simulated retail store using a Bayesian network-based probabilistic modeling tool to apply towards improving merchandise display and package design. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 193–196, 2011. © Springer-Verlag Berlin Heidelberg 2011

194

K. Suzuki, N. Nishiuchi, and M.K. Park

2 Methods 2.1 Experimental Design The following four aspects of customers' purchasing behavior were examined: purchasing gestures, glances made during the selection of merchandise, customer psychology concerning purchasing, and body information. First, to extract psychological information concerning purchasing, five college students were interviewed about purchasing merchandise using the evaluation grid method. The evaluation grid method, which is an interview method to reveal perceived attributes of respondents and assess their value system, has been applied towards merchandise development and market surveys. Based on the results of our interviews, “situation” was extracted as a letter-up item and the eight items of “flavor”, “texture”, “volume”, “price”, “previous knowledge of merchandise”, “package color”, “package graphics”, and “brand name” were extracted as letter-down items, and a questionnaire was then created, which was administered to students following product selection. To monitor purchasing behavior, three college students were asked to select the most appealing product among 25 snack food products that were displayed on fivetier shelving (Figure 1). The shelving was plain white in color, 90 cm in width and 130 cm in height, with five tiers of 25, 30, 40, and 45 cm in depth from the top to the bottom tiers, respectively. Participants were instructed to select a product while standing within 100 cm of the shelving. The participants’ purchasing behavior was recorded using a video camera that was placed 2 m from the participants, and an eye tracker (NAC, EMR-8) was used to record eye-gaze activity. There was no time limit for the user to select the most appealing product on the shelving. Following product selection, the participants were instructed to complete the questionnaire, which was created based on the interview results, and also included questions related to the participants' body information, such as age, height, gender, eyesight, dominant hand, and elapsed period since eating. 2.2 Bayesian Network A Bayesian network is a compression of the joint probability distribution (JPD) of a set of random variables and has been applied to create consistent probabilistic representations of uncertain knowledge in diverse fields, including medical diagnosis, fault diagnosis, and search algorithms. In this study, a Bayesian network was used to model customer purchasing behavior in a simulated retail store. Model construction was automatically performed using a Bayesian network with a Greedy search algorithm from measured data, and the fitness of a structure was measured by Akaike’s Information Criteria (AIC) [2], [3].

3 Results and Discussion Here, we conducted an experimental study to model customer purchasing behavior in a retail store using a Bayesian network to apply towards improving merchandise

Modeling of Purchasing Behavior for Application on Merchandise Display

195

display and package design. Factors affecting customers’ merchandise selection in the experimental retail store were computed using a Bayesian network-based probability method, and a purchasing behavior model was constructed from the results. Figures 2 and 3 illustrate two examples of the constructed purchasing behavior model. In example 1 of the purchasing behavior model (Figure 2), an association exists between the following five items: "Participant gazed at position D5", "Participant gazed at position C4", "Participant had eaten it before", "Participant desires to eat the product alone", and "What did the participant choose?". As a result of performing probabilistic inference, if all five of the conditions are satisfied, the probability that the participant will choose either B5, A2, or E3 is 40%, 40%, and 20%, respectively. In the second example of the purchasing behavior model (Figure 3), an association between the following seven items: "Participant gazed at position D2", "Participant gazed at position D1", "Participant gazed at position E1", "Participant gazed at position D4", "Participant gazed at position E5", "Participant gazed at position D3", and "Participant desires to eat the product immediately" was identified. After performing probabilistic inference, if the participant gazes at position D1 or D2, the probability that he (or she) will choose D3 or D4 is only 25%, while that of E1 or E5 is 75%. In the retail setting, several related products of the identical brand are typically displayed on the same tier of merchandise shelving. However, according to our findings of customers' eye-gaze activity, lining up several related products in a horizontal row is not effective for promoting the selection of a target brand. Our findings indicate that when merchandise is displayed on shelves at positions D1 or D2, it would be better to display related products at positions E1 or E5.

Fig. 1. Front view image (left) and the position number of products (right) on the merchandise display shelve

196

K. Suzuki, N. Nishiuchi, and M.K. Park

Participant gazed at position C4

Participant gazed at position D5

Participant had eaten it before Participant desires to eat the product alone

What did the participant choose?

Fig. 2. Example 1 of the purchasing behavior model

Participant gazed at position E1

Participant gazed at position D2

Participant gazed at position D1

Participant desires to eat the product immediately

Participant gazed at position D4

Participant gazed at position E5

Participant gazed at position D3

Fig. 3. Example 2 of the purchasing behavior model

From the present findings using the constructed model, we have proposed a method of effective merchandise display based on the characteristics of retail purchasing behavior. Our findings suggest that the modeling of purchasing behavior may provide a flexible strategy for targeting specific customers based on characteristics such as race, gender, and age, and allow optimization of the design of merchandise display shelves. To build a more realistic model of purchasing behavior, it is necessary to increase the number of participants in future experiments. In addition, we are examining if the seven-step model of the purchasing decision process that has been widely applied in the marketing field can be used as each node of a Bayesian network to understand consumer psychology concerning purchasing.

References 1. Ministry of Economy, Trade and Industry, Keizaisennryakutaikou, http://www.meti.go.jp 2. Ko, N. (ed.): Introductory of Service Science. University of Tokyo Press, Japan (2009) 3. Hirotug, A., Shunichi, A., Gensiro, K., Yosiyuki, K., Hidetosi, S.: Akaike’s Information Criterion AIC. Kyoritsu Pub., Japan (2007)

Decoding of Hand Shapes Based on ElectroMyoGraphic Signals during Playing Guitar Chords Hideaki Touyama and Masafumi Mizuguchi Toyama Prefectural University 5180 Kurokawa, Imizu, Toyama 939-0398, Japan [email protected]

Abstract. In this paper, aiming to study on a new entertainment direction, we investigated the decoding performance of complicated hand shapes from ElectroMyoGraphic (EMG) signals during playing guitar chords. Three healthy right-handed subjects participated in this experiment. During the experiment, they played four guitar chords (The major chords of ‘C’, ‘F’, ‘G’, and ‘A’) as well as being relaxed. The EMG signals were recorded from the left forearm by using 12 surface EMG electrodes. By using Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), the decoding performances could be more than 95% for all subjects. Our results will encourage the development of surface EMG based entertainment systems. Keywords: ElectroMyoGram (EMG), Guitar Chords, Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Entertainment.

1 Introduction In recent years, the useful applications based on ElectroMyoGraphic (EMG) signals [1] have been more and more studied. The amplitudes of the surface EMG signals during typical physical movements amount to 0.1-1.0 mV, and thus it is comparably easy to detect even with a simple measurement device. For example, the EMG system was implemented to videogames and it was one of the promising interfaces for entertainment and biomedical engineering [2,3,4]. In this paper, aiming to study on a new entertainment direction, we investigated the decoding performance of complicated hand shapes from EMG signals during playing guitar chords. This paper is organized as follows: the next section describes the EMG recording experiments and signal processing we have conducted. The third section presents the results of EMG classification. The last section presents discussion and conclusion. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 197–200, 2011. © Springer-Verlag Berlin Heidelberg 2011

198

H. Touyama and M. Mizuguchi

2 Experiments Three healthy right-handed subjects participated in this experiment. During the experiment, they played four guitar chords (The major chords of ‘C’, ‘F’, ‘G’, ‘A’) as well as being relaxed. Note that two of the subjects were expert and one has experienced a little in playing the guitar. During the experiment, the visual cues were presented on a computer display in order to instruct the tasks which the subject should perform (see guitar chord diagrams in upper figures in Figure 1). According to the visual instruction, the subjects performed the guitar playing with complicated hand shapes shown in lower figures in Figure 1.

Fig. 1. Guitar chord diagrams and corresponding hand shapes

The EMG signals were recorded from the left forearm of each of three healthy subjects (S1-S3) by using 12 surface EMG electrodes. The analogue EMG signals were amplified at a multi-channel bio-signal amplifier (Polymate II AP216, TEAC Corp. Japan). For future studies on real-time classifications, the amplified signals were sampled at 200 Hz, which is lower sampling frequency than those (500-1000Hz) in previous works. After full-wave rectification and smoothing the rectified EMG signals by using Butterworth filter, the signal processing of Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) was applied in order to evaluate the decoding performance using single-trial EMG samples.

3 Results We could detect clear EMG waveforms during playing guitar chords. Examples of the 12-channel EMG signals are shown in Figure 2. And furthermore, the decoding performances could be more than 95% for all subjects (S1:97.5%, S2:98.3%, S3:97.9%) as shown in Figure 3. The average decoding performance was 97.9%.

Decoding of Hand Shapes Based on ElectroMyoGraphic Signals

Fig. 2. Examples of 12-channel EMG waveforms during playing four guitar chords

199

200

H. Touyama and M. Mizuguchi

1 0.9 )%0.8 ( e0.7 c n a0.6 m ro fr0.5 e p gn0.4 id o0.3 c e d0.2 0.1 0

1

2 subject

3

Fig. 3. Decoding performances

4 Discussion and Conclusion The decoding performance was more than 95% and with this preliminary result the users of our EMG interfacing system would be able to play the virtual guitar using surface EMG signals. However, the number of guitar chords used in this study was limited (four major chords and relax hand shape). Thus, we are planning to investigate the other major chords (‘B’, ‘D’ and ‘E’) as well as minor guitar chords (‘Am’, ‘Bm’ ‘Cm’ ‘Dm’ ‘Em’ ‘Fm’ and ‘Gm’). Furthermore, for future online application it is required to realize real-time classifications during playing guitar chords [5]. Finally, our results will encourage the development of surface EMG based entertainment systems.

References 1. Kamen, G.: Electromyographic Kinesiology. In: Robertson, D.G.E., et al. (eds.) Research Methods in Biomechanics, Human Kinetics Publ., Champaign (2004) 2. Saponas, T.S., Tan, D.S., Morris, D., Balakrishnan, R., Turner, J., Landay, J.A.: Enabling Always-Available Input with Muscle-Computer Interfaces. In: Proceedings ACM Symposium on User Interface Software and Technology, UIST 2009 (2009) 3. Suzuki, K., Mito, G., Kawamoto, H., Hasegawa, Y., Sankai, Y.: Intention-Based Walking Support for Paraplegia Patients with Robot Suit HAL. Advanced Robotics 21(12), 1441– 1469 (2007) 4. Shin, D., Katayama, A., Kim, K., Kambara, H., Sato, M., Koike, Y.: Using a myoKinetic synthesizer to control of virtual instruments. In: Pan, Z., Cheok, D.A.D., Haller, M., Lau, R., Saito, H., Liang, R. (eds.) ICAT 2006. LNCS, vol. 4282, pp. 1233–1242. Springer, Heidelberg (2006) 5. Mizuguchi, M., Touyama, H. (in preparation)

Exploring Whole-Hand Gestures in a Tabletop Environment for Urban Modeling Peter Vandoren, Karel Frederix, Karin Coninx, and Frank Van Reeth Hasselt University – tUL – IBBT, Expertise Centre for Digital Media, Wetenschapspark 2, 3590 Diepenbeek, Belgium {firstname.lastname}@uhasselt.be

Abstract. Touch-based tabletop interaction with virtual environments (VEs) is a recent research interest. In particular, 3D navigation and object interaction on tabletops pose considerable research challenges given the flat, rectangular twodimensional workspace of an interactive tabletop. This paper explores the design of whole-hand gestures for urban modeling on an interactive, tilted tabletop system. These touch-based gestures fit a hybrid 2D/3D approach for navigation and object interaction in urban city modeling. In a formal user study the proposed whole-hand gestural interface was compared to a finger-based equivalent. The evaluation results reveal appreciation for aspects of the interaction concept, also illustrating the need for further tuning of the interaction concept and the input tracking. Keywords: interaction techniques, tabletop, gestures, touch-based interaction, navigation, urban modeling, user evaluation.

1 Introduction The emergence of multi-touch surfaces [1, 2, 3] has been quite a boost for research in touch-based interaction in the past decade. In 2007 Grossman [4] presented an overview of research advances in interaction techniques on interactive tabletops, indicating that three-dimensional interaction was largely undeveloped. Several novel techniques for 3D interaction have since been presented, either simulating 3D interaction with finger-based interaction while touching the surface [5, 6, 7, 8] or above surface interaction using vision or additional sensors [9, 10, 11]. Our approach explores interaction on the surface using whole-hand gestures, motivated by the versatility and the expressiveness of the human hand. We developed a set of novel gestures for navigating in a virtual environment and for modeling 3D objects (in our case, buildings in a virtual city). In the next sections we describe the interaction design and elaborate on the results of an initial user study, comparing whole-hand interaction and finger-based interaction.

2 Whole-Hand Gesturing in Urban Modeling The extensive experience of regular computer users with point-oriented, direct interaction techniques in 2D desktop applications generally results in a smooth C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 201–205, 2011. © Springer-Verlag Berlin Heidelberg 2011

202

P. Vandoren et al.

transition towards finger-based interaction on tabletops. Although people are proficient in whole-hand interaction in the real world, this kind of interaction is much less exploited in tabletop environments. Reasons for this absence include tracking issues and a rather large occlusion area, but the lack of suitable interaction metaphors might be one as well. We explored these issues in the TT-GUM (TableTop Gestures for Urban Modeling) platform, developed on an interactive touch-sensitive surface using frustrated total internal reflection (FTIR) [3]. The surface, measuring 80 by 60 centimeter, is tilted 45° to facilitate hand-based gestures. TT-GUM (fig. 1) offers two perspective views on the virtual city, a frontal view and a top view. The user switches to the desired view by touching a virtual button, located on the top right-hand side of the surface. TT-GUM offers gestures for navigation and object modeling in two separate interaction modes in order to reduce the complexity of the gestural interaction.

Fig. 1. The TT-GUM platform

The design of gestures for navigation in the virtual city is inspired by the movements of the Harrier jet aircraft [12], which is also able to perform typical helicopter actions (vertical take-off and hover). The basic posture for navigation is a flat hand that is positioned upright on the surface (fig. 2) and symbolizes the fuselage of the Harrier.

Fig. 2. Navigation a) forward b) turning left c) climbing d) descending

Pushing the hand forward initiates a forward travel of the virtual camera (fig. 2.a). If the hand is positioned in an angle with regard to the straight ahead position, the virtual camera turns in the direction indicated by the hand (fig. 2.b). Changing altitude of the virtual camera is achieved by altering the contact surface of the hand with the table. Lifting the front side of the hand, while keeping the back end on the table, results in a climb (fig. 2.c). Lifting the backside, while the front end of the hand remains on the table, commands the camera to descend (fig. 2.d). The white circle in the center of the surface defines a neutral area for horizontal camera travel, allowing only panning or changing the altitude. The concept for object modeling in TT-GUM is also based on whole-hand interaction. Object are moved (fig. 3.a) by positioning (two or) three fingers (normally

Exploring Whole-Hand Gestures in a Tabletop Environment for Urban Modeling

203

the three central fingers of the hand) on a building and dragging the building to the intended place. Positioning at least four fingers of the hand on or near a building initiates rotation (fig. 3.b). As it turns out that the little finger often lifts from the surface when a rather large rotating move (several tens of degrees) is made, the rotation requires at least a four-finger contact. Especially when using the top view on the virtual city, the similarity between the proposed rotation gesture and rotating a physical object emerges. Scaling a virtual building can be performed in one or two dimensions. The basic idea is that by positioning a flat hand on the edge of a building (fig. 3.c), the hand will stick to that edge as long as it touches the surface; so by moving the hand parallel to the edge, the building is scaled according to the movement. This technique, defined analogous to the “sticky finger” technique by Pierce [13], was extended in a dual dimension scaling gesture. By placing a cornering on the corner of a building (fig. 3.d), that corner sticks to the hand shape and hence enables concurrent scaling in two dimensions.

Fig. 3. Object manipulation: a) moving b) rotating c) scaling d) two-dimensional scaling

3 User Experiment 3.1 Design and Setup We organized a comparative user study, to gain insight in the value of the proposed whole-hand gestures and to receive subjective feedback from the test participants. We defined a finger-based gestural interaction equivalent, being touch-based and using at most 2 fingers to perform navigation or object manipulation, as a basis of comparison. The user population consisted of sixteen persons, all computer scientists, split into 2 groups based on their interaction experience with VEs. We defined a within-subjects test design, counter balanced on the interaction technique. Independent variables were the interaction technique, the presence of the wireframe aura representation around a building and the density of buildings in the virtual city. A test session for an interaction technique comprised a sequence of 15 navigational tasks, followed by three sequences of 20 object docking tasks each for move, rotate and scale operations. We scheduled the test conditions in a Latin square design to neutralize the effects of learning. After the test, the participants were asked to fill out a questionnaire. 3.2 Test Results and Discussion The finger-based interaction performed statistically significant faster than whole-hand interaction (using ANOVA, p<0.05), for navigation as well as object manipulation. With regard to the effect of city density or the use of a wireframe aura representation,

204

P. Vandoren et al.

we can’t report any statistically significant difference in task execution times. The subjective results indicate that the participants are convinced that finger-based interaction is faster and more precise. Finger-based interaction is generally considered to be more intuitive than hand-based interaction. When looking at the user preference for individual operations, 3 out of 16 users preferred travelling using whole-hand navigation, whereas 7 users preferred the panning operation in whole-hand navigation. We found similar results with regard to object manipulation: 3 participants preferred the translation using whole-hand interaction, whereas half of the participants opted for rotation according to whole-hand interaction and a quarter of the test group chose whole-hand interaction for scaling. Eleven participants mentioned some kind of frustration with whole-hand interaction, as the system did not react to their expectations when gesturing. Although the majority of the reported issues were due to incorrect gesturing by the participants, the problems of 4 participants were a consequence of above average differences in the hand shapes of these users and should be solved by updating the hand shape calibration in the touch tracking software. The height of the test person and the arms’ reach is also a matter to take into further consideration: 3 participants, being smaller than 1.7 meter, had some difficulty when scaling the height of a building as their hand was not correctly positioned along the top edge of the building because of insufficient arms’ reach. Several factors, probably in a combination, could be responsible for the considerable difference in task completion time between finger-based navigation and whole-hand navigation. A first reason of this difference is that hand-based navigation is a more continuous action, hence requiring also more consciously calculating, in contrast to the sequence of discrete actions in finger-based navigation. A second factor is the presence of a sideward move in finger-based navigation, which was not available in whole-hand navigation because of the airplane metaphor. An additional element is that hand-based navigation was more difficult to master than finger-based navigation, as substantiated by the larger learning curve of whole-hand navigation over the 15 navigational tasks. About half of the test population caused accidental moves of buildings when using whole-hand manipulation as they “released” the building incorrectly, by lifting the fingers not synchronously from the surface or by slightly dragging across the surface when lifting their hand. This situation will have led to greater attention and caution while performing the tasks using whole-hand interaction, at least with the users that weren’t able to master the gesture completely. Hence one might conclude that wholehand interaction was also more difficult to learn, albeit that 7 test participants managed to rotate buildings on average faster with whole-hand interaction than with finger-based interaction. As the whole-hand gesture for rotation has many similarities with rotating physical 3D objects on a table and as it supports proprioception, many users might have felt quite comfortable with this gesture and might therefore have expressed their preference for this gesture.

4 Conclusions and Future Work This paper presented whole-hand gestures for hybrid 2D/3D navigation and object manipulation in a tabletop environment. A comparative user study showed that, given

Exploring Whole-Hand Gestures in a Tabletop Environment for Urban Modeling

205

the current gesture design and implementation, finger-based interaction maintains the efficiency advantage. Most participants preferred finger-based interaction in general, but up to half of the participants expressed a preference for certain hand-based gestures and in particular for rotation. Besides enhancing the interaction design and the input tracking, we will perform further research on the application context where either finger-based interaction or whole-hand gestures are beneficial. Acknowledgements. The authors would like to thank their former colleagues, Rohita Khatiwada and Chris Raymaekers, for their valuable contributions to this research. Part of this research was performed in the context of the IBBT-project HiMasquerade.

References 1. Dietz, P., Leigh, D.: DiamondTouch: a Multi-user Touch Technology. In: 14th ACM Symposium on User Interface Software and Technology, pp. 219–226. ACM Press, New York (2001) 2. Microsoft Surface, http://www.microsoft.com/surface/ 3. Han, J.Y.: Low-cost Multi-touch Sensing through Frustrated Total Internal Reflection (FTIR). In: 18th ACM Symposium on User Interface Software and Technology, pp. 115– 118. ACM Press, New York (2005) 4. Grossman, T., Wigdor, D.: Going Deeper:a Taxonomy of 3D on the Tabletop. In: IEEE International Workshop on Horizontal Interactive Human-Computer Systems (2007) 5. Hancock, M., Carpendale, S., Cockburn, A.: Shallow-depth 3D Interaction: Design and Evaluation of One-, Two- and Three-touch Techniques. In: ACM Conference on Human Factors in Computing Systems 2007, pp. 1147–1156. ACM Press, New York (2007) 6. Hancock, M., ten Cate, T., Carpendale, S.: Sticky Tools: Full 6 DOF Force-based Interaction for Multi-touch Tables. In: ACM Conference on Interactive TableTops and Surfaces, pp. 145–152. ACM Press, Banff (2009) 7. Kruger, R., Carpendale, S., Scott, S.D., Tang, A.: Fluid Integration of Rotation and Translation. In: ACM Conference on Human Factors in Computing Systems, pp. 601–610. ACM Press, New York (2005) 8. Reisman, J.L., Davidson, P.L., Han, J.Y.: A Screen-space Formulation for 2D and 3D Direct Manipulation. In: 22nd annual ACM symposium on User Interface Software and Technology, pp. 69–78. ACM Press, New York (2009) 9. Hilliges, O., Izadi, S., Wilson, A.D., Hodges, S., Mendoza, A.G., Butz, A.: Interactions in the Air: Adding Further Depth to Interactive Tabletops. In: 22nd ACM Symposium on User Interface Software and Technology, pp. 139–148. ACM Press, New York (2009) 10. Benko, H.: Beyond Flat Surface Computing: Challenges of Depth-aware and Curved Interfaces. In: 17th ACM International Conference on Multimedia, pp. 935–944 (2009) 11. Takeoka, Y., Miyaki, T., Rekimoto, J.: Ztouch: an Infrastructure for 3D Gesture Interaction in the Proximity of Tabletop Surfaces. In: ACM Conference on Interactive Tabletops and Surfaces, pp. 91–94. ACM Press, New York (2010) 12. Harrier, http://www.britannica.com/EBchecked/topic/255852/Harrier 13. Pierce, J.S., Forsberg, A.S., Conway, M.J., Hong, S., Zeleznik, R.C., Mine, M.R.: Image Plane Interaction Techniques in 3D Immersive Environments. In: ACM symposium on Interactive 3D Graphics, pp. 39–44. ACM Press, New York (1997)

Input Interface Using Fingertip Ryo Wada and Tomohiro Hase Ryukoku University Seta, Otsu 520-2194 Shiga, Japan [email protected]

Abstract. This paper proposed an input interface that needs no keypad on small-sized portable equipment such as portable game machines. ON/OFF switches are provided on user's ten fingertips of both hands in our proposed method. Each of these fingertips is provided with one of control signals that work as binary number and so on. The combination of characters onto those fingertips is kept changeable by the user. A prototype was created to verify the proposal and evaluated whether it is effective as a user interface for data entry. This method has a feature that the needed characters can be entered if the user press user’s fingertips to any part of the user's body, for example, without needing any specific key pad on the surface of a small-sized portable terminal. Keywords: Input Interface, Fingertip, Glove, Switch.

1 Introduction Ten keys on the surface have conventionally been required to enter a numeral on a small portable information terminal. Also, we see the plus key, and the A, B, X and Y keys on the surface of portable game machines. More complicated input and more keys are necessary in order to input 26 alphabetical letters. Thus, conventional input methods needs specific key inputs. This paper proposed an input interface that does not require a keypad on the small portable equipment.

2 Proposed System 2.1 Outline of Proposal ON/OFF switches at the fingertips of a hand or both hands are employed in this proposal. The on/off signals are detected as binary system signals and treated as input signals by portable information terminals or personal computers. The binary signals are allocated to binary numbers instead of the ten decimal numbers, and are allocated to control signals instead of the plus key and A, B, X, and Y keys. This easier and less complicated input method is proposed to input 26 alphabetical letters and 50 characters from a Japanese syllabary. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 206–209, 2011. © Springer-Verlag Berlin Heidelberg 2011

Input Interface Using Fingertip

207

In addition, the combination of characters at the fingertips can be changed by users depending on their needs. 2.2 Outline of Experiment A prototype was prepared in order to evaluate the effectiveness of the proposed method from the viewpoint of the user input interface. Figure 1 shows a photograph of the total system and the input part. The rubber glove has on/off switches at each fingertip. The signal from a fingertip is detected as a binary signal and is input to a personal computer through a USB interface. In principle, they can input 10 bit signals using both hands, but we used 5 bit signals input using a hand in this experiment. In the experiment, five conversions of 5 bit signals are possible in a personal computer; (1) conversion to decimal signals; (2) conversion to the plus key and the A, B, X, and Y keys; (3) conversion to alphabetical letters; and (4) conversion from (1) to (3) above.

(a) SW on Glove

(b) I/O Board

(c) Total System

Fig. 1. Photograph of the proposed system

2.3 Experimental Results Figure 2 shows the block diagram of the experimental system that corresponds to Fig. 1. At first, 5bits signal obtained by the switches on the glove are input to the I/O Board as shown in Fig. 2. Next, these input signals are converted by LUT (Look Up Table) to group of the signals where the LUT are configured by PC. Finally, group of the signals are input to PC via USB. SW 1~5

31pattern

5bit Input Detection

USB LUT

To PC

I/F Input Mode Switching

Input by Fingers

From PC Input

Board

Fig. 2. Block diagram of the experimental system

208

R. Wada and T. Hase

(1) Conversion to decimal signal Generally speaking, the entry of numerals by means of the binary number system is not common, it is necessary to make teaching the minimum knowledge the first step. However, it was found that there was no difficulty in using binary numbers after practice. It was also found that by arranging the allocation of the fingers of both hands, input by 10 bits become possible, thus inputting 0 to 1023 in decimal numbers is possible. For alphabetical characters, it is possible to enter any of them by using the allocated ASCII codes. Table 1. Numerical Input Thumb

Index

Middle

Ring

Little

0

1

1

1

1

1

1

0

0

0

0

1

2

0

0

0

1

0

3

0

0

1

0

0

…

(2) Conversion to plus key and A, B, X, and Y keys This conversion also needs prior practice and a conversion table, but input speed increases as the skill level is enhanced. If the relationship between the fingers, for instance, up and down, right and left, A and B, and X and Y are improved, we can anticipate even user higher skill levels. Table 2. Game Pad Input Thumb

Index

Middle

Ring

Little

↑

0

0

0

0

1

↓

0

0

0

1

0

X

0

0

0

1

1

Y

0

1

1

0

0

…

(3) Conversion to alphabetical letters At an early stage of this investigation, input by ASCII codes was considered, but the direct input of 26 alphabetical letters is possible since 31 letters can be input at a maximum of 5 bits. As far as future investigation is concerned, further improvement of the skill level can be expected if a combination with specific meaning between the fingers and the letters is considered. This method has a feature that only pushing a fingertip on a certain part of the body will have the same function as input from a keypad, thus there is a possibility of eliminating the keypad from small portable information terminals. Table 3. Alphabet Input Thumb

Index

Middle

Ring

Little

a

0

0

0

0

1

b

0

0

0

1

0

c

0

0

1

0

0

d

0

1

0

0

0

…

Input Interface Using Fingertip

209

(4) Conversion from above (1) to (3) The special input was given, which made the conversion of each of four modes possible.

3 Conclusions This paper proposed an input method that does not need a keypad on small portable equipment. First, on/off switches at the fingertips of a hand or both hands are used. Next, the on/off signals are detected as binary system signals and treated as input signals by portable information terminals or personal computers. Further, the binary signals are allocated to binary numbers instead of decimal numbers, and to control signals instead of the plus key and A, B, X, and Y keys. An easier and less complicated input method to input 26 alphabetical letters was proposed. The combination of characters onto the fingertips can be changed by users depending on their needs. Experimental apparatus was created and the proposed system was validated. Even though each method has various tasks, such as the necessity of practice for users, the necessity of improving the allocation the fingers, and necessity of improving the input method, it was confirmed that the original purpose and target of this investigation was achieved.

References 1. IBM, Keyboard, http://www.01.ibm.com/software/globalization/topics/ keyboards/physical.jsp 2. Nintendo, DS, http://www.nintendo.com/ds 3. SONY, PSP, http://us.playstation.com/psp/ 4. The Peregrine, Wearable Interface - Large Glove, http://theperegrine.com/ 5. CEMETECH,Clove2, http://www.cemetech.net/projects/item.php?id=16

Recognition Method for Foot Written Characters Masahiro Yonezawa, Takako Nonaka, and Tomohiro Hase Ryukoku University, Yokotani, Seta-Oecho, Otsu, 5202194, Japan {t10m125@mail,[email protected],hase@rins}.ryukoku.ac.jp

Abstract. This paper proposed a method in which characters can be easily entered to an information device, by using the feet in place the hands. First, we determined input on the board was possible with an experimental system. Next, we confirmed recognition of characters directly write, and supported alphanumeric characters and, symbols. Finally we confirmed the recognition rate of the system by the input experiment of ten users. The recognition rates were more than 87 percent when including a fifth characters candidate, and our proposed user interface was confirmed as effective in recognizing handwritten characters. Keywords: Foot Written, Board, Recognition.

1 Introduction It may happen that a user cannot use their hands to enter information if they are disabled, or because the hands are being used for another purpose. This paper describes a method in which characters can be easily entered to an information device, by using the feet in place the hands. Figure 1 shows an image of the proposed system.

Display

User

Board Fig. 1. Image of proposed system

2 Proposed System This system consists of three sections: (1) a board on which the user enters the data by using both of their feet, (2) a means that recognizes the characters from the entered C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 210–213, 2011. © Springer-Verlag Berlin Heidelberg 2011

Recognition Method for Foot Written Characters

211

Start

character entry

Handwriting recognition Recognition results display no

Character strokes display Character database

・Uppercase letters ・Lowercase letters ・Numerals

Optimal character selection ｙes Execution/operation of selecting function Selected character display

End

Fig. 2. Flowchart of the experimental system

information, and (3) a display that indicates the results of the recognition. Figure 2 shows a flowchart of the experimental system By placing their feet on the board, the user enters different types of control information while checking the strokes of the character and the locus of entered characters on the display. Then the user selects the desired character from the

Fig. 3. Display screen image

212

M. Yonezawa, T. Nonaka, and T. Hase

Fig. 4. Operation of the prototype system

alternatives shown on the display as a result of character recognition. Figure 3 shows the display screen image, and Figure 4 shows the operation of the prototype system.

3 Verification Experiments After creating an experimental system, a subjective evaluation experiment was performed with ten subjects. The results showed that a character recognition rate of 87 percent was achieved. Figure 5 shows the recognition rate of the system. Future challenges include: (1) a one hour training session is required so that the system may be used without mental strain; (2) it takes considerable time to enter characters (one character per minute). We consider, however, the results are better than those of the eye-glance input method, which has been proposed for similar purposes.

6th 2.5

No Alternative 12.8

5th 3.6 4th 4.4

1st 53.6

3rd 9.4 2nd 13.6 Fig. 5. Character Recognition Rate

Recognition Method for Foot Written Characters

213

4 Conclusions A method has been proposed to use feet instead of hands to enter desired characters into an information device when the user cannot use their hands for some reason. Then an experiment system was created and a subjective evaluation experiment was performed. The results show that the proposed system is effective. Acknowledgments. This work was supported in part by the "High-Tech Research Center" Project for Private Universities: matching fund subsidy from MEXT (20062011) and Ryukoku University.

References 1. Wii Balance Board: Wii Fit Plus (March 25, 2011), http://wiifit.com/what-iswii-fit-plus/#pageWhatIsWiiFitPlus 2. Zinnia: Online hand recognition system with machine learning (March 25, 2011), http://zinnia.sourceforge.net/index.html

Part IV

Touch-Based and Table-Top Interaction

Sounds in Space: 3D Audio Experiences through Tangible Navigation Andrew Blakney and Sudhir Mudur Concordia University Montreal, Quebec

Abstract. This paper presents an experiment with 3D audio design and experience through a Tangible User Interface for 3D navigation. A generic 3D navigation metaphor, Navigational Puppetry, provides the user with a graspable viewpoint that allows them to ‘reach’ into the virtual world. An audio expert, specializing in relationships between ‘music, place, and mobility’, uses an audio-enabled prototype of this metaphor - the Navi-Teer - to populate a soundscape with graphical representations of sound elements. As navigation occurs, the audio environment yields unusual and complex 3D audio mixtures and spatial sound interactions. The experiment showcases Navigational Puppetry’s subtle interactive benefits of increased spatial orientation, tactile intimacy, easy capture of complex input and support for collaboration in a task that requires navigation to complete a larger goal.

1 Introduction The Navi-Teer interface is a Tangible User Interface (TUI) prototype for 3D navigation, which contains a superposition of egocentric and exocentric perspectives and a blending of navigational action and perception spaces. This affords an interaction that can best be described as ‘navigational puppetry’ - the user is a ‘puppeteer’ standing over a 'stage' containing a 'puppet', whose perspective view is available to the ‘puppeteer’ (see Figure 1). In the case of navigation for spatial sound, the puppet possesses the user’s ears and the stage embodies the audiovisual soundscape through which the puppet-ears explore and perceive sound. Researchers in the field of interfaces have recognized that interactions with sound are one of the most intensive interactive experiences within the known human experience [1, 2], and have experimented with this notion through the development of many diverse tangible or graspable audio interfaces [5, 7, 8]. By combining representation and control and through the blurring of input and output spaces, TUIs redefine the boundary between computer and user, allowing a broader and more intimate exchange [6], which makes them the perfect interactive vehicle for explorations into audio control and experience. The interactive and sensory experiences of visual navigation and auditory exploration / experience of a soundscape are interestingly related and complimentary. Specifically, both require continuous temporal control over multiple, and sometimes parallel, ‘degrees’ or ‘dimensions’, while maintaining certain macro and micro conceptual orientations. Also, both could potentially benefit from increased spatial C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 217–221, 2011. © Springer-Verlag Berlin Heidelberg 2011

218

A. Blakney and S. Mudur

awareness and tactile intimacy, affordances for excessive/irregular/complex input, and minimal (or at least transparent and/or intuitive) translation from action to perceived result. Combining the graspable advantages of TUIs with the task of 3D navigation is in area that has produced interesting preliminary results [3, 4], but which could benefit from further investigation.

Fig. 1. 3D Audio Experiment

2 Related Work Graphically and/or spatially enhanced approaches to 3D sound have been investigated and applied in many different domains, from game design to the creation of actual physical sound attractions. VRSonic (http://www.vrsonic.com/) and SoundLocus (http://www.soundlocus.com) are two examples of many available software packages enabling 3D audiovisual modeling that are ‘navigational’ in nature. There has been a vast amount of research regarding TUIs as interfaces for audio interactions [5, 7, 8] and a full review of this research is beyond the scope of this paper. However, an interesting work that highlights the potential for yielding new user experiences with tangible audio control is the Pendaphonics project [5]. This interface provides a very unique and physical way of experiencing a virtual 3D soundscape, which is linked to a physical interaction space in reality. The system consists of a large vertical display, providing the view of the soundscape to the user(s), and one or more suspended pendulums. As the pendulum is swung by one or more users through the physical interaction space, the sounds it generates change according to its relative virtual path through the soundscape. The nature of the interaction style affords very interesting explorations with sound control which are not possible without such an interface.

3 Navi-Teer Interface for 3D Audio The Navi-Teer interface prototype (see Figure 1) is composed of two NEC VT491 projectors: one projecting the puppet's first person view to a large vertical rearprojection screen and one is projecting the stage-view of the VE from above down onto the horizontal table-top interaction surface. The downward projector also displays a bounding box and look-orientation-direction arrow dynamically on top of the puppet. The puppet’s body is represented by a movable flat-bottomed base, containing 3 wireless function buttons, to be manually translated on a flat surface while being optically tracked in 2-DOF from above by a Quickcam Pro 4000 camera. The puppet’s head is represented by a graspable sphere which is mounted on top of the puppet-body to 'look around', being sensed isotonically in 3-DOF using an Intersense InertiaCube 3 wireless 3D orientation sensor. To enable exocentric stage

Sounds in Space: 3D Audio Experiences through Tangible Navigation

219

modifications, there is a generic tangible handle which slides around on the horizontal surface. The underlying application and VE was programmed in C++ using OpenGL (www.opengl.org), the ARToolkit (www.hitl.washington.edu/artoolkit/), and OpenAL (www.connect.creativelabs.com/openal). The Navi-Teer interface is intended to be used with one or two hands on the horizontal interaction surface while the user(s) is positioned in front of the large vertical display. Basic 2-DOF translation of the viewpoint/virtual-ears, achieved by moving the puppet-body base on the horizontal surface, corresponds to a 1-1 movement of the viewpoint/virtual-ears on a horizontal plane within the soundscape. If continuous movement across a large distance is required, the puppet may deploy a movement circle onto the horizontal plane around itself. A movement circle is simply a projected circle surrounding the puppet-body base. When the puppet-body is moved to make its bounding box break the boundary of the surrounding circle, the viewpoint/virtual-ears fly, or are translated, in the direction of the break at a speed relative to how far outside of the circle the edge of the puppet-body went (ie: the greater the distance outside the circle, the greater the speed). The puppet may fly either constantly flat along the current stage plane, or look-directed flying may be enabled. At any point, the 3-DOF of orientation of the viewpoint/virtual-ears can be modified by rotating the puppet-head.

Fig. 2. The Navi-Teer 3D Audio Experience

To enable basic soundscape modeling, the Navi-Teer affords a simple ray-casting method to grab, position, and orient sound objects in 3D space. Sound objects are represented within the VE as solid colour geometric shapes - either spheres (sounds with no directional orientation) or cones (sounds with directional orientation), whose sizes are relative to their volumes (See Figure 2). These sound-spheres and soundcones are imported into the virtual soundscape by bringing them out of a soundrepository cube, through ray-casting, in a predetermined order. The sound objects can be set to activate when the puppet is within certain proximities and they can also be given their own dynamic paths to move around while the puppet experiences the soundscape. Furthermore, all sounds can be simultaneously played / stopped in unison, or else each sound can be turned on/off independently by ray-selection. At any time, the path of the puppet through the soundscape can be recorded and played back to repeat a desired soundscape experience.

4 3D Audio Experiment We enlisted an audio designer / artist, whose work focuses on the relationships between “music, place, and mobility” [9], to use the Navi-Teer interface as a final 3D

220

A. Blakney and S. Mudur

mixing tool for a particular portion of a composition which contains instrument sounds from raw audio data recorded while riding a bus. He arrived with nine prepared audio .wav files, which were musical expressions of certain raw audio elements. Certain files corresponded to what would become the rhythm or percussive elements. Other sound files would become parts of the melody, such as various hums and ambient noises, which when arranged in a specific manner produced interesting melody combinations. The final composition was intended to be listened to on a personal music device while riding the same bus route. The expert wanted to simulate the beginning of the bus ride with an approach of a specific rhythm that would then blend into another rhythm and melody combination. To accomplish this, they first positioned a sound-sphere, representing the track for the introductory rhythm, in an open space. Then they positioned five other sound-spheres ahead of the previously placed sphere. Two rhythm spheres whose auditory interaction – depending on which rhythm is panned to more than the other – form the desired rhythm to be experienced after the introductory portion is completed. These spheres were placed far from each other, on both the left side and right side relative to the previously placed sphere. The three remaining spheres, relating to another rhythm element and a basic melody, were arranged to form a triangular structure with one sphere resting above an open space between two other spheres on the horizontal plane. There was now a simple soundscape which could be experienced with the Navi-Teer interface (see Figure 1). Once all the spheres were ‘singing’ properly, the expert traveled towards the triangular structure. Movements in and around that position in the soundscape achieved very interesting melody and beat interactions. During the entire interaction the expert was able to observe and grasp an interesting dual-perspective visual representation of the 3D audio mix. After the experiment was concluded, the expert gave an extended interview on their experience with the Navi-Teer interface. They stated that the Navi-Teer’s focus on space and movement, combined with the usual temporal aspects of sound and music manipulation, changes the entire audio interactive experience. In essence, they said, there is far less distance between the individual and ‘the sounds’ due to the blurring of input and output spaces. There was a noticeable increase in the tactile immediacy of actions and responses, which is actually a more natural and realistic way to work with sound. It allowed them to experiment with many different ideas on the spot, and also to perfect certain specific movements if so desired. In addition, they also commented on how this kind of audio interface adds a certain amount of observational or performance significance to the 3D mixing task that is not present with more traditional PC interfaces.

5 Conclusion We have presented an experiment in 3D soundscape design and experience that was afforded by an audio-enabled version of the navigational puppetry metaphor and the Navi-Teer prototype. The results outline an enhanced 3D audio interactive experience which was made possible through a graspable linking of sound, space, and time. According to feedback from our audio expert, approaching 3D soundscapes from a

Sounds in Space: 3D Audio Experiences through Tangible Navigation

221

‘puppetry’ perspective provided a more natural and intimate tactile interaction with sound and also enabled the task to have observational and / or performance significance. Acknowledgments. This research was supported in part through grants from NSERC, Canada, ENCS Faculty, Concordia and the GRAND NCE, Canada. We also thank Mr. S. Thulin for his participation in the audio experiment.

References 1. Bischoff, J., Gold, R., Horton, J.: Music for an interactive Network of Computers. Computer Music Journal 2(3), 24–29 (1978) 2. Crevoisier, A., Polotti, P.: Tangible acoustic interfaces and their applications for the design of new musical instruments. In: Proceedings of the 2005 Conference on New Interfaces For Musical Expression, pp. 97–100. National University of Singapore (2005) 3. Fjeld, M., Ironmonger, N., Schar, S.G., Krueger, H.: Design and Evaluation of four AR Navigation Tools Using Scene and Viewpoint Handling. In: Proceedings of INTERACT 2001 8th IFIP TC.13 Conference on Human-Computer Interaction, pp. 167–174 (2001) 4. Göttel, T.: ProBoNO: Transferring Knowledge Of Virtual Environments To Real World Situations. In: Proceedings of the 6th International Conference on Interaction Design and Children, IDC 2007, pp. 81–88. ACM, New York (2007) 5. Hansen, A.S., Overholt, D., Burleson, W., Jensen, C.N.: Pendaphonics: a tangible pendulum-based sonic interaction experience. In: Proceedings of the 3rd International Conference on Tangible and Embedded Interaction, TEI 2009, pp. 153–160. ACM, New York (2009) 6. Ishii, H.: Tangible Bits: Beyond Pixels. In: Proceedings of the TEI 2008, pp. 15–25. ACM, New York (2008) 7. Jordà, S., Geiger, G., Alonso, M., Kaltenbrunner, M.: The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI 2007, pp. 139–146. ACM, New York (2007) 8. Pedersen, E.W., Hornbæk, K.: Mixitui: a tangible sequencer for electronic live performances. In: Proceedings of the 3rd international Conference on Tangible and Embedded Interaction, TEI 2009, pp. 223–230. ACM, New York (2009) 9. Thulin, S.: There to Hear: Placing Mobile Music. Master’s Thesis. Concordia University (2009)

Multi-touch Surface Table with Multi-point Tactile Feedback Siam Charoenseang and Navakun Sribang Institute of Field Robotics, King Mongkut's University of Technology Thonburi, Thailand [email protected], [email protected]

Abstract. Currently, most of multi-touch surface tables do not provide any tactile feedback to the users. In this paper, a multi-touch surface table with tactile feedback is proposed. The proposed surface table provides the users with the ability of multi-point tactile feedback with variant patterns and levels of vibrations. A webcam is used to capture all finger-touch images as system inputs using computer vision. Interactive computer graphics is generated and projected on the surface table. In addition, the multi-point tactile feedback is produced by multiple motors attached below the display surface. Hence, the user will receive tactile feedback synchronized with multi-point interactive computer graphics. Keywords: Surface-based Display/ Multi-touch Display/ Tactile Feedback.

1 Introduction Multi-touch technology becomes more popular in daily life. It is integrated with several electronic devices such as mobile phones with touch screens, touch screen monitors, and etc. One of the advanced multi-touch products is a multi-touch surface table. It allows the user to directly interact with a system using finger touch on the screen display. Furthermore, it is able to support multiple users to work together on the same multi-touch display table. Earlier research works in the field of multi-touch surface have introduced various methods for touch detection. Jeff Han proposed the frustrated total internal reflection (FTIR) [1] technique that is based on the light totally reflected phenomenon. He implemented infrared lights inundating inside a piece of clear acrylic screen. When the user touches a surface, the infrared lights will leave the acrylic screen to the touching points captured by a camera behind the acrylic screen. Katz, I., Gabayan, K., and Aghajan, H. proposed a multitouch surface using multiple cameras technique for determining the fingertip’s positions [2]. The capacitive coupled technique for detecting finger touches on the tabletop front-projected display in DiamondTouch [3]. There are also touch screens with tactile feedback for small handheld devices developed by Ivan, P., Jun, R., and Shigeaki, M. [4, 5]. However, most of multi-touch surface table does not provide tactile feedback. Hence, this paper proposes a multi-touch surface table with multi-point tactile feedback that can provide haptic senses during the user interacts with a system. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 222–226, 2011. © Springer-Verlag Berlin Heidelberg 2011

Multi-touch Surface Table with Multi-point Tactile Feedback

223

2 System Overview Figure 1 shows the configuration of the proposed system. Hardware components of this system are a multi-touch display table and a tactile feedback system. This humancomputer interface provides the user with tactile feedback and 2D computer graphics on the display surface. A webcam attached in the multi-touch display table is applied for detecting finger touch image. Array of small motors, which are attached below the display surface, are used to generate multi-point tactile feedback synchronized with graphics.

Fig. 1. System Overview

2.1 System Components The dimension of the proposed multi-touch display surface is 1200 mm(W) x 700 mm(L) x 800 mm(H). The display surface is a clear acrylic sheet with 10 mm thick. Hardware components of proposed system are the webcam, infrared LEDs, video projector, and tactile feedback system as shown in Figure 2. Software includes the computer vision engine, graphics engine, and tactile engine.

Fig. 2. System Components

For the system’s vision and display, the webcam is modified for capturing 320 x 240 IR image with 30 frames/sec. Rear diffused illumination is the lighting techniques of this system with several infrared LEDs. The mirror is used for reflecting image projected from the video projector. It can increase a distance between the video projector and the display for magnifying the output image. For the tactile feedback,

224

S. Charoenseang and N. Sribang

actuators should be small, lightweight, and low power consumption. A small DC motors as shown in Figure 3 (a) are used for generating tactile feedback to the user. Thirty motors, which are attached below the display surface, are linked together with tiny copper wires. There are 5 rows of motors. Each row contains 6 motors and the space between motors is about 100mm as shown in Figure 3 (b).

(a)

(b)

Fig. 3. (a) The implemented small DC motors. (b) Thirty motors mounted below the display.

Figure 4(a) shows the overview of a tactile controller. The tactile controller consists of Arduino microcontroller [6] and a transistor circuit implemented for driving array of motors as shown in Figure 4(b).

(a)

(b)

Fig. 4. (a) Tactile controller: 1) Row motor driver circuit 2) Column motor driver circuit 3) Arduino microcontroller (b) Schematic diagram of a tactile feedback driver

For the part of system software, Community Core Vision (CCV) software library developed by NUI Group Community [7] is used in the computer vision engine. The computer vision engine is responsible for detecting position of finger touch on the multi-touch display table. The position of detected finger touch is then sent to the graphics engine via the TUIO [8] protocol. It is used to trig the event in graphics engine. The graphics engine renders 2D computer graphics projected on the display surface via a video projector. The tactile engine generates the impulse response through the small motors when the user interacts with the graphics. That engine also provides multi-point vibration synchronized with the computer graphics. The user will receive both graphics information and corresponding vibration when he/she touches the screen.

Multi-touch Surface Table with Multi-point Tactile Feedback

225

3 Experimental Results The experiments are set for testing the system performance and usability. The system performance covers the provided ability of the system input and output. The input image which captured by the camera is 320 x 240 pixels with 30 fps. The interactive working area for input and output is a screen with 30” diagonal. The graphics update rate is about 60 fps with the image resolution of 1024 x 768 pixels. This system can track the finger touches with velocity up to 120 cm/sec and up to 50 finger touches simultaneously. The tactile feedback system can response 30 points independently. The total area which can provide tactile feedback is about 46 cm x 60 cm. The smallest interactive area with tactile feedback is about 10 cm x 10 cm. 3.1 Usability To test the usability of the tactile feedback, the users were asked to find the 8 invisible virtual buttons with random positions on the display. The experiments are set in three conditions which are the invisible virtual buttons with only tactile feedback, ones with only audio feedback, and ones with both tactile and audio feedbacks. The number of correct positions and times used were recorded. From the results as shown in Figure 5(a), the invisible virtual buttons with both tactile and audio feedback can increase the number of correct positions with 5 percents compared to the ones with only tactile feedback and 11 percents compared to the ones with only audio feedback. Figure 5(b) shows the time used to find the correct positions of all 8 invisible virtual buttons. The operation time from the system with both tactile and audio feedback can be reduced with 49 percents compared with time from the system with only tactile feedback and 32 percents compared with time from the system with only audio feedback.

(a)

(b) Fig. 5. (a) Accuracy of correct identification of position (b) Time used

For another experiment, the users were asked to classify the vibration level which was generated into three levels. Before doing a test, the user must learn about each vibration level. From the experimental results, users can correctly identify the high vibration level with 48 percents, the medium vibration level with 62 percents, and the low vibration level 90 percents. The user’s satisfaction after using the system is evaluated through data from a questionnaire. From the results, the usability of display

226

S. Charoenseang and N. Sribang

size got the highest satisfaction score with 78 percents. The satisfaction score of the use of the virtual buttons with tactile and audio feedback is increased by 6 percents compared with the score obtained from the use of the virtual button without tactile and audio feedback. With tactile and audio feedback, it makes the user feel like he/she interacts with a real button. The lowest satisfaction score is related to accuracy of touch detection with 66 percents.

4 Conclusions and Future Works This research implemented the multi-touch surface table with multi-point tactile feedback. The proposed multi-touch table applied DI lighting technique for finger touch detection. The small motors which mounted below display surface are used for generating tactile feedback. From the experiment results, the tactile feedback helps user to find the invisible virtual buttons more easily and correctly. The virtual button, which provides tactile and audio feedback, helps user to feel more realistics. Various patterns and levels of the tactile feedback can be programed to increase the effectiveness of this proposed system. Furthermore, the accuracy of touch detection can be improved by adjusting the IR lights and selecting the suitable threshold for the binary image conversion. It is challenge to apply this system for entertainment applications, public relations, and computer-assisted instruction applications which the user can use them for self-learning. Acknowledgments. This research work is financially supported by the National Science and Technology Development Agency, Thailand.

References 1. Jefferson, Y.H.: Low-cost multi-touch sensing through frustrated total internal reflection. In: Proceedings of the 18th annual ACM symposium on User interface software and technology, pp. 115–118. ACM, Seattle, WA, USA (2005) 2. Katz, I., Gabayan, K., Aghajan, H.: A multi-touch surface using multiple cameras. In: Blanc-Talon, J., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2007. LNCS, vol. 4678, pp. 97–108. Springer, Heidelberg (2007) 3. Dietz, P., Leigh, D.: DiamondTouch: a multi-user touch technology. ACM, New York (2001) 4. Ivan, P., Shigeaki, M.: Tactile interfaces for small touch screens. In: Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology. ACM, Vancouver (2003) 5. Ivan, P., Jun, R., Shigeaki, M.: TouchEngine: a tactile display for handheld devices. In: CHI 2002 Extended Abstracts on Human Factors in Computing Systems. ACM, Minneapolis (2002) 6. Arduino, http://www.arduino.cc 7. Community Core Vision, http://www.ccv.nuigroup.com 8. TUIO, http://www.tuio.org

Suggested Considerations on the Design of Multi-touch Interfaces for Commercial Presentation Ting-Han Chen XxtraLab DESIGN Co., 1F, No.33, Sec.2, Zhongxiao East Road, Zhongzheng District, Taipei, Taiwan [email protected]

Abstract. This paper details several design considerations derived from multitouch projects for commercial presentation that we have conducted in the past. These considerations are suggested in the hopes that they may help future designers decide on appropriate design strategies and styles when designing a basic commercial multi-touch presentation interface. Keywords: Presentation design, multi-touch, design considerations.

1 Introduction Techniques for manipulating digital information with hands and fingers are now amongst the core studies of multi-touch surfaces in the realm of HCI. We must first thank previous research which has summarized useful suggestions for designing better multi-touch interface experiences [1, 2]. However, with regard to how these multi-touch interface experiences can accommodate specific presentational purposes, there has yet to be sufficient discussion on the design considerations that arise. This paper details several design considerations derived from practical multi-touch projects for commercial presentation that we have conducted. These considerations are suggested in the hopes that they may help future designers decide on appropriate design strategies and styles when designing a basic commercial multi-touch presentation interface. The multi-touch presentation interfaces we refer to are mainly wall sized and make use of visual representation (i.e. displaying graphics, videos, and texts) and optical multi-touch technology implementation models [3].

2 Projects Overview Multi-touch projects for different presentational purposes that we have conducted (Fig.1) include: (a) BriefWall – a casual presentation system in a multi-user environment, (b) MediaFaceX – a system for presentation for close quarters, (c) iOrange – a tailor-made presentation system for the real estate industry, and (d) MediaFaceXMS – a system for PowerPoint-like (slide-based) presentation. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 227–231, 2011. © Springer-Verlag Berlin Heidelberg 2011

228

T.-H. Chen

Each multi-touch project includes built-in multi-touch manipulation techniques, such as drag, touch-to-evoke, scale, and rotate, as well as other advanced techniques for more specific uses. Below we have elaborated on each of these projects.

Fig. 1. From left to right: BriefWall, MediaFaceX, iOrange, and MediaFaceXMS

2.1 BriefWall (BW) BW is a large multi-touch surface of 4.5M(W) x 2.4M(H) designed originally for the Intelligent Building Exhibition’s demonstrational future conference room during which it was used by the exhibition participants for their respective presentations. It contains 50 presentation slides for the different participants in the form of “virtual folders.” These virtual folders, located at the bottom of the screen, can be popped out in a standalone window when a user touches the virtual folder icon. Users are then able to move, scale, and switch slides in the pop-out window. The system thus allows multiple presenters to make their own window-based presentation by evoking different virtual folders on the same shared surface. 2.2 MediaFaceX (MFX) Based on our experience with BW, MFX, a 72-inch multi-touch screen for navigating rich media contents, was created. Media content such as pictures and videos can be resized, navigated, repositioned, and manipulated efficiently via defined complex gestures. The purpose of MFX is to modularize this technology into a presentation set and to understand the relationship between screen size and number of users. Two control interfaces are placed on the screen, meaning that up to two persons can share the 72-inch screen. 2.3 iOrange (iO) iO is designed to be used by salespersons of real estate companies when presenting their company's profile, products, and relevant real estate information. It deploys a full-wall screen and is for single presenter use. Special gestures are also designed for storing data from different regions and comparing different cases. The main idea behind iO is "Menu Follows You" which allows a presenter to pull up the navigational interface by just tapping twice on the 4M(W) x 2M(H) multitouch wall, no matter where the presenter is standing. It also acts as a simplified interface; when tapping on it, it will bring forth corresponding sub-menu items and change background contents. In this way, presenters are allowed to retrieve any content or information they desire simply by manipulating the iO interface.

Suggested Considerations on the Design of Multi-touch Interfaces

229

2.4 MediaFaceXMS (MFXMS) MFXMS is the advanced version of MFX which gives full screen manipulability for a single presenter. The software running the 140-inch multi-touch screen is designed for easy use and maintenance. The program automatically loads in JPEG images and FLV videos, and composes in a slide show sequence. Using two-finger manipulations on the controller interface, the presenter can browse, switch, or forward/rewind the slide sequence. The program also has a built-in whiteboard function, which enables the presenter to annotate on each slide using finger drawings.

3 Design Consideration Highlights By empirically studying the usage of these different interfaces, we have identified several issues concerning design considerations for multi-touch presentation interfaces. Simple vs. Complex Gestures. In every project, we have implemented basic multitouch gestures such as move, touch, drag, and scale. We call these “simple gestures” because presenters tend to know how to use them naturally without being taught. As well, some advanced gestures have also been implemented into every project. For instance, in BW, sliding two fingers down the screen will bring forth a folder swapping interface. Or, in MFX, two fingers touching the screen and rotating like turning a knob will switch slides. However, these special gestures will be unknown to the average user, thus presenters are required to take some time to become familiar with their use. Nevertheless, once presenters find a familiarity with these techniques, they will be able to manipulate them fluently. Full Screen vs. Screen for Multi-person. In regard to the arrangement of screen layout, in BW, users can evoke many presentation windows on the 4.5M wide surface. Approximately up to five people can share the screen at the same time. In contrast, iO and MFXMS are designed for single presenter use and deployed full screen functionality. With regard to the number of presenters, different dynamic screens and layout designs should be considered. However, it should be noted that there is the potential problem of boundaries when designing a screen for multi-person use. In the case of BW, we attempted to solve this problem by creating a collision detection system that ensured that two windows would bounce away if they collided with each other. On the other hand, in MFX, areas of overlap would instead appear at 50% transparency. One-to-one vs. One-to-many. On MFX and BW, the context was envisioned as one presenter to one audience member. Hence, the size and position of the items shown on the screen were designed with this in mind. We also set a minimum and maximum value for scaling images and texts in order to avoid over-scaling that might lead to low readability. Conversely, in iO and MFX, the presentation context was set as oneto-many, with an audience as big as 20 to 30 considered. Thus, we were required to consider how a group of 20-30 people could clearly see the multi-touch presentation over a certain distance. As well, we had to consider instances when the presenter’s body would block the screen from certain angles.

230

T.-H. Chen

Single vs. Dual Presenters. All of our projects are designed for “one presenter, one control interface.” As for shared multi-touch screen sets, it remains the same, only in that case becoming “one presenter, one window interface.” However, we found that in large screen size interfaces such as in BW, iO and MFXMS, our clients tend to arrange for at least two presenters when making a public presentation. One presenter speaks, while the other concentrates on manipulating the contents on the screen. This is done because to speak publicly and manipulate the multi-touch screen are two complex tasks; unless one is well practiced, it is not easy to speak freely while manipulating the screen with two hands. By a division of labor, the whole presentation goes more smoothly. Modularization vs. Customization. The interfaces of BW, MFX, and MFXMS are designed as modularized units. Their presentation formats, which use a slide-show based system, are designed for easy replacement and update. As for iO, it is customized and designed according to a workflow that real estate presenters find most useful when giving presentations. Their unique needs during presentations are taken into account and reflected in the interface design—which includes perspective regional map navigation, building categorization, and grouping functions for a building wish list with specially designed gestural techniques. Such functions are specifically aimed at real estate presenters and, therefore, may not be suitable for a general presentation format, or for use by presenters of other industries. Slide-based vs. Hyperlink-based. MFX, MFXMS, and BW all use a slide-based format, which means the presentation is carried out mainly through the switching of slides. iO, on the other hand, takes a hyperlink-based approach where the navigation of the presentation is more like browsing the web. Slide-based formats are more suitable for presenting sequential contents, while hyperlink-based is more appropriate for structured typology in which the presenter needs to jump among different contents or levels of contents interactively. Interface-follows-presenter vs. Presenter-follows-interface. In iO and MFX we have included an application that enables the presenter to call up the interface controller by tapping twice on the screen no matter where he/she stands. In the case of a multi-personal shared-screen, when two persons tap the screen at the same time, each interface controller will fly to the tapping point that is closest to its position. This “Interface-follows-presenter” application adds a large degree of freedom with regard to the presenter’s mobility. This application, however, is not present in BW and MFXMS, as such, the presenter must be next to the controller on the screen to use it, or drag the interface controller as he/she moves. Presenters who are in the habit of moving around when making a presentation often require some time to adapt to the fact that the interface controller does not automatically follow their movements. Uni-directional vs. Interactive. It has been shown that during a multi-touch presentation, a presenter will on occasion invite the audience to come on stage and manipulate the screen in order to increase the interactivity and vivacity of the presentation experience. For example, after presenters have finished presenting the BW system, they always invite audience members up to personally experience the multi-touch manipulation. According to our observations, members of the audience

Suggested Considerations on the Design of Multi-touch Interfaces

231

who are invited to try out the multi-touch interface require different learning times and levels of adaption depending on the complexity of the gestural techniques. Nevertheless, it is evident that a presenter can arrange different multi-touch experiences for the audience during different phases of the presentation in order to enhance the interactivity of the presentation experience.

4 Conclusion Above we have highlighted several design considerations derived from our experience. We believe that it will be of benefit for those having to decide on design strategy and style when designing a basic commercial multi-touch presentation that needs to conform to different design constraints or conditions. Furthermore, we hope this paper will succeed in fostering a formulation of framework and principles regarding the design of a better multi-touch presentation system.

References 1. Wang, C., Lee, H., Smeaton, A.F.: Designing Interaction for a Multi-touch Wall. In: 4th Irish Human-Computer Interaction Conference, Dublin, Ireland, ISBN: 1872327885 (September 2-3, 2010) 2. Bachl, S., Tomitsch, M., Wimmer, C., Grechenig, T.: Challenges for Designing the User Experience of Multi-touch Interfaces. In: Engineering Patterns for Multi-Touch Interfaces workshop (MUTI 2010) of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems, Berlin, Germany, pp. 365–366 (2010) 3. Teiche, A.: Multi-touch Technologies (2009), http://nuigroup.com

A Study on the C/R Ratio of Direct-Operation Multi-touch Interface Kuan-Hung Chen, Chun-Wen Chen, and Wenzhi Chen Department of Industrial Design, Chang Gung University 259 Wen-Hwa 1st Road, Kwei-Shan, 333 Tao-Yuan, Taiwan [email protected], [email protected], [email protected]

Abstract. Multi-touch interface is becoming an important part of consumer information products. Interface design should have more in-depth understanding to achieve better human-computer interaction. This study investigates the multitouch interface’s control/response ratio (C/R ratio) with rotation and moving tasks to understand the effects on participant’s performance and satisfaction. The results showed that the better C/R ratio is not the same for efficiency of different operating tasks. And the user satisfaction was affected by previous experience of the participants. Keywords: multi-touch, C/R ratio, sensibility.

1 Introduction Multi-touch interface is becoming an important part of consumer information products. Control/response ratio (C/R ratio) is a main factor in the design for such positioning devices. This study examined the user performance and satisfaction to understand the appropriate C/R ratio of multi-touch interface. Sanders & McCormick [1] extended man-machine interface’s C/R ratio concept and general application. The original term display was replaced by system response. C/R ratio is the reciprocal of gain. Low gain is low sensitivity. High gain is high sensitivity. It is explained by the knob interface (see Figure 1). The concept can be applied on the mouse’s sensitivity (see Figure 2). Stevenson, et al. [2] adjusted the C/R ratio of mouse to compare user performance. The result showed high C/R ratio could provide advantage in accuracy. When the input interface is replaced by a multi-touch one, the C/R ratio is a factor not obvious to investigate. North, et al [3] mentioned that the single-touch operation was developed in the concept of the original one-click mouse operation. The operation of multi-touch may require a different concept model. Perhaps in different tasks, the required sensitivity is different. Literature related to multi-touch is mostly on the performance and satisfaction of task-based operation. Fiorella, Sanna, & Lamberti [4], Yee [5], Esenther & Wittenburg [6] all used a variety of multi-touch control modes to analyze user performance and satisfaction. In the literature, three most commonly used multi-touch control modes are moving, rotating, and scaling. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 232–236, 2011. © Springer-Verlag Berlin Heidelberg 2011

A Study on the C/R Ratio of Direct-Operation Multi-touch Interface

Fig. 1. Low & high C/R ratio performance

233

Fig. 2. Low & high gain of mouse on display

2 Method There were ten participants, five males and five females, aged from 21 to 25. They were familiar with pointing operations. All had operation experience on single-touch and multi-touch interface, but they have not used multi-touch interface over 22 inches before. In this study, the used device of multi-touch screen was ACER T231H, 23 inches, the highest resolution 1920x1080 (pixel), viewable size 509.76 x 286.74 (mm). It adapted optical infrared LED panel technology. This study manipulated C/R ratio in two multi-touch modes, moving and rotating. The participants’ completion time, the number of errors, the number of touch and user satisfaction were collected and analyzed. Five levels of C/R ratio were used. 1/1 was set as median value. The highest and lowest levels were two times and half, as shown in Table 1. For each participant, the order of C/R ratio was set in random to avoid the participants’ learning effect. Table 1. Five C/R ratio levels and the related gain and sensibility C/R Ratio Gain Sensibility

2/1 1/2 Lowest

4/3 3/4 Low

1/1 1/1 Standard

3/4 4/3 High

1/2 2/1 Highest

The experiment system was designed with Adobe Flash CS5 that supported multitouch to carry out interface with two multi-touch modes of moving and rotating. Original objects size was set in 320x240 pixels with a poker pattern. Poker pattern could reduce the participant’s cognitive problems for the image direction. The participants operate the two multi-touch modes on the objects to complete all the tasks (see Figure 3 and Figure 4). The error range of target size was set in five percent (see Figure 5). If the object was not moved or rotated within the error range and the participants clicked "Finish" button, the system would log an error record. The system would pop up a window to alert participants to continue the experiment until the participants correctly placed the target within the error range. The study is a withinsubject design. Each participant was tested with all ten tasks of two modes and five C/R ratio levels. Satisfaction was evaluated upon completion of each task. Satisfaction evaluation adapted Likert scale with five levels: “very difficult to use (1 point)” to “very easy to use (5 points).

234

K.-H. Chen, C.-W. Chen, and W. Chen

Fig. 3. Rotating task

Fig. 4. Moving task

Fig. 5. The inside and outside error range of targets was limited to 5%

3 Results Table 2 shows the experimental results. With different levels of C/R ratio in the moving and rotating tasks, mean and standard deviation of completion time, number of operations, the number of errors and satisfaction are were presented in the table. For reading convenience, values of C/R ratio in the table are set in descending order, that is, from the low sensitivity to high sensitivity. The C/R ratio of the lowest completion time (13.30 seconds) for rotating task is 2/1, and the one of the highest completion time (28.90 seconds) is 4/3. For moving operation task, the C/R ratio of the lowest completion time (9.00 seconds) is 3/4, and the one of the highest completion time (11.90 seconds) is 1/1. There is no significant difference in task completion time among levels of C/R ratio (F = 1.222, df = 4, P = 0.351 > 0.05). In the aspect of touch number, the C/R ratio of the lowest number of operations (5.20) for rotating task is 2/1, and the one of the highest number of operations (8.40) is 4/3. For moving operation task, the C/R ratio of the lowest number of operations (2.40 seconds) is 1/2, and the one of the highest number of operations (4.10) is 2/1. There is no significant difference in number of operations among levels of C/R ratio (F = 0.841, df = 4, P = 0.502 > 0.05).

A Study on the C/R Ratio of Direct-Operation Multi-touch Interface

235

Table 2. Completion time, number of operations and errors, and satisfaction with C/R ratios C/R Ratio

2/1 Mean (sd)

4/3 Mean (sd)

1/1 Mean (sd)

3/4 Mean (sd)

1/2 Mean (sd)

Rotating 13.30 (3.09) 28.90 (16.00) 22.10 (22.52) 21.90 (12.31) 22.90 (11.96) 9.00 (4.80) 9.40 (2.87) Moving 10.50 (3.47) 11.50 (6.41) 11.90 (9.92) 8.40 (5.14) 5.30 (4.02) 6.10 (3.51) 5.90 (2.88) Touch num Rotating 5.20 (2.15) 2.90 (1.28) 3.20 (2.48) 2.60 (0.84) 2.40 (0.51) Moving 4.10 (1.19) 0.10 (0.31) 0.10 (0.31) 0.10 (0.31) 0.10 (0.31) Rotating 0.00 (0.00) Error 0.10 (0.31) 0.10 (0.31) 0.00 (0.00) 0.10 (0.31) Moving 0.00 (0.00) 1.90 (1.10) 2.70 (1.16) 2.20 (1.03) 2.00 (0.94) Satisfaction Rotating 2.50 (0.70) 4.00 (0.81) 4.20 (0.91) 3.90 (0.73) 3.40 (0.69) Moving 3.10 (0.73) Time

The average number of errors was close to zero. There is no significant difference in number of errors among levels of C/R ratio. For evaluation of satisfaction, in the rotating and moving task set with C/R ratio of 1/1, the score is the highest, with 2.70 and 4.20 respectively. The C/R ratio of the lowest score of satisfaction (1.90) in rotating tasks was 4/3. And one of lowest score of satisfaction (3.10) in moving task was 2/1. There is no significant difference in overall satisfaction among levels of C/R ratio. But in the moving tasks, there is significant difference in satisfaction among levels of C/R ratio (F = 3.351, df = 4, p = 0.017 < 0.05). There is a significant correlation between satisfaction and the completion time (person correlation coefficient = -0.517, p = 0.000 < 0.001). There is a significant negative correlation between satisfaction and number of operation (person correlation coefficient = -0.365, p = 0.000 < 0.001). In addition, the participants previous experience would had a significant effect in satisfaction (F = 3.194, df = 3, p = 0.027 < 0.06).

4 Discussion Due to participants’ experience of multi-touch was different, the result in tasks varied. Participants with little experience on touchscreen interface used higher number of operations in the rotating tasks. On the other hand, in the moving tasks they needed only single-touch operation and used the lower number of operations. However, this phenomenon may be caused by laboratory equipment’s error of multi-finger contact points, and needed to be investigated and validated in future experiments. Based on the completion time, the C/R ratio with best performance of overall tasks and rotating tasks is 2/1. On the other hand, the C/R ratio with best performance of the moving tasks is 1/2. Thus, it suggests that different task may require different C/R ratio. This argument requires more precise experiments and data to be verified. The correlation analysis between satisfaction and performance shows the longer time to complete the task, or the more numbers of operations needed to complete the task would reduce satisfaction. As the previous result, there is no significant difference in overall satisfaction among levels of C/R ratio. But only in the moving tasks, there is significant difference in satisfaction among levels of C/R ratio. It suggests the C/R ratio is related to the completion time and number of operations in the moving tasks.

236

K.-H. Chen, C.-W. Chen, and W. Chen

5 Conclusions In this study, experiments were conducted with the rotating and moving tasks of manipulated levels of C/R ratio to understand the effects of C/R ratio on the participants’ performance and satisfaction. The results show that different types of operating task may require different C/R ratio. This study can provide a reference to touchscreen interface design, but further research and data need to be verified. In the future research the experiment and devices need more revision and planning. The types of task and the number of participants can be added more to obtain more accurate information to validate the findings, and to provide more relevant results for design and research reference. Acknowledgments. This study was partly sponsored with a grant, NSC99-2410-H182-031, from the National Science Council, Taiwan.

References 1. Sanders, M.S., Mccormick, E.J.: Human Factors in Engineering and Design, pp. 334–382. McGraw-Hill, Singapore (1992) 2. Stevenson, R., Phillips, J.G., Triggs, T.J.: Mouse and Display Tablets as Cursor Control Devices. International Journal of Pattern Recognition and Artificial Intelligence 18, 1221– 1232 (2004) 3. North, C., Dwyer, T., Lee, B., Fisher, D., Isenberg, P., Robertson, G., Inkpen, K.: Understanding multi-touch manipulation for surface computing. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, R.O., Winckler, M. (eds.) INTERACT 2009. LNCS, vol. 5727, pp. 236–249. Springer, Heidelberg (2009) 4. Fiorella, D., Sanna, A., Lamberti, F.: Multi-touch User Interface Evaluation for 3D Object Operation on Mobile Devices. Journal on Multimodal User Interfaces 4, 3–10 (2010) 5. Yee, W.: Potential Limitations of Multi-touch Gesture Vocabulary: Differentiation, Adoption, Fatigue. In: Human-Computer Interaction: Novel Interaction Methods and Techniques, pp. 291–300. Springer, Heidelberg (2009) 6. Esenther, A., Wittenburg, K.: Multi-user Multi-touch Games on DiamondTouch with the DTFlash Toolkit. In: Maybury, M., Stock, O., Wahlster, W. (eds.) INTETAIN 2005. LNCS (LNAI), vol. 3814, pp. 315–319. Springer, Heidelberg (2005)

Multi-touch Table as Conventional Input Device Andreas Dippon1, Florian Echtler2, and Gudrun Klinker1 1

Technische Universität München, Dept. of Computer Science Boltzmannstr. 3, 85748 Garching, Germany {dippona,klinker}@in.tum.de 2 Munich University of Applied Sciences, Dept. of Computer Science Lothstr. 34, 80335 München, Germany [email protected]

Abstract. In order to improve the functionality of multi-touch devices, the possibility of using them as input devices for other computers needs to be reviewed. The idea is, to get rid of many different peripherals (e.g. keyboard, mouse, multi-touch pad) by using a single multi-touch display. Furthermore the display can be used as an additional monitor to show for example toolbars, which can be directly manipulated through multi-touch gestures. We implemented a prototype, which provides an adaptive keyboard that shows shortcut keys for different applications, a multi-touch pad as well as the option to drag&drop standard Windows widgets onto the multi-touch table, which can be controlled by direct touch input. A user study was conducted to test the current system and to get information about the further approach to this concept. Keywords: multi-touch, input device, libTISCH, virtual keyboard.

1 Introduction The idea of our project is, to bring big multi-touch screens into daily office life in the future. Instead of a normal desk, a multi-touch screen is used. The screen is used as an input device for a normal desktop computer and replaces all conventional input devices. A keyboard can be shown, as well as a multi-touch pad, a drawing table, a piano, a mixing desk and so on. At the same time it can be used as an additional monitor, so that programs and windows can be dragged onto the screen and be controlled with direct touch input. Several advantages are depicted in the following list: ─ changeability: you can simply change the shown virtual input devices, without having to rearrange your workspace. Therefore you can switch easily between different tasks, always using the appropriate input device. ─ adaptivity: the virtual input devices can adapt to the current language, program, etc. For example, the virtual keyboard can show the correct keyboard layout for different languages, or provide the user with additional information (e.g. when pressing the ctrl key in a program, the shortcuts of functions are shown on the keys). ─ scalability: the virtual input devices can be rescaled to meet each user's demands. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 237–241, 2011. © Springer-Verlag Berlin Heidelberg 2011

238

A. Dippon, F. Echtler, and G. Klinker

Yet there are currently several drawbacks: ─ haptic feedback: currently, multi-touch screens don't provide any haptic feedback, which makes it harder to write on an onscreen keyboard without looking at it, than on a physical keyboard. ─ size and costs: big, flat multi-touch screens are still very expensive and therefore not applicable as standard office desks. After reviewing related work, we describe our prototype implementation and then show the results of the conducted user study.

2 Related Work The inverse approach of augmenting keyboards with touch sensitivity and small displays in each key was already presented by several companies and researcher. A first commercial product with small displays in each key was presented with the Optimus Maximus Keyboard by Art Lebedev Studio[1]. Here, the keyboard layout can be changed via software and can show different languages or program related icons. The Touch-Display Keyboard project by Block et al.[2] improves this concept by adding touch sensors to each key and using the keyboard as an additional display for the computer. A quite similar project is the Adaptive Keyboard from Microsoft, which was used in the Student Innovation Contest at UIST 2010[3]. The touch sensitivity of the keys is removed in this keyboard, but an additional small multitouch area is added on top of the keyboard. The keys are transparent and the screen which is used for the multi-touch area continues under the keys, so it can be used to display different layouts on the keyboard. Another related project, which deals with the idea of combining a virtual desktop with a real one, is the Curve Project by Wimmer et al.[4]. They constructed a prototype for an interactive desktop, based on the concept of the DigitalDesk by Wellner[5]. The Curve blends a horizontal and a vertical interactive surface, which takes existing ergonomics research into account.

3 Prototype Implementation In order to test the introduced concept, a prototype was implemented on a multi-touch screen. For the multi-touch support we used the libTISCH library by Echtler[7]. The prototype features an adaptive software keyboard, a multi-touch pad and an area which can be used for direct touch input. Direct Touch The concept of direct touch input is, that users can drag windows or toolbars onto the multi-touch screen and use their fingers instead of a mouse or a touchpad. For example, the user can navigate through the file system or change colors in a graphics editor by directly touching the according buttons.

Multi-touch Table as Conventional Input Device

239

MTPad The implementation of the multi-touch pad (or MTPad) in our prototype is capable of moving the mouse cursor, left and right clicking and scrolling by using multiple fingers. For convenience, the MTPad looks like a standard touchpad of a notebook. The MTPad also takes advantage of the large area of the multi-touch display. Other than on a standard touchpad, the user doesn't have to move the cursor in several small steps, because he can start on the touchpad and just keep moving his finger on the multi-touch display till the cursor has reached the desired position. This effect doesn't interfere with the direct touch functionality, because each tracked touch is always related only to the area where it was sensed first. Keyboard The most important aspect of our concept is the software keyboard. To keep the adaption level of the user low, we decided to use the shape of a conventional keyboard, instead of testing new designs. The implemented keyboard supports different key layouts and program specific shortcut icons can be shown for several programs: Inkscape1, Windows Calculator and Windows Notepad (see figure 1).

Fig. 1. The adaptive keyboard with the shortcuts of Inkscape while pressing CTRL

4 User Evaluation In order to get a first impression of how people react to the introduced concept, a small user study was conducted to get some information about the acceptance of the system. Our first approach was to do a study of the entire concept. Although the outcome of such a test would be very valuable, there are several problems conducting the test. In order to test the concept for everyday work, the hardware would have to be integrated into a normal desk, such that the test person could sit and work there normally. Furthermore, like in the test conducted by Wigdor et al. on Living with a tabletop[8], the test subject would have to use the system for a long time to get used to it and in order to get significant test results. Instead we decided to conduct a shorter study about the implemented keyboard functions. The goal was to get information about the utility of the adaptivity of the 1

http://inkscape.org/

240

A. Dippon, F. Echtler, and G. Klinker

keyboard. The setup of the study consisted of a self-built 40” FTIR multi-touch table as the horizontal display and a vertical 42” TV as the main monitor. In the study, the users had to do two exercises in Inkscape, which were designed to first get in touch with the system and then focusing on the utility of overlaid shortcuts on the keyboard. In the first task, the users had to change the color of different objects by using direct touch on the color toolbar of the program, which could be dragged onto the multi-touch screen. In the second task, the users had to solve simple riddles by using the shortcuts of a set of commands on a set of objects, to get a certain object. On the left side of the TV screen the set of objects the users had to combine was shown, in the middle the set of commands they should use and on the right the object they should create. All shortcuts of Inkscape were visualized on the corresponding keys on the keyboard during the tests. After the tasks, the users participated in an interview about their impressions of the system. Eight users participated in the study, mostly male computer science students with an average age of 28. More than half of the users thought that the adaptive keyboard is very useful. Especially the shortcut icons were considered very useful in order to get used to non frequently used shortcuts. Two users also mentioned that they could imagine separate keyfields containing the shortcuts of a program. Especially the participants, who are frequently using programs with additional toolsets, like graphics editors, game editors or electronic layout editors, could imagine, that the use of direct touch input would improve their efficiency at working with those programs. Nearly all users stated that they could imagine using the system without haptic feedback for work where they don’t have to type longer texts, but for writing or programming they would prefer a keyboard with haptic feedback. More than half of the participants also mentioned that another important aspect that needs to be improved is the sensitivity and calibration of the system, so that unintended touches or mouse clicks are not triggered.

5 Future Work Adaptive Keyboard: The shortcut icon concept could support more programs and additional language layouts could be implemented. Another aspect is to make the keyboard more flexible, so that each user can define the shape individually (e.g. for users who prefer natural keyboards). The next evaluation step for the shortcut icons could be the comparison between different layouts of the shortcuts (e.g. ordered by menu appearance, groups of similar functions, etc.) and also the comparison between shortcuts on the keyboard versus shortcuts on additional buttons. Multi-touch input: A big challenge in the future will be the support of multi-touch gestures in standard programs. The task here lies in finding fitting gestures to control programs by multi-touch gestures rather than just mapping the gestures to conventional commands. Concept: The whole concept of a multi-touch desk in addition to the normal computer screen could be evaluated in a long term study, so that the participants get used to the system in their daily work, where they could eventually identify any drawbacks of the system.

Multi-touch Table as Conventional Input Device

241

6 Conclusion In our research, the concept of using multi-touch surfaces as input devices for other computers has been introduced. By using a multi-touch surface, which is built into a standard desk, conventional input devices could be replaced through this concept. In order to get a first impression of the usability of the concept, a prototype was implemented. In the prototype, an adaptive keyboard, a multi-touch pad and a direct touch area was realised. The adaptive keyboard featured the visualization of shortcut icons for several programs. The prototype was then evaluated in a small user study to gather information about problems of the concept and also to get suggestions for improving the system.

References 1. Art Lebedev Studio, http://www.artlebedev.com/everything/optimus/ 2. Block, F., Gellersen, H., Villar, N.: Touch-Display Keyboards: Transforming Keyboards into Interactive Surfaces. In: CHI 2010: Proceedings of the 28th International Conference on Human Factors in Computing Systems, pp. 1145–1154. ACM, New York (2006) 3. UIST, Student Innovation Contest (2010), http://www.acm.org/uist/uist2010/Student_Contest.html 4. Wimmer, R., Hennecke, F., Schulz, F., Boring, S., Butz, A., Hußmann, H.: Curve: Revisiting the Digital Desk. In: NordiCHI 2010: 6th Nordic Conference on HumanComputer Interaction, ACM, New York (2010) 5. Wellner, P.: The DigitalDesk Calculator: Tangible Manipulation on a Desk Top Display. In: UIST 1991: Proceedings of the 4th annual ACM Symposium on User Interface Software and Technology, pp. 27–33. ACM, New York (1991) 6. Bi, X., Grossman, T., Matejka, J., Fitzmaurice, G.: Magic Desk: Bringing Multi-Touch Surfaces into Desktop Work. In: CHI 2011 Conference Proceedings ACM Conference on Human Factors in Computing Systems (2011) (in Press) 7. Echtler, F., Klinker, G.: A Multitouch Software Architecture. In: Proceedings of NordiCHI 2008, pp. 463–466 (2008) 8. Wigdor, D., Perm, G., Ryall, K., Esenther, A., Shen, C.: Living with a Tabletop: Analysis and Observations of Long Term Office Use of a Multi-Touch Table. In: Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer Systems (2007)

Properties of Shadow-Cursor for Calibrating Screen Coordinates of Tabletop Displays Makio Ishihara1 and Yukio Ishihara2 1

Faculty of Information Engineering, Fukuoka Institute of Technology, 3-30-1, Wajiro-Higashi, Higashi-ku, Fukuoka, 811-0295 Japan 2 Innovation Center for Medical Redox Navigation, Kyushu University, 3-1-1, Maidashi, Higashi-ku, Fukuoka 812-8582, Japan [email protected], [email protected]

Abstract. This manuscript introduces the basic idea of a shadow-cursor for calibrating the screen coordinates of tabletop displays. The shadow-cursor is an invisible mouse cursor. We explain how the shadow-cursor works to calibrate the screen coordinates and discuss its properties of accuracy, and then show that the shadow-cursor has the potential to align the screen coordinates. Keywords: shadow-cursor, screen calibration, pointing interfaces, tabletop displays.

1 Introduction Human computer interaction or HCI becomes more important in helping people interact with computers more intuitively and implicitly. In the field of HCI, tabletop displays have been studied recently and become one of interesting challenges for application and interaction style. A tabletop display is a horizontal display which multiple users sit down around and use together. A tabletop display is most likely equipped with a touch screen so that the users can click on and drag contents on the screen in an interactive way. The contents could be water surface, a pool table, a satellite map etc. Nowadays tabletop displays are often used for collaborative work such as brainstorming and meetings where multiple people work together around a single tabletop display. This trend leads to a strong demand for a large screen space. A pointing interface is a system that locates the spot on a computer screen, which the user wants to focus on. One of key issues around a tabletop display is how to make the pointing interface if the tabletop display has a large screen so that the user cannot reach the contents by his/her own hands. In research, the trend is that the pointing interface functions by capturing the quadrilateral of the screen with a handheld camera and locating the center of the camera as a graphical cursor. Nintendo Wii takes this approach for pointing. Infrared LEDs are attached on a TV screen, which represent the quadrilateral of the screen. The Wii remote captures the LEDs and locates the graphical cursor in real time. This way works well as far as the camera captures the quadrilateral of the screen. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 242–246, 2011. © Springer-Verlag Berlin Heidelberg 2011

Properties of Shadow-Cursor for Calibrating Screen Coordinates of Tabletop Displays

243

One of solutions to this is to use Gray-code. Gray-code is a way to encode values 0 up to 2n-1 into n bits binary sequences with the property that only one bit changes between any two consecutive sequences. A gray-code binary pattern is a visual representation of the binary sequences at a specific bit. In our previous work [1], we used black and white gray-code binary patterns. They appear on the screen in order then the temporal pattern captured by a handheld camera at a position of the screen uniquely represents its own position. Thus it is not necessary that the handheld camera captures the entire screen but one position. In an environment of projection, Lee, J. et al. [2] proposed a method for making the visible gray-code binary patterns imperceptible by transmitting the gray-code binary patterns through frequency-shift keying or FSK modulation. They later made a prototype of a projection system that is capable of emitting both infrared and visible rays simultaneously. This way is applicable to tabletop displays if their screens are projection. Our pointing interface uses a common mouse so that it relies on neither the projection of infrared rays nor the quadrilateral of the screen. One of major problems with the use of a common mouse for pointing on a tabletop display is that the coordinates of the screen (hereafter the screen coordinates) are not always aligned with the coordinates of the mouse control space (hereafter the mouse control coordinates). Wigdor, D. et al. [3] showed that the correlation between display positions and mouse control orientations would have a significant impact on performance. In our first attempt to align the screen coordinates with the mouse control coordinates [4], we used a reflex in eye-hand coordination. The reflex in eye-hand coordination is a natural response to inconsistency between kinetic information of a mouse and visual feedback of the mouse cursor. We showed that the reflex depends on the angular distance between the screen coordinates and mouse control coordinates, and the reflex is also viable to align the screen coordinates with the mouse control coordinates. It however takes 4 seconds to complete the alignment. To improve the performance, we take another approach with the basic idea of a shadow-cursor in this manuscript. When we use a mouse, we track the mouse cursor at the same time to make sure that the mouse cursor is placed where we want. If we

Fig. 1. Screen coordinates calibration with shadow-cursor

244

M. Ishihara and Y. Ishihara

do not see the mouse cursor while moving the mouse, how accurately can we move the mouse cursor? Moving the mouse cursor accurately without seeing it plays an import role in aligning the screen coordinates with the mouse control coordinates. Here we call this invisible mouse cursor a shadow-cursor. We describe how it works in a later section. The aim of this manuscript is to conduct an experiment on accuracy of the movement of shadow-cursor and show that the shadow-cursor has the potential to align the screen coordinates. Section 2 introduces the basic idea of a shadow-cursor and describes how the shadow-cursor aligns the screen coordinates with the mouse control coordinates. Section 3 conducts an experiment on accuracy of the movement of shadow-cursor and discusses its results, and then shows that the shadow-cursor has the potential to align the screen coordinates. Section4 gives the concluding remarks.

2 Shadow-Cursor A shadow-cursor is an invisible mouse cursor. Figure 1 shows how the shadow-cursor works to obtain the angular distance between the screen coordinates and mouse control coordinates. In the figure, there is an angular distance of 30 degrees counterclockwise between these two coordinates. (1) An arrow appears on the screen and (2) the user moves the mouse in the direction indicated by the arrow and then (3) the mouse cursor will move at an angle of x degrees. In this case, x equals 30. Note that the mouse cursor is hopefully set invisible because it will move in the direction that the user doesn’t expect. In order to obtain the correct angular distance, it is necessary for the user to manipulate the mouse in the same direction as the arrow without seeing any other visual feedbacks. We call this a shadow-cursor. In the next section, we conduct an experiment on some properties of the shadowcursor and give an answer to how accurately we can move the shadow-cursor.

3 Experiment This section conducts an experiment on accuracy of the movement of shadow-cursor and shows that the shadow-cursor has the potential to align the screen coordinates. 3.1 Design We conduct an experiment on how accurately we can move the mouse without seeing any visual feedbacks. Figure 2 shows the design of the experiment. The rectangle represents a laptop screen of Dell XPS M1210, which is 261mm wide and 164mm high (1280x800 pixels in resolution). The laptop has a processor of 1.73GHz. It is also equipped with a graphics card of nVidia GeForce Go 7400. The subject sees the laptop screen right in front. A radial arrow appears in the center at each angle of -150 to 180 at 30 degrees intervals and the subject is asked to move the mouse in that direction, and then the system records the movement of shadow-cursor. There were 16 subjects between the ages of 21 and 23. All were right-handed, had an experience working with a mouse. Each subject had 10 trials at each angle, resulting in 120 trials.

Properties of Shadow-Cursor for Calibrating Screen Coordinates of Tabletop Displays

245

Fig. 2. Design of an experiment on accuracy of the movement of shadow-cursor

Fig. 3. Effect of radial arrow’s angles on accuracy

×

×

=

16 subjects 12 angles of radial arrow 10 trials 1920 total trials To avoid learning effects, the sequence of trials is in a random order but it is the same between subjects. 3.2 Effect of Radial Arrow’s Angles on Aaccuracy Figure 3 shows the effect of radial arrow’s angles on accuracy. The horizontal axis shows the radial arrow’s angles and the vertical one shows the subtraction of the observed angle from the target one at each angle across all the subjects. From the figure, there seems to be a certain relation between the radial arrow’s angles and accuracy. There are two increases around +/- 30 to 60 degrees, one subtle decrease in the center, and two drastic decreases at both sides. Analysis of variance shows that there is a significant impact of the radial arrow’s angles on accuracy [F(11,1908)=4.54074 at p=0.001]. The accuracy is however kept within +/- 4 degrees. 3.3 Effect of Individuals on Accuracy Figure 4 shows the effect of individuals on accuracy. The horizontal axis shows all the subjects and the vertical one shows the subtraction of the observed angle from the target one at each subject across all the arrow’s angles. From the figure, there seems to be a certain change in accuracy between individuals. The 11th and 12th subjects

246

M. Ishihara and Y. Ishihara

Fig. 4. Effect of individuals on accuracy

had worse accuracy than the others. Analysis of variance shows that there is a significant impact of individuals on accuracy [F(15,1904)=4.906141 at p=0.001]. The accuracy is however kept within +/- 4 degrees.

4 Conclusions This manuscript introduced the basic idea of a shadow-cursor for calibrating the screen coordinates of tabletop displays and discussed its properties of accuracy, and then showed that the shadow-cursor had the potential to align the screen coordinates with the mouse control coordinates. In our future work, we are going to implement a prototype of our pointing interface and evaluate its performance.

References 1. Ishihara, M., Ishihara, Y.: An approach to remote direct pointing using gray-code. In: AVI 2006: Proceedings of the working conference on Advanced visual interfaces, pp. 75– 78. ACM, New York (2006) 2. Lee, J., Hudson, S., Summer, J., Dietz, P.: Moveable interactive projected displays using projector based tracking. In: Proc. UIST 2005, pp. 63–72. ACM, New York (2005) 3. Wigdor, D., Shen, C., Forlines, C., Balakrishnan, R.: Effects of display position and control space orientation on user preference and performance. In: CHI 2006: Proceedings of the SIGCHI conference on Human Factors in computing systems, pp. 309–318. ACM Press, New York (2006) 4. Ishihara, M., Ishihara, Y.: Calibrating coordinates of a tabletop display with a reflex in eyehand coordination. IEICE Trans. on Information and Systems e93-d(10), 2862–2865 (2010)

Emotional Expression by a Person’s Grip on a Tactual Communication Tool Yasuhiro Matsuda and Tsuneshi Isomura Kanagawa Institute of Technology, Faculty of Creative Engineering, 1030 Shimo-ogino, Atsugi-shi, Kanagawa, Japan [email protected] Abstract. Tactual communication between people familiar to each other can express various emotions. But a social distance exists between unfamiliar people. The purpose of this study is the development of a tactual communication tool that conveys emotional communication during an oral conversation. In the present study, a tactual communication tool was examined, and the features of emotional expression by a person’s grip on the tool were analyzed. Ten subjects expressed six emotions by their grip. As a result, surprise and joy were expressed by one grip pattern, and anger and sadness by two grip patterns. Disgust and fear were expressed by different grip patterns. Keywords: Emotional communication, tactual communication, communication tool.

1 Introduction Elderly persons develop communication disorders in varying degrees by aging. Some of elderly persons (e.g. the hospitalized or institutionalized persons) have difficulties not only of verbal communication but also of emotional (nonverbal) communication. Tactual communication (interpersonal touching) is the most basic and primitive form of communication. Tactual communication between people familiar to each other can express various positive emotions. But a social distance exists between unfamiliar people, and we have a tendency to avoid the touch of these people. Recently, some tactual (haptic) communication systems have been developed. Ambe introduced a haptic communication system in interpersonal informal telecommunication [1]. The purpose of our study is the development of a tactual communication tool that conveys emotional communication during face-to-face oral conversation (e.g. a conversation between the elderly person and the institution staff). We designed the tactual communication tool with the concept of "using tactual communication but keeping a social distance" [2]. The tool consists of two soft rubber balls and a hose. A hole was made on the surface of each ball (diameter: 60 mm) so that both balls could be connected by the hose (inner diameter: 6 mm, length: 1200 mm). The interfaces between the balls and the hose were sealed by adhesive bond. In the experiment, two users each hold one ball. When one ball is gripped, air is pumped out of that ball and into the other. Thus, a user can grip the ball in different ways during an oral conversation, and the other user can recognize the different pressures of the ball (see Fig. 1). In a previous study, we analyzed the effectiveness of this communication tool C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 247–251, 2011. © Springer-Verlag Berlin Heidelberg 2011

248

Y. Matsuda and T. Isomura

during oral communication [2]. The results showed that the subjects mainly used the tool to respond to the grip and talk by the other subject and intended to emphasize their talk when they expressed emotions. In the present study, to analyze the features of emotional expression by using the tactual communication tool, the emotion expressed by the person’s grip on the tool was examined. Air pressure change sensor

Fig. 1. Tactual communication tool (left) and oral communication using the tool (right)

2 Methods The subjects were ten male college students. All subjects gave their informed consent after hearing a description of the study. The expressed emotions were the fundamental emotions of human beings: joy, anger, sadness, disgust, surprise and fear. We set six scenes for the subjects to express their emotions. The experiment was conducted for two days. On each day, three experimental sessions were conducted. In each experimental session, the subject expressed six emotions in a predetermined random order. The subject held the tactual communication tool with his fingers and palm totally contacting the soft rubber ball, so that the main hose was positioned between the thumb and index finger. In the previous study, this was the most frequent holding form for subjects showing their emotions with the tool [2]. The subject expressed an emotion by gripping the soft rubber ball for 15 seconds or less. The hose had a third branch in the center. This branch was connected to an air pressure change sensor (FKS-111, Fuji ceramics) (see Fig. 1 left). The air pressure change sensor and a high-speed video camera (DEWE-CAM, Dewetron) were connected to a data logger (DEWE-800, Dewetron). The data logger measured the air pressure change as a result of the grip and simultaneously recorded the image of the grip. The measurement range of the air pressure change was 1 Pa–133 kPa and the sampling frequency was 10 kHz. The image was 640×480 pixels and the sampling frequency was 70 Hz.

3 Results Fig. 2 shows the air pressure changes as a result of the grip. We calculated the grip duration and the maximum air pressure change. The mean of the grip duration as a

Emotional Expression by a Person’s Grip on a Tactual Communication Tool

249

function of emotions is shown in Fig. 3. An analysis of variance (ANOVA) revealed the significant main effects of emotion (F(5,354)=39.1, p<.001). The mean of the maximum air pressure change as a function of emotions is shown in Fig. 4, and the ANOVA revealed the significant main effects of emotion (F(5,354)=60.7, p<.001). The results of Scheffe tests on the emotions are indicated by the asterisks in Figs. 3–4.

Fig. 2. Air pressure changes as a result of the grip. The left side shows the long one-time grip (grip and hold 2 seconds, then release). The right side shows multiple grips with constant short intervals (grip 5 times for a total of 2 seconds).

Fig. 3. The mean of the grip duration as a function of emotions

Next, we observed the images of the grip and classified the grip technique and the pattern of the grip cycle into five types each. The grip technique was classified as a quick or a slow clench (the index, middle, ring and little fingers curling into the palm), a quick or a slow grasp (the index, middle, ring and little fingers do not touch the palm), and a combination of the two. Fig. 5 shows the percent distribution of the grip techniques as a function of emotions. The pattern of the grip cycle was classified into multiple grips with constant long or short intervals, a long one-time or a short one-time grip, and a combination. Fig. 6 shows the percent distribution of the patterns of the grip cycle as a function of emotions.

250

Y. Matsuda and T. Isomura

Fig. 4. The mean of the maximum air pressure change as a function of emotions

Fig. 5. The percent distribution of the grip technique as a function of emotions

Fig. 6. The percent distribution of the pattern of the grip cycle as a function of emotions

4 Discussion According to the experimental results, the subjects expressed joy and surprise by one grip pattern, anger and sadness by two grip patterns, disgust by three grip patterns and fear by five grip patterns. The features of emotional expression are as follows.

Emotional Expression by a Person’s Grip on a Tactual Communication Tool

251

Joy was characterized by a quick clench technique, and the pattern was multiple grips with constant short intervals for an average of 8 grips in 6 seconds. Surprise was characterized by a quick clench, and a strong and short one-time grip for an average of 1 second. Anger was characterized by the following: (1) a quick clench, and a long one-time grip for an average of 7 seconds; or (2) a quick clench, and multiple grips with constant long intervals for an average of 6 grips in 10 seconds. Sadness was characterized by the following: (1) a quick or slow grasp, and weak grips with constant short intervals for an average of 3 grips in 9 seconds; or (2) a slow clench, and a normal strength and long one-time grip for an average of 13 seconds. Disgust was characterized by the following: (1) weak grips with constant long intervals for an average of 4 grips in 8 seconds; (2) weak grips at first, then gradually stronger grips with constant short intervals for an average of 11 grips in 8 seconds; or (3) a strong and long one-time grip in an average of 9 seconds. Fear was characterized by the following: (1) a slow grasp, a weak and long onetime grip for an average of 9 seconds; (2) a quick grasp, multiple grips with constant long intervals for an average of 4 grips in 7 seconds; (3) a quick grasp, weak grips with constant short intervals for an average of 38 grips in 9 seconds; (4) a quick clench, a short one-time grip at first, then multiple grips with constant short intervals for an average of 16 grips in 8 seconds; or (5) a quick clench, a strong and short onetime grip at first, then strong grips with constant short intervals for an average of 20 grips for 7 seconds.

5 Conclusion In the present study, emotional expression using a tactual communication tool was examined, and the features of emotional expression were analyzed by a person’s grip on the tool. As a result, surprise and joy, and anger and sadness, were expressed by one and two common grip patterns, respectively. Disgust and fear were expressed by different grip patterns. We are planning to examine the features of emotional communication by using this tool.

References 1. Ambe, M., Ohmura, K.: The changes of the structure of informal tele-communication with haptic communication channel. Correspondences on Human Interface 2(3), 53–58 (2000) 2. Matsuda, Y., Shimbo, T., Isomura, T.: Tactual Communication Tool for Emphasizing Emotional Communication. J. Rehabil. Med. Suppl. (48), 162 (2010)

Effect of Target Size and Duration of Visual Feedback on Touch Screen Jeeyea Park and Kwang-Hee Han Department of Psychology Yonsei University, Sinchon-dong, Seodaemun-gu, Seoul, Korea [email protected], [email protected]

Abstract. This study investigated performance and touching experience as a function of duration of visual feedback and target size on a touch screen. Five duration of visual feedback by three target size within-subject experiment was conducted. Relationship between performance and duration of visual feedback has an inverted-U shape trend. Performance and touching experience evaluation was worse with small targets. There was significant interaction between target size and duration of visual feedback. At the small target size condition performance was different from the other sizes. People could pay more attention to visual feedback because small size target condition was difficult. Another possible explanation is that the presented visual feedback size was very similar to small target condition. Longer lasting visual feedback might over complicate things for people and lead to confusion with the target. Keywords: Touch screen, Visual feedback, Duration, Size, Touching experience.

1 Introduction Touch interface is more popular in computer input device. What we need for communication with a computer is just our fingers. This direct control method reduces our cognitive load and overcomes an interface area limitation. But the lack of tactile feedback makes us feel uncomfortable during use. For this reason, visual and auditory feedback may be important factors for improving performance [1]. Bender evaluated the effect of the duration of auditory feedback and target size on touch screen. He found that performance with large target is better, and error rates with small target are reduced if auditory feedback between 50 and 400ms is provided. But there was no study about duration of visual feedback. Previous research found that visual feedback is particularly important for touch screen than other sense feedback [2]. The purpose of current study is that to find the effect of duration of visual feedback and target size. Moreover we measure subjective evaluation on about the touching experience to find that how these factors affect to user experience. We provided Ripples [3] (fig. 1) as a visual feedback, which enables visualizations around each contact point on a touch screen. They found that users prefer a system utilizing Ripples and this feedback significantly improve the performance. The feedback presentation lasted for 100, 250, 500, 1000, and 2000ms from lifting the finger. 500, 1000ms durations are commonly used in mobile device touch screens [4]. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 252–256, 2011. © Springer-Verlag Berlin Heidelberg 2011

Effect of Target Size and Duration of Visual Feedback on Touch Screen

253

Fig. 1. Ripples: when contact is lifted, the visualization shrinks to the hit testing point [3]

2 Method Participants. Twenty-four participants (8 females, mean age 21.8 years, ranging from 18 to 28 years) from Yonsei University took part in the experiment for class credit. Experimental design. This study was a 3 x 5 factorial, within-subject design. The independent variables were target size and duration of visual feedback. We used three size of target for making levels of task difficulty. And feedback duration variable had five levels: 100, 250, 500, 1000, and 2000ms from lifting the finger. The 15 conditions (15 blocks) were presented in a randomly determined order. Each block has three trials and the first trial was always a practice trial to familiarize the participant with the task-feedback condition. Apparatus. The task was programmed in Microsoft Visual Basic 6.0 and presented touch screen (Samsung Sync Master 743BX, 17”, 1280 x1024). Touch screen mounted at 30 degree from horizontal for lower participant fatigue [5]. Multi-Directional Tapping Test This test is one of Non-keyboard input device tests for usability test. The test procedure is based on ergonomic practices and testing methods and measures in the ISO 9241 VDT ergonomic standard [6]. A circular arrangement of 16 equally-spaced, equally-sized, white-colored targets aligned to the horizontal and vertical center of background. As a trial started, target circle 1’s color changed to red. Participant’s task is to touch the red circle as quickly and as accurately as possible. If the target was successfully selected, the color of circle turns white and next target circle (2) color is turns red simultaneously. The diameter of the arrangement was fixed at 23cm, but target circle size was varied between trials. We used 1.5cm, 1.96cm, and 2.4cm diameter circles for task difficulties [3].

Fig. 2. Multi-directional tapping task

254

J. Park and K.-H. Han

Measures. In the tapping task, we measured the time to complete one trial and number of errors during the trial. These measurements used for analysis of participant’s performance. After completing a one condition (=1 block: 3 trials include practice trial), participants were asked to respond to questionnaires about their touching experience. The AttrakDiff2 questionnaire was used to measure pragmatic quality and attractiveness of the various conditions of task interface [7]. And we also measured perceived usefulness and perceived ease of use [8]. All questionnaires were rated using 7-pointing scales. Table 1. Questionnaires for user experience evaluation Category Description Endpoints(7-point scales) Pragmatic quality I think this touch interface Technic/People-centric is… Impractical/Practical Confusing/Clear Attractiveness Ugly/Pretty Bad/Good Repulsive/Pleasing Perceived Using this touch interface Disagree/Agree usefulness would improve my job performance. Perceived ease of I would find it easy to get Disagree/Agree use this touch interface to do what I want to do.

3 Results Performance (response time, error rate) and experience evaluation rating were evaluated by repeated-measure ANOVAs and further analyzed by post hoc Bonferroni. The results were analyzed by a 3 x 5 analysis of variance for three size of target and five duration of visual feedback. Size of Target. The analysis indicated that target size main effects were all significant. Increasing size of target significantly improved task performance [RT: F(2, 46) = 19.25, p < .001, ηp2 = .46; Error Rate: F(2, 46) = 8.68, p < .001] (fig. 3). In the post hoc, when target size was 15.0mm, response time (M = 10626ms) was significantly longer than 19.6mm (M = 9689ms) and 24.0mm (M = 9147ms) (p < .001). And these result was shown in evaluation of experience too [Pragmatic quality: F(2, 46) = 16.51, p < .001, ηp2 = .42; Attractiveness: F(2, 46) = 8.87, p < .001, ηp2 = .28; Perceived Usefulness: F(2, 46) = 13.30, p < .001, ηp2 = .37; Perceived Ease of use: F(2, 46) = 14.87, p < .001, ηp2 = .39] (fig. 4). Duration of Visual feedback. Main effect of duration of visual feedback was close to the .05 level of significance [F(4, 92) = 2.21, p = .07]. Relationship between response time and duration of visual feedback has an inverted-U shape trend. Other measures had no main effect of duration of visual feedback [Fs < 1].

Effect of Target Size and Duration of Visual Feedback on Touch Screen

255

Fig. 3. Performance (black: error rate; white: Fig. 4. Evaluations of touching experiences response time) as a function of target size follow target size

G

Fig. 5. Mean response time as a function of Fig. 6. Mean response time as a function of duration of visual feedback target size x duration of visual feedback interaction

Interaction between Target Size and Duration of Visual feedback. The two-factor interaction of target size and duration of visual feedback was significant [F(5.6, 130.2) = 2.23, p < .05, ηp2 = .09]. When target size was 15.0mm, there is no difference in duration of visual feedback condition from 100 to 1000ms. But performance rapidly decreased at the 2000ms condition. This tendency is different from the other two target sizes (19.6, 24.0mm).

4 Discussion The result of the study revealed effect of duration of visual feedback and target size on touch screen. Performance and touching experience evaluation was significantly worse with small target size condition (d = 15.0mm) than others. In other words, this condition was the only difference on the level of task, so we can call this condition “difficult level”. Relationship between performance and duration of visual feedback had an inverted-U shape trend (p = .07). However, except a condition of small target size, performance was better at 100, 200, and 500ms duration of visual feedback. The result of the interaction between target size and duration of visual feedback was

256

J. Park and K.-H. Han

significant. There was different performance as a function of duration of visual feedback during the difficult level of task. This tendency more corresponds to the earlier report, “feedback should be succinct [9]”, in case a reference point in a duration condition is 1000ms. Other sizes of target (19.6, 24.0mm) conditions were relatively easy to complete the task, so the reference point could be varied as the difficulty of task. Another possible explanation was that people didn’t pay attention to the visual feedback, moreover this feedback might makes people get confused. One limitation of this study is that presented visual feedback size was very similar to small target condition. Longer lasting visual feedback could might over complicate things for people and lead to confusion with the target. We conclude that task difficulty could create a different effect of duration of visual feedback on touch screen. If we design duration of visual feedback, we should consider its task difficulty.

References 1. Bender, G.T.: Touch Screen Performance as a Function of the Duration of Auditory Feedback and Target Size. Doctor. Wichita State University (1999) 2. Kim, J.H.: Multimodal Feedback Usability Analysis on Touchscreen-Applied Mobile Device. Master. Hongik University (2009) 3. Wigdor, D., Williams, S., Cronin, M., Levy, R., White, K., Mazeev, M., Benko, H.: Ripples: Utilizing Per-Contact Visualizations to Improve User Interaction with Touch Displays. In: Proceedings of UIST 2009, Victoria, British Columbia, Canada, October 4-7 (2009) 4. Choi, W.S.: A Study on the User Interface Design of Touch Screen Mobile Phone. Master. Kookmin University (2007) 5. Sears, A.: Improving Touchscreen Keyboards: Design issues and a comparison with other device, vol. 3, pp. 252–269 (1991) 6. Smith, W.J.: ISO and ANSI Ergonomic Standards for Computer Products. Prentice Hall PTR, Englewood Cliffs (1996) 7. Schrepp, M., Held, T., Laugwitz, B.: The influence of hedonic quality on the attractiveness of user interfaces of business management software. Interacting with Computers 18, 1055– 1069 (2006) 8. Davis, F.D.: Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13(3), 319–340 (1989) 9. Salvendy, G.: Handbook of Human Factors and Ergonomics, 3rd edn. John Wiley & Sons, New York (1984)

Development of an Economical Haptic Stimulus Device Greg Placencia, Mansour Rahimi, and Behrokh Khoshnevis Daniel J. Epstein Department of Industrial and Systems Engineering Viterbi School of Engineering University of Southern California, Los Angeles, CA 90089 {placenci,mrahimi}@usc.edu

Abstract. Human perception of haptic information is often complex and requires devices that are expensive and difficult to operate. We see increasing attention being given to finding simpler motions in haptic research to represent more complex haptic perception on the surface of the skin. Our research has indicated that lateral motions in multiple directions may provide sufficiently useful perception for information transfer. We present a design for an economical haptic stimulation device using off-the-shelf parts that are relatively easy to obtain and assemble. Our design was configured for lateral motion, but can be modified to deliver a variety of haptic stimulations. Keywords: Haptics, Haptic Communication Tangential Motion, Tactile Communication.

1 Introduction The sense of touch has been increasing tapped as a mode of interaction. However most high end haptic technologies are prohibitively expensive such as the stylus based Phantom line (SensAble Technologies) and the CyberGrasp glove system (Immersion Corporation). More affordable devices have entered the market such as Novint’s Falcon, force feedback device; and several Samsung consumer devices provide better tactile sensations using multiple forms of vibrations (Fig.1) Samsung P3 Novint ® Falcon Samsung Anycall Unfortunately multimedia player Cell Phone such devices offer relatively low Fig. 1. Miscellaneous Haptic Devices. Reprinted by permission levels precision. Moreover no currently available device adequately generates lateral motions which the authors sought to explore as a means of transmitting information. To bridge this gap, we C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 257–261, 2011. © Springer-Verlag Berlin Heidelberg 2011

258

G. Placencia, M. Rahimi, and B. Khoshnevis

developed a haptic device that was simple to construct from easily obtained materials, yet was capable of millimeter precision. The result was relatively easy to construct, inexpensive, and stable, yet required little space. Moreover it provides a flexible platform that can be adapted to other forms of haptic stimulation in addition to lateral motion.

2 Developing the Lateral Motion Device 2.1 Device Design The testing device consisted of two acrylic plates suspended by four threaded steel rods. The top plate supported the subject’s hand, while the bottom plate held the testing mechanism. A hole was cut into the center of the top plate through which subjects’ index finger was stimulated. The hole supported adjustable cutouts to accommodate variable finger sizes. Subjects calibrated the normal force they used to press down on the probe during testing by using a plunger attached to a counterbalance. Details of device iterations are reported in (1). A round, 5/16 inch diameter nylon probe was used to stimulate the finger pad. A Sensirion SHT11 temperature/humidity sensor was mounted to the top platform to measure ambient temperature and humidity near the fingerpad. The bottom platform supported two high torque servomotors stacked perpendicularly. Gears attached to the servos moved toothed tracks attached to two linear rails to generate linear motion. A third servo could generate rotary motion but was not used for our experiments. A probe was attached to the top servo using a screw and connecting bolt (Fig. 2).

Fig. 2. Lateral Motion Test Device

We controlled and monitored all electronics via a custom programmed PBASIC PS2sx microcontroller and Parallax Servo Controller Interface (PSCI, ver. 1.4) using PBASIC 2.5.

Development of an Economical Haptic Stimulus Device

259

2.2 Surface Contact Considerations We were concerned that viscoelastic and anisotropic haptic properties of the human finger pad could cue subjects instead of the lateral motion (2 - 9). To counter these effects we used a smooth round 5/16 inch (~ 8 mm) diameter nylon probe (μs = 0.25– 0.5N) as the point of contact (Fig. 3.).

Plunger

Mounted Probe (top view)

SHT1x Sensor

Mounted Probe (side view)

Fig. 3. Final Lateral Motion Interface Details

This minimized friction effects in all direction while presenting a sufficiently large contact area (~ 6 mm diameter × 1.25 mm deep) to ensure adequate stimulation (2–4). In addition we asked subjects to remove (unload) their fingers between testing sequences; a break of approximately 20–60 seconds depending on the respondent. To ensure subjects used consistent forces while pressing down on the probe, we placed a 1 N (~ 102 g) normal force on the plunger. Dynamic fingertip friction coefficients (μd) for nylon at the fingertip have been measured as 0.45–0.6 (5). We therefore calculated a 0.45–0.6 N tangential force at the fingertip during test that roughly corresponds to the 0.5 N tangential force used in (6) as baseline perception. 2.3 Confounding Concerns Due to Residual Motion We were also concerned that accurate perception of the lateral stimuli would be confounded by residual motion caused by either the device or the user. To mitigate device concerns, bolts were attached to each rod so as to sandwich each plate rigidly. In addition, rubber pads were attacked to the bottom of each rod’s base to prevent slippage. During testing, subjects were strapped to the device to prevent excessive hand motion and their arms supported. The index finger of their predominant hand was also immobilized using cloth tape with only the fingerpad exposed. Subjects were also monitored to ensure they consistently and properly placed the middle of the fingerpad on the probe.

260

G. Placencia, M. Rahimi, and B. Khoshnevis

3 Evaluation Summary During our experiments we asked 62 subjects (28 male; 34 female) to judge the direction of lateral motions at the index fingertip of their dominant hand, for 0.5 and 1.5 mm displacement in one of eight directions, separated by 45º. Our results showed no statistical difference accurately perceiving the motion among these directions for 1.5 mm displacement. For 0.5 mm displacement though, subjects were more sensitive to motion in the cardinal directions (front, back, left, and right) along the axis of the distal phalange, than along the diagonals (7). These results were found to be the same regardless of gender or age in five categories, ranging from 18 to 65+. However overall perception accuracy dropped as subject age increased. Full experimental methodology and comprehensive results of our experiments are reported in (8).

4 Future Development The viability of our design for testing lateral skin stimulation has been demonstrated above. Our design is relatively simple and inexpensive to implement, while very adaptable. Our controllers allowed us vary stimulus velocity, acceleration and direction via a simple PBASIC program; and tangential force is controlled by simply changing the static weight applied by the counter-balanced plunger. Our device was also capable of rotary motion that we did not use during our experiments. Therefore, further testing using rotary motion is envisioned, as well as using distance and direction under varied skin conditions. In addition, it has been suggested that testing subjects under varied states of awareness such as while intoxicated or under prescription medication may provide insight into how tangential haptic perception varies by outside influences. Acknowledgment. The authors would like to acknowledge the invaluable help of Dr. Kurt Palmer in the design of experiment and statistical analysis of the complex data set.

References 1. Placencia, G.: Information Transfer Via Tangential Haptic Stimulation. PhD Thesis. University of Southern California, Los Angeles (2009) 2. Liao, J.-C.: Experimental Investigation of Frictional Properties of the Human Fingerpad. MIT Thesis (September 1998) 3. Tan, H.Z., et al.: Human Factors for the Design of Force-reflecting Haptic Interfaces. In: Proceedings of Winter Annual Meeting of the American Society of Mechanical Engineers: Dynamic Systems and Control, vol. 55, pp. 353–359 (1994) 4. Savescu, A., Latash, M.L., Zatsiorsky, V.M.: A Technique to Determine Friction at the Fingertips. Journal of Applied Biomechanics 24, 43–50 (2008) 5. Sivamani, R.K., et al.: Coefficient of friction: tribological studies in man – an overview. Skin Research and Technology, 227–234 (2003) 6. Biggs, S.J., Srinivasan, M.A.: Tangential versus normal displacement of skin: relative effectiveness for producing tactile sensation. In: Proceedings of IEEE VR (2002)

Development of an Economical Haptic Stimulus Device

261

7. Placencia, G., Rahimi, M., Khoshnevis, B.: Sensing Directionality in Tangential Haptic Stimulation. In: Harris, D. (ed.) HCII 2009. LNCS, vol. 5639, pp. 253–261. Springer, Heidelberg (2009) 8. Placencia, G., Rahimi, M., Khoshnevis, B.: A heuristic to capture multi-directional lateral tactile perception. Theoretical Issues in Ergonomics Science (2011) (accepted for publications) 9. Levesque, V., Hayward, V.: Experimental Evidence of Lateral Skin Strain during Tactile Exploration. In: Proceedings of Eurohaptics. IEEE, Dublin (2003) 10. Martinot, F., Plénacoste, P., Chaillou, C.: Haptic Sounds and Vibrations of Human fingerprints. In: International Conference on Sensing Technologies (2005) 11. Moy, G., et al.: Human Psychophysics for Teletaction System Design. The Electronic Journal of Haptics Research 1(3) (2000), http://www.haptics-e.org 12. Wang, Q., Hayward, V.: In Vivo Biomechanics of the Fingerpad Skin Under Local Tangential Traction. Journal of Biomechanics 40, 851–860 (2007) 13. Pataky, T.C., Latash, M.L., Zatsiorsky, V.M.: Viscoelastic response of the finger pad to incremental tangential displacements. Journal of Biomechanics 38, 1441–1449 (2005) 14. Dinse, H.R., et al.: Tactile Coactivation Resets Age-Related Decline of Human Tactile Discrimination. Annals of Neurology 60, 88–94 (2006) 15. Pan, L., Zan, L., Foster, F.S.: Ultrasonic and viscoelastic properties of skin under transverse mechanical stress in vitro. In: IEEE Ultrasonics Symposium, vol. 2, pp. 1087– 1091 (1997) 16. Cua, A.B., Wilhelm, K.-P., Maibach, H.I.: Frictional Properties of Human Skin: Relation To Age, Sex and Anatomical Region, Stratum Corneum Hydration and Transepidermal Water Loss. British Journal of Dermatology 123, 473–479 (1990) 17. Drewing, K., et al.: First Evaluation of A Novel Tactile Display Exerting Shear Force via Lateral Displacement. ACM Transactions on Applied Perception 2, 118–131 (2005) 18. Vitello, M., Drif, A., Giachritsis, C.D.: Final Evaluation Report for Haptic Displays with Guidelines. TOUCH-HapSys. Towards a Touching Presence: High-Definition Haptic Systems (2006)

Feeling Home – Tangible Information Visualization in Smart Home Environments in Relation to the Concept of Transhumanism Florian Weingarten and Sahin Albayrak DAI-Labor, Technical University of Berlin, Ernst-Reuter-Platz 7 10587 Berlin, Germany {florian.weingarten,sahin.albayrak}@dai-labor.de

Abstract. Current trends in the area of smart and interactive environments, such as future homes, show a dramatically increasing number of networked appliances, electronic devices and sensors and a coherently rising amount of information and data flows. Instead of conveying this digital information only through a conventional graphical user interface, we propose a more subliminal way, following the human perception in the physical world. We use haptic feedback to make gathered data at home tangible for inhabitants and therefore extend their human capacities of perceiving otherwise invisible information. Keywords: Human Computer Interaction, Smart Environment, Smart Home, Tactile Feedback, Transhumanism.

1 Introduction Todays understanding of smart homes still coincides with the scenarios that have been conceived since the term ‘Smart Home’ has first been coined, approximately three decades ago. These homes have always been envisioned to offer inhabitants assistance while performing household chores and other everyday activities and therefore improve their quality of life. One thing that changed over the years is the increasing clarity on how to make the vision technically feasible and of how to make home automation and networking solutions available for a broad audience. The driving force behind this is the rapid evolution of the computer and its ongoing shift from a production tool to a ubiquitous companion in everyday life. While approaching the realization of smart home visions on a technical basis, new challenges arise in terms of how to interact with such an increasingly complex environment and its possibilities. Based on our previous and related work, we summarize the nature and components of smart home environments (Chapter 2), subsequently emphasize on information provision with a special focus on the ideas of transhumanism (Chapter 3), followed by a use case including a device prototype for tangible information visualization (Chapter 4).

2 Smart Home Environments Current trends in the area of smart and interactive environments, such as future homes, show a dramatically increasing number of networked artifacts like appliances, C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 262–266, 2011. © Springer-Verlag Berlin Heidelberg 2011

Feeling Home – Tangible Information Visualization in Smart Home Environments

263

electronic devices and sensors that can exchange data via various technologies and protocols. This emerging accumulation of interconnected artifacts is basically forming the smart environment as a distributed system [1]. Its potential lays in the interplay and dependencies of sensors and actuators. Therefore the range and accuracy of the envisioned assistance of inhabitants corresponds to the number and compatibility of networked artifacts, especially when the support is supposed to be context-sensitive. 2.1 Smart Home Artifacts Interconnected artifacts that represent the smart home can principally be assigned to one of two groups: sensors or actuators. • Sensors gather all sorts of information of the environment for example temperature, air quality, motion or light intensity. The amount of available information subsequently increases with each sensor. • Actuators are devices or appliances that can be controlled either automatically by the system or manually by the inhabitant. Considering the resulting heterogeneity of smart home networks, missing standards and protocols are of course a huge issue that needs to be addressed [2]. In this paper we leave the technical challenges out of scope and focus on the challenges of how to make those networks accessible and comprehensible for the inhabitants. 2.2 Home Operating System The availability of interconnected artifacts constantly producing data demands a runtime-environment, which handles the communication between all artifacts and provides access to their functions and related information. This idea has been pursued within several research projects like Service Centric Home (SerCHo, www.sercho.de) [3], Amigo (www.hitech-projects.com/euprojects/amigo/) or Smart Senior (www.smart-senior.de). Related to SerCHo, we previously showed an approach of how to simply control a complex and growing network of independent artifacts as well as how a large number of information flows can be managed through a user interface (UI) of a home operating system [1]. While the system primarily utilizes a graphical user interface with touch-based interaction to provide access to available information, we focus on its conveyance within the smart home in chapter 3, especially dealing with information that cannot be perceived autonomously by the human body. 2.3 Smart Home Interaction and Feedback Available solutions for home automation and home networking mostly utilize graphical user interfaces (GUI), displayed on a television screen, centrally located touch panels and recently also on mobile devices like phones or tablet computers. Especially related to those small devices, much research has been done in the field of tactile and audio feedback, revealing that multimodal feedback can improve the performance over only visual interfaces [4] [5]. Which feedback modality suits best highly depends on the context of the performed activity, and therefore should be

264

F. Weingarten and S. Albayrak

flexible in terms of mixing single modalities [6]. As information on the state of artifacts and device monitoring in general is one of the central aspects of smart home scenarios, multimodal feedback mechanisms can as well be used to communicate such data to the inhabitant. Making information tangible can help to convey data without the distraction of pro-actively accessing a computer system through a GUI.

3 Tangible Information Visualization Haptic feedback can help to make gathered data at home transparent for inhabitants and, in the context of transhumanism, extend their human capacities of perceiving otherwise invisible information. A more diffused and subtle transfer also offers possibilities to experience concealed activities at home beyond the common and limited way but without being distractive. Therefore, a physical prototype is used to convert an intangible value into a haptic signal pattern, which is conveyed to the inhabitant through direct body contact. Thus it is possible for inhabitants to actually feel the information surrounding them at home, like the amount of energy that is consumed by devices, network activities or simply if a letter was dropped into the mailbox. By uncovering such information commonly residing in the background in the described way, we blur the boundaries between the inhabitant and the smart home itself, making such complex environments more comprehensible, beneficial and enjoyable. 3.1 Transhumanism As stated by Bostrom, the movement of transhumanism is loosely defined. It promotes an interdisciplinary approach to understanding and evaluating the opportunities for enhancing the human condition and the human organism opened up by the advancement of technology [7]. We follow the idea of enhancing the human capacities of perception, especially of perceiving information that is only digitally available. The human perception corresponds to five senses: sight, smell, taste, touch and hearing. While lingering in physical space these senses can absorb parameters related to the condition of the environment. For example, the human body can feel the surrounding temperature, recognize light intensity, smell different odors and hear sounds. Considering a smart home environment housing numerous networked artifacts, a great amount of additional (digital) information is available that might be of interest but is not likely to be sensed by the human body. By offering data on different channels than the visual one, we evade the disadvantages of graphical representation in smart home environments, like demanding complete attention and focus of the inhabitant. We therefore equip the human body with a wearable device prototype that can convey signal patterns through vibration. Therefore it is possible to feel information and data, instead of accessing them actively through a display. 3.2 Device Prototype and Signal Patterns To transform an intangible value to signals inhabitants can perceive in a subtle way, an early prototype has been built as proof of concept. The setup consists of an Arduino microcontroller board (www.arduino.cc) and a small servomotor, which is

Feeling Home – Tangible Information Visualization in Smart Home Environments

265

used to create a certain vibration pattern through motion. The motor is placed inside a wristband and can therefore be worn with direct body contact to transfer information related to data available in the smart home, in form of haptic output. At first the controller has been programmed to generate different vibration patterns to evaluate the functionality and the possibilities such a feedback offers. Considering numerous information flows from a variety of sensors and actuators, the haptic signals that are utilized to convey the data have to vary as well. We therefore categorize data streams that might be conveyed (e.g. energy consumption) and assign them to a basic vibration pattern (e.g. a constant vibration or a stop-and-go pattern) that can change in intensity. By doing so, the inhabitant might be able to identify different data streams even all of them are transmitted by vibration. Furthermore the inhabitant should be able to define which data is received over the haptic channel.

4 Use Case To illustrate the usage and operating principles of tangible information visualization with a vibrating wristband, we describe how an inhabitant wearing the device can be informed about the overall level of energy consumption at home. The described case will be realized within our smart home testbed at the Technical University of Berlin. The inhabitant initially chooses the data that should be conveyed through the wristband, in this case the overall energy consumption. To prevent the wristband to constantly vibrate, a threshold is set as well, which defines the minimum value for the signal to start. By switching on the living room television, the wristband begins heavily to vibrate with an energy related pattern. By dimming down the ceiling lights, a slight weakening of the vibration can be felt, providing a feeling of being on the right way to reduce the current energy consumption without having reached the desired amount, yet. After the inhabitant completely switches off the ceiling lights, as well as turning down the electrical radiator, the wristband gently stops to vibrate, signaling the energy consumption falling below the threshold and below the desired maximum.

5 Discussion and Outlook In the paper we described the potential of using other ways than graphical representations to convey information available in a smart home environment to the inhabitants. Based on the general concepts of transhumanism we therefore proposed a more subtle way, following the human senses and the human perception in the physical world. For this purpose a device prototype has been build, which is currently used to demonstrate the potential of haptic feedback mechanisms to convey information within smart homes and to experiment with different signal patterns for various data streams. If this approach is practicable and offers the expected additional value needs to be validated in a user study. Therefore the use case described in chapter 4 will be realized by integrating the device into a smart home testbed at the Technical University of Berlin and by connecting it to a framework for the development of multimodal user interfaces, the Multi-Access Service Platform (masp.dai-labor.de) [8]. The realization will also include required improvements of the device prototype.

266

F. Weingarten and S. Albayrak

References [1] Weingarten, F., Blumendorf, B., Albayrak, S.: Towards Multimodal Interaction in Smart Home Environments: The Home Operating System. In: Designing Interactive Systems, Aarhus (2010) [2] Runge, M., Quade, M., Blumendorf, M., Albayrak, S.: Towards a Common Smart Home Technology. In: AmI – Blocks 2009 Smart Products: Building Blocks of Ambient Intelligence (2009) [3] Albayrak, S., Blumendorf, M., Feuerstack, S., Küster, T., Rieger, A., Schwartze, V., Wirth, C., Zernicke, P.: Ein Framework für Ambient Assisted Living Services. 2. Deutscher AAL Kongress, Berlin (2009) [4] Chang, A., O’Sullivan, C.: Audio-Haptic Feedback in Mobile Phones. In: Proceedings of CHI 2005, pp. 1264–1267. ACM, New York (2005) [5] Poupyrev, I., Maruyama, S.: Tactile Interfaces for Small Touch Screens. In: User Interface Software and Technology, pp. 217–220. ACM, New York (2003) [6] Hoggan, E., Crossan, A, Brewster, S., Kaaresoja, T.: Audio or Tactile Feedback: Which Modality When? In: CHI 2009, Boston (2009) [7] Bostrom, N.: Transhumanist Values. In: Adams, F. (ed.) Ethical Issues for the 21st Century. Philosophical Documentation Press (2004) [8] Blumendorf, M., Albayrak, S.: Towards a Framework for the Development of Adaptive Multimodal User Interfaces for Ambient Assisted Living Environments. In: Proceedings of HCI International, San Diego (2009)

Part V

Brain-Computer Interfaces and Brain Monitoring

Calibration Time Reduction through Source Imaging in Brain Computer Interface (BCI) Minkyu Ahn, Hohyun Cho, and Sung Chan Jun School of Information and Communications, Gwangju Institute of Science and Technology, South Korea {frerap,AugustCho,scjun}@gist.ac.kr

Abstract. Brain Computer Interface (BCI) is mainly divided into two phases; calibration phase for training and feedback phase. A calibration phase is usually time-consuming, thereby, being likely to raise subjects’ fatigue at the early stage. For more convenient and applicable BCI system it should be investigated to reduce such preparation (calibration) time before feedback phase. Beamformer is a source imaging technique widely used in MEG/EEG source localization problem. It passes only signals produced at the designated source point and filters out other signals such as noise. We conjecture information in source space may be consistent over well trained and good subjects. This idea facilitates to reuse existing datasets from the same or different subjects. Using IVa data in BCI competition III, we constructed a classifier from other 4 subject’s training data and performance was evaluated in source domain. In this work, we observed the proposed approach worked well, resulting in relatively good accuracies (73.21%, 74.21%) for two subjects. Keywords: Brain Computer Interface, Source imaging, Transfer learning, Zero training.

1 Introduction Brain computer interface (BCI) has been paid attention as one of interesting applications in bio-signal processing society [1, 2, and 3]. BCI system, in general, consists of two phases. One is a calibration phase and the other is a feedback phase. In the calibration phase, some amount of training data are collected, thus it is more likely to have users tired and distracted before the feedback phase (on-line phase). Matthias Krauledat [4] proposed one approach to reduce this phase called as “zero-training”. Another approach was reported using the application of source imaging for zerotraining [5]. Source imaging likely yields some features unseen in the sensor space and voxel points at the source space could be chosen at user’s discretion [6]. This means that for different sessions or subjects, the same voxel points in the coregistered source space can be chosen. In other words, for different sensor configurations (as it is varying over sessions or subjects), it is possible to project sensor data onto the fixed voxel points at the source space after co-registration of each subject into a typical head coordinated space. This point gives an advantage in that collected data for any sessions or any subjects under the same experimental paradigm C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 269–273, 2011. © Springer-Verlag Berlin Heidelberg 2011

270

M. Ahn, H. Cho, and S.C. Jun

could be used independently as additional trials for any calibration phase, which may possibly reduce calibration phase. This procedure is described in Fig. 1. For example, if some datasets (for different session or subject) are available under the same experiment, then these are projected onto the same source space through source imaging technique. We expect that variability between sessions or subjects at the source space would be diminished significantly because source imaging is likely to reduce some noise and yield more discriminative information. If it would be true, existing datasets would be used to do feature extraction and classification without collecting additional data. Thus, it is likely to reduce the calibration time and further to have zero-training achievable.

Fig. 1. Proposed zero training procedure

2 Datasets To see plausibility of our proposed idea, we used dataset IVa in BCI competition III [9]. The description of the datasets is as follows. Five subjects conducted motor imagery experiment (‘right’ or ‘foot’), and datasets were collected using 118 EEG channels. Different numbers of train or test trials were given for subjects (see Table 1). The thrill for this competition was to classify test trials using given train datasets for each subject. Due to the absence of sensor location, we used standard sensor locations for every subject that were obtained from the competition organizer. For more significant temporal and spectral information, we applied frequency filtering with 10Hz to 30Hz since the range certainly includes μ as well as β rhythm. The target signal was extracted from 0.4 sec to 2.4 sec after cue for imagery instruction. Table 1. Different number of trials for subjects Subjects Trials (Train / Test)

aa 168 / 112

al 224 / 56

av 84 / 196

aw 56 / 224

ay 28 / 252

Calibration Time Reduction through Source Imaging in Brain Computer Interface

271

3 Methods We propose to construct a classifier using source space features over subjects. For this purpose, we introduced three shell Ary 1 head model [7] and beamforming method for source imaging [8]. 3.1 Source Imaging by Beamforming Multi-layer Ary 1 model consists of three concentric spherical layers representing brain, skull and scalp from inner to outer. We considered an outmost layer of radius 100 mm for scalp and other layers were generated according to the radius and relative conductivity ratio of each layer, as tabulated in Figure 2. 1000 voxels were put on the inner-most layer representing brain cortex. To solve the multi-shell model we used Zhang’s method [7] and minimum-variance beamforming method [8] was applied for source imaging.

Fig. 2. Head model and 1000 voxels generated on the inner-most sphere

A minimum-variance beamformer is one of the representative adaptive spatial filters [8]. When m(t) is the spatiotemporal sensor measurement, source activity reconstruction is conducted through the following spatial filtering technique.

Qˆ (r, t ) = w T (r)m(t ), w(r) =

R −1l(r) , l (r)R−1l(r) T

(1)

where Qˆ (r, t ) is the estimated magnitude of the source activity at location r. Here l(r) and R denote lead-field vectors representing the meaning of a sensitivity matrix and a covariance matrix of the spatiotemporal signal m(t); R=<m(t)mT(t)>t and <•>t indicates the time average over a certain given time window. 3.2 Performance Assessment We have training datasets and testing datasets for each subject, so the performance was assessed only on testing datasets while training datasets were involved to construct a classifier. For comparison, we estimated classification accuracies for two ways; one way is a conventional sensor level approach that a classifier is generated

272

M. Ahn, H. Cho, and S.C. Jun

with subject’s original sensor training data only and then testing dataset is tested by this classifier. The other way is our proposed approach that sensor data is projected into source space by source imaging method and then a classifier is constructed by source features in the source space from other subjects’ data. In this assessment, Linear Discriminant Analysis (LDA) was applied to classify. For channel selection (to choose more discriminative channels among sensors or voxels), channels were sorted in the descending order of significance of student t-test. Finally, accuracies were calculated over the number of channels (in sensor or in voxel) to see the performance behavior as the number channels increases.

Fig. 3. Classification accuracy behavior between conventional approach using the subject’s training data and our proposed approach using source features of other subjects’ data

4 Results As a result, we observed the meaningful performance of our proposed approach for two subjects who are subject ‘aa’ and subject ‘ay’ (see Fig. 3). For both subjects, the proposed approach shows good accuracies which are highly above chance level (50%). For subject ‘aa’, our proposed approach performed even better than conventional approach using the subject’s own training data only. We did not observe this behavior for other three subjects (‘al’,’av’ and ‘aw’, not shown here). One interesting observation is that the subject ‘ay’ shows reasonably comparable performance even though the number of training trials was only 28, which is considerably small compared to the number of testing trials; it is inferred that these 28 trials well represented the entire feature spaces.

Calibration Time Reduction through Source Imaging in Brain Computer Interface

273

5 Conclusion One critical issue in BCI is to reduce the time-consuming calibration phase. This calibration phase is more likely to make users tired and distracted before the feedback phase, making it hard to collect consistent brain activity, and therefore degrading BCI performance. To overcome this, we proposed a zero-training approach using source imaging. From the investigation using well known dataset IVa in BCI competition III, we observed two subjects showed comparable performance and in one subject of them, our proposed approach performed better than conventional approach using subject’s own training data only. Our findings show some possibility that the proposed approach may reduce the calibration time by using other subject’s dataset. However, for better understanding, more thorough investigation is necessary, which is under study. Acknowledgements. This work was supported by a NRF grant (NRF-2010-0006135), and a NIPA grant (NIPA-2011-C1090-1131-0006).

References 1. Blankertz, B., et al.: The Berlin brain-computer interface presents the novel mental typewriter hex-o-spell. In: Proc. of the 3rd International Brain-Computer Interface Workshop and Training Course, pp. 108–109 (2006) 2. McFarland, D.J., et al.: Electroencephalographic control of three-dimensional movement. J. Neural Eng. 7 036007 (2010), doi: 10.1088/1741-2560/7/3/036007 3. Lotte, F., et al.: Exploring Large Virtual Environments by Thoughts using a BrainComputer Interface based on Motor Imagery and High-Level Commands. Presence: Teleoperators and Virtual Environments 19(1), 54–70 (2010) 4. Krauledat, M., Tangermann, M., Blankertz, B., Müller, K.-R.: Towards Zero Training for Brain-Computer Interfacing. PLoS One 3(8), e2967 (2008), doi:10.1371/journal.pone.0002967 5. Kamousi, B., et al.: Classification of Motor Imagery Tasks for Brain-Computer Interface Applications by Means of Two Equivalent Dipoles Analysis. IEEE Trans. Neural Syst. Rehabil. Eng. 13(2), 166–171 (2005) 6. Ahn, M., Cho, H., Jun, S.C.: Information Extraction from the Source Space for Brain Computer Interface. In: Proceedings of TOBI Workshop, vol. 2, pp. 29–30 (2010) 7. Zhang, Z.: A fast method to compute surface potentials generated by dipoles within multilayer anisotropic spheres. Phys. Med. Biol. 40, 335–349 (1995) 8. Sekihara, K., Nagarajan, S.S.: Adaptive spatial filters for electromagnetic brain imaging. Springer, Heidelberg (2008) 9. BCI Competition III, http://www.bbci.de/competition/iii/desc_IVa.html

How Much Features in Brain-Computer Interface Are Discriminative?– Quantitative Measure by Relative Entropy Sangtae Ahn, Sungwook Kang, and Sung Chan Jun School of Information and Communications, Gwangju Institute of Science and Technology, South Korea {stahn,scjun}@gist.ac.kr Abstract. Brain Computer Interface (BCI) gives opportunities to control a computer or a machine by imagination of limb movement, which activates somatosensory motor region in a discriminative manner. As far as it has been concerned, it has been not well investigated how much the given (extracted) features in BCI are discriminative in the sense of information theory. For this purpose, we cast the feature spaces corresponding to given conditions into probability spaces by yielding corresponding probability distributions. Then the relative entropy (measures to estimate the difference between two probability distributions) is introduced to measure the distance between these probability distributions. Such a distance represents well how two feature spaces are separable. We compare this distance with BCI performance (classification success rate) to see their correlation. Keywords: Brain Computer Interface, Relative Entropy, Information Theory.

1 Introduction The development of a brain computer interface (BCI) aims to provide a communication channel from a human to a computer or a machine, that is, it directly translates brain activity into sequences of control commands [1]. BCI classification success rate is a barometer to measure how well the BCI system works; it strongly depends on many factors such as a variety of preprocessing, features to be used, classification techniques. We have an intrinsic question of how much the given (extracted) features in BCI are discriminative. It has been investigated in many ways, but it has not been in the sense of information theory as far as it has been concerned. In this work, we try to investigate how two different conditioned EEG data is discriminative in a theoretical basis and how two feature spaces are well separable. To investigate these issues, the relative entropy as an information-theoretical measure is applied to several motor imagery movement EEG datasets and it is discussed how relative entropy and BCI success rate are related.

2 Data 2.1 Experimental Setup and Preprocessing Experimental EEG datasets were recorded from five healthy subjects. Each dataset contains only data from the 4 initial sessions without feedback. Visual cues indicated C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 274–278, 2011. © Springer-Verlag Berlin Heidelberg 2011

How Much Features in Brain-Computer Interface Are Discriminative?

275

for 3.5 seconds which of the following 3 motor imageries the subject should perform: (L) left hand, (R) right hand, (F) right foot. The presentation of target cues was intermitted by periods of random length, 1.75 to 2.25 seconds, in which the subject could relax. There were two types of visual stimulation: (1) where targets were indicated by letters appearing behind a fixation cross, and (2) where a randomly moving object indicated targets (inducing target-uncorrelated eye movements). From subjects ‘al’ and ‘aw’, two sessions of both types were recorded, while from the other subjects, three sessions of type (2) and one session of type (1) were recorded. EEG data were acquired from 118 channels attached on the scalp according to the extended international 10/20-system. Signals were then digitized at 1000 Hz with 16 bit (0.1 μV) accuracy and band-pass filtered between 0.05 and 200 Hz. In this work, we analyzed (R) right hand and (F) right foot datasets. In order to α or β rhythms, they were band-pass filtered between 8 and 30 Hz and down-sampled at 100 Hz. Also, we extracted temporal window of 1 second long for resting state and 3 seconds long for testing state (imagination period). It is known that right hand and right foot movements generate ERS/ERD signals mainly around C3 and Cz channels. Thus, probability density functions (pdf) within the short-time window (200 ms) corresponding to C3 and Cz channels are estimated through Gaussian Mixture Model (GMM) and EM (Expectation Maximization) algorithm. For each trial per each channel (time series of 4 seconds long), we generated informative samples as follows: EEG time series were averaged over moving short-time window (200 ms) and the window was sliding by 100 ms with overlapping 100 ms. Then for each trial a total of 40 samples were generated (10 samples from the resting state and 30 samples from the imagination period). This procedure was performed over all trials in the same manner. 2.2 Classification Success Rate We calculated classification success rate from five healthy subjects using CSP and FLDA (Fisher Linear Discriminant Analysis). This classification success rate is compared with an amount of discriminative information later. Table 1. Classification success rate of BCI Competition III datasets (IVa) Subjects

‘aa’

‘al’

‘av’

‘aw’

‘ay’

Success rate (%)

70.5

100

58.6

82.5

84.1

3 Methods 3.1 Gaussian Mixture Model (GMM) In order to calculate the pdf, we used Gaussian Mixture Model (GMM) [2]. GMM is a parametric probability density function expressed by a weighted sum of Gaussian density functions [3]:

276

S. Ahn, S. Kang, and S.C. Jun

M

p(x | λ) = ∑ w i g(x | μ i , Σ i ),

(1)

i =1

where x is a D-dimensional vector representing features, w i , i = 1,..., M are mixture weights, and g(x | μ i , Σ i ), i = 1,...M, are Gaussian density functions, each function is expressed by a D-variate Gaussian function of the form:

g(x | μ i , Σ i ) =

with mean vector constraint that

1 (2π ) D/2 Σ i

1/2

⎧ 1 ⎫ −1 exp⎨− (x − μ i ) ' Σ i (x − μ i )⎬ ⎩ 2 ⎭

(2)

μ i and covariance matrix Σ i . The mixture weights satisfy the

Σ iM=1w i = 1.

3.2 EM (Expectation Maximization) Algorithm EM (Expectation Maximization) algorithm is a general technique for finding a maximum likelihood solution having latent variables [2]. It is an iterative optimization method to estimate some unknown parameters Θ for the given measurement data U [4]. However, any “hidden” nuisance variables J should be integrated out as follows:

P(Θ | U ) =

∑ P(Θ, J | U )

J ∈J n

(3)

Then the posterior probability density function is maximized:

Θ* = arg max P(Θ | U ) Θ

(4)

There are two steps in EM algorithm, the first step is called the ‘expectation-step’ or E-step whereas the second step is called the ‘maximization-step’ or M-step. In the Estep,

t t f (J) ≅ P(J | U, Θ ) is

evaluated using the current guess Θ t , whereas in the M-step

it is optimized as follows:

t +1 t Θ = argmax [Q (t) + logP(Θ) with respect to the free Θ

variable Θ . 3.3 Relative Entropy The entropy of a random variable is a measure of the uncertainty of the random variable; it is a measure of the amount of information required on the average to describe the random variable [5]. The relative entropy is a measure of the distance between two distributions. In statistics, it arises as an expected logarithm of the likelihood ratio. The relative entropy D(p || q) is a measure of the inefficiency of assuming that the distribution is q when the true distribution is p . The relative

How Much Features in Brain-Computer Interface Are Discriminative?

277

entropy or Kullback-Leiber distance between two probability density functions p(x) and q(x) is defined as

D(p || q) =

p(x)

∑ p(x)log q(x)

x∈X

(5)

4 Results Figure 1 shows that relative entropy is higher in magnitude for testing state than for resting state. Distance between C3 and Cz channels increases relatively to resting state after onset of cue in Figure 1, especially subject ‘al’ or ‘ay’, but it does not in subject ‘av’, as shown in Figure 1. Based on these results, we found that the higher success rate has tendency in yielding the relatively higher relative entropy for testing state. After onset of cue, high amplitude of relative entropy means that difference of amount of information appeared noticeably between C3 and Cz channels.

Fig. 1. Time series of relative entropy. Time 0 represents the onset of cue.

5 Conclusion In general, classification success rate represents that how much the extracted features in BCI are discriminative, but we try to view them in a different manner, namely measuring information amount using relative entropy (measures to estimate the difference between two probability density functions). In relative entropy, we observed how far two different conditioned EEG data are distributed away. For each conditioned EEG data, we modeled a probability density function as a Gaussian Mixture Model and estimated it through EM algorithm. Then the relative entropy was adopted to measure discriminability between them. As a result, we observed visible

278

S. Ahn, S. Kang, and S.C. Jun

correlation between classification success rate and difference between averaged relative entropies over resting state and testing state. Thus, relative entropy may be a possible measure of extent of discriminability between two conditioned EEG data, thereby, being applicable to BCI. Acknowledgements. This work was supported by a NRF grant (NRF-2010-0006135), and a NIPA grant (NIPA-2011-C1090-1131-0006).

References 1. Blankertz, B., et al.: Spatio-Spectral Filters for Improving the Classification of Single Trial EEG. IEEE Transactions on Biomedical Engineering 52(9) (September 2005) 2. Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 4th edn. Elsevier, Oxford (2009) 3. Reynolds, D.: Gaussian Mixture Models, MIT Lincoln Laboratory, 244 Wood St., Lexington, MA 02140, USA 4. Dellaert, F.: The Expectation Maximization Algorithm. College of Computing, Gerorgia Institute of Technology Technical Report number GIT-GVU-02-20 February (2002) 5. Cover, T.M., Thomas, J.A.: Elements Of Information Theory, 2nd edn. Wiley, New Jersey (2006)

EEG-Based Measurement of Subjective Parameters in Evaluations Daniel Cernea1,2, Peter-Scott Olech1, Achim Ebert1, and Andreas Kerren2 1 University of Kaiserslautern, Department of Computer Science, Computer Graphics and HCI Group, P.O. Box 3049, D-67653 Kaiserslautern, Germany {cernea,olech,ebert}@cs.uni-kl.de 2 Linnaeus University, Computer Science Department, ISOVIS Group, Vejdes Plats 7, SE-35195 Växjö, Sweden [email protected]

Abstract. Evaluating new approaches, be it new interaction techniques, new applications or even new hardware, is an important task, which has to be done to ensure both usability and user satisfaction. The drawback of evaluating subjective parameters is that this can be relatively time consuming, and the outcome is possibly quite imprecise. Considering the recent release of costefficient commercial EEG headsets, we propose the utilization of electroencephalographic (EEG) devices for evaluation purposes. The goal of our research is to evaluate if a commercial EEG headset can provide cutting-edge support during user studies and evaluations. Our results are encouraging and suggest that wireless EEG technology is a viable alternative for measuring subjectivity in evaluation scenarios. Keywords: Evaluation techniques, Brain-Computer Electroencephalographic (EEG) interaction.

Interface

(BCI),

1 Introduction Measuring user satisfaction has been an important factor before introducing marketready goods and services for decades. Therefore, evaluations have been an essential step for developments in the field of Human-Computer Interaction (HCI). In fact, they are the decisive factor if a solution will really work as intended, based upon usability guidelines and user experiences. Setting up and performing an evaluation can be a time consuming process. Often, however, the real outcome is doubtful. Also, many widely used and accepted techniques have disadvantages. For example, "thinking aloud" encourages subjects to verbalize their thoughts and emotions, thus changing the users’ behavior. In our approach, a consumer market EEG device, the wireless Emotiv EPOC1 headset, is used to add value to standard evaluation methods like questionnaires. Instead of focusing on the EEG headset as an interaction device, we want to shift the focus to use the headset as an evaluation support device and even as a standalone device for fast first impressions. 1

http://www.emotiv.com

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 279–283, 2011. © Springer-Verlag Berlin Heidelberg 2011

280

D. Cernea et al.

In the following sections, we present an evaluation of the capabilities of the EPOC device itself and highlight the results. We continue by employing the EEG headset in two scenarios for measuring the emotional reactions of test subjects.

2 Related Work The history of Electroencephalography (EEG) dates back to the late 19th century [8]. Its use has been mainly medical in order to record the electrical activity of the brain, e.g., in the field of neuro-science to detect abnormal brain activity, like to diagnose epilepsy. Brain-Computer Interaction techniques (BCI) have been researched recently in order to provide interaction possibilities, e.g., for physically handicapped people [4]. Grimes et al. [2] advanced the field by investigating brain waves and how to classify working memory load with the help of an EEG devices. Scherer et al. [7] introduced an EEG-controlled Virtual Keyboard. The work of Horlings et al. deals with emotion recognition by using an EEG device [3]. Similarly, Mikhail et al. introduced a feature selection mechanism in [5], which can detect emotions out of noisy EEG data. In the work of Campbell et al. [1], the EPOC headset is used to interact with mobile phones (e.g. dial by using the headset instead of touch). Ranky et al. [6] propose to use the EPOC headset as an interaction device to control a robotic arm. After a training period, they obtained quite satisfying results.

3 The Wireless EPOC EEG Headset Originally designed as gaming device, the EPOC headset comes with preprogrammed features, which can be quickly employed in evaluation. These offer real-time feedback about the emotional reactions of a user. Using such an affordable EEG device for evaluation purposes means that the results are not externally influenced, except by the brain activity of the subject. In terms of hardware, the EPOC headset is quite non-intrusive, as it is enabled by a wireless connection and very light. It is capable to measure electrical brain activity by means of 14 saline non-dry sensors. The EPOC device has convenient features in its framework that enable the detection of a set of facial expressions and emotional states. Sadly, the algorithms for this are proprietary. As such, owners of an EPOC device have to rely on the manufacturer's encoding without much proof for the correctness of the detections. To overcome this and validate the output of the EEG headset, we have compared the results of the EPOC headset against the results from commonly accepted evaluation methods during two evaluations involving the detection of facial expressions and emotions. The following tests involved 12 subjects with a basic level of knowledge in computer usage. In terms of distribution, the test group contained four women and eight men, aging from 21 to 52, and with an average age of 29.75 years. The users have diverse ethnicity and varied cultural background. 3.1 Detection of Facial Expressions Facial expressions, especially when executed subconsciously, have the ability to reflect the persons’ inner feelings. Usually, video logs cannot be analyzed in real-time

EEG-Based Measurement of Subjective Parameters in Evaluations

281

and need to be interpreted after the completion of the subject’s task. To circumvent this problem, we considered capturing the facial expressions of the test subjects via the EPOC device. As the framework and coding of the detection of the facial motions is not accessible to us, we started by validating the results offered by the headset against more common evaluation methods―in this case, video log analysis. The subjects were equipped with the EPOC headset and positioned in front of a monitor and a webcam. They were given a sequence of words on the screen that represented facial expressions (e.g., smile, blink, etc.). Then, they were asked to execute them while the text was displayed and for as long and as often as they considered. The facial expression texts were given to the users randomly with each expression appearing at least 3 times. After the task was completed and the EPOC output data was available, the video logs and the information given by the headset were compared. The results show that the correct detection of the facial expressions varies between 70-100%, depending on the particular facial expression. 3.2 Detection of Emotional Reactions After checking the correctness of the results for facial expressions detection, we turned our attention to evaluating its capacities in correctly assessing the emotional state of the user. The subjects were given tasks that should provoke emotional reactions. These emotional responses measured by the EPOC were compared with the results of a questionnaire, posed to the subjects after the tasks were completed. If the results from the EEG headset and the questionnaire are close to identical, we can argue that the EPOC device is a viable alternative or supportive method in evaluation. The emotional states we tested for were calmness, meditation, engagement in dexterity and mental tasks, and excitement. The tasks the subjects had to complete involved watching informative videos, listening to music, and playing dexterity and memory games. Each task would generate one particular emotional response, and both the questionnaire and the EPOC output would focus only on reading that emotion. Afterwards, the users confirmed that the tasks were appropriate for generating the expected emotional response. During the tasks, the device returns a constant stream of values for each of the emotions mentioned above, at a rate of approximately 4 Hz. As these values fluctuate, we can deduce the changes that affect the user. The recognition of an emotion is triggered in two ways: by computing the angle of increasing slopes of the values during the task and by computing the difference between the maximum and minimum during a given task. An emotion is considered as triggered if we have a slope bigger than 30 or 60 degrees as well as if the max-min difference is more than 30% or 60%. These four thresholds, together with the neutral state for an emotion, depict a set of five possible levels for the values of an emotion as returned and interpreted by the EPOC device. Similarly, the questionnaire that assesses the emotional reaction of each user at the end of the task inquires about a particular emotion, and the subject has to answer in a 5-level point system: strongly disagree, disagree, neutral, agree and strongly agree (represented by 0, 0.25, 0.5, 0.75, and 1).

282

D. Cernea et al.

Fig. 1. Average difference between the EPOC device output and the questionnaire results for the emotional evaluation (left to right: calmness, meditation, engagement on dexterity task, engagement on metal task, excitement), with both on the same 5-point system: 0, 0.25, 0.5, 0.75, and 1. Black vertical lines represent the standard deviation for each average difference.

Figure 1 presents the results of our validation. Note that the average difference between the answers from the questionnaire and the EPOC output is between 0.185 and 0.068. To put this into context, an average difference of 0.25 means that on average the distance between the user's answer in the questionnaire and the EPOC output was one unit out of five. To statistically validate our results, we also computed the standard deviation for the average differences. The results are promising as the maximum standard deviation is 0.071. Also, we executed a paired sample t-test for the data obtained from each task (questionnaire results and EPOC output). In each of the five cases, there is no significant difference between the paired sets, suggesting that the EPOC device can offer quite accurate interpretations of human emotional states.

4 Evaluation Scenario Once we had confirmation for the correct results obtained via the EPOC headset in terms of facial expressions and emotional states, we continued with an evaluation scenario. We tested the emotional effects a spot-the-difference task would have on users. For this mostly mental/visual task, we presented the users three pairs of slightly different images sequentially. The emotional states that were incorporated are engagement, excitement, satisfaction and frustration. In detecting the levels of satisfaction and frustration, the emotional output of excitement and various facial expressions—like smiling or clenching—were considered. The highest average difference obtained between the EPOC outputs and the questionnaire answers was 0.33 for the excitement level. The other average differences are situated between 0.2-0.25, similar to the previously described validation. The standard deviation was computed for the differences, resulting in a maximum value of 0.12. A paired sample t-test for the data obtained from each emotion showed that all paired sets present no significant difference, except for the excitement emotion. A possible reason for this is that emotional excitement in an intrinsically mental task is hard to define.

EEG-Based Measurement of Subjective Parameters in Evaluations

283

5 Conclusion We tested the detection of facial expressions and emotional states with the Emotiv EPOC device and obtained promising results. Building on these results, we employed the EEG headset in an evaluation scenario producing encouraging outcomes in terms of using the headset as an evaluation device. While not knowing the exact purpose of wearing the EPOC device, one user even mentioned that he “would use this device in market research”, further suggesting that an evaluation approach based on a mobile EEG could open the door towards real-time efficient subjectivity measurement. Acknowledgments. This work was supported by the German Research Foundation (DFG, grant number 1131) as part of the International Graduate School (IRTG) in Kaiserslautern on “Visualization of Large and Unstructured Data Sets”.

References 1. Campbell, A., Choudhury, T., Hu, S., Lu, H., Mukerjee, M.K., Rabbi, M., Raizada, R.D.S.: NeuroPhone: Brain-Mobile Phone Interface using a Wireless EEG Headset. In: Proc. of MobiHeld 2010, pp. 3–8. ACM Press, New York (2010) 2. Grimes, D., Tan, D.S., Hudson, S.E., Shenoy, P., Rao, R.P.N.: Feasibility and Pragmatics of Classifying Working Memory Load with an Electroencephalograph. In: Proc. of CHI 2008, pp. 835–844. ACM Press, New York (2008) 3. Horlings, R., Datcu, D., Rothkrantz, L.J.M.: Emotion Recognition using Brain Activitiy. In: Proc. of CompSysTech 2008, ACM Press, New York (2008), article 6 4. Leeb, R., Friedmann, D., Mueller-Putz, G.R., Scherer, R., Slater, M., Pfurtscheller, G.: Selfpaced (Asynchronous) BCI Control of a Wheelchair in Virtual Environments: a case study with a tetraplegic. Intell. Neuroscience, 1–12 (2007) 5. Mikhail, M., El-Ayat, K., El Kaliouby, R., Coan, J., Allen, J.J.B.: Emotion Detection using Noisy EEG Data. In: Proc. of AH 2010, pp. 1–7. ACM Press, New York (2010) 6. Ranky, G.N., Adamovich, S.: Analysis of a Commercial EEG Device to Control a Robot Arm. In: Proc. of the 2010 IEEE 36th Annual Northeast, pp. 1–2 (2010) 7. Scherer, R., Mueller, G.R., Neuper, C., Graimann, B., Pfurtscheller, G.: An Asynchronously Controlled EEG-based Virtual Keyboard: Improvement of the Spelling Rate. IEEE Transactions on Biomedical Engineering 51(6), 979–984 (2004) 8. Swartz, B.E., Goldensohn, E.S.: Timeline of the History of EEG and Associated Fields. Electroencephalography and Clinical Neurophysiology 106(2), 173–176 (1998)

Fundamental Study of the Pictogram-Scanning-BCI Hiroyuki Inada and Hisaya Tanaka 1-24-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo 163-8677, Japan [email protected]

Abstract. We developed a pictogram-scanning-BCI system for neurogenic patients and people with severe physical disabilities. Pictograms are applied as a means for communication. Because pictograms have meanings in pictures, the pictogram method may communicate faster than the traditional letter method. We developed a Brain Computer Interface (BCI) with nine pictograms for athome care. We conducted comparative experiments of the pictogram-scanning method and traditional letter-scanning method to review input accuracy, input speed and bit rate. The BCI system showed nine pictograms on a 3x3 matrix to subjects, as well as 36 letters on a 6x6 matrix. A row or column of pictograms or letters was randomly flashed. As a result, the pictogram method was effective for particular communication such as the case of at-home care. Keywords: BCI, ALS, pictogram, CA, ERP, EEG.

1 Introduction It is extremely difficult for neurogenic patients and people with severe physical disabilities to communicate their intentions to others. To solve this problem, various Communication Aids (CAs) have been researched and developed, and there are cases where they were actually used in the practice of home medical care. In this study, we developed a pictogram-scanning BCI with electroencephalogram (EEG) which utilizes the P300 components of Event Related Potential (ERP). The Brain Computer Interface (BCI) is developed for patients in the actual scenes of home care and to meet the needs of patients with intractable neurological diseases. Furthermore, we conducted comparative experiments of the pictogram-scanning method and traditional letter-scanning method to review input accuracy, input speed and bit rate. The P300 is positive potential elicited at approximately 300 ms latency for attention to a specific stimulus. In this paradigm, the subject selected the target in two stimuli. In general, events in one of the two categories are rarely presented. Moreover, the subject is assigned a task that cannot be performed without categorizing the events. Under these circumstances, events in the rare category elicit an ERP characterized by a P300 component; the less probable the eliciting event, the larger the P300. Farewell and Donchin [1] described that a BCI uses the P300 component to enable patients to communicate with letters. The BCI system continually presents 36 alphabets to subjects on the display. They are set on a 6x6 matrix as visual stimuli. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 284–288, 2011. © Springer-Verlag Berlin Heidelberg 2011

Fundamental Study of the Pictogram-Scanning-BCI

285

1.1 Pictogram Pictograms are applied as one of the means for communication. They are usually used as signposts, and also as means of communication for cerebral paralysis or neurogenic patients; therefore pictogram-scanning-BCI is also considered as useful. It is supposed in this study that pictogram-scanning-BCI can transmit the subject’s intention faster than traditional letter-scanning-BCI. For pictograms utilized in the BCI, nine characters were selected from “Design Principles of Pictorial Symbols for Communication Support (JIS T0103)” stipulated by the Ministry of Economy, Trade and Industry in 2005 [3]. They conform with the criterion for selection “Den no shin” as well as with the requirement specifications that Tanaka, etc. [4] obtained from Table 1. Pictograms selected from JIS and their meanings

Yes/Good

No/Bad

Hot

Cold

What time is it?

Turn on the light

Suction

Bathroom

Ringing signal

Fig. 1. Screen for pictogram input system

Fig. 2. Composition of BCI system

286

H. Inada and H. Tanaka

research reports. We considered it desirable to conduct experiments with a smaller number to review usability of pictograms for the BCI. A 3x3 matrix was chosen so that conversational sentences can be developed with pictograms with a hierarchical structure. Table 1 shows pictograms prepared and their meanings. One pictogram has some meanings to enable communication. It is the difference between the pictogramscanning-BCI and letter-scanning-BCI. The pictogram-scanning-BCI requires only one input, while the letter-scanning-BCI requires multiple inputs to create a word. Nine pictograms were set on a 3x3 matrix on the screen, and selected characters were displayed (Fig.1).

2 Methods 2.1 BCI Composition The BCI composition is shown in Fig.2. The electroencephalogram (EEG) was recorded with electrode caps (g.tec g.EEGcap) based on the international 10-20 system. First, exploring electrodes were set at Fz, Cz, Pz, C3, C4, Oz, O1, and O2. Secondly, an earth electrode was set at Fpz and a reference electrode on an earlobe. The EEG from electrodes was intensified with biotic amplitude (g.tec g.USBamp). Filtering values were set at 0.1Hz for the high pass filter, 50 Hz for the low pass filter and 50Hz for the notch filter to hum noise. Linear discrimination analysis (LDA) was conducted in regards to the data obtained. Since the event related potential is influenced by the degree of attention, personal differences appear by each subject in terms of latency and amplitude. In consequence, we analyzed each of the subjects’ specific ERP features. 2.2 Subjects, Stimuli and Procedure Four subjects healthy and able to move their bodies participated in the experiment. The details were explained to them and they approved this experiment. The subject sat in a chair in the resting state, and looked at the 14-inch screen of a notebook computer. A “target” stimulus is defined as a subject’s intention when the characters were displayed in white on a black background. A “non-target” stimulus is defined as others when the characters were displayed in gray on the black background. In the task to analyze ERP features, the number of trials was set at 30, and five pictograms specified by the experimenter were entered. In the plenary task, the number of trials was set at 20. Tasks were conducted with the pictogram-scanning method and traditional letterscanning method to review input accuracy, input speed, bit rate, and accuracy of one word. As the standard stimulus, the inter-stimulus interval (ISI) was set at 100 ms and the stimulus onset asynchrony (SOA) at 175ms. ISI means the time when characters are white on a row or column, SOA means the time until next target character turns white. Their intervals were adjusted constantly at a ratio of 4:5 (ISI:SOA). One trial takes 1,050 ms (3x3 matrix) in the case of pictogram-scanning-BCI and 1,500 ms (6x6 matrix) in the case of letter-scanning-BCI. Furthermore, SOA was set at 70, 105, 140, 175, 210 or 240msec.

Fundamental Study of the Pictogram-Scanning-BCI

287

2.3 Assessment of BCIs To assess the performance of BCI, we need to analyze input accuracy, input speed and bit-rate. A bit-rate is mathematically expressed in the following. Note that N means the possible number selection, P means accuracy, and M means the number of decision-making per minute.

3 Results and Discussion Fig.3-6 shows the results of the analysis. Possible accuracy was approximately 80% or higher for BCIs with both methods on 140-240 SOA in Fig.3. The rate of correct answers tended to be better with letter-scanning-BCI than pictogram-scanning-BCI. The time required for input was 7 to 16 sec/characters in the case of pictogram and 10 to 31 sec/characters in the case of letter as shown in Fig.4. In all SOA, pictogramscanning-BCI input required shorter time than letter-scanning-BCI. In Fig.5, bit-rates in letter-scanning-BCI were 9.0 to 12.1 bits/min compared with 9.5 to 11.7 bits/min in pictogram-scanning-BCI. Thus, there was no notable difference between both methods when the SOA was 140ms or longer. While pictogram-scanning-BCI performed better than letter-scanning-BCI, it is necessary to consider the influence of the number of characters on the result. Pictogram-scanning-BCI had nine characters in the matrix of 3 rows and 3 columns. On the other hand, letter-scanning-BCI had 36 characters in the matrix of 6 rows and 6 columns. Thus, the BCI in the case of the former needs six stimuli to stimulate all matrices and the BCI for the latter needs 12 stimuli. For instance, if SOA was set at 70 ms, the BCI in the case of the former requires 420 ms for one trial, while the BCI for the latter requires 840 ms. This factor affected these results.

Fig. 3. Difference in accuracy by SOA

Fig. 4. Time required for character input

288

H. Inada and H. Tanaka

Fig. 5. Assessment of BCIs

Fig. 6. Accuracy of one word

In Fig.3 and 5, the performance was the same for both methods; however a pictogram has a meaning in one character enabling communication with one entry, and pictogram entry can transmit a subject’s intention to the BCI system faster than word spelling. Therefore, the pictogram method is effective for particular communication such as the case of at-home care. Fig.6 shows the ratio of correct words when a subject entered one word. Pictogram-scanning-BCI shows the percentage of correct answers higher than letter-scanning-BCI. To enter one word, a pictogram requires one character but a letter requires some characters. Possible reasons for this include the degree of attention. The longer it takes for subjects for input, they need to keep concentrated longer, i.e., pictogram-scanning-BCI puts less burden on subjects.

References 1. Donchin, E., Spencer, K.M., Wijesinghe, R.: The Mental Prosthesis - Assessing the Speed of a P300-based Brain-Computer Interface. IEEE Transactions on Rehabilitation Engineering 8(2), 174–179 (2000) 2. Fujisawa, K., Inoue, T., et al.: Communication with Visual Symbols - Japanese-version PIC. Brain Shuppan, 20–79 (1995) 3. Japanese Industrial Standards Committee: Design Principles of Pictorial Symbols for Communication Support JIS T0103 (2005) 4. Tanaka, H., Nozawa, A., Ide, H.: The Research Trend of Brain-Machine Interface without Body Actions. Human Interface Workshop, p. 5 (2004)

EEG Based Comparative Measurement of Visual Fatigue Caused by 2D and 3D Displays Young-Joo Kim and Eui Chul Lee Division of Fusion and Convergence of Mathematical Sciences, National Institute for Mathematical Sciences, KT Daeduk 2 Research Center, 463-1, Jeonmin-Dong, Yuseong-gu, Daejeon 305-811, Republic of Korea {yjkim,eclee}@nims.re.kr

Abstract. Visual fatigue is a very important issue of display market to implementing more comfortable display device in terms of human factor. This study is performed in order to measure the visual fatigue using objective evaluation method (i.e., EEG) in 3D display environment. We performed the difference of visual fatigue using a subjective method and compared it to an objective measure. As a result, the 3D content affected the power of brain wave in the beta frequency (14-25 Hz). Beta power was more strong at the viewing the 3D contents. Also, subjective result also showed more strong visual fatigue in the 3D condition than in the 2D condition. We found that visual fatigue significantly appeared amount of power in the beta band between stereoscopic (3D) image and 2D image. Keywords: Electroencephalogram, Visual Fatigue, Stereoscopic image .

1 Introduction Recently display market has been promoted with novel technology and devices. Among many kinds of display devices, the stereoscopic display device is strongly limelight with various 3D contents such as game, animation, movie and sports broadcasting, and so on. Although these 3D contents can give much immersion and interests, visual fatigue phenomenon is continuously raised as significant problem of watching stereoscopic display. There are some researches in relation to measure the amount of visual fatigue on the different display devices (LCD vs PDP) or contents (2D vs 3D) [1, 2, 3, 5]. Such visual fatigue measurement methods can be categorized into two groups such as survey based subjective methods and bio-signal measurement based objective evaluation ones [4]. In one previous research, they evaluated visual fatigue when watching 3D contents by using several questionnaires for subjective evaluation as well as objective evaluation using event relation potentials (ERPs) [5]. Despite some previous researches about objective visual fatigue measurement, more study is still needed to further explicate objective evaluation in order to quantitatively measure the amount of visual fatigue. In this paper, we perform comparative measure visual fatigues when respectively watching 2D and 3D displays by analyzing specific frequencies of multi-nodes EEG. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 289–292, 2011. © Springer-Verlag Berlin Heidelberg 2011

290

Y.-J. Kim and E.C. Lee

At the EEG measurement stage, subjective answers for some questionnaires are also collected in order to compare the relation between objective and subjective responses.

2 Method Participants were composed of five persons (male: 3, female: 2) between 22 and 24 years old. We paid each subject $30 for remuneration, and we requested them to prepare for the test by taking enough rest. In order to study the effect of 3D visual fatigue in terms of EEG, we presented two modes of same video clip such as 2D and 3D. All participants performed two tests by watching these two modes of the video clip. The length of each video clip was 15 minutes. After watching the video clip, they compiled some questionnaires about visual fatigue. The evaluation had seven points scales, and median point (4) corresponded that the participant did not definite judge the visual fatigue (see Table 1). Animation movie title is “Ice Age: Dawn of the Dinosaurs” produced by 21st Century Fox [6]. When subjects watched 3D video clip, the subjects wore 3D anaglyph red/green glasses on their eye. To make fair experimental condition, subjects when watching 2D video clip also wore glasses which has no red/green lenses. Table 1. Subjective assessment of visual fatigue

Scores

Comments

1

I didn’t feel visual fatigue (No fatigue)

2~3

I felt little visual fatigue.

4

I felt a little visual fatigue.

5~6

I felt more visual fatigue.

7

I strongly felt visual fatigue.

In order to collect brain-wave, we recorded the electroencephalogram (EEG) with electrodes attached according to the international 10-20 system at Oz, which was important electrode location for visual fatigue [7]. We positioned the ground electrode above the forehead to reduce electrical interference. The EEG signal was amplified with the EEG100c amplifiers (Biopac Systems Inc., USA) and digitized by using the PXI system (National Instruments). The signal analysis was performing using the Labview TM software. Sampling rate was 250Hz and EEG signals processed using band-pass filtering before data analysis. The independent T-test was performed to determine the mean differences in power value between the 2D and 3D contents. The significant level of statistical tests was set to 0.05.

EEG Based Comparative Measurement

291

3 Results Experimental results showed that significance was proved in a beta frequency band (14-25Hz) of EEG signals by comparison of watching 2D and 3D (p=.028; significant at a confidence level of p<0.05). In detail, the average power of the beta frequency band in case of watching 3D video clip (mean: 0.002601, standard deviation: 0.002016) was greater than that of 2D video clip (mean: 0.001778, standard deviation: 0.002491) at Oz positioned electrode. In alpha frequency band (8-13Hz) position, although the power of watching 2D video clip was greater than that of 3D one, no significance was found in the alpha band.

Fig. 1. This figure indicated averaged subjective evaluation value for visual fatigue. There was significant difference between 2D animation content and 3D animation content (*** = p<.001).

For subjective test, two questions about visual fatigue symptoms were given to subjects. According to the result of answers, we found that the amount of 3D visual fatigue was even greater than that of 2D (see figure 1).

4 Conclusion The aim of this study was to comparative examine visual fatigue caused by different visual stimuli (2D/3D video clips) in terms EEG frequency band. Results showed that EEG power increased during watching 3D video clip, compared with the case of watching 2D video clip. Especially, we found that the beta band had more significance compared with alpha band in terms of comparative watching 2D and 3D videos. This result supports the previous study that an increase in amplitude of eventrelated potentials (ERPs) is related to sensory stimulus [5], as well as the processing and execution of cognitive activity [7, 8]. From these results, two major issues need to be performed in future study. Firstly, the results are not supported by detailed information of eye responses (i.e., pupil size, blink rate etc). An eye measurement is necessary to be investigated for the

292

Y.-J. Kim and E.C. Lee

relationship between visual and EEG responses in case of watching stereoscopic scenes. Secondly, we will perform the EEG response according to the amount of visual depth fixation. Acknowledgments. The study was supported by the NAP (National Agenda Project) of the Korea Research Council of Fundamental Science & Technology.

References 1. Lee, E.C., Park, K.R., Whang, M., Min, K.H.: Measuring the Degree of Eyestrain caused by watching LCD and PDP Devices. International Journal of Industrial Ergonomics 39(5), 798–806 (2009) 2. Lee, E.C., Heo, H., Park, K.R.: The Comparative Measurements of Eyestrain Caused by 2D and 3D Displays. IEEE Transactions on Consumer Electronics 53(6), 1677–1683 (2010) 3. Choi, J., Kim, D., Ham, B., Choi, S., Sohn, K.: Visual Fatigue Evaluation and Enhancement for 2D-Plus Depth Video. In: 17th IEEE International Conference on Image Processing, Hongkong, pp. 2981–2984 (2010) 4. Lambooij, M., Ijsselsteijn, W.: Visual Discomfort and Visual Fatigue of Stereoscopic Display: A Review. Journal of Imaging Science and Technology 53(3), 030201-1–03020114 (2009) 5. Li, H.-C.O., Seo, J., Kham, K., Lee, S.: Measurement of 3D Visual Fatigue using EventRelated Potential (ERP): 3D Oddball Paradigm. In: IEEE 3DTV Conference, Istanbul, pp. 213–216 (2008) 6. Ice Age 3 web site, http://www.iceagemovie.com 7. Gomarus, H.K., Althaus, M., Wijers, A.A., Minderaa, R.B.: The Effects of Memory road and stimulus relevance on the EEG during a Visual Selective memory search task: An ERP and ERD/ERS study. Clinical Neurophysiology 117, 871–884 (2006) 8. Koze, J., Ukai, K.: Subjective Evaluation of Visual Fatigue Caused by Motion Image. Displays 29(2), 159–166 (2008)

A New Design of the Multi-channels Mobile and Wireless EEG System Chin-Teng Lin1, Wan-Ru Wang1, I.-Jan Wang1, Lun-De Liao1, Sheng-Fu Chen3, Kevin Tseng4, and Li-Wei Ko1,2 1

Brain Research Center & Dept. of Electrical Engineering, National Chiao-Tung University, Taiwan 2 Dept. of Biological Science and Technology, National Chiao-Tung University, Taiwan 3 Division of Medical Engineering Research, National Health Research Institutes, Taiwan 4 Dept. of Industrial Design, Chang Gung University, Taiwan [email protected]

Abstract. Most researchers acquired EEG by using standard measurement system like NeuroScan system, which includes AgCl electrode cap, SynAmps Amplifier and Scan software to provide good reliability for the acquisition of EEG data. However, it is still not convenient for Brain Computer Interface (BCI) application in daily life because of needing conduction gels to contact skins and being wired, expensive and heavy. Moreover, the conduction gel will trend to be drying, so it does not suitable for long-term monitoring. In this study, we developed a mobile and wireless EEG system. The system consists of frond-end 16-channel dry electrode cap, a miniature low-power wireless portable circuitry, and a back-end program receiving events and digital EEG data simultaneously. We demonstrate the recorded EEG data have high correlations between from our system and from NeuroScan system. Keywords: EEG, Brain Computer Interface, Mobile and Wireless EEG.

1 Introduction To be practical for routine use in many cognitive experiments in the real operational environment, the data acquisition system must be non-invasive, non-intrusive, lightweight, battery-powered and easy to don and doff. Further, it must enable a full range of head, eye and body movements. The only possible brain imaging modality fulfilling these criteria is electroencephalography (EEG). Electroencephalography is a powerful non-invasive tool widely used for both medical diagnosis and neurobiological research because it can provide high temporal resolution in milliseconds which directly reflects the dynamics of the generating cell assemblies. EEG is also the only brain imaging modality that can be performed without fixing the head/body. Substantial research [1-5] has shown that many features of EEG dynamics index the current state of subject alertness, arousal and attention. However, data collection in most EEG studies requires skin preparation and gel application to ensure good electrical conductivity between sensor and skin. These procedures are time consuming, uncomfortable and even painful for participants since skin preparation C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 293–298, 2011. © Springer-Verlag Berlin Heidelberg 2011

294

C.-T. Lin et al.

usually involves abrasion of the outer skin layer. Repeated skin preparation and gel application for EEG may also cause allergic reactions or infections. Further, the signal quality may degrade over time as the skin regenerates and the conductive gel dries. Advance electrode designs are needed to overcome these requirements and complications of adhesive contacts between EEG electrodes and the skin surface before routine EEG monitoring can be feasible in real-world environments. In this study, we developed a mobile and wireless EEG system. The system consists of frond-end 16-channel dry electrode cap, a miniature low-power wireless portable circuitry, and a back-end program receiving events and digital EEG data simultaneously. We demonstrate the recorded EEG data have high correlations between from our system and from NeuroScan system.

2 Mobile and Wireless EEG System Components Figure 1 shows our proposed 16 channel mobile and wireless EEG system consisted of three parts (1) 16-channel dry electrode cap, which is made of comb-like gilt on every dry electrode. The design of dry electrodes avoids the bad contact in hairy site and can fit the skull shape so that the conductivity is good enough to acquire EEG data. (2) a miniature low-power wireless portable circuitry, which including high-pass filters over 0.5Hz, amplifiers, a 16-to-1 multiplexer, a 12-bit analog-to-digital (A/D) converter, a power management circuit, and a wireless transmission circuit. The microprocessor filters 60Hz noise digitally. The amplifiers are designed about 5500 times. The power operates at 3V for low-consumption. The cut rate of all channels is 2k Hz. The A/D sampling rate is 125Hz for one channel. Moreover, the UART baud rate of Bluetooth is 115200 bit per second. Our proposed system can measure input voltage range about 272.7 uV compared to reference potential. (3) a back-end program, which simultaneously received events of experience and EEG digital data transmitted from Bluetooth. The program plotted the analog signal to display and recorded all data to a file with txt format.

Fig. 1. Components of the proposed 16 channel mobile and wireless EEG system

16-channel Dry Electrode Cap. The cap we used is shown in figure 1(a) which is consisting of 16 metal-made electrodes to acquire EEG data from cortex. We utilize nine probes (3 x 3 arrays) with metal material and spring-loaded features to form each dry electrode. Utilizing the dry electrode on the scalp can directly acquire the EEG signal and does not need to use conductive gels. Further, the signal quality may degrade over time as the skin regenerates and the conductive gel dries. Advance electrode designs are needed to overcome these requirements and complications of

A New Design of the Multi-channels Mobile and Wireless EEG System

295

adhesive contacts between EEG electrodes and the skin surface before routine EEG monitoring can be feasible in real-world environments. In addition, the dry electrode can also apply on the hairy site area to overcome the interference of acquisition problem via bushy hairs. The placements of total 16 dry electrodes are followed by the international 10-20 system. Miniature Low-power Wireless Portable Circuitry. Figure 1(b) shows the development of the proposed miniature, portable and wireless EEG-signal acquisition unit, which combining high-pass filters over 0.5Hz, amplifiers, a 16-to-1 multiplexer, a 12-bit analog-to-digital (A/D) converter, power management circuit, and wireless transmission circuit into one. Instrumentation Amplifier (IA) which has a high common-mode rejection ratio (CMRR) is good for the applications that the signal of interest is represented by a small voltage fluctuation superimposed on a voltage offset. The operational amplifiers are designed about 5500 times and filter the signals over 0.5 Hz. Next, the 16-channel input signals sequentially pass a 16-to-1 multiplexer with 2k-Hz cut rate every channel. The Analog-to-Digital (A/D) converter samples the data at 125-Hz rate all channels. The digital data are saved in buffers of a microcontroller and then are carried to wireless transmission circuit through UART. The baud rate of Bluetooth is 115200 bit per second. In the unit, The power management circuit operates at 3V for low-consumption and has recharge function. Thus, our proposed acquisition circuit can monitor input voltage range about 272.7 uV compared to reference potential and lossless the EEG signal of interest. EEG Data Acquisitions and Signal Processing. Figure 1(c) is the display of 16 channels EEG signal acquisition. A back-end record program, which simultaneously received events of experience and EEG digital data transmitted from Bluetooth wireless module. The program display the digital-to-analog signal on a frame in realtime and can also record all channel data to a file with TXT file format.

3 Comparison Experiments Design and Results For evaluating the data acquisition performance and feasibility of the proposed 16 channel mobile and wireless EEG system. We apply two kinds of testing experiments to the proposed system. First testing is using the simulation data generated by a function generator and the other one is real EEG signals acquisition comparison between our proposed wireless EEG system and the commercial NeuroScan system. Simulation Data Testing. Here we utilized the function generator to produce 5 Hz input signals and feed the simulated signals into one channel of the acquisition circuit. The record data received by the display program was compared to the standard sin wave generated by MATLAB. We generated the arbitrary simulated signals as the special EEG features for the system comparison. In Figure 2, the blue line is showed the arbitrary simulated signals generated by function generator and red line is the acquisition performance from our development of miniature low-power wireless portable circuitry. Although the sampling size of our record data is smaller than of the simulated wave, it’s no influence that the special features of Signals recorded by our system are highly correlated to the simulated signals via function generator.

296

C.-T. Lin et al.

Then, we generated four different frequency bands (5, 10, 15 and 20 Hz) simulated signals to test the comparison results. As shown in figure 3, after doing Fast Fourier transform, we can easily see that the major peak of the high power is consisted and matched in each frequency of the generated signals.

Fig. 2. Comparison Results of the Simulation Data Testing in Time Domain

Fig. 3. Comparison Results of the Simulation Data Testing in Frequency Domain

Real EEG Data Acquisition Testing. In this section, we acquire the real EEG signals from five different channel placements (Fp1, T3, Cz, T4, and Oz) in the international 10-20 system and show the comparison results between our proposed wireless EEG system and the commercial NeuroScan system. Table 1 shows the average correlations in time domain of the EEG signal fluctuations between our proposed wireless EEG system and the commercial NeuroScan system. It is clear to see that the correlations can reach 88.34%. Table 1. Correlation Results of the Real EEG Data Acquisition in Time Domain Placement FP1 T3 CZ T4 OZ Average

10-s Max correlation (%) 96.75 90.46 94.48 90.08 88.98 92.15

Fig. 4. Oz Signal Fluctuation between the proposed mobile and wireless EEG (Blue line) and commercial NeuroScan System (red line) and the correlation results (black line)

10-s Average correlation (%) 93.01 86.80 91.82 88.19 81.87 88.34

Fig. 5. Frequency Fluctuation in Oz channel between the proposed mobile and wireless EEG (Blue line) and commercial NeuroScan System (red line)

A New Design of the Multi-channels Mobile and Wireless EEG System

297

Here we plot the Oz signal fluctuation as the sample case shown in figure 4. It shows 10 sec results. EEG signal acquired from our proposed mobile and wireless EEG system is showed in blue line, red line is acquired by the commercial NeuroScan system, and the top black line shows the correlations of the 1 sec EEG signal fluctuation. Almost the correlations are higher than 85%. Table 2 shows the average correlations in frequency domain of the EEG signal fluctuations between our proposed wireless EEG system and the commercial NeuroScan system. It is clear to see that the correlations can reach 95.44%. Table 2. Correlation Results of the Real EEG Data Acquisition in Time Domain Electrode place FP1 T3 CZ T4 OZ Average

10-s Max correlation (%) 98.36 98.41 97.18 98.26 98.42 98.13

10-s Average correlation (%) 96.53 94.89 89.08 95.44 88.69 92.93

We also plot the frequency fluctuation of the Oz signal after FFT as the sample case shown in figure 5. It shows the 10 sec average correlation result. Blue line is the result of the proposed mobile and wireless EEG and red line is the result of the commercial NeuroScan System. Maximum correlation in 10 sec can reach 98.42% and average correlation in 10 sec can be 88.69%.

4 Discussion and Conclusion According to the experimental results, the data acquisition performance of our proposed 16-channel mobile and wireless EEG is highly correlated with the commercial EEG system (NeuroScan). The experimental results of this study also provides a new insight into the understanding of complex brain functions of participants actively performing ordinary tasks in natural body positions and situations in operational environments. Such data would be difficult or impossible to obtain in a standard EEG laboratory where participants are asked to limit their eye blinks, teeth clinching or other head/ body movements. These experimental results may also be applied in future studies to elucidate the limitations of normal human performance in repetitive task environments and may inspire more detailed study of changes in cognitive dynamics in brain-damaged, diseased or genetically abnormal individuals. The proposed system has many potential applications in clinical research and practice in such diverse fields as neurology, psychiatry, gerontology and rehabilitative medicine. Acknowledgment. This work was supported in part by the UST-UCSD International Center of Excellence in Advanced Bio-engineering sponsored by the Taiwan National Science Council I-RiCE Program under Grant Number: NSC-99-2911-I-009-101, in part by the Aiming for the Top University Plan of National Chiao Tung University, the Ministry of Education, Taiwan, under Contract 99W906, and in part supported by

298

C.-T. Lin et al.

the National Science Council, Taiwan, under Contracts NSC 99-3114-E-009-167 and NSC 99-2628-E-009-091. Research was also sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF10-2-0022. The views and the conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S Government. The U.S Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.

References 1. Lin, C.T., Ko, L.W., Chiou, J.C., Duann, J.R., Huang, R.S., Chiu, T.W., Liang, S.F., Jung, T.P.: Noninvasive Neural Prostheses Using Mobile & Wireless EEG. Proceedings of the IEEE 96(7), 1167–1183 (2008) 2. Pilutti, T., Ulsoy, G.: Identification of Driver State for Lane-Keeping Tasks. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans 29, 486–502 (1999) 3. Jung, T.P., Makeig, S., Stensmo, M., Sejnowski, T.J.: Estimating alertness from the EEG power spectrum. IEEE Transactions on Biomedical Engineering 44(1), 60–69 (1997) 4. Makeig, S., Jung, T.P.: Changes in alertness are a principal component of variance in the EEG spectrum. Neuroreport 7(1), 213–216 (1995) 5. Eoh, H.J., Chung, M.K., Kim, S.H.: Electroencephalographic study of drowsiness in simulated driving with sleep deprivation. International Journal of Industrial Ergonomics 35, 307–320 (2005)

An Experimental Comparison of Brain Activity in Professional and Non-professional Sewers during the Use of Sewing Needles Masako Omori, Yukari Morishita, and Asuka Kawakita Faculty of Home Economics, Kobe Women’s University 2-1 Aoyama, Higashisuma, Suma-ku, Kobe-city 654-8585, Japan [email protected]

Abstract. The purpose of this experiment is to analyze differences in brain activity of people using hand motor skills. The subjects were twelve healthy female adults with professional or non-professional sewing abilities. Near infrared radiation spectroscopy (NIRS) was used to measure the brain activity of each subject related to the movement of fingers during the “needling” process. Furthermore, the finger muscle activity was measured to assist in calculating the value of the integral during the needling. “Needling” performance (number of needle stitches, needle stitch intervals, and completion time) was also recorded. The study required the subjects to sew four types of stitches in 20 second intervals. The results show that while the subjects with professional skills made a significantly higher number of needle stitches, there was no significant difference between the non-professionals. No major muscle activity occurred in either group, although there was some increase in force used by the non-professionals. An increasing tendency was found in the presence of oxy-Hb in the brain activity, which corresponded with the rhythm of the muscle activity. In the professional group it was present in the prefrontal cortex and the motor cortex during the related movement of needling. These results are similar to those found in a previous study. The findings of this study may be useful in understanding the skill level of non-expert sewers by analyzing either the brain activity or changes in the muscle activity of the fingers. Keywords: Near infrared radiation spectroscopy (NIRS), sewing, cortical activation, muscle activity and skill level.

1 Introduction Kawabata and Narumi [1] reported a questionnaire survey that asked participants’ about their frequency in engaging in play, their confidence towards play, their study preferences, and their attitude towards making things by hand. Sewing seems to have a mutual relationship with skillfulness in the fingers and hands, and attitudes towards making things by hand. It may be, therefore, that sewing studies can nurture positive attitudes towards various other kinds of learning and activities using the hands, as well as help to develop skillfulness in fingers and hands. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 299–303, 2011. © Springer-Verlag Berlin Heidelberg 2011

300

M. Omori, Y. Morishita, and A. Kawakita

Recently, near-infrared spectroscopy (NIRS) has been developed that can measure cortical activation during dynamic movements such as walking and running [2, 3]. It has also been reported that motor learning consists of three or two distinct phases, designated the initial, intermediate, and advanced phases [4]. The purpose of this experiment is to analyze differences in brain activity of people using hand motor skills.

2 Methods 2.1 Subjects A total of twelve healthy subjects, two professional and ten non-professional sewers, participated in this experiment (mean age 25±9.4 years old). All participants gave written informed consent. The professional sewers were university teachers (mean age 48.5 years old). The non-professional sewers were sewing novices at Kobe Women’s University. However, they had a little experience with sewing needles in junior and high school. 2.2 Experiment Procedure Two different stitch intervals were used (4 mm and 2 mm). Sewing was also done with the. Thus, there were four conditions in total. EMG (electromograph) electrodes were attached to the subjects, who wore the optical-fiber probe of NIRS. They then sat on a chair in a relaxed manner with a cloth and needle for sewing. The interval included a pre-rest period of 20 seconds, stitching period of 20 seconds, and rest of 40 seconds before going to the next required set of stitches. Each cycle was done three times so that the four required stitches were done twelve times. The testing was done twice on two separate days, so that the subjects completed 24 sets of the required stitches. 2.3 Functional NIRS (fNIRS) Cortical activation was assessed in all subjects by task related changes in Hb oxygenation using an fNIRS system (FOIRE-3000; Shimadzu, Kyoto, Japan). Concentrations of oxy-Hb in the prefrontal cortex and motor cortex of subjects while they sewed were measured by a 48-channel fNIRS system. The fNIRS sensor was placed on the prefrontal cortex and motor cortex on the subject’s head, covering Cz according to the international ten-twenty system [5]. 2.4 Analyses fNIRS was used to measure the brain activity of the subjects including oxy-Hb, deoxy-Hb, and total-Hb around the cerebral cortex. The analysis included an assessment of the amount of relative change of oxy-Hb on 49 channels in the prefrontal cortex and motor cortex area of association during the movement in the needling period.

An Experimental Comparison of Brain Activity

301

PowerLab 4/25 (ADInstrments, inc.) was used in measuring the muscle activity in the fingers to assist in calculating the value of the integral during the needling. The muscle activity calculated the value of the integral during the needling.

3 Results 3.1 Number of Needle Stitches The needling performance (number of needle stitches, needle stitch intervals) was recorded, and it was found that the number of stitches of the professional sewers was higher than that of the non-professionals. However, no significant difference was seen among non-professionals. However, the non-professional subjects showed significant differences among the tasks (p<0.001) (Figure 1). The results of multiple comparison tests were 4 mm: 1 time–4 mm: 2 times (p<0.01), 4 mm: 1 time–2 mm: 2 times (p<0.001), 2 mm: 1 time–4 mm: 2 times (p<0.05), and 2 mm: 1 time–2 mm: 2 times (p<0.01). The number of needle stitches was increased 2 times than 1 time. Thus, the results suggested the effect of the sewing lesson.

number of stitches

***: p<0.001 **: p<0.01 inducement line presence absence

4mm

2mm 1time

2mm

4mm 2time

Fig. 1. The number of stitches of the professional sewers and non-professionals

3.2 Muscle and Brain Activity Muscle activity of the fingers and hands or a rhythm appeared in the professionals, and while the non-professionals applied excessive power, there was also little change in their muscle activity. Figure 2 shows the results for muscle activity of the professional and non-professionals. Figure 3 shows the result of integral value during the needling. Therefore, the increasing presence of oxy-Hb found with brain activity corresponded with the rhythm of the muscle activity, and in the professional group it was present in the prefrontal cortex and the motor cortex during the related movement of needling (Figure 2).

302

M. Omori, Y. Morishita, and A. Kawakita

a) professional right

left hand

b)non-professional

right h d

left hand

c)non-professional right h d

left hand

integral value(mV.s)

Fig. 2. Results of muscle and cortical activation change by sewing tasks

Left hand

Right hand

inducement line presence absence

2mm 4mm non-professional

2mm 4mm professional

2mm 4mm non-professional

2mm 4mm professional

Fig. 3. Result of integral value during the needling

An Experimental Comparison of Brain Activity

303

4 Discussion Motor skills progress from explicit control in the early stages of learning to more implicit or automatic control after thorough learning. Halsband et al. [6] suggested that motor learning consists of three distinct phases, designated the initial, intermediate, and advanced phases. In this experiment, globally increased levels of oxy-Hb were found in the motor cortex and prefrontal cortex of subjects in whom rhythmic muscle activity of the fingers and hands had appeared in the professionals and non-professionals. We did not observe any changes in oxy-Hb in the prefrontal cortex of professionals or non-professionals. However, markedly increased levels of oxy-Hb were seen in the motor cortex and prefrontal cortex of non-professionals in whom rhythmic muscle activity had appeared. This suggests that these subjects are at the advanced stage of motor learning. Therefore, the present study suggests that fNIRS systems may be useful for assessing the stage of motor learning in individual subjects who have undergone training to acquire sewing skills.

5 Conclusion The present study investigated brain activation during sewing using an fNIRS system, and revealed that activation of the prefrontal cortex and motor cortex are involved in learning of sewing skills, especially during the initial phase of motor learning. The results suggest that fNIRS may provide important insights into the future development of teaching, training and assessment methods for sewing skills.

References 1. Kawabata, H., Narumi, T.: An Analysis of Skillfulness in Fingers/Hands of School Children in Play and Study. Journal of Home Economics of Japan 60(2), 123–131 (2009) 2. Hikosaka, O., Nkamura, K., Sakaki, K., Nakahara, H.: Central mechanisms of motor skill learning. Curr. Opin. Neurobiol. 12, 217–222 (2002) 3. Jaeger, J.J., Lockwood, A.H., Van Valin Jr., R.D., Kemmerer, D.L., Murphy, B.W., Wack, D.S.: Sex differecese in brain regions activated by grammatical and reading asks. NeuroReport 9, 2803–2807 (1998) 4. Grantcharov, T.P., Bardram, L., Funch-Jensen, P., Rosenberg, J.: Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills. Am. J. Surg. 185, 146–149 5. Jaspers, H.: The ten-twenty electrobe system of the international Federation. Electroencephaloger 10, 37–375 (1958) 6. Halsband, U., Lange, R.K.: Motor learning in man: a review of functional and clinical studies. J. Physiol. 99, 414–424 (2006)

EEG-Based Measure of Cognitive Workload during a Mental Arithmetic Task Brice Rebsamen1, Kenneth Kwok1, and Trevor B. Penney2 1

Temasek Laboratories, National University of Singapore, Singapore Dept of Psychology, National University of Singapore, Singapore {tslrb,kenkwok,penney}@nus.edu.sg

2

Abstract. We collected EEG data from 16 subjects while they performed a mental arithmetic task at five different levels of difficulty. A classifier was trained to discriminate between three conditions: relaxed, low workload and high workload, using spectral features of the EEG. We obtained an average classification accuracy of 62%. A continuous workload index was obtained by low-pass filtering the classifier's output. The average correlation coefficient between the resulting workload index and the difficulty level of the task was 0.6. Keywords: EEG, mental workload, arithmetic, cognitive state.

1 Introduction Excessive cognitive workload can lead to delayed information processing, response errors, or a failure to respond, but a too low cognitive workload can also lead to errors due to boredom. Thus, the ability to measure cognitive workload in real-time might be valuable in enforcing industrial safety, in improving human-computer interfaces, and designing appropriate adaptive automation strategies [1, 2]. Analysis of workload using EEG data has been attempted in several studies. Gevins and Smith [3] discriminated three workload levels in a task involving sequential memorization of stimuli using spectral power in the theta and alpha frequency bands. Kohlmorgen et al. [4] designed an online system to measure workload from EEG, used it to manage high workload situations during driving and showed that this procedure improved the reaction times of drivers. Putze, Jarvis and Schultz [5] proposed a multi-modal approach, taking into account biological signals such as skin conductance, pulse and respiration and EEG, to classify several levels of workload during a driving task. In this paper we present the results of EEG data classification of a dataset obtained from 16 participants who performed a mental arithmetic task with five levels of difficulty. A continuous workload index based on the classifier's output is proposed and evaluated.

2 Method Data were collected from 16 students, 9 males and 7 females, with a mean age of 21.5 (SD=1.5) years. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 304–307, 2011. © Springer-Verlag Berlin Heidelberg 2011

EEG-Based Measure of Cognitive Workload during a Mental Arithmetic Task

305

EEG data was recorded from 16 channels (FP1, FP2, F3, Fz, F4, T7, C3, Cz, C4, T8, P3, Pz, P4, O1, Oz, O2) at 256 Hz with an ActiveTwo system from Biosemi, referenced to the nose. The task consisted of summing mentally two figures presented on the computer screen and comparing the result to a proposed answer, also presented onscreen. There were 5 possible difficulty levels. At level 1, the problems consisted of summing two one-digit figures, and each subsequent level included an extra digit such that at level 5 the problems consisted of summing two three-digit figures. The experiment was segmented in blocks of one minute, with a thirty-second rest period between each block. The rest period was introduced to avoid excessive fatigue, so that differences between block would be on workload only and not fatigue. Within a block, all problems were of the same difficulty level (1 to 5). After three repetitions of the six difficulty levels, a slide-show of landscape pictures (one picture every thirty seconds) was presented for five minutes to allow the participant to relax. Pictures were favored upon a fixation cross to avoid boring the subject too much. The whole session was then repeated (18 blocks of mental arithmetic and 5 minutes of relaxation).

3 Results 3.1 Behavior Analysis For each subject, we observed an increase of the mean response time with the difficulty level. Similarly, subjects made more errors for higher levels of difficulty. Although there were clear performance differences between subjects, they all showed the same behavior: response time and error rates increased with the difficulty level. We also investigated the effect of time on task on the response time and the error rate. For each subject we computed the difference between the first and last third of the trials. No statistically significant difference was found for either response time (p=0.06) or error rate (p=0.26). Hence, subjects did not seem to be affected by time on task (fatigue, boredom, etc.). 3.2 Classification EEG analysis was done using PyMVPA [6], a multivariate pattern analysis package for Python. The continuous EEG was first epoched in three second windows, with a two second overlap between consecutive windows. Epochs were labeled with either the difficulty level (LVL1 to LVL5) or as RELAXATION. This yielded approximately 2000 epochs, around 300 per label (900 for the RELAXATION condition). The power spectrum for each epoch was computed using the fast Fourier transform (FFT) and a Hanning window. The power spectrum was reduced to the total power in the following five frequency bands: delta (1Hz-4Hz), theta (4Hz-8Hz), alpha (8Hz14Hz), beta (14Hz-35Hz) and gamma (35Hz-45Hz). The resulting feature vector hence contained 80 features (16 channels and 5 frequency bands). A SMLR (Sparse Multinomial Logistic Regression) classifier [7] was trained to discriminate between three classes: relaxed, low workload (LVL1 and LVL2), and

306

B. Rebsamen, K. Kwok, and T.B. Penney

high workload (LVL3 to LVL5). For each class, a workload value was assigned: 0 for the relaxed class, 2.5 for the low workload class, and 5 for the high workload class. The system was evaluated using a two-fold validation procedure: the classifier was first trained on the first half of the data (up to after the first five minute relaxation period), and applied to the second half; it was then trained with the second half and applied to the first half. Across subjects, the mean accuracy was 62% (SD=4.3), which is significantly higher than the 33% chance level (p < 0.001). The classifier assigned to each epoch a discrete workload value corresponding to the output class. These discrete values were then low passed filtered over the previous 10 epochs, which yielded a continuous index of workload, ranging from 0 to 5 (an example of the result is provide in Fig. 1). We measured the correlation between the continuous workload index, and the task difficulty, which was taken as the difficulty level for the task condition and as 0 for the other conditions. Across subjects, the mean correlation coefficient was 0.6 (SD=0.08), which is significantly higher than chance (p < 0.001).

Fig. 1. An example of the resulting workload index: the task difficulty level is in black, the workload index in blue

4 Discussion We classified EEG data from 16 participants who were performing a mental arithmetic task with 5 levels of difficulty. For each participant, a classifier was trained to classify spectral features into 3 classes: relaxed, low workload, and high workload. The resulting accuracy was 62% on average across participants, which was significantly higher than chance. A continuous workload index was created by assigning a workload value to each class and low-pass filtering the result. The correlation between the task difficulty level and the resulting workload index was 0.6 on average across participants, which was also significantly higher than chance.

EEG-Based Measure of Cognitive Workload during a Mental Arithmetic Task

307

These results constitute a first step toward a real-time system to measure cognitive workload from EEG. However, development of an operational workload measurement system requires much additional work. In particular the following points will require our attention: Ó intra-subject consistency: once the system has been trained for a participant, does the prediction accuracy degrade with time? Ó inter-subject consistency: can a system trained with data from one participant be used with another participant? How does it affect the performance? Ó task independence: the current results were obtained with participants performing mental arithmetic in controlled conditions. What will the system give when the participant is performing a real-world task? Ó robustness to noise: how will the system behave in a non-controlled situation with more artifact sources (movements, sweat, etc.)?

References 1. Parasuraman, R., Mouloua, M., Molloy, R.: Effects of adaptive task allocation on monitoring of automated systems. Human Factors 38, 665–679 (1996) 2. Wiener, E.L.: Beyond the sterile cockpit. Human Factors 27, 75–90 (1985) 3. Gevins, A., Smith, M.: Neurophysiological measures of cognitive workload during humancomputer interaction. Theoretical Issues in Ergonomics Science 4(1), 113–131 (2003) 4. Kohlmorgen, J., Dornhege, G., Braun, M., Blankertz, B., Müller, K., Curio, G., Hagemann, K., Bruns, A., Schrauf, M., Kincses, W.: Improving human performance in a real operating environment through real-time mental workload detection. In: Toward Brain-Computer Interfacing, pp. 409–422. MIT Press, Cambridge (2007) 5. Putze, F., Jarvis, J., Schultz, T.: Multimodal Recognition of Cognitive Workload for Multitasking. In: The Car. Int. Conf. on Pattern Recognition (2010) 6. Hanke, M., Halchenko, Y.O., Haxby, J.V., Pollmann, S.: Statistical Learning Analysis in Neuroscience: Aiming for Transparency. Frontiers in Neuroscience, 4 (2010) 7. Krishnapuram, B., Carin, L., Figueiredo, M.A.T., Hartemink, A.J.: Sparse multinomial logistic regression: Fast algorithms and generalization bounds. IEEE Trans. on Pattern Analysis and Machine Intelligence, 957–968 (2005)

EEG Measurements towards Brain Life-Log System in Outdoor Environment Hideaki Touyama1 and Kazuya Maeda 1 Toyama Prefectural University 5180 Kurokawa, Imizu, Toyama 939-0398, Japan [email protected]

Abstract. In this paper, we studied electroencephalogram during ambulatory conditions in outdoor environment. Five healthy subjects participated in this experiment. The task of the self-paced walking subjects was to count the number of appearances of the target auditory stimulus using oddball paradigm. We observed P300 evoked potentials in ambulatory conditions in outdoor environment as well as sitting conditions in indoor environment. Our results are encouraging and make new direction to promising novel applications of ambulatory BCIs. Keywords: ElectroEncephaloGram (EEG), P300 Evoked Potentials, Ambulatory, Brain-Computer Interface (BCI), Life-Log, Outdoor Environment.

1 Introduction Brain-Computer Interfaces (BCIs) [1] provide communication channels that enable users to interact with computers only by means of brain activity. Most non-invasive BCIs are based on ElectroEncephaloGraphy (EEG), which provides a portable and a high time-resolution. Owing to these advantages, the novel BCI applications have been investigated in laboratories [2]-[4]. Recently, in research field of wearable computing, the concept of Life-Log technology has lately attracted considerable attention. Owing to dissemination of mobile, wearable or ubiquitous devices, people can record and collect the data (images, sounds, timestamp, positions etc.) of their experiences in their ordinary lives. With such technology, we might be blessed with a variety of useful applications such as ‘MyLifeBits’ [5]. One of the problems of the Life-Log technology is concerned with the data retrieval and indexing. In most cases, the data retriever has to operate the computer devices such as mouse and keyboard in order to seek one specific data that the user wants to pick up from the archives. Moreover, the automatic indexing to the Life-Log in which people are interested and find the specific emotions has not been established in the present stage. In this paper, we focused on the detection of brain activities of P300 evoked potentials in ambulatory context in outdoor environment towards Life-Log retrieval and indexing. It is well known the P300 evoked potential is widely used to drive BCI systems [2]. The P300 is a positive increase of the EEG amplitude which appears C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 308–311, 2011. © Springer-Verlag Berlin Heidelberg 2011

EEG Measurements towards Brain Life-Log System in Outdoor Environment

309

approximately 300 ms after the user has perceived a relevant and rare stimulus. The P300 is expected to be a very promising signal for potential Life-Log application. This paper is organized as follows: the next section describes the EEG recording experiments we have conducted. The third section presents the results of EEG waveforms in the ambulatory conditions. The last section presents discussion and conclusion.

2 Experiments Five healthy subjects participated in this experiment. During the experiment in outdoor, self-paced walking subjects could hear two brief sounds in the headphones. For comparison, the experiments were performed in sitting conditions in indoor environment. The EEG measurements were recorded in and around a campus of our university (Toyama Prefectural University). Two brief sounds were generated every 1s and appeared in a random order. One of the two brief sounds, the so-called target stimuli, was less frequent than the other nontarget stimuli. The task of the subjects in the oddball paradigm was to count the number of appearances of the target stimulus. As this stimulus was rare and thanks to the counting instruction, its appearance was expected to trigger a P300 in the subject’s EEG signals. The scalp electrodes were applied in order to perform EEG recordings. In this study, one-channel EEG signals were investigated from Cz according to the international 10/20 system [6]. A body-earth and a reference electrode were on a forehead and on a left ear lobe, respectively. The analogue EEG signals were amplified at a multi-channel bio-signal amplifier (Polymate II AP216, TEAC Corp. Japan) which is compact and portable. The subject was wearing a backpack in which the compact amplifier and computer were included. The amplified signals were sampled at 200 Hz. The digitized EEG data was stored in the computer. A subject during EEG measurement is shown in Figure 1.

Fig. 1. A subject during EEG measurement

310

H. Touyama and K. Maeda

3 Results An important point to check when doing P300 experiments is whether there was an actual P300 evoked by the target stimuli. In order to do so, we averaged on one hand the target trials altogether and on the other hand the non-target trials altogether. The resulting waveforms for electrode Cz are displayed in Figure 2. On these figures, we can observe a positive increase of amplitude appearing around 300 ms after t = 0 s (i.e., after stimulus presentation) in ambulatory conditions as well as sitting conditions. This confirms the presence of a P300 during the target trials. Note that we confirmed the subjects could perform the oddball tasks correctly by checking the number of the counts with button. -15

Target Nontarget

Ambulatory Condition

-10

]V [μ-5 ed uit 0 pl m A 5 G E E 10

150

0.2

-15

0.4

Time[sec]

0.6

0.8

Sitting Condition

-10

1

Target Nontarget

]V μ [ -5 ed uit 0 pl m A5 G E E 10

150

0.2

0.4

Time[sec]

0.6

0.8

1

Fig. 2. Grand average EEG waveforms in ambulatory conditions in outdoor environment (upper) and sitting conditions in indoor environment (lower). The solid and dotted lines correspond to target and non-target tasks, respectively.

4 Discussion and Conclusion In Figure 2, we observe P300 evoked potentials in ambulatory conditions in outdoor environment as well as sitting conditions in indoor environment. In the grand average EEG waveforms, the amplitude in ambulatory conditions is smaller than that in sitting conditions. This might be caused by the muscle artifact as well as the noise from swinging electrode lines. Lotte et al mentioned that being seated may reduce the signal noise, but it may also impact negatively the mental states of some subjects [7]. They hypothesized that in

EEG Measurements towards Brain Life-Log System in Outdoor Environment

311

the walking condition subjects were more motivated and engaged in the task than during the sitting condition which was a more passive task. Actually, in our experiment, the subjects were more motivated while walking in outdoor. Furthermore, Gwin et al reported the possibility of removal of movement artifact during walking and running conditions using tread mill in indoor environment [8]. It is important to reveal EEG classification performances using single trial EEG signals in outdoor, which we have not investigated in this work. However, Maeda et al reported the high classification performance could be obtained using single trial P300 signals in ambulatory conditions in outdoor environment [9]. It might be true that our results are encouraging and making a new direction to promising novel applications of ambulatory BCIs. Furthermore, our results may be applied to new input method in immersive virtual environment such as CAVE system, in which the users can move to some extent interacting with the virtual space or objects. Finally, our next work will be dedicated to confirm these results with more subjects and to explore new application using online Brain Life-Log System in order to extract the important events from the environments.

References 1. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain computer interfaces for communication and control. Clinical Neurophysiology 113(6), 767– 791 (2002) 2. Farwell, L.A., Donchin, E.: Walking off the top of your head: toward a mental prosthesis utilizing event-related brain potential. Electroencephalography and Clinical Neurophysiology 70, 510–523 (1998) 3. Pfurtscheller, G., Leeb, R., Keinrath, C., Friedman, D., Neuper, C., Guger, C., Slater, M.: Walking from thought. Brain Research 1071(1), 145–152 (2006) 4. Touyama, H., Hirose, M.: Brain Computer Interface via Stereoscopic Images in CAVE. In: Smith, M.J., Salvendy, G. (eds.) HCII 2007. LNCS, vol. 4557, pp. 1004–1007. Springer, Heidelberg (2007) 5. Gemmel, J., Bell, G., Lueder, R., Drucker, S., Wong, C.: My Life Bits: Fulfilling the Memex Vision. ACM Multimedia System Journal, 235–238 (2002) 6. Jasper, H.H.: The ten-twenty electrode system of the international federation. Electroenceph. Clin. Neurophysiol. 10, 370–375 (1958) 7. Lotte, F., Fujisawa, J., Touyama, H., Ito, R., Hirose, M., Lécuyer, A.: Towards Ambulatory Brain-Computer Interfaces: A Pilot Study with P300 Signals. In: 5th Advances in Computer Entertainment Technology Conference (ACE), pp. 336–339 (2009) 8. Gwin, J.T., Gramann, K., Makeig, S., Ferris, D.P.: Removal of Movement Artifact From High-Density EEG Recorded During Walking and Running. J. Neurophysiol. 103, 3526– 3534 (2010) 9. Maeda, K., Touyama, H.: Event-related potentials in ambulatory context for indexing of Life Log in outdoor. Technical Committee on Human Information Processing (HIP) (2010) (in Japanese)

Part VI

Ergonomics and Human Modelling Issues

On the Applicability of Digital Human Models for Personal Equipment Design Thomas Alexander and Jessica Conradi Fraunhofer Institute for Communication, Information Processing and Ergonomics (FKIE) Neuenahrer Str. 20, 53343 Wachtberg, Germany {Thomas.Alexander,Jessica.Conradi}@fkie.fraunhofer.de

Abstract. Digital Human Models (DHMs) have developed from academic research into valuable engineering tools. They offer a vast amount of functionality for modeling and simulation of anthropometric dimensions, reach, vision, movement, and comfort. Today’s DHMs are primarily used in automotive design. As our previous studies have shown, a simple transfer to other application domains (e.g. workplace design) may be possible but result into inaccuracies. With the following study we analyzed the applicability of two typical DHMs for the design of personal equipment and for identifying potential conflicts between different pieces of equipment. Such an application differs strongly from automotive applications, but many similar uses of DHMs can be observed in daily life. The analysis addresses different aspects: At first, the general functionality of the DHMs was screened. Required functions were either available instantly, available by workarounds, or unavailable. A subsequent verification and validation study analyzed potential shortcoming for the available functions. It consists of an empirical survey with different typical soldiers’ movements. The data include maximum reach, focusing a target with a binocular or aiming with different postures. A number of motion sequences were selected for the analysis. Results show that simple movements were modeled quite accurately. But complex movements were hardly possible to model and large inaccuracies were observed. An important functional shortcoming resulted from limited possibilities to integrate virtual objects of personal equipment into the DHM. This prevented an analysis of conflicts between personal equipment. Keywords: Digital Human Models, personal equipment, motion capture, verification & validation.

1 Introduction Spatial workplace and vehicle design have always been core domains for ergonomics and human factors. Body dimensions, postures and movements of future workers define basic requirements. In this connection, Digital Human Models (DHM) have been developed and continuously improved [1]. Most of them have a background in vehicle design and include primarily functionality addressing design issues from this domain [2, 3]. Results from our previous studies have shown that inaccuracies occur C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 315–319, 2011. © Springer-Verlag Berlin Heidelberg 2011

316

T. Alexander and J. Conradi

even with similar applications [4]. It is therefore necessary to be well aware of them and to consider them appropriately. With the following study we analyzed the applicability of two DHMs for the design of personal equipment. This is very different from workplace and vehicle design and it is expected that there are many restrictions and shortcomings. However, similar applications of DHMs are often observed in reality. In fact, designers and engineers are doing quite well in overcoming functional shortcomings by workarounds. This is critical because the results of these workarounds have high facevalidity and –especially with CAD-systems- are supposed to have a high degree of accuracy although this is often not the case. Therefore it is important to analyze and estimate the applicability of DHMs for such applications.

2 Method To evaluate the effects of applying DHMs for the design of personal equipment an insight analysis was carried out. It addresses three different aspects: Availability of general functionality, validity of anthropometric modeling and validity of biomechanic modeling. Two DHMs were selected in order to represent different application backgrounds: One has a background in car interior design (DHM RAMSIS) and many verification and validation studies have been carried out for this [5]. The other (DHM JACK) has a background in workplace design [6]. 2.1 Relevant Aspects for Applicability With regard to functionality the DHMs provide functions either instantly, they are available by workarounds, or unavailable. It is clear that the required functions for the design of personal equipment differed from those for workplace or car interior design. However, basic functions, e.g., anthropometric modeling, basic posture modeling, reaching to targets etc. are available for most DHMs. The next relevant aspects are verification and validation of the anthropometric and (static) posture model, especially with regard to functional measurements and reach ranges. The final aspect for applicability focuses human movement and motion dynamics. 2.2 Analysis of General Functionality The general functionality addresses basic issues of the DHM, e.g., anthropometric database, ranges of maximum reach, force and comfort ratings, posture simulation, and sight analysis on the one hand. On the other hand it considers functional and application-oriented issues, e.g., DHM-user group, platform, inclusion of additional objects in the analysis. The latter is important for the design of personal equipment because of the interaction of different pieces of equipment. It is often observed that missing functionality is substituted by workarounds. This is critical because it introduces errors.

On the Applicability of Digital Human Models for Personal Equipment Design

317

2.3 Anthropometric and Posture Model Most DHMs rely on standardized anthropometric dimensions and a few functional measures. But the majority of relevant postures for designing personal equipment is not standardized and requires additional manual adjustments. The same is true for different clothing – standard measures are captured without clothing which rarely occurs in daily life. For our analysis, the anthropometric dimensions of a small sample (n=30, age: 21.6 ± 2.0 yrs) were measured according to anthropometric standards. Different types of clothing were also considered. Virtual representations were set up using the DHM and subsequently modified to match with real-world postures. The simulated postures were then compared to postures from a motion-capture experiment. 2.4 Human Movement and Motion Dynamics Personal equipment is often used while walking or changing postures. Therefore it is important to consider the dynamic characteristics accordingly [7]. It is important that the DHM models and simulates relevant movements and motion dynamics correctly. In our analysis, participants performed different movements with different pieces of personal equipment. The motions were captured using an infrared, marker-based motion capture system. By identifying and marking 18 anthropometric landmarks it was possible to use the motion data for a comparison between simulated and real movements. Additional markers were attached to the personal equipment.

3 Results The analysis of functionality revealed that functionality is available or can be achieved by workarounds. An integration of virtual objects is limited and it does often not include collision detection or simulation of kinematic behaviors and physics. This is especially true for modeling of apparel and clothing which is not available for engineering DHMs. A detailed analysis of interrelationships between different pieces of personal equipment is not possible. It is limited to a pure visual analysis. The analysis of the anthropometric and static posture model showed differences between real and simulated dimensions. It is important to notice that neither of the DHMs was a real contour model, so that important measures as breadth and circumferences of body dimensions lack of accuracy. Differences of circumferences of arm and leg range between 2.7 cm (upper leg) and 6 cm (upper arm) on the average. Another effect is that soft tissue deformations are not considered and seating postures are over-estimated (e.g. seating height 0.5 – 3.9 cm larger than in reality). The inaccuracies increase during movements. An important functional measurement is reach. The analysis reveals that differences vary vastly between both models. This is shown in table 1. According to the results, it depends on the DHM whether reach is over- or underestimated. The analysis of the reach envelopes supports this finding. Again, the validity of the simulation depends upon the special model; our study revealed that one model calculated a reach envelope that was too small while the other an envelope that was too large. Fig.1 illustrates the difference. Notice that the real reach is between both spheres.

318

T. Alexander and J. Conradi Table 1. Differences [cm] in reach for both DHMS (real measurement is zero)

Maximum span Forward maximum reach

DHM RAMSIS Jack RAMSIS JACK

Mean 0,71 -6,87 10,24 -7,09

SD 3,28 3,86 3,31 3,87

Max 7,24 -0,2 18,26 2,39

Min -8,68 -15,7 4,13 -15,56

Fig. 1. Illustration of simulated reach ranges of DHM RAMSIS (yellow) and JACK (green)

Posture models have been developed and validated in particular for vehicle design. Standardized postures exist only for seating (in cars) and standing. A transfer to proposed applications including walking and changing of posture is limited. Yet, the validity of the results is unclear. There is a general need for detailed verification and validation studies and subsequent improvements. In case of postures similar to standard postures, the DHMs are capable of simulating realistic postures. With complexity of postures the accuracy of the results is vastly reduced and requires many manual actions. For the selected postures (and light clothing) differences of up to 16.7 cm for the elbow and up to 9.5 cm for the hip were found. For a large person inaccuracies of 22 cm were found. They increased with additional clothing. Modeling of complex motions with the two DHMs as an example was complex and intensive. None of them allowed a realistic simulation of specific movements. Although modeling was possible, it was very complex to model the movement and it required manual actions (e.g. posture modeling for keyframe animation, precalculation of joint angles). Because of missing functionality, the validation study was limited to single movements only. Fig. 2 shows an example of a successful modeling. The identical approach with the second DHM required additional workarounds. By connecting and assigning markers to a limited number of special virtual landmarks it is possible to replay motions, but a true simulation of expected movements was not possible this way.

On the Applicability of Digital Human Models for Personal Equipment Design

319

Fig. 2. Real and virtual keyframe-animation from standing to kneeling

4 Conclusions and Future Work The results show that an application of today’s DHMs for the design of personal equipment is limited although general functionality is available and workarounds are theoretically possible. In particular, additional functionality is required for including models of personal equipment and to model body contours more accurately. Simulation of complex movements is still very limited and allows only a modeling and replay of pre-captured movement sequences. Advanced biomechanical or biodynamic models might help improving. The results also show clearly that verification and validation studies are needed in order to analyze quality and applicability of DHMs in fields different from the original application.

References 1. Duffy, V.: Handbook of Digital Human Modeling for Applied Ergonomics and Human Factors Engineering. Taylor & Francis, Boca Ranton (2009) 2. Bubb, H.: Computer Aided Tools of Ergonomics and System Design. Human Factors and Ergonomics in Manufacturing 12(3), S249–S265 (2002) 3. Chaffin, D.: Digital Human Modeling for Vehicle and Workplace Design. Society of Automotive engineers, Warrendale (2001) 4. Alexander, T., Conradi, J.: Analysis of Anthropometry and Range Validity of the Digital Human Model RAMSIS. Paper #2001-01-2104. Transactions - Journal of Passenger Cars Mechanical Systems (2001) 5. JACK is a product of Siemens PLM Software, Munich, Germany 6. RAMSIS is a product of Human Solutions, Kaiserslautern, Germany 7. Alexander, T.: Functional Analysis and Modeling of Goal-Directed Arm Movements. Zeitschrift für Arbeitswissenschaft (ZArbWiss) (February 2010)

Discussing Validation of 3D Character Animation Demonstrating Ushiro-Ukemi Pedagogical Progression Mauro Cesar Gurgel de Alencar Carvalho1,2,3, Bruno Martins Carvalho1, Felipe Leal de Paiva Carvalho3, Heidi Dias Oliveira Junior2, Gerson Gomes Cunha2, Luiz Landau2, and Estélio Henrique Martin Dantas3 2

1 LaCiMovi – Movement Science Laboratory, Pedro II School, Rio de Janeiro, Brazil LAMCE–Laboratory of Computational Methods in Engineering–COPPE–UFRJ, Brazil 3 LABIMH – Laboratory of Human Bio-kinetics – UNIRIO – , Rio de Janeiro, Brazil [email protected], [email protected], [email protected], [email protected], {landau,gerson}@lamce.coppe.ufrj.br, [email protected]

Abstract. This study was designed to verify the evidence of validity, through: 1) literature review about ushiro-ukemi pedagogical progression. 2) An expert performed the movement and it was recorded to develop the animation. 3) Finally, an evaluation of 3D character animations processes by other experts’ opinions. 13 experts have evaluated three affirmatives through the Likert scale, and answered one question, also in ordinal scale, about the quality of the animation. The calculated medians for the first, second and third affirmatives were 5, 5 and 5 (fully agree), but, only for the 3rd affirmative about sitting ushiro-ukemi, the median was 4 (agree). The percentage of answers that scored higher than agree varied from 84,6 to 100%. The median of the 3D character animation was very good (4) and the percentage of acceptance was 100% (> good). The found definitional evidence of validity for these animations ensured their application in a learning material. Keywords: 3D character animation, Judo, sport skill, validation.

1 Introduction There are many available definitions for validation [1; 2; 3] or validity [4; 5; 6; 7; 8], varying according to the used approach or application. The references from the U. S. Department of Defense and related softwares for military use have shown detailed aspects about VV&A. The terms validation, verification and accreditation (VV&A) are commonly used in Computer Science, Software Engineering to attest the developed product fidelity, functionality and credibility. The Department of Defense [9] and the Department of Navy [10], both from USA, have presented a glossary and a manual with many close definitions about the Modeling and Simulation (M&S) Processes, validation, verification and accreditation. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 320–324, 2011. © Springer-Verlag Berlin Heidelberg 2011

Discussing Validation

321

Verification. The process of determining that an M&S implementation and its associated data accurately represent the developer's conceptual description and specifications. An informal question often applied to verification is "Was the M&S built right?" Validation. The process of determining the degree to which an M&S and its associated data are an accurate representation of the real world from the perspective of the intended use of the M&S. The informal question often asked for validation is "Was the right M&S built?" Accreditation. The official determination that an M&S application and its associated data are acceptable for use for a specific purpose. Accreditation seeks to answer the question "Is this the right M&S to use?" (DoN, 2004). Cook and Skinner [11] have also presented definitions and lited 75 different strategies to develop validation process for M&S. They divided these strategies in 4 groups: Informal, static, dynamic and formal, however these approaches are specifically directed for military M&S softwares. According to Howe and Mahar [12], definitional stage is the first stage during the validation process, though it is necessary to define the developed 3D animations as ushiro-ukemi representations.

2 Goals This study aimed to identify the 3D animations as representations of ushiro-ukemi pedagogical progression, to verify if 3D executed movements may be seen in many angles, to present and validate the overlapping technique for animating the biped performing ushiro-ukemi, and to evaluate the quality of the animation.

3 Sample The subjects were 13 experts in Physical Education and Judo (black-belt).

4 Materials and Methods 4.1 Video Videos were shot from front, back and sides views where a Judo teacher performed the laying, sitting, squatting and standing ushiro-ukemi. 4.2 Website Creation A website was idealized and built with a link, where the experts would be able to provide this validation: www.lacimovi.net/validacaoanime/index.htm. In this the face validation process was divided in three parts: introduction, animation process and questionnaire. The introduction part informed the experts about the validation proposal, the questionnaire goals and asks for their personal data. This part had four webpages,

322

M.C.G. de Alencar Carvalho et al.

one page for each movement of the ushiro-ukemi pedagogical progression: laying, sitting, squatting and standing (www.lacimovi.net/validacaoanime/01_deitado.asp is the first). Each page presented an animated image (.gif) showing the animation process developed at 3DS Max, where the video was set as viewport background and the biped movement was adjusted over the Judo teacher’s movement.

a

b

c

d

Fig. 2. (a) laying, (b) sitting, (c) squatting and (d) standing ushiro-ukemi animation at 3DS Max

This part presented three statements and one question to experts about the laying ushiro-ukemi. The affirmatives were evaluated through the Likert scale and the question according to the ordinal scale applied by Read and MacFarlane [13]. After answering this page, the experts were directed to the next page where the sitting ushiro-ukemi animation process and questionnaire was presented. The same procedure was followed by the squatting and standing ushiro-ukemi.

5 Results The first affirmative stated: “This 3D animation allows the (*) ushiro-ukemi visualization through various angles”. The experts’ medians were 5 (fully agree) and their percentage of agreement were: 91,3 (laying); 92,1 (sitting); 90,9 (squatting), and 92,1 (standing). The second affirmative stated: “This animation technique where the virtual human (biped) is overlapping the video allows presenting the (*) ushiro-ukemi flowing along time according to teacher’s movement”. The experts’ medians were 5 (fully agree) and their percentage of agreement were: 100 (laying); 92,3 (sitting); 90,9 (squatting), and 100 (standing). The third affirmative stated: “This 3D animation technique allows positioning the biped, adjusting its movement proportion every keyframe, according to the (*) ushiro-ukemi performed by the teacher”. The experts’ medians were 5 (fully agree) and their percentage of agreement were: 84,6 (laying); 84,6 (sitting); 90,9 (squatting), and 92,3 (standing). The fourth item was a question about the quality of the animation: “how do you evaluate this animation of the (*) ushiro-ukemi?” The animation scored “very good” for the sitting stages, but “excellent” for the others. The squatting stage scored 91,7 for the percentage of agreement, but 100 for the others. The % of agreement represents the amount of “fully agree” and “agree” answers. The median was applied to represent the answers central tendency measure, because they are based on an ordinal scale. was followed by the squatting and standing ushiro-ukemi.

Discussing Validation

323

6 Discussion and Conclusion The closest approach to this study goals was found at http://vva.msco.mil/Special_Topics/ HBR-Validation/default.htm, where DoD [14] discuss specifically the considerations for Human Behavior Representations (HBR). DoD affirms that the most commonly used strategy is face validation, obtained through experts opinions. However, while DoD (op. cit.) affirms that this validation technique is less reliable and complete, Cook and Skinner [11] deny that assuming that face validity, classified as informal, may lead to a misinterpretation. “While informal techniques have the advantage that they are relatively easy to perform and understand, their application is anything but unstructured. (…) Informal V&V techniques can be very effective if applied with structure and guidelines, and they are relatively low cost. Informal V&V techniques are effective for examining both the model and the simulation” [11]. This supports the strategy applied in this study for validating 3D character animation representing the pedagogical progression of ushiro-ukemi. Besides, during the animation process some steps were carefully taken to control some variables, to ensure a better representation of the ushiro-ukemi and to strengthen the internal and external validity [8]: a)

b)

c) d) e) f) g) h) i)

the original movement (in video) was performed by a Judo teacher (experienced performer), according to Kano’s [15] and Inokuma & Sato [16] descriptions and served as gold standard; animation was developed in one of the best programs for modeling and animation, 3D Studio Max, which already had a skeleton similar to humans, as a default, to develop the animation; a website was created to facilitate experts’ access and to permit the face validation of the animation to occur; the evaluators were chosen by their expertise in Physical Education and also in Judo (black belt holders); the site contains an introductory explanation to the experts about the task they would perform and also the proposed goals; three statements and a question was asked for each stage of the pedagogical progression; the answers were automatically sent to three different e-mail accounts to prevent loss of data; results were statistically treated for a proper description and interpretation; The medians and the percentage of agreement have shown enough evidence of validity for the 3D character animation, for the applied animation technique, for controlling the 3D animation positioning and speed, the type of the applied light and change the background color, and also for the quality of the animation.

The present study has shown evidence of face validity for the developed animation representing the ushiro-ukemi pedagogical progression. The applied technique was helpful for short duration movements, but it may not be advantageous (time x effort)

324

M.C.G. de Alencar Carvalho et al.

for long duration movements. These models may be applied in Augmented Reality systems for visualization and teaching Physical Education and Sports [17; 18].

References 1. Sargent, R.G.: Verification, validation, and accreditation of simulation models. In: Proceedings of the 2000 Winter Simulation Conference, pp. 50–59 (2000) 2. Balci, O., Ormsby, W.F., Carr, J.T., Saadi, S.D.: Planning for verification, validation, and accreditation of modeling and simulation applications. In: Proceedings of the 2000 Winter Simulation Conference, pp. 829–839 (2000) 3. DoD – Department of Defense of United States of America: Key concepts of VV&A (September 15, 2006a), http://www.vva.dmso.mil 4. Docherty, D.: Measurement in Pedriatric Exercise Science. Human Kinetics, 291–292 (1996) 5. Burton, A.W., Miller, D.E.: Movement skill assessment. Human Kinetics, 110–117 (1998) 6. Thrischler, K.: Medida e avaliação de Educação Física e Esportes. Manole, 66–72 (2003) 7. Rowe, D.A., Mahar M. T.: Validity. In: Measurement theory and practice in Kinesiology. Terry Wood and Weimo Zhu, Human Kinetics, 9–26 (2006) 8. Thomas, J.R., Nelson, J.K., Silverman: Métodos de pesquisa em atividade física. ARTMED, 280–296 (2007) 9. DoD – Department of Defense of United States of America: Glossary (December 5, 2006b), http://www.vva.dmso.mil/Glossary/default.htm 10. DoN – Department of Navy of United States of America: Modeling and Simulation Verification, Validation, and Accreditation Implementation Handbook, vol. I VV&A Framework (2004) 11. Cook, D.A., Skinner, J.M.: How to Perform Credible Verification, Validation, and Accreditation for Modeling and Simulation. CrossTalk. The Journal of Defense Software Engeneering (May 2005) 12. Mahar, M.T., Rowe, D.: A Practical Guidelines for Valid and Reliable Youth Fitness Testing. Measurement in Physical Education and Exercise Science 12(3), 126–145 (2008) 13. Read, J.C., Macfarlane, S.: Using the Fun Toolkit and Other Survey Methods to Gather Opinions in Child Computer Interaction. In: IDC 2006 (June 7-9, 2006) 14. DoD – Department of Defense of United States of America: Glossary (2001), http://vva.msco.mil/Special_Topics/HBRValidation/default.htm 15. Kano, J.: Kodokan Judo. Kodansha International (1994) 16. Inokuma, I., Sato, N.: Best Judo. Kodansha International (1986) 17. Carvalho, M.C.G.A., Dantas, E.H.M., Landau, L., Cunha, G.G.: Teaching Judo breakfall (ushiro-ukemi): an application of human modeling, movement simulation and augmented reality techniques. In: The International Workshop on Applied Modeling & Simulation 2006, Búzios, Brazil, p. 173 (2006) 18. Carvalho, M.C.G.A., Dantas, E.H.M., Landau, L., Cunha, G.G.: Augmented Reality applied to Ushiro-ukemi visualization. In: Annals of the 5th International Judo Federation World Research Symposium, Rio de Janeiro, Brazil, September 12, p. 35 (2007)

The Provision of Digital Information in the Seat Comfort of the Seat Design Kuen-Meau Chen1, *, Siu-Tsen Shen2, and Stephen D. Prior3 1

Department of Industrial Design, National United University, 1 Lien Da, Kung-Ching Li, Maioli 36003, Taiwan [email protected] 2 Department of Multimedia Design, National Formosa University, 64 Wen-Hua Rd, Hu-Wei 63208, Taiwan 3 Department of Product Design and Engineering, Middlesex University, London N14 4YZ, United Kingdom

Abstract. A great number of factors affect the uncomfortable of seats. Apart from external appearance, the most important is the comfort for users in designing an un-adjustable seat. A system or suggested principles which assist designers in designing the comfort of the seats to testees will benefit waist of users, and hence prevent them from potential discomfort and injuries. This study attempts to develop digital design platforms, and allows designers to measure the design scale of the comfort. We expect to find the association between seat parameter interaction and comfort. For example, when the seat height is much lower than the knee of testees, they can stretch calf forward if the space allows, so that their thighs contact the surface of the seat to share sciatic pressure in greater comfort. We believe to have changes of comfort in opposition to different seat parameter combinations. The final results will help effective reference parameters of comfort in seat design. Keywords: Seat design, prediction model, dynamic interaction, Digital Design.

1 Introduction While many factors influence the degree of seat comfort, few simple and practical tools are available for designers to assess seat design. The most important consideration when designing a non-adjustable chair is the comfort experienced by the targeted group. If these recommended guidelines are developed to help designers examine comfort in seat design, many users will benefit from better lumbar support by avoiding possible discomfort and injuries. Many researchers have studied seat design and size recommendations, but there is little information about the interaction of various parameters related to seat size in the literature. For example, a seat with an angle close to 90 degrees between its back and surface usually causes strong discomfort to users; however, the degree of comfort can be improved if lumbar support is added. In addition, a seat that tilts back more than 10 *

Corresponding author.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 325–329, 2011. © Springer-Verlag Berlin Heidelberg 2011

326

K.-M. Chen, S.-T. Shen, and S.D. Prior

degrees is worse for sitting in front of a low writing desk because it is difficult for users to lift their upper body when writing. This research should find other similar dynamic interactions to provide designers with simple guidelines to avoid reduced comfort when focusing on appearance. In addition, this study hope to understand the extremes of uncomfortable chairs, seat parameters’ proportional effects on the degree of comfort, make an integrated prediction about comfort assessment, and further develop human factor design standards for the interaction of seat parameters.

2 Literature Review People now spend long hours in a seat for work and leisure that results in musculoskeletal pain and discomfort [1]. However, the fit of seat size to body size does not necessarily determine comfort. Other factors include individual habits, seat pressure, types of tasks conducted when sitting, the muscles of different body parts, BMI, body fat percentage, allocation of body fat, thermal comfort resulting from the texture of the seat surface, and even microclimatic effects between the seat surface and the individual [2,3,4]. Many scholars have proposed various research perspectives and contributed valuable data for designers [5]. Eklund and Corlett (1987) compared two different types of seats and five different tasks. They found that seats or tasks that require individuals to bend the body-trunk often lead to more discomfort [6]. Zhao and Tang (1994) pointed out that better matching seat backs to users’ backs and lumbar outlines significantly improves comfort [7]. Relevant studies suggest that comfort and discomfort are two independent indicators for evaluation [8]. Many studies only include subjective measurements of discomfort [9]. Some studies treat comfort and discomfort as opposite directions in one dimension [10]. De Looze et al. (2003) noticed that most researchers do not separate the comfort and discomfort but combine them when including comfort and discomfort as evaluation indicators [11]. Therefore, this research refers to the results of Kyunga and Nussbauma in 2008 and of Gyouhyung et al. in 2008. This study believes that measuring the concept of comfort, one can identify differences in comfort more effectively than differences in discomfort [12, 13]. This research focused on the interaction of seat parameters and proposed a prediction model for degrees of comfort. Why is the interaction between parameters so important? Relevant studies point out that the contents of tasks rather than the types of seats influence muscular tension [14]. This indicates that in addition to seat size, guidelines must consider the type of task. When carrying out different VDT tasks, the different heights of screens may cause physical discomfort [15]. Accordingly, seat comfort not only emphasizes physical size, but also involves the interaction with task environment and content. Consequently, to achieve a more comprehensive prediction, researchers should adopt a biomechanical model for calculating the lumbar pressure caused by certain postures and assess the surface pressure of seats [16].

3 Research Method This experiment used adjustable chairs and asked participants to evaluate the degree of comfort of 27 differently sized seats based on three parameters: seat height,

The Provision of Digital Information in the Seat Comfort of the Seat Design

327

inclination angle of seat back, and height of lumbar support. By doing this, this study intended to discover the most comfortable seat and the interaction between parameters. 3.1 Preparation before the Experiment Before the experiment, participants were asked to stand for 10 minutes to relieve the impact of sitting for a long period. This study measured participant knee height and the distance from the knees to the back of the buttocks. Participants were asked to take off shoes and remove belts and large items from pockets. Participants were asked to try out three different combinations of the parameters to understand the range of seat sizes and to prevent biased feelings during the experiment. Participants were told to evaluate different combinations of seat parameters based on preferred posture. Finally, terminology and evaluation standards were explained to the participants. The first phase of the experiment involved three parameters discussed the most in the literature.The selected sizes for the experiment were: Factor 1 includes seat height at 33 cm, 39 cm, and 45 cm. Factor 2 includes lumbar support as 15 cm, 27cm, and no lumbar support. Factor 3 includes inclination angle of seat back as 90 degrees, 105 degrees, and 120 degrees. 3.2 Digital Information of Seat Design The development of this system in accordance with human-scale planning, including gender, height, table height adjustment, seat and so the measurement of the size (see Fig. 1).

Fig. 1. Digital information system of seat design

4 Discussion of the Results As presented in Table 1, this multivariate factor analysis reveals that the three major factors of seat height (F(2,944)=4.404,P<0.05), height of lumbar support (F(2,944)=12.130,P<0.05), and inclination angle of seat back (F(2,944)=47.297,P<0.05) all have significant impact on comfort, and that there is significant interaction between the inclination angle of seat back and the height of lumbar support (F(4,944)=10.784,P<0.05).

328

K.-M. Chen, S.-T. Shen, and S.D. Prior Table 1. Three-Way ANOVA Analysis

Variables Seat height Height of lumbar support Inclination angle of the seat back Seat height * height of lumbar support Inclination angle of the seat back * seat height Height of lumbar support* inclination angle of seat back Seat height * Height of lumbar support* Inclination angle of the back Error Sum

S 11.7 32.4 126.5 8.6 14.7 57.7 5.4 1228 1485.7

DOF 2 2 2 4 4 4

MS 5.8 16.2 63.2 2.17 3.6 14.4

8

.6

918 944

F 4.4 12.1 47.2 1.6 2.7 10.7

P 0.012* 0.000* 0.000* 0.166 0.027* 0.000*

.5

0.848

1.3

Table 1 reveals the significant interaction between seat height and the inclination angle of the seat back (F(4,944)=2.764,P<0.05). Making the inclination angle larger than 105 degrees significantly improves comfort. Another important finding is a higher (45 cm) seat causes less discomfort than a seat with a lower (33 cm) surface, even if the inclination angle is 90 degrees. A higher seat causes the least discomfort compared to seats with different heights so long as the inclination angle is larger (120 degrees). No significant difference was found between seat height and the height of lumbar support in the multivariate factor analysis (F(4,944)=1.624,P>0.05).

5 Conclusion The three factors adopted in this experiment are the most common and important parameters when designing seats. Seats with an inclination angle of 90 degrees but have no lumbar support receive low comfort scores. The discomfort may result from having to excessively bend the body trunk causing pressure on the abdominal cavity and the lumbar vertebra cannot retain a concave posture. By contrast, when the inclination angle is 120 degrees, seats without lumbar support are more comfortable. A possible explanation is that when the upper body remains straight, the lumbar outline forms a deeper concave; however, when the angle of the back is 120 degrees, the outline of the back is changed, and the concave of the lumbar vertebra is no longer obvious. Lumbar support of the same thickness may cause pressure to some tissues and lead to discomfort. From the perspective of lumbar support and seat back inclination angle, this study makes two observations. First, when the inclination angle is close to 90 degrees, increasing lumbar support can improve comfort. Second, when the inclination angle is larger than 120 degrees, removing lumbar support or reducing the thickness of the lumbar cushion relieves pressure on the lumbar area. Acknowledgement. This study is supported by National Science Counsel, Taiwan, with project number NSC-98-2511-S-239-001-MY2.

References 1. Ebe, K., Griffin, M.J.: Factors affecting static seat cushion comfort. Ergonomics 44(10), 901–921 (2001) 2. Gyi, D.E., Porter, J.M.: Interface pressure and the prediction of car seat discomfort. Applied Ergonomics 30, 99–107 (1999)

The Provision of Digital Information in the Seat Comfort of the Seat Design

329

3. Nick, C., Brynley, P., Hull, G.E.: An empirical study of preferred settings for lumbar support on adjustable office chairs. Ergonomics 41(4), 401–409 (1998) 4. Diebschlag, W., Heidinger, F., Kuurz, B., Heiberger, R.: Recommendation for ergonomic and physiological climatic vehicle seat design. Society of Automotive Engineers, T Paper no. 880055 (1988) 5. Zhang, L., Helander, M., Drury, C.: Identifying factors of comfort and discomfort. Human Factors 38(3), 377–38 (1996) 6. Eklund, J.A.E., Corlett, E.N.: Evaluation of spinal loads and chair design in seated work tasks. Clinical Biomechanics 2, 27–33 (1987) 7. Zhao, J.H., Tang, L.: An evaluation of comfort of a bus seat. Applied Ergonomics 25, 386– 392 (1994) 8. Sauter, S.L., Swanson, N.G., Waters, T.R., Hales, T.R., Dunkin-Chadwick, R.: Musculoskeletal discomfort surveys used at NIOSH. In: Stanton, N., Hedge, A., Brookhuis, K., Salas, E., Hendrick, H. (eds.) Handbook of Human Factors and Ergonomics Methods. CRC Press, Boca Raton (2005) 9. Hsu, W.-H., Wang, M.-J.: Physical discomfort among visual display terminal users in a semiconductor manufacturing company: a study of prevalence and relation to psychosocial and physical/ergonomic factors. American Industrial Hygiene Association Journal 64(2), 276–282 (2003) 10. Kee, D., Karwowski, W.: Joint angles of isocomfort for female subjects based on the psychophysical scaling of static standing postures. Ergonomics 47(4), 427–445 (2004) 11. De Looze, M.P., Kuijt-Evers, L.F.M., van Dieen, J.: Sitting comfort and discomfort and the relationships with objective measures. Ergonomics 46(10), 985–997 (2003) 12. Gyouhyung, K., Maury, A., Nussbaum, Kari, B.R.: Driver sitting comfort and discomfort (part I): Use of subjective ratings in discriminating car seats and correspondence among ratings. International Journal of Industrial Ergonomics 38, 516–525 (2008) 13. Kyunga, G., Nussbauma, M.A., Babski-Reevesb, K.: Driver sitting comfort and discomfort (part I): Use of subjective ratings in discriminating car seats and correspondence among ratings. International Journal of Industrial Ergonomics 38, 516–525 (2008) 14. Van Dieen, J.H., De Looze, M.P., Hermans, V.: Effects of dynamic chairs on trunk kinematics, trunk extensor EMG and spinal shrinkage. Ergonomics (2001) ISSN: 00140139 15. Seghers, J., Jochen, A., Spaepen, S.: Posture, muscle activity and muscle fatigue in prolonged VDT work at different screen height settings. Ergonomics 46(7), 714–730 (2003) 16. Kayis, B., Hoang, K.: Static three-dimensional modeling of prolonged seated posture. Applied Ergonomics 30(3) (1999)

The Effect of Damping in an Input Device on Human Positioning Performance Koen Crommentuijn and Dik J. Hermes Human-Technology Interaction Group, Eindhoven University of Technology, The Netherlands {k.j.crommentuijn,d.j.hermes}@tue.nl

Abstract. This study investigates the effect of damping in an input device on performance in a target acquisition task. In an experiment with 21 participants, five levels of damping were tested in the range of 0 to 10 Ns/m. Contrary to our expectation, the time required to acquire a target decreased as the damping value increased. The reduction in completion time was not caused by an increase in velocity, but by a decrease in the overshoot rate. It is therefore concluded that, unlike previous beliefs, some amount of damping in an input device can be beneficial to user performance. Keywords: damping, input device, human performance, positioning task.

1 Introduction This study investigates the effect of damping, a force resisting motion of an object relative to its velocity, on user performance. With popular research topics like teleoperation and haptic interaction, there is currently a high interest in specialized input devices. To allow optimum performance of a person using such a device, it is necessary to know which device characteristics affect user performance. Previous studies focused on, among others, device friction and device mass [1,2,3]. Jones and Hunter [4] investigated the effect of damping on user performance, much like the present study. They found that performance decreased as damping of the input device increased. The levels of damping tested were, however, relatively high, starting from 12 Ns/m. Furthermore, in their study, the device was operated from the shoulder joint, whereas many input devices are operated from the elbow joint and/or the wrist joint. As damping could have a different effect on user performance for values below 12 Ns/m or when a device is operated from a different joint, the present study investigates the effect of damping in the range of 0 to 10 Ns/m with an input device operated from the wrist and finger joints. The mass of the input device is also varied, because an interaction between damping and mass is expected.

2 Method Apparatus. The setup used in this study was identical to that described in Crommentuijn and Hermes [3]. Main features of this setup include a 1 degree-offreedom haptic device in which friction is virtually absent due to the use of air C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 330–334, 2011. © Springer-Verlag Berlin Heidelberg 2011

The Effect of Damping in an Input Device on Human Positioning Performance

331

bearings. Physics were simulated at a rate of 1000 Hz on a DS1103 PPC Controller Board (dSPACE). Damping was calculated by multiplying a damping constant with the velocity of the haptic interface. The experiment ran on a regular PC. The cursor and target were visualized on a 22″ LCD screen placed at approximately 70 cm from the viewer. The screen had a resolution of 1920 × 1080 pixels, and one pixel measured .248 × .248 mm. The movement of the cursor was scaled by a factor 5.22. Design. There were four independent variables: damping, additional simulated mass, target width, and target distance. Five levels of damping were tested: 0, 1.25, 2.5, 5 and 10 Ns/m. The joystick mass was varied between 0.53, 0.86, and 1.40 kg, by disabling or enabling an additional simulated mass of 0.33 or 0.87 kg. Targets were presented with three different widths, i.e., 21, 33, and 51 pixels, and at two different distances, i.e., 302 and 408 pixels. Hence, this study used a 5×3×3×2 within-subject design. The experiment consisted of multiple blocks of 25 positioning tasks. The first three blocks of the experiment were used to train a participant. One block of 25 positioning tasks contained one start target and a randomized set of targets with three different widths placed at two different distances, all combinations repeated four times. One setting of damping and joystick mass spanned three blocks. Thus, for each experimental condition (damping × mass × width × distance) twelve trials were run (3 blocks × 4 repetitions per block). The first block of each new joystick setting was regarded as a training block and not used in the analysis. The presentation order of the joystick settings was randomized across participants. The presentation order of the target widths and target distances were randomized per block. For each trial we recorded the completion time, the number of times the cursor went off a target (overshoots), the maximum velocity of the joystick, and whether an error was made (erroneous trial). Participants. Twenty-one participants, 17 male and 4 female, with no apparent physical impairment, took part in the experiment. They were unaware of the goals of this study. The average age was 20.5 years, with a range of 19 to 23. Two participants were left-handed. Procedure. Participants used their right hand to control the joystick, and their left hand to control the keyboard. An arm support was used to limit the arm movement to the wrist and fingers. A 1-dimensional target-acquisition task was performed. A red and yellow cursor had to be moved over a target that was represented by a blue rectangle. When the cursor was over the target the participant had to press the spacebar on the keyboard. The targets appeared left, right, left, and so on. An error was recorded when the spacebar was pressed when the cursor was not over the target. Participants were instructed to make, at most, one error per block, i.e., 1 in 25.

3 Results Data Preparation. On average, the erroneous trial rate of a participant was 5.3% (SD=3.7%). One participant, with an unsuccessful trial rate of only 0.5 %, was

332

K. Crommentuijn and D.J. Hermes

excluded from the analysis, as he/she focused too much on avoiding errors. To correct for deviations from a normal distribution, completion times were log-transformed before calculating an average score per participant per experimental condition. For the same reason, the square root of the average overshoot rate was used in the analysis. Trials with errors were excluded from the analysis. Trials with an extreme completion time were also excluded from the analysis. A trial with a log-transformed completion time resulting in an absolute z-score larger than 3.0, calculated for each experimental condition separately, was marked as an outlier. In total, 1054 of 17280 trials (6.1 %) were excluded from the analysis. Analysis of Completion Time. A repeated measures analysis of variance was performed with the average log transformed completion time as dependent variable and the damping value, joystick mass, target width, and target distance as independent variables. Contrary to our expectation, it was found that the completion time significantly decreased as the damping value increased (F(4,76)=16.1, p<.001). More in line with our expectation, it was found that the completion time significantly increased as the joystick mass increased (F(2,38)=73.1, p<.001). Furthermore, in line with Fitts’ Law, the completion time significantly increased as the target width decreased (F(1.5,28.9)=927.1, p<.001) and as the target distance increased (F(1,19)=327.2, p<.001). It was found (see figures 1 and 2) that the positive effect of damping on completion time significantly increased as the joystick mass increased (F(8,152)=2.7, p=.009) and as the target width decreased (F(3.9,73.5)=3.7, p=.009). Analysis of Maximum Velocity and Overshoot Rate. The positive effect of damping on completion time could be the result of an increased velocity. Hence, a repeated measures ANOVA was performed with the maximum velocity as dependent variable and the damping value, joystick mass, target width, and target distance as independent variables. It was found that the maximum velocity significantly decreased as the damping value increased (F(4,76)=94.0, p<.001), see figure 3. The reduction in completion time is therefore not caused by an increase in velocity. The ANOVA also showed that the maximum velocity significantly decreased as the joystick mass increased (F(1.3,25.2)=155.5, p<.001). The maximum velocity during a trial was significantly larger for targets at the largest distance than for targets at the shortest distance (F(1,19)=229.9, p<.001). Target width did not significantly affect the maximum velocity (F(2,38)=0.9, p=.40). Since the reduction in completion time is not explained by an increase in velocity, a third repeated measures ANOVA was performed with the square root of the average overshoot rate as dependent variable. It was found that the overshoot rate significantly decreased as the damping value increased (F(4,76)=59.2, p<.001), see figure 4. It can therefore be concluded that the reduction in completion time is caused by a more efficient acquisition of a target. The ANOVA also showed that the overshoot rate significantly increased as the joystick mass increased (F(1.5,29.0)=42.3, p<.001). The overshoot rate also significantly increased as the target width (F(2,38)=176.6, p<.001) and target distance (F(1,19)=80.4, p<.001) decreased.

The Effect of Damping in an Input Device on Human Positioning Performance

333

Fig. 1. The effect of damping and mass on Fig. 2. The effect of damping and width on completion time completion time

Fig. 3. The effect of damping and mass on the Fig. 4. The effect of damping and mass on the maximum velocity overshoot rate

4 Discussion A previous investigation showed that damping in an input device can decrease performance in a positioning task [4]. It was therefore expected that, with a moderate amount of Coulomb friction present, the addition of damping would increase the time necessary to acquire targets. Contrary to our expectation, an increase in damping resulted in shorter completion times. Furthermore, it was found that the positive effect of damping increased as the joystick mass increased and as the target width decreased. Subsequently, through analysis of the maximum velocity and overshoot rate, it was found that the positive effect of damping was not the result of an increase in velocity, but the result of a more efficient path towards the target. The results of this study are not in line with the conclusion drawn by Jones and Hunter [1] that a less viscous interface will result in better user performance. While in both studies a target acquisition task was performed, there are several differences in setup which could be the cause of the unexpected outcome. First, the range of damping values used in the present study was lower than the range used in the study

334

K. Crommentuijn and D.J. Hermes

of Jones and Hunter. Second, the input device in the present study is operated from the wrist and finger joints, and not from the shoulder joint. Furthermore, the inertia experienced by the user could have differed between the two studies, but the inertia experienced by the user was not reported in Jones and Hunter. Given these explanations, it is possible that the effect of damping on user performance follows a bell-curve. If this is the case, increasing the damping value will, initially, result in an increase in user performance. Increasing the damping value beyond its optimum will then decrease user performance. Alternatively, the positive effect of damping might only exist for input devices with a relatively large mass, e.g., larger than half a kilogram. It is therefore interesting to test whether the positive effect of damping still holds for interfaces with a low mass. It is also advised to test whether the effect of damping depends on the target distance, because only distances up to 20 mm (present study) and 40 mm [1] have been tested. It is generally assumed that human performance benefits from a low impedance of an input device. This study confirms that, for the values tested, an increase in interface mass indeed reduces human performance in a positioning task. However, this study also shows that a certain amount of damping can benefit human performance. When the task completion time is taken as performance criterion, damping, up till a value of 10 Ns/m, improved performance in a 1-dimensional target acquisition task. It is therefore advised to include some amount of damping in the design of an input device. Naturally, after a certain value of damping, performance will decrease again, and too high an amount of damping will cause fatigue or discomfort. So, more dedicated research is necessary to decide what amount of damping is optimum, also in relation to the task to be carried out. Acknowledgements. This work was supported by a grant from the Dutch Ministry of Economic Affairs (IOP-MMI/SenterNovem). The authors thank Dennis van Raaij and Ron Hendrix from the department of Mechanical Engineering for their extensive technical support.

References 1. Richard, C., Cutkosky, M.R.: The effects of real and computer generated friction on human performance in a targeting task. In: ASME IMECE Hapt. Symp., pp. 1101–1108 (2000) 2. Crommentuijn, K., Hermes, D.J.: The effect of coulomb friction in a haptic interface on positioning performance. In: Kappers, A.M.L., van Erp, J.B.F., Bergmann Tiest, W.M., van der Helm, F.C.T. (eds.) EuroHaptics 2010. LNCS, vol. 6192, pp. 398–405. Springer, Heidelberg (2010) 3. Crommentuijn, K., Hermes, D.J.: When stick-slip hinders human positioning performance. In: 11th IFAC/IFIP/IFORS/IEA Symp. on Analysis, Design, and Eval. of Human-Machine Systems, Valenciennes, France (2010) 4. Jones, L., Hunter, I.: Influence of the mechanical properties of a manipulandum on human operator dynamics. II. Viscosity. Biological Cybernetics 69, 295–303 (1993)

Performance and Comfort When Using Motion-Controlled Tools in Complex Tasks Ines Ann Heber1, Michael Oehl2, and Christine Sutter3 1

University Hospital RWTH Aachen, Department of Neurology, Section Neuropsychology, Pauwelsstr. 30, 52074 Aachen, Germany [email protected] 2 Leuphana University of Lüneburg, Institute of Experimental Industrial Psychology, Wilschenbrucher Weg 84a, 21335 Lüneburg, Germany [email protected] 3 RWTH Aachen University, Department of Work and Cognitive Psychology, Jägerstr. 17-19, 52056 Aachen, Germany [email protected]

Abstract. The use of interaction tools in modern work often challenges the human motor system, especially when these tools create awkward postures and discomfort (e.g., mouse arm syndrome). The question whether the trackball is a serious alternative to the mouse was evaluated in this experimental study in terms of motor performance, usability and comfort. In an applied pointing-selection task we varied gain and task difficulty. Results showed a considerably stronger impact of gain and task difficulty on the trackball than on the mouse, especially for the high gain trackball performance slowed down. Second, usability ratings were significantly better for the mouse than for the trackball (independent of the experimental condition). Finally, while the discomfort after mouse usage rose notably, trackball usage led to an even bigger increase in perceived discomfort. Keywords: Human-computer interaction, input devices, Fitts’ Law, repetitive strain injury.

1 Introduction The mouse is still the most widely used input device [1] since it is regarded as fast, precise and easy to handle. However, a rising amount of users experiences discomfort of the upper extremities. These mouse related repetitive strain injuries (RSI) are caused by repetitive movements, extreme wrist positions as well as high muscular load and tension [2, 3]. In recent years, manufactures of input devices have reacted to the rise of mouse-related RSI by developing alternatives considered to be less hazardous concerning muscular strain. Especially the thumb-controlled trackball has been advertized as being a healthy alternative by minimizing strain on the hand, wrist and arm. However, little is know about its efficiency, accuracy and muscular load. Furthermore, no analysis evaluates the trackball as a serious alternative to the mouse by comparing performance measures so far. Accordingly, the present study aims to provide a detailed analysis of motor performance and comfort of trackball and mouse and thus to evaluate the proposed ergonomic advantage of the trackball. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 335–339, 2011. © Springer-Verlag Berlin Heidelberg 2011

336

I.A. Heber, M. Oehl, and C. Sutter

2 Method Participants: N = 30 students (20 male, 10 female, mean age: 22 years) from the RWTH Aachen University. The two experimental groups were controlled for confounding effects in age, gender and ex ante muscular load based on preliminary questioning. Apparatus and stimuli: Participants sat in front of a desktop computer (266 MHz, Pentium III) and a 15” TFT screen (1024 x 768 resolution), equipped with either a thumb-controlled trackball (Logitech TrackMan Wheel©) or a standard mouse (Logitech MouseMan Wheel©). Derived from pre-testing, both device drivers were set as to produce the same cursor gain (pixel per second ratio). For the trackball medium speed was set at 5151 p/s (level 4) and fast speed at 5844 p/s (level 9). For the mouse medium speed was set at 4751 p/s (level 5) and fast speed at 5757 p/s (level 10). The task consisted of a serial point-click task (in conformance with ISO 9241-9, 2000) [4] and included moving a cross-hair cursor to a black square and to select the target by pressing the left button. A task consisted of 8 serial selections. Targets appeared in two sizes (2.5 and 5 mm) and at two distances (25 and 50 mm). The combination of target distance (near, far) and size (small, big) resulted in three possible movement difficulties (= Index of Difficulty, ID) according to Fitts’ law [4]: easy (ID = 2.6 bits), i.e., big and near target; medium (ID = 3.5 bits), i.e., big and far target or small and near target, respectively; hard (ID = 4.4 bits), i.e., small and far target. In order to exclude confounding effects of movement direction, the starting position of the cursor was varied: the cursor could appear at 45°, 90°, 135°, 225°, 270°, 315°, or 360° relative to the target. Procedure: In Experiment 1 n = 15 participants performed the task with the thumbcontrolled trackball. Participants completed one block of trials with medium and one block with high cursor speed. The order of speed conditions was counterbalanced across participants. Each block consisted of 48 training trials and 16 experimental trials. Subjects were instructed to work as fast and accurate as possible. The independent variables were cursor speed (medium vs. high) and ID (2.6, 3.5 small, 3.5 big vs. 4.4 bits). The dependent variables were movement time (MT; onset of cursor movement to target selection) and clicking errors (selections outside the target). After each block, participants rated the devices’ usability on a 6-point scale (1 = very easy to 6 = very hard). Muscular discomfort was rated before and after the experiment on a category partitioning scale [5] from 0 = no discomfort at all to 50 = severe discomfort. In Experiment 2 n = 15 participants worked with the standard mouse. Procedures and variables were the same as described for Experiment 1.

3 Results 3.1 Experiment 1: Trackball For the MT the 2 x 4 ANOVA (factors: “cursor speed”, “ID”) revealed significant main effects of cursor speed and ID (each p < .01). As depicted in Figure 1, MT increased by 14% with the fast cursor speed. An increase of ID also led to a rise in MT by 10% at medium and 13% at fast cursor speed. Though speed and difficulty did

Performance and Comfort When Using Motion-Controlled Tools in Complex Tasks

337

Fig. 1. Mean MT (ms) for trackball and mouse as a function of cursor speed and ID (bits)

not interact (p = .64). The analysis of clicking errors revealed no significant effects. Participants responded quite accurately with a mean error rate below 5%. 3.2 Experiment 2: Mouse For the MT (Figure 1) the 2 x 4 ANOVA (factors: “cursor speed”, “ID”) revealed significant main effects of cursor speed (p < .01) and ID (p < .05). MT increased by 30 ms (3%) at the fast cursor speed, and also with ID by 447 ms (36%) at medium cursor speed, and 437 ms (35%) at fast cursor speed. Though speed and difficulty again did not interact (p = .85). The analysis of clicking errors revealed no significant effect of cursor speed, but of ID. Although participants responded quite accurately with a mean error rate below 5%, clicking errors slightly increased with higher IDs (p < .05). 3.3 Comparison of Mouse and Trackball Performance Motor performance: The data was analyzed by a 2 x 2 x 4 ANOVA with the betweensubject factor “input device” and the within-subject factors “cursor speed” and “ID”. The results showed significant main effects of input device, cursor speed and ID (each p < .01). MT were generally 755 ms shorter for the mouse and this advantage was more pronounced within the fast speed condition (Δ 880 ms, p < .01) compared to the medium speed condition (Δ 631 ms, p < .01) yielding a significant interaction of input device and cursor speed (p < .01). Thus, trackball MT were significantly stronger influenced by cursor speed than mouse MT. Aditionally, we found a more distinct

338

I.A. Heber, M. Oehl, and C. Sutter

increase of MT as a function of ID for the trackball (783 ms = 38%) than for the mouse (442 ms = 35%) statistically confirmed by the significant interaction of input device by ID (p < .01). Interestingly, the medium IDs (big and far target vs. small and near target) differed with regard to MT in both cursor speed conditions for both trackball and mouse. According to Fitts’ Law, both size and distance combinations should result in equal MT. However, the combination of small target and short target distance did result in significantly increased MT compared to the other medium task difficulty for both cursor speeds and both trackball (p < .05) and mouse (p < .01). Comfort: Mean usability rating for the mouse was 1.9 (easy) and for the trackball it was 3.7 (rather difficult). The medium cursor speed was rated to be rather easy for the mouse (2.5) and difficult for the trackball (4.0). The fast cursor speed was rated to be easy for the mouse (2.2) and difficult for the trackball (4.3). While discomfort after mouse usage rose by 12 points, trackball usage lead to a significantly bigger increase in perceived discomfort (23 points). Mouse users complained about an increase in discomfort in the hand whereas trackball users experienced increasing discomfort in the fingers, elbow, shoulder and neck.

4 Conclusions The present study aimed to examine the trackball as an ergonomic alternative to the mouse. In our study, thumb-controlled trackball performance was significantly less effective than mouse performance. Both cursor speed and ID had (highly) significant effects on MT for both input devices, but trackball performance suffered significantly greater from fast cursor speed than mouse performance, and MT in the trackball were overall significantly longer than in the mouse. Effects of input device are recently more and more discussed in the light of sensorimotor transformations and compatibility. Findings give reason that trackball users obviously face a rather difficult transformation, since device movement is rotary and display movement linear [6, 7, 8]. In the mouse, device movement and cursor motion correspond highly, which results in better performance. These transformation issues might have resulted in worse trackball usability ratings in our study. Additionally, muscular discomfort whilst using the trackball was decidedly higher than for the mouse. While the trackball seemed to reduce discomfort of the hand, it increased for the arm, shoulder and the neck, supposedly due to an increase of static load. Nevertheless, results indicate that the trackball could be a useful alternative in terms of input device variability: Switching between devices might reduce some of the negative effects of both. Still the trackball should probably be used for simpler tasks and be operated with a slower cursor speed than the mouse. A further finding is worth to be discussed: In all conditions we found a significant difference in MT in both medium IDs. This result contradicts Fitts’ Law by indicating that target size seems to have a stronger influence on movement difficulty than target distance, while according to Fitts’ Law, they should be equally difficult. Our results are in correspondence to earlier studies, which also reported a greater impact of target size on performance [9, 10].

Performance and Comfort When Using Motion-Controlled Tools in Complex Tasks

339

In conclusion, ergonomic guidelines can be derived for the optimized application of motion-controlled interaction tools in laparoscopic surgery [11], virtual reality (for training and product development) or teleoperation. Acknowledgments. We wish to thank Heidrun Glienke and Martina Ziefle for research support.

References 1. Hastings, S., Woods, V., Haslam, R.A., Buckle, P.: Health risks from mice and other nonkeyboard input devices. In: McCabe, P.T., Hanson, M.A., Robertson, S.A. (eds.) Contemporary Ergonomics, pp. 312–316. Taylor & Francis, London (2000) 2. Burgess-Limerick, R., Shemmell, J., Scadden, R., Plooy, A.: Wrist posture during computer pointing device use. Clinical Biomechanics 14, 280–286 (1999) 3. Keir, P.J., Bach, J.M., Rempel, D.: Effects of computer design and task on carpal tunnel pressure. Ergonomics 42, 1350–1360 (1999) 4. ISO 9241-9: Ergonomic requirements for office work with visual display terminals – Part 9: Requirements for non-keyboard input devices. International Organization for Standardization. Beuth, Berlin (2000) 5. Heller, O.: Hörfeldaudiometrie mit dem Verfahren der Kategorienunterteilung (KU), Suprathreshold audiometry using the method of category partitioning (CP). Psychologische Beiträge 27, 478–493 (1985) 6. Proctor, R.W., Vu, K.-P.L.: Stimulus-Response-Compatibility Principles: Data, Theory, and Application. CRC Press - Taylor & Francis Group, New York (2006) 7. Sutter, C.: Sensumotor transformation of input devices and the impact on practice and task difficulty. Ergonomics 50, 1999–2016 (2007) 8. Sutter, C., Oehl, M., Armbrüster, C.: Practice and carryover effects when using small interaction devices. Applied Ergonomics 42, 437–444 (2011) 9. Sheridan, M.R.: A reappraisal of Fitts law. Journal of Motor Behavior 11, 179–188 (1979) 10. Sutter, C., Ziefle, M.: Psychomotor efficiency in users of notebook input devices: Confirmation and restrictions of Fitts’ law as an evaluative tool for user-friendly design. In: Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting, pp. 773–777 (2004) 11. Herring, S.R., Trejo, A.E., Hallbeck, M.S.: Evaluation of four cursor control devices during a target acquisition task for laparoscopic tool control. Applied Ergonomics 41, 47– 57 (2010)

Pen Tip Position Estimation Using Least Square Sphere Fitting for Customized Attachments of Haptic Device Masanao Koeda and Masahiko Kato Department of Computer Science, Faculty of Information Science and Arts, Osaka Electro-Communication University, Kiyotaki 1130-70, Shijonawate, Osaka, 575-0063, Japan {koeda,[email protected]}

Abstract. In this paper, a method for estimating the pen tip position of customized haptic device attachments is presented. Least squares sphere fitting is applied to the acquired point cloud data to estimate the radius and center of a sphere. Verification experiments were conducted and the experimental result shows that the proposed method has reasonable accuracy.

1 Introduction Conventionally, dental models [1] are used to impart hand skill education to dental students; however, this method it is not always effective. Dental patient robots that simulate real human motion have been developed [2, 3]. However, these robots are rarely introduced in education because of their high cost and difficult maintenance. We have been developing HAP-DENT [4], a computerized dental training simulator (Fig. 1). The developed simulator uses a combination of virtual reality and haptics. A suitable combination of virtual and real space would enable effective hand skill education for dental students. Operator with Head Tracker Virtual Teeth PC Monitor Half Mirror Combined Image Customized Attachment Teeth Model Haptic Device

Fig. 1. System Overview of HAP-DENT C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 340–344, 2011. © Springer-Verlag Berlin Heidelberg 2011

Pen Tip Position Estimation Using Least Square Sphere Fitting

341

Dentists use different tools to perform different dental operations, such as drills for grinding or scalers for removing dental calculus (Fig. 2). To build a simulator that creates a high degree of realism, it is necessary to prepare customized attachments (Fig. 3) that have the same shapes as real dental tools. Additionally, for simulation, it is necessary that the position of the tip of the attachment should be known. In this paper, we propose a method for estimating the pen tip position of customized haptic device attachments. We have studied a calibration method for multiple haptic devices [5]; however, the tip position was hard-coded in the method. The proposed method estimates the tip position by fitting the acquired point cloud data to a sphere using the least squares approach.

2 Estimation Method Using Least Squares Sphere Fitting The tip of a customized haptic device attachment is maintained at a certain position, and the attachment is manipulated in various directions to obtain a spherically arranged positioning data set. The radius and center of the sphere can be estimated by fitting the point cloud data to a sphere by the least squares approach. The estimated radius is equal to the length between the tip and default origin point.

(a) Grinding

(b) Scaling

(c) Probing

(d) Radectomy

Fig. 2. Typical dental tasks

(a) Default stylus

(b) Customized attachment

Fig. 3. Customized attachment for dental simulator

342

M. Koeda and M. Kato

Fig. 4 illustrates the coordination system. The acquired N-point cloud data are defined as b p si = ( xi yi zi )T i = 1" N . The center and radius of the sphere xc ,

yc , zc , and r = s p t , respectively, are calculated by the following equation

[x

c

yc

zc

r

]

⎡ a = ⎢− ⎣ 2

2 T

−

b 2

−

⎤ a 2 + b2 + c 2 − d⎥ 4 ⎦

c 2

T

(1)

where a, b, c, and d are given by

⎡ N 2 ⎢ ∑ xi i =1 ⎡a⎤ ⎢ N 2 2 ⎢ ⎢ b ⎥ ∑ yi z i ⎢ ⎥ = ⎢ i =1 ⎢c ⎥ ⎢ N 2 2 ⎢ ⎥ ⎢ ∑ zi xi ⎣d ⎦ ⎢ i =1N ⎢ ⎢ ∑ xi ⎣ i =1

N

N

∑ xi2 yi2

∑ xi2 zi2

∑y

∑y z

i =1 N

i =1 N

2 i

2 2 i i

i =1 N

∑z i =1 N

2 i

i =1 N

y

∑z

2 i

i =1 N

∑y i =1

2 i

∑z

i

i =1

i

⎤ xi ⎥ ∑ i =1 ⎥ N yi ⎥ ∑ ⎥ i =1 N ⎥ zi ⎥ ∑ ⎥ i =1 ⎥ N ⎥ ⎦ N

−1

⎡ N 2 ⎤ 2 2 ⎢ − ∑ ( xi + yi + zi ) xi ⎥ ⎢ iN=1 ⎥ ⎢− ( x 2 + y 2 + z 2 ) y ⎥ i i i i ⎢ ∑ ⎥ i =1 ⎢ N ⎥ ⎢ − ∑ ( xi2 + yi2 + zi2 ) zi ⎥ ⎢ i =1 ⎥ N ⎢ ⎥ 2 2 2 ⎢ − ∑ ( xi + yi + zi ) ⎥ ⎣ i =1 ⎦

(2)

3 Experiment and Result The experimental setup consists of a haptic device {Phantom Omni [6] (Fig. 4)}, a customized attachment (Fig. 5), and a metallic calibration block (Fig. 6). The length of the customized attachment, as measured using a ruler, is approximately 113 [mm]. Twelve small dimples (A-L, at 20 [mm] intervals) were placed on the surface of the calibration block. These dimples were used to curb tip movement during point-data acquisition.

∑

∑

b

ps

s

s

pt

b

Fig. 4. Coordination system

Actual measured value = 113 [mm]

Fig. 5. Customized attachment

Pen Tip Position Estimation Using Least Square Sphere Fitting

343

m] m] 20 [m [m [mm] 0 2 20 [mm] A 20 B C 20 [mm] D E F G H I J K L

(a) Overview

(b) Twelve small dimples on surface Fig. 6. Calibration block

Fig. 7. Snapshots of data acquisition process

Fig. 8. Sample of acquired point cloud data

Fig. 7 shows snapshots of the experimentation. Point cloud data (N = 1000) were acquired at each dimple, and the radius of the sphere was estimated using the proposed method. Fig. 8 shows sample point cloud data, and it can be seen that the points are spherically arranged. The results of the radius estimation experiment are listed in Table 1 and Fig. 9. The estimated average radius is 112.3 [mm], and this value is close to the measured length of the customized attachment.

344

M. Koeda and M. Kato Table 1. Experimental result

Length [mm]

Measurement Point Length [mm] Measurement Point Length [mm] Measurement Point Length [mm]

A 111.8 E 113.3 I 112.5

B 110.4 F 113.7 J 111.2

C 113.0 G 112.6 K 111.3

D 113.7 H 111.8 L 112.7

115 114 113 112 111 110 109 108 107 106 105 A B C D E F G H I J K L Measurement Point Fig. 9. Experimental result

4 Conclusion In this paper, a method for estimating the pen tip position of customized haptic device attachment was proposed. The proposed method was verified using a customized attachment, which is modeled as a grinding drill. It was found that the error was less than 3% and that the proposed method has reasonable accuracy.

References 1. Nissin Dental Products Inc., Simple Manikin II, http://www.nissin-dental.net/ products/DentalTrainingProducts/DentalSimulator/simplemannequ in/index.html 2. Uzuka, S., et al.: Development of Patient Simulation System: Application for Clinical Education. In: The IADR/AADR/CADR 85th General Session and Exhibition (2007) 3. Takanobu, H., et al.: Dental Patient Robot. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1273–1278 (2006) 4. Yamaguchi, S., et al.: Dental Haptic Simulator to Train Hand Skill of Student. In: Proceedings of the 3rd MEI International Symposium, p. 43 (2008) 5. Koeda, M., et al.: Estimation Method for Relative Position and Attitude Between Two Haptic Devices. In: Proceedings of 12th International Conference on Human-Computer Interaction, pp. 446–450 (2007) 6. Sensable Technologies, Inc., Phantom Omni, http://www.sensable.com/haptic-phantom-omni.htm

Corrected Human Vision System and the McGurk Effect Ladislav Kunc and Pavel Slavík Department of Computer Graphics and Interaction, Faculty of Electrical Engineering, Czech Technical University in Prague [email protected], [email protected]

Abstract. The McGurk effect test is a method to evaluate articulation of talking heads. Our work addresses the issue of corrected vision influence on the McGurk effect perception. We conducted an experiment to investigate this influence. Measured data shows different perception of participants with corrected vision in some cases. The results could help talking head evaluators to compare talking head implementations each other with elimination of the influence of corrected vision. Keywords: Talking head, McGurk effect, vision correction.

1 Introduction Voice based user interface is one of possible human-computer interaction methods. Spoken dialog systems provide new interaction modalities for user. This usually means verbal (speech) part of communication. We should not forget that the verbal part is closely tied to nonverbal part of our language in traditional human-to-human interaction. Humans listening to speech are used to focus on words. But during complex assessment of speaking person we process both parts of speech: nonverbal and verbal one [1]. Inclusion of some form of face-to-face interaction into spoken dialog technology enables the system to express nonverbal part of communication. Seeing virtual faces further humanizes computer user interfaces and makes them more acceptable for a common user [2]. For example talking heads provide means for additional nonverbal communication. Perceptually realistic visual articulation of a talking head is very important because humans are highly sensitive to perception of face muscles movements. The McGurk effect test is a method to evaluate speech articulation. Our work addresses the issue of corrected vision influence on the McGurk effect perception. The McGurk effect shows that humans use both hearing and vision modalities in parallel to perceive and understand speech. The first experiment was presented in [3]; there was dubbed videotape of visual ga syllable with audio ba syllable. Experiment’s participants thought that da syllable was pronounced. Our hypothesis is that people with corrected vision will judge the McGurk effect sequences differently than people with non-corrected normal vision. We conducted an experiment to validate our hypothesis. We prepared synthetic McGurk video sequences for participants and evaluated participants’ responses. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 345–349, 2011. © Springer-Verlag Berlin Heidelberg 2011

346

L. Kunc and P. Slavík

The next section gives an overview of related works on evaluation of talking heads articulation.

2 Related Work The primary goal of talking head articulation research is to produce the realistic visual articulation which is indistinguishable from real human. This task includes implementation methods and in the end evaluation of results. The quality of talking heads’ visual articulation has been measured using various methods. These comprise subjective evaluation, perception of speech in noisy environments and others. The most common method is subjective evaluation [4]. This method is based on getting comments and rating from naïve and expert participants. This provides information on quality of talking head articulation but does not give possibility to compare various talking head implementations. The second method – perception of speech in noisy environments improves the quality of results and facilitates comparisons of miscellaneous talking head implementations. Participants try to listen to talking head footage in noisy room and the ability of understand the words indicates how much they are able to improve the intelligibility of speech by lip-reading talking head [5]. Good results provide methods based on forced choice. Participants see videos and identify which animation is real or synthetic [6]. Extensive experiments of the McGurk effect using talking head were done in [7]. Interesting method based on perception of this effect was proposed in [8]. Participants are given synthetic talking head McGurk sequences and their confusion responses are measured. The last method was proved by judgments of participants with normal hearing and vision; not mentioned whether participants had corrected vision or not. Our work focuses on how corrected-to-normal vision influences perception of speech.

3 Experiment This section describes details of perceptual experiment conducted with aim to evaluate how corrected-to-normal vision influences perception of the McGurk effect on a talking head. The experiment should validate or reject our hypothesis: ─ People with corrected vision will judge the McGurk effect differently than people with non-corrected normal vision. Participants saw stimuli videos with synthetic McGurk effect and control sequences without McGurk effect. Totally 33 stimuli videos were prepared for participants. Videos contained possible combinations of artificial face, human face, text-to-speech audio and human audio modeling syllables ba – da – ga (see Table 1). The video sequences in human category were recorded by the means of usual web camera in resolution 640x480. Figure 1 illustrates what portion of face was recorded.

Corrected Human Vision System and the McGurk Effect

347

Table 1. Stimuli audio-video combinations prepared for participants. TTS stands for text-tospeech audio. Visual Human

Sound ba Visual ba Visual da Visual ga Talking head Visual ba Visual da Visual ga Black screen N/A

Sound da Visual ba Visual da Visual ga Visual ba Visual da Visual ga N/A

Sound ga Visual ba Visual da Visual ga Visual ba Visual da Visual ga N/A

TTS ba

TTS da

TTS ga

Visual ba Visual ba Visual ba Visual da Visual da Visual da Visual ga Visual ga Visual ga N/A N/A

Audio tracks were recorded using computer microphone headset. Recorded visual video sequences were mixed with particular audio tracks in video editing software1. Talking Head Application. Preparation of stimuli video sequences required application that is able to animate talking head model. ECAF toolkit application was used [9]. This toolkit displays and animates 3D talking head model of woman and produce output video sequences synchronized with either external audio file or integrated text-to-speech engine. The synchronization of audio track and phonemes animation of McGurk sequences was modified manually. Experiment Procedure. We prepared 33 video sequences (see Table 1) generated either by ECAF toolkit or as recorded human sessions. Each sequence is about 7 seconds long. The talking head or human repeats 6 times particular syllable ba, da or ga. The experiment was conducted remotely. Every participant observed the video sequences in his/her own computer in different but quiet environment. The test was handled by PHP web application with embedded Adobe Flash player for video playback (see Figure 1). It was not possible for participant to scale down the video from original resolution (640x480). The participants were both males and females in between ages of 23 to 26, university students of computer science. The whole experiment was anonymous and participants were instructed how to proceed the experiment verbally. At the beginning the web application asked participants whether they have or not have some vision correction. After this question a participant was navigated to the first video sequence. For each participant random order of video sequences was generated. The first three video sequences were training ones and the data from them was not used. Participant observed each video and fill-in the textbox with the text that he/she perceived. Each participant observed 36 (3 training ones + 33) video sequences. 3.1 Experiment Evaluation In total 32 participants took part in our experiment (41% had corrected vision). The data from experiment were manually normalized. For example: One participant 1

Adobe Premiere CS5.

348

L. Kunc and P. Slavík

Fig. 1. Experimental web application – a) example of human and talking head video sequence snapshots; b) the screenshot of web page with observation no. 3 which was seen by some participant. There is a text box for participant’s answer under the video player window.

answered ‘vaaa vaa va’ and the second participant answered ‘bababa’. The correct answer based on audio track should be ‘ba-ba ba-ba ba-ba’. After normalization the first answer was marked as correct and the second as incorrect. The graph in Figure 2 shows results for the McGurk sequences with the highest number of incorrect answers. Confusion video sequences are our main interest. The mean of incorrect responses for participants with normal vision is 3.89 (standard deviation 3.2). The mean of incorrect responses for participants with corrected-tonormal vision is 1.56 (standard deviation 1.85). This result denotes some differences in the McGurk effect perception by normal vision participants and participant with corrected vision. To make the results complete sequences with no confusion give mean of incorrect responses 1.2 (standard deviation 1.1).

Fig. 2. Results of talking head McGurk sequences with the highest number of incorrect answers. Last three results used text-to-speech generated audio.

Corrected Human Vision System and the McGurk Effect

349

4 Conclusion We conducted an experiment with the McGurk confusion audio-video sequences using talking head and human head. Our hypothesis was that people with corrected vision will judge the McGurk effect differently than people with non-corrected normal vision. The previous section presented results of our experiment and it is clearly visible that there are some differences between group of participants that had corrected-to-normal vision and participants with non-corrected normal vision. Results of our work could help researchers that want to evaluate their developed talking heads using the McGurk perception test. They should consider the vision correction of their participants in their results and report both groups separately to be comparable. Acknowledgement. This research has been partially supported by the MSMT under the research program MSM 6840770014. This paper has been also partially supported by the EC funded project VERITAS, contract number FP7 247765.

References [1] Knapp, M., Hall, J.: Nonverbal Communication in Human Interaction, 7th edn. Wadsworth Publishing, Belmont (2009) [2] Yee, N., Bailenson, J.N., Rickertsen, K.: A Meta-Analysis of the Impact of the Inclusion and Realism of Human-Like Faces on User Experiences in Interfaces. In: Proc. of CHI 2007, pp. 1–10 (2007) [3] McGurk, H., MacDonald, J.: Hearing Lips and Seeing Voices. Nature 264, 746–748 (1976) [4] Cosatto, E., et al.: Lifelike Talking Faces for Interactive Services. IEEE Special Issue on Human-Computer Multimodal Interface 91(2), 1406–1429 (2003) [5] Massaro, D.W.: A Framework for Evaluating Multimodal Integration by Humans and a Role for ECAs. In: Proc. of the 6th ICMI Conference, pp. 24–31 (2004) [6] Hack, C., Taylor, C.J.: Modelling ‘Talking Head’ Behavior. In: Proc. of British Machine Vision Conference, pp. 122–132 (2003) [7] Massaro, D.W., Stork, D.G.: Speech Recognition and Sensory Integration. American Scientist 86(3), 236–244 (1998) [8] Cosker, D., Paddock, S., Marshall, D., Rosin, P., et al.: Towards Perceptually Realistic Talking Heads: Models, Methods and McGurk. In: Proc. of the 1st Symposium on Applied Perception in Graphics and Visualization, pp. 151–157 (2004) [9] Kunc, L., Kleindienst, J.: ECAF: Authoring Language for Embodied Conversational Agents. In: Proc. of Text, Speech, Dialog Conference, pp. 206–213 (2007)

Facial Landmark Extraction for Lip Tracking of Patients with Cleft Lip Using Active Appearance Model Nayoung Lee1, Chuck Heaston2, Ada Rey1, Terry Hartman1, and Carroll-Ann Trotman3 1

Department of Orthodontics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA 2 3dMD, 100 Galleria Parkway, Suite 1070, Atlanta, GA 30339, USA 3 Professor, Department of Orthodontics, University of Maryland Dental School, 650 W. Baltimore Street, Room 6408, Baltimore, MD 21201 and Adjunct Professor, Department of Orthodontics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA [email protected], {reya,terry_hartman}@dentistry.unc.edu, [email protected], [email protected]

Abstract. Patients with cleft lip have trouble in communication because of facial disability resulting in both facial disfigurements and impairments in movement. Consequently, for individuals with facial functional impairments, lip tracking methods for analyzing these impairments are useful in treatment planning and diagnosis. In comparison with non-cleft control subjects, facial landmarks for lip tracking of patients with cleft lip need to be widely defined in the circumoral area in order to reflect an irregularly shaped mouth. In this paper, we discuss the idea of facial landmark extraction for lip tracking in patients with cleft lip. The idea is based on facial features, identified by using AAM (Active Appearance Model) algorithm for finding facial landmarks in the circumoral area. These landmarks are composed of the outer and inner contours of the mouth in the circumoral area. The inner contour of the mouth is represented as landmarks on the mouth. The outer contour of the mouth is divided into the upper outer contour and lower outer contour on the basis of the center line. Here, the center line is defined as an intersecting line which connects from the left corner to the right corner of the mouth. The distance from the lower lip to the jaw line in general is twice that from the bottom of the nose to the upper lip. Therefore, a set of artificial landmarks, the upper and lower outer contours, have been defined using a set of pre-determined distances. The distance from the lower lip to the lower outer contour used is twice that from the upper lip to the upper outer contour. Using these landmarks, we gathered objective measures of facial form in patients with cleft lip who have both form and functional deficits. All subjects were recruited from patients attending at the University of North Carolina School of Dentistry Craniofacial Center, Orthodontic, Pedodontic Clinics. Supported by: NIDCR GRANT #DE013814 & #DE019742. Keywords: Facial Landmarks, AAM (Active Appearance Model), Circumoral, Cleft Lip, Lip Tracking.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 350–354, 2011. © Springer-Verlag Berlin Heidelberg 2011

Facial Landmark Extraction for Lip Tracking of Patients

351

1 Introduction Patients with cleft lip have trouble in communication because of facial disability resulting in both facial disfigurements and impairments in movement. Consequently, for individuals with facial functional impairments, methods for analyzing these impairments are useful in diagnosis, treatment planning, and outcome assessment of surgical rehabilitative procedures [4]. For these reasons, researchers have attempted through various strategies to quantify facial functional deficits [4]. In comparison with noncleft control subjects, facial landmarks for lip tracking of patients with cleft lip need to be widely defined in the circumoral area in order to reflect an irregularly shaped mouth. Active Appearance Model (AAM), proposed by Cootes et al [5], is one of the most powerful model based algorithms [16]. It can be traced back to the active contour model (or “snakes,” [6]) and active shape model (ASM) [7]. Particularly, AAM decouples and models the shape and the texture of the deformable object, and is able to generate a variety of instances photo-realistically. Therefore, AAM has been widely used under various situations, for example, human eyes modeling [8], object tracking [9], facial and expression recognition system [10-13], gain analysis [14], and medical image segmentation and analysis [15,16]. In this paper, we discuss the idea of facial landmark extraction for lip tracking in patients with cleft lip. The structure of the paper is as follows. In Section 2, we describe the methodology. Finally, we discuss conclusion in Section 3.

2 Methodology We describe a method of facial landmark extraction in patients with cleft lip. The proposed method is based on facial features, identified by using AAM algorithm for Feature detection using AAM

Component labeling such as the eyebrows, nose and mouth

Label= Mouth?

No

End

Yes Detection of the left and right corner points

Finding artificial Landmarks in the circumoral area

End

Fig. 1. Overall flow of the system configuration

352

N. Lee et al.

finding facial landmarks in the circumoral area. Fig. 1 shows the overall flow of the proposed method of facial landmark extraction for lip tracking in patients with cleft lip. In practice, a face detector is used to first isolate regions containing faces only. Then, AAM is used for extracting facial landmarks. The coordinates of all landmarks for each image are stored as a vector, called shape

x = ( x1 , y1 , " , xn , y n )T , where xi and yi are the x and y coordinates of the i -th landmarks and n is the number of landmarks used [17]. In

(x), in the form

this paper, the facial landmarks were determined using 64 facial landmarks as shown in Fig. 2.

Fig. 2. 64 facial landmarks

The facial landmarks in the circumoral area are composed of the outer and inner contours of the mouth as shown in Fig. 3.

: outer contour of mouth : inner contour of mouth : center line Fig. 3. Facial landmarks in the circumoral area

Facial Landmark Extraction for Lip Tracking of Patients

353

Equation (1) describes the facial landmarks of the circumoral area.

Mouth = {moc , mic } where

(1)

moc and mic are landmarks of the outer and inner contour of the mouth,

respectively. The inner contour of the mouth is represented as landmarks on the mouth. The mic can be represented as equation (2).

mic = { plc , prc , pu , pl }

(2)

plc and prc are the left and right corner point of the mouth, respectively, while pu and pl are sets of landmarks on the upper and lower lip, respectively. The landmarks on the mouth are sorted in ascending order according to the x coordinate where

for detecting corner points. The left corner and right corner on the mouth are determined as the coordinates having minimum value and maximum value among the x coordinate values along inner contour of the mouth, respectively. The outer contour of the mouth is divided into the upper outer contour and lower outer contour on the basis of the center line. Here, the center line is defined as an intersecting line which connects from the left corner to right corner of the mouth. The distance from the lower lip to the jaw line in general is twice that from the bottom of the nose to the upper lip. Therefore, a set of artificial landmarks, the upper and lower outer contours, have been defined using a set of pre-determined distances. The distance from the lower lip to the lower outer contour used is twice that from the upper lip to the upper outer contour.

3 Conclusion We have discussed the idea of facial landmark extraction for patients with cleft lip. The proposed method is expected to result in enhanced extraction of facial landmarks in the circumoral area. Future work will involve extending our approach to lip tracking for patients with cleft lip. Acknowledgments. This study was supported by grant #DE013814 & #DE019742 from the National Institute of Dental Research (NIDCR).

References 1. Trotman, C.-A., Faraway, J.J., Phillips, C., van Aalst, J.: Effects of Lip Revision Surgery in Cleft Lip/Palate Patients. J. Dent. Res. 89(7), 728–732 (2010) 2. Phillips, C., Amphlett, B., Robbé, I.J.: The Long-term Effects of Water Fluoridation on the Human Skeleton. J. Dent. Res. (2011) 3. van Aalst, J.: Using Google Scholar to Estimate the Impact of Journal Articles in Education. Educational Researcher 39(5), 387–400 (2010)

354

N. Lee et al.

4. Weeden, J.C., Trotman, C.-A., Faraway, J.J.: Three Dimensional Analysis of Facial Movement in Normal Adults-Influence of Sex and Facial Shape. Angle Orthod. 61, 132– 140 (2001) 5. Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active Appearance Models. In: Proc. Eur. Conf. Comput. Vis., vol. 2, pp. 484–498 (1998) 6. Kass, M., Witkin, A., Terzopoulos, D.: Snakes-Active Contour Models. In: Proc. Int. Conf. Comput. Vis., pp. 259–268 (1987) 7. Cootes, T.F., Copper, D.H., Taylor, C.J., Graham, J.: Active Shape Models-Their Training and Application. Comput. Vis. Image Understanding 61, 38–59 (1995) 8. Wan, J., Renn, X., Hu, G.: Automatic Red-eyes Detection based on AAM. In: Proc. IEEE Int. Conf. Syst., Man Cybern., vol. 7, pp. 6337–6341 (2004) 9. Stegmann, M.B.: Object Tracking using Active Appearance Models. In: Proc. Danish Conf. Pattern Recog. Image Anal., vol. 1, pp. 54–60 (2001) 10. Xiao, B., Gao, X., Tao, D., Li, X.: A new Approach for Face Recognition by Sketches in Photos. Signal Process 89, 1567–1588 (2009) 11. Tao, D., Song, M., Li, X., Shen, J., Sun, J., Wu, X., Faloutsos, C., Maybank, S.: Bayesian Tensor Approach for 3-D Face Modeling. IEEE Trans. Circuits Syst. Video Technol. 18, 1397–1410 (2008) 12. Liebelt, J., Jing, X., Jie, Y.: Robust AAM Fitting by Fusion of Images and Disparity Data. In: Proc. IEEE Conf. Comput. Vis. Pattern Recog., pp. 2483–2490 (2006) 13. Zhang, T., Tao, D., Li, X., Yang, J.: Patch Alignment for Dimension Reduction. IEEE Trans. Knowl. Data Eng. 21, 1299–1313 (2009) 14. Li, X., Maybank, S., Yan, S., Tao, D., Xu, D.: Gait Components and Their Application to Gender Recognition. IEEE Trans. Syst., Man, Cybern. C, Appl. Rev. 38, 145–155 (2008) 15. Mitchell, S.C., Lelieveldt, B.P.F., van der Geest, R.J., Bosch, H.G., Reiber, J.H.C., Sonka, M.: Multistage Hybrid Active Appearance Model Matching-Segmentation of Left and Right Ventricles in Cardiac MR Images. IEEE Trans. Med. Imag. 20, 415–423 (2001) 16. Gao, X., Su, Y., Li, X., Tao, D.: A Review of Active Appearance Models. IEEE Trans. Syst., Man, Cybern. C, Appl. Rev. 40, 145–158 (2010) 17. Seshadri, K., Savvides, M.: Robust Modified Active Shape Model for Automatic Facial Landmark Annotation of Frontal Faces. In: Proceedings of the 3d IEEE International Conference on Biometrics: Theory, Applications and Systems, pp. 319–326 (2009)

Kansei Evaluation of the Projection for the Approach to Universal Design: Computerization of Tactile Sensibility Miyong Lee1, Kazuhiro Nishida2, and Yoshihiro Narita1 1 Division of Human Mechanical Systems and Design, Faculty of Engineering, Hokkaido University, Kita 13, Nishi 8, Kita-ku, Sapporo, Hokkaido, 060-8628, Japan 2 Division of Human Mechanical Systems and Design, Graduate School of Engineering, Hokkaido University, Japan {leemiyong,ynarita}@eng.hokudai.ac.jp, [email protected]

Abstract. This study focused on the projection of a shampoo container as a case of computerization of tactile sensibility. There is a projection distinguishing it from a conditioner container without having to look at the shampoo container. This study estimated the projection using Kansei evaluation about a feeling when we touch the container and aimed at investigating the characteristics of the projection. For this study, we produced two types of models that are different in the surface shape of the object. The individual models have three parameters (height/length/interval) that refer to the shape of the projection. As a result, height created the strongest impression of the three parameters of the projection. And there was hardly any difference in the impression in the plane type models and the curved type models. This study is an attempt to computerize the feeling by touching and it is expected to be useful as fundamental research of recognition of human ignorance. Keywords: Kansei evaluation, projection, computerization, tactile sensibility, SD method.

1 Introduction The Visionary of Universal Design, Ron Mace was an internationally recognized architect, product designer and educator whose design philosophy challenged convention and provided a design foundation for a more usable world. He coined the term "universal design" to describe the concept of designing all products and the building constructing environment to be aesthetic and usable to the greatest extent possible by everyone, regardless of their age, ability, or status in life [1]. Recently, universal design is utilized a lot in our lives including in environmental designs and daily life products [2]. In addition, a lot of studies on universal design are performed, but there is little approach about an impression and the usability of the user. This study focused on the projection of a shampoo container as a case of universal design. There is a projection that enables distinguishing the shampoo container from a conditioner container without having to actually look at the shampoo container. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 355–359, 2011. © Springer-Verlag Berlin Heidelberg 2011

356

M. Lee, K. Nishida, and Y. Narita

Traditionally, any study about the projection of a shampoo container has involved degree of recognition [3]. However, the feeling conjured by the projection is an important element for superior universal design. This study estimated the projection of the feeling conjured when touching the container, and investigated the characteristics of the projection using parameters. In other words, this is an attempt to computerize the sensation of the tactile feeling using Kansei evaluation [4]. Kansei is a peculiar Japanese term, which is often translated as a higher order function of the brain as a source of inspiration, intuition, pleasure/displeasure, taste, curiosity, aesthetics, and creation [5]. This information which is called Kansei information can become the fundamental data for recognition of human ignorance and development of a humanoid robot.

2 Overview of Experiment For this study, we produced two types of models that are different in the surface shape of the object by 3D plotter called MDX-40. One is a plane type model and the other is a curved type model. The materials of the model used chemical wood (RAKU TOOL SB0470). As wood is natural and commonly used material, we suggest that it has less influence than other materials. The size of plane type models is 25×50×150mm and the size of curved type models is 50×50×150mm. The individual types have three parameters (height, length and interval) that refer to the shape of the projection. The projection size of the standard model was selected based on the mean of the commercial shampoo containers. The height of the standard model is 0.7mm, the length is 6.0mm, and interval is 7.0mm. We produced six models other than a standard model. Two models varied only in height, two models varied only in length, and two models varied only in interval based on a standard model (table 1) (Fig.1). Impression evaluation was performed on the parameters using the semantic differential method (SD method) and a comparison of the characteristics. For the SD method, ten adjectives related to the feeling of touch were chosen; painful, beautiful, friendly, stiff, soft, smooth, rounded, good feeling, meaningful, easy to understand. The subject evaluated each model with five phases while touching a model with open eyes. This is the reason that the experiment for tactile sensibility for an ordinary person is not the experiment only for a handicapped person. Finally, the subjects chose two models that felt the best. There were 50 subjects (31males and 19 females) and the average age was 25.6( 10.4).

±

Table 1. The size of projection by three parameters models Height(mm) Length(mm) Interval(mm)

SM 0.7 6.0 7.0

H(-) 0.4 6.0 7.0

H(+) 1.0 6.0 7.0

L(-) 0.7 3.0 7.0

L(+) 0.7 9.0 7.0

I(-) 0.7 6.0 4.0

I(+) 0.7 6.0 10.0

Kansei Evaluation of the Projection for the Approach to Universal Design

357

Fig. 1. Top view of Models

3 Results and Analyses The experimental results were analyzed in three ways. The first was the result of SD method by each model, the second was choice of best model based on tactile sense, and the third was a comparison between plane type models and curved type models. For the first analysis, height created the strongest impression of the three parameters of the projection. For the height, when height causes the evaluation of “painful/stiff” to be higher, then the evaluation of “beautiful/friendly/soft/smooth/ rounded/good feeling” become lower. The evaluation of “meaningful/easy to understand” was the highest in the standard model (Tab. 2). If length became long, the evaluation of “smooth” became higher, and the evaluation of “friendly/soft/rounded” became lower. The evaluation of “painful/ beautiful/stiff/good feeling/meaningful/easy to understand” was the highest in a standard model. When the interval was long, the evaluation of “painful” became higher, and the evaluation of “good feeling” became lower. In a standard model, the evaluation of “beautiful/stiff/meaningful/easy to understand” was the highest and the evaluation of “friendly/soft/smooth/rounded” was the lowest. For the second analysis, “H( )” having the lowest height got the best evaluation of the touch (37%). “H( )” having the highest height got the worst evaluation of the touch (7%). From this result, it is thought that the height of projection influences tactile feeling above all. We found that the projection that the height is low, the length is long, and interval is short is the best tactile feeling (Tab. 3). For the third analysis, there was hardly any difference in the impression in the plane type models and the curved type models, and the only profound difference was a description of the container being painful to hold.

+

-

358

M. Lee, K. Nishida, and Y. Narita Table 2. Resualts of plane type models by adjectives models

SM -0.76

H(-) -1.48

H(+) 0.38

L(-) -0.92

L(+) -0.96

I(-) -0.98

I(+) -0.30

painful

M SD

±1.18

0.75

1.23

0.89

1.06

0.88

1.25

beautiful

M

-0.18

-0.42

-0.32

-0.24

-0.36

-0.46

SD

±0.95

0.28 1.23

0.92

0.95

0.93

0.97

0.98

friendly

M

-0.36

-1.10

-0.04

0.32

-0.26

-0.30

SD

1.20

1.28 0.66

0.78

1.08

1.01

1.05

0.98

stiff

M

0.60

1.28

0.12

-0.26

0.26

0.14

SD

1.08

-0.98 0.99

0.80

1.01

1.16

1.09

1.20

soft

M

-0.98

-1.18

-0.4

-0.14

-0.48

-0.32

SD

0.95

1.06 1.03

0.68

0.87

1.15

1.04

1.12

smooth

M

0.02

-0.84

-0.20

0.20

0.04

0.06

SD

1.21

1.40 0.87

1.05

1.08

1.22

1.06

1.19

rounded

M

-0.56

-1.10

-0.36

0.24

-0.40

-0.16

SD

1.10

1.00 1.10

0.83

1.18

1.01

1.06

1.17

good feeling

M

0.24

-0.64

0.20

0.22

0.50

-0.16

SD

1.05

1.24 0.79

0.97

1.18

1.15

1.00

1.10

meaningful

M

0.00

-0.02

-0.04

-0.06

-0.12

-0.20

SD

1.17

-0.16 1.08

1.10

1.20

1.08

1.09

1.10

easy to understand

M

0.76

-0.32

0.68

0.34

0.04

0.24

-0.22

1.12

1.21

1.29

1.17

1.19

1.24

SD

1.21

Table 3. The choice rate of best feeling model models choice rate (%)

SM 9

H(-) 37

H(+) 7

L(-) 8

L(+) 15

I(-) 13

I(+) 11

4 Conclusion This study aimed to evaluate tactile feeling using a Kansei evaluation of a rounded projection, such as that of a container of shampoo. Through an experiment using a production of the real model, we investigated the characteristics of a good projection of reasonable touch. Of the three parameters used to depict the shape of the projection, height created the strongest impression of the projection. The projection that the height is low, the length is long, and interval is short creates the best sensation when we touch a container. There was hardly any difference in the impression in the plane type models and the curved type models, and the only profound difference was a description of the container being painful to hold. It is thought that data about the human tactile sensation that were collected in this experiment have high utilization can provide valuable Kansei information [6]. Existing universal design was applied mainly on recognition and intelligibleness. In future, it will be necessary to develop universal design thinking about each user’s sensations as well as recognition. In future, we will evaluate the projection with a real product and will investigate the characteristics of the appropriate projection between the recognition and the sensation conjured by the projection.

Kansei Evaluation of the Projection for the Approach to Universal Design

359

References 1. The Center for Universal Design, NC State University, http://www.ncsu.edu/www/ncsu/design/sod5/cud/about_us/ usronmace.htm 2. Naoki, H., Yuko, K., Keiichi, W.: Universal Design of Multifunctional Peripherals Based on User Survey. Toshiba Review 65(2), 19–23 (2010) 3. Miyoshi, I., Sako, H.: Universal Design of Alcoholic Beverage Container-Proposal for tactile symbols representing alcoholic beverages. Shizuoka University of Art and Culture Bulletin 8, 105–115 (2007) 4. Harada, A.: On the definition of Kansei. In: Modeling the Evaluation Structure of Kansei Conference, vol. 2 (1998) 5. Lee, M., Lee, S.: Kansei evaluation of three-dimensional geometrical shapes using pictorial images. In: Proceedings of the 16th International Conference on Engineering Design ICED 2007, Paris, vol. 342 (2007) 6. Lee, M., Lee, S.: Kansei Evaluation in the Thickness of Physical Objects Based on Tactile Impression-Proposal of a Kansei Category Based on How We Feel about Objects. Journal of Japan Society of Kansei Engineering 9(1), 1–9 (2009)

A Framework of Motion Capture System Based Human Behaviours Simulation for Ergonomic Analysis Ruina Ma1, Damien Chablat1, Fouad Bennis1, and Liang Ma2 1

Institut de Recherche en Communications et Cybernétique de Nantes (IRCCyN) / Ecole Centrale de Nantes, 1 Rue de la No, 44321 Nantes Cedex 3, France 2 Tsinghua University, 100084, P.R. China, Beijing {ruina.ma,damien.chablat,fouad.bennis}@irccyn.ec-nantes.fr [email protected]

1 Introduction With the increasing of computer capabilities, Computer aided ergonomics (CAE) offers new possibilities to integrate conventional ergonomic knowledge and to develop new methods into the work design process. As mentioned in [1], different approaches have been developed to enhance the efficiency of the ergonomic evaluation. Ergonomic expert systems, ergonomic oriented information systems, numerical models of human, etc. have been implemented in numerical ergonomic software. Until now, there are many ergonomic software tools available, such as Jack, Ergoman, Delmia Human, 3DSSPP, and Santos, etc. [2-4]. The main functions of these tools are posture analysis and posture prediction. In the visualization part, Jack and 3DSSPP produce results to visualize virtual human tasks in 3-dimensional, but without realistic physical properties. Nowadays, with the development of computer technology, the simulation of physical world is paid more attention. Physical engines [5] are used more and more in computer game (CG) field. The advantage of physical engine is the nature physical world environment simulation. The purpose of our research is to use the CG technology to create a virtual environment with physical properties for ergonomic analysis of virtual human. In the following sections, firstly we will introduce motion capture system and the related data obtained and processed. Secondly we will illustrate the physical engine, and by using it we create our virtual physical environment. Thirdly, we propose a simulation framework using motion capture system and physical engine for ergonomic analysis. At last, we introduce a study case to present the ergonomic analysis based on a muscle fatigue model.

2 Motion Capture Motion capture techniques have been applied frequently to obtain the dynamic and natural motion information in human simulation tools [6]. Several kinds of tracking systems are available, such as mechanical motion tracking, acoustic tracking, magnetic tracking, optical motion tracking, and inertial motion tracking [7]. We use the motion capture system which is produced by NaturalPoint Company [8] to obtain the trajectory of human movement. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 360–364, 2011. © Springer-Verlag Berlin Heidelberg 2011

A Framework of Motion Capture System Based Human Behaviours Simulation

361

Optical tracking system camera placement: Our optical capture system is composed of three cameras around the work space. Markers are made from reflective material that could be tracked by camera sensitively, and they are attached to the human articulation. In order to track markers, multiple OptiTrack cameras must be arranged to have several overlapped fields of view. For best calibration and tracking results, it should be avoided to place al the cameras in the same plane, instead position the cameras at different angles. This will create an area called a capture volume in which tracking can occur. The overall capture system works at the frequency 100 Hz. It could offer a minimum sampling rate to have good image rendering, and this could provide sufficient detailed analysis of the static and dynamic human body motions.

Camera fixing Camera calibration

Data collection Data transmission Data save

Data preprocessing

Fig. 1. Camera Installation and Data Obtain Process

Data obtain process: Camera calibration here refers to the calibration of the multicamera system. It is the base of the following work. Dynamic camera calibration method which offered by the software was used to calibrate multiple cameras. In the Data Collection part, we open the interface about data collection. Markers are placed in the proper articulation, in order to get the relatively accurate position. In each articulation we put at least two markers and organise them as a tracker table. In this situation the average spatial position values of the markers will be the final trajectory values. In order to get the high quality trajectory, these markers moving in the capture volume space, should be viewed by at least two cameras. The data we get from the software is the format “.cvs”. In this format it contains much information more than we want. We must extract the information that we are interested in. In the data pre-processing part we do the data extraction using Matlab.

3 Physical Engine A physics engine is computer software that provides an approximate simulation of certain simple physical systems, such as rigid body dynamics (including collision detection), soft body dynamics, and fluid dynamics. They are use in films, computer graphics, and mainly in video games (typically as middleware), in which the simulations are in real-time. The term is sometimes used more generally to describe any software system for simulating physical phenomena, such as high-performance scientific simulation [9]. For the human being, the dynamic motion is related more about the mechanics. Doing analysis in the physics world is more meaningful. Physical engine development: A physics engine is basically like a black box; it uses a set of mathematical expressions to simulate physics. Data drives the physics engine to develop more quickly. There are two compelling advantages to use a physics

362

R. Ma et al.

engine in the simulation. The first one is time saving and the second one is quality of rendering. A physical engine provides us with the ability to have interactive effects in believable ways. Using it we could create a more realistic virtual world. Software Design: Bullet has been designed to be customizable and modular. The main components are organized as follows:

Fig. 2. Bullet Engine Software Design Structure [10]

Rigid Body Physics Pipeline: Before going into details, the Fig. 2 shows the most important data structures and computation stages in the Bullet physics pipeline. This pipeline is executed from left to right, starting by applying gravity, and ending by position integration, updating the world transform. The entire physics pipeline computation and its data structures are represented in Bullet by a dynamics world. When performing ‘stepSimulation’ on the dynamics world, all the above stages are executed. The default dynamics world implementation is the ‘btDiscreteDynamicsWorld’. Finally, based on the framework of Bullet, we created a virtual environment with physic and robotic properties. According to the digital human structure, we create our mannequin in the environment.

Worker’s motion Goemetric data Motion capture system Physical engine

Digital human model

Motion simulation in physical virtual environment

Dynamic data Muscle fatigue model

Ergonomic analysis

Fig. 3. Simulation Framework

A Framework of Motion Capture System Based Human Behaviours Simulation

363

4 Framework of Simulation The framework presented in Fig. 3 illustrating the workflow of the software system which was consisted by mainly four parts: The first part is using motion capture system to get the trajectory information of human body movements. The second part is using digital human model to get related geometric data and dynamic information data. The forth part is using physical engine to create a physical virtual environment and create the virtual human in the environment. The forth part is immerge the dynamic muscle fatigue model when the human doing the movement in order to do the ergonomic analysis. We use the physical engine to create the virtual physical environment and to create the virtual human in this environment. We have read the data obtained from the motion capture system and put it in a virtual human in the virtual physical environment. We had set up the dynamic muscle fatigue model, and immerge it in the virtual human is the next step work.

5 Case Study Considering the complexity of the whole body movement, at the very beginning we set the posture of an arm in moving an object as the research target. Arm geometric model: We put three groups of markers on the arm articulation position (the shoulder, the elbow and the wrist). In order to reduce the complexity and avoid the redundant situation, we suppose in the shoulder there is two degree of freedom, and in the elbow there is one degree of freedom and the wrist is fixed. The geometric model is as below: Table. 1. Geometry Parameters of Arm [11] j 1 2 3 4

σ(j) 0 0 0 0

α (j) π/2 - π /2 0 0

D(j) 0 0 D3 D4

θ(j) θ1 θ2 θ3 0

R(j) 0 0 0 0

D3 and D4 represent the length of upper and lower arm. Using the trajectory which we get from motion capture system and with the inverse kinematic model, we could calculate the articulation angle θ1, θ2, θ3. Ergonomic analysis---muscle fatigue model: Ergonomic analysis and assessment can be evaluated from different aspects according to the task requirement. At first, most of the ergonomic analysis is based on the scoring method. With the development of the ergonomic field, there come out some model to describe the working situations. Currently, we pay attention to the muscle fatigue effectors in ergonomic analysis. Recently, we have defined a muscle fatigue model [12] from the macroscopic point of view and using simple parameters to define the muscle fatigue trend when time elapses. We want to add the fatigue model in the human moving procedure. Below is the figure to illustrate the muscle fatigue trend with time goes by.

364

R. Ma et al.

Fig. 4. Muscle Fatigue Trend with Time [12]

6 Conclusions and Perspectives In this paper, we proposed a framework for human behavior simulation. We use physical engine to set up the virtual environment and the purpose of this framework is to enhance the ergonomic analysis. Ergonomic analysis is important challenge but we have concentrate our research on muscle fatigue model to use this model into the framework for trajectory planning is our final purpose. This is only a preliminary result. Thus, there still a many research to do in the ergonomic module and visualization module. Later we will take consideration of path planning and collision avoiding based on a multi-agent optimization approach in the physical world.

References [1] Karwowski, W., Genaidy, A., Asfour, S.: Computer-aided ergonomics. Taylor & Francis, Abington (1990) [2] Badler, N., Phillips, G.B., Webber, B.L.: Simulating humans: computer graphics animation and control. Oxford University Press, USA (1993) [3] Schaub, K., Landau, K., Menges, R., Grossmann, K.: A computer-aided tool for ergonomic workplace design and preventive health care. Human Factors and Ergonomics in Manufacuring 7(4), 269–304 (1997) [4] Chaffin, D.: Development of computerized human static strength simulation model for job design. Human Factors and Ergonomics in Manufacturing 7(4), 305–322 (1997) [5] Eberly, D.H.: Game Physics. Elsevier, Inc., Amsterdam (2004) [6] VSR Research Group: Technical report for project virtual soldier research. Tech. rep., Center for Computer-Aided Design, The University of IOWA (2004) [7] Foxlin, E., Inc, I.: Motion Tracking Requirements and Technologies. In: Handbook of Virtual Environments: Design, Implementation, and Applications, pp. 163–210 (2002) [8] http://www.naturepoint.com [9] http://www.en.wikipedia.org/wiki/Physics_engine [10] http://www.bulletphysics.com [11] Denavit, J., Hartenberg, R.S.: A kinematic notation for lower pair mechanism based on matrics. Trans. of ASME, Journal of Applied Mechanics 22, 215–221 (1955) [12] Ma, L., Chablat, D., Bennis, F., Zhang, W.: A new simple dynamic muscle fatigue model and its validation. International Journal of Industrial Ergonomics 39, 211–220 (2009)

Visual Perception Model for Sense of Materials Wenhao Wang and Toshikazu Kato Faculty of Science and Engineering, Chuo University 1-13-27 Kasuga, Benkyo-ku, Tokyo, 112-8551 Japan [email protected], [email protected]

1 Introduction Natural surfaces has a different materials, human can instantly feel these characteristics. For example, display of fruit on the supermarket shelves, we can determine only the eyes whether they are fresh. When shopping, we can quickly determine the quality of material goods, as well as light and heavy, soft and hard, and so on. We call this feeling “SITUKAN”, it is Japanese and means sense of materials. In a few years ago, NTT Communications what the Japanese companies did research on sense of materials. They published the results in the Nature [1], and said the brightness is an important reason of sense of materials. They used global features depicts the glossiness and semi-transparent of success. However, after our research found that, classify the glossiness and semi-transparent, local features more efficient than global features.

2 Experiment 2.1 Methods Local and Global Features Brightness histograms are widely used as global graphical features which show the distribution of brightness in image plane. Nevertheless, in visual perception process, photoreceptors extract not only global features, but local features such as brightness, luminance, edges and contrasts. By integrating and selecting global and local features extracted by photoreceptors, human beings perceive textures and shapes. Therefore, we should design local features to describe relations in neighboring pixels as well as global features. Although we can measure the relations of reference pixel γ on image plane P and R , we think relations in neighboring pixels properly N pixels around r a , … , a show local features. Therefore, we adopted 3×3 pixel window to compute local features of directions and curvatures of differentials. Fig. 1 shows the 28 local mask patters: “+” represents the reference pixel r and “*”represents displacement vectors R i 1, … ,28 . a ,a Even if capturing the same texture, the camera angle and the lightning conditions are not same. In addition, images in database are not always noiseless. If graphical features are sensitive to camera angle and noises on images, we can not apply them to C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 365–368, 2011. © Springer-Verlag Berlin Heidelberg 2011

366

W. Wang and T. Kato

Fig. 1. Local mask patterns for graphical features

similarity image retrieval. Therefore, we should design shift-invariant and noiserobust graphical features. When photoreceptors receive stimulus, they send a signal to higher level of visual perception process. A receptor receiving stimulus also sends signals to neighboring photoreceptors, telling them to turn down their own sensitivity to stimulus. This antagonistic interaction between a photoreceptor and its surround is often called lateral inhibition. Lateral inhibition is a mechanism which enhances contrast between stimuli. We defined tri-contrast as a mathematical model of lateral inhibition function by

(

Cont (i ) a1(i ) , a (2i ) , r =

{f (r + a

(i ) 1

)

} {

} (i = 1,...,28)

) − f (r) + f (r + a (2i ) ) − f (r)

f (r + a 1(i ) ) + f (r + a (2i ) ) + 2 f (r)

(1)

R are the displacement vectors (each “*” in Fig. 1 denotes a and where a and a a ), function r is stimulus defined on independent three-dimensional color space, and r P is the image coordinate vector (each“+” in Fig. 1 denotes r). A denominator and a numerator of Eq. (1) denote stimulus intensity and difference of stimulus, respectively. Since stimulus intensity will become stronger if stimuli of pixels around reference pixel r a , r a become strong, photoreceptor’s response a ,a ,r will become weaker. In addition, since tri-contrast value is normalized by stimulus intensity, it is scale-invariant to stimulus intensity and robust to noise. Lateral inhibition exists not only in photoreceptors but in various ganglion of higher level of visual perception process. Therefore, we think that lateral inhibition is useful to select and emphasis dominant factors extracted by photoreceptors. Grayscale Photo In our experiment, use the same size and shape of objects as the subject. One is painted with a gloss coating over the object, and the other is not treated semitranslucent objects. We were on the two objects can turn the disc. Whenever the rotary disk 2 degrees to take a photo with grayscale mode camera (Fig. 2). Then, we

Visual Perception Model for Sense of Materials

367

shot a total of 360 photos were divided into 2 groups. Group A is glossiness and Group B is semi-transparent sense. Finally, we will remove all the photos in the background, because this background has nothing to do with the sense of materials. 2

3

4

5

6

7

8

9

11

10

12

Fig. 2. 1-6 photos is part of Group A, 7-12 is part of Group B

Procedure First, we took out all the photos from the local and global features (Fig. 3), and put the data normalization. Next, we arbitrarily removed from each group (Group A,B) in 㻡㻜㻜 㻠㻡㻜 㻠㻜㻜 㻟㻡㻜 㻟㻜㻜 㻞㻡㻜 㻞㻜㻜 㻝㻡㻜 㻝㻜㻜 㻡㻜 㻜

㻠㻡㻜

O

㻠㻜㻜

P

㻟㻡㻜 㻟㻜㻜

㻝㻡㻜

㻞㻜㻜

㻝㻜㻜

M

㻟㻡㻜

㻟㻜㻜

㻟㻜㻜

㻞㻜㻜

㻞㻜㻜

㻝㻡㻜

㻝㻡㻜

㻝㻜㻜

N

㻟㻡㻜

㻣㻣 㻝㻡㻟 㻞㻞㻥 㻟㻜㻡 㻟㻤㻝 㻠㻡㻣 㻡㻟㻟 㻢㻜㻥 㻢㻤㻡 㻣㻢㻝 㻤㻟㻣 㻥㻝㻟 㻥㻤㻥

C

㻝

㻣㻢 㻝㻡㻝 㻞㻞㻢 㻟㻜㻝 㻟㻣㻢 㻠㻡㻝 㻡㻞㻢 㻢㻜㻝 㻢㻣㻢 㻣㻡㻝 㻤㻞㻢 㻥㻜㻝 㻥㻣㻢

㻝㻞㻜㻜

D

㻝㻜㻜㻜 㻤㻜㻜 㻢㻜㻜 㻠㻜㻜 㻞㻜㻜 㻜

㻜

㻜

㻝

㻟㻜㻜

㻝㻜㻜

㻡㻜

㻜

㻠㻜㻜

㻞㻜㻜

㻝㻜㻜

㻡㻜

㻝 㻝㻣 㻟㻟 㻠㻥 㻢㻡 㻤㻝 㻥㻣 㻝㻝㻟㻝㻞㻥㻝㻠㻡㻝㻢㻝㻝㻣㻣㻝㻥㻟㻞㻜㻥㻞㻞㻡㻞㻠㻝

㻝 㻝㻣 㻟㻟 㻠㻥 㻢㻡 㻤㻝 㻥㻣 㻝㻝㻟㻝㻞㻥㻝㻠㻡㻝㻢㻝㻝㻣㻣㻝㻥㻟㻞㻜㻥㻞㻞㻡㻞㻠㻝

㻤㻝 㻝㻢㻝 㻞㻠㻝 㻟㻞㻝 㻠㻜㻝 㻠㻤㻝 㻡㻢㻝 㻢㻠㻝 㻣㻞㻝 㻤㻜㻝 㻤㻤㻝 㻥㻢㻝

㻡㻜㻜

㻠㻜㻜

㻞㻡㻜

㻜

㻜

㻠㻡㻜

㻞㻡㻜

㻝㻜㻜

㻡㻜 㻝

㻤㻝 㻝㻢㻝 㻞㻠㻝 㻟㻞㻝 㻠㻜㻝 㻠㻤㻝 㻡㻢㻝 㻢㻠㻝 㻣㻞㻝 㻤㻜㻝 㻤㻤㻝 㻥㻢㻝

B

㻠㻜㻜 㻟㻜㻜

㻡㻜㻜

㻠㻜㻜

㻡㻜㻜

㻝㻡㻜

㻜

㻠㻡㻜

A

㻞㻜㻜

㻡㻜

㻡㻜㻜

㻞㻡㻜

㻞㻡㻜 㻞㻜㻜 㻝㻜㻜

㻝

㻟㻡㻜 㻟㻜㻜

㻝 㻝㻣 㻟㻟 㻠㻥 㻢㻡 㻤㻝 㻥㻣 㻝㻝㻟㻝㻞㻥㻝㻠㻡㻝㻢㻝㻝㻣㻣㻝㻥㻟㻞㻜㻥㻞㻞㻡㻞㻠㻝

㻝 㻝㻣 㻟㻟 㻠㻥 㻢㻡 㻤㻝 㻥㻣 㻝㻝㻟㻝㻞㻥㻝㻠㻡㻝㻢㻝㻝㻣㻣㻝㻥㻟㻞㻜㻥㻞㻞㻡㻞㻠㻝

Fig. 3. Local(O,P,M,N) and Global(A,B,C,D) features of brightness histograms

368

W. Wang and T. Kato

30, 60, 120 photos and put them into 3 samples as teaching data. Finally, let the computer after learning of the two features to classify all photos. Results Table. 1 shows that, with the reduction in the amount of teaching data, global features gradual decline in accuracy, but the correct rate of local features is not much change. Especially in the teaching data for the 30, the difference between the correct rate of 20%. And both are used, the correct rate has remained at 100%. This data clearly shows that classify the glossiness and semi-transparent, local features more efficient than global features. Table 1. Teaching data classification accuracy

120 photos

60 photos

30 photos

1. Local features 2. Global features

98% 89%

99% 80%

98% 78%

1+2

100%

100%

100%

References [1] Motoyoshi, I., Nishida, S., Sharan, L., Aselson, E.H.: Image statistics and the perception of surface qualities. Nature 447 (2007) [2] Oguma, M., Tsuboi, M., Horikoshi, T., Takaki, Y.: Subjective Evaluation of Appearance Reproduction of Moving 3D Images Produced by High-Density Directional Display. ITE 33(16), 1–4 (2009) [3] Spillmann, L., Werner, J.S.: Visual Perception. Academic Press, London (1990)

The Effects of Stereoscopic Display Luminance and Ambient Illumination on Visual Comfort Pei-Chia Wang1, Sheue-Ling Hwang1, Kuan-Yu Chen1, Jinn-Sen Chen2, Jinn-Cherng Yang2, and Hung-Lu Chang2 1

National Tsing Hua University, Industrial Engineering and Engineering Management Department, No. 101, Section 2, Kuang-Fu Road, Hsinchu, Taiwan 2 Industrial Technology Research Institute, Electronics and Optoelectronics Research Laboratories, No.195, Section 4, Zhongxing Road, Zhudong Township, Hsinchu County, Taiwan [email protected], [email protected], [email protected], {JIINSEN,OESJCY,hungluchang}@itri.org.tw

Abstract. In this paper, we investigated that display luminance and ambient illumination had significant effects on the perception of the imagery displayed, and their appropriate combination would be studied. The results showed display luminance and ambient illumination did not significantly influence on physiological fatigue. Ambient illumination statistically affected objective visual fatigue, but display luminance did not. Ambient illumination and display luminance significantly affected subjective comfort evaluation. However, the main effect of display luminance was statistically significant on visual discrimination performance, but ambient illumination was not significant. As for the combination effect of display luminance and ambient illumination, viewers felt the most comfortable and with the best visual performance in the high level of display luminance and the high level of ambient illumination. It is expected to find out the optimal condition to ensure the viewer’s visual comfort. As for the future work, the dynamic image factor may be taken into account. Keywords: display luminance, ambient illumination, visual comfort, fatigue, visual discrimination performance.

1 Introduction 1.1 Stereoscopic Vision The difference of human eyes’ horizontal distance induces the tiny discrepancy of image for seeing the same object, and this is the reason of the depth perception written in Euclid’s manuscript [1]. It is also called “binocular disparity” in psychology. In 1838, Charles Wheatstone created the first stereo image via two separate mirrors with an angle. He found that disparity was a sufficient and compelling stimulus for the perception of depth [2]. From now on, 3D displays have C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 369–373, 2011. © Springer-Verlag Berlin Heidelberg 2011

370

P.-C. Wang et al.

moved from a series of applications and become the mainstream as the development of the display technology. 1.2 The Relation of Display Luminance and Ambient Illumination The inadequate of the display luminance or the ambient illumination or both two derives to the accommodation of the brightness level to the pupil and thus induce the visual fatigue and to decrease the image quality. Several researches have studied the effect of ambient illumination in watching displays [3], [4], [5]. Visual comfort on CRT (Cathode Ray Tube) monitor was significantly decreased by enhancing the level from 300 Lux to more than 600 Lux of the ambient illumination [3]. Besides, the higher level of ambient illumination caused the display images to fade [4]. When subjects watched videos on an HDR (High Dynamic Range) display under the ambient illumination of 0.75 Lux, 8.5 Lux, 28 Lux, and 74 Lux, the visual comfort degree was the same. The viewer did not get visual fatigue under these four levels of environment illumination [5]. The contrast ratio decreases with increasing ambient illumination because the surface reflection makes the screen brighter [4]. As a result, the brightness of the display is also a factor of the visual fatigue.

2 Method 2.1 Subjects The subjects were 15 experts who are engineers of 3D display division in Industrial Technology Research Institute, and 9 novices who were graduate students in National Tsing Hua University. All subjects had a normal visual acuity, were neither color blind nor stereo blind. Only the subjects who passed the test of basic visual acuity, stereoscopic ability, and color blindness could participate in the formal experiment. 2.2 Stimulus Two kinds of materials were used in this study. They included 15 computer generated images which were created by 3ds MAX, and 15 photographic images which were taken by 3D camera. There were 10 experimental materials, which randomly displayed computer generated and photographic images, in the same treatment. Each experimental material contained two pictures with the same scene but included 10 differences between them. 2.3 Design of Experiment The experiment was a 3×2 within-subject design. One half of subjects participated the experiment in the low level of ambient illumination, and the others were in the high level. All subjects took part in the experiment under three levels of display luminance one by one.

The Effects of Stereoscopic Display Luminance and Ambient Illumination

371

The independent variables were illustrated as follows. 1. Display luminance: there were 15, 35, and 55 of the scale of stereoscopic display itself, which was 0.38 nits, 6.73 nits, and 47.3 nits respectively. 2. Ambient illumination: there were one yellow light bulb and seven yellow light bulbs, which were 55 Lux and 300 Lux respectively. There were four dependent variables. 1. Objective visual fatigue index: CFF (Critical Fusion Frequency). 2. Objective physiological fatigue index: HRV (Heart Rate Variability). 3. Subjective comfort evaluation: Likert’s five scale on the degree of “eyestrain,” “seeing the object clearly,” “psychological fatigue,” and “physical fatigue.” 4. Visual discrimination performance: the total sum of the differences of 10 experimental pictures in each treatment for subjects to find. 2.4 Procedure At first, the subjects read the purpose of the experiment and filled out the consent agreement form. Their HRV were measured by ANS Watch as the baseline in the beginning of experiment. There were three levels of display luminance. Each treatment took 20 minutes showing 10 experimental pictures with two minutes per experimental material on the stereoscopic display. Before and after watching 3D pictures in the same level of display luminance, the subjects took the objective visual fatigue by handy flicker. After each treatment, the HRV value was measured and subjective comfort evaluation was assessed. The picture and the level of display luminance and ambient illumination were randomly shown on the 3D display to eliminate any order effects. The duration of the experiment for each subject was about two hours.

3 Result 3.1 Objective Visual and Physiological Fatigues A significant main effect of ambient illumination on CFF, Z = -7.978, p-value < 0.001, was found based on Wald-Wolfowitz Test. Nevertheless, the main effect of display luminance was not significant. There was no significant interaction effect between display luminance and ambient illumination on objective physiological fatigue namely, HRV were found by MANOVA (Multiple Analysis of Variance). Moreover, no main effect of display luminance or ambient illumination was significant on HRV.

－

－

3.2 Subjective Comfort Evaluation The significant main effect of ambient illumination on “eyestrain”, Z = -7.978, pvalue < 0.001, “seeing the object clearly,” Z = -7.750, p-value < 0.001, “psychological fatigue,” Z = -7.978, p-value < 0.001, and “physical fatigue,” Z = -

372

P.-C. Wang et al.

7.978, p-value < 0.001, was found based on Wald-Wolfowitz Test. Moreover, the effect of display luminance was also significant on “eyestrain”, chi-square (2) = 21.971, p-value < 0.001, “seeing the object clearly,” chi-square (2) = 32.747, p-value < 0.001, and “physiological fatigue,” chi-square (2) = 8.379, p-value < 0.001. 3.3 Visual Discrimination Performance The results of ANOVA did not yield a statistically significant interaction between ambient illumination and display luminance on visual discrimination performance. A significant main effect of display luminance was found, F = 15.121, p-value < 0.001. However, no evidence was found indicating that the main effect of ambient illumination was significant.

4 Discussion 4.1 The Effect of Display Luminance and Ambient Illumination The two-way interaction was not statistically significant on objective physiological fatigue (HRV) and visual discrimination performance. The effect of display luminance was significant on subjective comfort evaluations, which were eyestrain, seeing the object clearly, and psychological fatigue, and on visual discrimination performance. In addition, the effect of ambient illumination affected objective visual fatigue (CFF) and subjective comfort evaluations, which were eyestrain, seeing the object clearly, psychological fatigue, and physical fatigue. 4.2 The Best Combination Level of Display Luminance and Ambient Illumination To consider the significant dependent variable of objective visual fatigue and subjective comfort evaluation, the average value of these variables was ranked by the degree of comfort. The highest sum number meant the best combination level of display luminance and ambient illumination. For watching 3D static pictures, viewers felt the most comfortable in the highest level of display luminance (34.2 nits) and the highest level of ambient illumination (300 Lux). Moreover, the combination level of the lowest display luminance (0.38 nits) and the lowest ambient illumination (55 Lux) caused the most fatigue.

5 Conclusion The result of this experiment did not show the significant physiological fatigue (HRV) in watching 3D static pictures for 20 minutes, but indicated significant objective visual fatigue (CFF) and subjective comfort evaluation. As a result, the duration of watching static pictures did not induce the discomfort feeling. The effect of display luminance was significant on subjective comfort evaluation and visual discrimination performance. Moreover, ambient illumination affected

The Effects of Stereoscopic Display Luminance and Ambient Illumination

373

objective visual fatigue and subjective comfort evaluation. The combination of higher level of ambient illumination and display luminance was better for the viewer in watching 3D pictures. Finally, the consequence of this study demonstrated that ambient illumination was the vital factor affecting the visual comfort. The dynamic images factor may be taken into account in the further study. Acknowledgement. The work is supported by funding from Industrial Technology Research Institute (no.:9351AA4110) and National Science Council (no.:NSC992221-E-007-085-MY2).

References 1. Smith, M.A.: Ptolemy and the Foundations of Ancient Mathematical Optics: A Source Based Guided Study, p. 57. American Philosophical Society, Philadelphia (1999) 2. Wheatstone, C.: Contributions to the Physiology of Vision: 1, On Some Remarkable and Hitherto Unobserved Phenomena of Binocular Vision. Philosophical Transactions of the Royal Society of London, 128–371 (1838) 3. Arne, A., Gunnar, H., Hans-Henrik, O.B., Magne, R.T.: Musculoskeletal, Visual and Psychosocial Stress in VDU Operators Before and After Multidisciplinary Ergonomic interventions. Applied Ergonomics 29, 335–354 (1998) 4. Len, C., Huang, K.: Effects of Ambient Illumination and Screen Luminance Combination on Character Identification Performance of Desktop TFT-LCD Monitors. International Journal of Industrial Ergonomics 36, 211–218 (2006) 5. Remple, A.G., Heidrich, W., Li, H., Mantiuk, R.: Video Viewing Preferences for HDR Displays Under Varying Ambient Illumination. In: Proceedings of the 6th Symposium on Applied Perception in Graphics and Visualization, pp. 45–52 (2009)

Preferred Setting of Keyboard and Mouse for Using a Supine Computer Workstation Hsin-Chieh Wu* and Ho-Rong Chu Department of Industrial Engineering and Management, Chaoyang University of Technology, No.168, Jifong E. Rd, Wufong District, Taichung, 41349, Taiwan, R.O.C. [email protected]

Abstract. The purpose of this study was to investigate the differences in performance and muscle effort between guideline and preferred setting of keyboard and mouse when using a computer in supine posture. The guideline setting is according to the rules of standard sitting posture; and the preferred setting is freely determined by the subjects. Ten healthy adults and ten people with disabilities of the lower extremities participated in this study. The experimental tasks included typing and mouse dragging tasks. The task performances and electromyography (EMG) were collected. The participants had significantly higher performances and lower muscle effort when using the mouse under preferred setting, as compared with the guideline setting. However, little difference in typing performance was found between the guideline and preferred settings. The results of this study are helpful to designing a supine computer workstation. Keywords: Universal design, EMG, typing posture, mouse operation.

1 Introduction The computer plays an important role in modern life, commonly used in daily life, both personal and professional use, and all industries rely on computer technology to enhance productivity and efficiency. General computer users have no ergonomic knowledge of computers, as frequency and duration of use increases, users feel related muscular and skeletal problems such as sore, painful and stiff muscles in the neck/shoulder, lumbar, and upper extremities, such parts have higher risk of muscular and skeletal discomfort by computer operation [1], and these problems develop slowly along with computer use, there are two major causes, the first is incorrect sitting postures, the second is too long a duration of continuous use of a computer. Most general computer operations consist of mouse, keyboard, and monitor watching, and work is largely repeated, especially for document processing, data entry, and researchers, furthermore, computer operation is done in a static sitting position, after prolonged periods, muscular and skeleton discomfort easily occurs, some literatures have pointed out that, dedicated personnel, performing computer related operations, had much more frequent muscular and skeleton soreness, pain and complaints [2]. *

Correspponding author.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 374–378, 2011. © Springer-Verlag Berlin Heidelberg 2011

Preferred Setting of Keyboard and Mouse for Using a Supine Computer Workstation

375

With technology advances, the population began to notice product health and safety importance, hence many computer-related workstations and aid equipments are developed to assist computer users and reduce discomfort of muscle and bones, such as the ergonomic mouse, ergonomic chair, ergonomic workstations, and arm support apparatus. These equipments have only limited effects of alleviating muscular and skeletal discomfort, and offer temporary convenience or comfort to users because current computer operations are static and in prolonged sitting positions, giving rise to muscular and skeletal discomfort. Modern employment of most people means prolonged operation of a computer in a sitting position, thus, fatigue easily occurs, and in the home, when changing posture to lying down, and using a computer for entertainment, chatting, or typing, while at ease, not only can release psychological stress, but also alleviate muscular and skeletal burdens. Lower-extremity disadvantaged population often feel very uncomfortable, sore muscles from the daily use of a computer, those not adapted to prolonged sitting [3], experience great discomfort related to computer operations and body posture. Therefore, this study was dedicated to understanding how to set up a keyboard and a mouse when using a computer in supine posture. It is hoped that people could lie in bed and operate a computer, feeling comfortable, convenience of operation, and alleviating muscular and skeletal discomfort [4]. The goal of this study was to study and compare the effect of keyboard and mouse setting during computer work with a supine posture. More specifically, the study was designed to test the following hypotheses during copy typing and mouse dragging tasks: (a) Preferred setting will result in better performance and lower muscle effort compared with the guideline setting during copy typing task. (b) Preferred setting will result in better performance and lower muscle effort compared with the guideline setting during mouse dragging task.

2 Methods Ten healthy adults and ten people with disabilities of the lower extremities participated in this study. All participants used the mouse with their right hand. The healthy participants use the computer for over ten hours per week. All the handicapped participants had experience of computer use. The experimental tasks included copy typing, mouse dragging and mouse stability operations. Participants completed a 20min copy typing task in each of two conditions: (a) guideline setting of keyboard and mouse; (b) preferred setting of keyboard and mouse. Testing order was randomized. Participants were given a 15min break between the two conditions. Immediately following the typing task, the mouse dragging task was performed. The mouse dragging task required the participant to move the mouse quickly following a tortuous path presented on the screen (Fig 1). During standardized computer work, the work performance in terms of typing score, typing speed, and mouse task completion time were registered. In the guideline setting condition, the keyboard and mouse were arranged to let the participant’ elbow bend at 90 degree, as shown in Fig. 2. In the preferred setting condition, an adjustable keyboard tray and mouse tray were used to be adjusted freely by the participant, as shown in Fig. 3.

376

H.-C. Wu and H.-R. Chu

Fig. 1. The illustration of screen image in the mouse task

Fig. 2. The guideline setting

Fig. 3. Two examples of preferred setting

Electromyographic (EMG) activity of the sternocleidomastoid and extensor carpi ulnaris muscle was recorded via a Biopac System MP150. Silver-silver chloride surface electrodes were positioned over the horizontal fibers of these four muscles. All electrodes remained in situ throughout and between the test periods. Root-meansquare(RMS) values of raw EMG signals were calculated for each of the recorded epochs. Mean RMS values were calculated for each task for each participant. These mean values were represented as a proportion of the maximum voluntary contraction (%MVC). One-way ANOVA was applied to examine the main effect. The level p < 0.05 was considered significant.

3 Results 3.1 Work Performances between the Two Settings The typing score and speed were similar between guideline and preferred settings, as shown in Table 1. The mouse task completion time was significantly shorter with preferred setting compared with guideline setting. However, no significant differences in typing score and typing speed were found between guideline and preferred settings.

Preferred Setting of Keyboard and Mouse for Using a Supine Computer Workstation

377

Table 1. Mean and standard deviation of operation performances Typing score

Typing speed Mouse task completion time (wpm) (sec) Setting Mean SD Mean SD Mean SD Guideline 691.8 289.4 37.1 14.9 459.1 150.7 Preferred 695.6 316.5 37.2 15.8 368.9* 178.0 *The mean value is significantly different from that in the guideline setting (p<0.05)

3.2 EMG Activity between the Two Settings Surface EMG %MVC values for Sternocleidomastoid and Extensor carpi ulnaris muscles under different setting conditions are listed in Table 2. For the copy typing task, the mean %MVC value of the sternocleidomastoid muscle was significantly greater under guideline setting compared with preferred setting; however, no significant differences in extensor carpi ulnaris muscle activity were found between these two settings. For the mouse dragging task, the mean %MVC value of the extensor carpi ulnaris muscle EMG was significantly greater under guideline setting compared with preferred setting; however, no significant differences in %MVC of sternocleidomastoid muscle EMG were found between these two settings. Overall the preferred setting of keyboard and mouse could lead to lower effort in some muscle no matter during typing or mouse dragging tasks. Table 2. Mean and standard deviation of %MVC values for the measured two muscles

Sternocleidomastoid muscle EMG

Extensor carpi ulnaris muscle EMG

(%MVC) (%MVC) Experimental conditions Mean SD Mean SD Copy typing task Guideline setting 10.65 11.07 18.79 12.46 Preferred setting 7.84* 10.11 18.68 12.23 Mouse dragging task Guideline setting 3.46 2.79 24.83 15.39 3.77 3.28 16.42* 7.81 Preferred setting *The mean value is significantly different from that in the guideline setting (p<0.05)

4 Discussion In the current study, computer work tasks were performed with a supine workstation either under the guideline setting or under personal preferred setting of keyboard and mouse. Interestingly, all of the participants preferred to put the mouse alongside their right hip. Personal preferred setting could lead to better performance during mouse dragging task, but had no significantly better performance during typing task compared with the guideline setting. This result means that the traditional guideline setting for sitting workstation is not suitable for using a supine workstation, especially dragging a mouse. In addition, the extensor carpi ulnaris muscle EMG results also

378

H.-C. Wu and H.-R. Chu

confirmed that preferred setting could lead to obviously lower muscle effort (16.42% MVC) during mouse dragging task than that of guideline setting (24.83% MVC). During the copy typing task, the guideline setting led to a significant increase in the neck EMG activity (10.65% MVC) as compared to the preferred setting (7.84% MVC). This result also suggests that provision of adjustable keyboard and mouse trays may be helpful to the people who require using a computer with a supine posture. Because this present study evaluated work performance and muscle effort for using a computer with only one supine posture, further research should be conducted to evaluate the effect of lying posture on performance and muscle effort. Acknowledgments. The authors thank the National Science Council of the Republic of China (Contract No. NSC99-2628-E-324-024) for financially supporting this research.

References 1. Wu, H.C., Lin, C.Y., Chen, H.C., Wang, C.H.: Relations between ergonomic risk factors and musculoskeletal disorders in computer operation. Journal of Ergonomic Study 9, 1–10 (2007) (in Chinese) 2. Wu, H.C., Chen, T.: The development of an electronic tool for diagnosing risk factors in computer operation. WSEAS Transactions on Computers 5, 1013–1019 (2006) 3. Haynes, S., Williams, K.: Product review of alternative computer workstations as possible workplace accommodations for people with chronic low back pain. Technology and Disability 19, 41–52 (2007) 4. Haynes, S., Williams, K.: Impact of seating posture on user comfort and typing performance for people with chronic low back pain. International Journal of Industrial Ergonomics 38, 35–46 (2008)

Part VII

Health and Wellbeing

An Interactive Multimedia System for Monitoring the Progressive Decline of Memory in Alzheimer's Patients Hala Al-Muhanna, Rawan Al-Wabil, Hailah Al-Mazrua, Noura Al-Fadhel, and Areej Al-Wabil Information Technology Department, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia {Halmohana,Ral-wabel1,Hal-mazrou1,Nalfadhel}@student.ksu.edu.sa, [email protected]

Abstract. This paper describes an assistive technology designed for longitudinal monitoring of memory decline for people with Alzheimer’s Disease (AD). While there are systems designed for rehabilitation of people with AD, supporting caregivers, and psychosocial intervention, there is a lack in technology support that provides quantitative measures of progressive memory decline that can assist physicians in clinical settings. An interactive autobiographical memory repository of images and sound recordings was developed to facilitate measuring memory recall and recognition. The system functionality and the user-centered design approach of involving geriatric psychiatry specialists and caregivers are described. Keywords: Alzheimer's Disease, Memory Decline, Caregivers, Elders, User-Centered Design, Caregiver Burden.

1 Introduction Accelerated cognitive decline and memory loss is a concern for rapidly aging populations. One of the most common reasons of memory loss is Alzheimer's Disease (AD), a neurological disorder which is an irreversible progressive form of Dementia affecting memory, cognition, and behavior [2]. Prevalence is growing worldwide affecting many facets of life. In Saudi Arabia alone, 50000 people were estimated to have been diagnosed with AD [9]. AD is also considered to be the sixth main cause of death in the United States [10]. An equal number of caregivers are affected who support People with AD (PwAD) with their physical needs of aging and episodic memory impairments. In 2008, there was an estimate for AD that 9.9 million caregivers provide 8.5 billion hours of care at a cost of $94 billion in the US [1]. Caregivers are often a spouse or a close relative of PwAD, and are considered the second or hidden victim of AD. Caregivers’ burden lies in the challenge of assisting PwAD physically and mentally, while at the same time trying to keeping track of progressive decline in cognitive abilities in interventional therapy. Recent research has suggested that earlier diagnosis facilitates earlier treatment of the disease, when medications have been shown to be most effective [8]. A challenge C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 381–385, 2011. © Springer-Verlag Berlin Heidelberg 2011

382

H. Al-Muhanna et al.

resides in monitoring memory loss at the onset of AD symptoms and for patients with Mild Cognitive Impairment (MCI). Recently, there has been increasing interest in the design and development of assistive technologies for PwAD. While this research serves the purpose of providing solutions for people who need them (e.g. individuals with AD, their caregivers, physicians), it also gives researchers opportunities to investigate and learn from people with AD whose requirements challenge the extent of designers’ and developers’ understandings [6]. Of particular interest to our current work are research studies examining technology solutions for people with AD that have been shown to be effective in facilitation social interaction and eliciting memory cues. Systems have been designed for early diagnosis, rehabilitation of AD patients [10], supporting caregivers [1][6], assisting them with their daily lives [4], and intervention [8]. Clinicians involved in the caring for people with AD often find it difficult to assess memory decline based on caregivers’ anecdotal evidence. This has led to the question of whether technology can assist physicians in accurately keeping track of accelerated memory decline. Zplay has been recently introduced for clinical and home use, which facilitates monitoring; however, longitudinal monitoring still relies on subjective or manual recording, which does not involve direct interaction between PwAD and the system [8]. The need for supporting physicians in monitoring decline of memory abilities signals opportunities for technology development. This has motivated us for developing "Monitoring Memory Streams", which is a software program aimed to help caregivers and physicians with that process. The inadequate understanding of the needs of users in this context has entailed adopting a UserCentered Design (UCD) approach for iterative needs assessments and evaluation of prototypes. Monitoring the accelerated decline in memory for patients with AD is a challenge in home settings. In clinical settings, physicians rely on neuropsychological measures. Longitudinal data is difficult to ascertain due to the challenge in eliciting accurate reports described by caregivers. A better understanding of how clinicians measure memory decline can help identify opportunities for technology to help this process and consequently reduce the burden on the caregivers. Reminiscence Therapy (RT) is a common psychosocial intervention for people with AD. RT involves eliciting memories with tangible objects such as photographs or artifacts. Evidence suggests that having PwAD engage in activities that are cognitively stimulating can slow down the rate of decline in cognitive abilities [1]. Multimedia that can be readily recognized by PwAD such as photos with faces of people familiar to PwAD or places of interest are often advocated in RT [8]. A better understanding of effective conversational prostheses and what visual and auditory stimuli are effective in triggering memories of past experiences can help improve the functionality of a monitoring system in assessing recognition rates over extended periods and consequently provide objective measures of decline. The aim of this project is to develop a system for monitoring the decline in memory for people with AD. The hypothesis is that a monitoring system used regularly by PwAD can help provide an accurate measure of cognitive abilities over long periods of time between clinical visits, decrease caregiver burden, by automatically keeping track of recognition rates of visual images and auditory cues. In this paper, we first present an overview of the system that we developed for monitoring memory decline of people with AD. Following that, we discuss our user-centered design approach in the design

An Interactive Multimedia System for Monitoring the Progressive Decline

383

and development of the system. Finally, we make some concluding remarks and discuss future work.

2 Monitoring Memory Streams: System Overview Our Monitoring Memory Streams system provides a technology solution for measuring memory decline for PwAD. The system includes customization features for caregivers to build a personalized photo and audio repository of four categories; namely people, places, events, and objects for the AD individual. Categories are based on psychological models of autobiographical memory [4], and have been applied in the context of assistive technologies for people with episodic memory impairment [1]. This is facilitated by populating the photo database with pictures of the AD individual’s relatives, friends, familiar places and surrounding objects, and the audio database with sound recordings of the people who are close and familiar to them. Recognition tests are designed by clinicians to measure the progressive decline of memory for PwAD with customized reporting. The system is intended to be used as a catalyst for facilitating interactions between caregivers and PwAD. Caregivers use the system with PwAD in recognition drills designed as activities similar to browsing photo albums, while the system keeps track of recognition rates of images and audio clips automatically over periods extending between clinic visits. The system generates reports of performance-tracking for enabling the specialists to monitor the memory decline or stability of PwAD. The system is implemented on a touch-pad portable device featuring direct manipulation interaction types with reduced complexity in which functionality is not hidden in menu structures. Evidence exists demonstrating that touch screens are effective interaction types for people with dementia [3]. The system reduces the burden on caregivers in long-term monitoring of PwAD by serving as a catalyst for facilitating a joint activity between caregivers and PwAD in relaxed home settings. Once an individual is diagnosed with AD, subjects are able to work on the interactive drills at home with caregivers. In clinic visits, memory decline can be examined in 3 to 6 month intervals depending on the stage of the AD. This system also facilitates examining the impact of medication on PwAD’s ability to respond or recognize people, places, objects or events in the multimedia drills as compared to other treatments. Systems that use personalized artifacts for PwAD have been shown to be promising in reminiscence therapy [4]; however, these have been criticized for burdening caregivers with collecting media. This issue has been addressed in our system by minimizing caregiver involvement to the initial setup stage. The system has a default test specifications, which can be customized by clinicians to match each PwAD’s needs. The tests' results are stored and customized reports can be generated for the specialists' use.

3 User Centered Design The system was developed with a UCD approach arising from the need expressed by physicians and caregivers of PwAD for a way of facilitating the monitoring process of accelerated memory decline for long periods between clinic visits. Physicians and

384

H. Al-Muhanna et al.

caregivers were involved as informants in the design and the development of the system, to provide insights into measures of cognitive decline to identify system functionality, and the needs of our targeted user population. Development of prototypes was carried out iteratively, with the input of a geriatric psychologist, neurologist, and caregivers throughout. Indirect involvement of PwAD was considered in early stages of the project due to the difficulties known to exist in eliciting requirements directly from PwAD [6]; however direct involvement of PwAd is planned for later stages of iterative prototyping and in the longitudinal trials of the monitoring system. In specifying the context of use for Monitoring Memory Streams, it was evident that our problem involves users who require specific design considerations that avoid complexity and over-stimulation and who have memory recollection issues, and is targeting specialists in the medical field. As designers and developers, a key challenge for us was to develop a system for people who have different life experience from our own. The main cycle of work was comprised of four activities. First, specifying the context of use in both clinic and home environments with three categories of users (i.e. specialists, PwAD and their caregivers). Second, specifying requirements involved examining software designed for RT and assistive technologies for PwAD as well as conducting requirements discovery sessions with specialists in fields such as geriatrics, psychiatry, and neurology. Caregivers of PwAD were involved as design informants to elicit insights into their cognitive and interaction needs. Third, producing design solutions involved iterative prototyping in two stages. First, the low fidelity prototype was designed using paper-based sketch formatted interface and was assessed by a neurologist. Second, a high-fidelity prototype comprised of an interactive application running on an iPad was examined by a geriatric psychiatrist. Cognitive walkthroughs of the design were conducted with the geriatric psychiatrist and refinement of measures and interaction design were revised accordingly.

Fig. 1. Prototypes used in requirements discovery sessions with physicians

4 Conclusion This project has presented a technology solution for healthcare practitioners for the purpose of monitoring memory decline of people with AD. The UCD approach

An Interactive Multimedia System for Monitoring the Progressive Decline

385

adopted in this project extends our understanding of how a monitoring system can be designed to meet the needs of clinicians and caregivers. At the stage of writing this paper, we have implemented the design and will evaluate it with our target user population in a longitudinal study. Acknowledgments. Our thanks to Noura Al-Zamel and Lubna Al-Hinti for their effort in the design and development of the system. We also thank Dr. Fahad AlWahhabi for his assistance in requirements gathering.

References 1. Alzheimer’s Association, 2008: 2009 Alzheimer’s Disease Facts and Figures, http://www.alz.org/national/documents/report_ alzfactsfigures2009.pdf 2. Alzheimer’s Association, “What is Alzheimer’s,” Alzheimer’s Association, http://www.alz.org/alzheimers_disease_what_is_alzheimers.asp 3. Astell, A., Ellis, M., Bernardi, L., Alm, N., Dye, R., Gowans, G., Campbell, J.: Using a Touch Screen Computer to Support Relationships Between People with Dementia and Caregivers. Interacting with Computers 22, 267–275 (2010) 4. Barsalou, L.W.: The Content and Organization of Autobiographical Memories. In: Remembering Reconsidered: Ecological and Traditional Approaches to the Study of Memory, pp. 193–243 (1988) 5. Cohene, T., Baecker, R., Marziali, E.: Designing Interactive Life Story Multimedia for a Family Affected by Alzheimer’s Disease: A Case Study. In: Proceedings of CHI 2005, pp. 1300–1303. ACM, New York (2005) 6. Hawkey, K., Inkpen, K., Rockwood, K., Mcallister, M., Slonim, J.: Requirements Gathering with Alzheimer’s Patients and Caregivers. In: Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility (Assets 2005), pp. 142–149. ). ACM, New York (2005) 7. Lee, M.L., Dey, A.K.: Providing Good Memory Cues for People with Episodic Memory Impairment. In: Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility (Assets 2007), pp. 131–138. ACM, New York (2007) 8. Makedon, F., Zhang, R., Alexandrakis, G., Owen, C., Huang, H., Saykin, A.: An Interactive User Interface System for Alzheimer’s Intervention. In: Makedon, F., Maglogiannis, I., Kapidakis, S. (eds.) Proceedings of the 3rd International Conference on PErvasive Technologies Related to Assistive Environments (PETRA 2010), article 35, 5 pages. ACM, New York (2010) 9. Mansour, T.: Alzheimer’s Disease (in Arabic). Alriyadh newspaper, http://www.alriyadh.com/2010/06/09/article533201.html 10. Gogia, P., Rastogi, N.: Clinical Alzheimer Rehabilitation, 1st edn. Springer, Heidelberg (2008) 11. Williams, T.: Alzheimer’s Disease Supersedes Diabetes as Sixth Leading Cause of Death in The United States, Alzheimer’s Association, http://www.alz.org/national/documents/release_061208_sixth_ final.pdf 12. Wilson, R.S., Aggrarwal, N.T., Branes, L.L., Cf, M.D.L., Le, H., Da, E.: Cognitive Decline in Incident Alzheimer Disease in a Community Population. Neurology 74, 951– 955 (2010)

Personal Smart Spaces for Diabetics Manal AlBahlal and Jalal AlMuhtadi College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia {AlBahlal,jalal}@ccis.edu.sa

Abstract. The Ad hoc pervasive computing provides an attractive vision for future computing and has a great influence on many fields that need to be smart with simple digital devices interacting and sharing services seamlessly and transparently. Healthcare is a key area that can benefit from smart digital spaces, especially extending services to out-of-hospital contexts. In this poster, we describe the design of a system, called Personal Smart Space (PSS), which provides an automated method for bootstrapping a personal space. Specifically, PSS will track a person's health and handle variations that may indicate a risk. The PSS is comprised of several services; namely, discovery management, event, description, and presentation. This poster describes the implementation and verification of this PSS for diabetic patients, which is comprised of 4 devices, 5 services and a coordinator. The PSS utilizes the UPnP protocol and XML standards to describe the devices and services to provide more flexibility. The novelty of this PSS lies in how the coordinator provides an interface to components (GPS, Camera, Glucose sensor, and mobile communication devices) and integrates a notification system as well as finding a backup device in cases of faults in one of the PSS components. Keywords: PSS, UPnP, XML, diabetic, smart devices, service components.

1 Introduction Today, Ad hoc pervasive computing provides an attractive vision for future computing and a great influence on education, health, entertainment and daily life. It forms a smart space, where digital and physical devices interact and share services seamlessly and transparently. Managing this smart space involves bootstrapping it, discovering devices in the surrounding area, adding, monitoring, and removing devices to from the cluster as needed. One of the most attractive fields, where procedures and tasks need to happen seamlessly and transparently, is the healthcare field. Researchers and healthcare practitioners are interested in looking for monitoring the patients' health in out hospital conditions. Monitoring health in that manner will ease and improve the quality of taking care of the elderly and disable people, minimize the need to go physically to the hospital, and reducing treatment cost. We design a system, called Personal Smart Space, which provides an automated method for bootstrapping a personal space once personal devices are near each other C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 386–390, 2011. © Springer-Verlag Berlin Heidelberg 2011

Personal Smart Spaces for Diabetics

387

and are able to communicate. PSS provides a plug and play solution that does not require users to reconfigure their devices each time a personal space is bootstrapped. PSS is component based where each component do a small task in the system. Combining these tasks will produce a huge, helpful job. The components communicate together once they are online without having previous knowledge of each other. A component can act as input that reads an environment variable, or as output that produces a value to output systems or a processing unit that accept a value and processes it or analyzes it to produce a result that in turn can be passed to other components. A component may play more than one role but the simplicity makes the system smarter and available. In PSS if a component fails, searching for a backup component is automatically started in order to keep the system working without any problem. Specifically, PSS will track a person's health and handle its changes that may indicate a risk. PSS will take the benefits of the existing technologies that keep the mobility of the person. It will sense the health changes by a smart wearable device, determine the person position by GPS, and may a live photo of it, exchange information by Bluetooth or Wi-Fi, and communicate over the Internet or the public network via GPRS.

2 PSS Framework Components We propose a lightweight middleware over UPnP [1],[2] that provides automated methods for bootstrapping a personal space once personal devices are near each other and are able to communicate. This automated bootstrapping and management of devices provides a plug-and-play solution that does not require users to reconfigure their devices each time a personal space is bootstrapped. The middleware architecture is shown in Fig. 1. The middleware consists of the following components.

Fig. 1. PSS Middleware Architecture

2.1 Management and Discovery Services Management service is responsible of bootstrapping the PSS, discovering devices in the surrounding area, adding, monitoring, and removing devices to from the PSS as

388

M. AlBahlal and J. AlMuhtadi

needed. New discovered devices are added to the network smoothly, they can be discovered by searching or if they announce about their presence. Devices should be monitored to keep a real state of their availability. If they are no longer existed or decide to leave the PSS then they should be removed smoothly taking into consideration their left role. 2.2 Event Service Event service is a publish-subscribe service where a device can subscribe to events of interest in order to receive a notification when the state of the events it subscribed in is changed. UPnP uses GENA as an event handler protocol. GENA defines an HTTP notification architecture that transmits HTTP notification using Multicast UDP. Multicast UDP only need to send a single notification request and it can be delivered to a large group of subscribers. To accomplish this, subscription arbiters handle the group of subscribers and when the notify resource sends a notification it forward it to them if they subscribed in. 2.3 Description Service Description service is responsible for knowing more about the device properties and services description it provides. UPnP using XML standard to describe the devices and services, retrieving the XML file from the file location attached with received packet provides more opportunity to use the device in appropriate way. 2.4 Presentation Service Presentation service is kind of viewing and controlling device remotely via html pages. This service depends on what the device offers in its page, what state it shows and what kind of interaction available.

3 Example Scenario We apply the concept of this PSS on a diabetic patient using a wearable sensor device that has an ability to measure blood glucose levels and send values to a nearby PDA, mobile phone or laptop. These devices in turn will analyze the received data and send alarms that show the state, position and a live photo of the patient to a predicated list of contacts if the level is above or below the diabetic patient’s normal level.

4 Simulation In order to test PSS and verify its functionality, we simulate 4 devices, 5 services and a coordinator. They are: Camera device that gives a live photo, GPS device that gives the location longitude and latitude of where it is, Glucose sensor that measure the glucose in the blood, laptop that has a screen and a threshold glucose checker and the coordinator.

Personal Smart Spaces for Diabetics

389

4.1 Show Active Devices and Build an Application At start, camera and GPS simulators start. Then the laptop starts, which includes the coordinator that sends a multicast general search requests then devices replies to these requests to represent its availability. If any devices lately join the environment, then they will send advertisement packets. For example, Glucose sensor starts and joins the PSS. Fig. 2 shows the coordinator interface which presents every device in the network and each service it contains and shows whether it is input, output or processing service. After we select all of them, these components will make up the application.

Fig. 2. Build an Application Screenshot

When the value of glucose is changed to 100, the sensor will send a notify message to all subscriber. The coordinator will post an action request to both displayscreen service in laptop device in order to view the value and to glucosechecker service in laptop device in order to processing the value. The glucosechecker service works on the received value and because the 100 in the normal range it will send an event that indicates that the state is “G” green. The value is also posted to display screen. The same is done with GPS and camera simulator. The screen will be look like as Fig. 3. When for example the glucose level is changed to a higher than the normal level (110), the value display in the screen is changed to 110, the state indicates that the case is dangers by letter” R” (Red) beside the GPS & camera image. Also SMS sent to phone numbers set by “set port and phones” screen. 4.2 Find a Backup Device We simulate two more devices one is a screen device, another is a threshold checker and then test if they can be a back up to the laptop that contains the two services if that laptop fails, in this case the coordinator now work on another separate device. The laptop that has processing & screen output is used in the testing scenario. If the Laptop fails, then the search for other services is started in order to stream Glucose values and process them.

390

M. AlBahlal and J. AlMuhtadi

After finding the displayscreen and glucosechecker back up, the application window will display the laptop but indicates that it is inactive but replaced and it should be removed because it no longer needed as shown in Fig. 4. Fig. 5. shows that glucose value is displayed in back up screen.

Fig. 3. Displayscreen Screen Shot

Fig. 4. Two devices replace the laptop that has two components screen Shots

Fig. 5. The Glucose value appears on this screenshot

References 1. UPnP Forum. Understanding UPnP: A White Paper (June 2000), http://www.upnp.org/ 2. Microsoft Corporation, Universal Plug and Play in Windows XP (August 2001), http://www.technet.microsoft.com/enus/library/bb457049 (TechNet.10).aspx

Quality and Usability Assessment for Health Information Websites: Can Commonly Used Evaluation Criteria Be Appropriately Applied to Assess Chinese-Language Websites? Chang Fang-Fang1, Ku Chia-Hua1, Wang Kung-Jeng1, and Wu Wei-Li2 1

Department of Industrial Management, National Taiwan University of Science and Technology, 43, Keelung Road, Sec.4, Taipei 10607, Taiwan, R.O.C. 2 Department of Chinese Language and Culture for International students, 2, Renai Rd., Sec. 1, Linkou Township Taipei County, 24449, Taiwan, R.O.C. {M9801003,kjwang,chiahuaku}@mail.ntust.edu.tw, [email protected]

Abstract. Healthcare websites must be evaluated for quality of content and usability by the target population because they are influential in healthcare decision-making. Many criteria have been developed to assess the quality and usability of health related websites. Nevertheless, most of these criteria are established from English language environment. Information technology is being developed and used by diverse groups of people, including users with different language background. We would like to discuss whether common criteria developed for English language web pages could assess the Chinese language internet environment. Quality and usability assessment criteria will be extracted and summarized by systematic reference search. Afterwards, language expert will evaluate the appropriateness of selected criteria on evaluating Chinese language websites. Keywords: Usability evaluation, Health informatics, Language.

1 Introduction Health information is one of the most important subjects that internet users search online and new topics are continually added to the line-up, such as food safety or drug recalls. A U.S. national telephone survey, conducted from August 9th to September 13th, 2010 by Pew Internet Project, has found that 80% of internet users looked online for health information. It was the third most popular online pursuit, following email and using a search engine [1]. People increasingly expect more from the functionality of health related websites and the web usability has become an important topic. Additionally, Internet World Stats presented its latest estimates for internet users by language. The biggest amount of internet user is English speaking people and the amount of Chinese speaking users comes after that as the second. The Chinese users have accounted for 22.6 % of all the Internet users in the world. The number of Chinese Speaking internet users has grown 1,277.4 % in the last ten years (2000-2010) [2]. As the number of non-English C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 391–394, 2011. © Springer-Verlag Berlin Heidelberg 2011

392

C. Fang-Fang et al.

resources available on the web is increasing, we should consider the factor of language while assessing the quality and usability of healthcare websites. Many organizations have developed quality assessment tools that seek to filter out poor quality health information [3].Nevertheless most of these criteria are established from English-language environment. Therefore, the applicability of these predefined criteria for different language environments should be evaluated. When the target audience differs substantially from the developers, it becomes urgent to tailor both design and evaluation methods [4]. The purpose of this study is to evaluate the feasibility of evaluative criteria to assess Chinese-language information related websites.

2 Methodology Common evaluative criteria were summarized by reviewing medical and informatics references. We systematically searched the following databases: Medline, PubMed, Science Director, and Web of Science by using the following keywords: “internet usability”, “usability evaluation”, “assessment criteria”, “medical informatics”, “health information” and “information quality.” Assessment criteria were extracted and grouped into categories according to their wording and description [5]. We ended the databases and information searching when it produced similar results. The expertbased inspection method will be performed to evaluate the feasibility of each criterion on assessing Chinese language health related websites. A linguistics expert will be involved in the criteria assessment process via an in-depth interview. Through the interview proceeding, criteria will be discussed for their applicability in accordance with the linguistics characteristics.

3 Result 3.1 Common Usability Criteria Fifteen usability criteria have been summarized and they are Purpose, Accuracy, Credibility, Attribution, Authorship, Currency, Disclosure, Useful links, Confidentiality, Completeness, Design and aesthetics, Readability, Navigation, Accessibility and availability, and Interactivity. After reviewing with a linguistics expert, six criteria are identified as linguistic relevance and included for further discussion. Table 1 lists the six criteria and their operational definitions [3, 5-9]. 3.2 The Characteristics between Chinese and English According to literature research, the nature of the representations and the mappings that are made between orthography, phonology, and semantics. The characteristics of English and Chinese languages are summarized in Table 2 [10, 11]. In terms of orthography, about 95% of Chinese characters can be divided into sub-units that form complex visual– spatial patterns. In contrast to Chinese, English has a more regular mapping of orthography[11, 12]. Regarding semantics and phonology, English has a more arbitrary relationship between semantics and phonology than Chinese does [11, 13]. In terms of structure, it indicates that the Chinese language is pictogram-based, whereas English is alphabet-based. In spite many of the different linguistics characteristics, Chinese and English has a similarity feature with the order of words in sentence.

Quality and Usability Assessment for Health Information Websites

393

Table 1. Operational definitions of each usability criteria Criteria Accuracy Design aesthetics

and

Operational definition The extent to which information is reliable and free from errors [8]. The ease with which it was possible to navigate around a website and find information required [5].

Readability

The quality of written language that makes it easy to read and understand [3].

Navigation

Ease of movement around interface and quickness of response [8].

Accessibility availability

and

Interactivity

Ease of access, fee for access, stability [5] Extent to which users can participate in modifying the form and content of a mediated environment in real time [9].

Table 2. The linguistics characteristic between Chinese and English Characteristics Differences Orthography(spelling)

Chinese

English

y Chinese orthography has complex visual–spatial patterns such as topdown, left to right, and outside to inside orientation.

y English orthography has a serial left to right structure of letter strings.

y Many Chinese characters encode meaning by including a semantic radical. y Chinese has a different orthographic system that has more clues to semantics but a less systematic relationship to phonology.

y At a mono-morphemic level, English does not have robust cues to meaning. y English has a more regular mapping between orthography and phonology than Chinese.

Structure

y The Chinese language is pictogram-based. y Chinese in which there are no explicit separators to indicate word boundaries. y Topic-prominent.

y English is alphabet-based. y Each English word is made up of letters and has a space on either side of it when it is written. y Subject-prominent.

Similarity The order of words

y The order of words in sentence is similar with that in English, i.e., of general sequence of subject, verb, object and then complement.

Semantics(meaning)

Phonology(sound)

394

C. Fang-Fang et al.

3.3 Expected Criteria for Assessing Chinese Language Health Websites Further result will present why some usability evaluation criteria may be languagedependent and how this conclusion has been drawn. After proceeding of expert discussion, the final result will illustrate which existing common criteria could be reliably assessed for quality and usability of Chinese web pages and how we should apply them.

References 1. Pew Internet Project, http://www.pewinternet.org/ 2. Internet World Stats: Usage and Population Statistics, http://www.internetworldstats.com/ 3. Eysenbach, G., Powell, J., Kuss, O., Sa, E.R.: Empirical studies assessing the quality of health information for consumers on the World Wide Web. JAMA: The Journal of the American Medical Association 287, 2691 (2002) 4. Moore, M., Bias, R.G., Prentice, K., Fletcher, R., Vaughn, T.: Web usability testing with a Hispanic medically underserved population. Journal of the Medical Library Association: JMLA 97, 114 (2009) 5. Kim, P., Eng, T.R., Deering, M.J., Maxfield, A.: Published criteria for evaluating health related web sites: review. Bmj 318, 647 (1999) 6. John, A.K.: A critical appraisal of internet resources on colorectal cancer. Colorectal Disease 8, 217–223 (2006) 7. Ream, E., Blows, E., Scanlon, K., Richardson, A.: An investigation of the quality of breast cancer information provided on the internet by voluntary organisations in Great Britain. Patient Education and Counseling 76, 10–15 (2009) 8. Usher, W.: General practitioners’ understanding pertaining to reliability, interactive and usability components associated with health websites. Behaviour & Information Technology 28, 39–44 (2009) 9. Whitten, P., Smith, S., Munday, S., LaPlante, C.: Communication Assessment of the Most Frequented Breast Cancer Websites: Evaluation of Design and Theoretical Criteria. Journal of Computer-Mediated Communication 13, 880–911 (2008) 10. Cheung, Y.-B., Thumboo, J., Goh, C., Khoo, K.-S., Che, W., Wee, J.: The equivalence and difference between the English and Chinese versions of two major, cancer-specific, healthrelated quality-of-life questionnaires. Cancer 101, 2874–2880 (2004) 11. Booth, J., Lu, D., Burman, D., Chou, T., Jin, Z., Peng, D., Zhang, L., Ding, G., Deng, Y., Liu, L.: Specialization of phonological and semantic processing in Chinese word reading. Brain Research 1071, 197–207 (2006) 12. Li, D.: A Study of Chinese Characters. Peking Univ. Press, Beijing (1993) 13. Leck, K.J., Weekes, B., Chen, M.J.: Visual and phonological pathways to the lexicon: Evidence from Chinese readers. Memory & Cognition 23, 468–476 (1995)

Computer Interaction and the Benefits of Social Networking for People with Borderline Personality Disorder: Enlightening Mental Health Professionals Alice Good, Arunasalam Sambhanthan, Vahid Panjganj, and Samuel Spettigue School of Computing, University of Portsmouth Portsmouth PO1 3AE, United Kingdom [email protected]

Abstract. This paper seeks to present the findings of a focus group and questionnaire in assessing how aware mental health professionals, who have experience with people with Borderline Personality Disorder (BPD), are in the extent of ICT based support for people with BPD. The methods used were both qualitative and quantitative and used descriptive data. Content analysis was used to explore specific themes and results were cross-examined between the two methods. The work should be viewed as an exploratory study into the viability and likely acceptance of a virtual support community specifically designed for people with BPD. The long term aim is to provide additional support for people with BPD, especially when they are in crisis and might be at a higher risk of harm. Keywords: Mental Health, Borderline Personality Disorder, Virtual Support, ICT, Face Book, Second Life.

1 Introduction The purpose of this research is to examine the level of awareness that mental health professionals have of ICT based support for people with BPD. The research presented here questions professionals on their awareness of Facebook support groups and other Web based resources. It also looks at how likely these professionals would accept a novel proposal for a virtual therapeutic type support environment, specifically for people with BPD. Treatment for severe mental illness does not often consider the potential that social networking can have in both reducing the impact of loneliness (Perese & Wolf, 2005), and the sense of 'feeling alone' (Neal & McKenzie, 2010), and so the use of ICTs in providing support for people with BPD could certainly be beneficial. People with Borderline Personality Disorder (BPD) represent 0.7% of the UK population (NICE, 2009) and are reported as more likely to seek psychiatric intervention than those with other psychiatric disorders (Rendu et al., 2002, cited in NICE, 2009). Whilst there is ICT based support specifically for people with BPD, this is limited predominantly to discussion forums, emails and social networking sites facilitated by Face Book.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 395–399, 2011. © Springer-Verlag Berlin Heidelberg 2011

396

A. Good et al.

Second Life virtual world communities could be beneficial in providing support for people with BPD who might be experiencing a sense of 'feeling alone', and who wish to seek support and useful information on resources available. Second Life, with over 15 million users (K Zero Universe, 2009) is growing as a medium for social interaction. A recent survey on healthcare related activities using Second Life shows that patient education and awareness building as the major health related activity undertaken. Mental health groups in Second Life featured the largest number of members at 32% of the total users (Norris, 2009). In terms of categories of groups, 15% of the health support groups in Second Life were dedicated to mental health. Second Life could then potentially offer the means to provide a virtual support system for people with BPD.

2 Method The methodology applied to this research comprises of two methods; a focus group and a follow up questionnaire to validate and supplement the data. Both qualitative and quantitative data were sought from the study. Participants included nineteen psychiatric staff, some who work as part of an outreach team and others as registered mental nurses in hospitals. All were professionals who had experience of working with people with BPD, most of whom were currently supporting and/or treating current patients with a BPD diagnosis. Both methods included the following themes: types of treatment and support provided; the need for additional support; the extent that staffs were aware of ICT support for people with BPD and how likely they were to accept and promote a virtual support environment. The data collected from the focus group was subjected to content analysis, which was then cross referenced against related questions within the questionnaire.

3 Results Participants comprising of mental health professionals from the focus group were questioned on the type and extent of support they provided for people with BPD. Responses ranged from pharmaceutical treatment, referrals to Dialectical behavior therapy (DBT) and simply ‘being there’ to listen and support.. All participants reported that the degree of time spent supporting people with BPD was significantly higher than their other clients. This was further elaborated on with one member of the group stating: “I think people (professionals) tend to shy away from it. I don’t think they know what to do with people with BPD. ” Other members confirmed they agreed. The questionnaires results provided further information, with all participants stating that in their experience, people with BPD require high levels of support. Furthermore, there were opinions expressed both in the focus group and the questionnaire that people with BPD were ‘untreatable’.

Computer Interaction and the Benefits of Social Networking

397

In spite of this evident need for high levels of support, professionals did not actively promote any types of self help forms of methods via the Internet. See figure 1. The results elicited from the questionnaire further support this.

Fig. 1. As a professional working with people with BPD, I encourage the use of specific self help methods via the Internet

The results from both methods showed that professionals felt that people with BPD were keen to meet others who shared the same diagnosis to meet and support one another. Results from this study have already indicated that the mental health professionals questioned did not promote self help methods via the Internet. When questioned on how aware they were on the availability of online support and resources specifically for people with BPD, including Facebook support groups and information sites where specific examples were given, none of the focus group participants conveyed any awareness of their existence. These results were further validated via a follow-up questionnaire. See figure 2.

Fig. 2: Awareness of mental health professionals of Web based support groups and resources for people with BPD

398

A. Good et al.

Whilst these professionals are evidently less aware of ICT based support, they were able to give example of offline resources such as the Steps Programme (A.A.), which is used to provide support to people with addiction and other behavioral problems. In continuing with theme of the need for additional support for people with BPD, one participant, representing outreach support workers stated: “If they have the ongoing support, over the computer 24/7, it could be useful. We only work 9 - 4.30 and no out of hours service. So we point people to Samaritans of Mental Health Line. In terms of an out of hours crisis, we can’t provide any support. Because that’s the thing isn’t it, they need support most of the time.” These results indicate that there is a clear need for additional support for people with BPD. While there are support groups available online. including those on Facebook, which could offer a degree of containment and/or support, there was no awareness of their existence. In spite of this lack of awareness, there is a keenness to promote any additional support available that might assist people with BPD. Moreover, when the idea was put forward of a virtual support environment, specifically catering for people with BPD, participants responded favourably. “Anything that would offer extra support for people with BPD, we would embrace.” Furthermore, all participants expressed an interest in being involved in a pilot study with the aim to research how a virtual support community system could be designed for people with BPD.

4 Discussion and Conclusions This research is limited in that only a relatively small sample of mental health professionals were involved. However, the results do suggest that these professionals feel that people with BPD have a higher need for support than their other clients. Whilst there is great potential for the use of social networking support groups for people with mental health problems, including Facebook and Second life, these professionals were not aware of their existence. In fact, this paper has highlighted a distinct lack of awareness from these professionals, of any online support specifically related to people with BPD. In spite of this lack of awareness, they were keen to promote any additional ICT related support that might benefit people with BPD and demonstrated a positive response to the proposal of a potential virtual therapeutic support community, specifically designed for people with BPD. Reviewed literature reports that social networking and second life are popular methods for the provision of support for people with general mental health problems (Perese & Wolf, 2005, Norris, 2009). With the economic impact that people with BPD place upon health resources, particularly in comparison to other service users (Rendu et al., 2002, cited in NICE, 2009), more awareness of ICT support would be useful to professionals for them to be able to advise their clients to utilise them.

Computer Interaction and the Benefits of Social Networking

399

Whilst the advent of social networking has given rise to online support groups for people with BPD, there is also potential scope to utilise resources such as second life in providing further support. Furthermore, there is some potential that the provision of additional support could have some impact upon reducing emergency hospital admissions for people with BPD, as well as a possibility for the subsequent decrease in the economic impact to the health services. This work is a continuation of the research carried out in exploring the potential of virtual therapeutic communities based on existing models of therapeutic hospitals and communities, as well as virtual treatments and support in treating people with BPD. An interdisciplinary approach to this research features collaboration from experts in HCI, forensic psychology and psychotherapy. Further studies will focus on developing a framework for the design of a virtual TC environment, which will be very much a user driven study, featuring mental health professionals and people with a diagnosis of BPD. Furthermore, research into exploring how effective avatars can be in enabling a sense of distancing when ‘off loading’ emotions and thoughts will be carried out. The proposed environment could potentially be used as a framework to support other mental health disorders. It should be emphasised that this proposal is intended to provide additional support and is in no way proposed as a replacement to existing methods of support and treatment. Following the research indicated here, there is a need for improved awareness of the availability of current online resources and support. A user-centred approach to the design of a virtual support environment specifically for people with BPD, will certainly go some way in disseminating the idea. However to ensure end user acceptance, awareness and promotion by key professionals will be crucial.

References 1. Alcoholics Anonymous, The Story of How More Than One Hundred Men Have Recovered From Alcoholism. Alcoholics Anonymous World Serv. Inc., (1976) 2. K Zero Universe chart. Q1, KZERO (2009) 3. Neal, D.M., McKenzie, P.J.: I did not realize so many options are available: Cognitive authority, emerging adults, and e- mental health. Library & Information Science Research (2010) 4. NICE, Borderline Personality Disorder – The NICE guideline on treatment and management. NICE, Royal College of Psychiatrists and the British Psychological Society (2009) 5. Norris, J.: The Growth and Direction of Healthcare Support Groups in Virtual Worlds (2009) 6. Perese, E., Wolf, M.: Combating loneliness among persons with severe mental illness: Social network interventions’ characteristics, effectiveness, and applicability. Issues in Mental Health Nursing 26(6), 591–609 (2005)

Design Improvement Requirements for the Upper Extremity Rehabilitation Devices in Taiwan Lan-Ling Huang1, Chang-Franw Lee1, and Mei-Hsiang Chen2 1

National Yunlin University of Science and Technology, Graduate School of Design, 123 University Road Section 3, Douliu, Yunlin 64002, Taiwan [email protected] 2 Chung Shan University of Medicine, Department of Occupational Therapy, 110 Jianguo N. Road Section 1, Taichung 40201, Taiwan

Abstract. This study aims to survey the most frequently used upper extremity rehabilitation devices (UERD) in Taiwan and how well their design meet the practical requirements of rehabilitation therapy. A questionnaire was prepared and sent to a sample of therapists in Taiwan hospitals. Analysis of the replied questionnaires can be summarized as follows: 1) The 185 respondents consisted of 68 males (37%) and 117 females (63%), with average age 31.3 yrs and work experience 7.2 yrs. 2) Therapists thought that the better ones of the existing UERD are Vertical tower, Stacking cones, Climbing board and bar, and Incline board. These devices are mainly used to treat patients’ movements of upper extremity in vertical stretching, flexion, and lifting. 3) The most common problem of the existing UERD is instability of the base. The most required improvement in design features is the adjustability of the functions. Keywords: rehabilitation devices, occupational therapy, product design.

1 Introduction Rehabilitation device is essential in the process of occupational therapy. In addition to the patients, occupational therapists are also major users concerning the use of rehabilitation devices. In a rehabilitation therapy process, the occupational therapist would assess the health situation of the stroke patient first, in order to choose appropriate devices and operating methods, and then design a series of therapy activities to help the patient to restore his/her movement abilities. A preliminary study has found that the current clinical use of upper extremity rehabilitation devices (UERD) vary in many different types and use patterns. In addition, several problems concerning the use of UERD and requirements for improvements have also been identified [1,2]. How to improve the design deficiencies of existing UERD for better rehabilitation result is the objective of this study. This paper reports the result of a questionnaire survey to professional occupational therapists in Taiwan about the use problems and design improvement requirements of UERD. The research results will be a useful reference for medical and rehabilitation device manufacturers as well as designers. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 400–404, 2011. © Springer-Verlag Berlin Heidelberg 2011

Design Improvement Requirements for the Upper Extremity Rehabilitation Devices

401

2 Current Situation of UERD As a preliminary study, a survey of the current situation and needs of the UERD was conducted by field observations and expert interviews. The results of the preliminary study are summarized in this section [1,2]: Table 1 shows the most commonly established and widely used UERD in hospitals in Taiwan. The hospital clinical use of UERD, according to treatment objectives, can be divided into two major categories: proximal type and distal type. A proximal type UERD is mainly used for movements in the shoulder and elbow positions to train a wide movement range for the upper extremity to restore its muscle strength and movement function. Most stroke patients in the beginning stage (with upper extremity movement recoverage in the Brunnstorm stages 2-4) need to use this type of device for rehabilitation. Examples of this type of devices are shown in Table 1. Based on the results obtained from the preliminary study, it is clear that the existing UERD still have many problems concerning their structure, usage, function, and psychological aspects, which all need to be dealt with in their further improvement designs. However, these points are simply a summarization from field observations and interviews to a small sample of hospitals and therapists. Whether or not these points are representative to the therapist population in Taiwan still needs to be justified. A more thorough survey to a larger sample of hospitals and therapists would be useful to clarify this point. Table 1. The most widely used UERD in clinics

D1. Exercise skate of arm

D2. Exercise skate of hand

D3. Single curved shoulder

D4. Horizontal tower

D5. Vertical tower

D6. Climbing board and bar

D7. Stacking cones

D8. Incline board

D9. Curamotion exerciser

D10. Upper bike

3 Questionnaire Survey and Results In order to more clearly understand the relative overall situation of the usage problems and improvement needs of the UERD as described in the previous section, this study further conducted a comprehensive investigation by questionnaire method, to survey professional therapists on each of the usage problem and improvement need. The questionnaire consists of mainly two parts: therapist personal profile and questions about the effectiveness, usage problems and improvement needs of the different

402

L.-L. Huang, C.-F. Lee, and M.-H. Chen

types of UERD. Ten widely used basic types of existing UERD are used in the questionnaire survey. Most of these UERD conventionally popular in clinics (D1-D8, Table 1); however, two of them (D9, D10, Table 1) are new devices which have been recently developed with an integration with information technology. Two types of question format were used: evaluation questions by a 5-point Likert scale with 1 signifying “strongly disagree” and 5 being “strongly agree”, and multiple choice questions (used for choice of features in design improvement). The questions for evaluation are about treatment effectiveness (Q1) and usage problems (Q2-9), as shown in Table 2. In addition, a multiple choice checklist concerning features (F1-F8) needed in the new design of UERD for improvement was also provided, as shown in Table 3. A total 530 questionnaires were sent out to 113 therapists in 113 hospitals in Taiwan. There were 48 hospitals sent back questionnaires (a response rate of 42%, out of 113 hospitals), and 185 valid questionnaires were received (a response rate of 35%, out of 530 questionnaires). The respondents consist of 68 males (37%) and 117 females (63%), with average age 31.2 yrs (SD=6.7) and work experience 7.1 yrs (SD=5.9). Table 2 lists the means of evaluation to the questions. The rightmost column lists the averages of the means of each question across all relevant devices. 3.1 Effectiveness Evaluation The results of Question 1 in Table 2 show that, in general, the therapists positively evaluated the treatment devices as effective (Mean = 3.5), with D5 (Vertical tower), D7 (Stacking cones) being the most effective, and D1 (Exercise skate of hand) and D2 (Exercise skate of arm) the least effective. According to the mode of operation of these devices, we can see that the most effective devices (D5, D7) are for training extension/flexion and lifting of the arm and the least effective devices (D1, D2) are for shoulder or elbow abduction and adduction; while other devices are for training arc movements of the arm. These results suggest that devices of higher performance require more upper limb movements in operation than those of lower performance. 3.2 Usage Problems of the UERD Questions 2-9 in Table 2 are about usage problems of the UERD. Some of the usage questions are related to all the devices (e.g., Q2-3), while others are only relevant to specific ones (e.g., Q6-9). In general, the most severe problems among the 4 questions asked for most devices are: Instability and Boring; and those for specific devices are Inconvenient installation, Straps easily loosen, and No difficulty level adjustment. For the usage problem of Boring (Q2), the therapists considered D1 and D2 being more severe in this respect. As for the problem of Easily damaged (Q3), D7 is more related to it. Furthermore, the Height not adjustable problem (Q4) applies more to D3, and Instability problem applies to D3, D5, and D4. As for the problems of Inconvenient installation (Q7), Straps easily loosen (Q8), and No difficulty level adjustment (Q9) are most related to D1, D2, and D3 respectively.

Design Improvement Requirements for the Upper Extremity Rehabilitation Devices

403

Table 2. Evaluation results of effectiveness, usage problems, and future design improvement of the UERD

Q1 Effectiveness Q2 Boring Q3 Easily damaged Q4 Height not adjustable Q5 Unstable Q6 Additional sandbag Q7 Inconvenient installation Q8 Straps easily loosen Q9 No difficulty level adjustment

Rehabilitation devices D1 D2 D3 D4 3.23 3.32 3.52 3.41 3.60 3.52 3.07 3.24 2.85 2.83 2.31 2.30 3.69 2.79 3.95 3.50 3.04 3.21 3.86

D5 3.74 2.90 2.55 2.78 3.58

D6 3.58 3.23 2.78 2.41 2.78 3.30

D7 3.68 3.02 3.62 2.21 3.14

D8 3.56 3.19 3.11 2.61 2.34

D9 3.45 2.97 2.56 3.29 3.43

D10 3.46 3.08 2.65 3.21 3.41

3.10

3.23 3.50 3.63

2.76

Avg 3.50 3.19 2.77 2.83 3.25 3.18 3.48 3.37 3.30

Table 3. Percentages of therapists selected the features to be included in new design of UERD

Features needed in new designs (%) F1 Interesting to use F2 Pack and store easily F3 Exchangeable components F4 Grips adjustable F5 Durable material F6 Easy to maintain and repair F7 Adjustable in function F8 Recording of movement data

Rehabilitation devices D1 D2 D3 D4 46 40 43 46 24 20 23 18 45 46 57 60 42 41 46 29 43 42 20 21 21 25 11 11 66 67 69 61 61 56 38 40

D5 42 21 70 28 28 13 55 35

D6 37 17 38 23 32 21 48 46

D7 45 22 57 38 51 11 39 33

D8 35 14 48 28 38 20 57 54

D9 25 16 47 35 15 13 40 43

D10 30 15 42 33 24 31 56 53

Avg 38.9 19.0 51.0 34.3 31.4 17.7 55.8 45.9

From the point of view of the 10 devices, the sampled therapists considered that D3, D1, and D2 seem to have more severe usage problems as compared to other devices, with each of them having two or three usage problems evaluated with a mean higher than 3.5 by the therapists. 3.3 Features Needed in the New Design of UERD This section contains the features of design improvement in addition to solving the existing usage problems of the UERD. Two general features concerning future UERD design (Integration design and New function design) were all evaluated highly by the therapists, with mean scores of 3.80 and 3.98 respectively. Table 3 shows the results of the multiple choice features required in new design of each of the rehabilitation devices. The values are presented in percentage of the 185 surveyed therapists. For example, 46% of the therapists selected that the feature of Interesting to use (F1) is required in the new design of the Exercise skate for arm (D1). Among the 8 features, the most chosen ones were: Adjustable in function, Exchangeable components, and Recording of movement data with average percentages of 55.8, 51.0, and 45.9 respectively. The feature of Adjustable in function (F7) was

404

L.-L. Huang, C.-F. Lee, and M.-H. Chen

mostly selected for seven devices (D3, D2, D1, D4, D8, D10, D5). The feature of Exchangeable components (F3) was mostly selected for 4 devices (D5, D4, D3, and D7). Similarly, the feature of Recording of movement data (F8) was mostly selected for 4 devices (D1, D2, D8, and D10). From the point of view of individual devices, those with the highest average percentage values of features to be added are: D1, D2, and D3. On the other hand, those with the lowest average percentage values of features to be added are: D9 and D6.

4 Conclusion This paper reports the results of a questionnaire survey to professional occupational therapists about the problems of existing upper extremity rehabilitation devices and their improvement needs. In summary, the following points can be concluded: 1. The majority of therapists consider that the existing UERD are effective in treatment for the patients. 2. There still exist some defects in the design of the devices, for instance, many devices are unstable to use due to lack of a firm base. Further integration of the devices into a few types with various functions is desirable. In addition, for some treatment requirement, new device should be introduced for such functions as: manual dexterity training, ADL-training, or sensory re-education, etc. 3. For improvement, the new design should at least incorporate the following features: a) adjustable in function, structure, and composition, b) providing exchangeable components, c) adapting information technology to allow the registration of the patient’s action data, and d) interesting to use so as to satisfy the user’s psychological needs. Acknowledgment. This study is supported by the National Science Council of the Republic of China with grant No: NSC 99-2221-E-040-009.

References 1. Huang, L.L., Chen, M.H., Lee, C.F.: The Investigation of Design Improvement Requirements for The Upper Extremity Rehabilitation Devices in Taiwan. In: 2010 Asia Pacific Industrial Engineering & Management System Conference, APIEMS 2010, Melaka (December 07-10, 2010) 2. Huang, L.-L., Lee, C.-F., Chen, M-H.: An investigation of the upper extremity rehabilitation devices for stroke patients. Journal of Science and Technology (2011) (accepted) (in Chinese)

Observation Research of Consumer Behavior for Marketing Decision Support Hideyuki Imai1, Noriko Hara1, and Toshiki Yamaoka2 2

1 Lion Corporation, Tokyo, Japan Wakayama University, Wakayama, Japan

Abstract. The purpose of this paper is to reveal a complaint and needs, insight from a practical way of observation such as brushing teeth. In our observation treated extreme users who will be likely to have the power of expression towards using goods as participants to reveal sub-consciousness under behavior. We take the adoption of a new ethnography. This way is defined by Fetterman, that it is combined to report a practical way of observation and interpretation. The behavior was recorded on video by their family not to disturb their own natural way. Moreover, we questioned the participants and cameramen about a behavior with watching videos to clear the gap between their behavior and recognition. We analyzed to report a practical way of observation, the participant’s impression and ideas from seeing video. We will discover subconscious dissatisfaction and needs. Under this approach, many participants did hard brushing every day, because they would not take artificial tooth. They have misunderstanding that the carefully brushing is strong brushing or long-time brushing. We found this approach helpful to understand customer insight that ‘brushing carefully’ meant ‘brushing strongly’ and participants relied on ‘quantity’ of brushing in other words. As a result, consumer discovery of insight will contribute to create more effective and efficient communication message for advertising and package, it will improve method of product and service development with high reality. Keywords: Insight, sub-consciousness, observation, ethnography, brushing carefully, advertising.

1 Introduction During the decrease in population and the change in composition of family, we can not find out customer needs and wants in the current way of research. Therefore it is clearly demanded more efficient and accurate marketing techniques based on observation of consumer behavior. The purpose of this research is to reveal a complaint and needs, insight from a practical way of observation such as brushing teeth. Therefore we can adapt marketing techniques to consumer needs, more efficient and accurate based on observation of consumer behavior.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 405–409, 2011. © Springer-Verlag Berlin Heidelberg 2011

406

H. Imai, N. Hara, and T. Yamaoka

2 Methods In our observation, we treated extreme users who will be likely to have the power of expression towards using goods as participants to reveal sub-consciousness under behavior. We take the adoption of a new ethnography. This way is defined by Fetterman [1], that it is combined to report a practical way of observation and interpretation. We sampled ten participants of teeth brushing observation research, who brush three times or twice, once a day. Participant age is from teenager to sixties and each age’s member is two participants. The behavior was recorded on video by their family.

3 Research Step The behavior was recorded on video by their family not to disturb their own natural way such as a following figure1. Moreover, we questioned the participants and cameramen about the behavior with seeing videos to clear the gap between their behavior and consciousness.

Fig. 1. A Novel technique of Observation and Research

1. Step1: Fact analysis. We analyzed to report a practical way of observation and the participant’s impression and ideas from seeing video. We will discover subconscious dissatisfaction and needs. 2. Step2: Concept of Structure Building. ‘The structure of insight’ in a collaborative task team. Members in this team need the ability of data analysis and understanding real intention behind behavior. To apply results in this research to marketing strategies and marketing mix plans, members in the team should be selected from Research and Development, Product Planning, Marketing, Advertising and Information and Research Department. Each member joins the first video analysis and studies consumer insight to construct the behavior process. Studying insight behind behavior is like understanding a scenario in a film. Therefore we call this

Observation Research of Consumer Behavior for Marketing Decision Support

407

task ‘Scenario Investigation’. Owing to this task, members can gain common experiences and work fully collaboratively. 3. Step3: Development of the concept. The new process would be demanding to make use of findings in this research for concept development. Product concepts used to be developed directly from fuzzy consumer needs. However, we will make propositions prior to concepts, which are more abstract idea like an image. ‘The structure of behavior and emotion’ reveals propositions then we create keywords, killer words and key images which can show consumer’s dissatisfaction, unsolved needs and unfulfilled desire. Founded keywords can be straight used in brand regulation to state the structure of new brand. It is necessary that we argue about the essence of customer insight sufficiently until we can represent the concept as one sentence. 4. Final step: Marketing planning. The marketing mix planning including concept test and communication strategies follow this procedure.

4 Results 4.1 Findings of the Insight from a Teeth Brushing Too Carefully 1. Fact analyzed. A following figure 2 and figure 3. cThe most typical participant is housewife, who brushed the teeth three times a day. Her brushing time was more six minutes. Average brushing time was about one minute past thirty second. Her brushing time was very long. dShe used the various tool of teeth, toothbrush, interdent brush and floth, very carefully. She was not realized retreating of gums. eShe was not brushing parts anywhere in mouth. It is biased to brushing time to the teeth by each region in mouth. Especially at the under front of teeth is most short time. 2. Finding Insight. cShe does hard brushing every day, because she would not take artificial tooth. She has misunderstanding that the carefully brushing is strong brushing or long time brushing. Consequently, she takes a gap between behavior and recognition. dTherefore she thinks a best action for the stain off the tooth that the brushing for good teeth is strong brushing and a lot times and a long time.

Fig. 2. Finding from investigation of a tooth brushing behavior

408

H. Imai, N. Hara, and T. Yamaoka

Fig. 3. Difference of the time to vrush the tooth at each region

5 Consideration 5.1 Findings from Methods of Behavior Observation For instance, we found two cases of brushing teeth in bath and 18 year-old girl brushing teeth in kitchen without comb in the morning. We believe these behavior patterns were recorded as they were. The questionnaires to participants disclosed their own gaps between their actual behavior and their consciousness. Seeing videos, participants also found the oddness of their ways and the shorter time in behavior than they imagined. The subconscious process was surprisingly revealed by their own findings, which was clearly different from findings provided by moderators. Moreover, it was also valuable finding that their family as cameramen filled the role of good moderators. Families were surprised at the differences between participants and them to watch behavior and asked questions straight. As a result, they succeeded in bringing out the background and insight behind behavior. Under this research, we chose extreme users as participants since we expected them to express deep subconscious in behavior. Consequently, it is more effective and objective approach to find insight in observation research of behavior and we do not have to depend on moderators anymore. 5.2 Consideration about Behavior Observation The participant with high concerns about tooth brushing was biased in region of brushing as well as other participant with little concern. They believed ‘brushing carefully’ meant ‘brushing strongly’ and they relied on ‘quantity’ of brushing in other words. Through these the observation, the members of Product Development strongly recognized the importance of view in terms of consumers’ actual behavior. We have developed new concepts which can transform the value of tooth brushing from ‘quantity’ to ‘quality’, such as ideas feeling of brushing by sound or touch. We feel confident this point of view, which we call it ‘signal’, will helpful for brand-new product

Observation Research of Consumer Behavior for Marketing Decision Support

409

development. To sum up, consumer discovery of insight will contribute to create more effective and efficient communication message for advertising and package to use killer words and it will improve method of product and service development with high reality. We believe it will support marketing managers who have to make decisions on making products concept and advertising concept.

6 Discussion Application of this video observation to marketing planning is still in process. We need to evaluate the effectiveness on products and advertising reflected concepts and killer words through this procedure after launch. What is more, we have to define extreme users and normal behavior so as to demonstrate findings to development staff persuasively. Hence, we need to take enough time for preparation. We have not developed the method to keep the quality of preparation then we have relied on our experiences for the present. That is to say, it is essential to smooth out the differences of opinions among ourselves including executives in advance.

References 1. Fetterman, D.M.: Ethnography: Step by Step. Applied Social Research Methods. Sage Publishing, Thousand Oaks (1989) 2. Tamura, H., Sugasaka, T., Ueda, K.: Designing a Smart Shopping-Aid System Based on Human-Centered Approach. eMinds: International Journal on Human Computer Interaction (2007) 3. 山岡俊樹, ヒット商品を生む観察工学, 共立出版 (2008)

Exercise Reminder Software for Office Workers Ahsen Irmak1, Rafet Irmak2, and Gonca Bumin3 1

Hacettepe University, Institute of Health Sciences, Program of Ergotherapy, Turkey 2 Ahi Evran University, College of Health Services, Turkey 3 Hacettepe University, Faculty of Health Sciences, Department of Ergotherapy, Turkey [email protected]

Abstract. Computers are widely used instruments in workplaces. Despite the low level of physical load, computer usage requires repetitive motions and staying in same position for a long time. That’s why great number of computer users may contract musculoskeletal disorders. Frequent rests can prevent musculoskeletal discomfort, but OW generally do not break their work until discomfort occurs. The rest breaks can be reminded by computer software. The purpose of this study was to develop exercise reminder software (ERS) for OW in Turkish Language. An exercise protocol was designed to prevent most common MSDs among OW. The software was designed under C# with video play, exercise description and exercise log properties and the source code is open for scientific purpose. The software is compatible with Windows XP and Vista. The developed software is the first known ERS in Turkish Language. Future studies may focus on integration of distance patient follow up systems. Keywords: computer software, exercise reminder, office workers.

1 Introduction Computers are widely used instruments in workplaces. When office workers (OW) use this instrument, they do not need high level of energy consumption but they have to stay in the same position for a long time and they have to do repetitive movements (1). Staying a static posture can change normal force distribution on joints of human body and may cause strains of human skeletal muscles (2). Repetitive movements can cause micro trauma on different type of tissues of human locomotion system. Both of the situations will cause degrease work performance and increases health expenses. To overcome these problems, ergotherapists study on preventive methods for OW. Some of the classical approaches are rearrangement of work place, ergonomic education of OW, preventive exercise programs and frequent rests (1). Frequent rests and preventive exercise programs have important role on prevention of musculoskeletal disorders (MSDs) among OW, but they generally do not break their work until discomfort occurs and they do not go on exercise programs regularly if supervision is unavailable or insufficient. To remind these rest breaks and supervise exercises by computer software become a popular method in recent years (1). These method have some advantages over classical approaches like cost effectiveness and degreased need for direct supervision. In addition C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 410–414, 2011. © Springer-Verlag Berlin Heidelberg 2011

Exercise Reminder Software for Office Workers

411

American Ergotherapy Association (AOTA) suggests use of exercise reminder software (3). But exercise reminder software is not common in languages other than English. It couldn’t be succeed to find any exercise and rest break reminder software for Turkish OW in the literature. The purpose of this study was to develop exercise reminder software for OW in Turkish Language.

2 Material and Methods Exercise Protocol. Therapeutic exercise was defined by DeLateur as “the prescription of bodily movement to correct impairment, improve musculoskeletal function or maintain a state of well-being” (4). For these purposes, therapeutic exercises can include stretching, strengthening or cardiovascular components. These exercises can be done in laying, standing or sitting position. Also tools like elastic resistance, balls and free weights can be used when doing the therapeutic exercise. OW are generally effected by MSDs which generally includes the involvement of spinal column and upper extremities. Musculoskeletal pain and joint stiffness are common complaints. Stretching exercises can degrease the complaints and prevent OW for future impairment (5). Also strengthening exercises should be included in the exercise protocol to maintain adequate function and maintain a state of well being. Cardiovascular exercises are helpful to improve endurance, maintain emotional well being and help to overcome stress in the work places. Cardiovascular exercises include walking, swimming and cycling like activities and it is difficult to do them in office conditions. The therapeutic exercise protocol was planned according to most common MSDs in OW which included preventive exercises that can be done in office conditions with no specific exercise tools. The protocol consists of gentle and mild level stretches and low level strengthening exercises. The frequency of the exercise was 1 exercise per 45 min. Software. Two databases were designed. One for exercise descriptions and one for exercise logs. Exercise description database includes information about video file of each therapeutic exercise, exercise description for OW and exercise description for ergotherapist. Exercise log database includes information about OW performance about exercise program. Algorithm was designed to provide flexibility to the user to continue exercise program from the last exercise he/she didn’t complete last time. The code was generated under C# (Microsoft Visual Studio 2008). User interface and exercise descriptions were in Turkish Language. The source code of the software is open for scientific purposes and it is possible to translate interface and exercise descriptions in other language without commercial use under the permission of authors. The software was tested under Windows XP and Vista. Target Population for the Software. The software was designed for general OW. The software was first applied on stenographers who work in Turkish Parliament. Stenographers are high level office workers who work on computer with time limit. They have to complete their work in given time.

412

A. Irmak, R. Irmak, and G. Bumin

Satisfaction of the Software. The user satisfaction was assessed by a 13 item questionnaire. (1)

3 Results The developed software includes a database for exercise descriptions and a database for exercise logs. 53 office exercises were recorded 1280-720 pixel resolution, 1048 kps video data rate and 24 bit video sample with no voice. This was done to not affect the conditions of work place. Exercise descriptions were presented as text next to video box. The software reminds exercises per 45 minutes. Software minimizes itself to task bar after OW has completed the exercise routine. OW can give feedback as “I did the exercise” and “I didn’t do the exercise”. If OW the feedback is “I didn’t do the exercise” the software begins from this exercise next time it come to the screen. User interface and exercise descriptions were designed in Turkish Language. The source code of the software is open for scientific purposes and it is possible to translate interface and exercise descriptions in other language without commercial use under the permission of authors.

Fig. 1. User Interface

The software was tested under Windows XP and Vista in experimental conditions and found compatible with these operating systems. Real condition test was done in Turkish Parliament, department of record office on stenographers. No technical problem was observed if NET Framework 3.5 or higher was initially installed. User satisfaction questionnaire was applied on 11 (8-73% Male, 3-27% female) OW whose mean age was 43±11 and mean working experience was 21±13 years. 873% of the participants were stenograph, 1-9% was expert stenographer and 2-18% were government official. The participants predicated that they used software 6±3 times per day and 11-100% of them suggest the software to their colleagues. Other statistics are presented in Table 1.

Exercise Reminder Software for Office Workers

413

Table 1. Results of user satisfaction questionnaire Item Effect on overall productivity Item

Easy to use Exercise instructions easy to follow

Empty 1 11% Empty

-

Empty

Item Video helpful in following the exercises Item Overall Satisfaction

Empty

-

Very Positive 1 11% Very Easy

Positive

Negatively

Very Negatively

Total Answers

8 89% Easy

Hard

Very Hard

9 Total Answers

5 45%

5 45%

1 9%

-

11

2 18% Very Helpful

9 82% Helpful

Somewhat Helpful

Not at all Helpful

11 Total Answers

5 45% Very Satisfied 4 36%

5 45% Satisfied

1 9% Unsatisfied

Very Unsatisfied

11 Total Answers

5 45%

2 18%

-

11

4 Conclusion User satisfaction questionnaire was done on a limited population but 89% of the users predicated that the software has positive effect on their overall productivity. The developed software is the first known ERS in Turkish Language. As previous studies of the authors the source code is open for scientific purposes under the permission of authors. This may help to develop exercise reminder software in languages other than English. Language of user interface and exercise descriptions is one of the important topics on access and adaptation of the software user. In this version, databases are offline. The ergotherapist collects log data per week by copying database to a flash memory. Also if there will be a need to make modification on exercise description or exercise video this should be done by manual access. These are the limitations of the current version. Future studies may focus on to develop online database functions for ergotherapist and integration of distance patient follow up systems.

References 1. Leonard Trujilloa, L., Zengb, X.: Data entry workers perceptions and satisfaction response to the “Stop and Stretch” software program. Work 27, 111–121 (2006) 2. Barredo, R.V., Mahon, K.: The effects of exercise and rest breaks on musculoskeletal discomfort during computer tasks: an evidence-based perspective. J. Phys. Ther. Sci. 19, 151–163 (2007)

414

A. Irmak, R. Irmak, and G. Bumin

3. McHugh Pendleton, H., Schultz-Krohn, W.: Pedretti’s Occupational Therapy Practice Skills for Physical Dysfunction. Elsevier, Missouri (2006) 4. Kottke, F.J., Stillwell, G.K., Lehmann, J.F. (eds.): Krusen’s Handbook of Physical Medicine and Rehabilitation, 3rd edn. WB Saunders Co., Philadelphia (1982) 5. Fisher, D.L., Andres, R.O., Airth, D., Smith, S.S.: Repetitive motion disorders: the design of optimal rate-rest profiles. Hum. Factors 35, 283–304 (1993)

Games for Health: Design Cognition-Focused Interventions to Enhance Mental Activity Hyungsin Kim1, Viraj Sapre1, and Ellen Yi-Luen Do1,2 1

GVU Center & College of Computing 2 College of Architecture, Georgia Institute of Technology, Atlanta, Georgia 30332, USA {hyungsin,viraj,ellendo}@gatech.edu

Abstract. Older adults want to proactively protect their aging cognition with various possible ways. Cognitive intervention has been proposed as an effective way to improve memory loss problems. In this paper, we present the CogStim Game, an iPad application for older adults to exercise their memory. For example, by practicing name-face matches through the medium of a game, older adults would have a chance to stimulate their cognition. Together with the game description, we will also present our design rationale behind the game development. The CogStim Game would provide a more accessible and effective way to address our society’s concerns due to an increasing aging population. Keywords: Games for Health, Senior Friendly Design, Cognitive Stimulation, Alzheimer’s Disease and Related Disorders (ADRD), Brain Exercise, Memory Enhancement Activities, Dementia.

1 Introduction Aging is one of the highest known risk factors for Alzheimer’s [1]. The chance to develop Alzheimer’s doubles about every five years after age 65 [2]. As there is no known cure for Alzheimer’s Disease and Related Disorders (ADRD), many older adults are concerned about their mental health and would like to proactively monitor their aging cognition [3]. Alternative treatments from both pharmacological and nonpharmacological approaches have shown to either slow down symptoms or preserve the patient’s current cognitive status [4]. Older adults tend to prefer cognitive interventions because they are usually safer and without side effects, compared to medications. They would like to seek help from non-pharmacological methods such as brain exercises, cognitive stimulation, and memory rehabilitation to delay the onset of ADRD [5]. The current practice of cognitive intervention conducted by special therapists requires a series of one-on-one or group meetings; thus, not many people would have a chance to try or receive the therapies. Besides these therapy approaches, research shows that active engagement with stimulating brain activities such as riddles or puzzle-solving may prevent age-related cognitive dysfunction [3, 6]. However, some people may stop doing these activities once they become mundane and boring. Studies show that in order to enhance the effectiveness of the intervention, the activities C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 415–419, 2011. © Springer-Verlag Berlin Heidelberg 2011

416

H. Kim, V. Sapre, and E. Yi-Luen Do

should be regularly practiced rather than performed on a short-term trial basis [7]. In this paper, we present our unique approach to integrate cognition-focused interventions as a computer game, the design rationales, and the Face Name Game as an example module of the CogStim Game.

2 The CogStim Game To increase accessibility and enhance personal engagement with cognitive intervention, we have designed and developed a technological intervention called the CogStim Game. The goal of the CogStim Game is to provide older adults with cognitive exercises that not only can improve cognitive outcomes, but can also provide predictive value in detecting cognitive impairment. The CogStim Game would encourage older adults to use their brains through diverse and fun mental exercises while actively monitoring their game play outcomes. If the outcome of a brain exercise is found to be worsening, it may be a timely detection of initial cognitive decline. Furthermore, with the CogStim Game, older adults concerned about their cognitive health may selfadminister cognitive intervention games, similar to what they can do at the gym to exercise and monitor their weight in order to prevent cardiac-related disease. Figure 1 shows an initial screen shot of the CogStim Game, in which a user can choose one of the four different game modules.

Fig. 1. A screen shot of the CogStim Game including four independent modules

In order to understand the technological potential to support cognition-focused intervention for cognitive impairment, we took the approach of gaining contextual knowledge, such as understanding the intervention process, interactions between therapists and patients in the intervention process, and their interactions with the intervention materials. Based on this knowledge we developed three design rationales, described in the following section.

Games for Health: Design Cognition-focused Interventions to Enhance Mental Activity

417

3 Design Rationales Based on user-centered design principles, we took an iterative design process to develop a series of game modules in CogStim. The design and development of the CogStim Game modules should meet the three design rationales below. 3.1 Developing Clinically Proven Game Contents The CogStim Game would support personalized, engaging, and stimulating cognitive activities through extensive game playing. There are three approaches to designing cognition-focused interventions: cognitive stimulation, cognitive training, and cognitive rehabilitation [5, 8]. Despite the conceptual differences in the definitions, the terms stimulation, training, and rehabilitation are used interchangeably. Our CogStim Game includes all of the three techniques. Since our goal is to provide learning through games, we can incorporate any technique to develop reasoning, memory, and speed of cognition. 3.2 Interacting with Technology The CogStim Game would be easy to use for older adults who are not familiar with computer use. Many older adults may have low computer literacy, with limited or no computer usage experience. If they cannot use computers, or if they have difficulty in interacting with technology, it could result in unexpected or adverse outcomes. In our previous research on a computerized dementia-screening tool, we learned that pen input is accessible, even for seniors who lack computer experience [9]. Furthermore, an 85-year-old study participant expressed that it was fun to draw using a stylus on a Tablet PC, and that drawing with a stylus on a smooth tablet was much easier than drawing with a pencil on a piece of paper. Some participants wanted to finger paint. Therefore, we designed interaction methods using touch or pen input for the CogStim Game. 3.3 Motivating Users The CogStim Game would motivate older adults to play every day as a routine exercise. We investigated ways to incorporate reward schemes to increase motivation and encourage game participation. For example, a personally motivating reward could be an automatic direct dial to a grandchild if they complete the daily game session. Another example would be putting an algorithm into the game so that if a user completes a certain session, their family members will then be notified by an instant short message or an email. The reward structure could also be accumulating points to exchange for coupons or gift cards that could be redeemed at participating local businesses.

4 The Face-Name Game As one of the CogStim Game modules, the Face-Name game is developed based on Cognitive Stimulation (CS). One of the CS exercises includes tasks such as recalling

418

H. Kim, V. Sapre, and E. Yi-Luen Do

recently learned items [5]. The Face-Name game has two modes: a training mode and a testing mode. In the training mode, users will be trained to associate a particular name with a particular face, while in the testing mode, the user will have to match names with faces. The interfaces provided by the game for each of these will vary, depending on the user’s particular stage of cognitive impairment. In the training mode, the game will try to reinforce memory by providing the user with multimodal cues about a person's name for users with Mild Cognitive Impairment (MCI) or the early stage of Alzheimer’s. For example, a user will be presented with images in groups of 7 and will be allowed to spend as much time as he or she wishes on each face-name pair. The interface for the current version has a photo of a person, along with a name in large letters. The interface says the name presented on the screen and prompts the user to say the name out loud while looking at the image. Figure 2 shows an example screen of the game which provides the three different pictures and a user need to select the right name that she had learned.

Fig. 2. A screen shot of the Face-Name Matching module in the CogStim Game

Only after the user has said the name out loud will he or she be allowed to navigate to the next image. The interface for users in advanced stages of Alzheimer's will be slightly different. In this case, the interface (a) will have no audio component, (b) will have smaller sets of images (3 per set) and (c) will change the face-name pairs, according to pre-decided timing. The interval for which the user will be allowed to see the face and read the name will be very brief (<2 seconds). This interval will allow the user's implicit memory to associate the face and name pair, while the user's explicit memory may not be able to make the association. This is deliberate on our part to enforce the user to tap his or her relatively undamaged implicit memory, even while the explicit memory is being eroded due to Alzheimer's. The testing mode of the game will provide users with a name and multiple faces per level. The users must drag the name to the correct face. As such, the interface for this mode will almost be the same for all users. The only difference will be the number of faces shown per level. For example, the number of faces shown to users with MCI will be more than the number of images shown to users in advanced stages of Alzheimer's.

Games for Health: Design Cognition-focused Interventions to Enhance Mental Activity

419

5 Conclusion and Future Work Early intervention is crucial in cases of MCI. Treating individuals with Mild Cognitive Impairment (MCI) may delay the progression of the disease and prevent it from developing. With CogStim games, older adults would be able to practice cognitive exercises anytime and anywhere. Once we complete the development of other modules, we plan to conduct randomized, controlled trials to evaluate the effectiveness of the CogStim Game. Ultimately, our research may contribute to understanding how computing technologies can advance our understanding of early cognition-focused intervention, the impact to the prevention and progression of disease, and quality-oflife issues for the aging population.

References 1. McCullagh, C.D., Craig, D., McIlroy, S.P., Passmore, A.P.: Risk factors for dementia. Advances in Psychiatric Treatment 7, 24–31 (2001) 2. Alzheimer’s Association: 2011 Alzheimer’s Disease Facts and Figures. Alzheimer’s and Dementia 7, 208–244 (2011) 3. Desai, A.: Healthy Brain Aging: Evidence Based Methods to Preserve Brain Function and Prevent Dementia, An issue of Clinics in Geriatric Medicine. Elsevier Health Sciences (2010) 4. Weih, M., Wiltfang, J., Kornhuber, J.: Non-pharmacologic prevention of Alzheimer’s disease: nutritional and life-style risk factors. J. Neural. Transm. 114, 1187–1197 (2007) 5. Moniz-Cook, E., Manthorpe, J. (eds.): Early Psychosocial Interventions in Dementia: Evidence Based Practice. Jessica Kingsley, London (2009) 6. Singh-Manoux, A., Kivimäki, M.: The importance of cognitive ageing for understanding dementia. AGE 32, 509–512 (2010) 7. Mahncke, H.W., Connor, B.B., Appleman, J., Ahsanuddin, O.N., Hardy, J.L., Wood, R.A., Joyce, N.M., Boniske, T., Atkins, S.M., Merzenich, M.M.: Memory enhancement in healthy older adults using a brain plasticity-based training program: a randomized, controlled study. Proc. Natl. Acad. Sci. USA 103, 12523–12528 (2006) 8. Robinson, L., Hutchings, D., Dickinson, H.O., Corner, L., Beyer, F., Finch, T., Hughes, J., Vanoli, A., Ballard, C., Bond, J.: Effectiveness and acceptability of non-pharmacological interventions to reduce wandering in dementia: a systematic review. International Journal of Geriatric Psychiatry 22, 9–22 (2007) 9. Kim, H., Cho, Y.S., Guha, A., Do, E.Y.-L.: ClockReader: Investigating Senior Computer Interaction through Pen-based Computing CHI Workshop on Senior-Friendly Technologies: Interaction Design for the Elderly, Atlanta, GA, pp. 30–33 (2010)

Promoting Positive Employee Health Behavior with Mobile Technology Design Hyungsin Kim1, Hakkyun Kim3, and Ellen Yi-Luen Do1,2 1

GVU Center & College of Computing 2 College of Architecture, Georgia Institute of Technology, Atlanta, Georgia 30332 {hyungsin,ellendo}@gatech.edu 3 John Molson School of Business, Concordia University, Montreal, Quebec Canada, H3G 1M8 [email protected]

Abstract. In this paper, we present an augmented mobile technology that can enhance the positive health behavior of employees, as well as companies’ social welfare. Our proposed mobile technology encourages walking behaviors among employees, which are then transformed into monetary incentives or a tangible donation. We draw on two social behavior theories for our technology design and also introduce a conceptual model that provides a “step-by-step” approach to enhance employees’ health and improve social welfare. Keywords: Health management system, corporate social responsibility, consumer-driven health care, health promotion and wellness.

1 Introduction The general lifestyle of people in the 21st century lacks basic recommended daily physical activity. This sedentary way of living causes several major public health problems [1]. Research shows that people who are physically active can reduce their risk of developing major chronic diseases such as coronary heart disease, stroke, and type 2 diabetes by up to 50%, as well as reduce the risk of premature death by approximately 20~30% [2, 3]. In addition, physical inactivity at all stages of life (i.e., from childhood to adulthood) contributes to negative effects (e.g., diseases, disorders, and premature mortality), and the risks tend to become greater as people age [4]. Related to sedentary life patterns, the significant amount of time people spent at their office working further reduces physical activity. At the same time, however, the dominant role of workplace in modern society makes it a great setting for health promotion to take place. Any new mechanism that can align health benefits for employees and benefits for firms can achieve this goal. Eventually, both parties can view health promotion programs targeting employees as a solution for a win-win outcome, and not as a trade-off relationship in which one wins, but the other loses. To this end, the current research intends to fulfill such an opportunity that resides in workplaces. Specifically, by choosing everyday walking as a focal type of physical activity, we propose a conceptual model based on the concepts of corporate social C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 420–424, 2011. © Springer-Verlag Berlin Heidelberg 2011

Promoting Positive Employee Health Behavior with Mobile Technology Design

421

responsibility and behavioral economics. Given the increasing demand for sustainability and corporate social responsibility (CSR), firms are being asked to perform more socially desirable functions and become an active part of the community. Unfortunately, we have yet to see an integrative perspective of health promotion design that considers the organic relations holistically (employees, firms, the communities). Therefore, we propose an augmented mobile technology that can facilitate the promotion of good health among employees as a solution to benefit all of the related stakeholders. In our proposal we show how everyday walking can have additional meanings in both economic and social conceptual models, and how employees, firms, and communities can all win by adopting these models.

2 Incentive-Based Human Behaviors The hedonic principle that people approach pleasure and avoid pain has been the basic motivational principle throughout the history of psychology [5]. How incentives affect decisions has been considered critical in economics [6]. People often respond to incentives in change or development of their behaviors. On the corporate side, firms use this theory to structure employees’ compensation in a way that the employees’ and owners’ goals are in alignment. Financial incentive, and the tools for encouraging certain behaviors or outcomes based on this theory, have been proven effective for obtaining best work performance [7]. Reflecting on the power of incentives to motivate desirable behavior, incentives at the workplace manifest themselves in a variety of forms, such as yearly bonuses, stock options, and pay for performance. In the context of promoting physical activity, the presence of incentives can encourage employees to develop healthy habits, such as regular everyday walking, as we will discuss later.

3 Corporate Social Responsibility Over the last two decades, corporate social responsibility (CSR) or sustainability has become a significant issue for business strategy. Media coverage of this issue has proliferated, together with public and governmental attention on this topic [8]. Corporate social responsibility (CSR) initiatives cover a wide range of activities, including traditional corporate philanthropy, socially responsible manufacturing practices, and community involvement. Prior research has found that CSR initiatives can have a positive impact on a company [9, 10]. For instance, “consumers want to buy product from companies they trust; suppliers want to form business partnerships with companies they can rely on; employees want to work for companies they respect; and NGOs, increasingly, want to work together with companies seeking feasible solutions and innovations in areas of common concern” [11]. CSR emphasizes that businesses should cooperate within society [12]. CSR means more than merely making financial contributions. People expect corporations to be engaged in their communities in a variety of ways. It also involves multiple stakeholders, including the government, shareholders, employees, consumers, the media,

422

H. Kim, H. Kim, and E. Yi-Luen Do

suppliers, NGOs, and the general public. Satisfying each group while minimizing conflicts will allow companies to develop a new win-win situation. The enhancement of employees’ health is a domain in which the CSR concept is readily applicable. Employee illness can increase healthcare costs. Further, it may reduce productivity or increase uncertainty via unanticipated absences. In fact, several research studies show that workplace health promotion programs can generate more profits than they incur in costs [1, 13]. Therefore, designing successful interventions in employee well-being will provide significant returns, such as increasing productivity, profitability, and savings in healthcare costs. CSR can also be used as a way to connect companies to non-profit organizations. In our proposed model, we describe dual routes by which an individual employee’s physical activity for good health can contribute to an increase in a company’s CSR.

4 Conceptual Models to Improve Employees’ Everyday Walking We propose a conceptual model to encourage employees’ everyday walking. Employees can opt for a commuting option requiring more walking. They can use their break time for a 10-minute walk, or they can use the stairs instead of elevators. They can even use lunchtime for 30-minute group walks outside of the building. 4.1 Direct (Monetary or Non-Monetary) Incentives Our model focuses on the individual employee’s role and its relationship with cost savings for companies. Figure 1 shows the interrelationship between employees and companies. The key notion herein is that employees’ walking behavior is compensated by companies through incentive systems, and companies can reduce insurance costs and enjoy improved productivity.

Employees’ walking

cost saving and performance various incentives

Companies

Fig. 1. Basic Conceptual Model

In this model, a company’s main roles are: 1) to promote employees’ walking by providing sufficient incentives, monetary or non-monetary (for example, promotion, benefit); and 2) to enlighten/educate employees regarding the importance of personal health management and provide easy steps for achieving good health conditions (e.g., walking). The three benefits the company would have are: 1) to reduce insurance costs; 2) to create an active work environment, which may induce job efficiency/efficacy; and 3) ultimately to improve employees’ well-being by enhancing their health.

Promoting Positive Employee Health Behavior with Mobile Technology Design

423

4.2 Indirect (Social) Incentives Our basic model can be extended to include another component, where corporations can exercise CSR initiatives in a broader context, and eventually become better corporate citizens: charitable giving toward non-profit organizations in their local communities. Figure 2 describes how the basic model is extended.

Employees’ walking

cost saving and performance recognition

donations and supports

Companies

positive public image through acknowledgement

Nonprofit organizations

Fig. 2. Extended Nonprofit-based Model

An additional role that the companies should play are: 1) to envision a societal role for firms among employees; 2) to add social meaning to the employees’ participation in the walking program; and 3) to select the causes or charity organizations as the beneficiaries. A company’s benefits are also extended: 1) to gain a positive brand image and social legitimacy from the public; 2) to increase awareness of the brand among prospective consumers; and 3) to gain tax benefits.

5 Technological Implementation To make this initiative involving employees’ walking and incentives proportionate to their walking activity, we employ mobile technology in our design. Currently, pedometers or step counters are popularly used as everyday exercise measurers or motivators. Pedometers are portable and can also be integrated into personal electronic devices, such as mobile phones. There are five main features implemented in the mobile phone application: Steps, Distance, Calories, Progress, and Donation. For example, the Steps icon shows the number of steps a person takes. The Distance icon displays a user’s walking distance in miles or kilometers. The Calorie icon shows a user’s calculated calories burned, based on the user’s exercise, considering the individual’s weight, height, walking distance and the level of the ground. By employing GIS technology, the difficulty level of the area will be calculated, which is then used for calorie calculation. For example, walking uphill will be differently calculated from walking on level ground or walking downhill. The Progress icon will show the temporal pattern or progress up to date. If a person sets a certain goal for a certain month, a status in comparison to the end point can be displayed, so as to further motivate the walker. Lastly, the Donation icon will enable the walker to allocate his/her incentives that have accumulated to either his/her own incentives or donations to nonprofits. Additionally, the walker will be given a list of nonprofits or causes, from which s/he can choose the beneficiary of his/her walking.

424

H. Kim, H. Kim, and E. Yi-Luen Do

6 Conclusion In this paper, we proposed a conceptual model that seeks to increase everyday walking for employees, decrease health insurance costs for employers, and ultimately contribute to our society’s well-being: a win-win situation. By using this overarching model, companies will simultaneously meet two goals: promoting their employees’ health, as well as increasing corporate social responsibility. An augmented mobile technology can play a new role as a facilitator to help companies increase cooperative social responsibility through a change in employees’ lifestyles. We use simple everyday walking as an example of physical activity. However, it can be extended and modified by applying this model to other forms of contributions from individual employees in addition to other methods of promoting companies’ corporative social responsibility.

References 1. Blackburn, G.L.: The ROI on Weight Loss at Work. Harvard Business Review 87, 30 (2009) 2. U.S. Department of Health and Human Services: The Surgeon General’s Call to Action to Prevent and Decrease Overweight and Obesity. U.S. Department of Health and Human Services, Public Health Service, Office of the Surgeon General, Rockville, MD (2001) 3. Smith, T.W., Orleans, C.T.: Prevention and Health Promotion: Decades of Progress, New Challenges, and an Emerging Agenda. Health Psychology 23, 126–131 (2004) 4. British Department of Health: At Least Five a Week: Evidence on the Impact of Physical Activity and its Relationship to Health. Department of Health Physical Activity, Health Improvement and Prevention (2004) 5. Higgins, E.T.: Promotion and Prevention: Regulatory Focus as a Motivational Principle. Advances in Experimental Social Psychology 30 (1998) 6. Mankiw, N.G.: Principles of Microeconomics. The Dryden Press, Orlando (1998) 7. Nohria, N., Groysberg, B., Lee, L.-E.: Employee Motivation: A Powerful New Model. Harvard Business Review 86 (July-August 2008) 8. Berns, M., Towned, A., Khayat, Z., Balagopal, B., Reeves, M., Hopkins, M.S., Kruschwitz, N.: Sustainability and Competitive Advantage. Sloan Management Review 51, 19–26 (2009) 9. Brown, T.J., Dacin, P.A.: The Company and the Product: Corporate Associations and Consumer Product Responses. Journal of Marketing 61, 68–84 (1997) 10. Sen, S., Bhattacharya, C.B.: Does Doing Good Always Lead to Doing Better? Consumer Reactions to Corporate Social Responsibility. Journal of Marketing Research 38, 225–243 (2001) 11. Guzzo, R.: Rethinking Rewards. Harvard Business Review 72, 158–159 (1994) 12. Maignan, I., Ferrell, O.: Corporate Citizenship as a Marketing Instrument. European Journal of Marketing 35, 457–484 (2001) 13. Makrides, L.: The Case for Workplace Health Promotion Program. Newsbeat of CACR 12, 1–4 (2004)

Believable Agents, Engagement, and Health Interventions Christine L. Lisetti Affective Social Computing Group, Florida International University [email protected]

Abstract. In this article we review some of the main contributions that the believable agents community has brought about and we review some of the main believable agent architectures existing to date. We also discuss the enormous potential that believable agents can bring in the domain of health, including health communication, health promotion, health counseling and psychotherapy. We discuss some of our work-in-progress aimed at building believable agents with dialog abilities to conduct supportive healthcare interventions for the management of chronic diseases and addiction issues.

1 Introduction The term “believable agents“ was originally coined to refer to autonomous artificial entities or agents, that are believable in a similar way that characters in movies or animation are believable (Bates1992; Bates1994; Bates & Loyall1997). For example Mickey Mouse is believable as an animated character in that its viewer can suspend his or her disbelief and experience that Mickey Mouse is real. Similarly, Rick - portrayed by Humphrey Bogart in Casablanca (1942) - is an equally believable and engaging character whom we believe is the irrevocably cynical owner of the Caf´e Americain (irrevocably until he meets the kid that is). In order to accomplish this illusion of life, believable agents do not need to be realistic (realism can even be proscriptive), but rather be carefully and artistically crafted to exhibit personality. Although believable agents have mostly co-evolved for and from the arts and entertainment applications, their desired abilities and features are very similar to what appears to be necessary to build engaging and effective computer-based health interventions. Indeed, although there is a crying need for computerized healthcare (Blumenthal2009) to facilitate access to prevention and/or chronic disease management programs (particularly for underserved populations), few patient-centered computer-based health care systems are available today. Furthermore, most of the existing e-health programs are mostly passively informative, presenting a variety of factual knowledge to their user, and consequently suffer from very high levels of attrition. In contrast, believable agents are designed to foster high levels of engagement with their users. Building healthcare delivery systems with believable agent architectures can C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 425–432, 2011. c Springer-Verlag Berlin Heidelberg 2011 

426

C.L. Lisetti

therefore potentially lead to successful health care programs, increasing accessibility of care (and ideally its quality as well). Believable agents can be created and integrated in the design of health communication strategies to communicate educational health literacy material in an engaging manner, and to deliver a wide variety of patient-centered tailored health-promotion interventions. In this article we review some of the main contributions that the believable agents community has brought about and we review some of the main believable agent architectures existing to date. We also discuss the enormous potential that believable agents can bring in the domain of health, including health communication, health promotion, health counseling and psychotherapy. We discuss some of our work-in-progress aimed at building believable agents with dialog abilities to conduct supportive healthcare interventions for the management of chronic diseases and addiction issues.

2 Believable Agents The believable agents community has been evolving over the last decade, since Joe Bates Oz project seminal paper on the role of emotions in believable agents (Bates1994) (portrayed very effectively and simply on the “woogles“ shown in Figure 1).

Fig. 1. Woogles in the Edge of Intention (from (Bates1994))

A variety of influencial believable characters can be recognized in the Sims (Wright2000), the Tamagotchi (Yokoi & Maita1996), Dogz (Stern1995), Nintendogs (Mizuki2005), or Touch Pet Doges, to name a few. AI-based believable agents are becoming increasingly powerful. Hap is an agentlanguage developed in the Oz project at CMU to construct non-linguistic believable agents. Hap provides mechanisms that support writing behaviors for characters, where a behavior is a piece of activity which can be high-level (a behavior for ”having fun”), or low-level (a behavior for moving the character’s body when they open a door). An extension to Hap to support natural language text generation was also built (Bates & Loyall1997). The extension allowed to integrated text generation with the agent’s actions, perceptions, inferences and emotions. As a result the agents could then use language and action together in order to best communicate, vary generated text based on emotional state or to express a specific personality, and issue text in real-time with desirable pauses, restarts and other timing effects.

Believable Agents, Engagement, and Health Interventions

427

Fac¸ade on the other hand, (created by one of the members of the Oz group) is a real-time 3D interactive narrative simulating being on stage with two live actors who want make a dramatic scene happen (see figure 2) (Mateas2003; Mateas & Stern2005). Facade is implemented using the ABL reactive planning language (based on the Oz Project language Hap mentioned above) which is designed specifically for authoring believable agents. In the case of Fac¸ade, the characters play roles in an interactive dramatic story world (inspired by the Edward Albee’s play Who’s afraid of Virginia Wolf?) (Mateas1999; Mateas2001; Mateas2003; Mateas & Stern2005).

Fig. 2. Facade (from (Mateas2003))

Another approach to building believable characters is found in Twig, inspired by the world of very fast robotic behaviors. Twig is a fast, AI-friendly procedural animation system that supports easy authoring of new behaviors (Horswill2009b). The system is particularly interesting for building novel scenarii because it provides a simplified dynamic simulation that is specifically designed to be easy to control. Characters in Twig are controlled by applying external forces directly to body parts, rather than by simulating joint torques. Twig’s puppetrystyle of control provides the simplicity of kinematic control within an otherwise dynamic simulation. Although less realistic than motion capture or full biomechanical simulation, Twig produces compelling, responsive character behavior (see figure 3).

Fig. 3. Twig playground (from (Horswill2009a))

428

C.L. Lisetti

It is also fast, stable, and supports believable physical interactions between characters such as hugging, punching, and dragging, and makes it easy to author new behaviors (Horswill2007; Horswill2008; Horswill2009a).

3 The Need for Computer-Based Approaches to Health Communication and Intervention Mental illness has been identified as the second leading cause of disability and premature mortality in the developed world. Yet the majority of people suffering from treatable mental health disorders do not have access to the required treatment. Similar statistical data can be found about at-risk people (e.g. substance abuse, HIV/AIDS, obesity), or people suffering from chronic condition (e.g. diabetes). Computer-assisted health interventions might be able to address this imbalance (Coyle et al.2007). If designed properly, believable agents could become very engaging therapeutic computer-based social companions aimed at promoting healthy behaviors by complementing regular therapeutic sessions. Patients could interact daily (or as needed) with their artificial agents who (or which) are personalized to their specific disorder(s), between the scheduled weekly sessions with their therapist. One type of intervention documented for treating substance abuse is referred as motivational interviewing (MI) (Miller & Rollnick1991). Although it was originally designed to treat alcohol and substance abuse, it has since then proven helpful for a wide variety of health promoting behaviors (Burke, Arkowitz, & Menchola2003). Unlike more traditional confrontation-of-denial counseling, motivational interventions are intended, through support, to increase the likelihood that people will make changes in their behavior toward healthy habits. It typically involves a brief assessment followed by feedback about the assessment results (Burke, Arkowitz, & Menchola2003). Such health promotion interventions (e.g. sexual risk reduction interventions) have proven most beneficial if they can individually tailor behavior change messages. Tailoring involves the use of the participants name (personalization), characteristics of the user such as gender (adaptiveness), or self-identified needs of the user (feedbackprovision) (Strecher1999;Lustria et al.2009). Tailoring can also be dialog-based by asking the user about their goals and by recommending choices about how to best achieve these goals. The key is that the personalized, adaptive, or feed-back-based tailored messages are more effective in promoting attitude and behavior change than the generic one size fits all content delivered by print or to groups (Strecher1999). Although additional studies are needed, a brief individualized computer-delivered sexual risk reduction intervention study (Kiene & Barta2006) has documented the risk reduction potential of interventions via computers for adolescents. These computer-based interventions might also prove particularly useful when dealing with sensitive private information (e.g. homosexual preference in nonsupportive culture, secret excessive drinking patterns). When revealing sensitive private information, participants have been found to be more candid with automated data collection systems than when interacting with a human interviewer (Evans, Edmundson-Drane, & Harris2000) (Plutchik & Karasu1991).

Believable Agents, Engagement, and Health Interventions

429

3.1 Talk-Based Health Interventions with Believable Agent Based on the need for computer-based health interventions described above, and the latest progress in believable agents, we posit that believable agent-based health interventions might be ideal to assist in delivering tailored health information to various populations (including at-risk populations). Believable agents created for health interventions will need to: – – – – – – –

have a believable personality and remaining engaging, be able to express empathy convincingly, be personalized to a user with a user- model, be highly adaptive in terms of characteristics (e.g. gender, race, culture, language), gather, remember (or learn from) feedback given voluntarily by a user, be able to engage in meaningful dialog, on a specific domain of expertise (since the full unconstrained natural language processing research problem is still unresolved).

In a previous article we described how believable embodied conversational agents (ECA) can be designed to conduct motivational interviews with participants (Lisetti & Wagner2008). ECAs are given predetermined scripted questions to elicit private information from the teenager, record the results, calculate the statistics (as an expert system connected with relevant databases) to be discussed during the feedback by the therapist. ECAs can then generate an animated feedback movie based on the therapists feedback that the teenager can keep and replay as needed (Lisetti & Wagner2008). Our ongoing work continues the implementation of agent-based motivational interviews, a system we refer to as Mobile and Accessible Telehealth Personal Avatar (MATPA). 3.2 Believable Agents in Virtual (Game) Environments for Health Interventions Another type of therapy involves exposure to disturbing stimuli or events at various increasing levels of intensity to address problems such as post-traumatic disorders, anxiety and phobic responses (Silverman & Kurtines1996). By being an active participant in the simulated reenactment of a traumatic or fearinducing situation, the patient can gain a sense of mastery by acting in a situation that previously paralyzed them. They learn to differentiate their perceptions of and reactions to such situations in ways that allow them to separate realistic and unrealistic elements in their cognition. Virtual environments are particularly efficient for example for treating phobias such as fear of spiders, fear of flying in planes, or fear of heights (Alcaniz et al.2006). In addition they have also been used to treat post-traumatic disorders, as for soldiers coming back from Iraq and troubled with PTSD. More recently, progress in ECA technology has made it possible to turn attention to problems involving human- human interactions or social phobias for which interaction with virtual interactive anthropomorphic avatars appears as a potential asset (Glantz, Rizzo, & Graap2003). Successful examples include suggestions for the treatment of public speaking anxiety (Pertaub Mel Slater & Barker2002), panic disorder and agoraphobia, and for a variety of other mental health problems (Riva2005).

430

C.L. Lisetti

Fig. 4. Simulated attachement behavior in Twig (from (Horswill2008))

These type of interventions could be very well designed within the Twig architecture, to design relevant interactions for children in the Twig environment and characters as shown in figure 4 (Horswill2009b), where the simple form and expressiveness of the characters can be manipulated to be quite engaging to children. Such games could also sense the child’s level of anxiety or stress using non-invasive bio-sensors or bio-feedback during the game session (Lisetti et al.2009). Such physiological readings could enable the game to adjust the level of exposure or difficulty of the game based on the stress that it generates and ensure that the exposure never becomes uncomfortable for the child.

4 Conclusion We have described how the current progress of in the field of believable agents can be relevant to enable the creation of computer-based health care interventions, for a variety of interventions and a variety of populations (e.g. culture, age, gender). Believable agents nowadays can be designed to possess enough intelligence to be engaging to users of a different groups and populations. Believable agents can be coupled with Artificial Intelligence techniques (e.g. user-modeling, expert-systems), for the development of meaningful and helpful computer-based interventions. Computer-based interventions are much needed are much needed to complement health care practioners and physicians’ work in a wide range of different contexts. For example, access to literacy material and interventions for underserved populations has been identified as seriously missing both in terms of actual access and of appropriate literacy level; some populations are pointed to computer-based material that does not match their literacy level (be it reading literacy, arithmetic literacy, or health literacy), resulting in poor or null level of impact and effectiveness. Similarly, access to chronic disease management material or interventions while at the home could substantially reduce the disabling effect of some chronic conditions, yet this access is quasi nonexistent. Computer-based approaches could address this gap. We have tried to demonstrate that believable agent-based interventions are much needed for health care, and that this work is feasible, given some careful artistic and AI design of believable agents. Much more research is needed in this area.

Believable Agents, Engagement, and Health Interventions

431

References [Alcaniz et al.2006]Alcaniz, M., Rey, B., Botella, C., Ba˜nos, R., Guillen, V., Quero, S., Garc´ıaPalacios, A.: Adaptive display for the treatment of emotional disorders: A preliminary analysis of effectiveness. In: Proc. CHI 2006 (2006) [Bates & Loyall1997]Bates, J., Loyall, B.: Personality-Rich Believable Agents that Use Language (1997) [Bates1992]Bates, J.: Virtual Reality, art, and entertainment. Presence: Teleoperators and Virtual Environments 1(1), 133–138 (1992) [Bates1994]Bates, J.: The Role of Emotion in Believable Agents. Communications of the ACM 37(7) (1994) [Blumenthal2009]Blumenthal, D.: Stimulating the adoption of health information technology. New England Journal of Medicine 360(15), 1477–1479 (2009) [Burke, Arkowitz, & Menchola2003]Burke, B.L., Arkowitz, H., Menchola, M.: The efficacy of motivational interviewing: A meta- analysis of controlled clinical trials. Journal of Consulting and Clinical Psychology 71, 843–861 (2003) [Coyle et al.2007]Coyle, D., Doherty, G., Sharry, J., Matthews, M.: Computers in talk-based mental health interventions. Interacting with Computers (2007) [Evans, Edmundson-Drane, & Harris2000]Evans, A., Edmundson-Drane, E., Harris, K.: Computer-assisted instruction: An effective instructional method for hiv prevention education. Journal of Adolescent Health 26, 244–251 (2000) [Glantz, Rizzo, & Graap2003]Glantz, K., Rizzo, A., Graap, K.: Virtual reality for psychotherapy: Current reality and future possibilities. Psychotherapy: Theory, Research, Practice, Training 40(1/2), 55–67 (2003) [Horswill2007]Horswill, I.: Psychopathology, narrative, and cognitive architecture (or: why ai characters should be just as screwed-up as we are). In: Proceedings of the AAAI Fall Symposium on Intelligent Narrative Technologies (2007) [Horswill2008]Horswill, I.: Men are dogs (and women too). In: Proceedings of the AAAI Spring Symposium Series on Agents, Personality and Emotions (2008) [Horswill2009a]Horswill, I.: Very fast action selection for parameterized behaviors. In: Proc. of the International Conference on Foundations of Computer Games (2009a) [Horswill2009b]Horswill, I.D.: Believable Physical Interactions. IEEE Transactions on Computational Intelligence and AI in Games 1(1), 39–49 (2009b) [Kiene & Barta2006]Kiene, S., Barta, W.: A brief individualized computer-delivered sexual risk reduction intervention increases hiv/aids preventive behavior. Journal of Adolescent Health 9, 404–410 (2006) [Lisetti & Wagner2008]Lisetti, C.L., Wagner, E.: Mental health promotion with animated characters: Exploring issues and potential. In: Proceedings of the AAAI Spring Symposium Series on Emotions, Personality and Behavior (AAAI SSS 2008), AAAI, Menlo Park (2008) [Lisetti et al.2009]Lisetti, C., Pozzo, E., Lucas, M., Hernandez, F., Silverman, W., Kurtines, W.: Programming in the second life virtual world. In: Proceedings of the 14th Annual Conference on CyberTherapy and CyberPsychology Conference - Studies in Health Technology and Informatics (SHTI) Series (2009) [Lustria et al.2009]Lustria, M., Cortese, J., Noar, S., Glueckauf, R.: Computer-tailored health interventions delivered over the web: Review and analysis of key components. Patient Education and Counseling 74, 156–173 (2009) [Mateas & Stern2005]Mateas, M., Stern, A.: Build it to understand it: Ludology meets narratology in game design space. In: DiGRA Conference, Vancouver, BC, vol. 2. Citeseer (2005) [Mateas1999]Mateas, M.: An Oz-Centric Review of Interactive Drama and Believable Agents. In: Veloso, M.W., Manuela (eds.) LNAI, pp. 297–328. Springer, Berlin (1999)

432

C.L. Lisetti

[Mateas2001]Mateas, M.: Expressive AI: A Hybrid Art and Science Practice. Journal of the International Society for Arts, Sciences and Technology 34(2), 147–153 (2001) [Mateas2003]Mateas, M.: Fac¸ade, an experiment in building a fully-realized interactive drama (2003) [Miller & Rollnick1991]Miller, W.R., Rollnick, S.: Motivational interviewing: Preparing people to change addictive behavior. Guilford Press, New York (1991) [Mizuki2005]Mizuki, K.: Nintendogs. Nintendo EAD (2005) [Pertaub Mel Slater & Barker2002]Pertaub Mel Slater, D., Barker, C.: An experiment on public speaking anxiety in response to three different types of virtual audience. Presence: Teleoperators and Virtual Environments 11(1) (2002) [Plutchik & Karasu1991]Plutchik, P., Karasu, T.: Computers in psychotherapy: an overview. Computers in Human Behavior 7, 33–44 (1991) [Riva2005]Riva, G.: Virtual reality in psychotherapy: Review. CyberPsychology and Behavior 8(3), 220–230 (2005) [Silverman & Kurtines1996]Silverman, W., Kurtines, W.: Anxiety and Phobic Disorders. Plenum Publishing Corporation, New-York (1996) [Stern1995]Stern, A.: Dogz. PF Magic (1995) [Strecher1999]Strecher, V.J.: Computer-tailored smoking cessation materials: a review and discussion. Patient Edu. Couns. 36, 107–117 (1999) [Wright2000]Wright, W.: The sims. Maxis/Electronic Arts (2000) [Yokoi & Maita1996]Yokoi, A., Maita, A.: Tamagotchi. Bandai Ltd. (1996)

Le-ADS: Early Learning Disability Detection System for Autism and Dyslexia Nor’ain Mohd Yusoff, Nor Syarafina Rusli, and Ruhaiza Ishak Faculty of Information Technology, Multimedia University Malaysia, 63100 Cyberjaya, Selangor, Malaysia [email protected], {syara.nor,bobogulz}@gmail.com

Abstract. Screening test is one of common approaches to detect learning disabilities among children. The Early Learning Disability Detection System for Autism and Dyslexia (Le-AdS) is developed to help primary school teachers to recognize signs and students’ behaviour. Studies and researches for the system have been done to understand these types of disorder. Research on the system architecture has also been carried out to know how the system should work based on the requirements and needs of the user. Interviews, reading and overview have been applied throughout the development process of this standalone software. This paper presents the work of Early Learning Disability Detection for Autism and Dyslexia (Le-ADS). Keywords: Mild Autism, Dyslexia, Screening System, Learning Disability.

1 Introduction In Malaysia, autism and dyslexia syndrome still have a cloud of stigma over it with many blaming parents for the misdeed that led the child to being punished [7]. According to Dyslexia Association of Sarawak president Dr. Ong Puay Hoon, there are more than 850 000 Form One students in our country who are unable to read and write appropriately because of these syndromes. This is a disturbing scenario because if these children are educated based on appropriate and suitable curriculum they may have chances in becoming successful and intelligent like any normal individuals. In order to educate these types of children, they need to be taught based on the correct learning methods. The Ministry of Education has provided special syllabus to teach dyslexic and autistic pupils. The problem is to categorize them accordingly. The existing method that was introduced by the ministry was basically based on teachers understanding on students’ academic performance, not characteristic of any learning disabilities. Furthermore, it does not provide students involvement directly. To divide these children accordingly, a reliable screening tool is needed to support the test done by teachers. The paper describes the problem statements, research methods and related works with regards to autism and dyslexia problem. It features present the system of Le-ADS, its features and implication to the users.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 433–437, 2011. © Springer-Verlag Berlin Heidelberg 2011

434

N. Mohd Yusoff, N.S. Rusli, and R. Ishak

2 Problem Statement In Malaysia, children with dyslexic and autism are being neglected in a way that they do not get a proper education compared to children without the problem of learning disorder. There are no integrated systems used by the Education Department in order to help teachers to screen and categorized these ‘special’ students accordingly. This will bring difficulties to inexperience teachers and those with deficient information on the disorders; and due to these it will directly affect students. The problems identified are as follows: • Inexperience teachers – New teachers graduating in the field are qualified teachers, but most of them have never gained a firsthand experience, except for what they learned in lectures or through briefings. • Lack of information on the symptoms related - Teachers need to do their own research and study on the symptoms in order to understand students ‘behaviour and learning problem. Information given by the parents or paediatricians is too general, in which just basically stated the disability type. • Students are being neglected - Students for both disabilities need different learning methods. If they are carelessly screened, they will be allocated in different classes with another type of learning techniques; the one which they don‘t essentially need. • ‘They do not look disabled’ condition - Since the disabilities did not revealed physical sign, some teachers will simply assume that the children are acting lazy and not focused in class. [1]

3 Research Methodology The methods included in the research are library search and interviews from selected participants. Library search involved extracting information from various reading sources that includes books, journals and research papers. Interviews involved respondents from NASOM (The National Autism Society of Malaysia), Temiang Primary School and Taman Tuanku Jaafar 2 Primary School. These references are accurate, comprehensible, and most importantly dependable in order to gain more information associated with learning disabilities, particularly autism and dyslexia. The interviews are extracted to gain the needed information. The selections are labelled as Sample I, Sample II and Sample III. For Sample I, the interview was conducted with the respondent from NASOM Intervention Program. Respondent is an experience teacher in training and managing children with Autism. Sample II shows the extraction from the interview with respondent from Temiang Primary School. Besides being a coordinator, the respondent is also experienced in teaching dyslexic students and has participated in several courses on the subject of Dyslexia. Sample III consists of extraction from interview with respondent from Taman Tuanku Jaafar 2 Primary School. The respondent has been a teacher for 16 years and is experienced in educating primary school students with learning disabilities.

Le-ADS: Early Learning Disability Detection System for Autism and Dyslexia

435

Sample I Interviewer: What is Autism? Respondent: A lifelong developmental disorder that prevents a child from having the ability to learn and to be able to master language and communication skills. Interviewer: What are the characteristic of an autistic child in terms of behaviour? Respondent: They do not have any eye to eye interactions with the people around them (unusual eye contact), tend to ignore questions or instructions and sometimes prone to be echolalia, unusual attachment to certain object (e.g. toy car), showing emotions inappropriately at unsuitable places and they have no fear of real danger. Interviewer: What are the characteristics of an autistic child in terms of their physical appearance? Respondent: An autistic child or adult look like any normal people with no disabilities. [5] Sample II Interviewer: What is Dyslexia? Respondent: Students that are dyslexic are those that have difficulties in recognizing alphabets, constructing words and phrases. At early stage, they tend to have difficulties identifying left and right; up and down and the concept of in front and behind. They have problems memorizing the words that have been taught to them; and have problem with words that are similar in sounds. Dyslexic people tend to use the right side of the brain more than the left side. These children are considered weak when it comes to language, logics and structures. Interviewer: What are the characteristics of a dyslexic student in terms of behaviours? Respondent: Behaves like normal children (normal here means that they can play, talk, understand instructions and can think of actions), unable to focus during class, likely to avoid reading activities. Interviewer: How about their criteria in terms of physical features? Respondent: Physically, these kids look exactly like normal kids with no difficulties. [3] Sample III Interviewer: What is Special Recovery Program? Respondent: The Special Remedial Program (‘Program Pemulihan Khas’) is an education plan set by the Malaysia Ministry of Education for primary school students that have complex problem of learning the basic skills of 3M. 3M is an abbreviation for “membaca, menulis, mengira” (read, write, count). Interviewer: Who are the students that are in the program? Respondent: The ones that are unable to read, write and count correctly; and those who cannot master the 3M skills because of surrounding factors not cognitive disabilities. Interviewer: How old are the students in this program?

436

N. Mohd Yusoff, N.S. Rusli, and R. Ishak

Respondent: The age for student in this program is eight to nine years old. They don‘t have to be in the program for the whole two year period. Once the students are able to adapt to 3M skills, they are shifted back to regular classes. [2]

4 LE-ADS System Le-ADS is a software to assist teachers and school principals to divide possible autistic or dyslexic students and the end result from this screening system will be a reference for teachers and school principals to provide appropriate learning curriculums for dyslexic and autistic children. Underlying Spiral Model Development Life Cycle, the software was implemented based on the qualifiers by [1], and findings from [2], [3] and [4]. The objectives of implementing Le-Ads as follows: • To assist teachers in categorizing dyslexic or autistic students • To help principals to differentiate dyslexic or autistic children in school • To associate dyslexic and autistic students with appropriate syllabus. 4.1 System Screen Shots: Examples on the Autism and Dyslexia Results The screen shots below show the examples on how results for autism and dyslexia are obtained. In the STUDENT PROFILE page shown in Fig 1, teacher enters students’ particulars and record. Next, in Fig. 2, teacher proceeds the test together with student by asking questions with the choices of Yes and No options. The Result Page in Fig. 3 displays Student’s Name, ID, Teacher’s Name and Test Result

Fig. 1. Student Profile Menu

Fig. 2. Examples of Questions

Le-ADS: Early Learning Disability Detection System for Autism and Dyslexia

437

Fig. 3. Examples of Autism or Dyslexia Learning Disability Result

5 Conclusions There are some significance in implementing the Early Learning Disability Detection System for Autism and Dyslexia (Le-ADS). The standalone software enables teachers to construct their own conclusion based on first-hand experience, not just depending on pediatricians’ report which sometimes can be too general or technical. The system that acts as an early detection system is able to help teachers in categorizing the students and proceed to deciding which curriculums are suitable for students, based on the type of disorder. Teachers will be able to provide more attention to students, corresponding to the students learning problems. With the existence of the software, the school principals may acknowledge that such disabilities cases exist and in a way exposed parents to the importance of detecting learning disabilities at early stage.

References 1. Abdul Wahab, M.H., Aziz, M.A., Asha’ari, F.J.: Expert System for Special Education (Final Year Project). Faculty of Information Science and Technology, Multimedia University Malaysia, Melaka (2007) 2. Abdullah, P.N.H.: Special Remedial Program in Sekolah Kebangsaan Taman Tuanku Jaafar 2, Sungai Gadut (March 31, 2010) (An Interview) 3. Darus, M.S.B.H.M.: Dyslexia Program in Sekolah Kebangsaan Temiang. Negeri Sembilan (April 2, 2010) (An Interview) 4. LUCID COPS, http://www.lucid.research.com (retrieved November 8, 2009) 5. Mintol, P.Y.: The National Autism Society of Malaysia (NASOM), Intervention Program. Seremban, Negeri Sembilan (March 29, 2010) (An Interview) 6. Mohd Yusoff, N., Abdul Wahab, M., Aziz, M., Jalil Asha’ari, F.: ESSE: Learning disability classification system for autism and dyslexia. In: Stephanidis, C. (ed.) UAHCI 2009. LNCS, vol. 5614, pp. 395–402. Springer, Heidelberg (2009) 7. Najar, E.: The Multi-Faceted Expression of Autism. New Straits Time Online (News Image Bank), p. 009 (July 1, 2010), http://www.nib.com.my (retrieved)

Interaction Design of Encouraging Daily Healthcare Habit with Communication Robots Jun’ichi Osada1, Tomoharu Yamaguchi2, Ryohei Sasama2, and Keiji Yamada2 1 NEC Design and Promotion Co. Ltd., 1753 Shimonumabe, Nakahara-Ku, Kawasaki, Japan 2 NEC Corporation, C&C Innovation Research Laboratories, 8916-47 Takayama-Cho, Ikoma, Nara, Japan [email protected], {r-sasama@cp,yamaguchi@az, kg-yamada@cp}.jp.nec.com

Abstract. This paper presents interesting findings and strategies for interaction design of a health support system for seniors. The system consists of a pedometer, a terminal with touch screen and a communication robot. Users can touch buttons on the screen and data of the pedometer as inputs. Speech from the robot and text messages on screen is used for output. Because of bringing a robot as a part of the system, this system requires new features beyond existing standards of usability and accessibility. Thus, the authors redefined three key features; a) natural, b) user friendly, and c) enjoyable for a development strategy of the interaction design with a communication robot.

1 Introduction Recently, isolation of elderly people has been considered an important social issue [1]. The health support system for seniors had developed to this demand [2]. The system design required two aspects; taking characteristic of seniors users into consideration, and keep users using the system every day without getting tired. Thus, “userfriendliness” and “enjoyment” are major important aspects. The authors tried to use the communication robot to meet these requirements. This paper presents interesting findings and strategies for interaction design learned through development of such a system.

2 System Specification and Application This system is composing of a pedometer, a terminal with touch screen and a communication robot “PaPeRo” as shown in Fig.1. The system accepts touches to buttons on the screen and the data of pedometer as inputs from the user. The pedometer is equipped with near-field communication device. Data of a pedometer is read by a sensor on top of the terminal. Speech from the robot and text messages displayed on screen is used for output to the user [2].

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 438–442, 2011. © Springer-Verlag Berlin Heidelberg 2011

Interaction Design of Encouraging Daily Healthcare Habit with Communication Robots

439

Fig. 1. A set of the health support system

“Health Check” to promote measuring weight and blood pressure, “Local Information” to provide news of local activities, and “Message” to exchange voice messages among users are the three major applications of the system. The system starts when a user puts the pedometer to a sensor or touches a button on the screen. The system assembles a flow of a task according to the trigger of the start (Fig.2). Those flows were designed by authors to refrect priorities of actions into the order of sequence in each situation.

Fig. 2. Flows of the tasks.

3 Requirements and Design Specification Major target users of this system are elderly people. The purpose of the system is to enable the users to use this system for a long term. Thus, “user-friendliness” and “enjoyment” are major important requirements. “User-friendliness” in this context means adapting to characteristic of elderly for accessibility and usability. Additionally, reduction of cognitive load is also important. “Enjoyment” can be achieved by conversational interactions, elements of surprise, and impressive presence.

440

J. Osada et al.

3.1 User-Friendliness Accessibility Design. GUI was designed to match to senior characteristic; colors, layout, size and font of text [3]. In addition, the words and phrases appear on the display are also selected to be simple enough (Fig.3).

Fig. 3. A snapshot of the screenp

PaPeRo’s behaviors, such as voice, utterance, and words and phrases are also designed to meet the senior characteristic. a) Voice: avoid the frequency band that is difficult for seniors to catch. b) Utterance: tempo is designed 15% slower than standard speech of PaPeRo. c) Words and phrases: Follow the display (more simple them). Usability Design. The screen layout was designed based on the usability design criteria. Buttons that a user can operate have consistency in display position and shape [4]. PaPeRo's motions are designed to navigate the user's operation (eg. PaPeRo’s faces direction indicates buttons that user can choose). Reducing Mental Load. It is necessary that the user build a mental model of the system so that the user may freely use the system. Senior feels more difficulty to build a new mental model than younger [5]. In addition, if their cognitive load is large, the false mental model will be built in them [5]. Thus, it is important to reduce their mental loads. In order to reduce their mental loads, the authors defined an interaction design policy. The system takes the initiative of the interaction. The user answers when it is necessary in the flows that the system handles. This policy aims to simplify the mental model which should be built.to reduces the user’s mental loads.

Interaction Design of Encouraging Daily Healthcare Habit with Communication Robots

441

3.2 Enjoyment “Enjoyment” is defined to change the interaction from “Operation” to “Play”. It will enable the users to keep using it every day without getting tired. The elements of the “Enjoyment” defined by the authors are shown below such as “conversational interaction”, “surprise”, and “presence”. Conversational Interaction. To use a communication robot does increase conversational interaction [6]. Thus, the authors decided to use PaPeRo as a conversational medium. Surprise. A surprise in the interaction is an unexpected reaction. It can be achieved by applying the patterns of “Two-man Stand-up Comedy” (in Japanese “MANZAI”) to robot’s dialogues. For example, when the robot get data on user's pedometer, he speaks number of steps and approximate distance with a funny metaphor. (e.g. “The distance is same as five times of height of Big Buddha in Nara.”) This is a combination of two MANZAI patterns which are called “the story-line and the punchline” and “non-sense” [7]. Presence. Make a user to keep using it every day without getting tired, it is necessary to attract the attention of the user to the system. On the other hand, the attention should not to disturb the user’s daily life. That means design of presence of the system needs to draw attention without interrupting user's daily life. The authors employed a function of “time signal” for this system and designed it based on patterns of “MANZAI”[7]. The time signal play “cuckoo clock” sounds. In Japan, “cuckoo clock” was very popular in old time. This sense of the “outdated” will archived “non-sense” which is one of the “MANZAI” patterns.

4 Findings and Strategies The system takes initiative of interactions. Thus, the users might hold the impression being controlled by the system. It is an important problem which should be solved. It is necessary to eliminate the impression that the user is being controlled by the system. It is also necessary to avoid to increase user's cognitive load. In addition, it is also required to enable the user to use the system every day. The authors tried two approaches to solve this problem. 4.1 Reduce the Negative Impression Well designed conversational interaction will change the impression of interaction from “operation” to “play”. 4.2 Create New Appealing Value PaPeRo’s behavior has variety of enjoyable elements, such as patterns of MANZAI, sound effects, etc. It is proposed to create appealing value for users by such interactions.

442

J. Osada et al.

5 Results Overview An experiment was performed with ten subjects for one month. As a result of analysis of the system record, it is found that all subjects used the system at least once a day or more. After the experiment, the subjects answered to a questionnaire. For a question concerning user's impression, the majority answered “It is enjoyable even if we hear it every day” about the utterance of PaPeRo in this system [8].

6 Summary The robot guided interaction system for motivating elderly people (or health support system for seniors) had developed. The reduction of the cognitive load is an important issue. In addition, it is necessary to eliminate the impression that the user is controlled by the system. Proposed conversational interaction design was changed the user’s impression of interaction from “operation” to “play”. The Authors found that designing interactions as conversation will create new appealing value for users. The findings and new definition of features will inspire new aspects for interaction design of an interactive system with communication robots and voice interface.

References 1. The Cabinet Office, Government of Japan: White Paper on Aging Society 2007 edn., ch. 3, Tokyo Office of Government Public Relations (2007) (in Japanese) 2. Sasama, R., Yamaguchi, T., Yamada, K.: An Experiment for Motivating Elderly People with Robot Guided Interaction. In: HCII (to be appered, 2011) 3. IAUD: IAUD UD matrix Information and Case study Trove, International Association for Universal Design (2010) (in Japanese) 4. Nielsen, J.: Usability Engineering. Morgan Kaufmann, San Francisco (1993) 5. Mori, R., Yamaoka, T.: Experiment on Interaction between Elderly User and Product with a Card Game. Bulletin of JSSD 55(2), 19–28 (2008) (in Japanese) 6. Osada, J.: Scenario and Design Process of Childcare Robot PaPeRo. In: Proceedings of the Conference on ACE 2006 (2006) 7. Osada, J.: Do you like “Kawaii” Robots? In: Proceedings of the Conference on ACE 2010 (2010) 8. Yamaguchi, T., Osada, J., Sasama, R., Yamada, K.: Encouraging Daily Healthcare Habit with Communication Robots. In: 14th International Conference on Human-Computer Interaction (HCII 2011) (to be appeared)

Can Digital Signage Help Consumers Eat Healthier? Anicia Peters1 and Brian Mennecke2 1

Human Computer Interaction Program, Virtual Reality Applications Center, 1620 Howe Hall 2 Supply Chain and Information Systems, College of Business, 3313 Gerdin, Iowa State University, Ames, Iowa, 50011, USA [email protected], [email protected]

Abstract. Digital signage has become prevalent in public spaces, but consumers are faced with many choices from various displays with the result that consumers often ignore important information. To “recover” customer attention, various retailers have recently begun to deploy dynamic digital menu boards, which are plasma screens that combine the power of video with multimedia content. We examine the effect of these boards on healthy eating, a topic that has been on the national agenda for some time as Americans struggle with obesity. The introduction of a specialized form of dynamic digital menu boards in fast food restaurants has the potential to increase unhealthy eating if retailers were to use them to “upsell” consumers to higher margin but less healthy menu items. By the same token, these boards could also be used to influence consumers to make healthier food choices resulting in benefits to the consumer and society. Keywords: digital dynamic menu boards, digital signage, healthy eating, consumer decision-making, vividness.

1 Introduction Our research question focuses on the effects of video and static image food ads in dynamic digital menu boards on consumer decision-making processes and, specifically, on healthy food choices. Digital signage is a type of public computergenerated plasma display screen that presents multimedia content [3]. Due to the prevalence of these displays in public spaces, the quest for more attention-grabbing features, and significant advances in technology has enabled digital signage venders to offer signage with advanced functionality and eye-catching content. Recently, many fast-food restaurants have begun to replace traditional menu boards with dynamic digital menu boards that are capable of displaying video and live news feeds [13]. A retailer can take advantage of one of digital signage’s prominent features; that is, the potential to immediately update content at the source outlet or from a centralized location [3, 13]. As an illustration of the importance of these emerging technologies, fast food restaurants such as Wendy’s have already begun to deploy and test video menu boards [13]. Wendy’s uses these technologies to embed video ads with the goal of “grabbing” the attention of the user and redirecting their focus to particular menu items. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 443–447, 2011. © Springer-Verlag Berlin Heidelberg 2011

444

A. Peters and B. Mennecke

Further, because the dynamic digital menu board is used to replace the static menu the digital boards are the only source of information about product offerings and, therefore, the consumer will by necessity attend to the content of the display when placing orders. Of course, while the primary content of these menu boards will display product features such as price and quantity, they can be used to display other product features such as nutritional information or to enhance the appearance of product offerings. As digital signage becomes more common, it will be more likely that consumers will filter out important contents such as nutritional information or other health related messages; therefore, understanding the factors that influence consumer perceptions about these displays and the display content is important.

2 Information Overload and Vividness Traditional static menus are commonly “cluttered” with information, which implies that dynamic digital menu boards containing video and moving images might foster additional information overload. Researchers have observed a “display blindness” that occurs as part of a coping strategy to deal with situations where displays offer abundant and cluttered information [7, 9]. In these circumstances, consumers ignore digital signage because it is often irrelevant to their task or is perceived as a distraction [3, 9]. Dynamic digital menu boards and the associated “clutteredness” might also contribute to information overload and negatively affect decision-making. Information overload occurs when an individual perceives that too much information needs to be processed within a given time and, as a result, the individual feels overwhelmed [1, 5, 8]. Research suggests that video content in digital signage stands out as an attention grabber due to its liveliness and video has the ability to overcome the “display blindness” effect [7, 9]. In our studies, we hypothesize that the vividness of video will “grab” the consumer’s attention and influence decision-making. We use the definition of vividness offered by Nisbett and Ross that “…information may be described as vivid, that is, as likely to attract and hold our attention and to excite the imagination to the extent that it is a) emotionally interesting, b) concrete and imagery provoking, and c) approximate in a sensory, temporal, or spatial way” [10]. We will design our manipulations with static images, text, and video in one condition to examine Taylor and Thompson’s thesis that suggests that a vividness effect has not been conclusively demonstrated as most studies manipulated vivid and non-vivid information separately and used a between participants contrast [12].

3 Consumer Decision-Making and Healthy Eating In order to investigate whether consumers’ decision-making can be influenced by the vividness of video ads and subsequently whether consumers can be influenced to make healthier food choices, we need to understand consumer decision-making processes. We used the basic 5-step consumer decision-making model of Engel et al. [4] due its simplicity. The steps are: a) motivation and need recognition, b) search,

Can Digital Signage Help Consumers Eat Healthier?

445

c) alternatives search, d) choice and e) outcomes. Nevertheless, we also realize that many moderating factors are present that may cause consumers to vary the decision rules they use and the process they undertake to make a decision [2, 6]. In our research model we will consider moderating factors such as the characteristics of the consumer, the relevance of the ad content to the task, the preference of the consumer, information overload conditions, and information display formats. At first, our supposition was that video displays would primarily function as a recommender system whereby the ads influence cognitive focus and influence decision-making strategies. Subsequently, we recognized that these technologies had the potential to positively influence choices made regarding healthy eating. For example, industry reports suggest that venders earn positive ROI from dynamic digital menu boards because consumers often preferentially choose offerings in video ads even when prices are higher [13]. As a result, video displays can potentially “guide” consumers to make healthy food choices when the display content includes healthy alternatives. Our study design takes into consideration that healthy food choices are subjective; for example, consumers might automatically assume that salads are healthy, but do not consider nutritional information for salad dressings. Because healthy food choices can be depicted as appetizing in video ads, we hypothesize that the vividness of moving imagery in the video ads will attract attention and lead consumers to evaluate their decisions in the context of health considerations. Further, if these displays include nutritional information we might be able to encourage consumers to focus on these criteria during the information search stage. Consumers might even truncate the information search stage for nutritional information when faced with “healthy” food options that are displayed using attractive and “eye-catching” imagery.

4 Hypotheses We have developed a research model from which we derive hypotheses as follows [11]: • H1: When a consumer is faced with too much information, a video ad in a dynamic menu board with greater vividness would be more likely to draw attention regardless of whether it features healthy or unhealthy options. • H1a: A video with more motion, bright colors, realistic representations, sharp images, and distinctive and unique content will be more likely to attract the attention of the consumer. • H2: The vividness of video will lead to the evaluation of fewer alternatives regardless of healthy choices. • H2a: The vividness of video will reduce the consumer’s perception of decision complexity. • H2b: The vividness of video will lead to high levels of involvement and reduce perceptions of decision complexity by allocating resources necessary to evaluate the stimuli fully and completely. • H3: Under conditions of information overload, the video will become a reference heuristic regardless of whether the content includes healthy information.

446

A. Peters and B. Mennecke

• H3a: The availability bias of the video will increase the likelihood of the featured item being purchased regardless of whether it was a healthy or unhealthy option. • H3b: The persuasiveness of the vividness effect will increase the likelihood of the featured item being purchased regardless of whether it was a healthy or unhealthy option. • H4: The vividness of video will lead to positive comparisons of the product that is actually consumed. • H4a: Positive judgments of the food item will lead to positive evaluations of the video ad as a recommender. We will conduct studies using a combination of methods such as laboratory experiments, a quasi-experimental design where we will manipulate digital signage content in a Café setting, and field studies. Eye-tracking will also be utilized.

5 Conclusion In this poster, we have hypothesized that consumers will use video ads as reference heuristic or as a recommender for which food choices they should make under conditions of severe cognitive load or simply as novices entering a specific fast food outlet. Product venders and other industry representatives suggest that dynamic digital menu boards are effective at influencing consumer behavior. As a result, we believe that these technologies could be used to improve healthy eating if they were used to promote products with healthier attributes. Although we have not yet collected data, our preliminary informal observations of peoples’ buying behavior in relation to viewing menu board items suggest that these technologies do have the potential to influence decision makers. When we first encountered these technologies, our first reaction was that these technologies had great potential to “up sell” customers to higher margin products but we subsequently realized that these technologies could also be used to promote healthy eating. We are hopeful that the results of this research will demonstrate that dynamic digital menu boards do indeed have the potential to positively influence consumers’ healthy eating behaviors and that venders will likewise adopt these technologies to promote healthy food alternatives.

References 1. Bettman, J., Luce, M., Payne, J.: Constructive Consumer Choice Processes. Journal of Consumer Research 25, 187–217 (1998) 2. Blackwell, R., Miniard, P., Engel, J.: Consumer Behavior, 9th edn. Harcourt, Inc., New York (2001) 3. Burke, R.: Behavioral effects of digital signage. Journal of Advertising Research 49(2), 180–186 (2009) 4. Engel, J., Blackwell, R., Kollat, D.: Consumer Behavior, 3rd edn., Hinsdale, Dryden (1978)

Can Digital Signage Help Consumers Eat Healthier?

447

5. Eppler, M., Mengis, J.: The Concept of Information Overload: A Review of Literature from Organization Science, Accounting, Marketing, MIS, and Related Disciplines. The Information Society, an International Journal 20(5), 1–20 (2004) 6. Hoyer, W.: An Examination of Consumer Decision Making for a Common Repeat Purchase Product. Journal of Consumer Research 11, 822–829 (1984) 7. Huang, E., Koster, A., Borchers, J.: Overcoming Assumptions and Uncovering Practices: When does the Public really look at Public Displays? In: Indulska, J., Patterson, D.J., Rodden, T., Ott, M. (eds.) PERVASIVE 2008. LNCS, vol. 5013, pp. 228–243. Springer, Heidelberg (2008) 8. Jacoby, J., Speller, D., Kohn, C.: Brand Choice Behavior as a Function of Information Load. Journal of Marketing Research 11, 63–69 (1974) 9. Muller, J., Wilmsmann, D., Exeler, J., Buzeck, M., Schmidt, A., Jay, T., Kruger, A.: Display Blindness: The Effect of Expectations on Attention towards Digital Signage. In: Tokuda, H., Beigl, M., Friday, A., Brush, A.J.B., Tobe, Y. (eds.) Pervasive 2009. LNCS, vol. 5538, pp. 1–8. Springer, Heidelberg (2009) 10. Nisbett, R., Ross, L.: Human Inference: Strategies and Shortcomings of Social Judgment. Prentice-Hall, New Jersey (1980) 11. Peters, A., Mennecke, B.: The Role of Dynamic Digital Menu Boards in Consumer Decision Making. In: CHI ACM Conference on Human Factors in Computing Extended Abstracts. ACM Press, New York (2011) 12. Taylor, S., Thompson, S.: Stalking the Elusive “Vividness” Effect. Psychological Review 89(2), 155–181 (1982) 13. The Buzz: Installation spotlight: POP appeal, http://svconline.com/ digitalsignage/features/digital_signage_wendys_0603/

Constraint-Based Nurse Rostering for the Valpara´ıso Clinic Center in Chile Renzo Pizarro1, Gianni Rivera1 , Ricardo Soto1 , Broderick Crawford1,2, Carlos Castro2, and Eric Monfroy2,3 1

Pontificia Universidad Cat´ olica de Valpara´ıso, Chile Universidad T´ecnica Federico Santa Mar´ıa, Chile 3 CNRS, LINA, Universit´e de Nantes, France {renzo.pizarro.h,gianni.rivera.r}@mail.pucv.cl {ricardo.soto,broderick.crawford}@ucv.cl {carlos.castro,eric.monfroy}@inf.utfsm.cl 2

Abstract. The nurse rostering problem consists in assigning working shifts to each nurse on each day for a certain period of time. In particular, for the Valpara´ıso Clinic Center, the problem comprises about 1600 assignments that must consider requirements related to minimal area or floor allocation as well as legal regulations. This planning is a difficult and time consuming task that currently is done by hand yielding often unsatisfactory results. In this paper, we provide a description of such a real-world problem and we show how it can be modeled and solved with constraint programming. Using this approach we provide an automatic generation of such rosters in a few seconds instead of by hand in some days.

Keywords: Constraint Satisfaction, Rostering.

1

Introduction

Nurse rostering problems (NRPs) are combinatorial problems which consist in generating rosters where required shifts are assigned to nurses over a scheduling period satisfying a number of constraints. Generic solutions implemented and designed for such problems are not always compatible with all hospital, clinics, and medical centers in general due to the wide variety of constraints. Number of nurses, legal regulations –which commonly differs from one country to another– as well as unexpected personnel wishes make the problem variant and hard. As a consequence, hospitals design rosters manually or attempt to implement a tailored system that satisfies their requirements and specific constraints. Generating a nurse schedule by hand often requires a lot of time and effort, and usually fails to satisfy the whole set of constraints that are critical for the hospital operation. The Valpara´ıso Clinic Center, located in the fifth region of Chile, established in 2004, nowadays has 60 nurses, 60 doctors and receives around 2000 patients per month. The building is organized by medical areas: Urgency, Medical Center, C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 448–452, 2011. c Springer-Verlag Berlin Heidelberg 2011 

Constraint-Based Nurse Rostering for the Valpara´ıso Clinic Center in Chile

449

Operating Rooms, Intensive-care Unit, and Hospitalization. Some of the main administrative processes are already automated, however, nurse rostering is done by hand, which is known to be time consuming and error-prone. Hence, a main goal is to design a system for automating the roster generation. Several approaches have been proposed to solve the traditional NRP. Earlier techniques were based on mathematical programming (MP). For instance, traditional methods such as linear, integer, and goal programming were employed to solve NRPs [11,10,13]. However, a main disadvantage is that NRPs may have too many or aplication-specific constraints to allow for a MP formulation. Other methods include metaheuristics such as tabu search [6,4], genetic algorithms [3,2], simulating annealing [12], and ACO [5]. Some generic solutions using constraint logic programming [8] have also been proposed, for instance INTERPID [1] is an industrial prototype for roster generation. Another approach proposes to use soft global constraints for an efficient NRP solving [9]. In this paper, we focus on presenting a real case of nurse rostering. We model the NRP at the Valpara´ıso Clinic Center as a Constraint Satisfaction Problem (CSP) and then we solve it via state-of-the-art Constraint Programming (CP) techniques. Our system has been implemented in the Ecli pse solver and generates suitable rosters within some seconds instead of manually some days. This paper is organized as follows: Section 2 gives an overview of CSPs and solving techniques. The Nurse Rostering Problem at the Valpara´ıso Clinic Center is described and modeled in Section 3. Finally, we conclude and we give some directions for future work.

2

Constraint Programming

Constraint Programming is a powerful programming paradigm devoted to the efficient resolution of constraint-based problems. It draws on methods from operational research, numerical analysis, artificial intelligence, and programming languages. CP has been largely used in different application areas, e.g, computer graphics, engineering design, database systems, scheduling, and even in molecular biology [7]. In CP, a problem is formulated as a Constraint Satisfaction Problem (CSP). This representation mainly consists in a sequence of variables lying in a domain, and a set of relations over such variables, namely the constraints. Formally, a CSP P is defined by a triple P = X , D, C where: – X is an n-tuple of variables X = x1 , x2 , . . . , xn . – D is a corresponding n-tuple of domains D = D1 , D2 , . . . , Dn  such that xi ∈ Di , and Di is a set of values, for i = 1, . . . , n. – C is an m-tuple of constraints C = C1 , C2 , . . . , Cm , and a constraint Cj is defined as a subset of the Cartesian product of domains Dj1 × · · · × Djnj , for j = 1, . . . , m. A solution to a CSP is an assignment {x1 → a1 , . . . , xn → an } such that ai ∈ Di for i = 1, . . . , n and (aj1 , . . . , ajnj ) ∈ Cj , for j = 1, . . . , m.

450

2.1

R. Pizarro et al.

CSP Solving

The basic CP idea for solving CSPs is to build a tree structure holding the potential solutions by interleaving two main phases: enumeration and propagation. In the enumeration phase, a variable and a value from its domain are chosen to create a tree branch. In the propagation phase, a consistency property is enforced to prune the tree, i.e., the values that do not lead to any solution are temporarily deleted from domains. In this way, the exploration does not inspect unfeasible instantiations accelerating the whole process. Algorithm 1 depicts a general procedure for solving CSPs. The goal is to recursively generate partial solutions, backtracking when an inconsistency is detected, until a result is reached. The algorithm uses two data structures: inst and D. The former holds the instantiations while the latter the set of domains. The variable k represents the current level of the tree and success is a boolean variable to be set to true when a solution is found. At line 3, a value is chosen. Then, the instantiate function is responsible for building the partial solutions and assigning them into the inst array. The consistent function decides whether the current instantiation can be extended to a full solution; additionally, it set success to true if the current instantiation is a solution. At line 9, the propagate procedure attempts to prune the tree by enforcing a consistency property on the constraints of the problem. At the end, restore reinitializes the k variable’s domain. Algorithm 1. A general procedure for solving CSPs 1: procedure solve(k : integer, inst : array) do 2: while D[k] = {} and not success do 3: a ← choose value f rom(D[k]) 4: inst ← instantiate(inst, k, a) 5: if consistent(inst, k, success) then 6: if success then 7: print solution(inst) 8: else 9: propagate(k, D, f ailure) 10: if not f ailure then 11: l ← choose variable() 12: solve(l, inst) 13: end if 14: end if 15: end if 16: end while 17: restore(k); 18: end procedure

3

The NRP Model

The Valpara´ıso Clinic Center uses two kinds of shifts: – Day Shift (D): starts at 8:00 AM and ends at 8:00 PM. – Night Shift (N): starts at 8:00 PM and ends at 8:00 AM.

Constraint-Based Nurse Rostering for the Valpara´ıso Clinic Center in Chile

451

Shifts must be assigned in the following order: Day 1: D, Day 2: N, Day 3: Off, Day 4: Off. A complete planning considers 28 days as it corresponds to the lowest common multiple of 4 (a cycle of nurse shifts) and 7 (the week). Hence, for a nurse i ∈ {1, . . . , nurses} and a day j ∈ {1, . . . , 28}, we consider the following set of variables, where 0 denotes day off, 1 denotes day shift, and 2 denotes night shift. Vi,j ∈ {0, 1, 2} The sequence: Day 1: D, Day 2: N, Day 3: Off, Day 4: Off, is modeled by means of five constraints. – A day shift must be followed by a night shift, for i ∈ {1, . . . , nurses} ∧ j ∈ {1, . . . , 27} (Vi,j = 1 ⇒ Vi,j+1 = 2) – A night shift must be followed by an off day, for i ∈ {1, . . . , nurses} ∧ j ∈ {1, . . . , 27} (Vi,j = 2 ⇒ Vi,j+1 = 0) – Two off days must be followed by a day shift, for i ∈ {1, . . . , nurses} ∧ j ∈ {1, . . . , 26} (Vi,j = 0 ∧ Vi,j+1 = 0) ⇒ (Vi,j+2 = 1) – A night shift must be followed by two off days, for i ∈ {1, . . . , nurses} ∧ j ∈ {1, . . . , 26} (Vi,j = 2) ⇒ (Vi,j+1 = 0 ∧ Vi,j+2 = 0) Then, we include the occurrences global constraint to ensure the required amount per day of day shifts, night shifts, and off nurses1 . The global constraint occurrences(V alue, V ars, N ) ensures that V alue occurs N times in the V ars list. Then, for j ∈ {1, . . . , 28}: – nurses/2 nurses must be off per day occurrences(0, Vj , nurses/2) – nurses/4 day shifts per day occurrences(1, Vj , nurses/4) – nurses/4 night shifts per day occurrences(2, Vj , nurses/4) Such a model has been implemented in the Ecli pse solver and launched on a 3.06GHZ Intel Core2 Duo with 2GB RAM computer running Ubuntu. Rosters are generated in less than one second for 60 nurses, less than 2 seconds for 200 nurses, and less than 7 seconds for 500 nurses. 1

Let us note that the amount of shifts required per day are regulated by administrative criteria of the Clinic Center.

452

4

R. Pizarro et al.

Conclusion and Future Work

In this paper, we have presented a simple nurse rostering model for the Valpara´ıso Clinic Center. The model has been implemented and solved in the Ecli pse solver allowing a fast nurse assignment. The solution introduced here is ongoing work, and it can clearly be extended by considering paramedics and specific rosters of some chief nurses (not considered here) as well as personal wishes. Including these new requirements may demand to investigate specific and global constraints such as the regular or the among constraint.

References 1. Abdennadher, S., Schlenker, H.: Nurse Scheduling using Constraint Logic Programming. In: Proceedings of AAAI/IAAI, pp. 838–843 (1999) 2. Aickelin, U., Dowsland, K.A.: An Indirect Genetic Algorithm for a NurseScheduling Problem. Computers & OR 31(5), 761–778 (2004) 3. Aickelin, U., Dowsland, K.A.: Exploiting Problem Structure in a Genetic Algorithm Approach to a Nurse Rostering Problem. Journal of Scheduling 3(3), 139–153 (2001) 4. Burke, E., Causmaecker, P.D., Berghe, G.V.: A Hybrid Tabu Search Algorithm for the Nurse Rostering Problem. In: McKay, B., Yao, X., Newton, C.S., Kim, J.-H., Furuhashi, T. (eds.) SEAL 1998. LNCS (LNAI), vol. 1585, pp. 187–194. Springer, Heidelberg (1999) 5. Crawford, B., Castro, C., Monfroy, E.: A Constructive Hybrid Algorithm for Crew Pairing Optimization. In: Euzenat, J., Domingue, J. (eds.) AIMSA 2006. LNCS (LNAI), vol. 4183, pp. 45–55. Springer, Heidelberg (2006) 6. Dowsland, K.A.: Nurse Scheduling with Tabu Search and Strategic Oscillation. European Journal of Operational Research 106, 393–407 (1998) 7. Rossi, F.: Handbook of Constraint Programming. Elsevier, Amsterdam (2006) 8. Jaffar, J., Maher, M.J.: Constraint logic programming: A survey. J. Log. Program. 19(20), 503–581 (1994) 9. M´etivier, J.-P., Boizumault, P., Loudni, S.: Solving Nurse Rostering Problems Using Soft Global Constraints. In: Gent, I.P. (ed.) CP 2009. LNCS, vol. 5732, pp. 73–87. Springer, Heidelberg (2009) 10. Miller, H.E., William, P., Gustave, J.R.: Nurse Scheduling Using Mathematical Programming. Operations Research 24(5), 857–870 (1976) 11. Semet, F., Vovor, T., Jaumard, B.: A Generalized Linear Programming Model for Nurse Scheduling. European Journal of Operational Research 107, 1–18 (1998) 12. Thomson, G.M.: A Simulated Annealing Heuristic for Shift-Scheduling Using Non-Continuously Available Employees. Computers and Operations Research 23, 275–288 (1996) 13. Warner, D.M., Prawda, J.: A Mathematical Programming Model for Scheduling Nursing Personnel in a Hospital. Management Science 19(4), 411–422 (1972)

Connecting with Dysphonia: Human-Computer Interface for Amyotrophic Lateral Sclerosis Patients Chun-Yang Su1 and Ju-Joan Wong2 1 Department and Graduated Institution of Industrial Design, College of Management, Chang Gung University, Taiwan R.O.C, 259 Wen-Hwa 1st Road, Kwei-Shan Tao-Yuan,Taiwan, 333, R.O.C. 2 Department of Industrial Design, National Yunlin University of Science & Technology, 123 University Road, Section 3, Douliou, Yunlin 64002, Taiwan, R.O.C.

Abstract. This research studied how Amyotrophic Lateral Sclerosis patients can communicate after losing speech and typing abilities. To create a friendly and useful HCI system, this research studied the graphical user interfaces (GUI) through participant observations, to understanding how to innovate a better communication device for ALS patients and the elderly, to gaining a better quality of life. Keywords: Amyotrophic lateral sclerosis (ALS), pictograph, human-computer interface (HCI), dysarthria, universal design (UD).

1 Introduction This study examined human-computer interfaces (HCI) for amyotrophic lateral sclerosis (ALS) patients that are able to communicate after losing the abilities of speech and to type. Creating an accessible HCI requires a universal design applied to people with disabilities or the elderly. To establish the needs of patients, we followed typical user-centered design procedures and interviewing ALS patients. We discovered what ALS patients wanted for daily living and what they really needed to create a new HCI that overcomes disabilities caused by disease. ALS is a motor neuron disease and is also known as Lou Gehrig’s disease. ALS causes muscles to degenerate until the patient becomes disabled. Most patients develop dysarthria and have dysfunctional hands. Previous research has shown another symptom is shoulder dysfunction. The health of the patient soon declines after this symptom. Prior to shoulder failure and death, other symptoms such as foot drop, fasciculation, spastic gait, and respiratory problems occur as well.[2] ALS does not lead to a vegetative state. ALS patients are conscious and can think and have emotions and feelings. Many ALS patients share their perspectives on life through broadcasting, publishing, and on the internet. They need to communicate with each other to realize they are not alone. Creating an HCI system for ALS patients has several distinct problems. The first problem is dysarthria. When a patient is unable to talk to others, they need a tool that C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 453–457, 2011. © Springer-Verlag Berlin Heidelberg 2011

454

C.-Y. Su and J.-J. Wong

allows them to communicate. Unfortunately, HCI systems are always manually controlled, using a mouse or a keyboard. However, ALS patients often have split hand syndrome and are unable to grip a mouse. Shoulder dysfunction makes them unable to lift up their hands to use a keyboard. The design of the user interface must be simple so that it can be easily controlled by ALS patients. To create a friendly and useful HCI system, we studied the use of graphical user interfaces (GUI). In 1973, Xerox PARC developed a personal computer called Xerox Alto, which was the first GUI computer. To understand if GUIs were appropriate, Xerox interviewed users and asked about their experience using the GUI. As a tool to be used in everyday life, an ALS HCI system must be able to be learned quickly and easily. Research has shown that patients are often elderly. After they become ill, patients typically had one year to figure out how to use the adapted devices. A method to help patients get use to the adapted device is to translate the functions into graphics through pictography. In computer science, this is also known as iconification. An HCI system can be a part of a cloud computing system on the client side. Patients can use the client interface to communicate with each other. By sharing information, patients will change their roles and form spontaneous self-help groups. Moreover, they can connect with different hospitals to share their situation. The attendants and other patients can understand them immediately.[3] To assess an HCI system, we used four steps. The first step is user identification. This creates a list of relevant characters including potential users. The second step is to gain the user requirements by making proposed applications are supported. The third step is to develop prototypes. The fourth and final step is to implement a final version of the application and create formal usability tests. (see Fig. 1)

Fig. 1. HCI system design method

An HCI system must be planned according to universal design principles to create a useful communication tool that will service the disabled and elderly by making it easily controllable. By using this cloud computing system, patients can exchange information, chat with each other, and even have access to entertainment. This project could help ALS patients and the elderly achieve a better quality of life.

Connecting with Dysphonia: Human-Computer Interface

455

2 Method Through participant observations, there is a lot of software developed for ALS patients to enhance their experience by using computers. These systems are controlled by different devices. Each has their own unique interface because of the different input methods. When patient's symptoms escalate, they must learn a new, yet unfamiliar device. They must try to communicate with the new system with an unfamiliar interface. When patients have dysarthria symptoms, they may use a normal keyboard until their hands are unable to function. Then the one-click control button fulfilled their needs for the first year. After their hands become completely dysfunctional, many patients already knew how to use the computer before they became ill. They were able to send e-mail, write articles, and chat online. In these cases, patients were already used to working a simple operating system, meaning that patients did not have to learn a new system. Setting up selections and key-in tools on a simple operating system interface was a better choice.

3 Result Grid 2 is software that allows patients to control a computer easily with onscreen keyboards. By using a one-click device, patients can select words by moving a target. The selection tool has vertical and horizontal markers. The vertical marker will move until the patient selects, they can select the horizontal marker. The point where they cross is the icon that the patient selected. In Chinese, patients will select about eight targets and then choose the words they really want after. This system can be installed on a normal operating system such as Windows. If the patient was a pc user, the tools will be learned quickly. In other cases, Patient A was using Grid 2, to key in about 100 Chinese words in an hour. Patient A used it to chat with friends and co-workers before. Patient A shares his story with others not only in the hospital but also in public. “One day, I received a message from Patient A. He thanked me, and told me that he is better now," said his nurse. Patient A has symptoms of dysarthria, and his feet and hands were disabled. He cannot move his mouth. The last muscle he can be use, is the left orbicularis oculi muscle. By moving his left eyebrow, Patient A uses Grid 2 with one-click. He can use the computer with the familiar system with only his eyebrow. In some cases, to patients that had not used a computer before they become ill, especially middle-aged patients, learning computer is a difficult challenge. Because the symptoms of ALS symptom worsen quickly, it is not suitable to learn a complex operating system. The ‘Spring’, the Maid is a choice for this type of ALS patient. The system is setup with eye-controlled equipment. The system has a simple interface that uses large icons, large buttons, and wide keyboards. Because eye-tracking systems are not precise, large icons help patient to control the computer more easily. This system applies to patients that were unfamiliar with using computers. The ‘Spring’, the Maid has a unique interface that is simple and clearly marked with large words. The patient can learn the system quickly.

456

C.-Y. Su and J.-J. Wong

Patient B never used any operating system. But, she is able to quickly key in words with ‘Spring’, the Maid. (see Fig. 2) When we interviewed patient B, she talked using this system. “Nice to meet everybody, thanks for the hospital and the system. I feel happy now,” she said. The eye-tracking system is a solution that works quickly. However, Patient C had some settings problems with this system. When we met Patient C, the nurse spent half an hour setting up the eye-tracking system. With light or physiology problems, the system is not a solution for some patients. The other reason is the cost of the system. An expensive system is only afforded by a few patients.

Fig. 2. HCI system design method

4 Conclusion The interface design needs for ALS patients will change as the patients symptoms change. As the symptom become worse, patients will need to change devices and adapt to a different interface. The suggestion for the interface is that it might be an adjustable display interface to overcome the problem of change and may be more suitable for every dysarthria patient.

References 1. Birbaumer, N., Kubler, A., Ghanayim, N., Hinterberger, T., Perelmouter, J., Kaiser, J., Iversen, I., Kotchoubey, B., Neumann, N., Flor, H.: The thought translation device (TTD) for completely paralyzed patients. IEEE Trans. Rehabil. Eng. 8(2), 190–193 (2000) 2. Eisen, A.: Amyotrophic lateral sclerosis: A 40-year personal perspective. J. Clin. Neurosci. 16(4), 505–512 (2009) 3. Feenberg, A.L., Licht, J.M., Kane, K.P., Moran, K., Smith, R.A.: The online patient meeting. J. Neurol. Sci. 139(suppl.), 129–131 (1996) 4. Kent, R.D., Vorperian, H.K., Kent, J.F., Duffy, J.R.: Voice dysfunction in dysarthria: application of the Multi-Dimensional Voice Program (TM). Journal of Communication Disorders 36(4), 281–306 (2003) 5. Kushniruk, A.W., Patel, V.L.: Cognitive and usability engineering methods for the evaluation of clinical information systems. J. Biomed. Inform. 37(1), 56–76 (2004)

Connecting with Dysphonia: Human-Computer Interface

457

6. Miller, R.G., Rosenberg, J.A., Gelinas, D.F., Mitsumoto, H., Newman, D., Sufit, R., Borasio, G.D., Bradley, W.G., Bromberg, M.B., Brooks, B.R., Kasarskis, E.J., Munsat, T.L., Oppenheimer, E.A.: Practice parameter: the care of the patient with amyotrophic lateral sclerosis (an evidence-based review): report of the Quality Standards Subcommittee of the American Academy of Neurology: ALS Practice Parameters Task Force. Neurology 52(7), 1311–1323 (1999) 7. Nijboer, F., Mellinger, J., Wilhelm, B., Matuz, T., Neumann, N., Wolpaw, J.R., Vaughan, T.M., Birbaumer, N., Kuchler, A.: The influence of psychological state and motivation on brain-computer interface performance in patients with amyotrophic lateral sclerosis - a longitudinal study. Front Neurosci. 4 (2010) 8. Nijboer, F., Birbaumer, N., Kubler, A.: Sensorimotor-rhythm-based brain-computer interface use in people with amyotrophic lateral sclerosis (ALS). Psychophysiology 41 (2004) 9. Tomik, B., Krupinski, J., Glodzik-Sobanska, L., Bala-Slodowska, M., Wszolek, W., Kusiak, M., Lechwacka, A.: Acoustic analysis of dysarthria profile in ALS patients. Journal of the Neurological Sciences 169(1-2), 35–42 (1999) 10. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Braincomputer interfaces for communication and control. Clin. Neurophysiol. 113(6), 767–791 (2002)

Assessing Health Information Websites for Inclusion of Web 2.0 Features Adam Townes and Michelle Rogers The iSchool at Drexel College of Information Science and Technology, 3141 Chestnut Street, Philadelphia, PA 19104, USA {amt74,Michelle Rogers,mlr92}@drexel.edu

Abstract. Health 2.0 is a facet of Web 2.0 that refers to special health applications where patients are able to take a more active role in their own care through the use of information communication technologies. Health 2.0 or Medicine 2.0 (used synonymously here) empowers patients not only through increased participation and interaction in online networks and communities but also enhances the user’s role from health care consumer to that of collaborator and contributor. This increased cooperation between providers and patients reflects a revised service philosophy. This expanded service philosophy emphasizes greater patient autonomy and choices regarding medical care and treatment decision.

1 Introduction The internet and Web 2.0 have taken their place as a persistent part of daily life for most people that choose or are able to participate in a networked society. Web 2.0 can be characterized as spanning the network of all networked devices, thus representing the resounding theme of networked communication, that of community defined by user participation and interaction. The creation of content and information through collective intelligence via personal, professional and casual network linkages creates richer user experiences and more user centered applications and content. This study evaluates several of the well known health oriented sites on the web on the basis of their provision of Web 2.0 features. The sites are then classified based on the number of service features they provide users to interact, comment, participate and create content. This study makes an assessment of the current state of Web/Health 2.0 features that are generally available on a varied selection of health and medically oriented websites. This will provide a more comprehensive image of the current state of online health applications in popular, government and other fairly well known sites in light of the perceived needs by patients. Increasingly Americans are being more proactive in becoming better informed in regard to their own health. Depending on their levels of education and income, the chances of their seeking health information via the internet will vary greatly with a few exceptions. These newly recognized information interactions can be credited to the growth in usage of broadband connections and advances in mobile computing. It is consequently important that sites providing health information should stay abreast of current developments and trends in user behaviors. The Pew Research Center C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 458–462, 2011. © Springer-Verlag Berlin Heidelberg 2011

Assessing Health Information Websites for Inclusion of Web 2.0 Features

459

indicates that up to 80 percent of internet users look for health information online. This is a significant amount of the U.S. population, since around three quarters of the population use the internet. [7] When viewed in a Web 2.0 context, a large portion of U.S. citizens are potentially reading and using this information in their everyday lives.

2 Web 2.0 Meeting People’s Health Information Needs Information seeking is an integral part of a person’s everyday life. In the last two decades, the internet has become an important source of information and media. The inception of Web 2.0 has only increased the internet’s usefulness as an information seeking tool. Healthcare is currently a mainstream issue on both a national and a personal level for many Americans. A recent study by the Pew Research Center, notes that about eight out of every 10 internet user’s searches for health information online. [4] The frequency of usage makes it the third most popular online activity after email and search engine usage. Research has shown that almost around nine out of every ten teens is connected to the internet somehow. [6] These numbers should come as no surprise and are indicative of trends in information search, usage, retrieval and information use environments. Health 2.0 represents a new service philosophy in the health care industry. This new service paradigm represents patients or health information seekers increased interest in health information and a desire to take on a larger role in their own care. It can be characterized by increased cooperation not only between providers but among information seekers and patients alike. Web 2.0 is well placed to facilitate this new philosophy and enhanced information service environment. Interaction on message boards allows users to discuss health concerns or issues with others that may have similar conditions or seek other the opinions of involved physicians in informing treatment decisions. Users can share favored content, resources, articles and advice. Web 2.0 can be seen to span the network of all networked devices thus forming a strong foundation of networked communication and by extension that of community. The formation of these online communities encourages the creation of content and information through the collective intelligence via personal, professional and casual network linkages. These linkages create richer user experiences as well more user centered applications and content.

3 Mobile Device and Accessibility Mobile devices are increasingly becoming a large part of frequent internet user’s means of access. The Pew Research Center indicates that fifty-seven percent of adults go online using a mobile connection; they further find that wireless internet users tend to be more heavily engaged than other uses that simply have hard line connections.[4] This would also suggest a greater intensity of usage on the part of these mobile users. Having mobile access provides users with the opportunities to perform searches at almost any time and almost anywhere. Yahoo has indicated that terms involving sexually transmitted disease, herpes and pregnancy are among the top searches performed using the mobile version of their site. [4] This might suggest that mobile devices are directly involved in an individual’s everyday life information seeking in

460

A. Townes and M. Rogers

reference to health occurrences or even worries. Thus they could be said to be more or less involved in an individual’s immediate information needs or spur of the moment decisions, at least in regard to sexual promiscuity or actions. [4] Again there are indications that this will be a consistent trend in the future. Teens, those aged between twelve and seventeen have very high rates of internet usage, with up to eighty-seven percent of U.S. teens use the internet. [7] Five years ago, around forty five percent of teens owned cell phones; by 2010 that number had risen to around seventy five percent. A staggering thirty percent rise in ownership and use. [8] The cell phone has become an integral part of many people’s live and if they statistics are any indication will continue to be so.

4 Web 2.0 Features Essentially, the main concepts that characterize Web 2.0 are user participation and interaction. Web 2.0 spans the network of all networked devices, thus representing the theme of networked communication through the creation of communities. The creation of content and information through the collective intelligence via personal, professional and casual network linkages can lead to richer user experiences and a focus on more user centered applications and content. In giving a definition for Web 2.0 Tim O’Reilly, founder of O’Reilly media named seven basic principles to characterize Web 2.0 rather than a concise definition. [3] To summarize, O’Reilly named the web as a platform, the harnessing of collective intelligence, data as the next Intel inside, end of the software release cycle or perpetual beta versions, lightweight programming models, software above the level of a single device and rich user experiences as the seven basic principles of Web 2.0. [5] An article from Modern Healthcare identifies three of them as being applicable to Health 2.0. First the service must be web based with a structure that encourages participation. The second calls for the indulgence of the wisdom of crowds or the harnessing of collective intelligence. The third is the provision of specialized data that is enhanced by analysis of the service provider and contributions by users. [2] In the context of this study, features that were assessed were identified as applying to the three principles that are named above in relation to Health 2.0. The features which were considered were provision of user registration, RSS feeds, a newsletter, message boards, a list of multimedia tools, links to social networks, personal tracking tools, mobile applications, a mobile version of the site, the availability of podcasts, blogs and a link to sharing sites such as Reddit or Digg. Each of the sites was evaluated on their inclusion of these Web 2.0 features.

5 Popular Health Information Sites and Web 2.0 Features The popular sites which the study reviewed appear to be in a transitional period. While just about all of them have incorporated some Web 2.0 features, many of them seem to be maintaining designs that are more reminiscent of Web 1.0 user experiences. The Web 2.0 features assessed as the basis of this study are meant to enhance both user experience and increase the services provided to users. When viewed in

Assessing Health Information Websites for Inclusion of Web 2.0 Features

461

light of the identified features, the sites tend to possess from three to six of the features assessed in the study. The most common features appear to be RSS feeds, newsletters, links to social networks, a list of multimedia tools, user registration and sharing links. The more interactive, collective intelligence features and content creations features are present, but tend to be less common across the websites represented in this study. They appear to be fairly well distributed across the different sites, but have a low frequency.

6 Features to Increase Interaction and Content Generation The low frequency features are the ones that are believed to encourage greater user interaction, generation of content, cooperation between patient and physician and mutual support. Interactional features such as message boards, personal tracking tools and mobile device related features such a mobile versions of site and availability of mobile applications appear to have lower frequencies of occurrence among the sites. The less common features are reflective of the expanded service philosophy that is definitive of the Health 2.0 paradigm in particular and three aforementioned Web 2.0 features that were identified as being applicable to Health 2.0. To be direct, it appears that many of the more popular health information sites have been somewhat slow to implement many of the more interactive and content generation type features or features that are related to mobile devices. As mentioned earlier, around sixty percent of adults use a mobile device to connect to the internet. These users tend to be more heavily engaged in the use of the internet. [4] Consequently assessing whether some of the more popular websites provide mobile versions could be an indication of whether they are truly meeting current user needs.

7 Discussion As a result of the mismatch between current health site offerings and the Web 2.0 landscape the following design considerations are suggested. In light of health information sites the content generation, information sharing and interaction oriented features should be incorporated. Inclusion of these features could contribute to greatly enhancing user experiences, increasing participation and the formation of supportive communities. These online communities and the generation of content through the interaction of users, composed of both patients, physicians and researchers will harness the power of collective intelligence and form a strongly health oriented archive of information that might be somewhat reminiscent of the usefulness and omnipresence of wiki type reference sources. The availability of specialized data or information to users, that is enhanced by active physician or health providers will contribute greatly to the fulfillment of potential information needs of users. Secondly the provision of mobile applications and site versions has the potential to enhance societal inclusion. Finally the Pew studies on mobile phone usage note that mobile device usage by minorities and teens tends to be higher than the rest of the U.S. population in

462

A. Townes and M. Rogers

general. Greater use of mobile technology has the potential for bridging some of the issues associated with the digital divide and information poverty.

References 1. Centers for Disease Control- Chronic Disease - At a Glance, http://www.cdc.gov/ chronicdisease/resources/publications/AAG/chronic.htm 2. Conn, J.: Health 2.0: The next generation of Web enterprises. Modern Healthcare (2007) 3. Eysenbach, G.: Medicine 2.0: Social Networking, Collaboration, Participation, Apomediation, and Openness. J. Med. Internet. Res. (2008) 4. Health Topics, Pew Research Center’s Internet & American Life Project, http://www.pewinternet.org/Reports/2011/HealthTopics.aspx 5. O’Reilly Media, What Is Web 2.0, http://oreilly.com/web2/archive/what-is-web-20.html 6. Schmidt, A.: The Young & the Wireless. School Library Journal 51(10), 44–46 (2005) 7. Teens and Technology, Pew Research Center’s Internet & American Life Project, http://www.pewinternet.org/Reports/2005/Teens-andTechnology.aspx 8. Teens and Mobile Phones, Pew Research Center’s Internet & American Life Project, http://www.pewinternet.org/Press-Releases/2010/Teens-andMobile-Phones.aspx

Encouraging Daily Healthcare Habit with Communication Robots Tomoharu Yamaguchi1, Ryohei Sasama1, Jun’ichi Osada2, and Keiji Yamada1 2

1 NEC Corporation, 8916-47 Takayama, Ikoma, Nara 630-0101 Japan NEC Design and Promotion Co. Ltd., 1753 Shumonumabe, Nakahara, Kawasaki, Kanagawa 211-8666 Japan {yamaguchi@az,r-sasama@cp,kg-yamada@cp}.jp.nec.com, [email protected]

Abstract. It is important for elderly people to be involved in local community to reduce the risk of being isolated. The authors are building a framework with communication robots for encouraging elderly people to participate in more social activities by providing local news that may be interesting. Since physical soundness is also essential for people to participate in such activities, selfmonitoring of physical conditions are involved into the framework. A robotguided interaction system is developed based on the framework so that a robot encourages the user to measure weight and blood pressure daily. The efficiency is estimated by an experiment.

1

Introduction

Nowadays, there is a lot of information on the Internet; however, few elderly people can obtain the benefits of this information [1]. It is not easy for elderly people to learn usage of a new ICT system [2,3]. Thus, the authors proposed a Robot-guided Interaction Framework for elderly people [4]. Once the user initiates an interaction, a communication robot initiates the following interaction sequences. The user can simply follow and respond to the guiding robot, and is not required to learn any operational sequence or mental model. An experiment on such guiding robots was performed with ten elderly subjects, and investigated how long they can use the system. As a result of an experiment, all subjects kept using the system almost every day until the end of the experiment period [5]. According to this result, the authors considered that the Robotguided Interaction Framework has certain efficiency for elderly people. Thus, the authors tried to apply it for motivating daily healthcare activities. Physical soundness is also essential for people to participate in social activities. It is well-known that weight and blood pressure are typical indices to check possibility of lifestyle disease. However it is not easy for most of people to measure them daily only with his/her own motivation. If a kind assistance is provided by someone who is close to her/him, many people can be encouraged to achieve daily self monitoring with less effort. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 463–466, 2011. © Springer-Verlag Berlin Heidelberg 2011

464

T. Yamaguchi et al.

2 Motivating People by Communication Robots The authors are building a framework for encouraging elderly people to participate in more activities triggered by interactions with communication robots. By providing news of local activities by the robots, it is expected that participation of elderly people in the activities will increase. It is also expected that they take actions for healthcare with guidance of the robots. 2.1 Activity and Communication Enhancement Communication robots are delivered to each elderly person’s home and placed at common places where people gather (Fig. 1). First, the robot at home provides news, and encourages participation in an activity, so that the elderly people have more opportunities to go out and meet others. Then, a robot at a common place proposes topics to enhance conversation among people who meet at that location. Finally, the robot at home also encourages the exchange of online messages with people who met at the common place in order to maintain longer and better relationships. Interactions between the robot and the user are designed based on Robot-guided Interaction Framework which decreases difficulties of using information systems for elderly people. Once the user initiated an interaction, a communication robot takes initiative of the interaction sequences.

Fig. 1. A Framework for Communication Enhancement for Elderly People

Encouraging Daily Healthcare Habit with Communication Robots

465

2.2 Encouraging Daily Self Monitoring Robot-guided Interaction Framework is applied also for encouraging a user to measure weight and blood pressure for daily healthcare. It is well-known that weight and blood pressure are popular index of physical condition of a human. Electric weight scale and electronic sphygmomanometer are popular in the market, and many people own one or more of those. However, it requires some effort to use them continuously for self monitoring in the busy life. It may be simply depends on strength of motivation of people. If a kind assistance is provided by someone else, many people can be encouraged to use those equipments daily with less effort. Thus, the authors build a communication robot which is connected to weight scale and electronic sphygmomanometer to encourage the user to use those healthcare equipments daily. In addition to a sequence of interactions to obtain news of local activities, the robot guides to measure weight and blood pressure step by step in a natural way [6]. The user can simply respond to the robot without learning any operational sequence to achieve all measurements. This efficiently lowers the barrier to use those healthcare equipments.

3 Experiment and Results An experiment is performed with 10 subjects (8 males and 2 females) to estimate the efficiency of the system. Ages of the subjects are between 70 and 85. They are mostly in good conditions without major disability. A few of them have hypertension and taking hypotensive drug to lower the blood pressure. A communication robot, a weight scale and an electronic sphygmomanometer are delivered to each subject (Fig.2), and investigated how long and how often s/he uses the system.

Fig. 2. Equipments at home. (Web Terminal, Robot and Sphygmomanometer. Weight scale is not shown here).

466

T. Yamaguchi et al.

As a result of 30 days experiment, all subjects kept using the system almost every day until the end of the experiment period. In average, subjects measured weight and blood pressure 0.7 times a day. According to interviews to the subjects, they had seldom or sometimes measured blood pressure before the experiment unless s/he has hypertension. 3/5 of subjects answered that they became to be more conscious about their physical conditions than they were before the experiment. Above results may implicate that the encouragement by the communication robot is effective to daily healthcare at a certain level. Further experiment with larger number of subjects is necessary for more accurate quantitative analysis.

4 Summary An interactive system with communication robots to motivate people for daily healthcare activities is introduced. The system is built based on a framework for encouraging elderly people to participate in more social activities by providing local news that may be interesting. Since physical soundness is also essential for people to participate in such activities, self-monitoring of physical conditions are involved into the framework. The robot encourages the user to measure weight and blood pressure daily. The efficiency is estimated by an experiment of 30 days with 10 subjects. Everyone continued to measure her/his weight and blood pressure until the end at 0.7 times per a day in average. Acknowledgements. This work was supported in part by the Japan Ministry of Internal Affairs and Communications through the project of “Ubiquitous Network Robots for Elderly and Challenged.” The authors would like to thank all the participants of the experiment, staffs of Uda city and Nara prefecture for kind support throughout the experiment.

References 1. Nielsen//NetRatings: Transition of Composition ratio of Web User in the Age Bracket (2006) (in Japanese), http://csp.netratings.co.jp/nnr/PDF/Newsrelease11072006_J.pdf 2. Ministry of Internal Affairs and Communications, Government of Japan: Report on Communications Usage Trend Survey 2010 Edition, ch. 3, Tokyo Office of Government Public Relations (2010) (in Japanese) 3. Harada, E., Akatsu, Y.: Cognitive Science about Usability, ch. 6, Kyoritsu (in Japanese) 4. Sasama, R., Yamaguchi, T., Yamada, K.: An Experiment for Motivating Elderly People with Robot Guided Interaction. In: 14th International Conference on Human-Computer Interaction, HCII 2011 (to be appeared, 2011) 5. Yamaguchi, T., Sasama, R., Yamada, K.: An Experiment to Motivate Elderly People by Daily Communication with Robots. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2010) Workshop on Network Robot Systems (2010) 6. Osada, J., Yamaguchi, T., Sasama, R., Yamada, K.: Interaction Design of Encouraging Daily Healthcare Habit with Communication Robots. In: 14th International Conference on Human-Computer Interaction, HCII 2011 (to be appeared, 2011)

Part VII

Learning, Education and Cultural Heritage

The Evaluation of the Applicability of Distance Education in Vocational Colleges by the Students of Erzurum Vocational College, Computer Technologies Department, Erzurum, Turkey Yusuf Ziya Ayik Ataturk University, Erzurum Vocational College, Computer Technologies Department, Erzurum, Turkey [email protected]

Abstract. In order to determine the opinions of the students about the applicability of distance education in vocational education, a questionnaire was applied to the first and second year students of Erzurum Vocational College, Computer Technologies Department. The data obtained from the survey, comprising 25 questions and directed to 154 students, were evaluated by statistical methods. The data evaluated by using SPSS 16.0 statistical packet program were also subjected to factor analysis and the 25 questions asked were reduced to 7 factors. The factors determined indicate the 61.53% of total variance. According to the indications of students, 7 factors affecting the education can be listed as follow: Quality of education, cost of education, being at a permanent work, equality, accessibility, success and habits. Keywords: Distance Education, Vocational College, Computer Technologies Department.

1 Introduction It is mainly the duty of education to train the qualified manpower to realize the social development. Education system has been carried out through formal education in a great ratio until recently. As a result of the fast growing technology and the new skills that computer and net have attached to our lives, education activities began to be carried out by modern methods independent from time and place. Usage of web-based net by computer technologies for the purpose of communication has made it possible to transfer education to distant people [10]. In an age of information, education and instruction are being carried via virtual universities, and new instruction-learning methods are improved by using satellite, fiber-optics, radio, television and web-based technologies. Distance education comes forth among these methods.

2 Historical Background of Distance Education in Turkey Distance education in Turkey was first expressed in 1927 in order to increase the literacy of the people, however, it remained only as an idea up to 1956. Ankara C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 469–473, 2011. © Springer-Verlag Berlin Heidelberg 2011

470

Y. Ziya Ayik

University introduced the distance education in 1956, and bank employees started to take education via letters [5]. The factual distance education application was started by establishing Open Education Faculty within Anatolia University after making necessary regulations in Higher Education Law. Especially with the spread of computer technologies in Turkey, some other universities apart from Anatolia University took important steps on the subject matter of distance education. Today there are a number of distance education faculties or colleges within the public and foundation universities giving graduate, undergraduate and pre-graduate education.

3 Definition of Distance Education It is difficult to make a clear definition of distance education because of its having extensive application fields and methods, and since it has a lot of factors to affect and to be affected. However, by means of different perspectives on these factors, of course, many definitions have been made. A comprehensive definition can be; It is a planned and systematic education application in which source and receivers are found in separate environments during the large part of instruction–learning processes, also which allows the chance of individualism, elasticity and independence to receivers from the point of age, objective, time, place and method, and where written and printed materials, audio–visual means and face to face education methods are provided through television and computer interactive technologies for the sake of instruction and learning [9].

4 Methods of Distance Education In distance education where learning is independent of place and time, the way that the instruction-education is fulfilled determines the method of distance education. In this system instruction–learning procedure or instructor–learner communication can be realized as synchronized, asynchronized or both together. Synchronized and asynchronized communications have superiorities or inferiorities to each other. Among the tools and technologies facilitating the learning in the desired time and place, there are electronic mail, sound mail, multi environment links and discussion groups. One of the advantages of asynchronized interaction is that it can allow the users to participate the communication from the place and at the time they desire according to their availability. Besides, it is always possible to participate all these activities again [11]. Synchronized distance learning uses tools and technologies such as conversation rooms, teleconference, video-conference, VoIP and video telephones to allow real time communication. It is relatively a less elastic choice as it is necessary to adopt the given time to participate the communication. Synchronized systems give the opportunity of benefiting more from the learning group alerting the sense of being less isolated for distance students. Feedback plays an important role in distance education, and synchronized communication allows spontaneous feedback.

The Evaluation of the Applicability of Distance Education in Vocational Colleges

471

5 Materials and Methods A questionnaire was applied to 154 people, who are first and second year students of Ataturk University, Erzurum Vocational College, Computer Technologies Department in order to determine their knowledge about distance education and their ideas on the applicability of distance education in Vocational Colleges. The data obtained from the survey were evaluated by using statistical methods. SPSS 16.0 statistical package program was used for the factor analysis of the data set. For the purpose of evaluating whether the data set is suitable for factor analysis or not, Kaiser-Meyer-Olkin (KMO) proficiency test was applied. This test is an index comparing the greatness of the observed correlation coefficients and the greatness of the partial correlation coefficients. In order to apply factor analysis to data set, the KMO ratio should be over 0.5. As the ratio rises, factor analysis application becomes more significant [4]. The KMO ratio of the questionnaire comprising 25 questions was found to be 0.791 by means of the statistical method applied. As the KMO was 0.791>0.50, and since Barlett test (sig.) was found to be significant, it can be said that data set is suitable for the factor analysis. Table 1. KMO and Bartlett Test Kaiser-Meyer-Olkin Measure of Sampling Adequacy. Bartlett's Test of Sphericity Approx. Chi-Square

,791 1,193E3

df

300

Sig.

,000

6 Survey Findings and Discussion In this study, it was aimed to research how distance education concept is perceived by Vocational College Students and how their success, whether positive or negative, is affected if the education they receive face to face is given to them through distance education. Besides, we wanted to make a statistical evaluation and determine the opinions of students about how much possible it is to carry out distance education by taking into account the factors such as characteristics of the lesson and the competence of the instructor. Seven factors were depicted at the end of the factor analysis applied on the data set replied by 154 students of Computer Technologies Department. The depicted factors explain the 61.513% of the total variance. The seven factors depicted as a result of the factor analysis can be classified as follows respectively; 1. 2. 3. 4.

Quality of Education Cost of Education Permanent Work Equality

472

Y. Ziya Ayik

5. Accessibility 6. Success 7. Habits The variable most affecting the factor quality of education is “Quality of education decreases in that the communication of learner-instructor in distance education cannot be so much as that of face to face education”. While the variable most affecting the cost of education is “Distance education is cheaper than face to face education”, the variable most affecting the permanent work is “Distance education helps those who are working to continue their education without losing their jobs”. On the other hand, “The assessment of the examinations will be more objective” most affects the factor equality, whereas the variables “There may be problems while linking to system or link cut in distance education” most affects the factor accessibility. The success factor is affected mostly by variable “The distance education for verbal courses cannot be at the same quality as that of face to face education” and the variable “The students who do not have individually studying habits cannot be successful in distance education” most affected the success factor.

7 Results Distance education is heavily practiced both in the world and in Turkey. It has lots of advantages despite some limitations. It has been observed from the survey applied to the students at Erzurum Vocational College, Computer Technologies Department that distance education has a lot of positive effects in institutions where vocational education is given, but it also has some deficiencies, many of which result from the peculiarities of vocational education. According to the students of the Computer Technologies Department, distance education is cheaper than face to face education, an important opportunity for the people who cannot attend formal education because of their jobs, and advantageous because of its characteristics like more equality in education. However, the students indicated the negative situations like technical problems that may occur during the education, application difficulties to be experienced due to the peculiarities of some subjects and the failure as a result of this problem, and the lack of studying habits by themselves as the disadvantages of distance education. As a result of above mentioned conditions, it appears that distance education is a form of education that should certainly be applied, but that the preparations and infrastructure, should be very well-planned, the teaching staff should be experienced and well trained. Taking into account only the drawbacks of asynchronized education, it is thought that performing both synchronized and asynchronized education together would be more beneficial.

References 1. Antalyalı, Ö.L.: Uzaktan Eğitim Algısı ve Yöneylem Araştırması Dersinin Uzaktan Eğitim İle Verilebilirliği, Süleyman Demirel Üniversitesi, İşletme Anabilim Dalı, Yüksek Lisans Tezi, pp. 8–10 (2004)

The Evaluation of the Applicability of Distance Education in Vocational Colleges

473

2. Beard, L.: Online Versus On-Campus Instruction: Student Attitudes & Perceptions. TechTrends 48(6), 29–31 (2004) 3. Çetiner, M.H., Gencel, Ç., Erten, Y.M., İnternete Dayalı Uzaktan Eğitim ve Çoklu Ortam Uygulamaları, 14. Uluslararası Bilgisayar ve Enformasyon Sistemleri Sempozyum Kitabı, pp. 532–539 (1999) 4. Kalaycı, Ş.: SPSS Uygulamalı Çok Değişkenli İstatistik Teknikleri, Asil Yayın Dağıtım, 2. Baskı, pp. 321–331 (2006) 5. Kaya, Z.: Uzaktan Eğitim, Pegem A Yayıncılık, pp. 27–28 (2002) 6. Koşar, E.: Öğrenme Teknolojileri ve Materyal Geliştirme, Pegem A Yayınları, 2. Baskı (2003) 7. Rıza, E.T.: Eğitim Teknolojisi Uygulamaları (1) İzmir Anadolu Matbaası, Genişletilmiş ve Geliştirilmiş 4. Baskı (1997) 8. Shihl, T.K.: A Survey of Distance Education Challenges and Technologies. International Journal of Distance Education Technologies, Cilt: 1 Sayı: 1 (2003) 9. Uşun, S.: Uzaktan Eğitim, Nobel Yayın Dağıtım, 7–8 (2006) 10. Varol, N.: İnternet’in Uzaktan Eğitimdeki Konumu (2007), http://www.ab.org.tr/ab01/prog/FTNurhayatVarol.html 11. White, C.: Language Learning in Distance Education. Cambridge University Press, Cambridge (2003)

An Evaluation of SignBright: A Storytelling Application for Sign Language Acquisition and Interpersonal Bonding amongst Deaf and Hard of Hearing Youth and Caregivers Melissa M. Burton, Chad Harbig, Mariam Melkumyan, Lei Zhang, and Jiyoung Choi Iowa State University, 1620 Howe Hall, Ames IA 50010 USA [email protected], [email protected], [email protected], {leiz,jy153}@iastate.edu

Abstract. Deaf or hard of hearing children of hearing caregivers face many challenges that have been shown to adversely impact their interpersonal interactions and development. Contemporary research indicates that many of these challenges stem from environmental factors, including a lack of exposure to language concepts during early developmental stages. This study investigates an innovative solution to foster connection and understanding between Deaf or hard of hearing children and hearing. SignBright is a storytelling application designed to promote connection between parents and Deaf or hard of hearing children. SignBright allows parents and Deaf or hard of hearing children to engage in the activity of storytelling, which promotes mutual growth and understanding, teaches language skills, and enhances parent-child bonds. This study is focused on evaluating qualities of SignBright such as the ease of navigation, design, layout, motivational characteristics, and the perceived effectiveness of SignBright. Keywords: Deaf, Children, Parent-Child Interaction, Communication, Sign Language, Educational Technology, Attachment, Child Development.

1 Introduction In the United States, 2 to 3 out of 1000 children are born deaf, and 90% or more of children with present-birth deafness are born into families where both parents are hearing [1]. While some hearing parents have adopted non-auditory means of communication, the vast majority lack the skills to communicate effectively with their children [2]. As a result, deaf children with hearing parents do not commonly receive essential exposure to language-driven interaction during early developmental stages. In addition, hearing parents often express feelings of failure or sorrow related to a child’s deafness, which they may view as a handicap [3]. Based on these factors, deaf children of hearing parents face many difficult challenges related to interfamilial perceptions and

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 474–478, 2011. © Springer-Verlag Berlin Heidelberg 2011

An Evaluation of SignBright

475

limited social engagement [5]. These challenges include socio-emotional, cognitive, and linguistic impairments, and relative emotional immaturity and dependence compared to hearing peers with hearing parents and deaf peers with deaf parents [4]. Furthermore, average deaf or hard-of-hearing students have been shown to exhibit low language related competency levels compared to hearing peers [2]. Relative performance of deaf children with hearing parents compared to deaf children with deaf parents is particularly enlightening, since it points to a fundamental difference in familial environmental conditions. As an example, Deaf parents commonly dialogue with their deaf children using sign language from a very early age, which has been shown to increase their linguistic aptitude and understanding of appropriate social cues [5]. Various technologies have been employed to good effect as educational tools for deaf and hard-of-hearing children, though many are limited in capability and rudimentary in form. However, technological solutions that adequately address the social needs of deaf children are nearly non-existent. An appropriate solution is required to connect deaf children and hearing parents and promote mutual understanding, provide deaf children with an effective means of expression, and foster cognitive development and language competency. We would like to draw your attention to the fact that it is not possible to modify a paper in any way, once it has been published. This applies to both the printed book and the online version of the publication. Every detail, including the order of the names of the authors, should be checked before the paper is sent to the Volume Editors.

2 Problem Identification Research revealed that the quality of early parent-child relationships can strongly influence academic and professional success, cognitive development, and feelings of social adjustment of deaf individuals, and that these impacts extend well into adulthood. Exposure to language and meaningful forms of expression were also identified as primary factors contributing to higher performance and socio-emotional well-being among the deaf [7]. Furthermore, our research indicated that interventions are most effective when they take place during early pre-verbal developmental stages, when children are typically learning proper responses to verbal cues and basic vocabulary through direct immersion [3,6,7,8]. In framing the design problem, we determined that an appropriate solution should expose children to relevant and multi-modal forms of language, including sign language and written English, during early developmental stages [12]. Additionally, the solution should give guidance and encouragement that is relevant to the specific child. The solution should also incorporate various techniques to promote acquisition of language skills in a clear and engaging manner, since the language-learning process is unique to each user and can be frustrating [9]. In considering forms for a solution, storytelling emerged as a particularly impactful and familiar mechanism to promote parent-child bonding, language development, and other social skills. Numerous child centered studies demonstrate the importance of storytelling and shared reading with parents. In early childhood (e.g., at kindergarten or at home), storytelling is well recognized as a means to support a child's development and to help the child express and assign meaning to the world, to

476

M.M. Burton et al.

develop communication, recognition, and recall skills, and to enforce relationships with peers and adults [10]. Furthermore, storytelling can be an important and intimate shared experience between a parent and child, and one which fosters personal familiarity and understanding. A storytelling solution designed primarily for hearing parents and deaf children may help to fill a crucial gap in interpersonal interaction between members of the target group, since many hearing parents are unequipped to engage in active, shared reading or storytelling with their children.

3

Proposed Solution

Our proposed solution, SignBright, is a storytelling application designed to promote connection between parents and deaf or hard-of-hearing children ages 2-5. SignBright allows parents and deaf or hard-of-hearing children to engage in the activity of storytelling, which promotes mutual growth and understanding, teaches language skills, and enhances parent-child bonds [10]. With the additional goal of providing a common language for children with hearing difficulties to communicate with caregivers, the SignBright application enables users to learn and practice American Sign Language (ASL) and Signed English (SE) forms of sign language [11,12]. SignBright is intended for use on multi-touch tablet devices, such as the Apple iPad. After experimenting with a number of form-factors, we believe that this format provides an experience that is most comparable to reading from a book, allowing parent and child to interact with the application while sitting in close physical proximity which results in interpersonal bonding between parent and child. Vibrant colors and patterns were chosen to actively engage users and address the “visual” nature of deaf perception. Primary interactive elements are emphasized through the use of bold forms and contrast, while secondary elements such as menus are allowed to fade to the back during engagement in learning activities. Primary interaction with elements is conducted through use of the SignFinder, a tool which docks in the top right-hand corner of the screen and expands to create a viewport when undocked. As users hover over elements with the viewport, they are presented with video depicting appropriate signing gestures for that element, accompanied with text. As users view the corresponding video for various elements, these videos will be added to a video collection that can be viewed at any time for practicing. By using a device that features a facing camera, users can record their own representation of the signed gestures and save them to the device to view in conjunction with the standard videos. In this way, users can visually analyze their gestures and refine technique as needed. In typical signing e-books with video, a stationary window is presented along with several lines of text. While this format is appropriate for some applications, stationary windows that do not strongly correlate to individual elements can lead to confusion as users attempt to determine which element is being signed at any given time. The SignFinder was employed to mitigate confusion by creating proximity between individual elements and associated signs. In addition, the SignFinder provides a novel tool with which to select individual elements while forming a story (Fig. 5), adding characteristics to a character (Fig. 3), or practicing signs (Fig. 7). In addition to input via the SignFinder, users will be able to navigate frames of a story or menus through a

An Evaluation of SignBright

477

familiar directional “swiping” gesture, which is common to many popular multi-touch applications. Other common multi-touch gestures, such as two-finger zooming of the SignFinder viewport, may be integrated, as well. Parents will begin by entering a child’s age and other pertinent data, which will allow for a customized user experience. Children will then create a character, or avatar, to represent themselves in stories (Fig. 2,3). SignBright provides many customizable options to make the application more personal. For instance, the avatar is intended to give children a greater sense of involvement in stories, and empower them to interact more readily with the world around them, and can be modified indefinitely as a child’s interests and self identity change. Regardless of age, children differ in their cognitive and linguistic abilities. For instance, 2-year-old children have more limited vocabulary than older children and are commonly limited to expression through very short phrases and sentences. However, infants are also more attentive to objects in their environment [3,6]. SignBright will initially address early language learning through presentation of simple objects in correlation with text (Fig. 4). These objects are selected based on relevance to a particular age. Parents can interact with the elements via the SignFinder tool, which will present them with video of related signed gestures. In this way, parents and children can begin to build a vocabulary that allows them to communicate freely and understand each other’s needs. As children continue to grow in age and ability, SignBright will adapt to their changing needs. At 3-4 years of age, children typically have increased ability to communicate. They are able to express their ideas clearly and tell some short stories [13]. At this level, children can begin to create their own stories by repeating the method described in figures 4 through 6. The content of this level focuses more on describing daily activities than simple objects. The higher level is designed for children who have an excellent grasp of sign language appropriate for 4-5 year old children. The content can range from humorous stories to more detailed fairy tales, with many joining segments. At this stage, SignFinder interaction will be used in conjunction with stationary video depictions to handle longer phrases. Children and families can benefit tremendously from acquisition of a shared language, to support proper child development and to strengthen shared interpersonal connections [5]. SignBright addresses both of these goals in an intuitive and innovative application that a child and parent can enjoy together. Furthermore, SignBright grows with users, making it an integral tool to encourage children and parents to keep developing their sign language abilities.

References 1. National Institute on Deafness and Other Communication Disorders (June 16, 2010), http://www.nidcd.nih.gov/health/statistics/quick.htm 2. Mayer, C.: What really matters in the early literacy development of deaf children. Journal of Deaf Studies and Deaf Education 12(4), 411–431 (2007) 3. Meadow, K.P.: Early manual communication in relation to the deaf child’s intellectual, social, and communicative functioning. Journal of Deaf Studies and Education 10(4), 321– 329 (2007)

478

M.M. Burton et al.

4. Meadow, K.P.: Deafness and child development. Univ. of California Press, Berkeley (1980) 5. Vaccari, C., Marschark, M.: Communication between parents and deaf children: Implications for social-emotional development. Journal of Child Psychological Psychiatry 38(7), 793–801 (1997) 6. Tait, M.E., Nikolopoulos, T.P., Lutman, M.E.: Age at implantation and development of vocal and auditory preverbal skills in implanted deaf children. Journal of Pediatric Otorhinolaryngology 71(4), 603–610 (2007) 7. Moeller, M.P.: Early intervention and language development in children who are deaf and hard of hearing. Pediatrics 106, 3 (2000) 8. Nicholas, J.G., Geers, A.E.: Will they catch up? The role of age at cochlear implantation in the spoken language development of children with severe to profound hearing loss. Journal of Speech, Language, and Hearing Research 50, 1048–1062 (2007) 9. Marschark, M.: Psychological development of deaf children. Oxford University Press, New York (1993a) 10. Peck, J.: Using storytelling to promote language and literacy development. The Reading Teacher 23(2), 138–141 11. Schick, B., Villiers, P., Villiers, J., Hoffmeister, R.: Language and theory of mind: A study of deaf children. Child Development 78(2), 376–396 (2007) 12. Stokoe, W.C.: Sign language structure: An outline of the visual communication systems of the American deaf. Journal of Deaf Studies and Deaf Education 10, 1 (2005) 13. Cassell, J., Ryokai, K.: Making space for voice: Technologies to support children’s fantasy and storytelling. Personal Ubiquitous Comput. 5(3), 169–190 (2001) 14. Adamo-Villani, N., Wilbur, R.: Two novel technologies for accessible math and science education. IEEE Multimedia 15(4), 38–46 (2009) 15. Adamo-Villani, N., Doublestein, J., Martin, Z.: Sign Language for K–8 Mathematics by 3D interactive animation. Journal of Educational Technology Systems 33(3), 243–257 (2005)

Collaborative Analysis and Communities of Practice in Health Sciences Juan Alberto Castillo M. PhD in Cognitive Psychology, ErgoMotion Laboratory, Medicine and Health Sciences School, Rosario University, RENATA – LMS in GRID computing project, National Minister of Education and COLCIENCIAS, Bogotá-Colombia [email protected]

Abstract. In the context of the information system in health sciences, the actor requires collaborative models for analysis and diagnoses process. In this purpose, we developed a learning community based on a distributed management model, the model is supported in grid development to facilitate the process of management, storage, distribution and selection of knowledge for perform process analysis, specifically in movement sciences, this project research analyzes the problems associated to learning the principles of human movement diagnoses. The study of the parameters required for the preparation of diagnostics, are presented to students to apply them in collaborative work sessions. Keywords: Ergonomics, Motion Analysis, Simulation, Community of Practice.

1 Introduction Today the knowledge visualization as a design activity focused in the solution of knowledge transmission problems; actually this is an area of extensive research in the contemporary context the information use in NTICS technologies and new media. The complexity of the interaction problems and the information management, with which individuals interact, is oriented to driving the need for more robust processes and tools more complex reasoning. In other hand the increased complexity in decision making problem and information management must be addressed with meaningful solutions for people. The problems in access to encrypted information depend on the establishment of codes or models use in packaging information and knowledge, the question is how every individual can coding and decode that information system. The present and future forms of interaction have changed the way of access to information and knowledge, so the design of these means is necessary to understand the ways of structuring knowledge and the mechanisms of encoding and decoding. Using a grid, we have created a learning community designed to develop jointly and simultaneously clinical assessments of problems, related to human movement survey, in order to develop collaborative diagnoses and develop common strategies for intervention strategies. Grid work permits while large groups of students observe the process of elaboration of these diagnoses. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 479–483, 2011. © Springer-Verlag Berlin Heidelberg 2011

480

J.A. Castillo M.

This project sponsored by RENATA (High speed national network), COLCIENCIAS and the National Minister of Education from Colombia, search implant a physical infrastructure, of services and applications in networks of high speed using GRID model, that allows integrate of existing projects in movement analyses at the University of Rosary and which they are tried to extend to the academic networks of high speed, for the development in university hospitals in Colombia. 1.1 Structure of GRID Project The project start with 3 courses for Colombian universities students, after that this LMS platform are including within academic programs of medicine and rehabilitation in others Colombian universities, the project have associate too university hospitals, they can accede to virtual resources, using high speed internet connections. The project uses a computational net (GRID), with a software of e-learning executed on this architecture. The computational net is tried to implant is an own net of MAAT known as MAAT-G and the software of e-Learning also are of MAAT, Known like MAATKNOWLEGE. In this sense, the first stage of this project will be joined by the Motion Analysis Laboratory at the University of Rosario and the motion analysis center of the health authority of the Autonomous University of Manizales .Thereafter, any associated RENATA University may enter the project and be part of the universities who want to be linked to the platform and resources offered through this computational grid. RENATA (National academic network) is leading the project GRID Colombia through the Red course with the participation of universities in Bogotá, Medellin and Bucaramanga. The main objective for RENATA is to build a computational grid system that contains universities and institutions that develop projects in the areas of climate, pollution, natural disasters, biodiversity, bioinformatics, health sciences and natural resources In Colombia, each university has implemented its platform GRID and one of its projects is the integration of these into a single GRID consolidated , managed infrastructure GRID as GILDA ( GLITE ) , Middleware Architecture for GRID Oriented Services ( MEN ) , ROCKS , G + . The project developed installs and configures a Computational Grid (GRID), with E - Learning software that runs on this architecture. The computational mesh installed, is a proper mesh called MAAT MAAT - G and e - Learning software is also MAAT, known as Maat - Knowledge. Ian Foster, co-director of the GriPhyN (Grid Physics Networks) [1]. Defines a Grid as a system that: 1) coordinates resources not subject to centralized control, 2) Use standard protocols and interfaces, open and general purpose and 3) deliver an important service quality. On the other hand Hai Zhuge [2]. GRID defines as "an intelligent networking environment and sustainable development that allows people and machines effectively capture, publish, share and manage knowledge resources". On the other hand, according to Hai [3]. "Knowledge GRID has social characteristics. In the real world, people live and work in a grid obeying social rules and social and Economic Laws. The grid Provider information flows and Knowledge flows.

Collaborative Analysis and Communities of Practice in Health Sciences

481

1.2 Community of Practice in Movement Analysis Learning using information and communication techniques in GRID, extending large -scale vision of a flexible, secure and coordinated educational and computational resources, and also expands the dynamic exchange between individuals, institutions and resources [4]. In this project, the objective of a community of practice is to meet different actors in a synchronic action, facilitated for the technologies of communication, the action reciprocates of the actors is oriented to the development of know-how, abilities and attitudes that should be stabilized in the time. In that community, the modelling of interaction aimed at diagnosis and troubleshooting can be seen in the model proposed by Morse, M [5] provides three scenarios for digital action in cyberspace, specifically identifies the electronic networks as arenas where you can develop a group think model, this is the means to enhance human inter-subjectivity and develop a model of interaction beyond the model of humanmachine interaction. In this sense, a distributed LMS in GRID lets you generate a model of user interaction on a technological platform for high performance. The Communities of Practice [5], as a process of social learning that occurs when people who have a common interest in a subject, our “Communities of practice are groups of students who share a concern or a passion for movement analysis, they do and learn how to do a better diagnosis, as they interact regularly in case study and online events.”

2 Design Model of Community of Practice The design of the learning community is based on the principle of diagnose is due to make from elaborations to understanding and structuring the movement problem. This action is due to develop in two dimensions: collective and individual. The objective is to facilitate the transference of the concrete experience to the theoretical elaboration. The search and exploration of the solution to identified the problem, must be oriented from the collective perspective, this with the purpose of constructing, diagnose could be anchored in the collective experience. The development of processes of reflection and discussion on-line and real time, facilitates the collective process of the problem deconstruction and construction of the solution. The GRID offers the possibility of a high volume of data flow. These data could be examined from individual point of view, also could be treated from the collective point of view. For the development of double flow process, a series of resources has been designed that can be consulted on line and that also can be shared by the members of the community. Additionally each individual can annex information that comes from its experience with the purpose of enriching the collective knowledge of the community. It is important to remember, that in the diagnose elaboration in the field of the health, is necessary the participation of several referring ones with the purpose of validating internally and externally the decisions that are due to take.

482

J.A. Castillo M.

Fig. 1. Model developed for the interactions design community. (Castillo, J. 2010)

2.1 Interaction Activities The interaction activities are developed in two contexts. The community organizes itself in different groups, each group interacts internally with the purpose of developing an own knowledge base. In the second context, the different groups interact to each other, in order to develop a common base of knowledge. The development of a common knowledge base, is organized in two dimensions, the dimension practices that it compiles the knowledge to do of each one of the experts who monitoring the community and the knowledge to make of each one of the students members of the community. The second dimension is oriented to the development of corpus theoretical specialized in the problems that must be treated in the knowledge field of the analysis of the movement. The interactions between the members of the community are developed in the practical and theoretical field. The development of abilities in these two fields with the purpose of sharpening the processes of analysis and diagnosis of the cases treated by each one the members outside the community. The development of the community requires east validation process in the experience of each individual in its personal practice of the processes developed collectively in the community members groups. Finally the development of a community based on GRID, facilitates the massive interchange of data (videos, photography’s, descriptions). With the purpose of allowing to the members the development a precise representation of the problem treated, it is important to emphasize that, in the movement analysis a high volume of objective information, is required it comes generally from systems of digital registry and analogous reports elaborated by the analysts.

Collaborative Analysis and Communities of Practice in Health Sciences

483

References 1. Foster, I., Kesselman, C., Nick, J.M., Tuecke, S.: The physiology of the grid: an open grid services architecture for distributed systems integration. Global Grid Forum (2002) 2. Zhuge, H.: A knowledge grid model and platform for global knowledge sharing. Expert Systems with Applications 22(4), 313–320 (2002) 3. Zhuge, H.: Communities and Emerging Semantics in Semantic Link Network: Discovery and Learning. IEEE Trans. Knowledge and Data Eng. (2008) 4. Foster, I., Kesselman, C., Tuecke, S.: The Anatomy of the Grid: Enabling Scalable Virtual Organizations. International Journal of High Performance Computing Applications 15(3), 200–222 (2001) 5. Lave, J., Wenger, E.: Situated Learning - Legitimate Peripheral Participation. Cambridge University Press, Cambridge (1991)

The Application of Interactive Media Display Technology in Environmental Science Learning Chun-Ching Chen and Chien-Ming Chen Graduate Institute of Interactive Media Design, National Taipei University of Technology, Taiwan [email protected], [email protected]

Abstract. The introduction of interactive technologies turns traditional concepts of teacher-centred teaching into student-centred learning pattern. Take environmental science learning as main subject, this study designs and develops an interactive educational device based on interactive media display technology, and to examine its acceptance. A simulated Earth is designed and embedded with interactive technologies to allow the users to interact with. System feedbacks comprise a projected animation on the ground and voice to explain the meaning of the user's action and the relation to environmental issue. The results show that most of the users can understand the purpose of the interaction and the contents of the environment issue which are introduced. It is concluded that the introduction of interactive technologies can arouse students’ interests in learning and improve their motivation as it creates new learning experience, particularly for popular science education. Keywords: interactive display technology, multimedia, environmental science.

1 Introduction Traditional teaching is a one-way instruction conducted by the teacher. It is easy to be implemented in a class and has remained Taiwan's main teaching methods. However, traditional teaching of the homogeneity of group learning only emphasizes the tasks but ignores the function of group learning [1]. Traditional education is a universal education, without considering individual differences of learners [2]. It is a teachercentered approach to learning, but this approach doesn’t take learners' individual differences into consideration. In this mode of education, the teacher cares about teaching schedule and accomplishments, and students are asked to be obedient. Current emphasis on the education shifts from teacher-centred to putting students in the centre of teaching activity. Digital interactive media technologies thus play a key role to achieve this goal. Interactive multimedia is to link various media to integrate with multimedia elements such as digital contents, photos, visual art, sound, animation and images to interact with people [3]. Interactive multimedia instruction is to use pervasive sound and colour, multi-strategy and interactive teaching materials, and manipulate teaching variables to enhance learning motivation, through observing the nature of the problem from the continuous trial and error to lead to solution [4]. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 484–488, 2011. © Springer-Verlag Berlin Heidelberg 2011

The Application of Interactive Media Display Technology

485

Therefore, the application of interactive multimedia can improve the traditional teaching by involving students in a simulated scenario to communicate with teaching system. Using teaching materials with interactive multimedia helps learning from the operation, as doing experiments in the laboratory to obtain practical and efficient learning. This will provide stimulating, interactive, two-way exchange of learning environment to enhance students' creativity and learning effects [5]. Human-computer interaction allows students to use multiple senses to learn and give feedback immediately; moreover, teachers can also understand students' learning status instantly. Digital media can be used in various types of courses. Previous work has investigated the application and its effects used in different platforms (e.g., [6]). These studies aim to incorporate interactive multimedia applications in basic science education (e.g., mathematics) to improve learning efficiency. However, if interactive teaching is introduced in teaching strategies and relevant scientific theories, such as social sciences in different levels, the strategies and the theories can be combined to carry out teaching curriculum and further to show its effectiveness [7]. With a variety of sensory stimulation interactive multimedia technology brings, it enables the learner to access information. Information and knowledge are closely related, which means accessing massive information allows learners to grasp the opportunity to learn. This study takes environmental science as a main subject, and introduces interactive multimedia display technologies in this popular science which is closely linked with our daily lives. It focuses on designing a physical interactive device, which allows learners to interact with learning contents. This work is tested through public exhibition and feedbacks from visitors in terms of its acceptance and the practibility of its application in learning.

2 Methods Environmental Science works for the interactive relation between human’s evolution and the change of the environment, so as to seek the approaches to synergistic evolution and sustainable development [8]. Environmental science education is to construct the context of human society, culture, economic development and the environment, in which the relationship is interdependent and the goal is to protect and improve the environment [9]. Environmental science education includes environmental awareness and sensitivity, knowledge of environmental concepts, environmental values and attitudes, environmental action skills and experience [10]. This work is based on environmental action skills and experience. Learners participate in a virtual environmental protection activity, and therefore to convey appropriate knowledge of environmental science. As environmental destruction and pollution is the result of competition for living space between human being and the nature, this work sets users’ operation as a key to open up the opportunity of protecting environment in their daily life and the influence of environmental action skills and experience on human society. This work takes the "Earth" as the core concept, and incorporates interactive technologies to allow the user's hands to determine sustainable living or damage on the Earth. The purpose is to educate learners to protect their living environment through their “both hands” and achieving the goal of applying interactive technology on popular science education.

486

C.-C. Chen and C.-M. Chen

This study uses interactive multimedia technology as the base of the system interaction, and interactive relation between human and the environment as the content of the system. The work is a working prototype, allowing users to give feedbacks in public exhibition. The interface is an interactive physical simulated Earth, combined with feedback animation coming after the users’ operation to convey educational information. Users move their both hands to attach the objects which could cause environmental damage on the earth. Pressure transmitting from the hands triggers the corresponding animation projected on the ground. The more pressure the “Earth” is, the more change the environment in the animation projects (Fig.1). The device can be operated by multiple users simultaneously. From the animation, users can learn the damage coming from human’s desires and to raise their awareness about environmental protection.

Fig. 1. Interaction of the system

Fig. 2. Appearance of the device

Fig. 3. Animations

This work consists of two main components, one is the physical device which can be operated, and the other one is the animations to provide system feedback. The device is an interactive simulated green floating “Earth”. It was built by acrylic hemisphere (diameter: 100cm) and covered with plastic turf to represent people’s living environment. Center at the device is a vertical projector to project animation on the ground, through a diameter of 50cm hole in the middle of the “Earth” surface (Fig.2). On the turf, various shaped objects represent plant, excavators, cars,

The Application of Interactive Media Display Technology

487

mink coats and money are implemented, which allows the learner to move and attach on the surface. Corresponding animations are then projected and played. "The earth" is embedded with Arduino platform and programmed by Processing to allow the user to interact by moving the objects. Adobe Flash CS4 and ActionScript 3.0 are used to construct the animation and interactive programme. The corresponding animation varies from different objects users selected. It includesthe change of nature by factories and excavators, the pollution coming from the vehicles, the death of wild animals due to mink coat, and peoples greed for money reflecting war…etc (Fig.3).

3 Results Named "Touch Earth", this work was publicly exhibited in Taipei 2010 Young Designers’ Exhibition (YODEX) in May 2010. During the exhibition visitors were invited to interact personally with the device and experience the learning (Fig.4). Interviews with various visitors (teachers, students and the public) and their feedbacks were also conducted to identify two aspects, people’s understanding on the interaction and information they received from the device, and the acceptance and practicability of applying interactive multimedia technologies to popular science education. The main results obtained are summarised as follows.

Fig. 4. Visitors interacted with the device

1. The public expects more intuitive interaction and richer visual presentation on the animation to increase the sensory stimulation. 2. According to teachers’ reponses, interactive multimedia teaching can actually increase the absorption of knowledge. However, in practice, producing such a device demonstration require professional technology input and space, as well as equipment. 3. From the learner's point of view, interactive multimedia teaching provides more attractiveness to arouse their interest and willingness in learning. Compared with reading from books, sound, visual and tactile stimuli provide easy but rich understanding of the knowledge. However, complicated learning contents may not be easy to be implemented on the device. 4. The relationship between teachers and learners could be improved. Parents look forward to seeing more application in their children's education.

488

C.-C. Chen and C.-M. Chen

4 Conclusion This study presents the following results: (1) interactive multimedia education may be suitable for small class teaching; (2) space and equipment constraints are the main issues in interactive multimedia teaching, however, the development of portable devices could show its potential in the future; (3) The interface should carefully consider both instructor’s and learner’s cognition to provide intuitive interaction and requirements. This study shows that interactive learning can indeed improve learning motivation and interest, and leads teaching activity from teacher-centered to studentcentered learning mode. Although interactive multimedia still cannot take over traditional teaching completely, it does shows its potential on helping students to achieve "true understanding and memory". Acknowledgments. Special thanks are given to Professor Lai-Chung Lee and KoChiu Wu, and co-designers Chia-Ni Chen and Wu-Chuan Wang of Graduate Institute of Interactive Media Design, National Taipei University of Technology for their support on work.

References 1. Johnson, D.W., Johnson, R.T.: Structuring Cooperative Learning: Learning Teachers. In: The 1987 Handbook of Lessons Plans for Teacher. Interation Book Company, Edina (1987) 2. Lin, Y., Nien, C.C.: The Comparison of Competency Based Education and Traditional Education. Journal of Manpower Training, 20–24 (February 2000) (in Chinese) 3. Wu, P.S.: Integrating Internet and Virtual-Reality Techniques to Build a Guide System of Bikeway -a Case Study of Tan-Ya-Shen Green Corridor. Master Thesis of Feng Chia University, Taichung (2007) (in Chinese) 4. Chang, Y.P.: A Study on the Performance of Summary Strategy Embedded in the Webbased Environment. Master Thesis of National Taiwan Normal University, Taipei (1999) (in Chinese) 5. Lin, Y.C.: A Study of Virtual-lab on WWW Affects Students’ Learning Achievement of Natural Science Learning at Elementary Schools. Master Thesis of National University of Tainan, Tainan (2001) (in Chinese) 6. Lu, C.H.: Learning Shihmen Reservoir Watershed Water Resources Issues —A Study of E-STS Instruction on the Students of Third Grade. Master Thesis of National Taiwan Normal University, Taipei (2010) (in Chinese) 7. Rhodes, F., Fishbein, M., Reis, J.: Using Behavior Theory in Computer-Based Health Promotion and Appraisal. Health Education & Behavior 24(1), 20–34 (1997) 8. Lin, M.J.: A Case Study of Er-chung Junior Life Education Forward the Implementation of Ecological Teaching in Science and Technology Area. Master Thesis of Ming Chuan University, Taoyuan (2005) (in Chinese) 9. United Nations Education, Scientific, & Cultural Organization.: The World’s First Intergovernment Conference on Environmental Education in Tbilisi. Eric/SMEAC Information Reference Center, Columbus, Ohio (1978) 10. Liu, C.N.: Multimedia Application of Interactive Online Digital Teaching on Senior High School Mathematics - Online Digital Teaching on Senior High School Mathematics. Master Thesis of Feng Chia University, Taichung (2008) (in Chinese)

Applying User-Centered Techniques to Develop a Radiology Teaching File System Marcelo dos Santos and Asa Fujino School of Communications and Arts (ECA/USP), University of Sao Paulo, Brazil {mar.santos,asfujino}@usp.br

Abstract. Education and training processes in diagnostic radiology requires systematic and comprehensive study of a large knowledge base of medical images. Electronic teaching files (ETF) systems in Radiology are meant to make learning easier and also to raise its quality. However, ease of use has been recognized as a key factor in influencing users’ acceptance of new technologies. Similarly, Radiology students’ interaction with the ETF should be designed according to their needs instead of requiring these students to adapt to the technology. In this work is presented the development of a digital radiology teaching file system. The proposed system has been created in order to offer a set of customized services regarding to users’ contexts and their informational needs. The current prototype has demonstrated the feasibility of developing a user-centered ETF system.

1 Introduction When Wilhelm Conrad Roentgen discovered a new kind of rays – named X-rays – he had also discovered a magic window for looking within the human body and painless examining the organs and bones. Before this discovery, a doctor had no way to view inside of a patient’s body other than to cut it open. At the present, we have seen great advances in medi