HCI International 2011 – Posters’ Extended Abstracts, Part II

Communications in Computer and Information Science

174

Constantine Stephanidis (Ed.)

HCI International 2011 – Posters’ Extended Abstracts International Conference, HCI International 2011 Orlando, FL, USA, July 9-14, 2011 Proceedings, Part II

13

Volume Editor Constantine Stephanidis Foundation for Research and Technology - Hellas (FORTH) Institute of Computer Science (ICS) N. Plastira 100, Vassilika Vouton 70013, Heraklion, Crete, Greece and University of Crete Department of Computer Science, Crete, Greece E-mail: [email protected]

ISSN 1865-0929 e-ISSN 1865-0937 ISBN 978-3-642-22094-4 e-ISBN 978-3-642-22095-1 DOI 10.1007/978-3-642-22095-1 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2011930138 CR Subject Classification (1998): H.4, H.5, I.2, H.3, C.2, D.2

© Springer-Verlag Berlin Heidelberg 2011 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Foreword

The 14th International Conference on Human–Computer Interaction, HCI International 2011, was held in Orlando, Florida, USA, July 9–14, 2011, jointly with the Symposium on Human Interface (Japan) 2011, the 9th International Conference on Engineering Psychology and Cognitive Ergonomics, the 6th International Conference on Universal Access in Human–Computer Interaction, the 4th International Conference on Virtual and Mixed Reality, the 4th International Conference on Internationalization, Design and Global Development, the 4th International Conference on Online Communities and Social Computing, the 6th International Conference on Augmented Cognition, the Third International Conference on Digital Human Modeling, the Second International Conference on Human-Centered Design, and the First International Conference on Design, User Experience, and Usability. A total of 4,039 individuals from academia, research institutes, industry and governmental agencies from 67 countries submitted contributions, and 1,318 papers that were judged to be of high scientific quality were included in the program. These papers address the latest research and development efforts and highlight the human aspects of design and use of computing systems. The papers accepted for presentation thoroughly cover the entire field of human–computer interaction, addressing major advances in knowledge and effective use of computers in a variety of application areas. This volume, edited by Constantine Stephanidis, contains posters’ extended abstracts addressing the following major topics: • • • • • • • •

Novel interaction environments Virtual and augmented environments Gestures, gaze and multimodality in HCI Touch-based and table-top interaction Brain-Computer interfaces and brain monitoring Ergonomics and human modelling issues Health and wellbeing Learning, education and cultural heritage

The remaining volumes of the HCI International 2011 Proceedings are: • Volume 1, LNCS 6761, Human–Computer Interaction—Design and Development Approaches (Part I), edited by Julie A. Jacko • Volume 2, LNCS 6762, Human–Computer Interaction—Interaction Techniques and Environments (Part II), edited by Julie A. Jacko • Volume 3, LNCS 6763, Human–Computer Interaction—Towards Mobile and Intelligent Interaction Environments (Part III), edited by Julie A. Jacko • Volume 4, LNCS 6764, Human–Computer Interaction—Users and Applications (Part IV), edited by Julie A. Jacko

VI

Foreword

• Volume 5, LNCS 6765, Universal Access in Human–Computer Interaction— Design for All and eInclusion (Part I), edited by Constantine Stephanidis • Volume 6, LNCS 6766, Universal Access in Human–Computer Interaction— Users Diversity (Part II), edited by Constantine Stephanidis • Volume 7, LNCS 6767, Universal Access in Human–Computer Interaction— Context Diversity (Part III), edited by Constantine Stephanidis • Volume 8, LNCS 6768, Universal Access in Human–Computer Interaction— Applications and Services (Part IV), edited by Constantine Stephanidis • Volume 9, LNCS 6769, Design, User Experience, and Usability—Theory, Methods, Tools and Practice (Part I), edited by Aaron Marcus • Volume 10, LNCS 6770, Design, User Experience, and Usability— Understanding the User Experience (Part II), edited by Aaron Marcus • Volume 11, LNCS 6771, Human Interface and the Management of Information—Design and Interaction (Part I), edited by Michael J. Smith and Gavriel Salvendy • Volume 12, LNCS 6772, Human Interface and the Management of Information—Interacting with Information (Part II), edited by Gavriel Salvendy and Michael J. Smith • Volume 13, LNCS 6773, Virtual and Mixed Reality—New Trends (Part I), edited by Randall Shumaker • Volume 14, LNCS 6774, Virtual and Mixed Reality—Systems and Applications (Part II), edited by Randall Shumaker • Volume 15, LNCS 6775, Internationalization, Design and Global Development, edited by P.L. Patrick Rau • Volume 16, LNCS 6776, Human-Centered Design, edited by Masaaki Kurosu • Volume 17, LNCS 6777, Digital Human Modeling, edited by Vincent G. Duffy • Volume 18, LNCS 6778, Online Communities and Social Computing, edited by A. Ant Ozok and Panayiotis Zaphiris • Volume 19, LNCS 6779, Ergonomics and Health Aspects of Work with Computers, edited by Michelle M. Robertson • Volume 20, LNAI 6780, Foundations of Augmented Cognition: Directing the Future of Adaptive Systems, edited by Dylan D. Schmorrow and Cali M. Fidopiastis • Volume 21, LNAI 6781, Engineering Psychology and Cognitive Ergonomics, edited by Don Harris • Volume 22, CCIS 173, HCI International 2011 Posters Proceedings (Part I), edited by Constantine Stephanidis I would like to thank the Program Chairs and the members of the Program Boards of all Thematic Areas, listed herein, for their contribution to the highest scientific quality and the overall success of the HCI International 2011 Conference. In addition to the members of the Program Boards, I also wish to thank the following volunteer external reviewers: Roman Vilimek from Germany, Ramalingam Ponnusamy from India, Si Jung “Jun” Kim from the USA, and Ilia Adami, Iosif Klironomos, Vassilis Kouroumalis, George Margetis, and Stavroula Ntoa from Greece.

Foreword

VII

This conference would not have been possible without the continuous support and advice of the Conference Scientific Advisor, Gavriel Salvendy, as well as the dedicated work and outstanding efforts of the Communications and Exhibition Chair and Editor of HCI International News, Abbas Moallem. I would also like to thank for their contribution toward the organization of the HCI International 2011 Conference the members of the Human–Computer Interaction Laboratory of ICS-FORTH, and in particular Margherita Antona, George Paparoulis, Maria Pitsoulaki, Stavroula Ntoa, Maria Bouhli and George Kapnas. July 2011

Constantine Stephanidis

Organization

Ergonomics and Health Aspects of Work with Computers Program Chair: Michelle M. Robertson Arne Aar˚ as, Norway Pascale Carayon, USA Jason Devereux, UK Wolfgang Friesdorf, Germany Martin Helander, Singapore Ed Israelski, USA Ben-Tzion Karsh, USA Waldemar Karwowski, USA Peter Kern, Germany Danuta Koradecka, Poland Nancy Larson, USA Kari Lindstr¨om, Finland

Brenda Lobb, New Zealand Holger Luczak, Germany William S. Marras, USA Aura C. Matias, Philippines Matthias R¨ otting, Germany Michelle L. Rogers, USA Dominique L. Scapin, France Lawrence M. Schleifer, USA Michael J. Smith, USA Naomi Swanson, USA Peter Vink, The Netherlands John Wilson, UK

Human Interface and the Management of Information Program Chair: Michael J. Smith Hans-J¨ org Bullinger, Germany Alan Chan, Hong Kong Shin’ichi Fukuzumi, Japan Jon R. Gunderson, USA Michitaka Hirose, Japan Jhilmil Jain, USA Yasufumi Kume, Japan Mark Lehto, USA Hirohiko Mori, Japan Fiona Fui-Hoon Nah, USA Shogo Nishida, Japan Robert Proctor, USA

Youngho Rhee, Korea Anxo Cereijo Roib´ as, UK Katsunori Shimohara, Japan Dieter Spath, Germany Tsutomu Tabe, Japan Alvaro D. Taveira, USA Kim-Phuong L. Vu, USA Tomio Watanabe, Japan Sakae Yamamoto, Japan Hidekazu Yoshikawa, Japan Li Zheng, P.R. China

X

Organization

Human–Computer Interaction Program Chair: Julie A. Jacko Sebastiano Bagnara, Italy Sherry Y. Chen, UK Marvin J. Dainoff, USA Jianming Dong, USA John Eklund, Australia Xiaowen Fang, USA Ayse Gurses, USA Vicki L. Hanson, UK Sheue-Ling Hwang, Taiwan Wonil Hwang, Korea Yong Gu Ji, Korea Steven A. Landry, USA

Gitte Lindgaard, Canada Chen Ling, USA Yan Liu, USA Chang S. Nam, USA Celestine A. Ntuen, USA Philippe Palanque, France P.L. Patrick Rau, P.R. China Ling Rothrock, USA Guangfeng Song, USA Steffen Staab, Germany Wan Chul Yoon, Korea Wenli Zhu, P.R. China

Engineering Psychology and Cognitive Ergonomics Program Chair: Don Harris Guy A. Boy, USA Pietro Carlo Cacciabue, Italy John Huddlestone, UK Kenji Itoh, Japan Hung-Sying Jing, Taiwan Wen-Chin Li, Taiwan James T. Luxhøj, USA Nicolas Marmaras, Greece Sundaram Narayanan, USA Mark A. Neerincx, The Netherlands

Jan M. Noyes, UK Kjell Ohlsson, Sweden Axel Schulte, Germany Sarah C. Sharples, UK Neville A. Stanton, UK Xianghong Sun, P.R. China Andrew Thatcher, South Africa Matthew J.W. Thomas, Australia Mark Young, UK Rolf Zon, The Netherlands

Universal Access in Human–Computer Interaction Program Chair: Constantine Stephanidis Julio Abascal, Spain Ray Adams, UK Elisabeth Andr´e, Germany Margherita Antona, Greece Chieko Asakawa, Japan Christian B¨ uhler, Germany Jerzy Charytonowicz, Poland Pier Luigi Emiliani, Italy

Michael Fairhurst, UK Dimitris Grammenos, Greece Andreas Holzinger, Austria Simeon Keates, Denmark Georgios Kouroupetroglou, Greece Sri Kurniawan, USA Patrick M. Langdon, UK Seongil Lee, Korea

Organization

Zhengjie Liu, P.R. China Klaus Miesenberger, Austria Helen Petrie, UK Michael Pieper, Germany Anthony Savidis, Greece Andrew Sears, USA Christian Stary, Austria

Hirotada Ueda, Japan Jean Vanderdonckt, Belgium Gregg C. Vanderheiden, USA Gerhard Weber, Germany Harald Weber, Germany Panayiotis Zaphiris, Cyprus

Virtual and Mixed Reality Program Chair: Randall Shumaker Pat Banerjee, USA Mark Billinghurst, New Zealand Charles E. Hughes, USA Simon Julier, UK David Kaber, USA Hirokazu Kato, Japan Robert S. Kennedy, USA Young J. Kim, Korea Ben Lawson, USA Gordon McK Mair, UK

David Pratt, UK Albert “Skip” Rizzo, USA Lawrence Rosenblum, USA Jose San Martin, Spain Dieter Schmalstieg, Austria Dylan Schmorrow, USA Kay Stanney, USA Janet Weisenford, USA Mark Wiederhold, USA

Internationalization, Design and Global Development Program Chair: P.L. Patrick Rau Michael L. Best, USA Alan Chan, Hong Kong Lin-Lin Chen, Taiwan Andy M. Dearden, UK Susan M. Dray, USA Henry Been-Lirn Duh, Singapore Vanessa Evers, The Netherlands Paul Fu, USA Emilie Gould, USA Sung H. Han, Korea Veikko Ikonen, Finland Toshikazu Kato, Japan Esin Kiris, USA Apala Lahiri Chavan, India

James R. Lewis, USA James J.W. Lin, USA Rungtai Lin, Taiwan Zhengjie Liu, P.R. China Aaron Marcus, USA Allen E. Milewski, USA Katsuhiko Ogawa, Japan Oguzhan Ozcan, Turkey Girish Prabhu, India Kerstin R¨ ose, Germany Supriya Singh, Australia Alvin W. Yeo, Malaysia Hsiu-Ping Yueh, Taiwan

XI

XII

Organization

Online Communities and Social Computing Program Chairs: A. Ant Ozok, Panayiotis Zaphiris Chadia N. Abras, USA Chee Siang Ang, UK Peter Day, UK Fiorella De Cindio, Italy Heidi Feng, USA Anita Komlodi, USA Piet A.M. Kommers, The Netherlands Andrew Laghos, Cyprus Stefanie Lindstaedt, Austria Gabriele Meiselwitz, USA Hideyuki Nakanishi, Japan

Anthony F. Norcio, USA Ulrike Pfeil, UK Elaine M. Raybourn, USA Douglas Schuler, USA Gilson Schwartz, Brazil Laura Slaughter, Norway Sergei Stafeev, Russia Asimina Vasalou, UK June Wei, USA Haibin Zhu, Canada

Augmented Cognition Program Chairs: Dylan D. Schmorrow, Cali M. Fidopiastis Monique Beaudoin, USA Chris Berka, USA Joseph Cohn, USA Martha E. Crosby, USA Julie Drexler, USA Ivy Estabrooke, USA Chris Forsythe, USA Wai Tat Fu, USA Marc Grootjen, The Netherlands Jefferson Grubb, USA Santosh Mathan, USA

Rob Matthews, Australia Dennis McBride, USA Eric Muth, USA Mark A. Neerincx, The Netherlands Denise Nicholson, USA Banu Onaral, USA Kay Stanney, USA Roy Stripling, USA Rob Taylor, UK Karl van Orden, USA

Digital Human Modeling Program Chair: Vincent G. Duffy Karim Abdel-Malek, USA Giuseppe Andreoni, Italy Thomas J. Armstrong, USA Norman I. Badler, USA Fethi Calisir, Turkey Daniel Carruth, USA Keith Case, UK Julie Charland, Canada

Yaobin Chen, USA Kathryn Cormican, Ireland Daniel A. DeLaurentis, USA Yingzi Du, USA Okan Ersoy, USA Enda Fallon, Ireland Yan Fu, P.R. China Afzal Godil, USA

Organization

Ravindra Goonetilleke, Hong Kong Anand Gramopadhye, USA Lars Hanson, Sweden Pheng Ann Heng, Hong Kong Bo Hoege, Germany Hongwei Hsiao, USA Tianzi Jiang, P.R. China Nan Kong, USA Steven A. Landry, USA Kang Li, USA Zhizhong Li, P.R. China Tim Marler, USA

XIII

Ahmet F. Ozok, Turkey Srinivas Peeta, USA Sudhakar Rajulu, USA Matthias R¨ otting, Germany Matthew Reed, USA Johan Stahre, Sweden Mao-Jiun Wang, Taiwan Xuguang Wang, France Jingzhou (James) Yang, USA Gulcin Yucel, Turkey Tingshao Zhu, P.R. China

Human-Centered Design Program Chair: Masaaki Kurosu Julio Abascal, Spain Simone Barbosa, Brazil Tomas Berns, Sweden Nigel Bevan, UK Torkil Clemmensen, Denmark Susan M. Dray, USA Vanessa Evers, The Netherlands Xiaolan Fu, P.R. China Yasuhiro Horibe, Japan Jason Huang, P.R. China Minna Isomursu, Finland Timo Jokela, Finland Mitsuhiko Karashima, Japan Tadashi Kobayashi, Japan Seongil Lee, Korea Kee Yong Lim, Singapore

Zhengjie Liu, P.R. China Lo¨ıc Mart´ınez-Normand, Spain Monique Noirhomme-Fraiture, Belgium Philippe Palanque, France Annelise Mark Pejtersen, Denmark Kerstin R¨ ose, Germany Dominique L. Scapin, France Haruhiko Urokohara, Japan Gerrit C. van der Veer, The Netherlands Janet Wesson, South Africa Toshiki Yamaoka, Japan Kazuhiko Yamazaki, Japan Silvia Zimmermann, Switzerland

Design, User Experience, and Usability Program Chair: Aaron Marcus Ronald Baecker, Canada Barbara Ballard, USA Konrad Baumann, Austria Arne Berger, Germany Randolph Bias, USA Jamie Blustein, Canada

Ana Boa-Ventura, USA Lorenzo Cantoni, Switzerland Sameer Chavan, Korea Wei Ding, USA Maximilian Eibl, Germany Zelda Harrison, USA

XIV

Organization

R¨ udiger Heimg¨artner, Germany Brigitte Herrmann, Germany Sabine Kabel-Eckes, USA Kaleem Khan, Canada Jonathan Kies, USA Jon Kolko, USA Helga Letowt-Vorbek, South Africa James Lin, USA Frazer McKimm, Ireland Michael Renner, Switzerland

Christine Ronnewinkel, Germany Elizabeth Rosenzweig, USA Paul Sherman, USA Ben Shneiderman, USA Christian Sturm, Germany Brian Sullivan, USA Jaakko Villa, Finland Michele Visciola, Italy Susan Weinschenk, USA

HCI International 2013

The 15th International Conference on Human–Computer Interaction, HCI International 2013, will be held jointly with the affiliated conferences in the summer of 2013. It will cover a broad spectrum of themes related to human–computer interaction (HCI), including theoretical issues, methods, tools, processes and case studies in HCI design, as well as novel interaction techniques, interfaces and applications. The proceedings will be published by Springer. More information about the topics, as well as the venue and dates of the conference, will be announced through the HCI International Conference series website: http://www.hci-international.org/ General Chair Professor Constantine Stephanidis University of Crete and ICS-FORTH Heraklion, Crete, Greece Email: [email protected]

Table of Contents – Part II

Part I: Novel Interaction Environments Measuring Human Interaction in Digital Television Using Profiles and Geolocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Valdecir Becker and Marcelo Kn¨ orich Zuffo

3

PC-Based Warning Mechanism System of Fall Risk in Elderly . . . . . . . . . Chih-Sheng Chang, Cherng-Yee Leung, and Jeih-Jang Liou

8

System of Systems for Sensor and Actuator Networks . . . . . . . . . . . . . . . . . Tiffany Elise Chua, Mark Merlo, and Mark Bachman

13

Smart Clothes Are New Interactive Devices . . . . . . . . . . . . . . . . . . . . . . . . . Gi-Soo Chung and Hee-Cheol Kim

18

Ebook Readers: An iPod for Your Books in the Cloud . . . . . . . . . . . . . . . . Ann-Marie Horcher and Maxine Cohen

22

The Ambient from the Young Passengers’ Perception in the Carriage of Taiwan High Speed Rail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jeichen Hsieh and Chan Yo Shan

28

An Effective Disaster Evacuation Assist System Utilized by an Ad-Hoc Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yasuki Iizuka, Kyoko Yoshida, and Kayo Iizuka

31

Locating Projectors Using Intensity of Reflected Beams Based on Phong Shading Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yukio Ishihara and Makio Ishihara

36

Embodied Communication Support Using a Presence Sharing System under Teleworking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yutaka Ishii and Tomio Watanabe

41

Visibility Experiment and Evaluation of 3D Character Representation on Mobile Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hiromu Ishio, Shunta Sano, Tomoki Shiomi, Tetsuya Kanda, Hiroki Hori, Keita Uemoto, Asei Sugiyama, Minami Niwa, Akira Hasegawa, Shohei Matsunuma, and Masaru Miyao Composite Context Information Model for Adaptive Human Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sukyoung Kim, Eungha Kim, and Youngil Choi

46

52

XVIII

Table of Contents – Part II

A Framework for a User Friendly Wireless Sensor Network Configuration System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Julia C. Lee and Lawrence J. Henschen FlexRemote: Exploring the Effectiveness of Deformable User Interface as an Input Device for TV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sang-Su Lee, Seungwoo Maeng, Daeeop Kim, Kun-Pyo Lee, Wonkyum Lee, Sangsik Kim, and Sungkwan Jung A Study of User Needs for the ‘Techno Kitchen’ . . . . . . . . . . . . . . . . . . . . . Martin Maguire, Colette Nicolle, Russell Marshall, Ruth Sims, Clare Lawton, Sheila Peace, and John Percival Consideration of the Human-Computer Interface in the Operation Room in the Era of Computer Aided Surgery . . . . . . . . . . . . . . . . . . . . . . . . Kazuhiko Shinohara ‘STISIM-Drive’ Meets ‘MotorcycleSim’: Using Driving Simulation Software to Develop a Unique Motorcycle Simulator for Rider Behavior Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alex W. Stedmon, David Crundall, Elizabeth Crundall, Rose Saikayasit, Editha van Loon, Alex Irune, Patrick Ward, and Neil Greig

57

62

66

72

76

AirportLogic: Usability Testing, Prototyping, and Analysis of an Airport Wayfinding Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bennett Stone and Yun Wang

81

Using on-Bicycle Rider Assistant Device While Cycling: A Hazard Perception Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chao-Yang Yang and Cheng-Tse Wu

85

Part II: Virtual and Augmented Environments Human–Robot Collaboration with Augmented Reality . . . . . . . . . . . . . . . . Siam Charoenseang and Tarinee Tonggoed Making Pixel Patterns Automatically for Camouflage - Using Color Information from Their Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Woon Jung Cho, Wonmi Ahn, Myung Shik Kim, Jeeyea Park, Seungduk Kim, and Kwang-Hee Han Virtual Bridge: AR-Based Mobile Interaction for Easy Multimedia Control of Remote Home Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DongJin Eun, Taik Heon Rhee, Seonghoon Kang, Minsuk Choi, Sangil Lee, and Hark-Joon Kim

93

98

102

Table of Contents – Part II

XIX

Design and Implementation of a Low-Cost Projected Virtual Reality System to Support Learning Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rodrigo G´ omez and Helmuth Trefftz

107

Interface Design to Support Situation Awareness in Virtual Puppetry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Keisha Harthoorn and Stephen Hughes

112

Immersive Video Game Based on Exercise Prescription . . . . . . . . . . . . . . . Daegun Kim and Changhoon Park

116

Assessing the Use of Cognitive Resources in Virtual Reality . . . . . . . . . . . William E. Marsh, Jonathan W. Kelly, Veronica J. Dark, and James H. Oliver

120

Augmented Reality Approach to Domestic Maintenance Tasks . . . . . . . . . Jorge Mart´ın-Guti´errez and Irene In´es Santos P´erez

125

Development of AR Display System for Dental Surgical Simulator . . . . . . Katsuhiko Onishi, Shota Ito, Yusuke Kawamura, and Hiroshi Noborio

130

Earthquake Disaster Prevention Support Tool – Visualization of Prevention Effectiveness by Utilizing Augmented Reality . . . . . . . . . . . . . . Kyoko Yoshida, Masahiro Urabe, Hayato Tsuchiya, Yasuki Iizuka, and Kayo Iizuka

134

Part III: Gestures, Gaze and Multimodality in HCI A Three-Dimensional Fingertip Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yangkeun Ahn, Kwangmo Jung, and Jiman Hong

141

Rule Based Trajectory Segmentation Applied to an HMM-Based Isolated Hand Gesture Recognizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jounghoon Beh, David Han, and Hanseok Ko

146

Head-Free, Remote Eye-Gaze Detection System with Easy Calibration Using Stereo-Calibrated Two Video Cameras . . . . . . . . . . . . . . . . . . . . . . . . Yoshinobu Ebisawa, Kazuki Abo, and Kiyotaka Fukumoto

151

Eye Position Effect on Audio-Visual Fusion Involves New Proposals for Multimodal Interface Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David Hartnagel, Alain Bichot, Patrick Sandor, and Corinne Roumes

156

A Virtual Mouse System Using Finger-Gestures of Twisting-in . . . . . . . . . Takashi Kihara and Makio Ishihara

161

Control of Five Finger of Computer Graphics Hand Using Electromyographic Signal Measured with Multi-channeled Small Laplacian Electrodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takuya Kimura and Akinori Ueno

166

XX

Table of Contents – Part II

Kinematic Analysis of Remote Target Pointing Hand Movements in a 3D Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yung-Hui Lee and Shu-Kai Wu Design and Implementation of Deformation Based Gesture Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wonkyum Lee, Sungkwan Jung, Sangsik Kim, Woojin Ahn, and Sang-su Lee

171

176

The Expansibility of User Interfaces Using Peripheral Multisensory Stimulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ju-Hwan Lee

180

Use of Hands-Free Mouse for Game Control . . . . . . . . . . . . . . . . . . . . . . . . . Moyen Mohammad Mustaquim

184

An Armband-Type Touch-Free Space Input Device for HCI . . . . . . . . . . . Dongwan Ryoo and Junseok Park

188

Modeling of Purchasing Behavior for Application on Merchandise Display and Package Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kotaro Suzuki, Nobuyuki Nishiuchi, and Mi Kyong Park

193

Decoding of Hand Shapes Based on ElectroMyoGraphic Signals during Playing Guitar Chords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hideaki Touyama and Masafumi Mizuguchi

197

Exploring Whole-Hand Gestures in a Tabletop Environment for Urban Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peter Vandoren, Karel Frederix, Karin Coninx, and Frank Van Reeth

201

Input Interface Using Fingertip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ryo Wada and Tomohiro Hase

206

Recognition Method for Foot Written Characters . . . . . . . . . . . . . . . . . . . . Masahiro Yonezawa, Takako Nonaka, and Tomohiro Hase

210

Part IV: Touch-Based and Table-Top Interaction Sounds in Space: 3D Audio Experiences through Tangible Navigation . . . Andrew Blakney and Sudhir Mudur

217

Multi-touch Surface Table with Multi-point Tactile Feedback . . . . . . . . . . Siam Charoenseang and Navakun Sribang

222

Suggested Considerations on the Design of Multi-touch Interfaces for Commercial Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ting-Han Chen

227

Table of Contents – Part II

XXI

A Study on the C/R Ratio of Direct-Operation Multi-touch Interface . . . Kuan-Hung Chen, Chun-Wen Chen, and Wenzhi Chen

232

Multi-touch Table as Conventional Input Device . . . . . . . . . . . . . . . . . . . . . Andreas Dippon, Florian Echtler, and Gudrun Klinker

237

Properties of Shadow-Cursor for Calibrating Screen Coordinates of Tabletop Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Makio Ishihara and Yukio Ishihara

242

Emotional Expression by a Person’s Grip on a Tactual Communication Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yasuhiro Matsuda and Tsuneshi Isomura

247

Effect of Target Size and Duration of Visual Feedback on Touch Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jeeyea Park and Kwang-Hee Han

252

Development of an Economical Haptic Stimulus Device . . . . . . . . . . . . . . . Greg Placencia, Mansour Rahimi, and Behrokh Khoshnevis

257

Feeling Home – Tangible Information Visualization in Smart Home Environments in Relation to the Concept of Transhumanism . . . . . . . . . . Florian Weingarten and Sahin Albayrak

262

Part V: Brain-Computer Interfaces and Brain Monitoring Calibration Time Reduction through Source Imaging in Brain Computer Interface (BCI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Minkyu Ahn, Hohyun Cho, and Sung Chan Jun

269

How Much Features in Brain-Computer Interface Are Discriminative? – Quantitative Measure by Relative Entropy . . . . . . . . . . . . . . . . . . . . . . . . Sangtae Ahn, Sungwook Kang, and Sung Chan Jun

274

EEG-Based Measurement of Subjective Parameters in Evaluations . . . . . Daniel Cernea, Peter-Scott Olech, Achim Ebert, and Andreas Kerren

279

Fundamental Study of the Pictogram-Scanning-BCI . . . . . . . . . . . . . . . . . . Hiroyuki Inada and Hisaya Tanaka

284

EEG Based Comparative Measurement of Visual Fatigue Caused by 2D and 3D Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Young-Joo Kim and Eui Chul Lee

289

A New Design of the Multi-channels Mobile and Wireless EEG System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chin-Teng Lin, Wan-Ru Wang, I.-Jan Wang, Lun-De Liao, Sheng-Fu Chen, Kevin Tseng, and Li-Wei Ko

293

XXII

Table of Contents – Part II

An Experimental Comparison of Brain Activity in Professional and Non-professional Sewers during the Use of Sewing Needles . . . . . . . . . . . . . Masako Omori, Yukari Morishita, and Asuka Kawakita

299

EEG-Based Measure of Cognitive Workload during a Mental Arithmetic Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Brice Rebsamen, Kenneth Kwok, and Trevor B. Penney

304

EEG Measurements towards Brain Life-Log System in Outdoor Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hideaki Touyama and Kazuya Maeda

308

Part VI: Ergonomics and Human Modelling Issues On the Applicability of Digital Human Models for Personal Equipment Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Thomas Alexander and Jessica Conradi Discussing Validation of 3D Character Animation Demonstrating Ushiro-Ukemi Pedagogical Progression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mauro Cesar Gurgel de Alencar Carvalho, Bruno Martins Carvalho, Felipe Leal de Paiva Carvalho, Heidi Dias Oliveira Junior, Gerson Gomes Cunha, Luiz Landau, and Est´elio Henrique Martin Dantas

315

320

The Provision of Digital Information in the Seat Comfort of the Seat Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kuen-Meau Chen, Siu-Tsen Shen, and Stephen D. Prior

325

The Effect of Damping in an Input Device on Human Positioning Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Koen Crommentuijn and Dik J. Hermes

330

Performance and Comfort When Using Motion-Controlled Tools in Complex Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ines Ann Heber, Michael Oehl, and Christine Sutter

335

Pen Tip Position Estimation Using Least Square Sphere Fitting for Customized Attachments of Haptic Device . . . . . . . . . . . . . . . . . . . . . . . . . . Masanao Koeda and Masahiko Kato

340

Corrected Human Vision System and the McGurk Effect . . . . . . . . . . . . . . Ladislav Kunc and Pavel Slav´ık

345

Facial Landmark Extraction for Lip Tracking of Patients with Cleft Lip Using Active Appearance Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nayoung Lee, Chuck Heaston, Ada Rey, Terry Hartman, and Carroll-Ann Trotman

350

Table of Contents – Part II

XXIII

Kansei Evaluation of the Projection for the Approach to Universal Design: Computerization of Tactile Sensibility . . . . . . . . . . . . . . . . . . . . . . . Miyong Lee, Kazuhiro Nishida, and Yoshihiro Narita

355

A Framework of Motion Capture System Based Human Behaviours Simulation for Ergonomic Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ruina Ma, Damien Chablat, Fouad Bennis, and Liang Ma

360

Visual Perception Model for Sense of Materials . . . . . . . . . . . . . . . . . . . . . . Wenhao Wang and Toshikazu Kato

365

The Effects of Stereoscopic Display Luminance and Ambient Illumination on Visual Comfort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pei-Chia Wang, Sheue-Ling Hwang, Kuan-Yu Chen, Jinn-Sen Chen, Jinn-Cherng Yang, and Hung-Lu Chang Preferred Setting of Keyboard and Mouse for Using a Supine Computer Workstation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hsin-Chieh Wu and Ho-Rong Chu

369

374

Part VII: Health and Wellbeing An Interactive Multimedia System for Monitoring the Progressive Decline of Memory in Alzheimer’s Patients . . . . . . . . . . . . . . . . . . . . . . . . . . Hala Al-Muhanna, Rawan Al-Wabil, Hailah Al-Mazrua, Noura Al-Fadhel, and Areej Al-Wabil Personal Smart Spaces for Diabetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manal AlBahlal and Jalal AlMuhtadi Quality and Usability Assessment for Health Information Websites: Can Commonly Used Evaluation Criteria Be Appropriately Applied to Assess Chinese-Language Websites? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chang Fang-Fang, Ku Chia-Hua, Wang Kung-Jeng, and Wu Wei-Li

381

386

391

Computer Interaction and the Benefits of Social Networking for People with Borderline Personality Disorder: Enlightening Mental Health Professionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alice Good, Arunasalam Sambhanthan, Vahid Panjganj, and Samuel Spettigue

395

Design Improvement Requirements for the Upper Extremity Rehabilitation Devices in Taiwan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lan-Ling Huang, Chang-Franw Lee, and Mei-Hsiang Chen

400

Observation Research of Consumer Behavior for Marketing Decision Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hideyuki Imai, Noriko Hara, and Toshiki Yamaoka

405

XXIV

Table of Contents – Part II

Exercise Reminder Software for Office Workers . . . . . . . . . . . . . . . . . . . . . . Ahsen Irmak, Rafet Irmak, and Gonca Bumin

410

Games for Health: Design Cognition-Focused Interventions to Enhance Mental Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hyungsin Kim, Viraj Sapre, and Ellen Yi-Luen Do

415

Promoting Positive Employee Health Behavior with Mobile Technology Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hyungsin Kim, Hakkyun Kim, and Ellen Yi-Luen Do

420

Believable Agents, Engagement, and Health Interventions . . . . . . . . . . . . . Christine L. Lisetti

425

Le-ADS: Early Learning Disability Detection System for Autism and Dyslexia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nor’ain Mohd Yusoff, Nor Syarafina Rusli, and Ruhaiza Ishak

433

Interaction Design of Encouraging Daily Healthcare Habit with Communication Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jun’ichi Osada, Tomoharu Yamaguchi, Ryohei Sasama, and Keiji Yamada Can Digital Signage Help Consumers Eat Healthier? . . . . . . . . . . . . . . . . . . Anicia Peters and Brian Mennecke Constraint-Based Nurse Rostering for the Valpara´ıso Clinic Center in Chile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Renzo Pizarro, Gianni Rivera, Ricardo Soto, Broderick Crawford, Carlos Castro, and Eric Monfroy

438

443

448

Connecting with Dysphonia: Human-Computer Interface for Amyotrophic Lateral Sclerosis Patients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chun-Yang Su and Ju-Joan Wong

453

Assessing Health Information Websites for Inclusion of Web 2.0 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adam Townes and Michelle Rogers

458

Encouraging Daily Healthcare Habit with Communication Robots . . . . . . Tomoharu Yamaguchi, Ryohei Sasama, Jun’ichi Osada, and Keiji Yamada

463

Part VIII: Learning, Education and Cultural Heritage The Evaluation of the Applicability of Distance Education in Vocational Colleges by the Students of Erzurum Vocational College, Computer Technologies Department, Erzurum, Turkey . . . . . . . . . . . . . . . . . . . . . . . . . Yusuf Ziya Ayik

469

Table of Contents – Part II

XXV

An Evaluation of SignBright: A Storytelling Application for Sign Language Acquisition and Interpersonal Bonding amongst Deaf and Hard of Hearing Youth and Caregivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Melissa M. Burton, Chad Harbig, Mariam Melkumyan, Lei Zhang, and Jiyoung Choi

474

Collaborative Analysis and Communities of Practice in Health Sciences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Juan Alberto Castillo M.

479

The Application of Interactive Media Display Technology in Environmental Science Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chun-Ching Chen and Chien-Ming Chen

484

Applying User-Centered Techniques to Develop a Radiology Teaching File System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marcelo dos Santos and Asa Fujino

489

The Study of the Interaction of Public Art with Digital Technology . . . . Shih Yin Huang and Ming-Shean Wang

494

Seven Wonders: An Interactive Game for Learning English as a Foreign Language in Junior High-School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . George Kapnas, Stavroula Ntoa, George Margetis, Margherita Antona, and Constantine Stephanidis Study-Buddy: Improving the Learning Process through TechnologyAugmented Studying Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . George Margetis, Stavroula Ntoa, Maria Bouhli, and Constantine Stephanidis

499

504

Improving Academic Performance and Motivation in Engineering Education with Augmented Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jorge Mart´ın-Guti´errez and Manuel Contero

509

Evaluation of Robot Based Embedded System Study Environment in Technical High School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yosuke Nishino and Eiichi Hayakawa

514

Extending Authoring for Adaptive Learning to Collaborative Authoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dade Nurjanah, Hugh Davis, and Thanassis Tiropanis

519

A Collaborative Tool for Communities of Practice to Share Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Justus N. Nyagwencha, Sheryl Seals, and Tony Cook

524

Classic Art for Modern People . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nikolaos Partarakis, Sokratis Kartakis, Margherita Antona, George Paparoulis, and Constantine Stephanidis

529

XXVI

Table of Contents – Part II

Information and Communication Technology (ICT) and Special Education System in the Kingdom of Saudi Arabia: A Case Study . . . . . Mukhtar M. Rana, Mohammad Fakrudeen, Mahdi H. Miraz, Sufian Yousef, and Alshammari Abderrahman Torqi ICT Training of Maestros of Primary Schools Located in Barrios Carenciados in Argentina. A Twofold Challenge: How They Can Master New ICT Technologies and Transform the Way They Teach . . . . . . . . . . . C. Osvaldo Rodriguez The Design of the Satellite Spaces for Informal Learning and Its Validity Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Syoko Shimora, Kazuyoshi Yamauch, and Natsuko Ohtake Window Control Interface to Attract Teacher’s Gaze Area for Watching a Reaction of Remote Learners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takumi Yamaguchi, Haruya Shiba, Naohisa Matsuuchi, Yusuke Nishiuchi, Kazunori Shimamura, and Takahiko Mendori Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

534

539

544

549

555

Table of Contents – Part I

Part I: Design Methods, Techniques and Knowledge Professional Graphic Designers Approaching Visual Interface Design . . . . Joanne Elizabeth Beriswill

3

Co-discovery Method and Its Application with Children as Research Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alessandra Carusi and Cl´ audia Mont‘Alv˜ ao

8

Get Your Mobile App Out the Door . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Heather Cottingham and Michele Snyder

13

Activity-Centered Design: An Appropriation Issue . . . . . . . . . . . . . . . . . . . Yvon Haradji, Germain Poizat, and Florence Mott´e

18

Conjoint Analysis Method That Minimizes the Number of Profile Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hiroyuki Ikemoto and Toshiki Yamaoka

23

Research on the Role of the Sketch in Design Idea Generation . . . . . . . . . Yuichi Izu, Koichirou Sato, and Yoshiyuki Matsuoka

29

Initial Perspectives from Preferences Expressed through Comparisons . . . Nicolas Jones, Armelle Brun, and Anne Boyer

33

Reducing Uncertainty in a Human-Centered Design Approach: Using Actor-Network Theory Analysis to Establish Fluid Design Guidelines . . . Ryan Kirk and Anna Prisacari

38

Verification of Centrality to Extract Proper Factors in Model Construction Process by Using Creativity Technique . . . . . . . . . . . . . . . . . . Kodai Kitami, Ryosuke Saga, and Kazunori Matsumoto

43

User-Centered Approach for NEC Product Development . . . . . . . . . . . . . . Izumi Kohno and Hiromi Fujii

48

Idea Creation Method Based on Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nozomi Koyatsu and Kazuhiko Yamazaki

53

Designing Interfaces for Home Energy Users: A Preference Study . . . . . . . Janelle LaMarche and Olga Sachs

58

Exploring the Relationship between Thinking Style and Collaborative Design Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chiung-Cheng Liao, Wenzhi Chen, and Hsien-Hui Tang

63

XXVIII

Table of Contents – Part I

Identifying Product Opportunity Based on Interactivity . . . . . . . . . . . . . . . Seungwoo Maeng, Daeeop Kim, Sang-Su Lee, and Kun-Pyo Lee

67

Idea Creative Method Based on Metaphor for Product Design . . . . . . . . . Takuya Mitsumaru

72

Persona-Storyboard Fusion: A Hybrid Approach to Improving Design Artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Michael Stewart, Jennifer Francois, Hongbo Zhang, and D. Scott McCrickard Studying Analysis Method for the Design Innovation . . . . . . . . . . . . . . . . . Takashi Inaba and Kazuhiko Yamazaki Balancing Trust and Automation Needs for Effective Home Energy Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hari Thiruvengada, Pallavi Dharwada, Anand Tharanathan, Wendy Foslien, Sriharsha Putrevu, and John Beane

77

82

86

Defining a Process for Cross-Product User Interface Consistency . . . . . . . Leslie Tudor and Cheryl L. Coyle

91

Tweaking HCI Methods for m-Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Al´ıcia Valls Saez, Muriel Garreta-Domingo, Gemma Aguado, and Marta Lopez Reyes

96

Part II: Usability and User Experience Contextual Awareness as Measure of Human-Information Interaction in Usability and Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Michael J. Albers

103

A Usability Model for Government Web Sites . . . . . . . . . . . . . . . . . . . . . . . . Deborah S. Carstens and Annie Becker

108

Usability Analysis of Website with Unconventional Layout . . . . . . . . . . . . Ro-Han Chang and Ying-Ya Su

113

Methodologies for Evaluating Player Experience in Game Play . . . . . . . . . Kimberly Chu, Chui Yin Wong, and Chee Weng Khong

118

How Does This Look? Desirability Methods for Evaluating Visual Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Edward S. De Guzman and Julie Schiller

123

An Analysis of Usage Patterns in Utilization of Interaction Styles . . . . . . Martin Dost´ al

128

On the Differences in Usage of Word Processing Applications . . . . . . . . . . Martin Dost´ al

133

Table of Contents – Part I

XXIX

Usability Study of TEL Recommender System and e-Assessment Tools United . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Beatriz Florian and Ram´ on Fabregat

138

Perceived Multimedia Quality: The Impact of Device Characteristics . . . Gheorghita Ghinea and Kyle J. Patterson

143

Usability Testing with Children: What We Have Overlooked . . . . . . . . . . . Hanayanti Hafit, Fariza Hanis Abdul Razak, and Haryani Haron

147

The Usability Assessment of Web-Based Learning Systems . . . . . . . . . . . . Chen-Wei Hsieh, Hong-Xon Chen, Yung-Chi Hsu, and Sherry Y. Chen

151

Using Pre-session Homework to Elicit More Insights during Web Usability Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Christopher Jewell and Franco Salvetti

156

Personalized ATMs: Improve ATMs Usability . . . . . . . . . . . . . . . . . . . . . . . . Armin Kamfiroozie and Marzieh Ahmadzadeh

161

The Effect of Induced Priming on Product Perceived Usability . . . . . . . . . Jihyun Kim, Myung Shik Kim, and Kwang-Hee Han

167

Who Are the People That Experience Soft Usability Problems? . . . . . . . . Chajoong Kim and Henri Christiaans

171

Gaze Analysis Tool for Web Usability Evaluation . . . . . . . . . . . . . . . . . . . . Takuo Matsunobe

176

Usability Evaluation for Software Keyboard on High-Performance Mobile Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takao Nakagawa and Hidetake Uwano

181

Usability Evaluation Method Employing Elements of “Thinking” and “Seeking” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nobuyuki Nishiuchi, Takehiro Ando, and Mi Kyong Park

186

Software Testing Method Considering the Importance of Factor Combinations in Pair-Wise Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ruoan Xu, Yoshimitsu Nagai, and Syohei Ishizu

191

Part III: Cultural, Cross-Cultural and Aesthetic Issues in HCI The Problematic of Beauty Performed in the Collaborative Action of Technology and Human . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HyunKyoung Cho

199

XXX

Table of Contents – Part I

The Politics of Collaborative Action of Technology and Human . . . . . . . . HyunKyoung Cho

204

What Makes Difference in Mobile Application Sales in Countries? . . . . . . Hyeyoung Eun, Hyunsuk Kim, and Sungmin Hong

209

hvMuseum: A Participatory Online Museum of Everyday Artifacts for Cultural Awareness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Young-ae Hahn

214

The Cross-Cultural Adaptation of the Work Role Functioning Questionnaire to Turkish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ahsen Irmak, Gonca Bumin, and Rafet Irmak

218

WARAI PRODUCT: Proposal to the Design Approach Designing the Product That Causes Laughter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takaaki Kamei and Kazuhiko Yamazaki

223

The Beauty Formation of Digital Media Projects via Sticking Jewels Techniques Applied to the Research and Education of Culture Creative Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jhih-Wei Lee and Chun-Ming Huang Leveraging Card-Based Collaborative Activities as Culturally Situated Design Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D. Scott McCrickard, DeMarcus Townsend, Woodrow W. Winchester, and Tiffany Barnes

227

232

Designing for Cultural Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anicia Peters, Britta Mennecke, Jos´e Camou, Kiraz Candan Herdem, and Lei Zhang

237

Can Culture Translate to the Virtual World? . . . . . . . . . . . . . . . . . . . . . . . . Raghavi Sakpal and Dale-Marie Wilson

242

Product Pleasure Enhancement: Cultural Elements Make Significant Difference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tyan-Yu Wu

247

Part IV: Cognitive and Psychological Issues in HCI An Optimal Human Adaptive Algorithm to Find Action-Reaction Word-Pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Arpit Agarwal, Rahul Banerjee, Varun Pandey, and Riya Charaya

255

Modeling Users in Web Transactional Tasks with Behavioral and Visual Exploration Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Areej Al-Wabil and Mashael Al-Saleh

260

Table of Contents – Part I

Evaluating Information Visualizations with Working Memory Metrics . . . Alisa Bandlow, Laura E. Matzen, Kerstan S. Cole, Courtney C. Dornburg, Charles J. Geiseler, John A. Greenfield, Laura A. McNamara, and Susan M. Stevens-Adams A Study on Human Error in the Interaction with the Computer Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Luiz Carlos Begosso, Maria Alice Siqueira Mendes Silva, and Thiago Henrique Cortez Psycognition: Cognitive Architectures for Augmented Cognition Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Karmen Guevara A Study on the Cognitive Differences between Beginners and Experts Regarding Cooking Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Keisuke Ishihara, Toshiki Yamaoka, Kazumi Tateyama, and Chinatsu Kasamatsu

XXXI

265

270

275

280

Red for Romance, Blue for Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ilyung Jung, Myung Shik Kim, and Kwang-Hee Han

284

Time: A Premise of Virtual Life . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hee-Cheol Kim

289

Extracts Cognitive Artifacts from Text through Combining Human and Machine Learning in an Iterative Fashion . . . . . . . . . . . . . . . . . . . . . . . . . . . Ryan Kirk

293

Modeling Human Behavior for Energy-Usage Prediction . . . . . . . . . . . . . . Anand S. Kulkarni, Karla Conn Welch, and Cindy K. Harnett

298

The Effect of a Visual Element on Musical Sensitivity . . . . . . . . . . . . . . . . Jieun Lee, Mitsuko Hayashi, and Masashi Nosaka

303

A Scrutinized Analysis Method of the Human Error Potential Due to the Introduction of New Digital Devices to Nuclear Power Plants . . . . . . Yong Hee Lee

308

Understanding Users by Their D.I.S.C. Personality through Interactive Gaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qin En Looi, Swee Lan See, Chi Shien Tay, and Gin Kee Ng

312

Modeling Attention Allocation in a Complex Dual Task with and without Auditory Cues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Brian McClimens and Derek Brock

317

XXXII

Table of Contents – Part I

Relationship between Emotional State and Physiological and Psychological Measurements Using Various Types of Video Content during TV Viewing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kiyomi Sakamoto, Shigeo Asahara, Kuniko Yamashita, and Akira Okada

322

Physiological Measurement Applied in Maritime Situations: A Newly Developed Method to Measure Workload on Board of Ships . . . . . . . . . . . Wendie Uitterhoeve, Marcella Croes-Schalken, and Dick Ten Hove

327

Physiological Correlates of Emotional State . . . . . . . . . . . . . . . . . . . . . . . . . Andrea K. Webb, Meredith G. Cunha, S.R. Prakash, and John M. Irvine A Study on the Operator’s Erroneous Responses to the New Human Interface of a Digital Device to be Introduced to Nuclear Power Plants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yeon Ju Oh, Yong Hee Lee, and Jong Hun Yun To Substitute Fast-Forward/Backward Keys for Numeric Keypad of TV Remote Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Horng-Yi Yu, Jui-Ping Ma, and T.K. Philip Hwang

332

337

342

Part V: Inclusive Design and Accessibility Digital Inclusion Index (DII) – Measuring ICT Supply and Usage to Support DI Implementation Planning Policies . . . . . . . . . . . . . . . . . . . . . . . Graziella Cardoso Bonadia, Nyvea Maria da Silva, and Cristiane Midori Ogushi Serious Game for Cognitive Testing of Elderly . . . . . . . . . . . . . . . . . . . . . . . Sangwoo Byun and Changhoon Park Leisure Activities for the Elderly–The Influence of Visual Working Memory on Mahjong and Its Video Game Version . . . . . . . . . . . . . . . . . . . . Chih-Lin Chang, Tai-Yen Hsu, Fang-Ling Lin, Chuen-Der Huang, and I.-Ting Huang

349

354

358

An Empathic Approach in Assistive Technology to Provide Job Accommodations for Disabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chien-Bang Chen

363

A Study on Interface Design Guidelines of Web Maps for Elder Users . . . Chun-Wen Chen and Kevin Tseng

368

Impact of Prior Knowledge and Computer Interface Organization in Information Searching Performances: A Study Comparing Younger and Older Web Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Aline Chevalier, Paulette Rozencwajg, and Benjamin Desjours

373

Table of Contents – Part I

XXXIII

Sketching Haptic System Based on Point-Based Approach for Assisting People with Down Syndrome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mario Covarrubias, Monica Bordegoni, and Umberto Cugini Helping Hands versus ERSP Vision: Comparing Object Recognition Technologies for the Visually Impaired . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marc A. Lawson, Ellen Yi-Luen Do, James R. Marston, and David A. Ross

378

383

Examining the Current State of Group Support Accessibility: An Expanded Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . John G. Schoeberlein and Yuanqiong Wang

389

Verbalizing Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lisa Tang and Jim Carter

394

Experiencing Accessibility Issues and Options . . . . . . . . . . . . . . . . . . . . . . . Lisa Tang, David Fourney, and Jim Carter

399

Adopting User-Centered Design for the Translating of Barrier-Free Design Codes/Regulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tsai-Hsuan Tsai, Wen-Ko Chiou, Huey-Yann Liao, and Tai-Xian Tseng

404

User Research for Senior Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kaori Ueda and Kazuhiko Yamazaki

409

Audio-Haptic Description in Movies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lakshmie Narayan Viswanathan, Troy McDaniel, and Sethuraman Panchanathan

414

Part VI: Social Interaction and On-line Communities Becoming Friends on Online Social Networking Services . . . . . . . . . . . . . . . Wonmi Ahn, Borum Kim, and Kwang-Hee Han

421

On-line Communication as a Part of the “Symbolic Politics” . . . . . . . . . . Evgeniy Ishmenev

426

A Displaying Method of Food Photos to Know Child’s Dietary Life for Parents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kenta Iwasaki, Kazuyoshi Murata, and Yu Shibuya

431

Memory Makers – The Experience of Camera Usage by Women . . . . . . . . Yu-Lung Kao and Ju-Joan Wong

435

Unique Motivation for Using Global Social Network Site in Korea . . . . . . Hyosun Kim and Kwang-Hee Han

440

XXXIV

Table of Contents – Part I

Color Image Effect of Online Community on Age: Focusing on Self-expression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jihyun Kim, Hyeryeong Kim, and Kwang-Hee Han

445

Gender in the Digital Age: Women’s Participation in Designing Social Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tˆ ania Cristina Lima and J´ ulio Cesar dos Reis

449

Study of Communication Aid Which Supports Conversation Held at Railway Station . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kaoru Nakazono, Mari Kakuta, and Yuji Nagashima

454

Sociable Tabletop Companions at “Dinner Party” . . . . . . . . . . . . . . . . . . . . Hye Yeon Nam and Ellen Yi-Luen Do

459

Quality of Community in Social Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kohei Otake, Tadakazu Fukutomi, and Tomofumi Uetake

464

Taiwanese Facebook Users’ Motivation and the Access of Information Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chun-Ming Tsai, Yu-Ting Huang, and Ji-Lung Hsieh

469

Connecting Generations: Preserving Memories with Thanatosensitive Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cyndi Wiley, Yun Wang, Ryan Musselman, and Beverly Krumm

474

Part VII: Work and Collaboration Introducing CAPER, a Collaborative Platform for Open and Closed Information Acquisition, Processing and Linking . . . . . . . . . . . . . . . . . . . . . Carlo Aliprandi and Andrea Marchetti

481

Secure Transmission of Medical Images by SSH Tunneling . . . . . . . . . . . . . Felipe Rodrigues Martinˆez Basile and Fl´ avio Cezar Amate

486

Service Components for Unified Communication and Collaboration of an SOA-Based Converged Service Platform . . . . . . . . . . . . . . . . . . . . . . . . . . Ki-Sook Chung and Young-Mee Shin

491

Fine-Grained Adaptive User Interface for Personalization of a Word Processor: Principles and a Preliminary Study . . . . . . . . . . . . . . . . . . . . . . . Martin Dost´ al and Zdenek Eichler

496

Development of Learning Achievement Index for Project Human Resource Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yusuke Emori, Takuya Furusawa, and Tsutomu Konosu

501

Table of Contents – Part I

Design and Development of Information Display Systems for Monitoring Overboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tadasuke Furuya, Atsushi Suzuki, Atsushi Shimamura, Takeshi Sakurada, Yoichi Hagiwara, and Takafumi Saito Fault Diagnosis of Induction Motors Using Discrete Wavelet Transform and Artificial Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . In-Soo Lee Study on Providing Multi-faceted Information on Technology Intelligence Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mikyoung Lee, Seungwoo Lee, Pyung Kim, Hanmin Jung, and Won-Kyung Sung

XXXV

506

510

515

Simulating Additional Area on Tele-Board’s Large Shared Display . . . . . Peter LoBue, Raja Gumienny, and Christoph Meinel

519

Components Based Integrated Management Platform for Flexible Service Deployment in Plant Factory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Aekyung Moon, Song Li, and Kyuhyung Kim

524

Development of the Many Nodes Connected and Simple Operated HD Remote Lecture System by Automatic Control . . . . . . . . . . . . . . . . . . . . . . . Takeshi Sakurada, Yoichi Hagiwara, and Tadasuke Furuya

529

Enhancing Flexibility of Production Systems by Self-optimization . . . . . . Robert Schmitt, Carsten Wagels, Mario Isermann, and Marcel Mayer

534

Do They Use Different Set of Non-verbal Language in Turn-Taking in Distributed Conferences? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hidekazu Tamaki, Suguru Higashino, Minoru Kobayashi, and Masayuki Ihara Floating 3D Video Conference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kun-Lung Tseng, Wen-Chao Chen, Tung-Fa Liou, and Kang-Chou Lin

539

544

Part VIII: Access to Information and Knowledge User Interface Design for the Interactive Use of Online Spoken German Journalistic Texts for the International Public . . . . . . . . . . . . . . . . . . . . . . . Christina Alexandris

551

How the Shapes of School Emblems for Colleges Convey Imagery . . . . . . Mu-Chien Chou

556

Extensible CP-Based Autonomous Search . . . . . . . . . . . . . . . . . . . . . . . . . . . Broderick Crawford, Ricardo Soto, Carlos Castro, and Eric Monfroy

561

XXXVI

Table of Contents – Part I

A Hybrid Approach to User Activity Instrumentation in Software Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Martin Dost´ al and Zdenek Eichler

566

Web Resource Selection for Dialogue System Generating Natural Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Masashi Inoue, Takuya Matsuda, and Shoichi Yokoyama

571

R&D Information System to Support Knowledge Creation . . . . . . . . . . . . Hyojeong Jin, Il Yeon Yeo, Youn-Gyou Kook, Byung-Hee Lee, and Jaesoo Kim

576

A New Method for Designing a Sitemap . . . . . . . . . . . . . . . . . . . . . . . . . . . . Soheila Khodaparasti and Marzieh Ahmadzadeh

580

On-line Handwritten Signature Verification Using Hidden Semi-Markov Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Daw-Tung Lin and Yu-Chia Liao

584

Accessing Previously Shared Interaction States through Natural Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Arthi Murugesan, Derek Brock, Wende K. Frost, and Dennis Perzanowski

590

Japanese Sentence Input Method Using Acceleration Sensor . . . . . . . . . . . Masaki Sugimoto, Kazufumi Nakai, Nobuo Ezaki, and Kimiyasu Kiyota

595

Where to Put the Search Concepts in the Search Result Page? . . . . . . . . . K.T. Tong and Robert W.P. Luk

600

Kansei Modeling on Visual Impression from Small Datasets . . . . . . . . . . . Shunsuke Uesaka, Kazuki Yasukawa, and Toshikazu Kato

605

A Movie Recommendation Mechanism Based on User Ratings in the Mobile Peer-to-Peer Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chian Wang and Dai-Yang Lin

610

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

615

Part I

Novel Interaction Environments

Measuring Human Interaction in Digital Television Using Profiles and Geolocation Valdecir Becker and Marcelo Knörich Zuffo Laboratório de Sistemas Integráveis, Departamento de Engenharia de Sistemas Eletrônicos, Escola Politécnica da USP, Av. Prof. Luciano Gualberto Trav. 3 n. 158, 05508-900 SP São Paulo Brasil [email protected], [email protected]

Abstract. This paper presents an audience measurement system for Brazilian digital free-to-air television, with profiles and geolocation introduction. The work uses activity theory framework for analyzing the context in which the audience and the interaction occur. There are nine user and behavior profiles proposed. The geographic distribution of the audience is shown by maps. Keywords: Audience measurement, profiles, geolocation.

1 Introduction This paper presents an audience measurement system for digital free-to-air TV using profiles and geolocation. Nine user and behavior profiles were developed from the activity theory analysis used in human-computer interaction for analyzing the context in which the interaction occurs, This allowed for a more detailed understanding of the viewers as compared with the traditional audience measurement. In addition, the system provides maps with the behavior and use in front of television. This information is useful in developing the content and precise advertising campaigns. Traditionally, the audience measurement is based on tuned channels, generating data about time before television and most watched channels [1]. The measurement is done by companies like Ibope [2], in Brazil, and Nielsen [3], in United States, which connect audimeter on the receivers [11]. The audimeters get the tuned channel and send the information to a database. This process is limited in scope (total ratings, share, gross rating points and some profiles, such as age and social class) and causes discomfort to the viewer because of the need of an audimeter [1, 4, 10]. The audience measurement system reported in this paper eliminates the need for such audimeters.

2 User Identification With the TV digitalization, one can use software that collects more detailed and precise data, without audimeter use [5,6]. The receiver [7] itself collects and sends C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 3–7, 2011. © Springer-Verlag Berlin Heidelberg 2011

4

V. Becker and M.K. Zuffo

data. In the system developed here, the receiver connects to a database, informing the current channel and the number of people watching television. This article describes the evolution of the system, incorporating behavior and television use profiles in addition to the information related to user's location. For this purpose, the users at the measurement point are registered (Fig. 1). The definition about which households and user should be in the sample is obtained statistically, as typical audience measurement panels. For demonstration purpose, we defined five measurement points with 20 users, distributed among nine profiles, described below.

Fig. 1. Identification screen, which allows the association between the user and the profiles

When switched on, the receiver authenticates on the database using an IP connection, and receives information about the registered users. Then the user identifies itself. The receiver sends the information to the database with the location of it. The database does the association between identification and the profiles stored on it.

3 Geolocation After the user identification, the receiver sends to a database the information with a Postcode number of the locality. The user registers the Postcode with the measurement software at the first time of turning on the TV with the measurement software. With this information, maps are prepared for the audience behavior, relating the location and with the profiles. The maps are generated using a methodology called Geofence [8], which designs a virtual perimeter with smaller maps in geographic areas pre-established by the Postcode number. Thus, it is possible to cross information between the user profiles and the audience, and locate them geographically.

Measuring Human Interaction in Digital Television Using Profiles and Geolocation

5

4 Audience Profiles The profiles were defined based on activity theory, which is employed to interpret the use of technology and to develop systems for human computer interaction. Activity theory is mainly used in the activity-focused design, which starts the analysis not with the user, but with the activities required to complete the tasks. The analysis starts in the context of system use and the task actions, considering the activities involved in the process and the artifacts used in mediation [9]. This is the point where the activity theory becomes relevant to the development of user profiles and behavior with regard to television. The viewer uses technology (artifacts and tools) to establish a relationship with the content within a social context. Artifacts mediate this communication. From this context and such mediation, nine user and behavior profiles were defined, which can be analyzed individually or together. 1. 2. 3. 4. 5. 6.

Sex – male or female composition of the audience Age – how old is the audience on the average Disability profile - the need for accessibility On line profile - internet access and frequency of use TV audience profile – the average time watching TV Commercial break profile - engagement in programming, change the channel during commercial breaks. 7. Crowd profile - how is the TV being watched, alone or accompanied. 8. Incoming profile – social class and average incoming 9. Technological profile - using technologies such as PC, cable TV, DVD player / Blu-ray / VCR, number of TVs The profiles one, two and eight are already used by the measurement institutes, like Nielsen and Ibope. The profiles one, two, three, eight and nine are previously recorded with information provided by users. The profiles four to seven are updated monthly, based on the stored data. These profiles can be applied to both fixed and mobile reception. The main difference between these two modes is the technology of collecting information, which should be transparent to those who access the information. Besides the division into two lines of analysis, based on people who see TV (profiles 1-7) and the environment they occupy (profiles 8 and 9), one can also add profiles to the composition (profiles 1, 2 , 3, 8 and 9) or audience behavior ( profiles 4, 5, 6 and 7). The information provided by the system can be accessed through an IP address. The user can select information by applying filters on the audience, share, on a particular profile or compose maps with profiles and behavior over the time. The intersection of information about audience and the different profiles allows detailed analysis about the composition and behavior of people watching TV. Fig 2 shows an example of the obtained data.

6

V. Becker and M.K. Zuffo

Fig. 2. Example screen with information about profile composition of the audience. In this case, with the Technological profile.

5 Conclusion This work proposes significant reformulation of the way the audience is traditionally analyzed. Currently the most important audience information is only related to the time that the channel is tuned on and the number of people watching it. To this information we have added details of behavior and location, allowing the broadcasters to better plan the TV schedule and providing advertisers better investment targets. The combination of profiles and the geographic area allows us to understand the viewers' tastes and motivations. This understanding is essential to improving the television content.

References 1. Webster, J.G., Phalen, P.F., Lichty, L.W.: Rating Analysis: The Theory and Practice af Audience Research, 3rd edn. Lawrence Erlbaum Associates, Inc., Londres (2006) 2. IBOPE. Instituto Brasileiro de Opinião Pública e Estatística, São Paulo (2010), http://www.ibope.com.br (accessed 01/10/2010) 3. Nielsen: São Paulo (2010), http://en-us.nielsen.com (accessed 01/10/2010) 4. AIMC. El uso de la tecnología en la medición de las audiencias, Spain (2008) 5. Becker, V., Zuffo, M.: Audience measurement system for digital television. In: Adjunct Proceedings - EuroITV 2010, pp. 111–114. Tampere University of Technology, Tampere (2010) 6. Becker, V., Zuffo, M.: Medição de audiência em ambientes de TV digital. In: Proceedings of the XXXIII Congresso Brasileiro de Ciências da Comunicação. Caxias do Sul, RS (2010)

Measuring Human Interaction in Digital Television Using Profiles and Geolocation

7

7. Costa, L.C.P., Herrero, R.A., Nunes, R.P., De Biase, M.G., Zuffo, M.K.: Over the Air Download for Digital Television Receivers Upgrade. IEEE Transactions on Consumer Electronics 56(1) (2010) 8. Munson, J.P., Gupta, V.K.: Location-based notification as a general-purpose service. In: Proceedings of the 2nd International Workshop on Mobile Commerce, Atlanta, Georgia, USA, pp. 40–44 (September 28, 2002) 9. Kuutti, K.: Activity Theory as a potential framework for human-computer interaction research. In: Nardi, B. (ed.) Context and Consciousness: Activity Theory an Human Computer Interaction, pp. 14–44. MIT Press, Cambridge (1995) 10. Balnaves, M., O’Regan, T.: Survey wars: the explosion of measurement in audience ratings. In: Papandrea, F., Armstrong, M. (Org.) Record of the Communications Policy & Research Forum 2009, Sydney, pp. 256–263 (2009) 11. Nielsen: Anytime Anywhere Media Measurement: A Progress Report, USA (June 14, 2007)

PC-Based Warning Mechanism System of Fall Risk in Elderly Chih-Sheng Chang1, Cherng-Yee Leung2, and Jeih-Jang Liou1 1

The Department of Product and Media Design, Fo Gung University, Taiwan 2 The Department of Industrial Design, Tatung University, Taiwan [email protected]

Abstract. It is difficult to recognize and classify movement patterns correctly. We have developed and evaluated a method to classify movement using contact forces during sit-to-stand (STS) movement in elderly people. We used the key points of the ground reaction force (GRF) to identify the sequence of important time points in the STS movement of elderly people. The data showed that the reaction forces had three distinctive patterns: typical-modal (t-modal), multimodal (m-modal), and incompetent-modal (i-modal). This is the first study to identify STS patterns based on objective force data and key points. The STS patterns can be used to supplement the subjective Berg Balance Scale (BBS) score to provide a more precise analysis of STS movement in the elderly. Keywords: BBS, GRF.

1 Introduction The population is aging in all developed countries, and as a result, the number of people requiring additional strategies to remain independent is likely to increase. Performance on STS tests has been found to predict subsequent disability, illness, falls, and hip fractures [4]. Currently, STS movement is investigated using observational performance tests to assess the risk of falling or performance measurements [7]. The observational performance tests, which tend to be subjective, use several assessment tools that combine measures of balance with measures of gait and mobility to determine a person’s risk of falling, e.g., the Berg Balance Scale [2] and the Tinetti Gait and Balance Assessment [8]. The BBS relies on interviews conducted by rehabilitation professionals and was developed as a performance-oriented measure of balance in elderly individuals [1]. Although the BBS has excellent test–retest reliability for the assessment of standing balance, Newton [5] reported a wide range of ability among elderly subjects who had the same mode score on the BBS. This finding suggests that the BBS score does not provide a precise measurement of performance ability. The present study describes the key points of STS movement in elderly people and introduces a PC-based approach to the assessment of STS patterns, with a view to supplementing the subjective BBS to provide a more precise analysis of STS movement in elderly people.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 8–12, 2011. © Springer-Verlag Berlin Heidelberg 2011

PC-Based Warning Mechanism System of Fall Risk in Elderly

9

2 Methods 2.1 Participants Twenty healthy elderly subjects (age, 68.95 ± 4.59) participated in the study; their mean height, weight, and body mass index (BMI) were 158.50 ± 5.85 cm, 59.39 ± 10.24 kg, and 23.55 ± 3.91, respectively. The selection criteria were (1) over 65 years old; (2) no acute medical illness in the past 3 months; (3) no orthopedic diagnosis; (4) no muscular disease; (5) BBS score ≥41 (6) BI score ≥60 (7) MMSE score >17; and (8) IADL score of ≥7. 2.2 Apparatus In the present study, the GRF was defined as the force applied to the ground by the buttocks and/or the feet. Vertical GRFs were recorded from two separate force platforms (size, 500 × 500 mm; accuracy, 0.20 N) composed of eight load cells. A personal computer simultaneously saved the two sets of GRF data at 1000 Hz using a 16-bit analog-to-digital converter acquisition board (NI PCI-6220; National Instruments Inc., Austin, TX, USA) and dynamic amplifiers. 2.3 Procedure The subjects sat on an armless chair of standard height (40 cm) on a buttock-force platform with their arms folded across the chest. A back support on the chair was used to ensure that the subjects’ trunks were leaning back in a standard position, and their bare feet were positioned on the leg-force platform. No other restrictions were imposed on the initial position. Each subject performed the task in a comfortable and natural manner and at a self-selected speed. 2.4 Measurements Figure 1 shows selected GRF parameters. Two curves (B, buttock weight; L, leg weight) describing the vertical GRF as a function of time were obtained from the force platforms. Curve T was the sum of forces B and L. The time taken to stand up was calculated from these curves. Bs, Ls, Ts, the onset of the B, L, and T curves: Tp, Lp, maximal GRF at curves T and L; B0, B at zero. Prior to analysis, the data were smoothed using the moving-average method. The force data indicated that the different sequences of the STS task exhibited specific characteristics that formed distinct patterns. Bs, Ls, and Ts occurred immediately after initiation of the action cue when the difference between the GRF at that time point and the previous value did not equal zero. Seat-off was defined as the time at which the thighs lost contact with the chair, when the instant of curve B was at B0. Tp and Lp occurred when the GRF for curves T and L, respectively, were maximal. The GRF oscillated following seat-off, and the STS phase ended when the GRF reached body weight.

10

C.-S. Chang, C.-Y. Leung, and J.-J. Liou

Fig. 1. The GRF parameters (N15)

2.5 Statistical Analysis The GRF time recordings were transformed logarithmically (Box & Anderson, 1955). A normal distribution test was performed, followed by a one-way ANOVA with repeated measures to evaluate the effect of Bs, Ls, and Ts. Significant results were followed up using the least significant difference (LSD) method as a post hoc test. The mean times of Tp, Lp, and seat-off were compared in a similar manner. A P-value <0.05 was deemed statistically significant.

3 Results The mean time ± the standard deviation (SD) for Bs, Ls, and Ts were 6.40 ± 0.24, 6.52 ± 0.27, and 6.60 ± 0.35, respectively. Different aspects of onset significantly affected the time of the activating moment (F = 5.67, P < 0.05). The LSD indicated that Bs occurred significantly earlier than Ls and Ts, and that Ls and Ts did not significantly affect the activating moment. The mean times for Tp, Lp, and seat-off were 7.53 ± 0.20, 7.59 ± 0.24, and 7.62 ± 0.21, respectively. These different time points significantly affected the time of the activating moment (F = 14.62, P < 0.001). Tp, Lp, and seat-off appeared in sequence; the LSD analyses indicated that the time of Tp, Lp, and seat-off were all significant. Based on the characteristics indicated by the force data, the STS movement sequences were classified into three patterns (Figure 2). The mean BBS score of the t-modal, m-modal, and i-modal patterns were 49.88 ± 2.64, 46.29 ± 4.54, and 43 ± 2, respectively. Figure 3 shows the relationship between the force pattern and the BBS score.

PC-Based Warning Mechanism System of Fall Risk in Elderly

11

Fig. 2. The three characteristic force patterns observed during the STS movement in elderly participants. From left to right: typical-modal (t-modal), multi-modal (m-modal), and incompetent-modal (i-modal).

Fig. 3. The relationship between the force pattern and the BBS score

4 Discussion Our data indicate that Bs is the onset of the STS movement in elderly people and that Ls and Ts are less accurate indices in comparison with Bs. Thus, using Ls or Ts to identify STS onset may underestimate the duration of the STS movement in elderly people. However, Chang and colleagues [3] reported that GRF component Bs and Ls, which were not significant in the present study, could both be used to mark the onset of the STS movement in healthy adults. The age of the participants may explain the difference between our findings and those of Chang. Tp and Lp are the maximum force points of the body and legs respectively. Their physical significance is different from that of seat-off, which represents the moment when the buttock leaves the seat. Although disagreement exists over the order of these three points [6], Chang and colleagues [3] reported the order of Tp, Lp, and seat-off in healthy adults using two high-frequency force platforms. Our findings in elderly subjects are consistent with those of Chang. The force data indicated that the STS movement task could be divided into three distinct patterns. In general, the curves of t-modal pattern are smooth, regardless of the B, L, or T curve. The curves of the m-modal pattern have multi-peak characteristics in curves B, L, and T. The curves of the i-modal pattern oscillate

12

C.-S. Chang, C.-Y. Leung, and J.-J. Liou

widely, particularly the B and L curves. The large amplitude swings in the i-modal pattern were caused by multiple outputs from one subject who tried, but was unable to complete the STS movements. In our study, the relationship between the BBS and STS patterns derived from the objective instrument was linear (Figure 5). Moreover, the SD of the t-modal and i-modal patterns on the BBS score was smaller than that of the m-modal pattern. Thus, the same BBS score may be associated with different STS patterns. For example, a subject with a BBS score of 48 may have shown either a tmodal (smooth movement) or m-modal (unstable movement) pattern, whereas another subject with a BBS score of 44 may have been able to stand up with no assistance (mmodal) or been unable to perform the STS task (i-modal) on the first try. Thus, the objective STS patterns provide a more precise assessment of balance in the elderly than does the more subjective BBS. Aknowledgments. This research was supported by a grant from the National Science Council (NSC99-2221-E-431-002) and (NSC99-2221-E-036-032).

References 1. Berg, K.O., Wood-Dauphinee, S.L., Williams, J.I., Gayton, D.: Measuring balance in the elderly: preliminary development of an Instrument. Physiotherapy Canada 41, 304–311 (1989) 2. Berg, K.O., Maki, B.E., Williams, J.I., Holliday, P.J., Wood-Dauphinee, S.L.: Clinical and laboratory measures of postural balance in an elderly population. Archives of Physical Medicine and Rehabilitation 73, 1073–1080 (1992) 3. Chang, C.S., Leung, C.Y., Liou, J.J., Tsai, W.W.: Evaluation of key points in the sit-tostand movement using two force platforms. Perceptual and Motor Skills 111(2), 496–502 (2010) 4. Lipsitz, L.A., Jonsson, P.V., Kelley, M.M., Koestner, J.S.: Causes and correlates of recurrent falls in ambulatory frail elderly. Journal of Gerontology 46, M114-M122 (1991) 5. Newton, R.A.: Balance screening of an inner city older adult population. Archives of Physical Medicine and Rehabilitation 78, 587–591 (1997) 6. O’meara, D.M., Smith, R.M.: The effects of unilateral grab rail assistance on the sit-to-stand performance of older aged adults. Human Movement Science 25, 257–274 (2006) 7. Rogers, M.E., Rogers, N.L., Takeshima, N., Islam, M.M.: Methods to assess and improve the physical parameters associated with fall risk in older adults. Preventive Medicine 36(3), 255–264 (2003) 8. Tinetti, M.E., Williams, T.F., Mayewski, R.: Fall risk index for elderly patients based on number of chronic disabilities. The American Journal of Medicine 80, 429–434 (1986)

System of Systems for Sensor and Actuator Networks Tiffany Elise Chua, Mark Merlo, and Mark Bachman University of California, Irvine, 3312 Engineering Gateway, Irvine, CA 92697 {tchua,mmerlo,mbachman}@uci.edu

Abstract. Sensor and actuator networks are often the backbone of control applications. They are used in many different fields, such as health care, home automation, and industrial control. Despite the prevalence of sensor and actuator networks, there is a lack of tools to support the rapid prototyping of sensing and control applications. We have developed a system to support a generic platform for rapid prototyping of sensing and control applications in which different sensors and actuators are brought together to perform specific functions. At the heart of the system-of-systems is the Control INterface to Devices and Instruments (CINDI), a small Linux-based box with a number of standard interfaces. The system was built with simplicity in mind and to support component reuse. Keywords: system of systems, sensor and actuator networks, rapid prototyping, pervasive computing, ubiquitous computing.

1 Introduction Many applications involve utilizing a controller to process inputs from various sensors and send outputs to various actuators. Such systems can be viewed as a “systemof-systems” of sensors and actuators, which on their own may be operational, but are networked together to perform a common specific function. For example, Hata et al [1] built systems of systems for elderly health care to prevent potential patient falls by producing an alert whenever the patient is about to get out of the bed. An ultrasonic transducer detected the presence or absence of a patient in bed, and by an air pressure sensor with air tube embedded in the mattress monitored the heart rate of the patient. Used together, the sensors provided a fairly accurate method of detecting when a patient is about to get out of bed using a fuzzy logic controller with two inputs. In industrial automation, systems of systems are commonly used for data acquisition, equipment monitoring, advanced process control and complex machine control. Although a simple programmable logic controller may do the job when the humanmachine interface only includes simple buttons and lights, an advanced system of systems for industrial control may be able to provide higher level data handling and higher processing speeds, with more extensive communications capabilities and better human machine interfaces. The development of custom-built or dedicated system of systems involving sensor and actuators usually requires microprocessor programming and communication C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 13–17, 2011. © Springer-Verlag Berlin Heidelberg 2011

14

T.E. Chua, M. Merlo, and M. Bachman

protocol development. In addition, custom-built systems typically consume a considerable amount of time and effort in maintenance. A number of platforms have been proposed to shorten the development cycle by providing modular and reusable components. Phidgets, the hardware equivalent of widgets, have been developed to provide prebuilt programmable modules such as simple I/O controllers, servo controllers, and proximity sensors for PC-based control [2,3]. Various phidgets can be combined to create a system of systems. Another example is the Atlas Sensor Platform, which consists of both hardware and middleware. It provides a hardware node built with a stack of interchangeable layers – the device layer, the processing layer and the communication layer. This allows the various types of devices to be connected and provides various options for connecting to the controller that resides in the middleware. The systems have been applied to various fields such as home automation [4] as well as industrial applications [5]. Both systems provide modularity and extensibility, and allow the user to setup a custom controller for a networked system of sensors and actuators through an application programming interface and specify the algorithms for processing inputs and producing outputs. In control applications, several types of controllers are commonly used. For example, aside from simple logic or sequential controllers, the proportional-integralderivative (PID) controller is one of the most common. With a few parameters, namely the variable set point, proportional, integral and derivative constants, a PID controller can be configured to bring an input variable to its set point by adjusting a manipulated variable based on the error difference between the input(s) and set point in a feedback loop. The PID controller can be cascaded in an application with multiple inputs, or built in parallel for multiple outputs. Fuzzy logic controllers are also commonly used to combine logic with linear control. Other frequently used components include data loggers, indicators, and alarms.

2 System Design and Implementation Here we discuss the development of a system to support a system of systems of sensor and actuators. The system is designed to be easy to use, with no programming required on the part of the user to get running. The system provides pre-built controllers for common tasks. To use, users simply specify the inputs and outputs of the controllers. In the following sections, we describe the system design and architecture of a Control INterface to Devices and Instruments (CINDI). 2.1 Block Diagram The block diagram is shown in Fig. 1. The system consists of CINDI middleware running in a Linux-based operating system. The system has a number of USB ports through which motes can be directly connected. The system is designed for hierarchical control, with CINDI at the top of the hierarchy, motes passing data to and from CINDI, and sensors, actuators and other devices connected to motes. Currently, only CINDI motes are supported, but future versions may allow the use of custom devices.

System of Systems for Sensor and Actuator Networks

15

CINDI

Mote 1

Mote n Mote 2

Sensor

Actuator Sensor

Actuator

Fig. 1. Block diagram of a system of systems for sensors and actuators

2.2 Architecture The architecture of CINDI is shown in Fig. 2. The top box represents CINDI itself, and the bottom box represents a CINDI mote connected to sensors and/or actuators. In the top box, the interface layer shows the various types of connections possible. At present, only the USB-Serial interface is implemented. The OS driver layer is provided by the Linux operating system. The device layer contains the input and output handlers. Information on device capabilities and how to receive and send data from and to the device resides in the configuration database. The controller layer contains controllers for input and output devices from the device layer. Controller configurations are also stored in the configuration database. A configuration tool is provided to configure devices and controllers in the system. CINDI Controller Layer Controller 1

Controller 2

Controller n

Config Database

Device Layer Dev 1

Dev 2

Dev 3

Dev n

OS Driver Layer USB Driver

Bluetooth Driver

USB

Bluetooth

Zigbee Driver

...

Interface Layer Zigbee

...

Config Tool TCP/IP

USB Application Layer ADCs / DACs

Mote

Sensors Actuators

Fig. 2. CINDI Architecture

16

T.E. Chua, M. Merlo, and M. Bachman

The bottom box shows the architecture of a CINDI mote. The mote has a USB interface, configured as a serial communication port. A PIC-based microcontroller on the mote receives data from and sends data to built-in analog-to-digital converters on the chip. Sensors and actuators connected to the CINDI mote. 2.3 Use Cases Use cases are shown in Fig. 3 to describe the operation of the system. To configure the CINDI box, an administrator selects the controller to be used for the CINDI mote. Controller setup involves creating a controller from one of the pre-built controller templates, and setting up the input, output, and timing. The controller can be configured to begin to operate at startup or to operate only when manually started. Once configured, the system is ready to use. A user may manually start a controller that is not configured to begin at startup.

Fig. 3. CINDI Use Cases

3 Summary and Future Work Designing a control system for a wide variety of system of systems is a challenging problem. The type of controller needed is different for each application domain. Simple controllers were built in this example, but the system can be extended in the future to provide different types of single-input-single-output (SISO), single-input-multipleoutput (SIMO), multiple-input-single-output (MISO), and multiple-input-multipleoutput (MIMO) controllers. The architecture of CINDI makes it suitable for hierarchical control, in which CINDI is at the top of the hierarchy and sensors and actuators communicate to each other through CINDI. Although the system is simple and does not support real-time applications, it is easy to use and setup for rapid prototyping. Currently the system only supports sensors and actuators connected through USB-based CINDI motes. Future versions may allow the user to configure the system to support custom devices or motes with a defined I/O protocol for reading/writing data through other types of interfaces such as Bluetooth and Zigbee.

System of Systems for Sensor and Actuator Networks

17

References 1. Hata, Y., Yamaguchi, H., Kobashi, S., Taniguchi, K., Nakajima, H.: A Human Health Monitoring System of Systems in Bed. In: IEEE International Conference on System of Systems Engineering, pp. 1–6 (2008) 2. Greenberg, S., Fitchett, C., Tn, C.: Phidgets: Incorporating Physical Devices into the Interface. In: CHI (2001) 3. Marquardt, N., Greenberg, S.: Distributed physical interfaces with shared phidgets. In: Proceedings of the 1st international conference on Tangible and embedded interaction - TEI 2007, ACM Press, New York (2007) 4. Bose, R., King, J., Pickles, S., Elzabadani, H., Helal, A.: Building Plug-and-Play Smart Homes Using the Atlas Platform. In: Proceedings of the 4th International Conference on Smart Homes and Health Telematic (ICOST), Belfast, the Northern Islands (2006) 5. Sollacher, R., Niedermeier, C., Vicari, N., Osipov, M.: Towards a Service Oriented Architecture for Wireless Sensor Networks in Industrial Applications (2007)

Smart Clothes Are New Interactive Devices Gi-Soo Chung1 and Hee-Cheol Kim2,* 1

Textile Fusion R&D Department, Korea Institute of Industrial Technology (KITECH), Ansan, Gyeonggi-Do, Korea 2 School of Computer Engineering/UHRC, Inje University, Obang-Dong 607, Gimhae, Gyeong-Nam, Korea [email protected], [email protected]

Abstract. Clothes are more than wears. They are interactive wearable devices. Recently, smart clothes have gained interest among researchers and practitioners, and shown us the potential as comfortable mobile computers in a ubiquitous world. This paper presents the two cases where the digital garment that we developed have been tested and applied to: communication and video display, and healthcare. While recognizing that the garment is still insufficient for ordinary users, we hope that its problems will be gradually resolved with the technological development. Keywords: Digital yarn, interactive device, smart clothes, wearable computing.

1 Introduction Worn on the user’s body, wearable computers should be equipped with four elements: context awareness, minimal cognitive overload, no user interaction with certain tasks, and integration with outfit [1]. Smart clothes are natural devices with which we wear for 24 hours, and best fitted to fulfill the elements. In this paper, we describe our digital garment and the two cases where it has been tested. Even if its development is at an experimental stage and tests are small scaled, we suppose that the cases can provide some hints and directions for further development.

2 Digital Garment The garment consists of both ordinary yarns and digital yarns enabling data transmission, as well as different electronic modules including sensors. Therefore, the development of the digital garment demands fusion technology of fiber engineering and information technology. Figure 1 shows an overview of the digital garment that we developed. *

Corresponding author of the paper.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 18–21, 2011. © Springer-Verlag Berlin Heidelberg 2011

Smart Clothes Are New Interactive Devices

19

Fig. 1. An overview of the digital garment

Among many technologies required, the production of digital yarns is most crucial. The first author of the paper has developed the digital yarn [2]. Its major material is copper alloy, which means it has conductivity. Therefore, the data can be transmitted through it. The yarn in the clothes transfers both analog and digital signals, and is divided into two parts (see Figure 2), core and outer parts. Its core part consists of 7 micro wires and a special resin. Its outer part is covered with dyed normal yarn. In particular, electric resistance of the yarn is much lower, 7.5 Ω/m, than previous conductive yarns [3-5].

Fig. 2. Digital yarn developed by Dr. Chung

20

G.-S. Chung and H.-C. Kim

3 Two Cases 3.1 Communication and Video Display The digital garment is made by ordinary yarns on the one hand, and digital bands on the other hand for the data transmission. Here, a digital band is a set of 10-30 digital yarns that are used for a communication line. Figure 3 shows us a testing environment. A CCD camera to record scenes is connected with a digital band by which recorded videos are transmitted real-time to an LCD monitor. This confirms that digital yarns transfer the video data properly without transmission delay. In fact, the digital yarn’s transmission speed is very high. It is about 80 Mbps. It implies that an 800MB movie file can be transmitted within one and a half minute approximately. However, we also have to admit of inconvenient aspects in the garment. The monitor in the garment is too heavy to carry all the time. The connectors between digital bands and ordinary yarns are a bit big so that users can feel uncomfortable when they wear the garment. However, we learned that this environment could be used for special tasks where inconvenience might be endured, rather than for ordinary lives.

Fig. 3. Video display with the digital garment

3.2 Healthcare The digital garment with biosensors to measure ECG (Electrocardiography) was tested for the purpose of healthcare. Here, the ECG signals acquired from the sensors are transmitted to the monitor through the digital yarns so that the user could see them in the monitor. The result was promising that the signals were accurate enough when compared with the data acquired without the garment. Thus, we found the huge potential for health management with the digital garment. However, we still need a technological advance to develop more comfortable and non-invasive sensors when the clothes are fitted with them. We are working in this direction [6].

Smart Clothes Are New Interactive Devices

21

Fig. 4. Measuring ECGs with the digital garment

4 Conclusion This paper gave a brief introduction to the digital garment, and presented the two cases in terms of its usage. The purpose for testing it in the cases was not to gain nor evaluate usability, but to ascertain the potential of new interactive clothes. Whereas technological advancement is requested, we believe that the digital garment will become a wearable computer in practice in a near future. Acknowledgments. This work is funded by the Korean Ministry of Knowledge Economy (#10033321).

References 1. Pezzlo, R., Pasher, E., Lawo, M.: Intelligent Clothing. IOS Press, Amsterdam (2009) 2. Chung, G.S., An, J.S., Lee, D.H., Hwang, C.S.: A Study on the Digital Yarn for the High Speed Data Communication. In: The 2nd International Conference on Clothing and Textiles, pp. 207–210 (2006) 3. Textile Wire version 03.01-e, Elektro-Feindrhat-Ag, Switzerland 4. Bekaert (a firm in Belgium), Bekinox VN 12/2x275/175S/316 L/HT, http://www.swicofil.com/bekintex.html 5. Linz, T., Kallmayer, C., Aschenbrenner, R., Reichl, H.: Embroidering Electrical Interconnects with Conductive Yarn for the Integration of Flexible Electronic Modules into Fabric. In: IEEE International Symposium on Wearable Computing, Osaka, Japan, (October 19–21, 2005) 6. Kim, H.C., Chung, G.S., Kim, T.W.: A Framework for Health management services in nanofiber technique-based wellness wear systems. In: IEEE Healthcom 2009, Sydney, Australia, pp. 70–73 (December 16–18, 2009)

Ebook Readers: An iPod for Your Books in the Cloud Ann-Marie Horcher and Maxine Cohen Nova Southeastern University, USA [email protected]

Abstract. The new way to carry books or literature is to not carry it at all. Not only have computing resources and data moved to the cloud, so has the latest novel or self-help book. The key to the acceptance of this technology is usable design that anticipates a reader’s needs while still invoking a comfortable form factor – the book [1]. Though e-book readers have been available for a decade, the latest wave of e-book readers has achieved a new level of technology acceptance. This trend and potential acceptance of future models can be understood by applying task-technology fit theory [2] to a usability study of the Barnes and Noble Nook and Amazon Kindle ebook readers. Keywords: ebook, digital libraries, usability, task-technology fit, gender bias.

1 Introduction Computing resources are moving to the cloud [3], along with digital repositories of information [4]. Beyond simply replicating the content of books, the ebook evolves that repository to a new form that retains the value of the book to deliver organized units of information, but has the flexibility for new paths for access [5]. The ebook has also transformed the previous form factor of books with variable weight and size to a totally mobile, but predictable shape [6]. Beyond the predictable size of the device, the content on the device has the advantage of flowing between a series of interoperable devices [7]. The typical mobile user is manipulating a group of on-body devices [8]. Initially the development of digital libraries focused on technology at the expense of usability [9]. Similar to the problems experienced in the design of secure systems [10], usability, like security, cannot be tacked on at the end. The greater problem was not whether digital repositories could be built, but whether a user community would accept online reading. Even younger segments of the population, such as university students [11], have traditionally preferred print. The technological limitations of computer screens created a usability barrier to the acceptance of online reading [5]. Even though the superiority of the ebook for sharing and delivery was acknowledged, the eye fatigue from prolonged on-screen reading, as in reading an entire book, made the format undesirable. The recognition of the impact of typography upon the readability of text preceded the evolution to the ebook [12]. The presentation of text information on a screen with less resolution than paper exacerbated the issue [13], resulting in decades of studies recommending guidelines for formatting screen text [14, 15]. Still the gold standard for clarity remained the black on white configuration of printed text [16]. The screen size limitations of small-screen C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 22–27, 2011. © Springer-Verlag Berlin Heidelberg 2011

Ebook Readers: An iPod for Your Books in the Cloud

23

devices like PDAs made clarity of display even more critical to usability [17]. Because of portability similar to the printed book, the small-screen devices became a natural target for the ebook. The navigation of material was second barrier. Moving through the electronic version of information was slower, and the users could not easily find certain phrases, or flip to other sections as they could using a printed version [5]. Once again, the form factors of small screen devices were a critical factor to ebook acceptance, particularly for the recreational reader. Using alternative navigation techniques such as grab-and-drag on a touch screen instead of a keyboard improved the user experience for interaction with large tables and maps [18]. The navigation experience paralleled the user experience with a physical book. Similarly, navigation of an ebook needed to equal the convenience of flipping through a physical book, or jumping to a section based on the table of contents.

2

Planning

The usability testing focused on three major task workflows commonly performed by a reader interacting with text material regardless of the storage media. These major areas are shown in Figure 1.

Fig. 1. Typical workflows for ebook reader usage

The user must progress through the previous capabilities to succeed at the tasks at the higher levels. The first two sections have close equivalents in the physical book. The enhanced capabilities address areas where the ebook has a design advantage over the printed counterpart. The test goals are to achieve feedback in the following areas that have been documented in the literature as usability barriers to ebook adoption. 2.1 Test Methods A comparison of usability between the ebook readers was chosen because this most directly parallels the user experience in field when selecting this capability. Experimental design in usability studies is as critical as the design of the interface being evaluated [19]. Small errors in design, such as confirmation bias in the task workflow [20] can overshadow the results. The replication of the normal environment of the user provides context for the usability and influences the results [21, 22]; therefore the usability testing was done in the field at various locations on a college campus with Internet availability.

24

A.-M. Horcher and M. Cohen

2.2 Participants Participants were required to have a computer and use it regularly. This established a minimal expertise level. The study participants were selected to represent a range of demographic groups in terms of age, sex, education, and computer expertise. The subjects were also selected for their interest and willingness to try out ebook readers. Table 1. Ebook reader usability barriers Usability Barrier Screen readability Navigation

Portability/Physical

Network connection

Literature Reference Kang, Y.-Y., Wang, M.-J. J., & Lin, R. (2009). Usability evaluation of E-books. Chen, N., Guimbretiere, F., Dixon, M., Lewis, C., & Agrawala, M. (2008). Navigation techniques for dual-display e-book readers. Kang, Y.-Y., Wang, M.-J. J., & Lin, R. (2009). Usability evaluation of E-books. Churchill, D., & Hedberg, J. (2008). Learning object design considerations for small-screen handheld devices. Walker, G. H., Stanton, N. A., Jenkins, D. P., & Salmon, P. M. (2009). From telephones to iPhones: Applying systems thinking to networked, interoperable products. Serif, T., & Ghinea, G. (2008). Mobile information access in the real world: A story of three wireless devices.

2.3 Procedure The participants performed a list of tasks with each ebook reader. Each task tested an aspect of ebook reader usability as described by the usability barriers in Table 1. The study compared the Barnes & Noble Nook and the Amazon Kindle 3. 2.4 Measurements As part of the entry questionnaire the subject was asked to assess their attitude toward ebook readers in a three point Likert scale. The same question was asked at the conclusion in an exit questionnaire. The overall time to complete all tasks was recorded, as well as number of times the user requested assistance. The observer noted the comments made by user while thinking aloud for later analysis. The tasks for which the subject requested assistance were also noted. The time and date of session was recorded. At the conclusion of the session, the observer interviewed the subject to get a usability rating for every task, and for each ebook reader. In addition to giving a rating on a 5 point Likert scale for 16 tasks on each ereader, the subject was also asked for an overall preference of Barnes &Noble ereader or Amazon Kindle ereader.

3 Results The selection of preferred software was not consistent with the software receiving the higher composite score of usability. In cases where it was inconsistent, the subjects

Ebook Readers: An iPod for Your Books in the Cloud

25

stated that one usability characteristic was so overwhelming that it out-weighed the other considerations. In some cases this overwhelming reason resulted in the selection of the Amazon Kindle. In the others, it resulted in the selection of Barnes &Noble. The subjects mentioned they liked how the ebook looked on the screen, and used that criteria alone to make their choice. This is an indication that the screen usability may have greater weight than the other usability barriers. Of the five usability barriers, the tasks related to navigation received the lowest scores. No subject felt navigation deserved the highest score. In contrast, the portability and screen readability tasks received the highest ratings for both ebook readers. This suggests that the use of e-ink has resolved the readability problem, even where the layout of the screens might create an accessibility issues. The ability of users to move around the book and manipulate the information needs further refinement to reach achieve the level of usability desired by the typical subject.

4 Discussion and Conclusions In addition to the results from the questionnaires, observations of each subject also yielded some interesting points. Neither age, nor computer expertise was a predictor or how the subjects perceived usability. Across the board, though, women reacted differently to the technology. There was greater anxiety among the female subjects about whether they were performing the test correctly. Furthermore, due to the design of the ebook readers, female subjects did not get a success experience when using the device correctly. The buttons were too stiff to respond to the typical feminine pressure, touch screens did not perceive the stroke of the women’s fingers. This suggests the devices are not calibrated to a norm that includes both genders. The impact of the device not responding is two-fold. The usability becomes suspect, and the subject loses confidence in their ability to control the device [23]. When linked to research that notes the impact of social-cultural issues on early adoption [24], this lack of usability to one gender points to a bias in design. Since men dominate the technology fields [25], and the early adopters [26], it would follow that first generations of device design reflect a male aesthetic. This is an area for further examination. Women also reacted differently to the size and shape of the device. In each case before beginning the task list, the female subjects examined the size, shape and texture of the device. Though not measured specifically in the results, this reaction to texture and color suggests there may some usability issues related to whether the subject feels they can hold the device comfortably.

References 1. Tsakonas, G., Papatheodorou, C.: Exploring usefulness and usability in the evaluation of open access digital libraries. Information Processing & Management 44(3), 1234–1250 (2008) 2. Goodhue, D.L., Thompson, R.L.: Task-Technology Fit and Individual Performance. MIS Quarterly 19(2), 213–236 (1995)

26

A.-M. Horcher and M. Cohen

3. Khmelevsky, Y., Voytenko, V.: Cloud computing infrastructure prototype for university education and research. In: Proceedings of the 15th Western Canadian Conference on Computing Education, pp. 1–5 (2010) 4. Kratky, A.: Re-thinking reading in the context of a new wave of electronic reading devices. In: Proceedings of the First international conference on Human-computer interaction, tourism and cultural heritage, pp. 1–11 (2011) 5. Kang, Y.-Y., Wang, M.-J.J., Lin, R.: Usability evaluation of e-books. Displays 30(2), 49– 52 (2009) 6. Golovchinsky, G.: Reading in the office. In: Proceeding of the 2008 ACM workshop on Research advances in large digital book repositories, pp. 21–24 (2008) 7. Walker, G.H., et al.: From telephones to iPhones: Applying systems thinking to networked, interoperable products. Applied Ergonomics 40(2), 206–215 (2009) 8. Ashbrook, D., Lyons, K.: Ensembles of on-body devices. In: Proceedings of the 12th international conference on Human computer interaction with mobile devices and services, pp. 503–504 (2010) 9. Blandford, A., et al.: Disrupting digital library development with scenario informed design. Interacting with Computers 19(1), 70–82 (2007) 10. Baskerville, R.: Information systems security design methods: implications for information systems development. ACM Comput. Surv. 25(4), 375–414 (1993) 11. Moyer, J.E.: Teens today don’t read books anymore: a study of differences in interest and comprehension in multiple modalities. In: Proceedings of the 2011 iConference, pp. 815– 816 (2011) 12. Tinker, M.A., Paterson, D.G.: Studies of typographical factors influencing speed of reading. VII. Variations in color of print and background. Journal of Applied Psychology 15(5), 471–479 (1931) 13. Hassan, T., Hu, C., Hersch, R.D.: Next generation typeface representations: revisiting parametric fonts. In: Proceedings of the 10th ACM symposium on Document engineering, pp. 181–184 (2010) 14. Garcia, M.L., Caldera, C.I.: The effect of color and typeface on the readability of on-line text. Computers & Industrial Engineering 31(1-2), 519–524 (1996) 15. Humar, I., Gradisar, M., Turk, T.: The impact of color combinations on the legibility of a Web page text presented on CRT displays. International Journal of Industrial Ergonomics 38(11-12), 885–899 (2008) 16. Scharff, L.V., Ahumada, A.J.: Why is light text harder to read than dark text? Journal of Vision 5(8), 812 (2005) 17. Churchill, D., Hedberg, J.: Learning object design considerations for small-screen handheld devices. Computers & Education 50(3), 881–893 (2008) 18. Burigat, S., Chittaro, L., Gabrielli, S.: Navigation techniques for small-screen devices: An evaluation on maps and web pages. International Journal of Human-Computer Studies 66(2), 78–97 (2008) 19. Gray, W.D., Salzman, M.C.: Damaged merchandise? a review of experiments that compare usability evaluation methods. Hum.-Comput. Interact. 13(3), 203–261 (1998) 20. Brown, S.A., et al.: Expectation confirmation: An examination of three competing models. Organizational Behavior and Human Decision Processes 105(1), 52–66 (2008) 21. Maguire, M.: Context of Use within usability activities. International Journal of HumanComputer Studies 55(4), 453–483 (2001) 22. Kay, J.: A test-first view of usability. Interacting with Computers 21(5-6), 347–349 (2009)

Ebook Readers: An iPod for Your Books in the Cloud

27

23. Hurtienne, J., et al.: Physical gestures for abstract concepts: Inclusive design with primary metaphors. Interacting with Computers 22(6), 475–484 (2010) 24. Park, S., Yoon, S.-H.: Separating early-adopters from the majority: The case of Broadband Internet access in Korea. Technological Forecasting and Social Change 72(3), 301–325 (2005) 25. Todd, K., Mardis, L., Wyatt, P.: We’ve come a long way, baby!: but where women and technology are concerned, have we really? In: Proceedings of the 33rd annual ACM SIGUCCS fall conference, pp. 380–387 (2005) 26. Chau, P.Y.K., Lung Hui, K.: Identifying early adopters of new IT products: A case of Windows 95. Information & Management 33(5), 225–230 (1998)

The Ambient from the Young Passengers' Perception in the Carriage of Taiwan High Speed Rail Jeichen Hsieh and Chan Yo Shan Tung Hai University, Dept. of Industrial Design, Taiwan [email protected], [email protected]

To explore the Taiwan High Speed Rail passenger experience of environmental attitudes, the concept car as a potential environmental factors (Table 1) to Hershberg & Cass environmental significance of the semantic scale (Table 2) (Xu Leiqing, 2004) as Measurement tools of environmental factors in the compartment of the bipolar adjective semantic differential method is usually the number must be used about 10 to 30 is more suitable, it will be adjusted to 2to 3 group to obtain more stable reference. Online questionnaire with Taiwan High Speed Rail train running video will be a reference tool to assist measurement. The video contents directly play to compare the experience feedback of fifty passengers.

Table 1. Potential environmental of high-speed rail travel

Potential environmental factors Passenger vehicles

乘乘

Changes in acceleration and deceleration Stationary traffic Speed Air Quality Temperature and humidity Path Noise Light Seating range Color

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 28–30, 2011. © Springer-Verlag Berlin Heidelberg 2011

The Ambient from the Young Passengers' Perception

29

Table 2. Scale selection

Concept Overall rating Evaluate the effectiveness Evaluation of the unique aesthetic Activities Space Effect Neat and clean Organization Temperatures Bright

Main scale

－the bad of useful － useless unique－ plain positive － negative small － spacious ugly － fine clean － dirty casual －mess warm － cold light － dark good

Alternative scale

－unpleasant friendly － hostile interesting － boring complex － simple private － public rough － smooth well organized － messy formal －free hot － cool cheerful － boring pleasant

The Carriage and Use of Environmental Objects According to the Taiwan High Speed Rail High Speed Rail website Inside information on the object for investigation of high-speed rail passengers traveling within the scope of physical activity when environmental objects, finishing as the standard range of cars and commercial cars (Table 2.3), does not contain a barrier-free space, toilets, vending machines, public telephones. Environment provides an object inside a passenger seat on the journey to provide the needs of Taiwan High Speed Rail interior finishing environmental objects such as table 3. Research show that although many high-speed rail seat space to facilitate the design and intimate personal use universal design considerations, there are still inadequacies in the actual use of passengers when the environment was due to restrictions that could result in the occurrence of passenger interference between the activities , are summarized as follows: 1. Table design in the seat behind the current seat back angle adjustment, back seat passengers to use the table immediately affected, and the appropriate table can not be smoothly adjusted. 2. For standard car seats configured as 3 +2, crowded, passengers will feel sitting in the middle than the narrow space available. 3. Standard Cabin narrow the distance between each seat, so passengers to use the table, almost the desktop can be temporary seat at a glance. 4. Seat can be rotated 180 degrees relative to facilitate understanding of the passengers sit, also happens to be equal to all the people are no table can be used.

30

J. Hsieh and C.Y. Shan Table 3. Environmental Investigation Picture

Passenger Information System

Standard class seats

Business class seats

Function LED electronic display boards to provide passengers the train station and meteorological information. Standard car seats configured as 2 +3 seating, seat back angle can adjust, and the seat can be rotated 180 degrees, seat length of 54cm, seat width 43cm, 55cm. Business Class seats for the 2 +2 seating configuration, seat back angle adjustment to fit passenger demand, the seat can be rotated 180 degrees, seat length 54cm, seat width 47cm, seat 57cm. Seats are provided for each seat dining table, maximum weight 10kg

Picture

Function Hidden hangers for objects, maximum weight 2kg

Hidden hangers Seat reading light at the top, providing visitors to read as the auxiliary light Reading light

Pedal

Pedal line with ergonomic design to provide passenger leg room for stretching and support

Personal music listening system allows travelers to choose five music stations Music system

Power outlets

Note: From the Taiwan High Speed Rail, http://www.thsrc.com.tw

Power outlet facilities enable travelers to use all kinds of electronic products such as notebook computers, PDA, etc.

An Effective Disaster Evacuation Assist System Utilized by an Ad-Hoc Network Yasuki Iizuka1, Kyoko Yoshida2, and Kayo Iizuka2 1

School of Science, Tokai University, Kanagawa, Japan [email protected] 2 School of Network and Information, Senshu University, Kanagawa, Japan {k-yoshida,iizuka}@isc.senshu-u.ac.jp

Abstract. This paper proposes a system that supports effective evacuation from danger using an original Distributed Constraint Optimization Problem (DCOP) algorithm DiSTaS-Anne (Distributed Tabu Search with simulated Annealing). The use of DCOP facilitates the assisted optimization of people’s evacuation timing, by estimating the location of evacuees. This system enables assistance in terms of evacuation guidance to be given to relieve congestion, by calculating evacuation routes via an ad-hoc network of evacuees’ mobile devices (phones, PCs, etc.), intercommunication function and location information. Keywords: optimization, Distributed Constraint Optimization Problem, disaster evacuation.

1 Introduction At times of disaster, or immediately prior to such periods, smooth evacuation is one of the key issues. However, it is also rather difficult to achieve, because people tend to rush forward when faced with disaster. Crowded evacuation passageways of buildings in the event of fire, or roads congested with cars containing people fleeing from predicted hurricanes are examples. Conversely, sometimes people do not take evasive action and do nothing based on the effect of mass psychology. Traffic jams occur when many people or vehicles attempt to traverse a limited route simultaneously and moreover, result in an evacuation time that is longer than would otherwise be required. By making each individual wait for a while to ensure an acceptable flow, avoiding traffic jams becomes feasible. The system the authors are proposing in this paper facilitates the assisted optimization of people’s evacuation timing, by estimating the location of evacuees. This system enables assistance to be given in the form of evacuation guidance to relieve congestion, by calculating evacuation routes via an ad-hoc network of evacuees’ mobile devices (phones, PCs, etc.), intercommunication function and location information. The requirements for the mobile devices are to be equipped with wireless LAN (Wi-Fi). Locations of evacuees are estimated by mobile devices using the positional relationship between the device and the wireless base station. The evacuation

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 31–35, 2011. © Springer-Verlag Berlin Heidelberg 2011

32

Y. Iizuka, K. Yoshida, and K. Iizuka

route and timing of each evacuee are calculated by distributed processing using evacuees’ devices connected by an ad-hoc network. Some related works concerning disaster evacuation simulation systems exist using multi agent, which tend to be focused on pre-existing simulation. The focus of this paper is on evacuation guidance at the time of disaster.

2 Issues about Disaster Evacuation When disaster occurs, to facilitate evacuation, information on the current situation of each site is useful [1], hence the authors’ proposal for a real-time disaster situation mapping (RDSM) system. Knowing “which evacuation route is safe” (by using this system or not), the next issue is “how to evacuate safely and effectively.” However, smooth and effective evacuation is not always easy. People tend to rush forward to passageways which are perceived to be safe, which results in congestion. The more people rush, the greater the congestion. The provision of additional appropriate information concerning evacuation guidance may make it feasible to evade congestion, and successfully shorten evacuation times. The requirement and process for the additional appropriate information concerning evacuation guidance is as follows: • • •

Determining people’s individual locations. Plan regional and short-term range evacuation. Provide information about appropriate evacuation routes and appropriate evacuation timing.

Usually, disaster countermeasures offices will be set up in organizations, in order to determine and provide appropriate evacuation routes. However, planning and providing appropriate information rapidly is not always easy. In addition, such countermeasures offices have limited resources. An effective disaster evacuation assistance system must cope with these issues.

3 Disaster Evacuation Assist System 3.1 Major Function The mobile devices in the system estimate their location via the global positioning system (GPS) or their relative position from Wi-Fi base stations, and communicate with devices in their vicinity via an ad-hoc network. Using the latter, devices autonomously bring up evacuation plans using preset knowledge inputs of evacuation and the current situation. Plans would be devised considering traffic of evacuation routes of places such as buildings, related constraints, and the need to avoid congestion. Evacuation routes and timing are abstracted from the plans and will be displayed on users’ (evacuees’) devices.

An Effective Disaster Evacuation Assist System Utilized by an Ad-Hoc Network

33

Receive Wi-Fi signals

Estimate location

Solve distibuted constraint optimization problem

Fig. 1. Usage Image of the Disaster Evacuation Assist System

3.2 System Features Features of the disaster evacuation assist system are listed below: •

•

•

Makes it feasible to swiftly calculate optimized solutions and use fewer computer resources: By using distributed constraint optimization problems (DCOP), which involve problem-solving logic used under a decentralized environment, it becomes feasible to solve large scale problems though the calculation amount of each agent (devices, in this case) is relatively small. For disaster evacuation, optimized solutions must be swiftly obtained, hence the authors propose the application of DiSTaS-Anne (Distributed Tabu Search with simulated Annealing) for computation logic. The details of DiSTaS-Anne are described in the next chapter. Makes it feasible to estimate location automatically: As for outdoors, estimating the location (degree of latitude and longitude) is feasible by using GPS. Mobile devices which are GPS-equipped are common these days. As for indoors, location information can be specified based on the relative position to Wi-Fi base stations, and enabled by matching the radio field intensity of Wi-Fi signals from at least two base stations. This scheme is in practical use such as PlaceEngine [6]. Usable in everyday situations (not just during disasters): This system is realized as an example of application software of mobile devices. By installing it on devices that users carry around every day, it can be used in emergencies.

4 Scheme for Calculating Appropriate Evacuation Route and Timing (DiSTaS-Anne) To ensure an appropriate evacuation route and timing, the authors propose the application of DiSTaS-Anne (Distributed Tabu Search with simulated Annealing) for a disaster evacuation assist system.

34

Y. Iizuka, K. Yoshida, and K. Iizuka

4.1 About DiSTaS-Anne DiSTaS-Anne is one of the methods of Distributed Constraint Optimization Problems (DCOP). DCOP has recently attracted attention as a problem solving framework under a decent decentralized environment in the field of artificial intelligence (AI)[2][3] DCOP is a framework that attempts to solve various constraint satisfaction problems (CSP), namely a classical AI problem solving framework, with an enhanced problem area in a decentralized environment. In DCOP, knowledge is expressed in the form of constraints. Formally㸪DCOP are defined as follows. A set of variables , , ⋯ , , in which each variable is assigned a value taken from a finite and discrete domain , , ⋯ , , and each variable is also assigned to multiple agents , , ⋯ , . Constraints : × → { , } are defined between and and a cost function : × → ℝ exists for each constraint.

．

,

=

,

=

ℎ

Where, < . The agents only have information about which is assigned to , which is constraint of , and cost function ∗ . At this time, the purpose of DCOP is to obtain an assignment for variable A that minimizes the summation ( )=∑ (A) of the cost function. In DCOP, the problem is solved while the values of the variables are exchanged between the agents tied together by constraints through message transmission. Well-known algorithms used to solve DCOP include ADOPT [4], and distributed stochastic search algorithms (DSA) [5]. When using DCOP for real-life problems, problems must be solved in distributed environments with minimal calculation resources and promptly . Under such circumstances, a fast and efficient approximate solution is desired. The authors had proposed Distributed Tabu Search with simulated Annealing (DiSTaS-Anne) in order to solve DCOP. DiSTaS-Anne is a method which is based on distributed search, with added features of Tabu Search and simulated annealing. Its redeeming feature is the ability to swiftly obtain high-quality approximate solutions. The evaluation result of this algorithm is shown in Figure 2.

4.2 Evacuation Planning Process Using DiSTaS-Anne The evacuation planning process using DiSTaS-Anne in the disaster evacuation assist system as proposed by the authors is described as follows: Knowledge such as evacuation routes or their capacity is preset in the form of a constrained condition. Devices can obtain information on numbers and distribution of neighboring evacuees by ad-hoc communication between devices. Inputting this information, situations are solved as DCOP. The solutions are shown to each evacuee on their devices. The solution are then recalculated and updated periodically to ensure optimized solutions.

An Effective Disaster Evacuation Assist System Utilized by an Ad-Hoc Network

35

Fig. 2. Effectiveness Evaluation of DiSTaS-Anne

5 Conclusion and Future Works In this paper, the authors proposed a disaster evacuation assist system using DiSTaSAnne. Using this system, effective evacuation derivation would become possible with limited system resources via ad-hoc communication and distributed processing. Flexibility of system resources is important when disaster occurs. Future work is planned, including the detailed formulation of constraint conditions and estimation of calculating time of DiSTaS-Anne applied to disaster evacuation assist system. The effectiveness will be verified by detailed simulation.

References 1. Iizuka, K., Iizuka, Y., Yoshida, K.: A Real-time Disaster Situation Mapping System for University Campuses. In: The 14th International Conference on Human Computer Interaction (2011) 2. Iizuka, Y., Takeuchi, I.: A Multiplexed Method for Distributed Constraint Optimization Problems. IPSJ Journal, Information Processing Society of Japan 50(12), 3136–3149 (2009) 3. Iizuka, Y., Suzuki, H., Takeuchi, I.: Multi-agent Tabu Search Method for Distributed Constraint Satisfaction Problems. The IEICE Transactions on Information and Systems (Japanese Edition) J90-D(9), 2302–2313 (2007) 4. Modi, P.J., Shen, W.-M., Tambe, M., Yokoo, M.: Adopt: Asynchronous distributed constraint optimization with quality guarantees. Artif. Intell. (161), 149–180 (2005) 5. Zhang, W., Wang, G., Xing, Z., Wittenburg, L.: Distributed stochastic search and distributed breakout: Properties, comparison and applications to constraint optimization problems in sensor networks. Artif. Intell. (161), 55–87 (2005) 6. PlaceEngine, http://www.placeengine.com/PlaceEngine

Locating Projectors Using Intensity of Reflected Beams Based on Phong Shading Model Yukio Ishihara1 and Makio Ishihara2 1

Innovation Center for Medical Redox Navigation, Kyushu University, 3-1-1, Maidashi, Higashi-ku, Fukuoka 812-8582, Japan [email protected] 2 Faculty of Information Engineering, Fukuoka Institute of Technology, 3-30-1, Wajiro-Higashi, Higashi-ku, Fukuoka 811-0295, Japan [email protected]

Abstract. In this work we focus on handheld projector-camera systems and discuss how to find out where and in which orientation those projectors are. We propose a way to find out them by measuring the intensities of the beams reflected on the screen. The point is that our system does not rely on fiducial markers which are usually stuck on the screen to help to locate the projector. Additionally, in our system, a laser-based projector is used, and the intensity of beams being projected and reflected on the screen is modeled based on Phong shading model. Through the evaluation, we show the potential of our system in HCI areas. Keywords: beam intensity, Phong shading model, laser-based projector, real-time calibration, projector-camera system.

1 Introduction Because of the downsizing and portability, projectors are easily installed anywhere and widely used in various ways as well as in common use such as presentations. Usually projectors are fixed on-centered to the screen. If projectors are put angled, the projected images will be distorted. Sukthankar et al. [1] built a projector-camera system where the projector is allowed to be off-centered to the screen. In the system the projector projects a calibration pattern and the camera captures it on the screen. From the correspondences between the calibration pattern and observed calibration pattern, the mapping between the camera and projector coordinates is obtained. And also the mapping between the camera and screen coordinates is obtained by observing the outline of the screen. These two mappings give the desired mapping between the projector and screen coordinates. By using the desired mapping, the projected image is corrected to appear not-distorted. In contrast to using one projector, other researches focus on multiple projectors to display a single tiled image on a large screen. Chen et al. [2] aligned 24 projectors to project a large seamless image. Likewise this system needs to be calibrated by obtaining the mappings mentioned above. On the other hand there is another way of use of a projector, which is allowed to move while projecting images. In this case, real-time calibration, or quick calibration, C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 36–40, 2011. © Springer-Verlag Berlin Heidelberg 2011

Locating Projectors Using Intensity of Reflected Beams Based on Phong Shading Model

37

is required whenever the projector moves. If the mapping between the projector and screen coordinates is obtained every time the projector moves, the calibration pattern will interrupt the projection. So, the calibration pattern cannot be used. Instead, the outline of the quadrilateral illuminated by the projector is used. Beardsley et al. [3] and Rehg et al. [4] built a projector-camera system capable of real-time calibration. Four fiducial markers are stuck on the screen, which show the four corners of a rectangle where the projected image is supposed to appear. The camera captures both the outline of the quadrilateral illuminated by the projector and the four fiducial markers. Before images are projected, they are pre-warped first, and then projected on the screen. This pre-warping is always performed. As a result, those images appear not-distorted within the four fiducial markers even while the projector is moving. In addition, projector-camera systems capable of real-time calibration are useful for HCI because the current orientation and position of those projectors are available all the time when in use. The work of Beardsley et al. [3] is a good example of projector-camera systems used in HCI. In their system, a mouse cursor is placed at the center of the illuminated quadrilateral while projecting the desktop image of MS-Windows within the four fiducial markers. The cursor moves when the projector is panned. At the time, as mentioned above, the desktop image is kept within the four fiducial markers. So the projector can be used as a mouse. However fiducial markers make the interaction area be limited around the markers. In this work, we aim to extend the interaction area by no use of fiducial markers. Previously we proposed a way to find out the orientation and position of a projector by measuring the intensities of the beams reflected on the screen [5]. As that technique needs no fiducial markers, the screen can be extended as far as the reflected beams are observed. In that work, a DLP projector was used and the intensities of the beams were modeled by their own defined equations. Recently, handheld laser-based projectors were released by AAXA Technologies Inc. Those projectors benefit from use of laser, so the projected images are always in focus. This feature could solve a problem posed in the previous work, that the projected images become blurred badly when the DLP projector is paned in either direction. Therefore, in this work the DLP projector is replaced with the laser-based projector. This will also simplify how the intensities of the reflected beams are calculated because of no need to take into account blurring caused by the projector being out of focus. That is, the intensities depend only on the positions of the projector and camera. In computer graphics area, Phong shading model [6] plays an important role in providing a way to give the intensities of the beams reflected on objects with respect to the positions of light sources and an observer. Thus, in this work, we take an advantage of Phong shading model to create the intensity curves of the beams reflected on the screen. And then, the orientation and position of the projector is obtained based on the intensity curves. The rest of this manuscript is organized as follows. In Section 2, how projectors are located is explained. And the intensities of the beams reflected on the screen are modeled in Section 3. In Section 4, our system is evaluated, and the concluding remarks are given in Section 5.

38

Y. Ishihara and M. Ishihara

Fig. 1. Our projector-camera system. A camera is attached on a projector, and captures the images projected by the projector.

2 How to Locate Projectors First of all it is assumed that projectors project beams of the same intensity in every direction when a one-colored image, such as a white-filled image, is projected. Fig. 1 illustrates our projector-camera system, where a camera is attached on a projector. ߠ௛ and ߠ௩ are the angles at which the projector is angled horizontally and vertically toward the screen, respectively. ‫ ݐ݊ܥ‬is the center of the projector's perspective on the screen, and ‫ ܦ‬is the distance between the projector and ‫ݐ݊ܥ‬. ܲ is a position on the screen. ߠଵ and ߠଶ are the horizontal and vertical angles to ܲ from ‫ݐ݊ܥ‬. ܱ is the nearest position on the screen from the projector. The angle at the projector between ܱ and ܲ is obtained as follows. ݂ఏ (ߠ௛ ,ߠ௩ ,ߠଵ ,ߠଶ ) = acos(…‘• ߠ௛ …‘• ߠ௩ …‘• ߠଵ …‘• ߠଶ െ …‘• ߠ௛ •‹ ߠ௩ •‹ ߠଶ െ •‹ ߠ௛ •‹ ߠଵ …‘• ߠଶ )

(1)

When intensities are observed at ܲ௧ , ܲ௟ and ܲ௥ as ‫ܫ‬௧ , ‫ܫ‬௟ and ‫ܫ‬௥ , respectively, then the projector can be located, or ߠ௛ , ߠ௩ and ‫ ܦ‬can be derived from the following simultaneous equations. ‫ܫ‬௧ = ݂ூ ቀ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡͲǡ ‫ܫ‬௟ = ݂ூ ቀ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡ െ ‫ܫ‬௥ = ݂ூ ቀ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡ

ఏ೛೓ ଶ ఏ೛೓ ଶ

ఏ೛ೡ ଶ

ሻǡ ‫݂ܦ‬ఏ ሺߠ௛ ǡ ߠ௩ ǡ Ͳǡ Ͳሻൗ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡͲǡ

ǡͲሻǡ ‫݂ܦ‬ఏ ሺߠ௛ ǡ ߠ௩ ǡ Ͳǡ Ͳሻൗ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡ െ ǡͲሻǡ ‫݂ܦ‬ఏ ሺߠ௛ ǡ ߠ௩ ǡ Ͳǡ Ͳሻൗ݂ఏ ሺߠ௛ ǡ ߠ௩ ǡ

ఏ೛೓ ଶ ఏ೛೓ ଶ

ఏ೛ೡ

ሻቁ,

(2)

ǡͲሻቁ,

(3)

ǡͲሻቁ,

(4)

ଶ

where ݂ூ (ߠ, ‫ )ܦ‬gives the intensity of the beam reflected at ‫ ݐ݊ܥ‬when the projector is angled by ߠ in either direction and is placed ‫ ܦ‬away from ‫ݐ݊ܥ‬. ߠ௣௛ and ߠ௣௩ are the horizontal and vertical angles of the projector’s perspective.

Locating Projectors Using Intensity of Reflected Beams Based on Phong Shading Model

39

3 Intensity Curves It is already explained in the previous section that the projector can be located if ݂ூ (ߠ, ‫ )ܦ‬is known. In this section, ݂ூ (ߠ, ‫ )ܦ‬is modeled by observing the intensities of beams by the camera. A white-filled image was projected and the reflected beams from ‫ݐ݊ܥ‬ were observed by the camera at 5ൈ6 = 30 positions: ‫ = ܦ‬50, 75, 100, 125, 150 cm and ߠ ൌ Ͳ௢ , ͳͲ௢ , ʹͲ௢ , ͵Ͳ௢ , ͶͲ௢ , ͷͲ௢ . Fig. 2 shows the intensities of the observed beams. The intensities range from 0 to 255. Eqn. (5) of Phong shading model was best fit to those intensities as shown in Fig. 2. ଵ

݂ூ (ߠ, ‫ܫ = )ܦ‬௖ = ௔ା௕ሺଶ஽ሻାሺଶ஽ሻమ ൛‫ܭ‬ௗ ‫ܫ‬௣ …‘• ߠ ൅ ‫ܭ‬௦ ‫ܫ‬௣ …‘• ఈ ሺʹߠሻൟ + ‫ܭ‬௔ ,

(5)

where ‫ܫ‬௖ and ‫ܫ‬௣ are the intensity of the beams observed by the camera and the one projected by the projector, respectively. ܽ, ܾ, ‫ܭ‬ௗ , ‫ܭ‬௦ , ߙ and ‫ܭ‬௔ define the intensity curves. ‫ܫ‬௣ was set to 255 because a white-filled image was projected. The set of optimal values was obtained by the steepest descent method as follows. ܽ ൌ9582.668, ܾ ൌ18.613, ‫ܭ‬ௗ ൌ14781.993, ‫ܭ‬௦ ൌ െ2148.044, ߙ ൌ1.026, ‫ܭ‬௔ ൌ 10.388.

Intensity

200 Observed Modeled

150 100 50 0 50

75

D (cm)

100

125

150

0

10

20

30

40

50

θ (deg.)

Fig. 2. Intensity curves

4 Evaluation Finally we evaluated our system. The projector and camera were put at each of the same 30 positions but ߠ௩ was kept to 0 in order to check the accuracy of ߠ௛ . The orientation of the projector and the distance from the screen, or ߠ௛ and ‫ ܦ‬were estimated by Eqn. (3) and (4). Fig. 3 shows the result. There are about ͸௢ error on ߠ௛ and 5 to 10 cm error on ‫ܦ‬. The estimation errors could be caused by fluctuation in the intensities of the observed beams.

40

Y. Ishihara and M. Ishihara

Fig. 3. Accuracy of estimated values of ߠ௛ and ‫ܦ‬

5 Conclusions In this work, we proposed a way to find out the orientation and position of a projector by measuring the intensities of the beams reflected on the screen. Specifically, there were two main changes from our previous projector-camera system. That is, a laserbased projector was used and intensity curves were modeled based on Phong shading model. From the evaluation, it showed the successful first step towards use of handheld projector-camera system without fiducial markers, especially for HCI purposes. In the future, we will discuss the effect of the estimation errors on usability in application use, and also improve the estimation process for higher accuracy.

References 1. Sukthankar, R., Stockton, R.G., Mullin, M.D.: Automatic keystone correction for cameraassisted presentation interfaces. In: Tan, T., Shi, Y., Gao, W. (eds.) ICMI 2000. LNCS, vol. 1948, pp. 607–614. Springer, Heidelberg (2000) 2. Chen, H., Sukthankar, R., Wallace, G., Li, K.: Scalable alignment of large-format multiprojector displays using camera homography trees. In: Proc. VIS 2002 IEEE Computer Society, pp. 339–346 (2002) 3. Beardsley, P., Baar, J.V., Raskar, R., Forlines, C.: Interaction Using a Handheld Projector. IEEE Computer Graphics and Applications 25(1), 39–43 (2005) 4. Rehg, J.M., Flagg, M., Tat-Jen, C., Sukthankar, R., Sukthankar, G.: Projected light displays using visual feedback. In: Proc. ICARCV 2002, vol. 2, pp. 926–932 (2002) 5. Ishihara, Y., Ishihara, M.: Locating a projector using the strength of beams reflected on a screen. In: Proc. AUIC 2007, vol. 64, pp. 47–50. Australian Computer Society, Inc. (2007) 6. Phong, B.T.: Illumination for computer generated pictures. Communications of the ACM 18(6), 311–317 (1975)

Embodied Communication Support Using a Presence Sharing System under Teleworking Yutaka Ishii1 and Tomio Watanabe2, 3 1

Information Science and Technology Center, Kobe University, 1-1 Rokkodai, Nada, Kobe, Hyogo 657-8501, Japan 2 Faculty of Computer Science and System Engineering, Okayama Prefectural University, 111 Kuboki, Soja, Okayama 719-1197, Japan 3 CREST of Japan Science and Technology Agency [email protected], [email protected]

Abstract. We have proposed the concept of a presence sharing system Ghatcha [GHost Avatar on a Telework CHAir] in which the users’ embodiment is not indicated by the avatars but by the chairs that suggest the presence of avatars. This system provides the same communication space for the users’ embodiment, thus creating a feeling of working alongside remote workers. In this paper, we develop prototype systems using a virtual or a real environment. And the effectiveness of the virtual prototype system is confirmed in the experiment. Keywords: Embodied Interaction, Avatar, Remote Communication, Telework, Remote Operating Chair.

1 Introduction A telework increases productivity and operational efficiency by offering employees the flexibility to work from their home offices. It would gain popularity as it can be utilized in different ways. However, the quality or efficiency of work might deteriorate as a result of a telework as it leads to a sense of isolation or a lack of concentration. Thus, it is important to examine remote collaboration support in detail. Remote collaboration has various purposes and applications, and it is expected to support for each situation. The subjects of this research are not remote users performing a group task but individual users performing their own specific tasks wherein all their co-workers also perform tasks with the same aim such as a job of a home-based worker or individually pursuing online distance learning. In the case of tasks that are not synchronized, however, the video image might contradict our expectations. In order to solve the problem, Honda et al. proposed a virtual office system “Valentine” using an awareness space and provided a work support environment for home-based workers [1]. Moreover, various media communication methods have been proposed for practical use, such as the design of a communication environment, which aims at maintaining and fostering human relations for family members living apart, or a communication system wherein the furniture or daily necessities, which are separated in different rooms, can be linked [2],[3]. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 41–45, 2011. © Springer-Verlag Berlin Heidelberg 2011

42

Y. Ishii and T. Watanabe

However, when a user’s own avatar is used as a communication media for an embodied interaction, many issues arise with regard to the relation between the users and their avatars. For example, if a human-type avatar is used, the correspondence of the user’s motion and that of the avatar’s would be hindered by input devices. Otherwise, the appearance of an avatar cannot appropriately represent a user’s embodiment. The embodiment is not always indicative of the avatar information in the input/output of the smooth interaction system. It would be useful to integrate the embodiment with the environment information for the development of an effective interaction support system. Hence, this paper proposes a new presence sharing system called Ghatcha: GHost Avatar on a Telework CHAir [4]. The Ghatcha system is based on the embodiment of the environment information of the chair motion rather than that of the avatar. The present study aims to recreate the environment in which remote users interact with their co-workers in the same virtual office, and enhance their motivation in performing their tasks. In this paper, a prototype of the system using a virtual environment is developed, and the effectiveness of this system is demonstrated by an evaluation experiment.

2 Development of the Prototype System Using CG The prototype system using a CG avatar was developed. In this system design, the chair motions are measured by various sensors such as a gyroscope, an accelerometer, or a magnetic sensor. The virtual chair motions are represented based on the measurements, and are shared on the network. The mutual motions of each user are transmitted to the office model from the shared communication space. This collaborative system determines the third interaction space with the chairs for each remote coworker. The CG prototype system is generated by an HP workstation xw4200 (CPU: Pentium4 3.6GHz, RAM: 512MB, NVIDIA Quadro FX3400), OS: Windows XP Professional SP1, and DirectX9.0b. The frame rate is 30 fps. The chair motions are measured by a laser sensor mouse (Logicool MX Air) attached under the chair. The communication scene using the system is shown in Fig. 1. This example displays only the user’s human type avatar.

Fig. 1. Communication scene using the prototype system

Embodied Communication Support Using a Presence Sharing System under Teleworking

43

3 Development of the System in the Real Environment Not only a virtual environment, the prototype system in a real environment was developed as shown in Fig. 2. The system was constructed by the 4 servo motors through the USB cable. User's chair motions were measured by the same way as the virtual system. User can put any objects for the substitute of his/her partners on the chairs, including the one of himself/herself. This miniature system provide interaction awareness between users using their embodiments.

Fig. 2. Prototype system in the real environment

4 Evaluation of the Experiment Using the Virtual System 4.1 Experimental Setup The system evaluation experiment was performed by the prototype system using CG in the previous chapter 2. The subjects consisted of 10 pairs, and they worked on a simple task wherein they made paper cranes by folding pieces of paper. The task was repeated twice using two scenes: one where the chair system was connected with the motions of the user and another where they were not connected. The subjects were ordered to fold the papers as much as they could. After the task was finished in each scene, the user’s behavior was observed during a waiting period of 3 minutes. The only information that was shared through the system was the motion of the chair. Only the user’s human type avatar was displayed in addition to both the users’ chairs. Thus, the user makes his/her presence felt not as the chair but as the avatar. The partner’s avatar was not represented in the virtual space. The subjects answered the questionnaire after the task in each scene. They were provided an explanation of the conditions and the setting of the experiment, and they agreed to the experiment before the experiment started. The time taken to conduct the experiment was about 40 minutes on average including the waiting time and the time taken to answer the questionnaire. The example of the evaluation experiment scene is shown in Fig. 3.

44

Y. Ishii and T. Watanabe

Fig. 3. Example of an evaluation experiment

4.2 Sensory Evaluation The two scenes were evaluated on a seven-point bipolar rating scale ranging from –3 (lowest) to 3 (highest), in which 0 denotes a moderate score. For the sake of convenience, the results of the means and the standard deviations are shown in Fig. 4. The questionnaire consisted of eight categories: four categories on the impression of the work and the other four categories on the evaluation of media communication. In most of the categories, the significant difference between the two scenes was obtained by administering the Wilcoxon’s rank sum test; a significance level of 0.1% for the items of “Do you feel like sharing the same space with a partner?”, and “Do you feel like working together with a partner?” A significance level of 1% was obtained for the items “Do you enjoy your task?” “Do you believe that you could associate yourself with the character?” and “Do you recognize a partner’s motion?” The effectiveness of the prototype system is evinced by the positive evaluation of each category in the scene that the chair motions were connected. The scene that the chair motions weren’t connected has a negative evaluation that is significant at the 5% level for the item “Are you bored by your task?”

Fig. 4. Results of the questionnaire

Embodied Communication Support Using a Presence Sharing System under Teleworking

45

In addition, comments such as “I didn’t move too much while folding pieces of paper into the figure of a crane” and “I was pleasant to manipulate my chair after working the task” are obtained as responses of a free description of the respondents’ opinions in the questionnaire. The effectiveness of the system can be observed in the waiting time after working rather than during the working time.

5 Conclusion In this paper, the prototype systems using a virtual or a real environments were developed. And, the evaluation experiment was performed using the virtual prototype of the system, and the effectiveness of the system was demonstrated by a sensory evaluation.

References 1. Honda, S., Tomioka, H., Kimura, T., Oosawa, T., Okada, K., Matsushita, Y.: A companyoffice system “Valentine” providing informal communication and personal space based on 3D virtual space and avatars. Information and Software Technology 41, 383–397 (1999) 2. Miyajima, A., Itoh, Y., Itoh, M., Watanabe, T.: TSUNAGARI Communication: The design of the communication environment which aimed at keeping and fostering human relations and a field test for family members living apart. The Transactions of Human Interface Society 5(2), 171–180 (2003) (in Japanese) 3. Tsujita, H., Tsukada, K., Siio, I.: SyncDecor: Appliances for Sharing Mutual Awareness between Lovers Separated by Distance. In: CHI 2007 Conference Proceedings and Extended Abstracts, Conference on Human Factors in Computing Systems, pp. 2699–2704 (2007) 4. Ishii, Y., Osaki, K., Watanabe, T.: Ghatcha: GHost Avatar on a Telework CHAir. In: Proc. of HCI International 2009, pp. 216–225 (2009)

Visibility Experiment and Evaluation of 3D Character Representation on Mobile Displays Hiromu Ishio1, Shunta Sano1, Tomoki Shiomi1, Tetsuya Kanda1, Hiroki Hori1, Keita Uemoto1, Asei Sugiyama2, Minami Niwa1, Akira Hasegawa1, Shohei Matsunuma3, and Masaru Miyao1 1

Graduate School of Information Science, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8601, Japan 2 School of Engineering, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8601, Japan 3 Nagoya Industrial Science Research Institute, Noa Yutsuya Bldg 2F, 1-13, Yotsuya-dori, Chikisa-ku, Nagoya 464-0819, Japan [email protected]

Abstract. In the experimental study, we measured visibility and readability of text characters presented on a small 3D liquid crystal display (LCD) and evaluated features of the 3D character representation for application to mobile devices. For the visibility evaluation, we focused on time lag for first recognition, time required for full reading, and maximum distance of 3D objects popping out from the fixed display. We also report the dependency of the results on the age of subjects in comparison with 2D and 3D representations. Keywords: 3D character representation, liquid crystal display (LCD), mobile display.

1 Introduction Recent advances in stereoscopic vision technology have brought us wide range of display applications ranging from large TVs and public screens to small devices such as mobile phones and portable games. In this trend, it is most probable that 3D representations not only of scenery images but of texts will become widely used on such devices in the near future. A useful application may be “augmented reality” (AR) [1], where computer generated additional information such as text characters, images, etc. are superimposed on a real visual environment in order to complement human sensory information. In AR applied to mobile devices, text information may need to be presented more effectively in a stereoscopic way depending on the location of a real object to be augmented in 3D. Nevertheless, there is no index available in such a case to identify visibility and readability of the text information. Therefore, it is important to investigate visibility and readability of text characters displayed on mobile 3D devices [2, 3]. In the experimental study, we measured visibility and readability of characters presented on a small 3D liquid crystal display (LCD) and evaluated features of the 3D C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 46–51, 2011. © Springer-Verlag Berlin Heidelberg 2011

Visibility Experiment and Evaluation of 3D Character Representation on Mobile Displays

47

character representation for application to mobile devices. For the visibility evaluation, we focused on time lag for first recognition, time required for full reading, and maximum distance of 3D objects popping out from the fixed display. We also report the dependency of the results on the age of subjects in comparison with 2D and 3D representations.

2 Method The subjects in this study were 109 healthy males and females (age 20-78). The experiment consisted of three parts (Exp. I, II and III) which were carried out in sequence to each of them. We obtained informed consent from all subjects, and approval for the study from Ethical Review Board in the Graduate School of Information Science at the Nagoya University. Experiment I. Visibility Evaluation in Pop-out Distance Measurement of 3D Images. The aim was to evaluate visibility of 3D images for each subject and, for those who succeeded in stereoscopic view, to measure maximum distance of a 3D object virtually popping out from the fixed display. The details of experimental setup were as follows: We first measured pupil distance (PD) of each subject. Then we set a small (3.8 inch) LCD 40 cm in from of subjects under illumination condition of 500 lx. It is a naked-eye 3D display (a test model of SHARP LYNX 3D SH-03C) with resolution of 800 x 480 pixels and can present 3D images by using parallax barrier system that separates two different images to present to the right and left eyes with parallax barrier on the display (Fig.1, Fig. 2.).

Fig. 1. SHARP LYNX 3D SH-03CTM

Fig. 2. Parallax barrier display

Fig. 3. 3D images with a spherical object moved forward and back continuously with a cycle of 10 seconds

48

H. Ishio et al.

There we presented 3D images where a spherical object moved forward and back continuously with a cycle of 10 seconds (see Fig. 3). The spherical object movies called Power 3D™ were kindly supplied by Olympus Visual Communications, Corp. In principle, the spherical object appears as a 3D image at 40 cm (i.e., the location of LCD) and moves toward the subjects in front of them. We asked them to gaze at the center of the spherical object and to answer whether they could view the object in 3D during the motion (Fig.4). We further asked those who succeeded in stereoscopic view to point out the nearest location of the object with their finger, and measured the maximum distance of the object popping out from the fixed display. Experiment II. Visibility Evaluation in Time Lag Measurement for First Recognition of 3D Text Characters. The aim was to evaluate visibility of 3D text characters for each subject and, for those who succeeded in stereoscopic view, to measure time lag for first recognition of the 3D text characters. The details of experimental setup were as follows: The subjects first watched the small 3D display which showed nothing (i.e., black background only). At some point, we presented text characters (lowercase and uppercase alphabet letters and numbers aligned still at random, see Fig. 5) in 3D on signal to the subjects. Then we measured the time required for first recognition of the 3D characters as they are. For comparison, we chose large (H 5.0 mm x W 1.3 mm each) and small (H 2.5 mm x W 0.63 mm each) size of characters in the experiment.

Fig. 4. Succeeded subjects in stereoscopic view were asked to point out the nearest location of the object with their finger, and measured the maximum distance of the object popping out from the fixed display

Experiment III. Readability Evaluation for Full Reading of 3D Text Characters. The aim was to evaluate readability of 3D text characters for subjects who succeeded in recognition of those characters, by measuring time required for full reading. The details of experimental setup were as follows: The subjects were 49 males and females (age 20-78) who could recognize 3D text characters. We measured time required to finish counting the number of capital character “A” which was involved in a random sequence of characters in 3D. For comparison, we used the same sizes of large and small characters in this experiment as were adopted in Exp. II. We also carried out the same kind of counting number experiments for the same sizes of characters by using 2D (i.e., ordinary nonstereoscopic) representation.

Visibility Experiment and Evaluation of 3D Character Representation on Mobile Displays

49

3 Results In Exp. I, 87.2% of the subjects could and 12.8% of those could not recognize 3D images where a spherical object moved forward and back continuously. We show the number of subjects who could and could not recognize the 3D images in each age group in Fig. 6. We could not, however, find any specific tendency between visibility of 3D images and age group of subjects. In the case of subjects who could recognize 3D images, the maximum distance of the object popping out from the fixed display was measured as 10.2 cm (mean) ± 1.8 cm (SD). In Fig. 7, we show the relation between pupil distance of the subjects and maximum pop-out distance (mean value) of the 3D object. We find that our experimental results in general agree well with a theoretical prediction calculated by using PD.

Fig. 5. Text characters used for 3D representation. Arrows in the right picture indicate instructed reading directions.

In Exp. II, 78.3% and 77.4% of the subjects could recognize the large and small characters in 3D, respectively. For the subjects who could recognize 3D characters, the time required for first recognition of those characters in large and small size was 9.2 sec (mean) ± 5.3 sec (SD) and 9.0 sec (mean) ± 10.1 sec (SD), respectively. In Fig. 8, we show the relation between the mean time required for first recognition of 3D characters and age group of the subjects. In the case that the represented characters in 3D are smaller, it tends to require more time especially for subjects of over 60 years old to recognize those characters.

Fig. 6. Number of subjects who could and could not recognize the 3D images in each age group

50

H. Ishio et al.

In Exp. III, we measured time required for full reading of 3D text characters. The results are 9.2 sec (mean) ± 5.3 sec (SD) for large 3D characters, 9.0 sec (mean) ± 10.1 sec (SD) for small 3D characters, 12.6 sec (mean) ± 6.0 sec (SD) for large 2D characters, and 16.0 sec (mean) ± 13.5 sec (SD) for small 2D characters. In Fig. 9, we show the relation between the mean time required for full reading of 3D text characters and age group of the subjects. There, depending on the character size, we find significant difference in the mean time required for full reading. That is, in the case that characters are larger, the mean time required for full reading decreases in both 2D and 3D representations. In such a case, there is almost no difference in the mean time in 3D and 2D representations. On the other hand, in the case that characters are smaller, there is almost no difference in the mean time in 3D and 2D representations for subjects under 40 years old while it requires more time for subjects over 40 years old to read characters in 3D representation, compared to the case of 2D representation.

Fig. 7. Pupil distance of the subjects and maximum pop-out distance (mean value) of the 3D object

Fig. 8. Mean time required for first recognition of 3D characters in each age group of the subjects

Fig. 9. Mean time required for full reading of 3D text characters in each age group of the subjects

4 Conclusions In conclusion, whether we can recognize 3D characters or not does not so much depend on our age. The time required for first recognition of 3D characters increases in the case of smaller characters especially for people over 60 years old. There is not so

Visibility Experiment and Evaluation of 3D Character Representation on Mobile Displays

51

much difference in time required for full reading of 2D and 3D characters in larger size while the time significantly increases in 3D representation of smaller characters for people over 40 years old. Therefore, we should carefully choose suitable size of characters in 3D representation not to lose their readability especially for elderly people.

References 1. Cawood, S., Fiala, M.: Augmented Reality: A Practical Guide, Pragmatic Bookshelf, Raleigh, North Carolina (2008) 2. Hasegawa, S., et al.: Aging Effects on the Visibility of Graphic Text on Mobile Phones. Gerontechnology 4, 200–208 (2006) 3. ISO 13406-2: Ergonomic Requirements for Work with Visual Displays Based on Flat Panels –Part 2 Ergonomic Requirements for Flat Panel Displays (2001)

Composite Context Information Model for Adaptive Human Computing Sukyoung Kim, Eungha Kim, and Youngil Choi Service Cognitive Control Team, Electronics and Telecommunications Research Institute, 138 Gajeongno, Yuseong-gu, Daejeon, 305-700, Korea {ksk0314,ehkim,yichoi}@etri.re.kr

Abstract. This study is to propose the composite context information model that provides an appropriate model depending on the situation around user. In our research, describe the definition of composite context information, propose the reuse method of the preexisted context information and design the composite context information process structure to improve performance. For developing the user-centric context-aware computing, above all important thing is, provide the high-level concept like human beings in knowledge framework and many application layers use it, don’t care the inner structure for processing complex context information. Keywords: composite context information, context-aware computing.

1 Introduction Nowadays, many researchers or institute is to predict the context-aware computing one of the future internet’s technologies. Context-aware computing can be used in various fields by grafting and very close a complex reality to the human being. That is, to success the context-aware computing, is important to model from the human being to computer being. But it is not easy, because all of humans being do make the model to process by computer, beside has become more complex according to develop human world. Moreover, rapidly evolving the web’s paradigm needs the like thing with human beings, and supporting the standardized infra of context-aware can be reduce the development cost of application. Despite the many context-aware system proposed, since the concept of a meaningful context similar in many ways represented a different vocabulary, it is not easy to reuse the context information by most early context-aware system. The important thing is the modeling method to extract from status to context and select the appropriate service. According to the modeling method, the context-aware system can be support the concept of high-level context or can be share the context information. Development of the technologies like that the service delivery platform or the network generation service overlay network depend on to select or to composite the appropriate service by the context. So, like this, to structure the service system with the context process architecture shall be subject to the user situation oriented and provide the mechanism from context to context information. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 52–56, 2011. © Springer-Verlag Berlin Heidelberg 2011

Composite Context Information Model for Adaptive Human Computing

53

Beside, Many researchers have been defined for the context and context information, each definition are limited that applies to services and application. For increasing reuse and share the high-level context information, for reducing the complexity in application and for enhancing the standardization of context information, we decide to define the CCI(composite context information). As define the CCI, we suggest the structure that construct of multi-layered representation architecture which is consist of the unit context information.

2 Propose the CCI Model and Process This chapter describes the overview of our research method. And we descried briefly the related study in terms of the context information modeling. 2.1 Definition of CCI We define the CCI which is “the enhanced high-level context information by connecting or compositing of composite information related with multi-entity to decide and provide for service of system”, in here, entity can be the things that is generate the situation or associate with human. Fig.1 shows the concept of above definition. Many entities generate the context information and the CCI made by composite with each the context information. The right side of fig.1, describe the main criteria for constructing the CCI, and we can find the CCI’s contents consist of DL and SWRL.

Fig. 1. According to define the CCI, describes the concept of CCI with service and address to the main criteria of CCI

2.2 Analysis of Modeling Method for CCI We search and analyze the related study of context-aware system, and more detail analyzes the modeling method of each system and main element to satisfy that compose the CI. As result of that, we can draw the fig.2 which describes the main elements and main contents to enhance the performance of context-aware middleware [2, 3]. We can know the important thing which is select the model to apply the above features when construct the CCI model and framework.

54

S. Kim, E. Kim, and Y. Choi

Fig. 2. As the result of analysis, the main elements and main contents to enhance the performance of context-aware middleware

Besides, to decide the modeling method, compare with the modeling and processing method of early context-aware system. Result to this, we compare with the architecture, context model, context process and historical context data through the existing R&D systems consist of CASS, CoBrA, Context Management Framework, CORTEX, Gaia, Hydrogen and SOCAM. We decide the whole structure our test system, architecture is centralized middle in service network and context model is OWL DL+SWRL and context processing is context reasoning engine based on Pellet. 2.3 Design the Structure of CCI Process and Data Architecture In this section, we proposed the design of CCI structure to process the context information and data repository architecture to enhance the CCI decision. Fig.3 shows the structure of CCI process with CCI Repository and reusing process to reuse the existed context-aware context information. As we can see the Fig.4, our

Fig. 3. Design the structure of CCI processing and the CCI or the Context Information repository

Composite Context Information Model for Adaptive Human Computing

55

study focus the CCI interpreter and Rule processing role of decision the CCI List and inference the rule of CCI. Two part of our structure, in order to decide the processing of CCI, first matching process with required context to CCI List in the repository, second decision the function type to process for dynamic or changeable context from outside and third, rule processing that is CCI and rule extraction to translate for transmission to reasoning engine. Besides, we design the data structure of CCI in repository to enhance the speed of searching, reasoning and reusing the CCI or CI. Fig.4 is show the layered structure of CCI to repository, and as explain to our strategies in repository and sequence of processing CCI, and recombine with CI rule and CCI rule Op list to express CCI.

Fig. 4. Design the multi-layered structure to CCI processing by combine with CI rule index and CCI rule Op list

Fig. 5. Example of the modeling for expression the “Availability” one of CCI

56

S. Kim, E. Kim, and Y. Choi

3 Modeling the CCI by OWL DL Finally, as we can see fig.5, we modeling the CCI with defined vocabulary “Availability” that is consist of context entities connected with device, network, location, service and user. The expression of “Availability” is composite with unit expression of each entity, and we can find the high-level CCI like as “ServiceEnable”, “ResponseStatus”. All of CCI is subclass of “CompositeContext” class and “CompositeContext” is subclass of “Context” class. First, this expression is store to the CCI schema repository and it is used for matching and reasoning when composite with the CCI rule index and CCI rule operator.

4 Conclusion For developing the user-centric context-aware computing, above all important thing is, provide the high-level concept like human beings in knowledge framework and many application layers use it, don’t care the inner structure for processing complex context information. So do this, we suggest the whole structure and model for CCI. To develop the study, we have to build the repository and casual system architecture and test our proposed method. And we have a plan to construct the user-defined CCI repository that is to enhance the volume and quality of the CI and accumulate the knowledge about the variety of CCI. Acknowledgments. The project related to this paper is performed by the Ministry of Knowledge Economy and the Korea Evaluation Institute of Industrial Technology [Research and development of composite context based adaptive service path configuration technology (10035146)].

References 1. Dey, A.K.: Providing Architectural Support for Building Context-Aware Applications. PhD Thesis, Georgia Institute of Technology (November 2000) 2. Biegel, G., Cahill, V.: A framework for developing mobile, context-aware applications. In: Proceedings of the 2nd IEEE Conference on Pervasive Computing and Communication, pp. 361–365 (2004) 3. Chen, H., Finin, T., Joshi, N.: An Ontology for Context-Aware Pervasive Computing Environments. The Knowledge Engineering Review 18(3), 197–207 (2003) 4. Patkos, T., Bikakis, A., Antoniou, G., Papadopouli, M., Plexousakis, D.: Design and challenges of a semantics-based framework for context-aware services. Int. J. Reasoning-based Intelligent Systems 1(1/2) (2009) 5. Preuveneers, D., Berbers, Y.: Adaptive context management using a component-based approach. In: Kutvonen, L., Alonistioti, N. (eds.) DAIS 2005. LNCS, vol. 3543, pp. 14–26. Springer, Heidelberg (2005) 6. Baldauf, M., Dustdar, S., Rosenberg, F.: A survey on contextaware systems. Int. J. Ad Hoc and Ubiquitous Computing 2(4), 263–277 (2007)

A Framework for a User Friendly Wireless Sensor Network Configuration System Julia C. Lee and Lawrence J. Henschen Northwestern University, Evanston, IL, 60208 USA [email protected], [email protected]

Abstract. Wireless sensor networks (WSNs) are being embedded into the fabric of our everyday lives. But there design still requires expertise in computer science and engineering in addition to the expertise for the application to which the WSNs are directed, and this slows the progress towards WSN solutions to important modern-day problems. We outline a system which may ultimately allow designers with little or no computer background to design WSNs. The system is based on a well-crafted user interface combined with a knowledge base about WSN hardware and software.

1 Introduction In 1990 HCI research pioneer Mark Weiser remarked that “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it”[1]. (Wireless) sensor network applications are leading the way to “fabric” the modern technologies into our “everyday life”. Gehrke and Liu [2] note “ … the growing pervasiveness of small-scale computing devices throughout the physical world, and the wireless sensor network community is a driving force behind this”. Currently, designing a wireless sensor network (WSN) requires expertise in many different areas from hardware to software, from system to application, from tools to abstract modeling, etc. Our work focuses on the development of a system that will assist developers of WSN applications who have little or no computer hardware/software background. The system will prompt the user with questions related to the application, provide choices for answers, and then combine those answers with its knowledge base and models to automatically create configuration and software for the application. This approach illustrates that one important solution to the application of new technology is to combine knowledge with a good user interface.

2 Review of Issues Related to Wireless Sensor Networks We briefly review some of the important issues related to WSN design and development so that the dimension of the complexity of the problem can be recognized by the reader. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 57–61, 2011. © Springer-Verlag Berlin Heidelberg 2011

58

J.C. Lee and L.J. Henschen

• Different application type or area: Application areas range widely, each with its own peculiar requirements [3][4]. Examples include: military (e.g. surveillance), environmental monitoring, health (e.g. patient monitoring), engineering, home/office, robotics, smart parking lots, interactive toys, interactive museums, detecting car thefts, vehicle tracking and detection, and many more. • Network configuration and routing: Network configuration focuses on the number and distribution topology of the nodes [5][6]. Routing focuses on how to get the information from the nodes to a base station. There are many choices, and the pros and cons of each depend on the application, the number of nodes involved, the sensor nodes chosen, etc. A user with little expertise in these topics would find it impossible to make good choices. • Node configuration: Sensor nodes available for application vary greatly in terms of features and capabilities [7], and users not familiar with computer engineering concepts would find it difficult to understand the differences. These details can be hidden from the users if a system has the intelligence to connect the user’s requirements to the information /knowledge bases. • Software and firmware: The most popular operating system for sensor nodes is TinyOS [8]. Applications on TinyOS based systems are written mostly in nesC [9]. However, mastering TinyOS or nesC is not an easy job for users who do not have a computer science background. • Other critical issues include [3]: WSNs often have large numbers of nodes; they are often densely deployed; nodes are prone to failures; the topologies can change frequently; nodes are limited in power, computational capacity, and memory; WSNs use broadcast communications instead of point-to-point; etc. From the above we can see the broad spectrum of knowledge required for WSN development, and this knowledge is separate from the issues specific to the application (e.g. environmental monitoring) [3][4][5].

3 An Automatic Sensor Network Configuration System Figure 1 shows an overview of the kind of system we are studying to help noncomputer engineers design WSNs. The system includes the following components: 1. A reference knowledge database: contains up to date information about wireless sensor networks including but not limited to types of sensors, types of motes, drivers, software libraries, distribution topologies, routing algorithms, etc. 2. A question and answer (QA) system: A hierarchy of carefully designed questions will guide the user through the “interview” process. The questions are mostly multiple choice menus; the user can easily choose from a set of pre-designed answers. When the user chooses one answer from a higher level, a lower level of relevant questions and answers will be made available to the user. Answers will be mapped into parameters that will be used by the other components of the system. 3. A static model: This model deals with the static aspects of the WSN application system, e.g. node/mote type, number, distribution topology, static routing scheme,

A Framework for a User Friendly Wireless Sensor Network Configuration System

59

etc. This component uses the parameters generated by the QA system and the information from the reference knowledge database to generate a configuration file for the chosen hardware components. 4. A dynamic model: This model deals with the dynamic aspects of the WSN application system, e.g. sensor data collection, processing and communication, etc. This component also uses parameters generated by the QA system, information from the reference knowledge database, and the configuration data from the static model to generate the software for the nodes of the application system. With such a system the users (i.e., application experts) will concentrate on the description of the application, not on computer engineering issues such as programming and hardware configuration.

Fig. 1. An Automatic Configuration System for Wireless Sensor Network Applications

4 Some of Our Major Approaches Due to space limitations we can only briefly introduce some of our approaches to constructing the system sketched in the previous section. 4.1 Smart Questions and Answers The interface between the user and our system is the QA system. Questions are designed hierarchically. The top level questions categorize the application that the user is developing, for example “military”, “environmental”, etc. Sub-categories allow the system to get into more details of the application; for example in the “environmental” category, one could choose from: “air pollution monitoring”, “animal migration monitoring”, etc., and then specify other information such as the monitoring area, measurement types, measurement frequency, etc. Because the questions are all related to the user’s application and the data entry is by clicking, a non-computer engineer will find it very easy to describe the application of interest. 4.2 Taxonomy Taxonomy will play a crucial role in the design of the QA sub-system, and there are several taxonomies described in the literature. In [10], WSNs are categorized as

60

J.C. Lee and L.J. Henschen

“Event Detection and Reporting”, “Data Gathering and Periodical Reporting”, “Sink Initiated Querying”, and “Tracking Based Applications”. In [11], the authors classify the communications within a sensor network into “application” and “infrastructure” and further classify the “application” into “cooperative” and “non-cooperative”. They classify data delivery into four categories: “continuous”, “event driven”, “observer (sink)-initiated”, and “hybrid”. Lan Bai, et. al. [12] categorize sensor network applications into “archetypes” based on 8 properties of sensor network applications: “mobility”, “initiation of sampling”, “initiation of data transmission”, “actuation”, “interactivity”, “data interpretation”, “data aggregation”, and “homogeneity”. Taxonomy will be used in many ways in our system, but particularly in the user interface (i.e., the QA system). For example, if the user indicates the WSN is of type “Data gathering and periodic reporting”, the system will prompt for the kinds of values to be measured and the sampling and reporting periods. 4.3 Knowledge and Deductive Rules Rules will be used to generate application specific parameters based on the answers chosen by the user. For example, a rule like “transportation” & “parking” & “existence of car” & “hourly” Æ (infrared&ultrasonic&light&webcom) & (((SBT80)&(Tmote Sky)) || Mica2) would indicate that a transportation application dealing with parking, in which detection of cars is the information sensed and reported hourly, should use four sensors per node (infrared, ultrasonic, light, and webcam) and one of two commercially available nodes (Tmote Sky or Mica2, [7][13]), etc. Additional QA could further refine these recommendations. 4.4 Models and Parameters Modeling (static and dynamic) plays an important role in our system. Our current work focuses on generating the programs that run on a node, and this is done by the dynamic model of our system. We think that in a WSN all the activities performed by nodes can be modeled as events; this includes sensing data, transmitting data, and system activity (e.g. put some part to sleep) among others. We model each event as a 6-tuple E=(P, C, A, R, L, D). C is a set of condition expressions (one of which can be time). A is a function or action, and R is a set of result values for the actions. L is the location id of the event. P is the set of events that this event depends on, and D is the set of events that depend on this event. For example, in the smart parking lot application, the arrival of a car is an event. The condition of the event will be that the measurements of the four sensors exceed the thresholds for those measurements. The actions may include recording the occupancy of this parking position, starting to count the time for this car, etc. This event triggers another event, namely sending a message to the control (sink) node with the node's identity L.

A Framework for a User Friendly Wireless Sensor Network Configuration System

61

5 Conclusion A tool like we have described, with a well-crafted user interface and an intelligent system behind it, will allow users with little or no WSN training to design WSNs in their application field.

References 1. Dix, A., Finlay, J., Abowd, G., Beale, R.: Human-Computer Interaction, 3rd edn., p. 181. Prentice Hall, Edinburgh Gate (2004) 2. Gehrke, J., Liu, L.: Sensor Network Application. Internet Computing (2006) 3. Akyildiz, I., Su, W., Sankarasubramaniam, Y., Cayirci, E.: Wireless Sensor Networks: A survey. Computer Networks 38, 393–422 (2002) 4. Arampatzis, T., Lygeros, J., Manesis, S.: A survey of Applications of Wireless Sensors and Wireless Sensor Networks. In: Proceedings of the 13th Mediterranean Conference on Control and Automation, pp. 719–724. IEEE Press, New York (2005) 5. Lewis, F.: Wireless Sensor Networks. In: Cook, D., Das, S. (eds.) Smart Environments: Technologies, Protocols, and Applications. John Wiley, New York (2004) 6. Tan, J.: A Scalable Graph Model and Coordination Algorithms for Mobile Sensor Networks. In: Li, Y., Thai, M., Wu, W. (eds.) Wireless Sensor Networks and Applications, pp. 65–83. Springer, Heidelberg (2008) 7. Hempstead, M., Lyons, M., Brooks, D., Wei, G.: Survey of Hardware Systems for Wireless Sensor Networks. Journal of Low Power Electronics 4, 1–10 (2008) 8. Levis, P., Madden, S., Polastre, J., Szewczyk, R., Whitehouse, K., Woo, A., Gay, D., Hill, J., Welsh, M., Brewer, E., Culler, D.: TinyOS: An Operating System for Sensor Networks., http://www.cs.berkeley.edu/~culler/AIIT/papers/TinyOS/levis0 6tinyos.pdf. 9. Gay, D., Levis, P., Culler, D., Brewer, E.: nesC 1.1 Language Reference Manual, http://nescc.sourceforge.net/papers/nesc-ref.pdf 10. Iyer, A., Kulkarni, S., Mhatre, V., Rosenberg, C.: A Taxonomy-based Approach to Design Large-scale Sensor Networks. In: Li, Y., Thai, M., Wu, W. (eds.) Wireless Sensor Networks and Applications, pp. 3–33. Springer, Heidelberg (2008) 11. Tilak, S., Abu-Ghazaleh, N., Heinzelman, W.: A Taxonomy of Micro-Sensor Network Models. Mobile Computing and Communications Review, 28–36 12. Bai, L., Dick, R., Dinda, P.: Archetype-Based Design: Sensor Network Programming for Application Experts, Not Just Programming Experts. In: Proceedings of the International Conference on Information Processing in Sensor Networks, pp. 85–96 (2009) 13. Lee, S., Yoon, D., Ghosh, A.: Intelligent Parking Lot Application Using Wireless Sensor Networks. In: International Symposium on Collaborative Technologies and Systems CTS (2008)

FlexRemote: Exploring the Effectiveness of Deformable User Interface as an Input Device for TV Sang-Su Lee1, Seungwoo Maeng1, Daeeop Kim1, Kun-Pyo Lee1, Wonkyum Lee2, Sangsik Kim2, and Sungkwan Jung2 1

Dept. of Industrial Design, KAIST. 335 Gwahangno Yusung-Gu, Daejeon, Republic of Korea 2 KAIST Institute, KAIST. 335 Gwahangno Yusung-Gu, Daejeon, Republic of Korea {sangsu.lee,maengoon,up4201,kplee}@kaist.ac.kr, {wklee,sskim98,skjung}@itc.kaist.ac.kr

Abstract. We tried to explore the possibility of using deformation-based interaction as a new input method to remote control. We selected eight basic commands to control TV and design deformation-based gestures for each command based on prior studies regarding organic user interface. A prototype, the FlexRemote, was made and tested by informal user study. The study showed that the users have higher preferences to the FlexRemote in comparison with the conventional input devices and the users can make intuitive interactions through physical deforms. From this positive explorative study, we are expecting that this study will present a new direction in intuitive and fun interaction method with TV in living room environments, which is gaining greater importance. Keywords: interaction design, deformation-based interaction, organic user interface.

1 Introduction The conventional TV is mainly composed of relatively simple interactions such channel or volume control. However, recently studies have actively been made on the use of TV as a home display which supports various media activities in the living room environment. Thus, methods to interact with TV in a remote distance in the living room are getting greater importance. Meanwhile, many researchers have been seeking new interface opportunities to move beyond traditional input methods, such as WIMP [8]. Various studies have been made to find out the possibility of deformation as a new interaction method [1,2,6,7]. This study sought the possibility of using organic user interface (OUI) [3], which is a new method different from the conventional ones in the remote distance interaction for TV. OUI is not a fixed form, but can change its form organically according to functions making intuitional interactions possible. We expected that the study will make more intuitional and interesting interactions possible. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 62–65, 2011. © Springer-Verlag Berlin Heidelberg 2011

FlexRemote: Exploring the Effectiveness of Deformable User Interface

63

This study tried to design and verify what effect the interactions through deforms have on the actual users as input devices for remote TVs. A prototype was made based on previous studies for understanding user towards deformable interface as an input method, and informal user tests were conducted. For the prototype we made, deformable gestures which were obtained through our prior user reference were mapped concerning commands which are applicable to information devices. FlexRemote, which is OUI device and can implement a total of 8 general commands through deformable gestures, was made. This device of 2-dimentional flexible plastic material can be easily flexed in any direction. This study looked into the effectiveness of deformable interactions as a remote input device which is remote from the display. It is expected that this study will present a new direction in comfortable interactions with TV in living room environments, which is gaining greater importance.

2 Designing Deformation-Based Interface We designed the FlexRemote, a deformation-based input device for smart TV, based on prior study regarding deformation-based interaction. For the commands for smart TV, we selected basic commands of general information device which is also applicable to the smart TV. We categorized commands into four: commands for device status, for navigation, for content manipulation, and for action. Two commands were selected for each category, total eight basic commands were selected from existing Category Status

Commands and Gestures

(Folding right to left)

(Folding left to right)

Power on

Power off

(Bending right side upward)

(Bending left side upward)

Next Channel/Next

Previous Channel/Next

Navigation

Content manipulation (Bending upward)

(Bending downward)

Volume up/Zoom-in

Volume down/Zoom-out

Action

(Bending the upper right corner)

(Swing)

Favorite channel/Scrap content

Delete content/Back

Fig. 1. Eight basic commands and deformation based gestures we designed for FlexRemote

64

S.-S. Lee et al.

studies [1,2,4,5,7]. More complex commands, such as typing texts, were excluded due to the difficulty of performing them with a deformation-based gesture at this exploratory study. For eight basic commands we have chosen, eight different deformationbased gestures was assigned according to our prior study of understanding how users manipulate deformable display as an input device [4]. In this prior work, we tried to understand deformation-based user gestures by observing users interacting with artificial deformable displays with various levels of flexibility, and we could gain userdefined gestures for 11 commands. We could map our 8 TV commands with those prior user-defined gestures. Figure1 shows how our 8 commands mapped with userdefined gestures.

3 Implementation and User Study We made our FlexRemote prototype based on our design. It consists of 16 flex sensors so that it can recognize eight deformation gestures we selected in prior section. (Figure2, left) We setup the virtual TV environment based on Windows PC system. User can control 8 commands we designed by manipulating our FlexRemote prototype. With this prototype we made, an informal user study was conducted with the students in our departments. We gave each participant a use case scenario which involves all 8 commands we designed. First we show them how those 8 commands work with 8 different deformation-based gestures, and then we asked participants to control TV by themselves (Figure2, right). Also we asked participants to do the same task using keyboard and mouse combination. From an in-depth interview with each participant, most users told us their higher preferences to the FlexRemote in comparison with the conventional input devices such as traditional remote controller or keyboard/mouse. Unlike the conventional method where users have to pay attention to the input device and the screen alternatively, the users can make intuitive interactions through physical deforms without moving their vision from TV. Most of the users became easily familiar with the commands. Regarding workload, the interviews showed that physical demand was rather high, but their joy from the new manipulation was greater than the physical demand. This may mean that physical demand can be an issue as the new effect in a long term, studies should be made on long-term effects of deformable interactions, and also comparative studies should be made with other methods such as body gestures which are recently studied.

Fig. 2. FlexRemote prototype (left) and user study (right)

FlexRemote: Exploring the Effectiveness of Deformable User Interface

65

4 Conclusion and Future Works In this study, we tried to explore the possibility of using deformation-based interaction as a new input method to remote control. We selected basic commands to control smart TV and design deformation-based gestures for each command based on prior studies regarding organic user interface. A prototype, the FlexRemote, was made and tested by informal user study. The study showed that the users have higher preferences to the FlexRemote in comparison with the conventional input devices and the users can make intuitive interactions through physical deforms. From this positive explorative study, we are expecting that this study will present a new direction in intuitive and fun interaction method with TV in living room environments, which is gaining greater importance. For the future works, we are planning to conduct user study within the real context; in the real living room environment when people actually watching TV programs. From this further study, we could gain understanding of social acceptability [6] of new interaction method which is also very important as a practical solution. Acknowledgment. This research was supported by WCU(World Class University) program through the National Research Foundation of Korea funded by the Ministry of Education, Science and Technology (R33-2008-000-10033-0). This work was also partially supported by the Industrial Strategic technology development program, KI002156 funded by the Ministry of Knowledge Economy(MKE, Korea).

References 1. Gallant, D.T., Seniuk, A.G., Vertegaal, R.: Towards more paper-like input: fexible input devices for foldable interaction styles. In: Proc. UIST 2008, pp. 283–286. ACM Press, New York (2008) 2. Holman, D., Vertegaal, R., Altosaar, M.: PaperWindows: Interaction Techniques for Digital Paper. In: Proc. CHI 2005, pp. 591–599. ACM Press, New York (2005) 3. Holman, D., Vertegaal, R.: Organic user interfaces: designing computers in any way, shape, or form. Communications of the ACM 51(6), 48–55 (2008) 4. Lee, S., Kim, S., Jin, B., Choi, E., Kim, B., Jia, X., Kim, D., Lee, K.: How users manipulate deformable displays as input devices. In: Proc. of CHI 2010, pp. 1647–1656. ACM, New York (2010) 5. McNeill, D.: Hand and Mind: What Gestures Reveal about Thought. University of Chicago Press, Chicago (1992) 6. Rico, J., Brewster, S.: Usable gestures for mobile interfaces: evaluating social acceptability. In: Proc. of CHI 2010, pp. 887–896 (2010) 7. Schwesig, C., Poupyrev, I., Mori, E.: Gummi: User Interface for Deformable Computers. In: Proc. CHI 2003, pp. 954–955. ACM Press, New York (2003) 8. Van Dam, A.: Post-WIMP user interfaces. Commun. ACM 40(2), 63–67 (1997)

A Study of User Needs for the ‘Techno Kitchen’ Martin Maguire1, Colette Nicolle1, Russell Marshall1, Ruth Sims1, Clare Lawton1, Sheila Peace2, and John Percival2 1

Loughborough Design School, Loughborough University, Ashby Road, Loughborough, Leics, LE11 3TU, UK [email protected] 2 Faculty of Health & Social Care, Horlock Building, 110 The Open University, Walton Hall, Milton Keynes, MK7 6AA, UK

Abstract. A project, Transitions in Kitchen Living (TiKL) has been conducted to study kitchen histories and current use by older people. A multidisciplinary team developed a common interview methodology to collect people’s personal kitchen histories and their views on their current kitchen. Now and in the future ambient assistance and applied electronic technology might provide additional functionality to make kitchens easier to use for all ages. This paper relates some of these technical developments to user needs identified in the TiKL project and reports on the results of a survey to obtain older people’s views on them. Keywords: kitchen, advanced technology, older people, people with disabilities.

1 Introduction The kitchen is an important space in the home. In order to support easier use and independent living for older people, it is desirable to provide, a kitchen environment that is comfortable so that tasks can be performed with a minimum amount of unnecessary effort and strain. Good design and physical ergonomics can help to achieve this. However to expand the possibilities to support people as they age, computer technology, if developed in a subtle and sensitive way can help people to conduct tasks in the kitchen. They can also help to monitor a person’s well being to detect if someone has fallen and needs help. There are many issues related to ethics, privacy, intuitiveness and ease of use that need to be considered before such systems are implemented [1]. It is well known that such technological developments are no substitute for an older person at home having human contact and support from friends and family. This paper develops the work of a project (Transitions in Kitchen Living - TiKL) conducted by social gerontologists at the Open University and ergonomists at Loughborough University in England and funded by the UK’s New Dynamics of Ageing Programme, to study people’s lives in relation to the kitchen [2, 3]. Forty eight older people, living in a range of accommodation in Bristol and Loughborough, took part. Each participant was interviewed about their ‘kitchen history’ – all the different kitchens they have experienced in their lives, and also their abilities, needs, likes and dislikes in relation to their current kitchen. Many of the participants had C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 66–71, 2011. © Springer-Verlag Berlin Heidelberg 2011

A Study of User Needs for the ‘Techno Kitchen’

67

developed simple and practical solutions to overcome the problems they faced. These included adapting cupboards for easier access, installing task lighting in dark corners, erecting lower work surfaces for use while seated (possibly in a wheelchair) and repositioning plugs for easier access. Smaller appliances such as table top dishwashers and microwave/cooker combinations were also found suitable when there was little space available. All these ideas were assembled and incorporated into a ‘Kitchen Living Guide’ to help other people adapt their kitchens to better meet their changing needs as they grow older. A natural extension of this study was to review the possibilities of using more advanced technology to make kitchen life easier. A small survey was also conducted with a group of older group of people to obtain their reactions to some of the ideas and concepts behind the ‘techo kitchen’.

2 General Problems and Requirements in the Kitchen The TiKL kitchen study reviewed the current kitchen environment and identified a range of general problems that people faced. Participants were asked about their capabilities with respect to kitchen activities. The following chart (Figure 1) shows the number of participants who did have specific capability problems. 30 25 6 20 15

3

2

Some probs. 4

10

16

15 5

Problems

20

9

0 Sight

Hearing

Reaching

Dexterity

Fig. 1. Numbers of participants experiencing physical limitations out of total of 48 (‘Some problems’ indicates those saying they had no problems but then described problems or coping strategies)

It can be seen that the physical limitation of reaching (up, down or in front) affected 26 people (54% of the sample), while problems of lack of dexterity and strength, poor sight and hearing also affected significant proportions of the sample. The interview study identified specific task related problems in the kitchen resulting from these physical limitations as described by the project participants. It also recorded solutions that people had adopted and provided a stimulus to think about new possible solutions.

68

M. Maguire et al.

Table 1 below lists 16 common problems that people might face in the kitchen (both older and younger people) alongside solutions that could be adopted to address them. The second column shows traditional kitchen design and ergonomic solutions that arose during the study. The third column presents more advanced technological solutions that could be adopted or developed. Table 1. Comparison of possible ‘low tech’ and ‘hi tech’ solutions to problems identified in the kitchen study Problem

Possible ‘low tech’ solution

Possible ‘hi tech’ solution

1. Poor lighting in parts of kitchen for doing tasks 2. Tiredness and needing to sit down to carry out tasks 3. Difficulty in reaching window or blind over sink

Install under cupboard lights or table lamps with switches Table or lower work surface for sitting or perching stool Reposition catches to be at bottom of window frame

4. Bending down to low or reaching up to high shelves. 5. Reading small instructions on packaged food. 6. Bending down to check cooking in oven/take out pan 7. Transporting food from oven to work surface or table. 8. Drudgery of washing up

Relocate shelving height. Pull out or carousel shelving. Improved lighting to read instructions or magnifier Oven placed at waist height. Light- check without opening Trolley to help transfer items between kitchen-dining room Wash up in stages

Activity is sensed and light is turned on automatically Work surface motorized so can be raised or lowered Remote control opening and closing of windows/blinds Motorised cupboards that can be moved up and down Gadget to scan and read instructions aloud.

9. Concern that iron, oven or kettle left on when out

Return to house or contact neighbour to check

10. Hearing door bell or phone with kettle on 11. Needing help with shopping. 12. Lack of space for ironing board in the kitchen 13. Lack of plug points. 14. Kettle too heavy to lift to tap to fill. 15. Wash basket too heavy to take outside. 16. General cleaning (floor, on cupboards, behind units)

‘Robot’ to transfer items between kitchen-dining room Dishwasher installed (perhaps small unit) Auto shut off of these items if left on too long or if go out

Neighbour, friend or relative helps with transport. Additional space (e.g. utility room) for board to be set up. Additional electricity points installed

Visual alert if door bell pressed or phone rings Online shopping service. Fridge notifies of items need. Pull out ironing surface built into kitchen units. Future gadgets may be powered wirelessly

Smaller kettle that can be lifted more easily. Lighter washing basket. Use with smaller loads. Easy clean floor and work surfaces.

Hot water unit to provide hot water ‘on tap’ ‘Robot’ to carry clothes to outside ‘Robot’ kitchen cleaner with access behind units

A Study of User Needs for the ‘Techno Kitchen’

69

3 Survey to Explore Requirements for Technological Support Drawing upon the list of possible innovations in Table 1, a similar group of older people (21 in total, 16 females, 5 males, from 50 to 85 and an average age of 71) were surveyed to think about a range of possible technological enhancements that might support them in the kitchen. Each respondent was asked to state whether they would want each innovation or not. A summary of the survey results are shown in Table 2. Table 2. User reactions to ‘hi-tech’ innovations in the kitchen (N=21) Innovation 1. Press button on fridge to read out food items present or run out of.

% Approval 33%

Comments Would help with shopping. Good idea for freezer where more food stored. Opening door less saves energy. Unsure of technology behind it. Would you trust it? Would also need to know if items in date. Would need to log food items in and out.

2. Sensor alert if smoke, CO, fire or gas leak when you are out.

76%

Would help with safety. Good idea. Who would come to deal with it? If return to house and has been leak, putting on light could cause fire.

3. Quick cooling oven hob.

76%

Would be safer especially for grand children. Good idea – have sometimes been caught out by residual heat on ceramic hob.

4. Shut off electrical equipment when you leave the house.

90%

Yes it is easy to go out and forget that things still on. Would be safer. Save money on electric not being used. Peace of mind.

5. Washing machine has only 6 buttons for easy use.

75%

Only use 2 or 3 programmes most of time. Less confusing. Too many programmes never used. Am happy with dial and choice.

6. Open/close windows and blinds with button press.

76%

Would be handy as window situated over sink. Would make life easier often have to stretch to close them. Would be safer. Blind may hit items on window sill. Would make us lazy.

7. Water sensor welfare alert if taps not run for some time.

43%

Daughter regularly checks I’m OK. Useful for emergencies e.g. if someone has fallen. You would have to be able to cancel the system if went on holiday.

70

M. Maguire et al. Table 2. (Continued)

Innovation 8. Automatic task light when working in dark corner.

% Approval 71%

Comments Very useful. Could see better and avoid eye strain. Could save energy. Safety and emergencies in the dark. I wouldn’t need to stay in the corner. Need good light all the time.

9. You or others alerted if water on floor indicates possible flood.

75%

Useful – water can run a long time before noticed. Would be safer and avoid waste. Too many alarms may make you worry about too many minor things.

10. Simple touch screen to order shopping and have delivered.

24%

Save carrying heavy shopping. Like the choice or to check quality in store. Feel too complicated for the people intended to help. If became disabled friends would assist.

11. Recipe display on wall so easier to follow when cooking.

52%

May make it more efficient and save writing recipe’s down. Don’t follow recipes. Prefer to print with large font. Interested to see prototype.

12. Instructions read out about food preparation and cooking actions.

24%

Cooking an effort so could help if tired. Would be useful if can’t read cooking instructions label. I know how to cook things. Prefer to cook at my pace.

There was also some interest in adjustable wall cupboards that could move up and down (42% approval) electrically on runners for easier access to higher cupboard shelves. One person stated that “As a 4 foot 11 inch person this would be wonderful and would suit households with people of different heights and reach capability”. Another said that this “Would bring the cupboard down to their eye line and reach”. There was keen interest in a worktop that can be lowered temporarily for use while seated (67%). It was thought that this “Would keep people’s interest in cooking and independence later in life”. However it was also said that this could reduce cupboard space potential in the kitchen. Another idea was a facility in the kitchen to read out the small print (instructions, ingredients) on food packaging or other products (50%). Participants were also asked to propose their own innovations. There was a requirement from two people for more help to be energy or water efficient in the kitchen, for example, a plug to cut down on energy use in the kitchen by avoiding standby or a better mixer tap so that hot water was delivered quickly rather than running it until it warmed up. Three people suggested a facility (perhaps in the fridge) to help assess food quality e.g. “Has egg or cheese gone off?” Some people wanted to hand over complete tasks to technology: “Something to do the ironing” and “A robot that could cook, clean, serve food, do the shopping and the washing and drying up!” The results of the survey show that there is enthusiasm for certain types of technological support in the kitchen provided it is well thought through and designed

A Study of User Needs for the ‘Techno Kitchen’

71

to meet people’s needs. An understanding of older people’s viewpoints is important to ensure that such developments are useful and practical to kitchen users in the future.

References 1. Gill, J.: Ambient intelligence - paving the way - how integrated services can deliver a more inclusive society. In: EU RTD Framework project COST 219 (2008) 2. Maguire, M., Nicolle, C., Marshall, R., Sims, C., Lawton, L., Peace, S., Percival, J.: A study of the age-friendliness of kitchens. In: INCLUDE Conference, Helen Hamlyn Centre for Design, Royal College of Art, London (April 18-20 , 2011) 3. Sims, R.E., Marshall, R., Maguire, M., Nicolle, C., Lawton, C., Peace, S., Percival, J.: Older people’s experiences of their kitchens: Dishes and wishes, Contemporary and Human Factors 2011. In: Anderson, M. (ed.) Proceedings of Ergonomics & Human Factor 2011, Stoke Rochford, UK, April 12-14, pp. 387–393. Taylor & Francis Group, London (2011)

Consideration of the Human-Computer Interface in the Operation Room in the Era of Computer Aided Surgery Kazuhiko Shinohara School of Health Science, Tokyo University of Technology, Nishikamata, Ohta, Tokyo,144-8535, Japan [email protected]

Abstract. Human-computer interactions in the operation room (OR) are rapidly increasing as a result of advances in endoscopic surgery and the implementation of computerized physicians order entry systems (CPOE). Not only computerized medical devices and robotics, but also various image guidance systems with augmented reality to support endoscopic surgery, are being developed and some are already widely used in the OR. While CPOE provide the surgical team with quick and wide-ranging access to individual patients’ medical records and images while in the OR, essentially the procedures undertaken by surgeons and scrub nurses cannot be changed. In this study, ergonomic problems involving the human-computer interface in the operation room were investigated for the smooth uses of computers in this era of computer-aided surgery. The amount of cords, cables and medical tubing is increased between 3- and 5-fold over that used in the conventional OR, which carries latent risks for fault connections between the equipment and slipping and tripping accidents among the OR team. The use of a CPOE requires additional movement to the computer terminal during surgery. If fingerprint recognition is used as the CPOE biometrics system, surgeons and scrub nurses cannot access the CPOE themselves during operations. As countermeasures against such problems, the integration of cables and wires, improvement of the WIFI environment and electromagnetic compatibility with medical devices, and implementation of tablet PCs with voice recognition or touch screens are required. Moreover, it is essential for the human-computer interface in the OR to be further improved by accounting for OR-staff’s work flow. Keywords: computer aided surgery, operation room, human-computer interface.

1 Introduction Human-computer interactions in the operating room (OR) are rapidly increasing as a result of advances in endoscopic surgery and the implementation of computerized physician order entry systems (CPOE). Endoscopic surgery is a typical example of minimally invasive surgery, in which internal organs are operated on via small-caliber channels under endoscopic view. Much of the endoscopic procedure depends on computerized medical electronic devices, including surgical robots and image-guidance C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 72–75, 2011. © Springer-Verlag Berlin Heidelberg 2011

Consideration of the Human-Computer Interface in the Operation Room

73

systems. Thus endoscopic surgery leads the way in computer-aided surgery. Furthermore, implementation of CPOE in the OR is rapidly progressing in Japan. These systems started as an order and reservation entry system, however their scope is becoming broader and is leading to the realization of a “paperless and filmless hospital”. The aim of this study was to investigate the problems associated with the implementation of the human-computer interface during the era of computer-aided surgery.(Fig.1)

2 Materials and Methods We compared the workflow and ergonomic problems of the OR staff during a typical operation for colon cancer using computer-aided endoscopic surgery with a CPOE versus conventional open surgery with paper-based medical records. In this study, the CPOE refers not only to an order and reservation entry system but also to the system for entering and viewing laboratory data and clinical images, such as ultrasonography, computed tomography, and X-ray images.

3 Results 3.1 Additional Devices Required for the Endoscopic Surgery and CPOE in OR For endoscopic surgery, additional medical electronic apparatus such as an electronic endoscopy system (including endoscope, light source, image controller, and video recorder), an automatic insufflator (pneumoperitoneum apparatus), and an ultrasonic cutting and coagulation system are required. In the case of robotic surgery, a surgical manipulator such as the da VinciTM system is also required. All this apparatus is computer controlled. By contrast, the only medical electronic apparatus required for conventional open surgery is the electrocautery device. Thus, human-computer interaction during endoscopic surgery has increased compared with conventional surgery. The number of cords, cables, and medical tubes between the operative field and the medical apparatus has also increased between 3- and 5-fold compared with conventional open surgery. 3.2 Additional Work and Problems Associated with CPOE in the OR To access a patient’s medical records and imaging data (such as computed tomography, ultrasonography, and X-ray images) during surgery, surgeons and anesthesiologists must move to the CPOE terminals each time. Also, difficulties have been encountered in accessing data on the CPOE terminals. Similarly, nurses and clinical engineering technicians who operate various medical devices must also move to and access the CPOE terminals for personal authentication of the patient, reading the patient’s medical records and laboratory data, and ordering laboratory tests. Nurses must also access the system during the operation to record events, such as the infusion of intravenous fluids or administration of drugs. The clinical engineering technicians must input data from off-line medical devices. In the case of a filmless and paperless OR, nurses and technicians must additionally record separately the lot number of blood and implants that are used during surgery.

74

K. Shinohara

a

b Fig. 1. A scene of endoscopic surgery(a) and CPOE terminal in the OR(b)

4 Discussion Human-computer interactions in the OR are rapidly increasing in this era of endoscopic surgery and CPOE compared with in the era of conventional open surgery and paper- and film-based medical records. Since modern endoscopic surgery began in the 1990s, indications for its use have greatly increased. The main merit of endoscopic surgery for the patient is minimal invasiveness, but it brings the disadvantage that surgeons have to operate on the internal organs via small-caliber channels with restricted tactile sense and visual information. To compensate for this, many types of medical electronic apparatus have been developed. Not only computerized medical devices and robotics, but also various image-guidance systems with augmented reality to support endoscopic surgery are already widely used in the OR. All these types of apparatus are controlled by computers. In addition, the implementation of CPOE is progressing rapidly in Japan and may eventually lead to the realization of a paperless and filmless hospital. The system covers e-prescriptions, medical records, hospital accounts, and logistic systems among others. It includes not only verbal and numerical medical records but also medical images, such as X-rays, computed tomography scans, magnetic resonance images, endoscopy, and photographs of surgical specimens. The system allows for secure and rapid transmission of medical information as well as improves patient safety by providing computerized references and check systems for prescribing medication. CPOE systems also provide the surgical team with quick and wideranging access to patients’ medical records and images while in the OR. However there is also resistance in the OR to adapting to using CPOE due to perceptions regarding the impact of the human-computer interaction on the busy and stressful workflow in the OR. From this study of the ergonomic issues involved in the human-computer interface in the OR, several problems are revealed. The number of cords, cables, and medical tubes is increased compared with conventional open surgery. This increase provides latent risks for faulty connections between the equipment and also for slipping or

Consideration of the Human-Computer Interface in the Operation Room

75

tripping accidents among the OR team. The use of a CPOE requires additional movements to the computer terminal during surgery. Such movement from staff members’ positions to the CPOE terminal not only complicates workflow, but also increases the risk that individuals may neglect their duties; this is especially true for anesthesiologists, clinical engineering technicians, and nurses who must closely monitor machines. These movements provide an additional workload for the OR staff and potential risks for slipping/tripping accidents within the team. For surgeons and scrub nurses, there are difficulties in operating the keyboard and mouse of the computer terminal during surgery. If fingerprint recognition is used as the CPOE biometrics system, surgeons and scrub nurses cannot access the CPOE themselves during operations without interrupting the surgery to access the system. As a countermeasure against these problems, cables and wires must be integrated, and improvements are required in the WiFi environment and electromagnetic compatibility with medical devices, as well as in the use of tablet PCs with voice recognition or touch screens. Application of a mobile terminal with eye-movement control and/or voice control may be one solution. However, care must be taken in the design of the human-computer interface to avoid disturbing the intuitive and finely coordinated activities of surgeons during surgery. Workflow analysis of the OR staff during surgery is a fundamental requirement for the development of an ergonomically acceptable human-computer interface in the OR in this era of computer-aided surgery.

5 Conclusion For the smooth and safe implementation of medical electronic devices and CPOE in computer-aided surgery, it is essential for the human-computer interface to be further improved to accommodate OR staff characteristics and workflow. Basic education on computer information technology is also necessary as part of medical education and training.

References 1. Shinohara, K.: Implemantation problems of computerized physician order entry system in the operation room. In: Proceeding of Human computer interaction 2009 (CDR) (2009) 2. Shinohara, K.: Safety and ergonomic problems on the clinical practice of gastrointestinal endoscopy. In: Proceeding of 2nd International conference on Applied Ergonomics (CDR) (2008) 3. Benedict, A.J.: Electronic Prescribing Adoption. In: Proceeding of 2nd International conference on Applied Ergonomics barriers and influencing factors (CDR) (2008)

‘STISIM-Drive’ Meets ‘MotorcycleSim’: Using Driving Simulation Software to Develop a Unique Motorcycle Simulator for Rider Behavior Research Alex W. Stedmon1, David Crundall2, Elizabeth Crundall2, Rose Saikayasit1, Editha van Loon2, Alex Irune3, Patrick Ward2, and Neil Greig4 1

Centre for Motorcycle Ergonomics & Rider Human Factors, University of Nottingham, UK [email protected] 2 School of Psychology, University of Nottingham, UK 3 Dimax Technology Ltd, Nottingham, UK 4 The Institute of Advanced Motorists, IAM House, London, UK

Abstract. In order to compare groups of road users who have fundamentally different skills, attitudes and behaviours, one of the first in-depth motorcycle simulation studies of its kind was conducted. The project was developed using the innovative ‘MotorcycleSim’ simulator designed and built at the University of Nottingham. The simulator is a research tool to investigate aspects of motorcycle ergonomics and rider human factors and is the first of its kind in the world to incorporate ‘STI-SIM Drive’ software that allows motorcyclists to ride a full size motorcycle and interact with a virtual riding environment (VRE). To build a simulator that was both fit for the purpose of research and provided the desired levels of fidelity associated with real world riding, a user-centred design process was adopted from the outset (based in principles of ISO:13407). Keywords: transport simulation, motorcycles, rider behavior.

1 Introduction Capturing road user experiences in the real world is extremely valuable and underpins the applied nature of ergonomics within human-machine interaction [1]. However, due to practical, safety and ethical issues surrounding real world research, simulators offer the potential to develop alternative research tools [2]. Simulators offer a level of abstraction from the real world by providing an artificial environment in which users can experience characteristics of a real system [2]. Whilst there are limits to the degree of realism that can be achieved in simulators, if they are designed well, they offer high levels of ecological validity [1]. Underpinning this is a key human factors question regarding the level of realism required for a simulator to serve its intended purpose (e.g. training, research, product development, etc) based on a fundamental understanding of user-centred design principles [3]. From the outset, a detailed user-centred design methodology was adopted based on principles of ISO:13407 [4]. Without such an approach there is always a risk of specifying a solution that fails to support the users’ understanding of the target C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 76–80, 2011. © Springer-Verlag Berlin Heidelberg 2011

‘STISIM-Drive’ Meets ‘MotorcycleSim’: Using Driving Simulation Software

77

application [5]. A questionnaire survey incorporated 13 characteristics that participants rated on a 5-point scale [1]. The results indicated that the riders required the simulator to lean like a real motorcycle, have acceleration and braking effects and for the controls to work in a realistic manner. From additional comments that were made, riders highlighted the level of realism the simulation software might offer and how other traffic and road conditions could be incorporated into the virtual riding environment (VRE). Furthermore, a number of riders felt the simulator should be capable of replicating vibration and allow for riding with a passenger. The user requirements formed the basis of the development of MotorcycleSim as well as highlighting development routes for the future [1].

2 MotorcycleSim: An Overview MotorcycleSim has been built using a full size and fully equipped Triumph Daytona 675 motorcycle (kindly supplied by Triumph Motorcycles). The existing motorcycle controls (e.g. throttle, brake lever, brake pedal, gear selector, clutch lever) were modified or adapted to work with the simulation software in a realistic manner. MotorcycleSim uses commercially available ‘STISIM-Drive’ software which is an industry leader and has a worldwide support community. Although primarily a driving simulation package, the software can be used to build interactive VREs that allow for strictly controlled experimental repeatability in a laboratory setting as riders experience identical scenarios which are not possible on the road [3]. Within the ‘STISIM-Drive’ software it is possible to change vehicle characteristics to alter the simulated handling and performance profile of the motorcycle. Given focus of the software on driving applications, the vehicle characteristics have been fine-tuned to mimic a motorcycle and to a large extent this has been achieved by ‘fooling’ the software that the vehicle is a small and lightweight car with the acceleration and braking characteristics of a motorcycle. Alongside this, bespoke software and engineering solutions were developed to integrate the motorcycle hardware with the simulation software [2]. The visual representation (projected onto a large screen in front of the rider) can be manipulated in a number of ways: • enhanced acceleration and braking effects are presented by altering the degree of dynamic pitch in the visual scene as the motorcycle accelerates and slows down; • the scenery tilts as the rider steers in a particular direction to enhance the perception into leaning into a corner • the rider’s field of view can be increased to take account of peripheral visual cues. MotorcycleSim can be used in static or dynamic modes. In the dynamic mode the motorcycle tilts as the bespoke software takes data for speed and the steering angle in order to calculate an angle of lean (Figure 1).

78

A.W. Stedmon et al.

Fig. 1. The unique titling action of MotorcycleSim

MotorcycleSim, has been designed to offer four different simulation modes: static simulation with ‘STISIM-Drive’; cornering/leaning training; cornering/leaning training with ‘STISIM-Drive’; and an advanced dynamic model with ‘STISIM-Drive’ that aims to offer a realistic riding experience by utilising a comprehensive dynamics model to control the leaning of the motorcycle as the rider navigates the VRE.

3 Integrating MotorcycleSim with ‘STISIM-Drive’ In order to interact with the ‘STI-SIM Drive’ software, the mechanical movement of the throttle, braking, gear selection and steering angle of the motorcycle was interfaced with a standard automotive games controller unit (Figure 2). This solution was used as no precedent for a motorcycle controller exists with ‘STISIM-Drive’ and as such, a novel solution had to be explored: • throttle – the throttle on the motorcycle was connected to the games controller accelerator pedal via the standard throttle cable. When the throttle on the motorcycle is applied the throttle cable pulls the spring loaded foot pedal down. The degree of movement can be calibrated to different ranges to represent different motorcycle throttle characteristics. • braking – a limitation of the games controller is that the standard car brake pedal interface only offers a single braking input. On motorcycles using both the front and rear brake together or independently is a key riding skill and so an innovative solution was developed. Integrated and distributed ratios were developed so that the functional fidelity of the brakes is therefore preserved as the user is required to use both brakes to get the maximum braking effort (as would be expected on a real motorcycle). • gear selection – the motorcycle gears are interfaced with the ‘STISIM-Drive’ software using two ‘push to make’ switches wired to auxiliary buttons on the gear selector unit. The device was designed and manufactured to operate as a real motorbike, and not interfere with the rider’s natural riding position and gear changing action, and to have a minimal impact on the aesthetics of the motorcycle.

‘STISIM-Drive’ Meets ‘MotorcycleSim’: Using Driving Simulation Software

79

• steering angle - the steering wheel was removed from games controller so that the steering column could be connected to the steering unit of the motorcycle. A bespoke design allows for an initial free rotation of the column to allow for the ‘STISIM-Drive’ ±720o steering calibration.

Fig. 2. Integrated motorcycle controls with an automotive games controller

4 Using and Evaluating MotorcycleSim MotorcycleSim provided the basis for an in depth investigation of rider behaviour given its unique ability to test rider behaviour in a controlled environment [6]. Within a main riding scenario various sub-scenarios were developed and additional tasks were designed within the integrated experiment approach in order to explore a wide range of skills, attitudes and behaviours across three rider experience groups. The study was designed to compare Novice, Experienced and Advanced trained riders and the process adopted allowed for comprehensive data collection and analysis with minimal disruption to participants or biases creeping into the data [6]. By combining ‘STISIM-Drive’ and ‘MotorcycleSim’ this approach demonstrated the power and innovation of exploring a variety of rider attributes in a single investigation. Furthermore, and in line with the iterative principles of ISO:13407, an evaluation of MotorcycleSim has been conducted which have provided evidence to support the effectiveness of static simulation as well as ideas for future development [3].

5 Conclusions MotorcycleSim is at the forefront of motorcycle simulation. No other simulator exists like it, that provides a full size, interactive, realistically controlled, accurate performance measurement platform which can be used to investigate motorcycle design, rider equipment, rider behaviour, road safety and rider training and competence. A major strength in its development has been the underlying user-centred approach in capturing

80

A.W. Stedmon et al.

user requirements at the earliest opportunity which have provided a basis for the initial design and onward development of MotorcycleSim. The potential of transport ergonomics in all its guises is vast and novel methods and procedures are likely to emerge in order to better understand the fine detail of road user behaviour. Simulated trials will continue to provide a valuable research tool for investigators and given the investment in technologies and equipment, it is often necessary to design integrated approaches that offer the greatest potential to capture data on many levels. Acknowledgement. The authors would like to thank The Institute of Advanced Motorists, Phoenix Distribution (Arai helmets) and ‘STISIM-Drive’ for their support.

References 1. Stedmon, A.W., Brickell, E., Hancox, M., Noble, J., Rice, D.: MotorcycleSim: a usercentred approach for an innovative motorcycle simulator. In: Golightly, D., Rose, T., Bonner, J., Boyd Davis, S. (eds.) Creative Inventions & Innovations for Everyday HCI. The Ergonomics Society, Leicester (2008) 2. Stedmon, A.W., Young, M., Hasseldine, B.: Keeping it real or faking it: The trials and tribulations or real road studies and simulator research in transport research. In: Bust, P. (ed.) Contemporary Ergonomics 2009. Taylor & Francis, London (2009) 3. Stedmon, A.W., Hasseldine, B., Rice, D., Young, M., Markham, S., Hancox, M., Brickell, E., Noble, J.: MotorcycleSim: an evaluation of rider interaction with an innovative motorcycle simulator. The Computer Journal (2009), doi:10.1093/comjnl/bxp071 4. Earthy, J., Sherwood Jones, B., Bevan, N.: The improvement of human-centred processes: Facing the challenge and reaping the benefit of ISO:13407. International Journal of Human Computer Studies 55(4), 553–585 (2001) 5. Stone, R.J.: Virtual Reality. Virtual and Synthetic Environments: Technologies and Applications. In: Karkowski, W. (ed.) International Encyclopaedia of Ergonomics and Human Factors. Taylor & Francis, London (2001) 6. Stedmon, A.W., Crundall, D., Crundall, E., Saikayasit, R., Ward, P., van Loon, E., Irune, A.A.: Advanced Training & Rider Performance. University of Nottingham (2010)

AirportLogic: Usability Testing, Prototyping, and Analysis of an Airport Wayfinding Application Bennett Stone¹ and Yun Wang² 1

Department of Human Computer Interaction, Iowa State University, USA 2 Department of Graphic Design, Iowa State University, USA [email protected], [email protected]

Abstract. As air travel continues to be prevalent in our lives, and as the use of mobile devices designed to assist us become more common, the creation of wayfinding applications for mobile devices continues to be a challenge for both developers and interface designers. Unfortunately, many airport specific wayfinding applications do not live up to user expectations. In preliminary project phases, usability testing and evaluation of existing applications was performed, followed by prototyping and evaluation of a proposed application. Task-based user testing was performed to address two primary questions: 1) are current and proposed applications efficient for users?, and 2) is the information architecture and navigation suitable for average users? A prototype application and corresponding website were developed, and tested, with user testing confirming enhancements over previous airport way finding applications. Keywords: user interface design, mobile application, wayfinding.

1 Introduction Responsible for predicting user needs and technological capacity, most airport specific way finding applications ultimately do not live up to user expectations. After investigating a several airport navigational tools and applications for the iPhone, it was discovered that there is no single application that provides both basic and specific functionality necessary for travelers to effectively locate essential airport services and locations. The primary problem is that existing airport wayfinding and navigational assistance applications, which are designed for use on mobile devices have failed to provide both effective user interfaces, and useful way finding tools and information, essentially rendering them useless to travelers. Many existing applications contained unnecessarily large amounts of information in visual interfaces that were nearly impossible for individuals to fully use. In addition, several other applications were specific to only certain airport attractions such as pubs, airlines, or shopping, leaving all other potential points of interest up to the traveler to locate without the assistance of the application. In this project, the tasks included creating a simplified and intuitive mobile application for the iPhone in order to assist with common wayfinding tasks in both United States and global airports, as well as a developing corresponding websites that C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 81–84, 2011. © Springer-Verlag Berlin Heidelberg 2011

82

B. Stone and Y. Wang

would provide valuable pre-travel assistance with tasks such as booking airline, hotel, and rental car reservations; and creating an effective linking system between the mobile application and website. Tasks within the mobile application are assigned a rank order based on preliminary user feedback and evaluation of existing mobile applications in order to increase the speed and efficiency of wayfinding and location determination through a more intuitive user interface. Using task-based user testing with eight participants for the full testing, each participant was asked to complete several tasks based on a researcher provided scenario. Each participant was asked to complete basic operations within the AirportLogic application such as navigating to a gate within a selected airport, locating power outlets, and locating public transportation to the nearest city.

2 Method Preliminary user testing was performed using six participants selected from a design course, using existing airport wayfinding applications in order to effectively determine both necessary and unnecessary functionality, as well as isolating problem areas within each application. After preliminary user testing, a prototype application was developed and produced as a task-specific-functional1 prototype. Having designed the application to meet only essential users needs while traveling, the navigational structure was simplified through several stages of refinement prior to user testing, which was shown to be successful (100% of participants were able to perform the requested tasks). This task based user testing was designed to address three primary questions: 1) is the application efficient for the user? Does it allow a user to locate primary facilities, functions, and directions within an airport? 2) is the information architecture and navigation suitable for average users? and 3) are average users able to expediently find what they are looking for within the application?

Fig. 1. Left to right: Identification of problem areas during user testing, and solutions

1

Only pages, and information relevant to assigned tasks airport wayfinding application were made available. Participants were presented with an error screen when attempting to navigate to areas of the application that were not part of the user testing.

AirportLogic: Usability Testing, Prototyping, and Analysis

83

2.1 Participants Eight participants were randomly selected from a pool of participants for this research, each travelling by airplane at least 1 time per year (round trip). Table 1. Participant Information

2.2 Procedure Participants were asked to complete a pre-testing user profile indicating gender, native language, university status, computer and mobile application usage frequency, specific mobile travel application usage, and number of annual trips by airplane, after which they were directed to a task list. Participants were instructed to proceed through each task while thinking aloud. The list of tasks they were asked to complete are shown in Table 2. AirportLogic testing was performed using an iPhone simulator on a 15” MacBook Pro, using an external two-button scroll mouse while user application interactions were recorded internally using QuickTime’s screen recording feature. A researcher was present throughout the user testing to answer questions, take notes, and when necessary, redirect participants. After completion of user testing, participants were asked to complete a post use questionnaire that requested feedback on application usability and functionality, organization, overall impressions, and freeform comments. 2.3 Measures Data collected included user information, total number of user clicks per task, total time per task, number of errors per task, number of incorrect menu items selected, and number of incorrect icons selected2, and information regarding perceived usability. Data was analyzed and grouped by demographic variables, task, and user average as compared to ideal3 number of clicks or time spent on task. 2

3

Incorrect icon selection was determined by participant selection of a navigational icon that had already been selected, or was available with fewer steps. Ideal refers to the maximum number of selections, or time as determined by researchers through preliminary testing of the AirportLogic application.

84

B. Stone and Y. Wang Table 2. Task list and description

Locate and initialize AirportLogic Locate an international airport Navigate to a gate Locate a restroom near the gate Locate a power outlet near the gate Locate favorites within the application Locate a Starbucks near current location Locate public transportation to the city

Used to determine icon legibility No analysis: single option only Locate and identify terminal 1, concourse C, gate 20 Locating the nearest restroom to current user location Locating the nearest power outlet to current location Locate frequently used options within AirportLogic Locate Starbucks, and add to favorites menu. Multiple-step task Locate a train from the airport to downtown.

3 Results User testing showed positive results for the AirportLogic prototype; 100% of participants were able to successfully complete the user testing, 87.5% of participants were satisfied with the application, and there were no tasks rated as “difficult” or “problematic” by users. While a number of participants selected areas on screen that were not attached to menu items, resulting in an increased number of attempted item selections for locating tasks located within map display items, the majority of tasks were completed using minimal item selection, and were rated as being successful as compared to the researcher ideal number of selections and time periods.

4 Discussions and Conclusion This analysis of existing airport wayfinding applications, and design of a more focused and usable wayfinding application proved to be successful. Though there were problems identified within the interface used for testing and analysis, user ability to navigate through primary functionality was dramatically improved, while the final implementation of the AirportLogic application addresses all user-testing located issues. This ultimately led to the generation and implementation of innovative new ways of allowing individuals to navigate to primary locations of interest within airports. Future directions for this research include the implementation of iPhone-based prototype for user testing, as well as performing user testing within an airport.

Using on-Bicycle Rider Assistant Device While Cycling: A Hazard Perception Assessment Chao-Yang Yang and Cheng-Tse Wu Department of Industrial Design Chang Gung University Kweishan, Taoyuan, 333, Taiwan [email protected]

Abstract. Technologies have been developed to assist the cyclist for precise training exercise such as monitoring their body and bicycle state. Due to the limited position placing on-bicycle rider assistant device and riding posture, it enlarges the essential view range of the chance the cyclist distracted from the road. The risks using the system have been evaluated through an interactive video-based hazard perception/reaction test in which response times were measured. 5 participants, 3 male and 2 female participated a 2-phase section including 7 traffic scenes in a 5 minutes cycling exercise with and without using on-bicycle rider assistant device. As the results, average reaction times tended to increase 0.33 seconds and average mis-maintained pushing cadence time was 31.4 seconds when cycling with the device assistance. It is concluded that cadence control assisted by the device can interfere the efficiency of hazard perception, an important safety factor of cycling. Keywords: on-bicycle rider assistant device, hazard perception, response time.

1 Introduction Cycling becomes a popular leisure activity due to green issues and increasing request of changing lifestyle. For different purpose, bicycle has been designed into different functions such as road bike is for speeding and mountain bike is for crossing multi terrain. Cycling is seen as substitutions of fashion, taste and professional sport in which various add on accessories have been innovated e.g. riding information device becomes necessary for cyclist to monitor the distance, speed, heart rate, Revolution Per Minute (RPM), road slope, elevation, and GPS map. Road bicycle in nowadays can reach the speed of 40 kilometer per hour with ease and the risks to involve into an accident are high and it is possible to be fatal. Apart from riding, using these devices has created additional tasks that may increase the cyclist’s mental load and distract their attention to the road conditions. Recent researches of transportation safety mainly focus on car driving. Brown, Tickner and Simmonds (1969) have pointed out that two types of resource interference are generated from using mobile phone in car driving: interference between visual and hand controlling requirement; interference between conversation and driving behavior. This behavior could increase risk of accident by weakening C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 85–89, 2011. © Springer-Verlag Berlin Heidelberg 2011

86

C.-Y. Yang and C.-T. Wu

driver’s performance in speed control, straightness maintenance, sudden hazard response and ignorance of road signs (McKenna & Crick, 1997; Sagberg & Bjørnskau, 2006). Using telephone at the time of the crash was 2.4 times more than the crash of being at fault (Sagberg, 2001), and risk of accident was decreased with response time of hazards perception (Sagberg & Bjørnskau, 2006). However, these researches may not be enough to indicate the harness using latest On-Bicycle Rider Assistant Device (OBRAD) during riding a bicycle, as cycling activity is more dynamic and physical exercise intensive. For example, to maintain the speed, legs are pushing the paddles according to the road conditions. This could stress cyclist’s mental load and weaken his/her judgment of hazards. Therefore, we aimed to investigate cyclist’s safety risks through assessing his/her response time in detecting the hazard on the road in using OBRAD.

2 Methods To prevent uncontrolled factors such as weather or traffic, as shown in Fig 1, the experiment environment was setup in the lab with a size 48 (suitable for 158~170cm tall cyclist) steel tube road bicycle, which was reported as high crash risk type (Rodgers, 1997), Tacx™ Fortius Multiplayer T1930 Virtual Reality Cycle Trainer and Tacx™ Trainer software version 2.0 (T1990.02). Epson EB-410We short throw projector was used to project a 4:3, 230cm diagonal length, screen. The screen was positioned 170cm in front of the participant’s eyes. Garmin® Edge705 OBRAD were employed to display information with the size 36pt Display font; the display of the device was 40cm to the participant’s eyes. 5 Chang Gung university students, 3 male and 2 female, age between 18 and 22 years old, have more than 10 years bicycle riding experience, were selected to participate this experiment. Before the experiment took place, the bicycle were adjusted to the participant’s comfort i.e. the seat post were adjusted by the participant’s inseam length. To reduce unnecessary mental load, the gears were fixed in 50 (chainwheel) by 14(freewheel) and the break was deactivated. Human response on surrounding stimulus is discussed in a three-steps process: stimulus identification, response selection and response programming. Stimulus is detected through sensory organs e.g. eye, ear or skin and recognized through comparison of previous memory. Second, the type of physical reaction is selected and then human body will generate a series of muscle movement to perform the reaction. The duration of this mental processing and physical movement is counted as reaction time (Sagberg & Bjørnskau, 2006; Schmidt & Lee, 2005). Response time is the duration from a human receiving a stimulating signal to the first response (Sagberg & Bjørnskau, 2006). To test the response time, we selected obvious stimulus such as oncoming and overtaking cars to avoid unnecessary association. A button on the control bar for thumb clicking was designed to minimize individual physical difference in performing response. A red dot signal was transmitted to the screen and recorded with the main screen where response time were counted as the duration between the first appearance of hazard to the red dot signal showed up in the screen.

Using on-Bicycle Rider Assistant Device While Cycling

87

Fig. 1. The experiment appartus

Fig. 2. Examples of hazards in the video (up-left: oncoming car in white; up-right: oncoming car in the shade; lower-left: car overtaking; bottom-right: oncoming car in dark color)

The participants were asked to ride a 5 minutes section of rural area. The resistances controlled by the cycle trainer changed according the road conditions, in which the range of slope was controlled within ±3%, to prompt the participant monitoring the device and adjust their RPM. A practice section was first commenced for participants to warm up and get used to control their RPM assisted by the device. In task 1 (controlled group), participants clicked the button once they percept a developing hazard that could endanger cycling e.g. oncoming car or car overtaking (as shown in Fig 2) (Sagberg & Bjørnskau, 2006). There were 7 situations that could

88

C.-Y. Yang and C.-T. Wu

possibly develop into hazards occurred in the section and the duration of the critical intervals varied between situations. After 15 minutes break, the participants commenced task 2 in which the participants were additionally asked to perform a secondary task: maintaining leg cadence in 50±5 revolutions per minute (RPM). The times and duration that the cadence was not maintained within the tolerance were counted.

3 Results As shown in Table 1, without using the device, participants generally reacted the hazards within 0.6~1 seconds. However, participant’s performance was decreased in respects of speed control and hazard perception when they used the device. The average response time (standardised for each situation and averaged across situations) tended to be slightly longer in the experiment group (task 2). In the 5 minutes ride the participants averagely 0.33 seconds more in responding the hazards when use the device than without using it. They mis-controlled the RPM range for 31.4 seconds in which most of them associated with hazard situation. For example, the participants A failed to maintain their RPM 6 times in which 4 of them happened during the critical of hazards. Additionally, a false alarm happened on the time spot 04:13, the participant mistaken a white guardrail as a developing hazard. Table 1. The response time to the hazards situations (RT: response time)

Situation 1. In the town, a van appeared from shade 2. Outside the town, a white car overtook from left side. 3. On the straight road, a black van appeared from the far end of the road. 4. In a sharp right turn, a white car appeared. 5. In a sharp left turn, a white car appeared. 6. In a sharp right turn, a black car appeared from left and was willing to join the main road. 7. On the straight road, a white car appeared from the far end.

Time spot

Task 1 RT (sec)

Task 2 RT (sec)

Hazard missed

RPM missed

00:21

0.6

1.6

0

3

00:31

1

1.4

0

3

01:20

0.6

2.1

0

4

01:39

0.6

0

1

1

01:53

0.8

1.2

0

0

02:12

0.8

0.4

0

0

02:31

1

1

0

2

Using on-Bicycle Rider Assistant Device While Cycling

89

4 Discussion The results have suggested that using the information device while riding bicycle has increased the cyclist’s mental load and affect cycling performance. The complete set of the device should be designed in consideration of the possible risks it brings up. For example, the idea that having auditory stimulation could enhance human reaction performance e.g. speed and accuracy may be implemented in assisting cyclist to percept hazards around. It is possible that the relationship between individual experience and response time differs between situations (Sagberg & Bjørnskau, 2006). The participants were sensitive on the situation that hazards appear around them but far end of the road. Riding distance, reported as an important crash factor (Rodgers, 1997), will be considered in the future research. The results have implied the future research to look at the issues of the interface design, observation behavior, body posture, possible neck tiredness caused by repetitive glancing the device and different cycling conditions such as physical tired or uphill road. Acknowledgements. This research was supported by grants (NSC 99-2221-E-182049) from the National Science Council of Taiwan, and we thank them for financially supporting this research.

References Brown, I.D., Tickner, A.H., Simmonds, D.C.V.: Interference between concurrent tasks of driving and telephoning. Journal of Applied Psychology 53(5), 419–424 (1969) McKenna, F.P., Crick, J.: Developments in hazard perception. Transport Research Laboratory, Crowthorne (1997) Rodgers, G.B.: Factors associated with the crash risk of adult bicyclists. Journal of Safety Research 28(4), 233–241 (1997), doi:10.1016/S0022-4375(97)00009-1 Sagberg, F.: Accident risk of car drivers during mobile telephone use. Internation journal Vehicle Design 26(1), 57–69 (2001) Sagberg, F., Bjørnskau, T.: Hazard perception and driving experience among novice drivers. Accident Analysis & Prevention 38(2), 407–414 (2006), doi:10.1016/j.aap.2005.10.014 Schmidt, R., Lee, T.: Motor control and learning: a behavioral emphasis, 2nd edn. Human Kinetics, Champaign (2005)

Part II

Virtual and Augmented Environments

Human–Robot Collaboration with Augmented Reality Siam Charoenseang and Tarinee Tonggoed Institute of Field Robotics, King Mongkut's University of Technology Thonburi, Thailand [email protected], [email protected]

Abstract. This paper presents an implementation of human-robot collaboration by using augmented reality technique for providing necessary information to human operator. In this research, human operator and a robot arm share the same workspace in virtual object assembly task. The virtual objects are created in the form of 3D computer graphics that is superimposed on the real video image. While working in assembly task, the robot will assist the human operator by loading the virtual objects. Furthermore, a task planner controls all robots’ operations accordingly to human actions. Using augmented reality, human operator will receive robot’s task plan in the form of computer graphics during assembly task. The computer-generated information will support human operator’s decision for a suitable next step action. Keywords: Human - Robot Collaboration, Augmented Reality, Task Planning.

1 Introduction During recent years, augmented reality technology, which presents computer graphics information superimposed on real time video image, gains more interests. Previous research works implemented augmented reality to provide necessary information in the form of computer graphics to the operator during working with real objects [1],[2],[3]. Augmented reality is also used in human-robot collaboration task. Human operator will receive the augmented task information while working with robot [4],[5]. However, human and robot do not share the same workspace in previous related research works. Hence, this paper presents the development of human-robot collaboration with augmented reality system in the common workspace. The task presented in the research is a virtual object assembly. During human- robot operation, human will obtain virtual objects, graphics instruction, and robot’s action information through augmented reality technique. Moreover, robot will receive control commands from human operator to assist in transferring virtual objects.

2 System Overview This research allows the collaboration between human and robot in virtual assembly task. The virtual objects which are used in this task are in the form of 3D computer C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 93–97, 2011. © Springer-Verlag Berlin Heidelberg 2011

94

S. Charoenseang and T. Tonggoed

graphics and rendered on the detected marker plates. During assembly process, the operator will receive all graphics information which is 2D guidance texts, 3D symbols, and 3D virtual objects through LCD display. In addition, the operator can allow or decline the assistance from robot in transferring marker on which virtual objects are superimposed. 2.1 System Configuration Figure 1 shows the system configuration developed in this research. It consists of an operator, 5 DOF robot arm, computer, LCD screen display, video camera, and maker plates. During virtual object assembly, the operator and robot will move virtual objects on marker plates as system guidance. USB video camera is used to capture image of the workspace. The computer is responsible for image processing, task planning management, rendering all graphics on LCD screen, and controlling robot operation. The robot performs as an operator’s assistance. Its operations depend on task plan and operator’s decision commands.

Fig. 1. System Configuration

2.2 System Data Flow From data flow diagram in Figure 2, this system consists of 4 main components which are vision manager, graphics manager, task manager, and robot manager. First, ARToolKit [6] software library is used in the vision manager to process captured video images and obtain positions and orientations of targeted marker plates. Then, the vision manager sends targeted objects’ positions and orientations to the task manager. The task manager is responsible for generating action plan using STRIPS planning algorithm in assembly task. Next, the robot manager computes forward/inverse kinematics, reads robot’s joint angles, and sends commands to control the robot’s movement. The last component is the graphics manager which generates 2D and 3D computer graphics to present all guidance information and virtual objects on LCD display to the operator.

Human–Robot Collaboration with Augmented Reality

95

Fig. 2. System Data Flow Diagram

2.3 System Operation When system starts, it presents graphics instructions so that the operator can select the assembly model as shown in Figure 3(a). After selecting the assembly model, several virtual objects are rendered and superimposed on the markers. The system then generates a task plan of selected assembly model. That plan is interpreted to manage system states, present graphics information, and control the robot. The 2D graphics shows suggestion texts and information related to robot action while 3D graphics presents assembly guidance arrows such as suitable positions and orientations of virtual objects over the real video image on the LCD screen as shown in Figure 3(b) and 3(c). During the operation, robot will assist the operator to transfer targeted object after “Yes” virtual object is selected as shown in Figure 3(d). The operator can decline any robot action by selecting “No” virtual object. If the operator does not follow the suggested task plan, the system will generate a new task plan for that assembly task automatically.

(a) Model Selection

(b) Rotation Guide

(c) Robot Selection

(d) Robot Operation

Fig. 3. System Operation

3 Experimental Results There are 3 experimental sets to evaluate the proposed system in the areas of system performance, usability, and values for specific task.

96

S. Charoenseang and T. Tonggoed

3.1 System Performance Robot’s repeatability is one of the system performances. At least 6 targeted positions in the robot workspace are tested for obtaining the robot’s repeatability in both x and y axes. This experiment results indicate that the repeatability errors in x and y axes are 0.52 and 0.35 millimeters, respectively. Next, the graphics update rate is about 17 fps with the image resolution of 640 x 480 pixels. This system has ability of tracking the targeted markers which are moved with velocity less than 10 centimeters/second. 3.2 Usability After using the proposed system to do virtual assembly task, questionnaire is used to collect user’s satisfaction on the system’s usability. The results indicate that most of users are satisfied with the system’s usability. Using graphic arrow helps the users to understand the virtual object’s direction more easily. Hence, the usability of guidance arrows indicating the virtual object’s orientation got the highest satisfaction score with 92 percents. However, most users did not pay much attention at the suggestion text during operation. As the result, the usability of suggestion texts got the lowest satisfaction score with 74 percents. 3.3 Values for Specific Task This proposed system is applied in a virtual assembly task. In this task, the users were asked to assemble the virtual object as a given picture on the instruction sheet. This assembly task is set in three conditions as assembly with/without any system's information and assembly with system's information along with robot's assistance. The goal of this experiment is to reduce the operation’s time when the users work with this system’s assistances. Since the 3D virtual objects are mostly complicated, this task is difficult for the user who does not have much skill in 3D graphics assembly. Therefore, assembly without any system’s information took the most averaged assembly time at 182 seconds. With graphics guidance information, it can reduce averaged assembly time into 64 seconds or 64.84 percents of assembly time obtained from system without graphics guidance information. The averaged assembly time with system’s information and the robot is 144 seconds or 20.88 percents of assembly time obtained from system without graphics guidance information. The speed of this operation depends on the robot’s speed. After the experiments, the questionnaire is used to collect the user’s satisfaction. The results show that most of the users are pleased with this proposed system for the virtual assembly task. Because of using guidance graphics overlaid on real time video during assembly task can reduce the operating time. Therefore, the ability of system’s suggestion assisting user to assembly faster got the highest satisfaction score with 87 percents. Since the robot must work with human operator, it is set to operate at the low speed. Some users are not satisfied with this operation speed so the satisfaction of robot’s assistance got the lowest score with 81.33 percents.

Human–Robot Collaboration with Augmented Reality

97

4 Conclusions and Future Works The human- robot collaboration with augmented reality for virtual assembly task was proposed. The system’s assistance using augmented reality and the robot can reduce the operation times of the virtual assembly task. Furthermore, the use of augmented reality in the assembly task can save cost and training time. The virtual objects in assembly task can be changed into various object’s models by importing from the CAD applications. Moreover, the graphics guidance can help the user to understand the assembly task more easily so the assembly time is reduced. Based on the experimental results, there are many issues that can be developed to improve the system performance and usability. Since the operator does not pay much attention to read graphic texts during operation, the augmented guidance information should be graphic symbols instead of texts. The assembly time of operating with the robot can be reduced by increasing the robot’s speed. However, the change of robot’s speed should be considered along with human’s safety. For more natural communication, the interaction between robot and human can be improved by using gesture or speech recognition. Acknowledgments. This research work is financially supported by the National Science and Technology Development Agency, Thailand.

References 1. Triebfürst, W.W.G.: ARVIKA: Augmented Reality for development, production and service. In: Proceedings of the 1st International Symposium on Mixed and Augmented Reality. IEEE Computer Society, Washington, DC, USA (2002) 2. Pathomaree, N., Charoenseang, S.: Augmented reality for skill transfer in assembly task. In: IEEE International Workshop on Robot and Human Interactive Communication ROMAN 2005, pp. 500–504 (2005) 3. Yuan, M.L., Ong, S.K., Nee, A.Y.C.: Assembly Guidance in Augmented Reality Environments Using a Virtual Interactive Tool. Singapore-MIT Alliance Symposium, Singapore (2005) 4. Otmane, S., Mallem, M., Kheddar, A., Chavand, F.: Active Virtual Guides as an Apparatus for Augmented Reality Based Telemanipulation System on the Internet. In: Proceedings of 33rd Annual Simulation Symposium (SS 2000), Washington, DC, USA, pp. 185–191 (2000) 5. Green, S.A., Chase, J.G., XiaoQi, C., Billinghurst, M.: Evaluating the Augmented Reality Human-Robot Collaboration System. In: 15th International Conference on Mechatronics and Machine Vision in Practice M2VIP 2008, pp. 521–526 (2008) 6. Human Interface Technology Laboratory, ARToolKit, http://www.hitl.washington.edu/artoolkit/documentation

Making Pixel Patterns Automatically for Camouflage – Using Color Information from Their Background Woon Jung Cho1, Wonmi Ahn1, Myung Shik Kim1, Jeeyea Park1, Seungduk Kim2, and Kwang-Hee Han1 1

Cognitive Engineering Lab. Department of Psychology, Yonsei University, Japan 2 Department of Computer Science, Yonsei University, Japan [email protected], [email protected]

Abstract. Camouflage is an attempt to obscure a target’s outline and match its background for hiding the target. Detectability of a target depends on the features of the background and changes in the surrounding. Thus, we have to consider visual information from the background and apply them to the development of an effective camouflage pattern. As dynamic environments are essential for designing an adaptive pattern, controlling digital images and making patterns automatically by use of computer can be efficient alternatives through the designing process of development and evaluation. In this study, we made the automatic pixel patterns program for the camouflage assessment. For making patterns, color information derived from background was used and pixel-dot type of pattern was considered in view of the digital pattern. Automatic stimuli-pattern maker developed in this study follows several steps; load an image, select an area, extract colors, and create a pattern. This automatic implementation has some benefits for design processes for camouflage pattern development. Further studies are needed not only for program-upgrade but also for improvement of color strategy through analyzing camouflage features. Keywords: camouflage pattern, camouflage assessment, photosimulation, adaptive pattern.

1 Introduction Recently, we heard of the breaking news that British a Defense Company is developing the invisible tank. Using attached electric sensors on the surface of the tank it could process color, line and shape from surroundings, and then project them on a tanks’ exterior. This invisible tank technology, so-called ‘e-camouflage system’ aims to merge target and background. In the context of military, these demands for finding the adaptive camouflage strategy in dynamic environment conditions will grow as battle conditions vary. Detectability of a target depends on the features of the background and changes in the surrounding. Thus, we have to consider visual features like colors or patterns from C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 98–101, 2011. © Springer-Verlag Berlin Heidelberg 2011

Making Pixel Patterns Automatically

99

environments and apply them to the design process for an effective camouflage. In addition, empirical detectability data of developed patterns on dynamic surrounding conditions are required for effective camouflage pattern rather than designers’ subjective point of view. In this study, we developed the automatic pixel pattern program for photosimulated camouflage assessment aimed to investigate how visual features from background could affect a target’s ability of camouflage.

2 Camouflage Pattern Design and Assessment 2.1 Camouflage Pattern Design Camouflage is an attempt to obscure the target’s signature and match its background[1,2,6] in order to hide or disguise the identity of the target. In the same context, the purpose of camouflage pattern is to minimize the noticeability and maintain the visibility of pattern at the same time[2,5]. To satisfying the two contradicting purposes for camouflage, features of camouflage pattern have to be similar with the visual features of the targeted background as much as possible. Some features like color, shape, and size among others can be critical factors for designing patterns. Designers have to deal with these factors sufficiently and increase similarity with the targeted region for effective camouflage design. So far, most camouflage patterns were generated by the subjective view of expertdesigners who could deal carefully with characteristics of background conditions like color and pattern, and so on. But now, these traditional approaches can incur relatively high cost and time to create adaptive camouflage patterns as the military contexts are changed[3,5]. If dynamic environments are essential for designing adaptive patterns like the ‘e-camouflage’ system, controlling digital images and making patterns automatically by computer can be efficient alternatives through the design process of development and evaluation. 2.2 Camouflage Assessment by Photosimulation Evaluation of the camouflage effectiveness is also a fundamental step for optimal camouflage pattern. There are two camouflage assessment methods; one is field observation, the other is photosimulation[4]. The former performs the camouflage assessment on natural field conditions. While this field observation is able to obtain most practical evaluation data, it has some potential risk factors disturbing detection performance of camouflaged targets. The latter, in contrast, photosimulation performs on the controlled experimental situation, so it is made possible to produce rich statistical data from different groups of observers with the same image sets. Moreover, it has some strengths in designing patterns in respect that we are able to collect images continuously under the identical conditions and spare the additional expense [4]. Photosimulation is used to design or evaluate the developed pattern for dynamic and various military contexts because it is implemented on digital environments [3,4]. Actually, it is usual to apply color information as quantitatively computable data to experimental settings. These

100

W.J. Cho et al.

simulated results are applicable to real military environments, but potential influence factors in natural situations are not considered. In this study, we designed the visual target search experiments applying photosimulation and developed the automatic program for making pattern-stimuli under a dynamic background for the purpose of comparing camouflage effectiveness of patterns easier.

3 Automatic Program for Making Camouflage Pattern Stimuli Making patterns automatically from digital images is beneficial to camouflage assessment experiments using photo simulation. The reason is that generation of patterns is easier for various backgrounds and quantitative descriptions of background features. 3.1 Overall Flow of Automatic Program The automatic pattern-stimuli maker developed in this study follows several steps for generating patterns as the outcome. (1) At first, load a photo used as a background image and (2) assign target area. After target area is selected, (3) this program extracts colors based on color information of the selected area, and (4) lastly creates a pixel pattern using extracted colors which are based on color features of the background.

Fig. 1. Overall flow of developed program

3.2 Use of Color Information and Generation of Pattern Stimuli If the colors in the camouflage pattern are well extracted from the surroundings, they are used to allow the target to blend into the background and reduce any visual cues[2,3,5]. Several countries investigated color characteristics of different environments and applied different color schemes to development of patterns for dynamic surroundings[2]. In many ways, the use of color information derived from

Making Pixel Patterns Automatically

101

background images plays an important role in detectability of camouflaged targets. Further, it is critical to extract the limited numbers of appropriate colors. In this program, we used color information as a best feature for effective camouflage and pixel-dot type of pattern was considered in view of the digital pattern. This automatic pattern program derives selected numbers of colors based on color frequency information of the selected area and then replaces all pixels of the targeted area with extracted frequent-colors according to the similarity measure.

4 Discussions In this study, we developed the automatic pixel pattern program for the camouflage pattern stimuli, which is aimed to investigate how visual features from background could affect detectability of camouflaged targets through photo-simulated camouflage assessment. As a result of the developing and testing the automatic pattern stimuli program, it was determined that strategy used for color processing was less effective for considering color characteristics of a background in some conditions. Nevertheless, this implementation deserves to be considered continuously through the design process for camouflage pattern development. For better effectiveness, further studies are needed not only to upgrade the program but also to improve color strategy through analysis of camouflage features. Acknowledgement. This work has been supported by the Low Observable Technology Research Center program of Defense Acquisition Administration and Agency for Defense Development.

References 1. Copeland, A., Trivedi, M.: Models and metrics for signature strength evaluation of camouflaged targets. Algorithms for synthetic aperture radar imagery IV, 194–199 (1997) 2. Friskovec, M., Gabrijelcic, H., Simoncic, B.: Design and Evaluation of a Camouflage pattern for the Slovenian Urban Environment. Journal of Imaging Science and Technology 54(2), 020507-(11) (2010) 3. Mitchell, K., Staples, C.: Methods for deriving optimum colours for camouflage patterns. In: RTO SCI Workshop on Search and Target Acquisition in Netherlands (1999) 4. Peak, J.: Guidelines for Camouflage Assessment Using Observers. Nato Research & Technology Organization Neuilly-Sur-Seine RTO-AG-SCI-095 (2006) 5. Ramsley, A.: Camouflage patterns-effects of size and color, technical report released on DTIC online (1976) 6. Riley, T.: Testing visual Camouflage, innovations report, National centre for computer animation in Bournemouthh University (2004)

Virtual Bridge: AR-Based Mobile Interaction for Easy Multimedia Control of Remote Home Devices DongJin Eun, Taik Heon Rhee, Seonghoon Kang, Minsuk Choi, Sangil Lee, and Hark-Joon Kim Samsung Electronics Co., Ltd. 416 Maetan3, Yeongtong, Suwon, Gyeonggi 443-742, Republic of Korea {dj.eun,taikheon.rhee,shoon.kang,minsuk1.choi, lignas.lee,harkjoon.kim}@samsung.com

Abstract. This paper proposes an interaction for controlling multimedia contents of remote devices using a mobile device based on AR technology. Using a real-time object recognition method, home devices detected by the camera of a mobile device are displayed on the camera preview screen along with the thumbnails of their own multimedia contents around the recognized positions. A user may drag a multimedia content which he or she wants to play, and drop it onto another target home device which he or she wants to play the content through. The user study showed that the proposed interaction expects higher usability since once a home device is registered with its device name when registering its image shown on the mobile camera for the object recognition, this matching process is no longer necessary when a user controls the device through the mobile device. Keywords: Mobile Interaction, Augmented Reality, Mobile AR, DLNA, Contents Sharing, Real-Time Object Recognition.

1 Introduction Recently, most of homes have several home devices holding multimedia contents, such as televisions, digital video recorders, photo frames or PCs, and the interaction issues between home devices have taken on great important. Although standardized technologies for transferring multimedia contents among home devices, including DLNA (Digital Living Network Alliance) [1], are spreading rapidly, the user interfaces to find other home devices or multimedia contents on them are still complex so that users not only need many steps to control the home devices. Moreover, users feel a lot of trouble in matching a device itself with its name or identifier displayed on the other device which is to connect with it. Most of the commercial DLNA solutions use a list of home devices with text and icon, when a user needs to connect a server device with a display device. Figure 1 shows the example screenshot of selecting remote devices in commercial solutions, Samsung AllShare [2] and Braintransfer DLNA remote [3]. As shown in Figure 1, a user should always memorize the correct name of the target device whenever he or she wants to control remote devices. In addition, such a series of selection steps is C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 102–106, 2011. © Springer-Verlag Berlin Heidelberg 2011

Virtual Bridge: AR-Based Mobile Interaction for Easy Multimedia Control

103

Fig. 1. Example screenshots of selecting remote devices in commercial solutions

required in the commercial solutions and it makes the user interface look complex and difficult to use. This paper proposes an interaction for controlling multimedia contents of remote devices using a mobile device based on AR technology. When a user focuses on one or more home devices through the camera of a mobile device, they are detected by a real-time object recognition method on the camera preview screen of the mobile device. The position of each recognized device is displayed on the touch-enabled camera preview screen, along with the thumbnails of its own multimedia contents around the recognized position. Then the user drags the thumbnail of a multimedia content which he or she wants to play, and drops it onto the recognized position of another target device which he or she wants to play the content through.

2 Virtual Bridge: Interaction Design Since Tani et al. [4] proposed the concept of using live video images to make users interact with a real-world object located at a distance, a number of studies have been utilized the metaphor in various usage scenarios. Boring et al. [5] mentioned a number of related works which adopted the “interaction through video” metaphor, while they proposed Touch Projector, a mobile interaction through video. In their work, the Touch Projector allows users to manipulate content shown on displays at a distance by touching the image of the content taken by a mobile camera. Our proposed interaction called the Virtual Bridge focuses on virtually bridging two real-world home devices by a mobile controller. In other words, our interaction connects a Digital Media Server (DMS), which stores content, with a Digital Media Renderer (DMR), which play content from a DMS, by the touch interaction on live video images taken by the camera of a mobile Digital Media Controller (DMC), which finds the content on DMS and plays it on DMR. The definitions of DMS, DMR and DMC are introduced on the DLNA specification [1]. The name of the “Virtual Bridge” reflects the concept that a user’s bridging interaction of two devices in the real world through the virtual space on a live video image. Figure 2 shows the walkthrough of the concept of the Virtual Bridge. Note that the mobile controller (DMC) itself may take on a role of content server (DMS) and directly connect its content to the remote device (DMR) for rendering.

104

D. Eun et al.

Fig. 2. User scenario flow. (a) A user focuses on one or more home devices through the camera of a mobile device. (b) The position of each recognized device is displayed on the touchenabled camera preview screen, along with the thumbnails of its own multimedia contents around the recognized position. (c) The user drags the thumbnail and drops it onto the position of another target device. (d) The content corresponding to the thumbnail is played through the target device.

The Virtual Bridge interaction needs pre-registration procedure of the home devices as shown in Figure 3. Once a home device is registered with its device name when registering its image shown on the mobile camera for the object recognition,

Fig. 3. Pre-registration procedures. (a) A user takes a snapshot of the target device to register and extract features of the image. (b) The snapshot with the extract features is matched with the device name for the real-time object recognition afterwards.

Virtual Bridge: AR-Based Mobile Interaction for Easy Multimedia Control

105

this matching process is no longer necessary when a user controls the device through the mobile device.

3 Implementation and Evaluation To compare usability of the proposed interaction with Samsung AllShare [2], a conventional approach applied in commercial products, we implemented a prototype of Virtual Bridge interaction controller. The controller (DMC) prototype was implemented as an android application on a Samsung Galaxy Tab (SHW-M180S). A variant of SIFT algorithm [6] has been implemented for the real-time recognition of home devices on live video images. The DLNA functions in Windows Media Player 7 application on a notebook PC and Samsung Galaxy-S, a smartphone with Samsung AllShare were used for the content server (DMS). Also, a television supporting Samsung AllShare was used for the content renderer (DMR). All the devices were assumed to be connected to a Wi-Fi network in advance. Figure 4 shows examples of the implemented prototype in use.

Fig. 4. Prototype implementation of the Virtual Bridge. (a) Searching content through a server (smartphone). (b) Moving content to a renderer (TV).

To measure user-satisfaction with the proposed interaction compared with the commercial solution, a questionnaire based on the IBM computer usability satisfaction questionnaire [7] was given to 10 participants, who were aged between 25 and 36 with a mean of 29.7 years. The questionnaire consisted of 7 items in total, using a 5-point Likert scale from strongly agree (5) to strongly disagree (1). Figure 5 presents a summary of the results. In general, the proposed interaction gained better results than the conventional approach. Especially, the participants felt that it was more enjoyable, more intuitive, and easier to learn than the conventional approach. From the questionnaire results with comments of the participants, we may infer that the results are due to the direct matching between the device name listed in a controller and its image shown in the live video image.

106

D. Eun et al.

Fig. 5. Average user feedback regarding the interaction techniques

4 Conclusion and Future Work This paper proposes an interaction for controlling multimedia contents of remote devices using a mobile device based on AR technology. While the conventional approach always requires matching a remote device and its name, the proposed interaction makes a user connect content from a server to a render intuitively on the camera preview screen of a mobile controller. The proposed interaction relies on performance of the real-time object recognition algorithm to detect home devices on live video images. The currently implemented algorithm only detects a home device when its image is taken in the similar direction with the one taken at the registration time. In the future, we plan to overcome such a limitation by improving the recognition algorithm robust on direction variations, as well as introducing proximity sensors or indoor position determination technologies. Meanwhile, we also plan to find various user scenarios beyond the multimedia control, based on interaction among home devices.

References 1. Digital Living Network Alliance, http://www.dlna.org 2. SAMSUNG AllShare, http://www.samsung.com/au/tv/internet-tv.html 3. DLNA Remote, Braintransfer Inc., http://braintransfer.de/en/services/mobilesolutions/ dlnaremote 4. Tani, M., Yamaashi, K., Tanikoshi, K., Futakawa, M., Tanifuji, S.: Object-oriented video: interaction with real-world objects through live video. In: Proc. CHI 1992, Monterey, CA, USA, pp. 2287–2295 (1992) 5. Boring, S., Baur, D., Butz, A., Gustafson, S., Baudisch, P.: Touch Projector: Mobile Interaction through Video. In: Proc. CHI 2010, Atlanta, GA, USA, pp. 2287–2295 (2010) 6. Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. Int. J. Computer Vision 60(2), 91–110 (2004) 7. Lewis, J.R.: IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use. Int. J. Hum.-Comput. Interact. 7(2), 57–78 (1995)

Design and Implementation of a Low-Cost Projected Virtual Reality System to Support Learning Processes Rodrigo Gómez and Helmuth Trefftz Virtual Reality Lab , EAFIT University Medellín, Colombia [email protected]

Abstract. Virtual Reality technologies have been successfully incorporated into the learning processes and potential new applications in education are explored continuously. We found that one of the difficulties to popularize its use in the educational context, in countries with emerging and developing economies, is the cost of hardware required to generate satisfactory immersive experience. In this work we considered virtual reality from the perspective of human-computer interaction to support learning processes. The characteristics of low-cost projected virtual reality system (PVR) proposed, combines study and integration of available technology solutions, the development of an image synchronization routine that enables the use of a single video projector and the design of a printable pattern that preserve the state of polarization on the projection screen. Keywords: Virtual Reality, projection surface, polarization.

1 Introduction The Virtual Reality (VR) is a strong tool for the learning support process, including tangible and abstract concepts, increase critical performance in activities such as surgical procedures and technical transfer skills [1,2]. Immersion in the VR experience is determined as a key element in the cognitive variables associated with the processes of learning [3]. The technology used in VR is the main limitation for immersive experience. The computer graphics is the spearhead for generation of three-dimensional digital worlds and so far is limited by the processing power, not by lack of algorithms [4]. Something alike occurs with visualization hardware, even though today we have massive HD stereoscopic screens, the first 3D optical perception still requires the use of passive and active goggles. The autosteroscopic display are still far away from popularized as a tool for daily technology VR that support learning processes [5]. In the case of most emerging & developing countries, the costs associated with IT equipment for virtual reality, and their rapid obsolescence, have made quite difficult the incorporation of VR as a tool to support the learning processes [6]. Notwithstanding the foregoing, the software development and 3D digital content have been identified as a potential factor to increase the economic competitiveness of emerging economies, where there was a latent need for hardware related to VR technology [7,8]. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 107–111, 2011. © Springer-Verlag Berlin Heidelberg 2011

108

R. Gómez and H. Trefftz

This paper considers the critical points for the implementation of a virtual reality system design that includes: a single video projector, a polarize rotary disk filter like a beam shutter type, a printable pattern that preserves the state of polarization on the projection screen.

2 Related Work The PVR system was treated as a set of devices and software which combined allow stereo viewing digital content and the interaction with them [9]. We identified three critical subsystems for the operation of a PVR system Fig.1(a), the display, data transmission and interaction subsystem.

Fig. 1. PVR system scheme based on three subsystems (a). Schematic display subsystem (b).

2.1 PVR Subsystems Display subsystem, Responsible for viewing digital content and devices for the whole image makers (video projector),image filtering (polarizers), alignment, and optical image formation (projection surface). As shown in Fig. 1(b), unlike traditional configurations, requiring the use of two video projectors [10] required for generating the stereoscopic selective filtering of each image perspective to the user corresponding eye. Other works [11] have shown how the selective control of each image at a frequency of 30 Hz for each eye has been sufficient to generate an acceptable stereoscopic. Guided by [11] and considering new technologies for the selection of images [12] PVR system was designed with a single video installation projector and a rotary disk-shaped polarizer. In order to obtain the perspective perception in the passive PVR, conventionally projected two images with different points view simultaneously [10], so the display surface (Projection screen), must retain the polarization of each image. Projection surfaces are called "Silver" screen, in the commercial use, and have higher associated costs against the standard projection surfaces. Other studies have put forward proposals alternatives to preserve the polarization with good results performance [13]. Send data subsystem, includes hardware and software necessary to receive and send information subsystems according to the characteristics and requirements

Design and Implementation of a Low-Cost Projected Virtual Reality System

109

thereof. At this time, the stereoscopic driver that have better performance uses GPU data processing [14]. It supports the majority of branded graphics cards, and is designed to work with most 3D hardware solutions.

Fig. 2. Schematic send data Subsystem

Subsystem interaction is defined as the set of peripherals which is effected by interaction with the digital scene. In the market offers a wide range of devices that meet this function, standing between haptic and kinetic devices [15]. The Wii controller has been tested as a successful tool human-computer interaction technologies related to VR. A wide range of applications and libraries available in different programming languages under the GNU (General Public Licences); on the other hand, it was considered a low cost hardware (40 USD per unit) [16].

3 Experimental Procedures and Results The display subsystem design was set with reference to the optical axis of projection, the polarized filter was placed into the lens front of the video projector (Figure 3), in order that the light that comes from the video projector, is polarized when goes through the disk.

Fig. 3. Rotary polarizer for display subsystem: (a) Schematic representation, (b) montage implemented

For the rotation synchronization of the polarizer with the sequence of projected images, we developed a routine called "digital image synchronization stereoscopic (DSIS)" (figure 4), synchronization sequence image. DSIS routine maintains the cross-polarization between the projected images in sequence and compensates the natural delay generated by the rotation of the motor at a constant revolution 60 Hz.

110

R. Gómez and H. Trefftz

Fig. 4. Timing diagram of the DSIS routine

In order to maintain the standard of usability in the system application proposed, a mono images output application from BINO software was implemented on the VGA output signal to DSIS input routine, establishing itself as the reference signal for synchronizing video and disk rotation frequency polarizer, thus has an open loop to maintain state sequential polarization between the projected images. For the projection surface, we designed a printable pattern that successfully retained polarization stage (Figure 5); cost implementation was minimal compared to the Silver screen available on market.

Fig. 5. Gray scale for printables patrons. The projection surface had better perfomace whit printable patron number 6.

4 Conclusions and Future Work One of the main differential PVR system designed over conventional systems was the use of a single video beam. Which avoided the need for optical axis alignment of the images, it requires additional study to compare the level of immersion of this design compared to conventional. The timing sequence of image for the cross polarization, could consider an alternative technique for stereoscopic display and its performance depends on the delta of change in the state of images polarization. The designed printable patron can be used as a projection surface PVR system with similar benefits to the “silver” screen. However it requires a holographic quality printer resolution in order to optimize its functionality.

References 1. Ausburn, L.J., Ausburn, F.B.: Effects of desktop virtual reality on learner performance and confidence. Opening a line of inquiry, Journal of Industrial Teacher Education 45(1), 54– 87 (2008)

Design and Implementation of a Low-Cost Projected Virtual Reality System

111

2. Lapointe, J.-F., Robert, J.-M.: Using VR for Efficient Training of Forestry Machine Operators. Education and Information Technologies 5(4), 237–250 (2000) 3. Psotka, J.: Immersive training systems: Virtual reality and education and training. Instructional Science 23(5-6), 405–443 (1995) 4. Goodnight, N., Wang, R., Humphreys, G.: Computation on Programmable Graphics Hardware. IEEE Computer Graphics and Applications, 12–15 (2005) 5. Javidi, B.: Three-Dimensional Television, Video and Display Technology. Springer, New York (2002) 6. Pickles, J.: The social implications of geographic information systems. Guilford Press, New York (1995) 7. Commission for Communications Regulation COLOMBIA government.: Analysis of the ICT sector in Colombia: Evolution and Challenges. Document Analysis and Relation of Infrastructure Center Industry Knowledge (2010) 8. Dutta, S., Mia, I.: The Global Information Technology Report 2008–2009 Mobility in a Networked World. World Economic Forum Editors (2009) 9. Churchill, E.F., Snowdon, D.N.: Collaborative Virtual Environments. In: Munro (ed.) Springer, Heidelberg (2001) 10. Gutiérrez, M., Vexo, F., Thalmann, D.: Stepping into Virtual Reality. Springer, Heidelberg (2008) 11. Lee, K.-H., Jang, T.-J.: A study on a stereoscopic display system using a rotary disk type beam shutter. In: Smith, M.J., Salvendy, G. (eds.) HCII 2007, Part I. LNCS, vol. 4557, pp. 868–875. Springer, Heidelberg (2007) 12. Masat, M., Minami, A.: High-speed polarizing device and high-speed birefringence measuring apparatus and stereoscopic image display apparatus utilizing the polarizing device. US Patent No 7,742,169 B2. (June 22, 2010) 13. Zelle, J.M., Figura, C.: Simple, low-cost stereographics: VR for everyone. In: SIGCSE Bul., vol. 36, pp. 348–352. ACM, New York (2004) 14. Open inventor architecture group.: The Oficial Reference Document for Open Systems. Addison-Wesley, Reading, MA, USA (1994) 15. DeFanti, T.: VR: Past, present, and future. In: International Conference on Image Processing (ICIP 1998) vol. 3, p. 1 (1998) 16. Lee, J.C.: Hacking the Nintendo Wii Remote. IEEE Pervasive Computing, 39–45 (JulySeptember 2008)

Interface Design to Support Situation Awareness in Virtual Puppetry Keisha Harthoorn and Stephen Hughes Department of Computer Science University of Northern Iowa {harthook,sthughes}@uni.edu

Abstract. We propose virtual puppetry as a potential mechanism for enhancing students’ presence in a virtual learning environment. To develop this style of interaction will require substantial attention to the user interface in order to promote the operator’s situation awareness of both the real and virtual environments. This presentation describes the development of an initial prototype and some of the ongoing concerns for controlling virtual puppets. Keywords: Virtual Puppetry, Interface Design, Situation Awareness, Virtual Environments, Presence, Virtual Heritage.

1 Motivation Immersive theaters (Domes, Caves, etc) and other interactive large-screen displays are becoming increasingly popular installments in museums to allow patrons to experience “Virtual Heritage” exhibits. Likewise, portable versions of these exhibits are finding their way into classrooms. Experiencing ancient structures first-hand can give patrons or students a powerful familiarity with these environments that they would not otherwise have [1]. The central element of these displays is usually a threedimensional computer model of an ancient space. One of the fundamental concerns for many of these virtual learning environments is how to cultivate “presence”, the subjective feeling of being part of the virtual environment. This is based on the long held belief that the increased student presence will yield opportunities for deeper learning [2]. In hopes of increasing presence, considerable effort goes into modeling and rendering environments to make them visually accurate and believable; however, the illusion of presence can be easily disrupted by other external factors. We propose that, in addition to visual fidelity, the communication channel between teachers and students also plays a significant role in students’ presence. Many educational VR systems involve teachers and students collaboratively viewing a virtual model. In this scenario, all of the participants are on the outside of the virtual world, looking in. Therefore, any interaction between teacher and student occurs outside of the virtual environment and necessarily disrupts the sense of presence. This disruption can be avoided by moving all participants into the virtual space, allowing them to be represented in the environment through personal avatars, providing an interaction style is reminiscent of Second Life [3]. While this ensures that all interaction takes place in the virtual world, students must divert some of their C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 112–115, 2011. © Springer-Verlag Berlin Heidelberg 2011

Interface Design to Support Situation Awareness in Virtual Puppetry

113

attention to manipulation of their character. This may come at the expense of focusing on the material to be learned. An alternative model allows students to interact directly with a teacher represented virtually in the environment. While technology is readily available to automate virtual characters, their actions often appear transparently scripted, especially when the character needs to interact with a real person [4]. To allow robust interaction between the audience and virtual characters probably still demands human interpretation of audience questions, reactions and gestures. We therefore suggest that to students’ presence in educational VR systems could be enhanced by interacting directly with a virtual character that is controlled by an unseen human operator; a puppeteer. To develop this style of interaction will require substantial attention to the user interface for the human operator. People have the advantage of being able to tap into a wealth of deictic behaviors and gestures that are practically involuntary. Moreover, there are social constraints for interacting with the audience (making eye contact, for example) which are second nature for human presenters, but need to be directly activated by a puppeteer. Not only must the virtual character support various gestures and behaviors, but they must also be easily and effectively toggled by the operator. Situation awareness is the ability of an operator to perceive, comprehend and act in a highly dynamic environment [5]. The ability to establish and maintain situation awareness is heavily influenced by the design of the interface. In this application, the operator to be aware of events that are taking place in both the environment and in the audience and be able to respond to each appropriately. For this reason, it is critical for the interface to aggressively support the situation awareness of the puppeteer.

2 Design Principles and Prototype The development of our virtual puppeteering interface was driven by principles that are consistent with the interface design guidelines proposed by Endsley [5]. Specifically, we attempt to present a view of the overall environment, focus the viewer on aspects of the environment that are relevant, take into account both topdown and bottom-up processing and simplify interaction demands on cognitive load. 2.1 Multiple Views Our first principle recognizes that the framework must be capable of generating and displaying multiple distinct views to communicate information about both the virtual environment and audience. For our prototype, we worked with the Unity game engine [6] in order to generate the virtual puppet and environment. Three distinct views of the virtual information were designed: 1.

A first-person perspective of the environment allows the puppeteer to see the environment from the eyes of the puppet. 2. A third-person view shows the puppet in the context of the environment as seen by the audience. 3. A map view provides the context of the character within the environment from a bird’s eye view.

114

K. Harthoorn and S. Hughes

In addition to the views of the virtual environment, a live view of the audience can be maintained using a video feed from a webcam. 2.2 Focus+Peripheral Display Since the puppeteer will be required to monitor and interact with both the audience and the virtual environment, it is critical that the puppeteer is able to quickly assess the state of a particular view and understand what intervention is needed. The second principle that we attempted to design for recognizes that all information related to the operator’s goals should be available, but information relevant to the current task should be more prominently displayed. To achieve this, we designed a multi-screen Focus+Peripheral display (see Figure 1). This array of physical display devices consists of four standard-size monitors with larger display projected above them. The projected screen serves as the primary screen Fig. 1. Focus+Peripheral Display for interaction, and is known as the “Focus” screen. The four smaller screens hold peripheral information and allow the operator to monitor other views of the environment. When information contained on a peripheral display requires the attention of the operator, he/she must swap that peripheral display to the focus display. Note that the physical location of each peripheral view remains constant so that the operator can always find the desired view. In our initial development, we compared two techniques for bringing the display into focus: 1) Warp - A button press corresponding to the desired peripheral screen causes it to take focus; the location of the mouse pointer remains in the same relative screen location. 2) Point-to-Screen - The user gestures toward the peripheral screen that they wish to activate; the mouse pointer is repositioned to the relative entry/exit point of the focus screen. An informal comparison suggests that the point-to-screen seemed intuitive, but the warp technique was faster and more natural. 2.3 Puppet Capabilities: Marotte Interaction The term puppet covers a wide range of manipulative devices. Puppets vary from simple finger puppets up to complex marionettes, which are controlled by any number of bars, levers and strings. While it is compelling to have control of a fully articulated puppet, we sought to create a system that minimizes the control options. Granting complete, direct control of the virtual character’s posture and gestures would potentially overload the operator’s working memory at the expense of situation awareness. Moreover, this level of control may not be practical for instructors without an extensive knowledge of advanced puppetry. To this end, we attempted to distill the puppet interface down to its minimum. To capture the essence of an environmental

Interface Design to Support Situation Awareness in Virtual Puppetry

115

guide, we adopted the model of a Marotte, which is a modified rod puppet [7]. With a physical Marotte, the puppeteer is capable of manipulating the puppet’s head and body using a central rod. The puppeteer uses a free hand to act directly as the puppet’s hand. This allows the puppeteer to control the location and direction of gaze of the puppet, as well as perform basic hand gestures to point or manipulate small objects. In the virtual version of our puppet, we focused on head and hand gestures. To further simplify controls of the puppet, we limited the interaction with each of the views to a simple context-driven selection. The puppeteer controls the puppet exclusively by selecting an element of the display; the puppet responds with the appropriate head and/or hand actions given the context of the display. For example, when the audience view is active, the puppeteer may click on the face of an audience member; the puppet responds by acknowledging that individual with a hand gesture and making eye contact. Likewise, clicking on a location in the map view causes the puppet to navigate (move its body) to that location. As the prototype is evaluated, the capabilities of the puppet may need to be enhanced, but this basic operation forms a foundation for exploring puppet mediated instruction For example, the puppeteer can direct the puppet interact with the environment by clicking on various objects from the puppet’s 1st person perspective. Affordances for object types can be embedded in the environment model, still minimizing the demand on the puppeteer to issue complex commands.

3 Evaluation Pending ongoing evaluation and revision of this prototype, we intend to conduct a more a formal study on the effects of this design on the puppeteers situation awareness and puppet mediated instruction impacts student presence.

References 1. Jacobson, J.: Ancient Architecture in Virtual Reality; does Visual Immersion Really aid Learning? Dissertation, School of Information Sciences. University of Pittsburgh (2008) 2. Devine, K.: Place and Time: Creating Contextualized Presence for Virtual Heritage. In: Proc. 13th Intl Conference on Virtual Systems and Multimedia, pp. 65–72 (2007) 3. Second Life, http://secondlife.com/ 4. Mateas, M.: Expressive AI. Leonardo. Journal of the International Society for Arts 34(2), 147–153 (2001) 5. Endsley, M.: Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors: The Journal of the Human Factors and Ergonomics Society 37(1), 32–64 (1995) 6. Unity Game Engine, http://unity3d.com/ 7. Puppet Theater, http://www.puppettheater.com

Immersive Video Game Based on Exercise Prescription Daegun Kim and Changhoon Park Dept. of Game Engineering, Hoseo University, 165 Sechul-ri, Baebang-myun, Asan, Chungnam 336-795, Korea [email protected], [email protected]

Abstract. It is increasingly desirable to have good health. And, regular exercise is the most effective way to improve fitness. But, most people who repeat the exercise easily feel the boredom and tend to give up. This paper propose an immersive video game based on exercise prescription. The game is designed to ensure safety and effectiveness exercise. And, exercise intensity and time can be adjusted automatically with monitoring the player’s physical status. We expect this kind of game will help people increase their interest and motivation. Keywords: Health, Video Game, Exercise Prescription, Safety, Effectiveness.

1 Introduction Advances in medicine have resulted in increase of life expectancy and it is increasingly desirable to have good health. Physical fitness is considered a measure of the body’s ability to function efficiently and effectively in work and leisure activities, to be healthy, to improve the immune system, and to prevent disease. And, regular exercise is the most effective way to improve fitness. But, most people who repeat the exercise easily feel the boredom and tend to give up[1]. This paper introduces an overview of an immersive video game based on exercise prescription which provides not only entertainment but also a program of exercise[2]. We expect this game will help people increase their interest and motivation, so exercise can be a part of their daily routine.

Fig. 1. Game concept design C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 116–119, 2011. © Springer-Verlag Berlin Heidelberg 2011

Immersive Video Game Based on Exercise Prescription

117

This game is intended to support aerobic exercise to improve heart and lung function have a strong organization, has the effect of the blood vessels. And we believe that it is important to ensure exercise safety and effectiveness. So, we apply exercise prescription to game mechanics. Then, game can adjust exercise intensity and time for a player based on heart rate.

2 Immersive Game for Exercise This game provide a modified exercise bike and wireless heart rate monitor. We have modified exercise bike as a game controller. This bike has rotation sensors connected to the handlebar and pedals. This make it possible to exercise and play a video game at the same time with pedaling and steering. Especially, the level of resistance in the pedal can be controlled by PC connected via USB. And, wireless hear rate monitor is used to estimate calorie expenditure and exercise stress. These interface for exercise for exercise prescription

Fig. 2. Exercise bike and wireless hear rate sensor

Using exercise bike, player can control a game character’s speed and direction in the track. This means that we want to use fun to encourage physical activity. This game will give exercise results along with the enjoyment of game competition. And the resistance of the pedal is adjusted dynamically depending on the slope of the track in the game. The higher slope of virtual world is, the more the pedal resistance increase. At this moment, player can experience virtually uphill and downhill while watching the game screen. Thus, the design of the track is closely related a exercise program.

3 Design and Implementation Using exercise bike, player can control a game character’s speed and direction in the track. This means that we want to use fun to encourage physical activity. This game will give exercise results along with the enjoyment of game competition. And the resistance of the pedal is adjusted dynamically depending on the slope of the track in the game. The higher slope of virtual world is, the more the pedal resistance increase. At this moment, player can experience virtually uphill and downhill while watching the game screen. Thus, the design of the track is closely related a exercise program.

118

D. Kim and C. Park

In this paper, we aim to propose an effective and safe exercise program by using game technology. To achieve this goal, the racing track is designed by applying the following prescription principals. First, it would be efficient to perform aerobic exercise at least 20 minutes per session. Because, it takes at least that long for you to start burning fat. Second, warm-up and cool-down add a few minutes to exercise routine in order to reduce risk of injury and improve athletic performance. Warming up gradually revs up your cardiovascular system, increases blood flow to your muscles and raises your body temperature. And, cooling down after your workout may help gradually reduce the temperature of your muscles and regulate blood flow. Third, to get the most health benefits from aerobic activity, we should exercise at a level strenuous enough to raise heart rate to target zone. And, target heart rate zone is 50 to 75 percent of maximum heart rate (HRmax). The HRmax is the highest heart rate an individual can safely achieve through exercise stress, and depends on age. The most accurate way of measuring HRmax is via a cardiac stress test. For general purposes, people instead typically use a formula to estimate their individual maximum heart rate. The most common formula encountered, with no indication of standard deviation, is HRmax = 220 − age.

Fig. 3. Racing track based on exercise prescription principals

This game is developed based on an object oriented framework for virtual reality applications, NAVER[3, 4]. And, we have two different versions depending on 3d rendering engine, OpenScneGraph and Gamebryo. OpenSceneGraph is an open source high performance graphic toolkit. And, Gamebryo is a cross-platform 3D graphics engine and targeted at game development. This project has been executed with 5 students for a period of 9 months.

Fig. 4. Screenshots from the game

Immersive Video Game Based on Exercise Prescription

119

Fig. 5. User test

4 Conclusion and Future Work In this paper, we proposed an immersive video game to provide a program of exercise as well as entertainment. This game aims to support exercise safety and effectiveness. To achieve this goal, we apply exercise prescription to game mechanics. Then, the game can adjust exercise intensity and time for a player based on his or her heart rate. We expect this game will help people increase their interest and motivation, so exercise can be a part of their daily routine. Acknowledgement. This research project was supported by the Sports Promotion Fund of Seoul Olympic Sports Promotion Foundation from Ministry of Culture, Sports and Tourism.

References 1. Mokka, S., Vaatanen, A., Heinila, J., Valkkynen, P.: Fitness computer game with a bodily user interface. In: Proc. Of the 2th Intl. Conference on Entertainment Computing, vol. 38 (2003) 2. Van Dan, N., Kameyama, M.: Bayesian-networks-based motion estimation for a highly-safe intelligent vehicle. In: SICE-ICASE International Joint Conference, pp. 6023–6026 (2006) 3. Park, C., Ko, H., Kim, T.: NAVER: Design and Implementation of VR Framework for Gyeongju VR Theater. Computer and Graphics 27(2), 223–236 (2003) 4. Park, C., Hirose, M., Ko, H.: A platform for interactive VR storytelling. In: Pan, Z., Aylett, R.S., Diener, H., Jin, X., Göbel, S., Li, L. (eds.) Edutainment 2006. LNCS, vol. 3942, pp. 193–202. Springer, Heidelberg (2006)

Assessing the Use of Cognitive Resources in Virtual Reality William E. Marsh1, Jonathan W. Kelly1,2, Veronica J. Dark1,2, and James H. Oliver1 1

Human Computer Interaction Graduate Program, Iowa State University, 1620 Howe Hall, Ames, Iowa, USA 2 Department of Psychology, Iowa State University, W112 Lagomarcino Hall, Ames, Iowa, USA {marsh,jonkelly,vjdark,oliver}@iastate.edu

Abstract. Due to system limitations, interactions in virtual environments are often unnatural and this may impact performance. During learning, unnatural interactions draw from a finite pool of cognitive resources, meaning that those resources cannot be used for a concurrent, possibly more important task. Because users typically have primary objectives to accomplish in the virtual world, we argue that interaction techniques and other system design choices should account for task compatibility. We use a dual-task paradigm to study resource usage during locomotion tasks varying in their similarity to real-world locomotion. In one experiment, unnatural locomotion interfaces required additional spatial resources compared to natural movements. Some participants used unique strategies unlikely in traditional dual-task studies, possibly due to the high level of immersion. Keywords: Virtual reality, dual-task paradigm, working memory, locomotion, cognitive resources, user interfaces.

1 Introduction Due to system limitations, interactions in virtual reality (VR) often require responses that are unlike those in the natural world and that are based on limited sensory feedback. These unnatural responses may affect a user’s performance at interface manipulations and attempts to interact with the system are likely to interfere with performance on other tasks as well. During the interface-learning phase, unnatural interactions draw from a pool of cognitive resources [1], meaning that those resources cannot be used for a concurrent, possibly important task. Users often have primary objectives to accomplish in the virtual world, aside from interface manipulation, and understanding how interface-learning affects performance on these objectives may enable better interface design. 1.1 Limitations of Virtual Reality Hardware limitations in VR often lead to unnatural input techniques to perform actions in the virtual world. Locomotion is one noteworthy example because a C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 120–124, 2011. © Springer-Verlag Berlin Heidelberg 2011

Assessing the Use of Cognitive Resources in Virtual Reality

121

majority of virtual environments require that the user have the ability to move about while accomplishing tasks [2]. VR systems cannot support completely natural locomotion through an infinite virtual world due to boundaries such as tracking ranges or walls. Many techniques have been proposed to address these problems and to make aspects of locomotion more natural. Examples include treadmills [3], giant “hamster balls” [4], and methods of tricking the user with modified sensory feedback [5]. System limitations affect not only the input techniques, but also the feedback provided to the user by the system. Systems often have low-resolution graphics and a reduced field-of-view, for example. We are interested in understanding what the actual impact of using these systems is on concurrent tasks performed in the environment. 1.2 Dual-Task Paradigm Psychology researchers have distinguished between different pools of cognitive resources [6]. Most multi-component models of memory distinguish between verbal and visuo-spatial resources, and some research supports further partitioning the visuospatial pool into separate visual and spatial components. Verbal resources are needed by tasks requiring storage or manipulation of verbal items, while visuo-spatial resources are used for tasks involving storage or manipulation of visual or spatial information. Baddeley's [7] often cited model of working memory makes the distinction between verbal and visuo-spatial resources and further describes a “central executive” that serves to direct attention, mediating access to the other two components. A dual-task selective-interference paradigm is used to determine which set of resources a given task requires. In this paradigm, a task of interest is administered concurrently with other simultaneous tasks of known resource demands. If there is selective interference, above a general dual-task deficit, between the task of interest and a simultaneous task, it can be concluded that the task of interest requires some of the same resources as the simultaneous task. A commonly used simultaneous cognitive task is the span task in which participants are asked to remember a span of items. The nature of the items, either verbal (numbers, letters, or words) or spatial (locations), determines the type of resource that is used. For example, if a concurrent spatial span task disrupts performance at a task with unknown cognitive requirements more than a verbal span task does, then the unknown task is presumed to require spatial resources [7].

2 Using the Dual-Task Paradigm in Virtual Reality Due to the limitations of VR, interactions are likely to compete for resources from the pools described above. The dual-task paradigm has not been widely used in VR, but it can be applied to examine the cognitive resources required for different types of VR interfaces. Cognitive resource usage in VR is interesting for several reasons: • When interacting with a VR system, a user typically has other more important, resource-demanding tasks to complete in the environment than just using the

122

W.E. Marsh et al.

interface. VR is often used in domains with high cognitive demands, such as teleoperation of unmanned vehicles [8]. • Locomotion through virtual environments provides a particular challenge beyond what might normally be encountered in desktop computer applications. These environments are often large or even infinite and it is impossible to provide a fully natural interface within the physical bounds of the VR hardware. • The immersive nature of VR means that system designers have a high degree of control over all aspects of a user’s experience, including cognitive task demands.

Fig. 1. Examples of verbal and spatial memory task flow

We conducted a study applying the dual-task selective-interference paradigm to investigate cognitive resource usage during virtual locomotion tasks using interfaces varying in naturalness. Each participant was assigned one of three interfaces (between subject): gamepad, body-based, and real walking (in increasing order of assumed naturalness). Each participant completed basic movement tasks (sidestep, walk forward, rotate, duck) concurrently with different working memory conditions (within subject): spatial span, verbal span, and no task. In these scenarios, a user was presented with memory items (see Fig. 1), asked to complete movements, and then tested on the memory items. The verbal span task required a participant to remember a sequence of numbers and the spatial span task required memory of a sequence of boxes presented in different spatial locations within a grid. We predicted that the spatial task would interfere with locomotion the most and that the verbal task would interfere very little in comparison to the no-task condition. We customized the span task difficulty to each participant’s individual ability level by assigning sequences of five items if the participant could successfully complete two practice tasks at the fiveitem level. Otherwise the difficulty was set to four items. Although performance is often slower in a dual-task situation (a general dual-task deficit), results on many measures showed faster performance when participants had a concurrent task than when they did not. Many participants indicated during the postexperiment interview that they found it difficult to perform the movement tasks while simultaneously remembering a verbal or spatial sequence. We believe that users hurried to complete the movements in order to get back to devoting all of their resources to the working memory tasks. Unfortunately, because of the nature of the movements used, we had no measure of movement accuracy to assess the possibility of a speed-accuracy tradeoff. Memory task results showed marginal significance of interface group and of the interaction between group and memory task. As predicted, the locomotion tasks interfered more with the spatial span tasks than the verbal tasks. Spatial memory items

Assessing the Use of Cognitive Resources in Virtual Reality

123

were forgotten more than verbal items and stopping performance in the body-based group was worse when combined with a spatial task than a verbal task. The post-experiment interviews also revealed other relevant aspects of interacting with the VR interface. Users reported memory problems caused by being startled by the ducking task, in which an I-beam came floating overhead. A few users in the bodybased and gamepad groups believed that rotation tasks might have interfered with the memory tasks. Finally, there is the possibility that participants may have adopted strategies to remember the spatial span task that were specific to full-scale VR. For example, one user reported that in order to accomplish the spatial span task, she imagined placing her hands and feet on the squares, as in the game Twister. We note that the imagined motor actions may have interfered with the locomotion movements.

3 Conclusions and Future Work The experiment described above illustrates the kind of information that can be obtained using the dual-task selective interference paradigm to assess cognitive issues relating to VR. While this study focused on an input technique, the limited perceptual fidelity provided by VR systems may also lead to cognitively demanding strategies. We plan next to investigate the cognitive implications of restricting a user’s field-ofview. The design will be similar to the one described above except that the betweensubjects variable will be field-of-view (wide, narrow) instead of locomotion interface. If we understand the types of task conflicts that may exist in a given domain, we can strategically make system design choices and possibly even create systems that adapt according to the user’s current task load. The unexpected result that participants performed some locomotion tasks better when there was a memory task than with no task should be further explored. While we were able to measure accuracy in the span tasks, the locomotion tasks that we used involved only small, fast, direct movements that did not allow us to get a good measure of accuracy. In future research we need tasks that also allow us to measure accuracy, so that we can determine if there might be a speed/accuracy tradeoff, whereby improved performance on the memory task comes at a cost to the locomotion task. We also would like to ensure that users understand that completing movements faster will not get them to the memory recall faster. Finally, several users believed that they were capable of more difficult memory tasks and generally performance was high, so in future studies we may increase the task difficulty to more fully tax users. Acknowledgments. This research was funded by a grant from the Air Force Research Laboratory.

References 1. Anderson, J.R.: Acquisition of cognitive skill. Psychological Review 89(4), 369–406 (1982) 2. Bowman, D.A., Kruijff, E., LaViola, J.J., Poupyrev, I.: 3D User Interfaces: Theory and Practice. Addison-Wesley Professional, Boston (2005)

124

W.E. Marsh et al.

3. Darken, R.P., Cockayne, W.R., Carmein, D.: The Omni-Directional Treadmill: A Locomotion Device for Virtual Worlds. In: 10th Annual ACM Symposium on User Software and Technology, pp. 213–221. ACM, New York (1997) 4. Medina, E., Fruland, R., Weghorst, S.: Virtusphere: Walking in a human size VR ”hamster ball”. In: Human Factors and Ergonomics Society 52nd Annual Meeting, pp. 2102–2106. Human Factors and Ergonomics Society, Santa Monica, CA (2008) 5. Razzaque, S., Kohn, Z., Whitton, M.C.: Redirected walking. In: Eurographics, pp. 289–294. European Association for Computer Graphics, Geneve, Switzerland (2001) 6. Navon, D.: Resources - A theoretical soup stone? Psychological Review 91(2), 216–234 (1984) 7. Baddeley, A.: Is working memory working? The fifteenth Bartlett lecture. The Quarterly Journal of Experimental Psychology: Section A 44(1), 1–31 (1992) 8. Foo, J.L., Knutzon, J., Kalivarapu, V., Oliver, J.H., Winer, E.: Three-Dimensional Path Planning of UAVs in a Virtual Battlespace using B-Splines and Particle Swarm Optimization. Journal of Aerospace Computing, Information, and Communication 6, 271– 290 (2009)

Augmented Reality Approach to Domestic Maintenance Tasks Jorge Martín-Gutiérrez and Irene Inés Santos Pérez Universidad de La Laguna, Dpto. Expresión Gráfica en Arquitectura e Ingeniería, Avda. Astrofísico Francisco Sánchez s/n 38206 La Laguna, Spain [email protected], [email protected]

Abstract. Augmented reality technology is becoming more familiar and utilized at user level thanks to new generation of smartphones provided of webcam, big format tactile screens and internet connection. This allows creating applications which make certain human tasks nicer and more comfortable. AR technology has been applied to mechanical engineering field, concretely for assistance on repairing and maintenance tasks of complex and specific systems using expensive systems which are not very useful nor accessible for performing simple or domestic tasks. In this work we intend making this technology approachable for applications and tasks which are more common to everybody. For example, two AR technology based tools, one belonging to a smartphone platform and another using a head mounted display (HMD) glasses connected to a PC. Both allow helping user while performing installation, maintenance and adjustment of a mountain bike brakes system.

1 Introduction In next decade we will access real world through augmented reality; this process will reach global audience in 2021 according to the “Realidad Aumentada: una nueva lente para ver el mundo”- report [1]. Augmented reality (AR) is a technology which allows real-time superimposition of virtually generated images and information over real world images meaning adding a synthetic virtual part to the real one. AR is a technology helping enrichment of our perception of reality making possible real world information is complemented by digital one. AR experiences have become easy to use and especially mobile. Progress made in mobile devices, particularly smartphones with cameras but also in different technologies which combine real world with virtual information allowing the chance to enjoy this applications making possible AR accessing broad consumer sector through actual devices such as smartphones, game consoles and computers with webcams. Gartner1 consultant states AR will achieve broad public reach gradually in a 5 to 10 years horizon. Although AR has already begun to introduce in the market, its use is not quite extended yet but a great growth is expected within the next few years thanks 1

http://www.gartner.com/it/page.jsp?id=1447613

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 125–129, 2011. © Springer-Verlag Berlin Heidelberg 2011

126

J. Martín-Gutiérrez and I.I. Santos Pérez

to mobile devices (especially smartphones). Trend in virtual reality systems is becoming more transportable, comfortable and user friendly each time. Actually, computers and smartphones are both devices chosen although is expected that development of display technologies will allow creation of glasses or lens allowing showing virtual information right in front of human eye.

2 Augmented Reality in Maintenance Tasks Industrials There are few contributions where augmented reality technology is applied to machines’ mounting and maintenance and most of them are developed through research work. In less accessible fields like aerospace, applications have been developed for supporting maintenance staff performing their tasks [2, 3, 4]. In the same application field Schwald [5], introduces an AR system for training & assistance in maintenance of complex industrial equipment using an optical see-through head mounted display. ARMAR project, has developed, designed, implemented and tested on users a beta version of an AR application for support of army mechanics during routine maintenance tasks inside an armored vehicle turret [6, 7]. Actually, mechanics from the army and manufacturers like Boeing use AR glasses when staff works on vehicles, glasses show repairs step by step, target necessary tools including textual instructions. This kind of experience supports learning as well training of specific tasks. I+D department of automotive giant BMW2 (through BMW research project), include among their working lines development of an AR application supporting mechanics while performing maintenance, diagnose and repair of any fault. A recently published study has also applied AR technology for maintaining photovoltaic solar installations equipment [8]. According to these references, it could be thought that use of AR technology in repair and maintenance fields is subject to complex systems although that’s not the case because it’s actually available to anyone. A few years ago, AR apps demanded specialized equipment which wasn’t portable. Nowadays applications for laptops and smartphones merge digital information into the real world quick and easily. Design of easy apps makes communication between people and computers more natural each time allowing those who are not used to computers can interact with the system. So, the use of augmented reality may be extrapolated to nearer and common environments for everybody.

3 Project Target In this work we have developed an application based on AR use for support, repair and maintenance of a standard mountain bike brakes system. Also, it focuses on making the AR system useful as training for all bicycle users targeting substitution of the troublesome maintenance manuals and assembly instructions. Augmented manual (AR_Brakes-V manual) is developed to visualize in two platforms: smartphone or computer (through HMD). The first of both has been developed for being accessible 2

http://www.bmw.com/com/en/owners/service/augmented_reality_workshop_1.html

Augmented Reality Approach to Domestic Maintenance Tasks

127

to everybody meanwhile the second is the most suitable platform from machines requiring more complex maintenance tasks. It’s intended performing a usability study in the very near future for comparing both platforms. This work is a pilot study, which may be extensible to all electronic devices, vehicles, home appliances, etc, as they are all provided of user manuals for configuration or installation and usually understanding their instructions becomes a tough task. The possibility of creating those manuals in augmented reality as assistance during training, configuration and installation of those equipment could make learning how to use them much easier.

4 Generic AR-Manual for Devices Project consisted in developing an augmented manual with instructions of steps for installing the brake lever, V-brakes and replacement of the cartridge shoe providing also assistance for adjusting the cable’s tension. Augmented manual is presented in spiral binding so it can be leant over the mountain bike’s handlebar. Each page has a marker that belongs in AR to every task that must be accomplished. The application requires accurate position and orientation tracking in order to register virtual elements in the real world and so we have used a marker-based method. Therefore, the system requires a webcam for capturing the real world. The captured image recognizes virtual objects on the visible markers. Program used for developing scenes in PC has been Build_AR [9] while AndAR [10] software was used for creating mobile devices scenes both being GNU software – General Public license). 3D modeling of the parts manipulated during tasks was performed using Autodesk Inventor 3D software. Besides, textual information is added about tools used in every step. 3D graphics are animated so user can understand gestures that should be carried out. User provided with a mobile phone including camera will lean the augmented book over the bike’s handlebar where an included adapter should be set. Through the phone’s screen he can see virtual parts superimposed over the real ones (displaced respecting marker) (see Fig.1). Besides seeing animated components on the real environment, user will also be able to listen to a recorded voice which will guide him through every task following instructions accordingly.

Fig. 1. Augmented scenes with HMD and PC platform

Fig. 2. Augmented Smartphone

scenes

with

128

J. Martín-Gutiérrez and I.I. Santos Pérez

Most accessible and common device suggested is mobile phone although an AR glasses with earphones included could also be used (Fig.2). Traditional user manual will be substituted by several markers and AR application would be available for download from the product’s website.

5 Conclusion This study introduces a simple augmented reality system seeking incorporation of this technology to home appliances and usual vehicles for providing help in their configuration or maintenance. An application is introduced allowing user visualizing in a smartphone the augmented scenes for performing installation, substitution and maintenance tasks on a mountain bike brakes system. The AR system can substantially improve quality of maintenance services while saving costs as the same AR system can be used for different versions of several products (for example, Vbrakes is standard for many bikes models). This kind of apps can be extended for setup and installation of electrical appliances (TV recorder and programmer, Hi-fi systems, home cinemas, game consoles, etc.). We propose substitution of the usual user manual for setup and maintenance of home appliances or devices for AR based portable applications. User may access product’s website from his mobile device (iPhone, smartphone) for download and installation of the AR application. As a future work we propose performing a usability study for knowing the efficacy, efficiency and satisfaction of created AR apps drawing conclusions about viability of introducing these apps for user’s performance of common tasks. Acknowledgements. The Spanish Ministry of Innovation and Science, through the “Not oriented Fundamental Investigation Projects” (“Improvement for spatial reasoning and visualization through technologically developed tools” Project ref. TIN2010-21296-C02-02) partially supported this work.

References 1. Ariel, Fundación Telefónica: Realidad Aumentada: Una nueva lente para ver el mundo, Planeta edn. Madrid, Spain (2011) 2. Rahardja, S., Wu, E., Thalmann, D., Huang, Z.: Visualization of intelligent maintenance systems through mixed reality. In: Proceedings of the 7th International Conference on Virtual Reality Continuum and its Applications in Industry, VRCAI 2008 (2008) 3. Haritos, T., Macchiarella, N.D.: Augmented Reality (AR) for Aircraft Maintenance Technician’s Training. In: Proceedings of Society of Applied Learning Technologies: New Technologies (2007) 4. Macchiarella, N.D.: Augmenting Reality as a Medium for Job Task Training. Journal of Instruction Delivery Systems 19(1), 21–24 (2005) 5. Schwald, B., Laval, B.: An augmented reality system for training and assistance to maintenance in the industrial context. In: 11th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, pp. 425–432 (2003)

Augmented Reality Approach to Domestic Maintenance Tasks

129

6. Henderson, S., Feiner, S.: Augmented Reality for Maintenance and Repair (ARMAR). Technical Report AFRL-RH-WP-TR-2007-0112, United States Air Force Research Lab (2007) 7. Henderson, S., Feiner, S.: Evaluating the Benefits of Augmented Reality Documentation for Maintenance and Repair. IEEE Transactions on Visualization and Computer Graphics (2011) 8. Benbelkacem, S., Zenati-Henda, N., Belhocine, M., Bellarbi, A., Tadjine, M.: Augmented Reality Platform for Solar Systems Maintenance Assistance. In: Proceeding of the International Symposium on Environment Friendly Energies in Electrical Applications. In: EFEEA 2010 (2010) 9. Build_AR. Human Interface Technology Laboratory New Zealand (HIT Lab NZ) (2010.03.07), http://www.buildar.co.nz/ 10. AndAR project. Android Augmented Reality (2010.03.07), http://code.google.com/p/andar/

Development of AR Display System for Dental Surgical Simulator Katsuhiko Onishi, Shota Ito, Yusuke Kawamura, and Hiroshi Noborio Osaka Electro-Communication University, 1130-70 Kiyotaki, Shijonawate, Osaka, Japan [email protected]

Abstract. In this paper, we describe about a display system for dental surgical simulator. The design concept of our system is that the system allows the user to learning the dental surgical methods with the real hand and body posture. Therefore our system set a display, which shows virtual teeth model and real teeth and gums, close to user’s hand position and allows the user to manipulating object directly. Keywords: dental surgical simulator, direct manipulation, augmented reality.

1 Introduction In dental surgical region, the most of surgical training methods are used plastic teeth or live patients. These methods are good for training surgical skills, for example the usage of surgical tools, the surgical procedure and so on. But it is difficult to execute tasks many times because it needs the new plastic teeth at each task or live patients. Therefore many kinds of dental surgical simulator have been proposed [1], [2], [3], [4]. They allow users to learning dental surgical methods by using the unique interface. But almost of them has just only typical computing display system. The user cannot have an experience about real hand position and the body posture in the dental surgery. In this paper, we describe about a display system for dental surgical simulator. The system allows the user to learning the dental surgical methods with the real hand and body posture. Our system set a display close to user’s hand position and allows the user to operate surgical tasks directly. And it shows combined image with virtual teeth model as a surgical target and a real teeth model as other parts of the patient dental model. For adapting any view images of user’s head position, the system measures the head position and a real teeth model position by using 3DOF magnetic sensor.

2 AR Display System In dental surgery, the surgical environment is almost common that a patient lay down in front of a dentist and the dentist operates surgical tasks to him/her. In such situation, the dentist hands position and body posture is different from that of operating traditional dental surgical simulation. And in the dental surgical procedure, C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 130–133, 2011. © Springer-Verlag Berlin Heidelberg 2011

Development of AR Display System for Dental Surgical Simulator

131

it is needed to hold the surgical tools and restrict the motion for accuracy tasks, for example, drilling teeth and so on. In such situation, the dentist usually put his/her hand on the other teeth as restricting the hand motion for detailed operation. To achieve the manipulation like dental surgical operation with such kind of user’s hands position and body posture, our system set the half mirror horizontally between head position and hands position as shown in Figure 1. And the system shows combined image with virtual teeth and teeth model to user. Furthermore to adapt user’s head motion for probing and confirming surgical target, it measures user’s head motion and teeth model. And it shows the virtual teeth on LCD display with adequate view volume from the user’s head position. And it needs accuracy for operating tasks to such kind of small teeth that a dentist has to use a surgical tool with hold it on a parts of around target teeth. To achieve such kind of operating environment, our system uses real teeth model around surgical target teeth and the user is able to hold the hands on it while they execute surgical tasks and they enable to operate surgical simulation tasks precisely. User with head tracker LCD display Virtual teeth Half mirror Combined image

Surgical tool Teeth model

Fig. 1. System image

3 Prototype Display System The overview of our prototype display system is showed Figure 2(a). The application is implemented on Windows PC with OpenGL. And it measures the user’s head position and the teeth model position by using 3DOF magnetic sensor. Our system shows users a teeth model and virtual teeth projected on display by reflecting on the half mirror. Figure 2(b) and Figure 2(c) are examples of view images that the user is able to see on the half mirror. It measures a user’s view position and show the adequate view volume from the position. The system allows the user to display virtual and real teeth model simultaneously from any user’s view positions.

132

K. Onishi et al.

(a) System overview

(b) User’s view 1

(c) User’s view 2

Fig. 2. Prototype display system

4 Conclusion and Future Work In this paper, we proposed a display system of dental surgical simulator. Our system enables users to show the combined image of virtual teeth and teeth model. In our display methods, user can see the combined image from any point of view by tracking their head position and model position. We made prototype system and confirmed of our method. As future work, we are planning to implement our method to one of the other dental surgical simulator which can generate realistic virtual teeth and surgical tools and the user can manipulate it by using the haptic interface.

Development of AR Display System for Dental Surgical Simulator

133

Acknowledgments. This is supported in part by 2010 Grants-in-aid for Scientific Research(No.22360109) from the Ministry of Education, Culture, Sports, Science and Technology, Japan.

References 1. Noborio, H., Kawamoto, Y.: Digital Collision Checking and Scraping Tooth by Dental Bar. In: The 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 227–234 (2010) 2. Rhienmora, P., Gajananan, K., Haddawy, P., Suebnukarn, S., Dailey, M., Supataratarn, E., Shrestha, P.: Haptic augmented reality dental trainer with automatic performance assessment. In: The 15th international conference on Intelligent user interfaces (IUI 2010), pp. 425–426 (2010) 3. Kim, L., ParkAn, S.: Efficient Virtual Teeth Modeling for Dental Training System. International Journal of CAD/CAM 8(1), 41–44 (2008) 4. Yau, H., Tsou, L., Tsai, M.: Octree-based virtual dental training system with a haptic device. Computer-Aided Design & Applications 3, 415–424 (2006)

Earthquake Disaster Prevention Support Tool －Visualization of Prevention Effectiveness by Utilizing Augmented Reality Kyoko Yoshida1, Masahiro Urabe1, Hayato Tsuchiya1, Yasuki Iizuka2, and Kayo Iizuka1 1

School of Network and Information, Senshu University, Kanagawa, Japan {k-yoshida,iizuka}@isc.senshu-u.ac.jp 2 School of Science, Tokai University, Kanagawa, Japan [email protected]

Abstract. Japan has faced major problems with large-scale earthquake countermeasures due to its location in an earthquake zone. The Building Standards Act of Japan has been amended since previous major earthquake damage and new buildings have been built with a safer design than ever before. However, measures against falling or overturning of furniture, which could injure people inside buildings, are not sufficient. In this paper, the authors propose an earthquake disaster prevention support tool with which one can experience the shaking of furniture with or without earthquake disaster countermeasures using a 3DCG animation function of Augmented Reality (AR) and three applications of this tool. Keywords: earthquake disaster prevention, Augmented Reality (AR), disaster simulated experience.

1 Introduction Japan has faced major problems with large-scale earthquake countermeasures due to its location in an earthquake zone. The probability of the metropolitan area of Japan being struck by an inland earthquake within thirty years is 70% and the number of dead and injured people when it occurs is estimated to be about 250,000 at the maximum [1] [2]. The Building Standards Act of Japan has been amended since previous major earthquake damage and new buildings have been built with a safer design than ever before. However, measures against falling or overturning of furniture, which could injure people inside buildings, are not sufficient. In the Great HanshinAwaji Earthquake in 1995, furniture fell over and scattered in about 60% of the houses that were not fully or partially destroyed. Nearly half of the indoor injuries were due to falling or overturning of furniture [3] [4]. To solve these problems, we developed an earthquake disaster prevention support tool (AR Earthquake Disaster Prevention kit), with which one can experience the shaking of furniture with or without earthquake disaster countermeasures using a 3DCG animation function of Augmented Reality (AR). In this paper we also propose three applications of this tool. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 134–138, 2011. © Springer-Verlag Berlin Heidelberg 2011

Earthquake Disaster Prevention Support Tool

135

2 Background It is considered that preventive measures against the predicted inland earthquake in the metropolitan area would minimize damages. In the questionnaire survey that we conducted targeting the students of Senshu University, however, it was found that many students took no earthquake countermeasures because they thought that such earthquake countermeasures would cost a lot or they did not know about such countermeasures. One of the reasons for the slow progress of earthquake disaster countermeasures is that people do not know the cost of the countermeasures. In consideration of this current situation, we propose in this paper the AR Earthquake Disaster Prevention kit to visualize the cost effectiveness of earthquake disaster countermeasures using AR. AR is the technology that shows pictures of an actual environment (or part of it) taken with a Web camera and composites additional virtual information with the pictures.

3 AR Earthquake Disaster Prevention Kit The AR Earthquake Disaster Prevention kit shows a 3DCG animation of an estimated disaster caused by an earthquake of Marker differing scales, using the basic tool of the AR Furniture Card (Figure 1). The AR Furniture Card Image of contains a marker that shows furniture Name of furniture furniture, expected seismic Countermeasure: intensity scale, description about Taken/Not taken whether an earthquake disaster countermeasure is taken or not (two cards are prepared for the Expected seismic intensity same conditions with and without countermeasures being taken), and name and price of the products Product name and price of earthquake used for the countermeasure (if the countermeasure card is for furniture with the countermeasure being taken). The marker is read by the Web camera Fig. 1. AR Furniture Card and a 3DCG animation of furniture, such as bookshelf and desk, is shown on a PC monitor to simulate the shaking of the furniture during an earthquake of the expected seismic intensity. The method of shaking of the furniture, which depends on the seismic intensity scale and the earthquake disaster countermeasures, is simulated according to existing research results, movies and other data in order to reproduce actual shaking during an earthquake of the expected scale [5]. In the following three applications of the AR Furniture Card are presented.

136

K. Yoshida et al.

3.1 AR Earthquake Disaster Countermeasure Effect The AR Earthquake Disaster Countermeasure Effect reads two AR Furniture Cards: One is for furniture with the earthquake disaster countermeasure taken and the other is for the same furniture without the countermeasure. The AR Earthquake Disaster Countermeasure Effect realizes the shaking of furniture during an earthquake with or without the earthquake countermeasure being taken. With this tool one can see how the earthquake disaster countermeasure makes a difference when the furniture falls over during an earthquake. It visualizes in a 3DCG animation how furniture shakes and objects inside the furniture fall out (Figure 2). On the AR Furniture Card of the furniture for which an earthquake countermeasure is taken, the name and price of the items used for the countermeasure are described, which enables us to see immediately the effect and cost of the earthquake disaster countermeasure. With this tool one can recognize the danger that arises if no earthquake disaster countermeasure is taken. This tool thus aims to promote the countermeasure to be taken to prevent furniture from falling over. This tool is used Fig. 2. Earthquake Disaster Countermeasure Effect mostly on desks. 3.2 AR Earthquake Disaster Floor Layout The AR Earthquake Disaster Floor Layout is a tool by which one can place the AR Furniture Cards of bookshelf, bed, table, etc. on a room planning drawing, similar to considering one’s own room layout, to simulate the shaking of furniture during an earthquake(Figure 3). With this one can see dangerous and safe areas inside the room during an earthquake and design the furniture layout of each room in consideration of safety during an earthquake disaster. This tool can also be used for the layout design for office rooms of companies. The tool is used mostly on desks. 3.3 AR Earthquake Disaster Simulated Experience With the AR Earthquake Disaster Simulated Experience, one can place AR Furniture Cards in a room and see in full scale on a large screen what the room looks like during an earthquake (Figure 4). With the AR Earthquake Disaster Simulated Experience, one can simulate furniture for which no earthquake countermeasure is taken and virtually experience in full scale the situation whereby the furniture shakes and falls over during an earthquake of JMA seismic intensity scale 7 or lower. One can thus acknowledge the importance of earthquake disaster countermeasures with

Earthquake Disaster Prevention Support Tool

Fig. 3. AR Earthquake Disaster Floor Layout (Left)

137

Fig. 4. AR Earthquake Disaster Simulated Experience (Right)

this tool. This function is realized by creating 3DCG objects on the same scale as actual furniture. This tool is expected to be used in a 3D space, e.g. private room.

4 Evaluation A questionnaire survey was conducted targeting 85 students of Senshu University to measure the change in their consciousness between before and after their experience of the AR Earthquake Disaster Countermeasure Effect. The result showed that the number of students who had concern in earthquake countermeasures increased by 11.8%, the number of students who were more scared of earthquakes increased (with the number of students who were not scared of earthquakes decreased by 7.1%), and the number of students who did not take earthquake countermeasures but want to take measures increased by 3.7% (Table 1). We therefore consider that the AR Earthquake Disaster Countermeasure Effect raises people’s awareness of disaster and their interest in earthquake disaster countermeasures. Table 1. Result of questionnaire survey Are you concerned in earthquake countermeasures?

Yes Neither yes or no. No

Are you scared of earthquakes?

Before(%) 32(37.6) 26(30.6)

After(%) 42(49.4) 21(24.7)

Before(%) 45(52.9) 7(8.2)

After(%) 47(55.3) 11(12.9)

27(31.8)

22(25.9)

33(38.8)

27(31.8)

Do you want to implement an earthquake countermeasure? (Question to those who do not have earthquake countermeasure.) Before(%) After(%) 22(40.7) 24(44.4) 14(25.9) 14(25.9) 18(33.3)

16(29.6)

We received at the exhibition comments such as “I became aware of earthquake disaster in visible ways,” “I think that knowing about earthquake disaster countermeasures is not enough. The countermeasures would be more promoted if sales companies of countermeasure products against earthquake disasters utilize them.”

138

K. Yoshida et al.

5 Conclusion In this paper we proposed the AR Earthquake Disaster Prevention kit which utilizes AR technology and showed the evaluation result. Future problems include resolution enhancement of the model to improve the reality sensation, increase of diversity of earthquake countermeasure products for furniture, application of the special effect to objects other than furniture by combining specific models, and translating the model scenarios into reality.

References 1. An estimation of ground motion map of Japan, http://www.jishin.go.jp/main/chousa/09_yosokuchizu/index.htm 2. A report on the anticipated damage caused by Tokyo Metropolitan Earthquakes, The committees for technical investigation on countermeasures for Tokyo Metropolitan Earthquakes, the Central Disaster Management Council, Cabinet Office, Government of Japan (2005), http://www.bousai.go.jp/jishin/chubou/taisaku_syuto/pdf/ higaisoutei/gaiyou.pdf 3. A report on investigation of house damage in the Great Hanshin-Awaji Earthquake, Architectural Institute of Japan (1996), http://www.fdma.go.jp/html/life/kagu9.html 4. Promotion of Measures to Secure Furniture, Tokyo Fire Department (2004), http://www.tfd.metro.tokyo.jp/inf/h16/i024_02.htm 5. Fukuwa Laboratory, Nagoya University, http://www.sharaku.nuac.nagoyau.ac.jp/laboFT/fall_furniture/gaiyou.html

Part III

Gestures, Gaze and Multimodality in HCI

A Three-Dimensional Fingertip Interface Yangkeun Ahn1,2, Kwangmo Jung2, and Jiman Hong1 1

2

Soongsil University, Seoul, South Korea Korea Electronics Technology Institute, Seoul, South Korea [email protected]

Abstract. This paper proposes a method to recognize the fingertip threedimensionally using infrared stereo cameras. The proposed method can be used for human-computer interactions with a three-dimensional display and is designed to provide robust performance against finger trembling, kinematic errors, and sensor noise. This paper describes the proposed method in detail and also presents implementation results. Keywords: Fingertip Recognition, Stereo Camera, Spatial Touch.

1 Introduction This paper proposes a three-dimensional interface using a fingertip with no separate equipment to ensure interactions between the three-dimensional image projected onto the space and the user. The image projection system discussed here is a device that projects a two-dimensional image onto the concave mirror, resulting in binocular parallax and enabling the user to gain a sense of visual depth. This system provides the user with an immersive interface. The use of the existing touch screen technique to ensure interaction between the visualized image and the user entails the following problems: First, the input device is exposed to the user when a physical touch screen is installed, thus undermining the user’s sense of immersion. Second, the touch screen equipment should recognize the depth of the finger, as the image perceived by the user is not two-dimensional, but conventional touch screens are incapable of recognizing depth. In order to provide the user with an immersive interface, therefore, an optical method should be chosen such that the existence of the touch screen is unknown to the user. Also, the location of the user’s finger should be identified in a threedimensional space so as to facilitate interaction with the user. This paper proposes a method using two infrared cameras to identify the location of the finger in threedimensional space (see Fig. 1). In order to implement the proposed method, this paper discusses detection of the location of the fingertip within images, alignment of the fingertip points of two images to calculate the three-dimensional location, and the design of a linear Kalman filter to deal with image noises.

2 Methodology and Test Details The method proposed in this paper can be summarized as follows: First, infrared light is projected onto the finger using the infrared emitter. Second, the images of the C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 141–145, 2011. © Springer-Verlag Berlin Heidelberg 2011

142

Y. Ahn, K. Jung, and J. Hong

reflected finger are recorded with infrared stereo cameras. Lastly, the pixel coordinates of the finger extracted from the two images and the cameras’ intrinsic parameters are used to calculate the three-dimensional location of the finger. The overall process is described in Fig. 2. This section introduces details of each method.

Stereo Camera

Screen

Infrared LED Array

Stereo

Camer a Jig

Hand

Fig. 1. Conceptual diagram of proposed device and configuration of testing equipment Input Image

+

-

Source Image

Thresholding

Background Image

Input Image

+

+

-

Source Image

Thresholding

Background Image

Distance Disparity Calculation

Stereo Matching

Fig. 2. Conceptual diagram of processing process

2.1 Configuration of Testing Equipment The testing equipment used in this paper is shown in Fig. 1. The infrared stereo cameras are used here to minimize errors caused by the subject(s) other than the finger. The impact of noise can be reduced, as infrared wavelengths alone are accepted from near distance. Two Chameleon (a model from Point Grey) cameras are

A Three-Dimensional Fingertip Interface

143

installed on the stereo jig, as shown in Fig. 1. An infrared radiation device is used to project infrared light onto the finger and extract the finger’s image. 2.2 Extraction of Fingertip Image This section explains how to extract the image of the fingertip from the fed images. This method consists of hand area extraction, binarization, fingertip area extraction, and Kalman filter-based stabilization. When the system operates, the background image of the environment, where the equipment is located, is obtained by showing the hand in the image. The obtained background image is used in the difference operation with a subsequently entered image to remove the background from the latter image, and the resulting image consequently contains the hand area reflected by infrared light only. The extracted image of the hand is a gray-scale image with each pixel having a value from 0 through 255. Binarization is carried out to clearly distinguish the finger area from the remaining area. 2.3 Extraction of Fingertip Area To identify the coordinates of the fingertip from the binarized hand area, a filter bank is created based on human observations. The filter bank is a method to connect filters with different conditions in series and obtain desired results in each phase to ultimately gain the fingertip’s image. This method breaks the geometrical forms of the fingertip into three phases in total, producing powerful results from a combination of simple rules. The geometrical characteristics of the fingertip can be summarized as follows [1]: − The finger area remains white within the binarized image; − The whole of the small circle S is included in the white area in Fig. 3; and − The large circle L cuts the fingertip area in two points only (i.e., points a and b) in Fig. 3. The method used here applies the aforementioned three filters consecutively to take the ultimately remaining area as the fingertip area candidate. The algorithm for this method can be expressed as follows:

Fig. 3. Geometric analysis of fingertip and optic axes of two non-transversal cameras

144

Y. Ahn, K. Jung, and J. Hong

For each pixel (x,y): For each filter: If i-th rule at pixel (x,y) is false Skip pixel; Else Mark (x,y) as fingertip End if; End for; End for; Return fingertip candidates; 2.4 Calculation of Three-Dimensional Location of Fingertip This paper assumes that the cameras are calibrated in advance. The intrinsic parameter matrices of the two cameras, obtained through calibration, are denoted as here; all locations are specified with the left camera as the basis. Fig. 3 describes how the optic axes of the two cameras do not exactly intersect with the fingertip points extracted from the stereo vision. Thus, the minimum distance crossing the two rays orthogonally should be derived [2]. The extrinsic parameters of the calibrated stereo cameras are referred to as , which represent the left rotating matrix (=I), the right camera’s rotating matrix in relation to the left side, the location of the left camera (=[0,0,0]), and the right camera’s location with the left camera as the basis, respectively. The new parameters of the stereo vision can be written as below: (1) To identify the three-dimensional location of the finger using new parameters, : the following linear system should be solved to derive the parameters

, (2)

To solve the above equation, the matrix linear system:

is derived as given below to set up the (3)

, can be derived from The parameter set, fingertip can be determined on this basis using .

, and the location of the

2.5 Test Results The system is used to test the user’s inputs. When the user enters rectangular, triangular, and circular movements, the results also resemble those movements. Performing computation 20 times per second, the system can be reliably used as an interface.

A Three-Dimensional Fingertip Interface

145

Fig. 4. Test results

3 Conclusion This paper has proposed an infrared stereo camera-based method for threedimensional fingertip recognition. Using the proposed method, we identified the location of the fingertip with the series filter bank, stabilized it using the Kalman filter, and obtained the three-dimensional coordinates of the fingertip with the left camera as the basis. To assess the level of accuracy in the future, it will be essential to videotape already known movements and compare the two results. Also, research is needed on a correction algorithm designed for 1:1 correspondence with the designated image.

References 1. Letessier, J., Berard, F.: Visual Tracking of Bare Fingers for Interactive Surfaces. In: UIST 2004, ACM, New York (2004) 2. Trucco, E., Verry, A.: Introductory Techniques for 3-D Computer Vision. Prentice Hall, Englewood Cliffs (1998-2006)

Rule Based Trajectory Segmentation Applied to an HMM-Based Isolated Hand Gesture Recognizer Jounghoon Beh1, David Han2, and Hanseok Ko1,3 1

Institute for Advanced Computer Studies, University of Maryland, College Park, USA {jhbeh,hsko}@umiacs.umd.edu 2 Office of Naval Research, USA [email protected] 3 School of EE, Korea University, Seoul, Korea [email protected]

Abstract. In this paper, we propose a simple but effective method of modeling hand drawn gestures based on their angles and curvature of the trajectories. Each gesture trajectory is composed of a unique series of straight and curved segments. In our Hidden Markov Model (HMM) implementation, these gestures are modeled as connected series of states analogous to series of phonemes in speech recognition. The novelty of the work presented here is the automated process we developed in segmenting gesture trajectories based on a simple set of threshold values in curvature and accumulated curvature angle. In order to represent its angular distribution of each separated states, the von Mises distribution is used. Likelihood based state segmentation was implemented in addition to the threshold based method to ensure that gesture sets are segmented consistently. The proposed method can separate each angular state of training data at the initialization step, thus providing a solution to mitigate ambiguity on initializing HMM. For comparative studies, the proposed automated state segmentation based HMM initialization was considered over the conventional method. Effectiveness of the proposed method is shown as it achieved higher recognition rates in experiments over conventional methods. Keywords: Trajectory segmentation, hand gesture recognition, hidden Markov model, HMM initialization.

1 Introduction The proper presentation of hand gesture motion is essential for the performance of an automatic hand gesture recognition system. The trajectory of motion is widely used as a distinct feature to represent not only hand gesture recognition [1],[2] but also action recognition [3] and hand-written character recognition [4],[5],[6]. Hand gesture in this work is represented by a trajectory composed of spatiotemporal sequence of hand positions in 2-D. In addition gesture trajectory can be decomposed into several strokes and curves that maintain a consistent angular tendency. Hand signal modeling in most of the related research has been based on classifying each of such angular tendencies, and connecting them to a Markov chain C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 146–150, 2011. © Springer-Verlag Berlin Heidelberg 2011

Rule Based Trajectory Segmentation

147

to add temporal property such as Hidden Markov Model (HMM) [7]. In HMM each stroke or curve is modeled as a state with statistical parameters such as mean, mixture weight, and variance that are optimized by Baum-Welch re-estimation formulae [8] with training data. However, in practice, some of the parameters such as a number of states comprising a gesture vocabulary are determined manually. The number of states is an important parameter because an excessive number of states can result an over-fitting problem if the number of training samples is insufficient compared to the number of model parameters. When there are insufficient number of states, HMM’s ability to discriminate correctly in turn is reduced. A common way of determining the optimal number of states is by trial and error. Gunter and Bunke [4] optimized the number of state by iterative refinement of the recognizer performance. Starner, Weaver, and Pentland [1] determined the number of state as a constant regardless of gesture pattern. Lee, Kim, and Kim [5] used the skeleton pattern of handwritten character to determine the number of state for on-line handwriting recognition. Elmezain et al. [2] empirically assigned the number of state based on the hand motion trajectory. In this paper, we propose a Figure-based Trajectory Segmentation (FTS) algorithm to automatically determine the number of states based on a simple set of rules in terms of trajectory angles.

2 Proposed Method 2.1 Figure-Based Trajectory Segmentation Gesture trajectories are acquired by sampling centroid locations of the hand. The proposed method of the FTS segments the trajectory into a series of states by dividing them as either a straight line or a curve. For a straight line segment, curvature of the segment is expected to be negligibly small. For a curved section, its curvature value is finite and is expected to remain below some value. By a simple thresholding, these two types of trajectory segments can be easily separated. If, on the other hand, the curvature is significantly large, it can be inferred that the trajectory exhibited an abrupt heading change such as a vertex. Additional segmentation may be performed on curves by considering their cumulative value of the curvatures by another threshold. Upon exceeding the threshold, the curve section is segmented, and the trajectory is considered to have transitioned into another segment. Our implementation of the FTS is illustrated in Fig. 1. Curvature is computed [3] by

k (t ) =

|| v(t ) × a(t ) || || v(t ) ||3

(1)

where v(t) and a(t) denote velocity and acceleration of hand motion at time t respectively. The cumulative curvature angle is defined by k c (t ) = where

τ

t

∑ k (i) || v(i) ||

i =t −τ

denotes the time at previous segmentation point.

(2)

148

J. Beh, D. Han, and H. Ko START

Read data Calculate curvature Y

curvature > thc

Segment

N Accumulate curvature angle Y

cumulated angle > tha

Initialize cumulation

N N

End of data Y

thc: threshold for line segmention tha: threshold for curve segmention

END

Fig. 1. Flowchart of the proposed FTS

The number of segment is adjustable by tuning the angular change threshold. In this work its value is set to 80˚. In Fig. 2 the examples of segmentation for hand gesture trajectory “A” and “D” are depicted.

(a) Hand motion trajectory for letter “A”

(b) Hand motion trajectory for letter “D”

Fig. 2. Example hand motion trajectories “O” mark denote the detected turning point of segmentation

2.2 Determining Number of States The proposed FTS method was applied to a training database of a set of alphanumeric hand gesture symbols. Due to variability of gesture motion from different individuals, it has been observed that a same gesture symbol may be divided into different number of segments. Among various segmentation lengths of these symbols, we selected the number of states shown to be the most frequent in the training set as the number used for the construction of the HMM models. 2.3 HMM State Parameter Initialization Since the angular heading of hand motion is considered as an input feature of our recognizer, we employ the von Mises probability density function (PDF) [9] to represent the state distribution of the angular heading, x. The von Mises PDF is given by

Rule Based Trajectory Segmentation

f ( x | μ,κ ) =

eκ cos( x − μ ) 2πI 0 (κ )

149

(3)

where I0(·) is the modified Bessel function of order zero. μ and κ are, respectively, the circular mean and concentration that are analogous to the mean and variance in the normal distribution. For each state in each gesture HMM, the segmented trajectory data are used to estimate the parameter μ and κ initially. Then, we refine the segment boundary (turning point) by aligning input feature to corresponding state using (3). This procedure is repeated iteratively until there is no change in the segment boundary position. After this refinement the final update of μ and κ are to be the initialized parameters.

3 Performance Evaluation 3.1 Experimental Settings The following are the set of alpha-numeric symbols we have chosen for our initial study: 1. Numbers: 1, 2, 3, 4, 5, 6, 7, 8, 9 2. Alphabet: A, B, C, D, X, Y, Z 3. Symbols: ↑, ↓, ←, →, @ The hand gesture database consists of samples from 22 people. Three separate hand gesture sets were collected from each individual. For evaluation, 11 people’s data are used for training the gesture recognition HMM, and the remaining data were used for testing. Video frame rate in the database was 40 Hz and the image resolution was 160 x 120.

3.2 Results We compared the performance of the proposed method to conventional approaches [5, 6]. In Table 1, the fixed number method sets the number of state of each HMM to a constant, and the Bakis method sets the number of state of each HMM to the average length of the corresponding sequence of feature multiplied by a constant f. In our implementation of the fixed number method, the best result was obtained with 4 states, and for the Bakis method we obtained the best result with setting f to 0.08. Table 1. Performance comparison

Recognition error rate (%)

Fixed number 9.53

Bakis 12.99

Proposed method 7.50

150

J. Beh, D. Han, and H. Ko

From the result the proposed method achieves the lowest error rate. Since training the HMM is based on the maximum likelihood estimation principle, a proper initial estimation is essential for obtaining globally maximized likelihood function for good recognizer performance. The proposed segmentation inherently clusters angular tendencies in a gesture symbol trajectory. It determines the number of states in HMM consistently, therefore it would also result proper initialization the parameters of each state. Another advantage of the proposed method is that it automatically determines the number of states, unlike other method that require manual trial-and-error procedure, thus it simplifies the training part of the algorithm.

4 Conclusions A rule based trajectory segmentation method to initialize hand gesture HMM is proposed in this paper. The number of state in HMM is automatically determined by the number of segments in the hand gesture by a simple rule based algorithm. Those segments are used to initialize statistical parameters of state of each HMM. From the experimental result, the proposed method reduced the error rate by an average of 31.8% over the conventional methods. Advantages of the proposed segmentation method are that the training phase is simple and yet consistent in making good initial estimates for the HMM.

References 1. Starner, T., Weaver, J., Pentland, A.: Real-time American sign language recognition using desk and wearable computer based video. IEEE Trans. PAMI 20(12), 1371–1375 (1998) 2. Garg, P., Aggarwal, N., Sofat, S.: Vision based hand gesture recognition. World Academy of Science, Engineering and Technology 49 (2009) 3. Rao, G., Yilmaz, A., Shah, M.: View-inveriant representation and recognition of actions. International Journal of Computer Vision 50(2), 203–226 (2002) 4. Gunter, S., Bunke, H.: HMM-based hand written word recognition: on the optimization of the number of states, training iterations and Gaussian components. Pattern Recognition 37, 2069–2079 (2004) 5. Lee, J.J., Kim, J., Kim, J.H.: Data-driven design of HMM topology for on-line handwriting recognition. In: The Proceedings of 7th International Workshop on Frontiers in Handwriting Recognition, pp. 239–248 (2000) 6. Gunter, S., Bunke, H.: Optimizing the number of states, training iterations and Gaussian s in an HMM-based Handwritten Word Recognizer. In: The Proceedings of the 7th International Conference on Document Analysis and Recognition (2003) 7. Juang, B.-H., Rabiner, L.R.: A probabilistic distance measure for hidden Markov models. AT&T Technical Journal 64(2), 391–408 (1985) 8. Baum, L.E.: An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes. Inequalities 3(9), 1–8 (1972) 9. Fisher, N.I.: Statistical Analysis of Circular Data, Ch. 3.3.6 The von Mises distiribution, pp. 48–56. Cambridge University Press, Cambridge (1993)

Head-Free, Remote Eye-Gaze Detection System with Easy Calibration Using Stereo-Calibrated Two Video Cameras Yoshinobu Ebisawa, Kazuki Abo, and Kiyotaka Fukumoto Faculty of Engineering, Shizuoka University, Johoku 3-5-7, Hamamatsu, 432-8561 Japan [email protected]

Abstract. The video-based, head-free, remote eye-gaze detection system based on detection of the pupil and the corneal reflection was developed using stereocalibrated two cameras. The gaze detection theory assumed the linear relationship; θ=k|r’|. Here, θ is the angle between the line of sight and the line connecting between the pupil and the camera, and |r’| indicates the size of the corneal reflection - pupil center vector. Three novel easy calibration methods were proposed; ‘automatic’, ‘one-point’, and ‘two-point’. In the ‘automatic’, the user does not have to fixate the specific location in the PC screen. In the ‘one-point’, the angular difference between the optical and visual axes of the eye was determined and used for compensation. The ‘two-point’ was proposed to compensate the nonlinear relationship between |r’| and θ, which occurs when θ is large. The precision of gaze detection was compared among the three methods using the developed gaze detection system. Keywords: eye-gaze detection, pupil, corneal reflection, user calibration, head-free, infant.

1 Introduction So far, several non-contact, remote, eye-gaze detection systems allowing head displacements have been developed for human monitoring or human interface [1][2]. However, the current systems require some effort by the user that he or she must fixate several points on a PC screen for gaze calibration [3][4]. Moreover, in many of the other systems, a movable range of the user’s head is narrow. We have already developed a precise head-free eye-gaze detection system with three cameras [5]. To detect pupil easily and to generate the corneal reflection, which are necessary to estimate the eye-gaze point, the near-infrared LEDs were installed in each camera. Two of the cameras are stereo-calibrated wide view cameras, which detect 3-D positions of the pupils. Another one is a narrow view camera with a pantilt drive unit. The latter camera tracks one of the pupils and detects the angle of the line of sight by high resolution. The angle was determined using the corneal reflection - pupil center vector. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 151–155, 2011. © Springer-Verlag Berlin Heidelberg 2011

152

Y. Ebisawa, K. Abo, and K. Fukumoto

Although the resolution of eye-gaze detection of this system was high, the system can detect only one eye. Moreover, the pan-tilt unit may obstruct the progress of development of the system by its high cost and dynamic mechanism, and so on. So, in the present study, we develop two camera systems in which the function of the narrow camera is installed in those of the stereo-calibrated two cameras. In addition, in the three camera system, we proposed two easy calibration methods: the moving target and two-point calibration methods [5]. Both methods used only the inclination of the corneal reflection - pupil center vector for calibration. Accordingly, by using the size as well as the inclination of the vector, the improvement of the precision is expected. Moreover, in the two-camera system, four points of gaze on the screen can be detected. Averaging them may increase the resolution of gaze detection. Recently, the recent research showed that the gaze distribution of the infant with autism is peculiar [5], and the application of the eye-gaze detection system to the autism diagnosis support system is expected. In addition, the medical screening at the stage of the infant is hoped for. However, it is difficult to make infants fixate several calibration points. So we propose three easy calibration methods and compare them in the developed two-camera system experimentally.

2 System Configuration and Eye-Gaze Detection Theory Each optical system of the developed two-camera system consists of a NTSC B/W video camera having a near-infrared sensitivity, a 25 mm lens, an infrared pass filter (IR80), and light sources having many infrared LEDs (Fig. 1(a)). The two cameras are driven with slightly shifting synchronization. The LEDs are lit synchronized with the exposure period (500µs). A PC captures the images of the user’s face via an image grabbing board and then detects the centers of a pupil and a corneal reflection for each are detected. The two cameras are separately set under a 19 inch display. By camera calibration, the camera position, O, comes to be known in the world coordinate system (Fig. 1(b)). By stereo matching, the 3-D pupil center position, P, is detected. Let a gaze point, Q, be the intersection between the line of sight and a visual object plane (PC screen, known), the line of sight and the position of Q are determined by the two angles (θ and φ) and the position of P. To understand this theory easily, we define a virtual gaze plane, H, which is perpendicular to line OP and passes the camera position, O. The plane rotates along with the pupil position. The intersection of plane H and line PQ is defined as point T. The position of T in the coordinates system of H is expressed by θ and φ. In the gaze detection theory, the angels, θ and φ, are related to the corneal reflection - pupil center vector, r, (Fig.1 (b)). Here, actually the vector is calculated as the corresponding 3-D vector, r’, and is used. The direction of r’, φ’, is defined as the angle from the horizontal axis in the camera image. Actually φ’ is compensated by the pose of the camera. First, the theory assumes φ’=φ. Here, φ is angle between line OT and the horizontal axis on the plane H. This horizontal axis is defined as the nodal line between plane H and the horizontal plane in the world coordinate system. Second, the theory assumes a linear relationship between |r’| and θ, as follows.

Head-Free, Remote Eye-Gaze Detection System with Easy Calibration

θ = k |r’|

153

(1)

Therefore, obtaining the value of k in eq. (1) means the gaze calibration. However, in general, there is the several-degree difference between the visual axis and the optical axis of the eye. This difference produces the errors in gaze point detection. To determine the point precisely, the user must fixate one calibration point at least. In addition, when θ is large, the relationship between |r’| and θ may become nonlinear. It depends on the user individual. Since the system in this study has the two cameras, the gaze detection theory is expanded as shown in Fig. 2.

(a)

(b)

Fig. 1. (a) System appearance and (b) gaze detection theory. (a) The light source arranging the infrared LEDs having two different wavelength in two concentric rings (inner: 850 nm, outer: 940 nm) is attached to each camera. When the inner LEDs turn on, the pupils become bright. When the outer LEDs turn on, they become dark. These two groups of LEDs are alternately turned on synchronously with the field of the camera. By differentiating the consecutive bright and dark pupil images, the pupils become conspicuous in the difference image due to cancellation of the background images except for the pupils. The pupils are detected in the difference image. (b) See in the text.

Fig. 2. The expanded gaze detection theory. P indicates one of the two eyes. OL and OR indicate the positions of the two cameras. If the calibration value, k, is determined, the two lines of sight can be calculated for one eye. For another eye, the two lines of sight can be also determined.

154

Y. Ebisawa, K. Abo, and K. Fukumoto

3 Eye-Gaze Calibration Methods We tried the following three calibration methods. (a) ‘Automatic’ calibration method: The user does not need to fixate any known point. In Fig. 2, the vectors OLTL and ORT R are determined by k, |r’L|, |r’R|, φL’ and φR’. TL and TR are projected onto the gaze projection plane, G. To minimize the distance between the two projected points (T L’ and T R’) on plane G, the value of k was determined. This method assumes to detect the optical axis of the eyeball. (b) ‘One-point’ calibration method: The user fixates one point presented at the center of the PC screen. The horizontal and vertical angular differences between the optical and visual axes (ΔQx, ΔQy) are recorded. The value of k is also determined by the same method as method (a). They are used for gaze detection. (c) ‘Two-point’ calibration method: The user fixates not only the calibration point presented at the center but also another point presented at the top of the screen. This method is proposed to compensate the nonlinear relationship between θ and |r|; this may occur when θ is large. The following equations are used.

⎧k |r ' | θ=⎨ 1 ⎩θ B + k 2 (|r ' |-|r ' B |)

k 1|r ' | ≤ θ B k 1|r ' | > θ B ,

(2)

where θB and |r’B| indicate the averages of θ and |r’| when the user fixated the center calibration point. The values of k and (ΔQx, ΔQy) obtained by method (b) are used for gaze detection. The value of k is used as k1.

4 Experiments Three university students participated in the experiments. In the calibration procedure, the subject fixated the two calibration points; the center and top of the screen. After calibration procedure, the subject fixated nine targets evenly arranged on the PC screen one by one. 30 gaze points (1 sec) were acquired for each target. In both procedures, the distance between the subject’s face and the PC screen was approximately 80 cm. Fig. 3 (a)-(c) compared the gaze point distributions among the three calibration methods for subject KY. Each dot shows the average of both eyes’ gaze points. Fig. 3 (d) compares the average gaze detection errors in visual angle among the three calibration methods. The three subjects showed the average errors of 2.16 deg in the ‘automatic’, 1.37 deg in the ‘one-point’, and 0.94 deg in the ‘twopoint’ calibration method. In another experiment, the same three subjects moved back and forth by ±10 cm and right and left by ±5 cm. At seven positions within the range, they fixated the nine targets. Here, either was selected from among the one-point and two-point calibration methods depending on subjects. The average and SD of the gaze error for the three subjects was 0.92±0.40 deg.

Head-Free, Remote Eye-Gaze Detection System with Easy Calibration

155

Fig. 3. Comparison of results among three easy calibration methods

5 Conclusions The eye-gaze detection system using the stereo-calibrated two cameras was developed. The three novel easy calibration methods were proposed. The automatic and one-point calibration methods may be useful for the infant. If calibrated well, the developed gaze detection system experimentally showed the average gaze error less than 0.92±0.40 deg for 20 cm-back and forth, and 10 cm-right and left head movements.

References 1. Guestrin, E.D., Eizenman, M.: General theory of Remove Gaze Estimation Using the Pupil Center and Corneal Reflections. IEEE Trans. on Bioml. Eng. 53(6), 1124–1133 (2006) 2. Zhu, Z., Ji, Q.: Novel Eye Gaze Tracking Techniques under Natural Head Movement. IEEE Trans. on Bioml. Eng. 54(12), 2246–2260 (2007) 3. Tobii Technology, http://www.tobii.com/ 4. Eye Tech Digital Systems, http://www.eyetechds.com/ 5. Kondou, Y., Ebisawa, Y.: Easy Eye-Gaze Calibration using a Moving Visual Target in the Head-Free Remote Eye-Gaze Detection System. In: Proc. of VECIMS 2008, pp. 145–150 (2008) 6. Jones, W., Carr, K., Klin, A.: Absence of Preferential Looking to the Eyes of Approaching Adults Predicts level of Social Disability in 2-Year-Olds Toddlers with Autism Spectrum Disorder. Archives of General Psychiatry 65(8), 946–954 (2008)

Eye Position Effect on Audio-Visual Fusion Involves New Proposals for Multimodal Interface Design David Hartnage, Alain Bichot, Patrick Sandor, and Corinne Roumes Département Action & Cognition en Situation Opérationnelle, Institut de Recherche Biomédicale des Armées, BP 73, 91223 Brétigny-sur-Orge, France [email protected]

Abstract. Combination of audio and visual information is expected to ensure an efficient interface design for spatial information. Then, we focus on Audiovisual (AV) fusion referred to the perception of unity of audio and visual information despite there spatial disparity [1]. Previous experiment showed that AV fusion varied over space mainly with horizontal eccentricity [2]. As audio spatial information is coded in relation to head position and visual information is coded relative to eye position, question arises on eye position effect. The current psychophysical experiment investigates the effect of horizontal eye position shift on the variation of AV fusion over the 2D frontal space. Results showed that eye position affects AV fusion. Current data support the need for including eye position inputs when displaying redundant visual and auditory information in integrated multimodal interfaces. Results are discussed considering the probable effect of visual display structure.

1 Introduction A wide variety of working situations such as teleoperation, simulation for training and air operations, require the use of a virtual 3D restitution of the world. The so-called Virtual Reality can remain partial in the case of Augmented Reality when computed data are displayed overlaying the natural perception of the environment. The HumanMachine Interface (HMI) necessarily provides the operator with information in a non natural way. The human has to deal with a new kind of reality characterized by new physical rules. In order to design more ecologically the interface and to ensure the expected level of efficiency, it is worth understanding how the human assumes these new realities. A present HMI design challenge concerns multisensory integration. Combining different sensory information provides faster reaction time [3] and ensures a redundant information processing. These benefits are decisive in time determinant situations and may enhance situation awareness. This defy implies to study the way human perceive auditory and visual information as coming from the same location in space. This phenomenon is referred to as audio-visual (AV) fusion. The present paper concerns the rules of AV fusion in space and their involvement for HMI design. Perception of the world is basically multimodal; the world is not spread over sensory cues, it is perceived as unified [4]. Experimental results have shown an enhancement of cell responses for multimodal stimuli in various locations of the C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 156–160, 2011. © Springer-Verlag Berlin Heidelberg 2011

Eye Position Effect on Audio-Visual Fusion Involves New Proposals

157

nervous system [5][6]. Association of sensory cues provides many advantages: it is ecological, accurate, efficient and reliable; information is redundant; and responses are faster. Information coming from different modalities may be perceived as one. This phenomenon of perceptual fusion is usually compared to the ventriloquism effect [7]: the perception of the spatial location of a sound is biased in the direction of a visual stimulus (visual capture). Audio-visual fusion (AV fusion) has been investigated by Godfroy et al. [2]. They showed that AV fusion capability varied with the eccentricity of the bimodal stimulation in the participant’s perceptive field, the smallest spatial windows laying in the median sagittal plane. AV spatial windows were found to be symmetrical in relation to that plane. So, AV fusion space was considered as head-centered. These variations over space followed closely the spatial resolution of audition. There are quite a large number of reference frames depending on the sensory system involved, the context, and the goal [8]. Each of them can be egocentric (relative to the subject), allocentric (relative to the environment) or even geocentric (relative to the earth). What kind of frame of reference supports the AV fusion in space? Actually, primary unimodal egocentric reference frames are different whether information involves the visual sensory system or the auditory system. As vision information is initially coded relative to the retina, which depends on the position of the eye, the reference frame for vision is considered as eye-centered. For audition, spatial information depends mainly on inter-aural time differences (ITD), inter-aural level differences (ILD) in azimuth, and spectral cues in elevation [9]. These cues depend on the position of the head; so, the auditory reference frame is considered as head-centered. The aim is to define the limits of human AV fusion perception when the eyes are shifted in order to display trustworthy bimodal information whenever the operator does not look straight ahead, which may be the most common situation. Results will be discussed in the frame of the suggested applications.

2 Methods 2.1 Subjects Seven volunteers participated in this study, 4 women and 3 men aged, from 25 to 45. They all had normal or corrected to normal vision and no auditory defects. 2.2 Experimental Setup Stimuli Control The subject was located at the center of an acoustically-transparent, hemi-cylindrical screen, 120 cm in radius and 145 cm in height. The subject’s head was maintained by a custom bite-board with the eyes at a 110-cm height. The head was oriented 10° leftward of the center of the screen to increase the space of investigation when a fixation spot was presented 20° to the right. The orientation of the gaze was monitored with an ASL 504 (50Hz) eye-tracker placed 45 cm in front of the subject at a level lower than the field of investigation to prevent from any visual masking. To

158

D. Hartnage et al.

avoid any effect of allocentric cues, the experimental room was in total darkness, and noise level was reduced as much as possible (< 39 dBA). Stimuli could only be presented if the subject was looking at the red fixation spot with an angular error less than 1.66° for a mean duration of 500 ms (300-700 ms). Such a feedback between the eye-tracker sampling and the experimental software was used guarantee the dissociation of the two references frames at the bimodal stimulus onset. The bite-board permitted the control of the head position (i.e. the auditory reference frame) and the eye-tracker permitted the control of the eye position (i.e. the visual reference frame). The visual part of the bimodal 500ms stimulus was a 1° spot of light, it was provided by a green laser beam (MELLES-GRIOT, 532 nm, 5 mW) attenuated with optical filters in order to reduce luminance at 3 cd.m-2 (<5µW). The auditory part of the bimodal stimulus was a 49 dBA broad-band pink noise delivered by one of the 19 loudspeakers (LS) located behind the screen, oriented toward the subject’s head. Disparity For each of the 7 loudspeakers at eye level, 61 disparities between the spot and the sound source were tested. The spot could be displayed 0° to 20° horizontally apart from the center of the associated loudspeaker. At 0° of azimuth the spot could be displayed 0° to 30° apart with a 5° step in elevation. For all other loudspeakers only 9 disparities were tested. 2.3 Task The subject had to judge the perception of unity emerging from the bimodal stimulus using a joystick. When the spot and the pink noise were perceived as coming from a unique and common location in space, “fusion” response was selected by pulling the joystick. When the two unimodal stimuli were perceived as issuing from two distinct locations, the “non fusion” response was selected by pushing the joystick.

3 Results For most of the loudspeakers, fusion limits in elevation could not be established. Subjects still responded “fusion” for more than 50% of the trials for disparity up to 30°. AV fusion limits in azimuth varied along the horizontal dimension in the perceptive field (F(6,1243)= 21.783; p<.001). This variation appeared laterally deviated in the direction of eye shift in the reference frames dissociated condition. No significant difference was obtained between the 2 conditions (F(1,1243)= 0.973; ns) but the interaction between conditions and eccentricity was significant (F(6,1243)= 4.93; p<.05). The signs of the differences were analyzed. Differences between the two conditions were significant for 5 paired eccentricities out of the 7 tested. Moreover, the sign of the differences changed as if the curve of the dissociated condition would have been shifted 10° to the right (toward eye shift, 50%).

Eye Position Effect on Audio-Visual Fusion Involves New Proposals

159

4 Discussion The present basic experiment was conducted in order to study the effect of gaze shift on AV fusion. Results show that lateral gaze shift influences AV fusion perception in azimuth but not in elevation. Data collected in this experiment must be taken into account when considering HMI design. Designers from both virtual environments and enhanced perception used head tracker, and the displayed data changes in relation to head movement. As a consequence, gaze is assumed to be in line with head position when displaying visual and auditory information. It can be suggested currently that the lack of eye monitoring may bias the expected perception of visual and auditory redundant information. This question is particularly important for aircraft Head Mounted Display’s (HMD), where time is determinant. In such a 3D dynamic environment where the relative position of the various actors may dramatically change, it may be fruitful for the pilot to take advantage of the high mobility of the eyes compared to the dynamic of the head equipped with helmet. Indeed, the spherical shape of the eye, it’s location in the hemispherical socket, make any change in the eye orientation effortless. So it can be inferred that pilots usually favor eye movement instead of head movement. In many situations, especially in combat, there are high Gz accelerations, head movement will be limited and pilots will use more eye movement. Even though, it can be argued that this situation is very complex compared with psychophysics tasks, the present results have to be considered to avoid error in multimodal display. The experiment reported in this paper deliberately involved non informative visual and auditory stimuli. No allocentric cues were available. The bimodal stimulus was the only sensory input to process. Such conditions are in crucial opposition with HMIs where a high flow of data is concerned. Nevertheless, such basic data are not available in the literature. The ideal technological solution would take into account head and eye position to change the display in relation to the position of the eye and head to match with the perceptual space of the operator. To overpass this technical difficulty, it may be easier to exchange a hardware refinement for a software manipulation. If AV fusion naturally varies according to head and eye orientation, can an additional item in the information displayed provide a counter-measure to the eye shift effect? The present experiment was conducted in total darkness, with no additional visual cue provided. However, previous experiments have shown that visual display can influence AV fusion perception as well. Perhaps visual cues like linear symbols enhancing the edges of the display or a peculiar spatial arrangement of the features structuring visual space can give an allocentric reference frame to the subject. This allocentric reference frame would compensate for dissociation between auditory and visual reference frames. These data suggested further investigations to determine whether visual or auditory additional markers in the perceptive field may counteract eye inputs effect on audio-visual fusion. Moreover, such selected visual and auditory stimuli may also counterbalance the reported tendency of an upward offset of the perceived location of auditory sources in darkness [10].

160

D. Hartnage et al.

References 1. Bertelson, P., Radeau, M.: Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Perception & psychophysics 29(6), 578–584 (1981) 2. Godfroy, M., Roumes, C., Dauchy, P.: Spatial variations of visual-auditory fusion areas. Perception 32(10), 1233–1245 (2003) 3. Diederich, A.: Intersensory facilitation, vol. 369. Peter Lang, Frankfurt (1992) 4. Gibson, J.J.: The senses considered as perceptual systems. Greenwood Press, Westport (1966) 5. Stein, B.E., Meredith, A.M.: The merging of the senses. The MIT press, Cambridge (1993) 6. Giard, M.-H., Peronnet, F.: Visual-auditory Integration during Multimodal Object recognition in Humans: A Behavioral and Electrophysiological Study. Journal of Cognitive Neuroscience 11(5), 473–490 (1999) 7. Jack, C.E., Thurlow, W.R.: Effect of degree of visual association and angle of displacement on the ventriloquism effect. Perceptual and Motor Skills 37, 967–979 (1973) 8. Paillard, J.: Motor and representational framing of space. In: Paillard, J. (ed.) Brain and space, pp. 163–182. Oxford University Press, Oxford (1991) 9. Blauert, J.: Spatial Hearing. In: The psychophysics of human sound localization, The MIT press, London (1983) 10. Pedersen, J.A., Jorgensen, T.: Localization performance of real and virtual sound sources. In: New Directions For Improving Audio Effectiveness, pp. 29.1–29.30. RTO, France (2005) 11. Perrot, D.R., Saberi, K.: Minimum audible angle thresholds for sources varying in both elevation and azimuth. Journal of the Acoustical Society of America 87, 1728–1731 (1990) 12. Witkin, H.A., Asch, S.E.: Studies in space orientation. IV. Further experiments on perception of the upright with displaced visual fields. Journal of Experimental Psychology 38, 762–782 (1948)

A Virtual Mouse System Using Finger-Gestures of Twisting-in Takashi Kihara1 and Makio Ishihara2 1

Graduate School of Computer Science and Engineering, Fukuoka Institute of Technology, 3-30-1, Wajiro-higashi, Higashiku, Fukuoka, 811-0295 Japan [email protected] 2 Faculty of Information Engineering, Fukuoka Institute of Technology, 3-30-1, Wajiro-Higashi, Higashi-ku, Fukuoka, 811-0295 Japan [email protected]

Abstract. In this manuscript, we introduce the finger gesture of twisting-in to make an input device and conduct an experiment on performance. The finger gesture of twisting-in is a series of movements of a finger twisting and pushing. It gives us the feeling of twisting and pushing a button. The experiment shows that our input device has the potential to be practical and usable. Keywords: twisting-in, finger gesture, virtual mouse, input device.

1 Introduction A keyboard and mouse is a common input device. In recent years, various portable input devices have been manufactured. For example, NIPPON STYLE [1] manufactures a portable input device called a virtual keyboard using a semiconductor laser (see Fig. 1, left) and Apple manufactures iPad with a touch screen. In research, the trend is to use multiple off-the-shelf cameras in order to track the user’s fingertip or his/her hand in three dimensional coordinates in order to assemble an input device. The user moves his/her fingertip or hand before the cameras to be interpreted to computer’s commands. In this manuscript, we use a single camera to track the fingertip. When using two or more cameras, we have to adjust the position of every camera. However, when there is one camera, it does not take the time and effort into the setting. As for single camera-based input devices, Y. Hirobe et al. [2] built an input device which recognizes the fingertip from a picture taken by the attached single camera and enables the user to perform a series of movements of the fingertip to organize the corresponding computer’s command (see Fig. 1, right). This manuscript focuses on a user-friendly gesture-based input device so that the user can manipulate it in a more natural and easier way.

C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 161–165, 2011. © Springer-Verlag Berlin Heidelberg 2011

162

T. Kihara and M. Ishihara

Fig. 1. Virtual key board (left) and In-air typing interface for mobile devices using a single camera (right)

2 Finger Gesture of Twisting-in Fig. 2 shows an example of the finger gesture of twisting-in. The finger gesture of twisting-in is a series of movements of a finger twisting and pushing. For example, we do this when thrusting the cork of wine. Since this finger gesture is familiar with our everyday activities, it gives us the feeling of twisting and pushing a button, and provides us with a more natural and intuitive interface to computers. In this manuscript, we use the finger gesture of twisting-in to generate a couple of mouse events.

Fig. 2. Finger gesture of twisting-in

3 Design of Virtual Mouse System Fig. 3 outlines the system. This system consists of a PC and web camera. The system tracks an index finger with the camera and recognizes the index fingertip as a mouse cursor. If we twist our index finger before the camera, we can click on and drag an item depending upon the fingertip’s position and orientation. To perform this, the system analyzes the finger gesture of twisting-in from a sequence of pictures taken by the camera and transmits the corresponding mouse event to the operating system. At first, the system extracts skin color from each picture taken by the camera to make a binary image that represents a boundary of the

A Virtual Mouse System Using Finger-Gestures of Twisting-in

163

index finger. Next, the system detects the position and orientation of the index fingertip from the binary image. Finally the system tracks this information to generate mouse events. Fig. 4 shows how the system generates mouse events. There are three sorts of mouse events: move, clickin and clickout. Move event represents movement of the mouse cursor and clickin event represents the mouse button pressed, and the clickout event represents the mouse button released. For example, clickin event is transmitted when the index finger is leaned to more than 30 degrees counterclockwise in order to drag.

Fig. 3. Outline of our system

Fig. 4. Timing of mouse events’ generation

4 Experiment and Result To evaluate the system, we conducted an experiment on performance. We used the one-direction tapping test described by ISO(9241-9). Two rectangles of width w and with a center-to-center distance d were presented to the subject (see Fig. 5). The task was to point and click, along one axis, within each rectangle 25 times. There were 5 subjects between the age of 23 and 24. Each subject was asked to perform the task 6 trials for each of our system, a common mouse and touchpad. The index of difficulty varied over the trials. The index of difficulty is given by the following equation. Index of Difficulty =

d w

(1)

The elapsed time and the number of mis-clicks outside rectangles during each trial were measured.

164

T. Kihara and M. Ishihara

Fig. 5. One-direction tapping test

Fig. 6 and Table 1 show the results. In Fig. 6, the horizontal axis shows the elapsed time in second and the vertical one shows the effective index of difficulty. The effective index of difficulty is given by the following equation. Effective Index of Difficulty =

d we

we = 4,133 sx

(2) (3)

where sx is the standard deviation of the selection coordinates in a horizontal axis. Table 1 shows the average of the number of mis-clicks outside rectangles at each index of difficulty for our system, a mouse and touchpad.

Fig. 6. Index of difficulty as a function of elapsed time

Fig. 6 implies our system requires more time to complete the task than the others. This is because the image processing uses CPU power much enough to cause a certain delay in refresh rate. However, Table 1 implies the result seems fine if the index of difficulty is four or less. We expect our virtual mouse system to work well if the image processing performance is improved.

A Virtual Mouse System Using Finger-Gestures of Twisting-in

165

Table 1. Number of mis-clicks outside rectangles Index of difficulty Mouse TouchPad Our system

1

2

3

4

5

6

0.4 0.6 1.6

1.2 1 1.4

1 0.8 0.6

2.2 1.4 2

1.8 1 6.6

2 2 4.4

5 Conclusion In this manuscript, we introduced the finger gesture of twisting-in to make an input device and conducted an experiment on performance. The finger gesture of twistingin is a series of movements of a finger twisting and pushing. It gives us the feeling of twisting and pushing a button. The experiment showed that our input device seemed not practical but usable. In our future work, we are going to improve image processing speed with CUDA technology to reduce the delay in refresh rate.

References 1. http://www.tanomi.com/limited/html/00034.html NIPPON STYLE Virtual Key Board (VKB) 2. Hirobe, Y., Funahashi, K., Niikura, T., Watanabe, Y., Komuro, T., Ishikawa, M.: In-air Typing Interface for Mobile Devices using a single Camera. ITE Technical Report 33(23), 66–68 (2009), IST2009-32, ME2009-108

Control of Five Finger of Computer Graphics Hand Using Electromyographic Signal Measured with Multi-channeled Small Laplacian Electrodes Takuya Kimura and Akinori Ueno Department of Electrical and Electronic Engineering, School of Engineering, Tokyo Denki University, 2-2 Kanda-Nishiki-cho, Chiyoda-ku, Tokyo 101-8457, Japan [email protected], [email protected]

Abstract. The authors propose a new simple method for controlling five fingers of computer graphics hand respectively using electromyograms signal. We manufactured five double-polar concentric ring electrodes, for measuring surface Laplacian electromyograms(Laplacian EMGs) from flexor muscle corresponding to each finger flexing action. Laplacian EMGs can detect the current flowing into the component that is proportional to the vertical surface current density. The experimental results shown the each manufactured electrode could measure the Laplacian EMG corresponding to each finger flexion without any interference and that the developed interface could control the CG hand in real time. These results infer that the proposed approach is promising as an EMG-based human-computer interface or human-machine interface. The next challenges to be addressed are to conduct the evaluation test for many subjects and to facilitate electrodes attachment. Keywords: EMG, Human interface, Laplacian electrode.

1 Introduction In recent years, devices using EMG such as EMG prosthetic hand has been developed to control the computer and human interface. And, the Laplacian electrode can detect signals along the axis into the origin perpendicular to the surface. A pilot measuring circuit and a dedicated interface circuit were also assembled by reference to previous our approach [1]-[2]. Therefore, multiple Laplacian electrodes is expected to be detected independently of each finger on each EMG. The purpose of this study are Laplacian multi-channel EMG signals detected using multiple electrodes, and development of device of control of five finger of CG hand using the EMG signal.

2 Theoretical Basis of Body Surface Laplacian The use of the body surface Laplacian has been first proposed by Hjorth in electroencephalogram in 1975 [3] and by He and Cohen in electrocardiogram in 1992 [4], but not in EMG. Considering a local orthogonal coordinate system (x, y, z) with C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 166–170, 2011. © Springer-Verlag Berlin Heidelberg 2011

Control of Five Finger of Computer Graphics Hand

167

origin at a point on the body surface where the z axis is orthogonal to the body surface, the Laplacian EMG, LS, is defined by applying a Laplacian operator to the body surface potential φ as follows: ⎛ ∂ 2φ ∂ 2φ ⎞ ⎛ 1 ⎞ ∂J LS = −∇ 2 xyφ ≡ −∇ xy ⋅ ∇ xyφ ≡ −⎜ 2 + 2 ⎟ = −⎜ ⎟ z ⎜ ∂x ⎟ ∂ x ⎝ σ ⎠ ∂z ⎝ ⎠

(1)

Thus the Laplacian EMG signal is negatively proportional to the normal derivative of the normal component of the current density at the body. Therefore, Laplacian EMG is supposed to be sensitive to the firing of the measurement. Accordingly, the signal is potentially useful as an input for human interfaces, because it is less susceptible to the interferences caused by the activity of approximal muscles than the conventional EMG signal.

3 System Configuration 3.1 Laplacian Electrode In this study, fabricated Laplacian electrode is bipolar concentric ring electrode. Bipolar concentric ring electrode can be realized by disk electrode and forming the electrodes on the outer ring of concentric circles. And taking the difference between the two electrodes can detect the vertical component proportional to the current flowing into the origin. Therefore, the Laplacian electrodes can be to measure potentials from a localized area smaller diameter. This fabricated electrode, the two electrodes area is 50mm2, the distance between the electrodes is 1.0mm, diameter electrodes is 14mm. 3.2 Detection of EMG Fig.1 shows the block diagram of the Laplacian EMG measurement circuit. This circuit is composed of high input impedance instrumentation amplifier, notch filter, high pass filter, low pass filter, inverting amplifier, additional type Driven-RightLeg(D.R.L.)circuit. Laplacian electrode skin contact impedance is high because it is small, therefore inst.amp. with high input impedance first-stage IC was used. The circuit constants in the H.P.F. and L.P.F. were set to obtain a cutoff frequency of 10 and 1000Hz, respectively. Notch filters were used to reduce 50Hz interference. Amplification factor of 50,000 times the entire circuit. Additional type D.R.L. circuit was tried for S/N improvements by feedback signal to the Body-Earth, the signal adding the residual signal of five channels. 3.3 Signal Converter Fig. 2 shows the block diagram of the signal converter circuit. This circuit is composed of H.P.F., Absolute value circuit, Integrating circuit, Inv.Amp., Comparator, Buffer, Relay circuit and USB Gamepad. The first stage is H.P.F. with a cutoff frequency of 53Hz, which cuts the low frequency component of EMG in order

168

T. Kimura and A. Ueno

Laplacian Electrode Inst. Amp.

-

Notch Filter 50Hz

H.P.F. 10Hz

L.P.F Inv. 1kHz Amp.

Buffer

×5

Electrode + (Body-Earth) D.R.L.

Fig. 1. A block diagram of the Laplacian EMG measurement circuit/

to reduce baseline variation. In the second stage, full-wave rectified signal to the Absolute value circuit In the third stage, the time constant of 0.1s to smooth at the integration circuit. Then, the comparator input signal is amplified 10 times. A pulse outputs that exceeds the threshold of the comparator. The comparator has hysteresis to prevent chattering. Shorts the gamepad button and send to PC when the input signal of high the at relay circuit.

EMG Amp.

PC

H.P.F.

Absolute value circuit

USB Gamepad

Relay circuit

Integrating circuit

Buffer

Inv. Amp.

Comparator

Fig. 2. Block diagram of the signal converter circuit

4 Evaluation of Developed System Three male volunteer aged 22 were instructed to sit down on a chair. Laplacian electrode was fixed by surgical tape and rubber bands to each subject's right forearm. Arrangement of electrodes was fixed to the best part out of the fingers being in EMG at pilot experiment. We control for EMG than CG hand when flexion of the five finger. In addition, similar experiments were performed with disposable electrodes fabricated same diameter attached to the same site. And it is compared the discrimination probability. In addition, the discrimination probability if we could control without interfering with the other fingers, finger obtained at 5 × 20 sets of 100 trials. Show the discrimination probability of disposable electrodes in Table 1, discrimination probability of Laplacian electrodes in Table 2.

Control of Five Finger of Computer Graphics Hand

1ch Thumb 2ch Index

169

1.77Vp-p 14.6Vp-p

3ch Middle 4ch Ring

10.7Vp-p 11.2Vp-p

5ch Little

4.58Vp-p 2s Fig. 3. Recordings of Lalacian EMG during individual finger flexions Table 1. Discrimination Probability of disposable electrodes

Subject A Subject B Subject C

1ch Thumb

2ch Index

3ch Middle

4ch Ring

5ch Little

Total

0%

0%

0%

55%

0%

11%

65%

15%

10%

5%

50%

29%

35%

0%

90%

5%

95%

45%

Table 2. Discrimination Probability of Laplacian electrodes

Subject A Subject B Subject C

1ch Thumb

2ch Index

3ch Middle

4ch Ring

5ch Little

Total

95%

75%

65%

100%

100%

87%

100%

85%

95%

80%

90%

90%

100%

85%

70%

95%

100%

90%

5 Conclusion The result is a high probability Laplacian EMG without interfering with other finger flexion can be measured in response to each finger, five fingers can be controlled in real time showed that CG hand. The next challenges to be addressed are to conduct the evaluation test for many subjects and to facilitate electrodes attachment. In addition, on-off only control to the

170

T. Kimura and A. Ueno

CG, and it is expected to be high discrimination probability is obtained by performing signal processing such as pattern recognition.

References 1. Miyazawa, K., Ueno, A., Mori, H., Hoshino, H., Noshiro, M.: Development and Evaluation of a Wireless Interface for Inputting Characters using Laplacian EMG. In: 28th Ann. Int. Conf. IEEE EMBS, pp. 2518–2521 (September 2006) 2. Ueno, A., Uchikawa, Y., Noshiro, M.: A Capacitive Sensor System for Measuring Laplacian Electromyogram through Cloth –A Pilot Study-. In: 29th Ann. Int. Conf. IEEE EMBS, pp. 2518–2521 (September 2006) 3. Hjorth, B.: An on-line Transformation of EEG Scalp Potentials into Orthogonal Source Derivations. Electroenceph. Clin. Neurophysiol. 39, 526–530 (1975) 4. He, B., Cohen, R.J.: Body: Body Surface Laplacian ECG Mapping. IEEE Trans. Biomed. Eng. 39(11), 1179–1191 (1992)

Kinematic Analysis of Remote Target Pointing Hand Movements in a 3D Environment Yung-Hui Lee and Shu-Kai Wu Department of Industrial Management, National Taiwan University of Science and Technology, No. 43 Kee-lung Road, Section 4, Taipei, Taiwan, ROC [email protected]

Abstract. The study investigates kinematics of target pointing hand movements in a free-hand, touchless, 3D environment. Consistent with Fitts’ original information processing model, target pointing hand movement time in a 3D space increased with an increase of distance, and increased with a decrease of target size. The relative primary submovement movement time (% of total movement time) for target with depth (67.24%) was higher than that for targets without depth (64.49%). Pointing targets in the upper half of the reference framework required a lifting up of the arm to work against the gravity, and thus cost more than those in the lower half. For targets with depth, they required an even elevated upper-arm, thus result a longer “primary submovement time”, a decreased “peak velocity”, and a decreased relative time to peak velocity. Keyword: Target pointing hand movement; 3D, depth perception, hand/arm kinematics.

1 Introduction As an alternative to traditional input devices (mouse, trackball, joysticks, and etc.), “3D target acquisition hand movement” was explored as an input function to various appliances that require hand-free and no–touch interaction with computer. In comparing with wearable input devices, hand movement seem to be a more natural alternative, can be considered as a part of the environment, and be operates remotely (Nakamura, et al., 2008). Lee et al. (2010) investigates and models the performance of target pointing hand movements in a free-hand, touchless 3D environment. Their results indicate that these movements are constrained in several ways. As the distance to a target is increased, or the target size is decreased, movement time is increased. Locations of the targets affected the performance. Moreover, the pointing target “Forward” position in the Z-axis (3.88 s) required more time than any other position. Recent analyses of the kinematics of pointing movements reveal an invariant shape characterized by an initial high velocity peak, which is sometimes followed by one or more low velocity, secondary peaks (Thompson et al., 2007). The goal of the current work is to clarify the nature of the above mentioned constraints on the performance of target pointing hand movements in 3D environment. In particular, we will investigate the combined effect of and interaction among target size, distance, and location on the C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 171–175, 2011. © Springer-Verlag Berlin Heidelberg 2011

172

Y.-H. Lee and S.-K. Wu

kinematics of pointing movements. We believe that the effects on movement kinematics from multiple sources will help to reveal the nature of the speed/accuracy tradeoff, and consequently, how these constrained movements are organized based on task relevant information.

2 Method 2.1 Subjects Ten volunteers, aged 22–26 years (23.4 ± 1.5 years old), self-declared right-handed, participated in this study. They were paid voluntary for the study. 2.2 The 3D Control A 3D computer vision based systems was used to track free hand, on-touch, control movement in real time. Figure 1 illustrated the hand/head position tracking and the target acquisition hand movement. The system is capable of recognizing gestures at 20 Hz. It was then the velocity and the accelerations of the hand movements were calculated. Details of system validations can be found in Lee et al. (2011).

Fig. 1.Two cameras tracked hand/head positions and target acquisition hand movements. Subjects sat 3 meters in front of a display screen.

2.3 The 3D Perspective Display Targets, cursor, and the visual database were computer generated and were presented on a 42-inch LCD screen (BENQ S-Series). Since targets were presented in a 2D perspective display, shadow of the target on the mesh with linear perspective and a forward-looking schematic was used to provide visual information in 3D. 2.4 Tasks There are a total of 17 targets on a reference framework (17 positions). Eight targets are located in the middle frame (with z=0, without depth). They are in the X-axis, Yaxis, and XY plan (a 2D plan). Other 9 targets were located, with depths, in the z-axis. Spherical targets are one of three sizes (1.8, 2.8 and 3.8 cm in radius) and are located in one of the three distances (8, 12, and 16 cm). Please see Lee et al. (2011) for illustration of the design of the tasks.

Kinematic Analysis of Remote Target Pointing Hand Movements in a 3D Environment

173

2.5 Performance Measurements Movements for each trial were divided into primary submovement and secondary submovmeent followed Thompson, et al. (2007). A reaction time was determined as the time from the target appearing on the screen and the velocity of the hand exceeding 8% of peak velocity. The remaining time constituted the total movement time used in the analysis. This time was further parsed into a primary phase and a cumulative secondary phase. In addition, measures of interest in the analysis were the mean velocity and the peak velocity of the primary submovment, relative time to peak velocity (% of total movement time), and relative primary submovement movement time (% of total movement time). Followed Fitts (1954), we deduced Fitt’s regression models based on moving distances (D) and target sizes (S). A calculation of the task difficulty index (ID=log2 2D/S) shows that ID ranged from 2.07 to 4.15. 2.6 Experimental Procedure To avoid mixing the change of target sizes with the change of target sizes as they recede in distance from an observer in 3D spaces, a split plot design was adopted, where the target size was selected as a whole plot. Target acquisition tasks were repeated in three different days for each subject. The experiment involved 3 days of experiment for each subject, each day consisted of three sessions and a total of 153 (3 sizes × 3 distances × 17 positions) targets were acquired. 2.7 Data Analysis A split-plot design was used for the data analysis, with target size as a whole plot effect. Factors of target distance and position were randomized within the plot. Tukey Test was used for post hoc comparison. An alpha level of 0.05 was selected as the minimum level of significance.

3 Result and Discussion 3.1 Primary, Secondary Submovement Time Primary submovement time: Target pointing hand movement time in a 3D space increased with an increase of distance, and increased with a decrease of target size. Post hoc analysis indicates that the time for a 3.8 cm target (1.59±0.96s) was significantly shorter than that for 2.8 cm (1.77±1.02s), followed by 1.8 cm (2.47±1.69s) (p<0.05). The time for a distance of 8 cm was 1.56±1.24s, which was significantly shorter than that for 12 cm (1.93±1.18s), followed by 16 cm (2.34±1.43s) (p<0.05). There was also a significant two-way interaction effect between “depth” and “position” (F(7, 63)= 7.07, p<0.01). This interaction suggests that the time for targets without depth and in the lower part of the spherical frame was shorter than that for targets with depth. However, the time for targets with depth and in the upper part of the spherical frame was shorter than that for targets without depth. Secondary submovement time: A total of 72% tasks that had “secondary submovement(s).” Only “Target size” significantly affected secondary submovement

174

Y.-H. Lee and S.-K. Wu

time (F(2, 18)= 195.12, p<0.01). Post hoc analysis indicates that the mean time for a 3.8 cm target (0.81±1.32s) was significantly shorter than that for a target of 2.8 cm (1.06±1.53s), followed by that for 1.8 cm (2.34±2.49s) (p<0.05). Relative primary submovement movement time (% of total movement time): ANOVA results showed that “size” and “distance” had a significant effect on the relative time to peak velocity. Figure 2 illustrated the relative time to the total movement time, mean velocities, peak velocities, and relative time to peak velocity as a function of index of task difficulty and effect of “depth”. Post hoc analysis indicated a larger percentage for targets with size of 3.8 cm (70.90±27.27%) than that of 1.8 cm (57.12±26.38 %). There was a larger percentage for targets with a distance of 16 cm (70.05±23.02 %) than that of 8 cm (58.89±30.38 %). Data in Figure 2 (a) showed that the percentage for target with depth (67.24%) was higher than that for targets without depth (64.49%). The primary, or ballistic, portion of a rapid aiming task is believed to be under feedforward control, with secondary submovements reflecting the ‘‘homing-in’’ or corrective phase.

(a)

(b)

(c)

(d)

Fig. 2. The (a) relative time to the total movement time, (b) mean velocities, (c) peak velocities, and (d) relative time to peak velocity as a function of index of task difficulty and effect of “depth”.

3.2 Velocity and Related Kinematic Variables Mean velocity: Mean velocity was significantly affected by “target size” and “distance”. In addition, ANOVA results reveal a significant two-way interaction effect between “depth” and “distance” (F(2, 18)= 12.49, p<0.01). This interaction suggests that targets “with depth” have significantly lower mean velocities when they were with longer “distances”.

Kinematic Analysis of Remote Target Pointing Hand Movements in a 3D Environment

175

Peak Velocity: Peak velocity was significantly affected by “distance” and “position”. ANOVA results indicate that the “depth” of the targets also affected peak velocity (F(1, 9)= 171, p<0.01). The peak velocity for targets without depth was 14.31±4.61 cm/s, which was significantly faster than that for targets with depth (12.98±3.70 cm/s) (p<0.05). ANOVA results also reveal a significant two-way interaction effect between “depth” and “distance” (F(2, 18)= 15.44, p<0.01). This interaction suggests that targets with larger index of task difficulty have a higher peak velocity as Fig. 2 (c) illustrated. This situation required participants to execute the largest movements of the hand along with large forces (Kuhtz-Busckbeck et al., 1998). For targets with depth in the upper half, they required an even elevated upper-arm, thus result a longer “primary submovement time” and a lower “peak velocity”. Relative time to peak velocity: Both “size” and “distance” had a significant effect on the relative time to peak velocity (% of total movement time). There was also a significant two-way interaction effect between “depth” and “position” (F(7, 63)= 6.48, p<0.01). Please see Fig. 2 (d) for illustration. The percentage for targets decreased linearly with the increased of index of task difficulty, indicating participants spent an increased amount of time in the deceleration phase of the movement. The increased mass for movements in depth reduced the peak velocity, resulting in more corrective, secondary submovements (i.e., smaller relative time to peak velocity), consistent with the role of task constraints in the speed/accuracy tradeoff.

References 1. Fitts, P.M.: The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47, 381–391 (1954) 2. Kuhtz-Buschbeck, J.P., Stolze, H., Boczek-Funcke, A., Johnk, K., Heinrichs, H., Illert, M.: Kinematic analysis of prehension movements in children. Behavioural Brain Research 93(12), 131–141 (1998a) 3. Lee, Y.H., Wu, S.K., Liu, Y.P.: Performance of Remote Target Acquisition Hand Movements in a 3D environment, Human Movement Sciences (2010) (in printing) 4. Nakamura, T., Takahashi, S., Tanaka, J.: Double-crossing: A new interaction technique for hand gesture interfaces. Computer Human Interaction 8th Asia Pacific Conference, 292–300 (2008) 5. Thompson, S.G., McConnell, D.S., Slocum, J.S., Bohan, M.: Kinematic analysis of multiple constraints on a pointing task. Hum. Movement Science 26, 11–26 (2007)

Design and Implementation of Deformation Based Gesture Interaction Wonkyum Lee1, Sungkwan Jung1, Sangsik Kim1, Woojin Ahn1, and Sang-su Lee2 1

KAIST Institute for IT Convergence, 335 Gwahangno, Yuseong-Gu, Daejeon, South Korea 2 Department of Industrial Design, KAIST, 335 Gwahangno, Yuseong-Gu, Daejeon, South Korea {wklee,skjung,sskim98,woojinahn}@itc.kaist.ac.kr, [email protected]

Abstract. We present an approach of designing and implementing a novel interaction in a future flexible display. By featuring flexibility of flexible displays, deformation-based gestures are employed as input techniques for interacting with digital contents. We designed gestures for the selected tasks, commonly found in commercial device, based on our prior study investigating how users prefer to manipulate flexible displays. We devised a concept of device which use the deformation based gesture interaction and implemented a prototype based on currently available technology. Sensors mounted on the prototype enable us to recognize deformation without visual sensors which are conventional in gestural recognition. The prototype is evaluated with applications that support our interaction by participants. Evaluation results show that deformation based gesture interaction can increase intuitiveness and fun. Keywords: interaction design, deformation-based interaction, flexible display, organic user interface.

interaction,

gesture

1 Introduction While recently various researches about user interface are studied by many researchers, the new concept of user interface, Organic User Interface (OUI), has drawn much attention in the HCI community. OUI represents that deformation of shape or form can be means of interacting with computers [1]. Various interaction methods employing deformation were proposed by many researchers and these studies showed the possibilities of deformation-based gesture interaction in flexible display. Schweisg et al. proposed a bendable computer prototype, in which zoomingin and zooming-out are implemented with bend sensor [2]. Twend was a prototype that uses bending gestures as input for a mobile device based on optical bend sensor [4]. In Paperwindows [3], paper-like interaction has been simulated with projector. Lee et al. investigate the user-preferred manipulation in flexible display based on user observation, and it gives a guideline of designing deformation-based interaction [5]. Our work is extension of these researches with user-centered interaction design and practical implementation of deformation-based gesture recognition. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 176–179, 2011. © Springer-Verlag Berlin Heidelberg 2011

Design and Implementation of Deformation Based Gesture Interaction

177

2 Designing Interaction We conducted user-centered approach to design deformation-based gesture interaction. By considering that most frequently used contents of information devices are music, videos, texts, and images, we extracted the common commands in interacting with the contents. The common commands are anticipated to be essential commands in flexible display targeting mobile information device. We categorized the commands with depths that distinguish the processes from beginning to end of interacting with the contents (Fig. 1).

Fig. 1. Set of commands and deformation-based gestures designed for flexible display

The deformation-based gestures were defined and assigned into the commands based on prior study [5]. In this study, 31 participants were asked to deform the given 3 types of A4-sized sheets as means of manipulating imaginary display, in which each type has the distinct elasticity. 343 gestures observed when users deform thin plastic were considered in our interaction scenarios, and the most preferred gesture for each command was selected in terms of average preference, average planning time and agreement score [6]. Selected set of gestures consists of bending, flipping, translating and swaying. The obtained set of gestures gains totally more than 75% preference from the evaluation results and the gestures were mapped into commands for completion of designing interaction (Fig. 1).

3 Implementation We made our prototype based on our designed interaction. Since there were, to our knowledge, few commercial flexible displays that provide full flexibility, mounting flexible display was ruled out and graphic output was displayed on rigid screen. When user manipulates our prototype with deformation, the screen showed the interaction result corresponding to the gesture user made. The prototype (Fig. 2) that we implemented consists of two boards; one is flex board implemented on flexible printed circuit board (FPCB) and another is MCU board implemented on rigid printed

178

W. Lee et al.

circuit board (PCB). The dimension of flex board is 25.5cm x 26cm, and the thickness is 3mm at most, enough to be integrated into possible future flexible display. 16 bend sensors at 16 edges both sides (at 8 edges each side), 5 accelerometer sensors at 4 corners and at center and 1 gyroscope sensor are mounted on flex board to measure the deformation of flex board. The dimension of the MCU board is 6 cm x 9 cm, and the thickness is 11mm at most. The MCU board has TI MSP430 MCU and the MCU reads the sensor data at a frequency of 200 Hz and the sends the sensor data to computing machine, Windows PC in this paper, via USB interface. The concept of recognizing deformation was use of developable surface model [7]. This model enabled us to estimate the deformation of planar sheet in limitations of measurement resolution. The bend sensor read only the degree of total bending without details ranging actual bend sensor, however guiding rule, along which flat sheet is bent, were obtained from the bending angle data of edges. All of our deformation based gestures were categorized by guiding rule, and mapping into gestures were conducted by matching between guiding rules from measurement data and each reference data corresponding to each gesture. Additional data of accelerometer sensors and gyroscope sensors was employed not only to recognizing other non-deformation based gestures such as flipping, swaying, and translating but to help recognize deformation with orientation status.

Fig. 2. Prototype for deformation-based gesture interaction

4 Evaluation We conducted evaluation of proposed interaction with user study. Participants, 25-32 years old university students, were asked to interacting with Google Earth application, E-book applications and media player application implemented in Windows PC environment. Without any instruction, they were asked to manipulate the prototype making arbitrary gestures. Participants were also asked to use mouse or keyboard with interacting same application. From user observation and in-depth interview, we figured out that deformation based gesture interaction helped user pay attention more than conventional input

Design and Implementation of Deformation Based Gesture Interaction

179

device and that they adapted to the new manipulation in a short time without instruction. Proposed interaction features analog interaction, for example, when user bends the prototype more, user feels turning page faster. The analog interaction is difficult to experience in mouse/keyboard, while the analog interaction provided great satisfaction. It was also found that although burden coming from physical manipulation to make appropriate gestures weakened efficiency, it increased joy in feeling varying shape of device according to deformation.

5 Conclusion and Future Works In this study, we proposed deformation based gesture interaction as a means of interacting flexible display device with digital contents. User-preferred interaction was designed based on prior research. We implemented a prototype with practicability and evaluated proposed interaction with user study. It is concluded that proposed interaction increases intuitiveness, joy and concentration and shows the possibilities of means of interacting future flexible display devices. However, new application for proposed interaction is demanded due to the limitations of traditional GUI based application, in which the deformation based gestures were not full of efficiency. High accuracy in recognizing the gestures, integration with real flexible display and an exploratory study on relation between deformation based gesture input and possible additional input are also our future works. Acknowledgments. This work was supported by the Industrial Strategic technology development program, KI002156 funded by the Ministry of Knowledge Economy(MKE, Korea).

References 1. Holman, D., Vertegaal, R.: Organic user interfaces: designing computers in any way, shape, or form. Commun. ACM 51, 48–55 (2008) 2. Schwesig, C., Poupyrev, I., Mori, E.: Gummi: User Interface for Deformable Computers. In: Proc. CHI 2003, pp. 954–955. ACM Press, New York (2003) 3. Holman, D., Vertegaal, R., Altossar, M., Troje, N., Johns, D.: Paperwindows: interaction techniques for digital paper. In: Proceedings of the CHI 2005, pp. 591–599. ACM Press, New York (2005) 4. Herkenrath, G., Karrer, T., Borchers, J.: Twend: twisting and bending as new interaction gesture in mobile devices. In: CHI 2008, pp. 3819–3824. ACM Press, New York (2008) 5. Lee, S., Kim, S., Jin, B., Choi, E., Kim, B., Jia, X., Kim, D., Lee, K.: How Users Manipulate Deformable Displays as Input Devices. In: CHI 2010, pp. 1647–1656. ACM Press, New York (2010) 6. Wobbrock, J.O., Aung, H.H., Rothrock, B., Myers, B.A.: Maximizing the Guessability of Symbolic Input. In: CHI 2005, pp. 1869–1872. ACM Press, New York (2005) 7. Perriollat, M., Bartoli, A.: A Quasi-Minimal Model for Paper-Like Surfaces. In: IEEE Computer Vision and Pattern Recognition, pp. 1–7. IEEE Press, Los Alamitos (2007)

The Expansibility of User Interfaces Using Peripheral Multisensory Stimulation Ju-Hwan Lee Department of Newmedia, Korean German Institute of Technology, Seoul, Korea [email protected]

Abstract. The present study has explored the expansibility of using everyday objects for game interface to provide additional multisensory stimuli. The perceived expansibility of game interfaces was investigated by measuring user various responses such as the degree of presence, immersion, and enjoyment. Offering actual sensations assumed in the virtual environment of game could enhance user experience by effectively accessing their emotions. Participants were provided with one of four output modes: basic condition, room light stimulation, chair vibration stimulation, & both light and vibration. Consequently, tactile stimulation has resulted in higher presence and immersion than visual did. The result also suggests that a gamer could experience the expanded interfaces with additional multisensory output, bringing more enjoyment. These tendencies are especially recognized for high immersive tendency group. These findings can be applied to various fields of game interface design. Keywords: Multisensory, Multimodal, Computer Game, User Interface, Expanded Game Interface.

1 Introduction A growing trend in game interface design is using multimodal input and output user interfaces. Previously, much of efforts to improve the game interface are centered on high fidelity graphics and sound, but the success of console game platforms has suggested other alternatives, for example an input device affording natural gesture and movement. It is presumed that a player feels more engaged and immersed in the game by evoking their various sensations beyond merely processing visual information. Presence is defined as the subjective experience of being in one place or environment even when one is physically situated in another [1]. We empirically investigated the expansibility of perceived user interfaces by adopting presence that defines epistemological dimension regarding spatial perception. Immersion, often synonymous to presence, refers to the sensation of being surrounded by a completely other reality. According to previous game studies, immersion is measured by the absence of awareness of real environment along with complete immersion in virtual game environment [2, 3]. Therefore, there are always possibilities that additional stimuli from peripheral space exterior game interface might work as a hindrance to gamer immersion in the game. However, research applying multisensory stimulation C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 180–183, 2011. © Springer-Verlag Berlin Heidelberg 2011

The Expansibility of User Interfaces Using Peripheral Multisensory Stimulation

181

to game interface has displayed that a social interaction activated by input devices supporting natural movement has enhanced immersion [4]. The result indicates that virtual and real environments are not an exclusive dimension, offering a gamer an expanded experience by a specific environmental connection. There are two common methods to improve gamers’ experience by displaying multi-sensory information in game interface. The first is to offer sensory stimuli similar to those simulated in the virtual environment (i.e. the scent of a flower given in the scene of full bloom flowers in 4D movie theatre) [5]. Displaying useful information such as remaining energy level, ammunition is second approach in which gamers benefit in processing information valuable to their mission more efficiently, increasing their motivation and immersion to the game [6]. These approaches could improve gamers’ enjoyment and satisfaction by accessing their emotion along with multisensory experience. Based on theoretical backgrounds, we have explored the possibility of using everyday objects for game interface to provide additional visual and tactile stimuli. The perceived expansibility of game interface was emphatically investigated by measuring user’s subjective responses such as presence, immersion, and enjoyment. We also measured the task performance since the potential relationship between presence and task performance is arguable whether performance is benefited or deteriorated by the enhanced presence. Offering actual sensations assumed in the virtual environment of game could enhance users’ experience by effectively accessing their emotions. We measured participants’ immersive tendency with using Immersive Tendency Questionnaire [1], since individual differences also play a critical role in experiencing presence along with technological components of media.

2 Experiment 2.1

Methods

Participants. 12 Participants (male: 8; female: 4, average age = 24.4) took part in this experiment, after administrating Immersive Tendency Questionnaire to measure their immersive tendency in virtual environment or game. Materials. A simple shooting game was developed and connected to Arduino board in order to synchronize events in game with external visual and tactile stimuli additionally displayed. In the event of blast of an enemy, tactile feedback was provided by vibrator installed on their seats for tactile stimuli and a flicker lasting a second by a light bulb for visual stimuli in the event of blast in the game. Procedure. Participants played the game after instruction of game. Shooting down enemies and avoiding crush with them simultaneously was the key of the game. Performance was measured by game score which a gamer gain from bringing down enemies during playing time. After the session, the questionnaire regarding immersion, presence, and enjoyment was administered for their subjective evaluation. 2.2 Results The participants’ subjective rating data were analyzed by a series of repeated measured analyses of variance (ANOVAs) with four levels of game output interface

182

J.-H. Lee

modes as an independent variable: basic, visual stimuli (light bulb), tactile stimuli (vibration of chair), and both tactile & visual stimuli(vibration plus light), with comparing low immersive tendency group with high immersive tendency group. The main effect of four output interface modes in the score of performance, immersion, presence, and enjoyment were not significant, but immersion [performance: F(3,11)=1.11, p>.05; immersion: F(3,11)=3.48, p<.05; presence: F(3,11)=1.51, p>.05; enjoyment: F(3,11)=1.63, p>.05]. However, on the inside of these results, the discrepancy between basic and tactile condition and the one between basic and simultaneous tactile and visual condition in immersion are the most significant [basic and tactile stimuli: F(1,11)=7.61, p<.05; basic and both visual & tactile stimuli: F(1,11)=5.04, p<.05]. Tactile stimuli have resulted in higher presence and immersion than visual stimuli did. The strong response toward tactile stimuli rather than basic condition was consistent with the result of the one in presence [tactile stimuli: F(1,11)=7.74, p<.05], supporting the existing research that immersion and presence are positively related. High immersive tendency group was likely to feel more immersed, presence, enjoyment to different levels of output interface modes, while

Fig. 1. A) Four levels of game output interfaces, B) An illustration of the experimental set-up used in this study

Fig. 2. The mean scores of participants’ immersion, presence, and enjoyment responses as a function of four output interface modes and immersive tendency groups

The Expansibility of User Interfaces Using Peripheral Multisensory Stimulation

183

low immersive tendency group showing less difference among different levels [immersion: F(1,10)=9.39, p<.05; presence: F(1,10)=2.00, p=1.87; enjoyment: F(1,10)=5.00, p<.05]. Especially, high immersive tendency group displayed the increase of enjoyment for visual condition and both visual & tactile stimuli condition than low immersive tendency group.

3 Discussion The result suggests that a gamer could experience the expanded interface with additional multisensory output, bringing more enjoyment. These tendencies are specially recognized for the higher immersive tendency group. The finding can be applied to various game interface designs. Additional sensory cues, for example, can be used to assist players or even distract them to increase difficulty level. Future studies are expected in relation to different game interface designs and genres.

References 1. Witmer, B.G., Singer, M.J.: Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence 7(3), 225–240 (1998) 2. Reeves, B., Nass, C.: The Media Equation. CSLI Publications/Cambridge University Press, Stanford (1996) 3. Roskos-Ewoldsen, D.R., Roskos-Ewoldsen, B., Dillman Carpentier, F.: Media Priming: An updated synthesis. In: Media Effects: Advances in theory and research, pp. 74–93. Routledge, London (2008) 4. Lindley, S.E., Le Couteur, J., Berthouze, N.L.: Stirring up Experience through Movement in Game Play: Effects on Engagement and Social Behavior. In: CHI 2008, Florence, Italy, pp. 511–514 (2008) 5. Riva, G., Davide, F., IJsselsteijn, W.: Being There: Concepts, effects and measurements of user presence in synthetic environments. IOS Press, Amsterdam (2003) 6. Nesbitt, K.V., Hoskens, I.: Multi-Sensory Game Interface improves Player Satisfaction but not Performance. In: 9th AUIC 2008, Wollongong, Australia, pp. 13–18 (2008)

Use of Hands-Free Mouse for Game Control Moyen Mohammad Mustaquim Department of Informatics and Media Uppsala Universiy, Uppsala, Sweden [email protected] Abstract. Interaction using gaze is not a new concept. It has been ready to deliver the promises to many people who suffer from severe disability and the only means of communication for them is eye or some muscle movement. The ideas proposed here construct a system that can collect gaze data, studies the oculomotor functions and capture natural human behavior. The hands-free mouse concept here is based on this idea which will provide the user to move the cursor and select object using eye, or any other muscular movement for the operation of game control. The concept of game in this study is, a tool that can help users with special needs doing exercise in an informal way. This can be helpful for patients who are under physiotherapy. A game interface can be used to show progresses of user’s muscle movement using scanning technique challenging the users for advancing certain level while exercising parallel. Keywords: Gaze, Eye Tracking, Inclusive Design, Special Need, Game.

1 Introduction Design for all or universal design is a focused topic in the world of designer, engineer and industry. Computer games which are not considered anymore as an entertainment issue for children or young adults are also not beyond the thought of universal design. Achieving inclusivity in computer games design is a difficult job. The principal thought that comes to general people’s mind when talking about inclusive game is that a game that gives accessibility to people with special needs. Although this is or will not be the primary focus of inclusive games design, it is and will be one of the major issue of all time- how can we design game that is accessible for everyone regardless of their age or physical disabilities. Using human modalities, the idea of game control for special need people is a vital topic of research going on in human computer interaction field. As the problem sometimes is that not everyone is able to or not comfortable enough to use their hand for using traditional input device in computer such as keyboard and mouse, the use of other inputs for controlling action came to the discussion in researchers mind. Computer games is not an easy thing to control either for people with such limitations and special needs. However to include them in the service like games it is important to consider multiple input controlling methods therefore. 1.1 Alternative Input in Computer Games A game controller is a device used for controlling the playable character or objects or otherwise providing input in a computer game. The controller is usually connected to C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 184–187, 2011. © Springer-Verlag Berlin Heidelberg 2011

Use of Hands-Free Mouse for Game Control

185

a game console or a computer by means of a wire, chord and also by means of wireless communication. Typical game controllers are keyboards, mice, joysticks, game pads etc. The principal function of a game controller is to control the movement or actions of a playable body-object or otherwise influence the events in a video- or computer game. However, certain groups of people have limited accessibility or no accessibility at all to such game controllers because of their physical limitations. Alternative ideas for game control are therefore important to consider for such groups of people. Speech is a common approach of controlling game. Using user emotion to control game action is dependent on detecting emotion in user’s speech command which is vital for disable people. Another approach is gaze based input. As mentioned earlier it is quite expensive as this system uses computer vision and cameras. Use of bio-signal interfaces such as brain computer interface which is based on EEG signals is an alternative but it is sensitive to noise hence not a very good idea to use for supporting people with disabilities for controlling game which is quite receptive issue for them. 1.2 Theoretical Concept of Hands-Free Mouse The demonstrated system called HaMCoS in [1] and the evaluation of the system in a case study in [2] is the motivation of this paper’s proposal of a hand free mouse system for controlling game. The system proposed in [1] uses Electromyogram or EMG signals that is originating from the muscle of the user. Intentional muscle contractions are detected depending on whether or not the intensity of the input signal exceeds a certain threshold. It then decides on the instant the mouse cursor starts moving, changes direction or stops[1]. The system responds by detecting a contraction event and the muscular activity in our case for controlling game is the brow muscle, although the muscle of user’s choice can be used for detecting contraction. A sensor can be placed on top of forehead of the user using a temporary adhesive for instance. One sensor is enough for the purpose as the sensor is used to detect one muscular movement only. However the idea of small muscle contraction is considered to be a single click of mouse whereas the double contraction is comparable to double mouse clicks. This is how the sensor in the proposed system detects the signal from the user sensor. 1.3 Game-As a Tool for People with Special Needs Using gaze technology people with special needs can be highly benefited if customized system can be fashioned for them to use the gaze technology as an input method. Game as mentioned earlier is not only for entertainment anymore, but the concept of game is viewed in a different point now a days. Using the concept of HaMCoS [1] if the sensor placed on someone’s muscle can detect muscular movement and send a signal to the computer then this input signal can be used to control a graphical user interface in an interactive way. While designed properly, this can be a motivational tool for special need people to accomplish different tasks; for

186

M.M. Mustaquim

instance ‘physiotherapy in an entertaining way’ without the help of a physiotherapist at certain cases. This leads less physical meeting between patient and physiotherapist which can save money for the patient. The following section describes the method and proposed model for this idea.

2 Method The method for creating such facility for ‘special needs’ group of people has two segments. Creating an entertaining game where user activities are stored as data is number one. While the use of sensor in a proper way in subject’s area of interest in body part where they want to do exercise, is number two. An EOG, Electrooculography (depending on eye movements), sensor can be placed on forehead or near eyebrow or some specific area of the face where the user want to train themselves to recover from paralysis situation due to stroke or other neurological issues. EOG is actually a hybrid approach of EMG. For exercising hands there can be different options. Use of Electromyogram (EMG) sensor on top of hand can be a simple approach. However the Gyroscope and Accelerometer can also be used in this case. Depending on the level of accuracy the sensor for hand can be chosen. However, EOG and EMG is the focused discussion in this research as they both detect muscular movement, hence the proposed model talks about EMG method to imply the sensor for receiving data from user’s muscle movement from upper or lower limb and triggers the GUI of the game. 2.1 Proposed Model For facial movement detection the EOG sensor is placed on the forehead as shown on figure1. To make a feasible use of the sensor on the facial and head area, two things were considered. Gaze accuracy describes the angular average distance from the actual gaze point to the one measured by the eye tracker. Gaze precision describes the spatial variation between individual gaze samples. Gaze accuracy and gaze precision are typically measured in degrees of visual angle. While Gyroscope is the best method to detect motion from upper or lower limb (hand for example), EMG sensor is still chosen for few reasons. EMG is not influenced by gravity and it’s sensitive to remote muscle activity and interference. Intimate contact with subject is needed for EMG sensors and small sensing elements are available with no influence on motion. Figure 2 shows an EMG sensor placed on subject’s hand while Figure 3 shows the block diagram of the complete system.

Fig. 1. A sample sensor placed on user’s forehead

Fig. 2. An EMG sensor placed on user’s hand

Use of Hands-Free Mouse for Game Control EOG SIGNALS

USER USER

187

EMG AND/OR EOG SENSOR

GUI Of GAMES EMG SIGNALS

Fig. 3. Block diagram of the proposed system

3 Result and Discussion The type of proposed hands-free system for specific purpose, for controlling game should be fast, efficient, and reliable. Gaze interaction was found to be slower than mouse/hand interaction and more erroneous than head interaction during some usability tests. Usability evaluations of gaze systems should not focus only on efficiency, however. High availability of system components and multimodal input options are also important factors contributing to effectiveness use of the systems. Proper design of an interactive user interface that gives the user a kind of feelings of competing on a game environment by rising psychological challenges to achieve a certain success level, can thus be an useful idea for many people who are in need of physiotherapy. Further research, usability testing by real users can thus lead this research to be an interesting topic of interest.

References 1. Torsten Felzer, R.N.: How to operate a PC without using the hands. In: Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 198– 199 (2005) 2. Felzer, T., Nordmann, R.: Evaluating the Hands-Free Mouse Control System: An Initial Case Study. In: Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I., et al. (eds.) ICCHP 2008. LNCS, vol. 5105, pp. 1188–1195. Springer, Heidelberg (2008)

An Armband-Type Touch-Free Space Input Device for HCI Dongwan Ryoo and Junseok Park Electronics and Telecommunications Research Institute., 161 Gajeong Dong, Yuseong Gu, Daejeon, Korea {dwryoo,parkjs}@etri.re.kr Abstract. In this paper, the design of an armband-type non-touch space input mouse for human computer interface is presented. Demand for a non-touch type input system has been increasing with the development of large-display technology. One of the non-touch type input system is a system based on gesture recognition. There is a lot of research on gesture recognition systems based on vision, but the presented device has advantages in terms of accuracy. The globe type input system is already developed, but the users do not want restrictions on finger actions. Unlike a glove type device, an armband device type does not constrain finger movements. We designed an armband- type space input mouse by using arm movement and wrist gesture actions. Keywords: Armband, Non-touch, HCI, 3D space.

1 Introduction Interaction technologies between a computer and a user are changing not only from interactions based on a traditional keyboard and mouse in the desktop environment, to the interactions based on a user's natural behavior in a three-dimensional input space, but also from interface based on audio-visual displays to realistic interfaces by using human touch senses. A variety of research for input device is being conducted [1]-[4]. Demand for a non-touch type input system has been increasing with the development of large-display technology. One of the non-touch type input system is a system based on gesture recognition. There is a lot of research on gesture recognition systems based on vision, but the presented device has advantages in terms of accuracy. Users would like to control devices by using small motions. The globe type input system is already developed, but the users do not want restrictions on finger actions. Unlike a glove type device, an armband device type does not constrain finger movements. In this paper, the design of an armband-type non-contact space input for human computer interface is presented. An armband-type non-touch 3D space input mouse is I/O interface technology for manipulating various devices using a user's arm/wrist gestures. The user wears the device on the wrist in contactless 3D space environment, such as a large smart interactive TV or HMD. We designed an armband-type space input mouse by using arm movement and wrist gestures. The practical considerations of an armband-type non-touch space input system are the contactless interface based on multi-sensor; the natural behavior of users such as arm, and wrist motion recognition; and convenience, such as small size unrestricted motions. C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 188–192, 2011. © Springer-Verlag Berlin Heidelberg 2011

An Armband-Type Touch-Free Space Input Device for HCI

189

2 Design of an Arm Band-Type Space Input Mouse We designed an armband-type space input mouse that uses arm movements and wrist gestures. Unlike a glove type device, the armband device type does not constraint on the finger movements. Therefore, wearing the armband, users can type on a keyboard and wear gloves. This type is a non-touch space input mouse that uses arm movements and wrist gestures(bending actions), as consisting of gyro sensors, accelerometers and proximity sensor array. The point of the mouse can be moved by recognizing arm movements from data processing of the IMU sensors, such as gyro sensors, and accelerometers. From data processing the proximity sensors array, a user can right/left-click input with wrist bending gestures.

Fig. 1. Non-touch 3D space input mouse: Armband-type

This space input system can also be used by a surgeon wearing surgical gloves to see MRI images etc. In other words, this device can also be used as a space input mouse in areas with special environmental constraints. It can be used by disabled people with impaired fingers, as well as for game, and portable handheld information devices. Like a PC mouse, this armband type space Input mouse also has six kinds of basic functions such as up, down, left, and right move and left click, and right click. Figure 1 shows how to click by using the non-touch 3D space input mouse(armbandtype). This armband-type space input system consists of IMU sensors to recognize arm gestures, and proximity sensors array to recognize wrist bending gesture.

3 Implementation and Experimental Results 3.1 Non-touch 3D Space Input Mouse Prototype The armband-type non-touch 3D space input mouse is I/O interface technology for manipulating various devices using a user's arm and wrist gestures with wearing on

190

D. Ryoo and J. Park

the wrist in contactless 3D space environment, such as large displays or HMDs. The focus of the second type of mouse prototype is on improving accuracy. And Haptic feedback was also considered. In addition, by using the wrist gestures, this system is available to some disabled people with impaired finger. The non-touch 3D space input mouse board(armband-type) and the USB type receiver dongle for wireless communication are shown in figure 2. The USB type receiver dongle is