Modern Approaches in Applied Intelligence, Part II - IEA AIE 2011

Lecture Notes in Artificial Intelligence Edited by R. Goebel, J. Siekmann, and W. Wahlster

Subseries of Lecture Notes in Computer Science

6704

Kishan G. Mehrotra Chilukuri K. Mohan Jae C. Oh Pramod K. Varshney Moonis Ali (Eds.)

Modern Approaches inApplied Intelligence 24th International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2011 Syracuse, NY, USA, June 28 – July 1, 2011 Proceedings, Part II

13

Series Editors Randy Goebel, University of Alberta, Edmonton, Canada Jörg Siekmann, University of Saarland, Saarbrücken, Germany Wolfgang Wahlster, DFKI and University of Saarland, Saarbrücken, Germany Volume Editors Kishan G. Mehrotra Chilukuri K. Mohan Jae C. Oh Pramod K. Varshney Syracuse University, Department of Electrical Engineering and Computer Science Syracuse, NY 13244-4100, USA E-mail: {mehrotra, mohan, jcoh, varshney}@syr.edu Moonis Ali Texas State University San Marcos, Department of Computer Science 601 University Drive, San Marcos, TX 78666-4616, USA E-mail: [email protected]

ISSN 0302-9743 e-ISSN 1611-3349 ISBN 978-3-642-21826-2 e-ISBN 978-3-642-21827-9 DOI 10.1007/978-3-642-21827-9 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2011929232 CR Subject Classification (1998): I.2, H.3-4, F.1-2, C.2, I.4-5, H.2.8 LNCS Sublibrary: SL 7 – Artificial Intelligence © Springer-Verlag Berlin Heidelberg 2011 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Preface

There has been a steady increase in demand for efficient and intelligent techniques for solving complex real-world problems. The fields of artificial intelligence and applied intelligence cover computational approaches and their applications that are often inspired by biological systems. Applied intelligence technologies are used to build machines that can solve real-world problems of significant complexity. Technologies used in applied intelligence are thus applicable to many areas including data mining, adaptive control, intelligent manufacturing, autonomous agents, bio-informatics, reasoning, computer vision, decision support systems, fuzzy logic, robotics, intelligent interfaces, Internet technology, machine learning, neural networks, evolutionary algorithms, heuristic search, intelligent design, planning, and scheduling. The International Society of Applied Intelligence (ISAI), through its annual IEA/AIE conferences, provides a forum for international scientific and industrial communities to interact with each other to develop and advance intelligent systems that address such concerns. The 24th International Conference on Industrial, Engineering and Other Applications of Applied Intelligence Systems (IEA/AIE-2011), held in Syracuse, NY (USA), followed the IEA/AIE tradition of providing an international scientific forum for researchers in the diverse field of applied intelligence. Invited speakers and authors addressed problems we face and presented their solutions by applying a broad spectrum of intelligent methodologies. Papers presented at IEA/AIE-2011 covered theoretical approaches as well as applications of intelligent systems in solving complex real-life problems. We received 206 papers and selected the 92 best papers for inclusion in these proceedings. Each paper was reviewed by at least three members of the Program Committee. The papers in the proceedings cover a wide number of topics including feature extraction, discretization, clustering, classification, diagnosis, data refinement, neural networks, genetic algorithms, learning classifier systems, Bayesian and probabilistic methods, image processing, robotics, navigation, optimization, scheduling, routing, game theory and agents, cognition, emotion, and beliefs. Special sessions included topics in the areas of incremental clustering and novelty detection techniques and their applications to intelligent analysis of time varying information, intelligent techniques for document processing, modeling and support of cognitive and affective human processes, cognitive computing facets in intelligent interaction, applications of intelligent systems, nature-inspired optimization – foundations and application, chemoinformatic and bioinformatic methods, algorithms and applications. These proceedings, consisting of 92 chapters authored by participants of IEA/AIE-2011, cover both the theory and applications of applied intelligent

VI

Preface

systems. Together, these papers highlight new trends and frontiers of applied intelligence and show how new research could lead to innovative applications of considerable practical significance. We expect that these proceedings will provide useful reference for future research. The conference also invited three outstanding scholars to give plenary keynote speeches. They were Ajay K. Royyuru from IBM Thomas J. Watson Research Center, Henry Kauts from the University of Rochester, and Linderman from Air Force Research Laboratory. We would like to thank Springer for their help in publishing the proceedings. We would also like to thank the Program Committee and other reviewers for their hard work in assuring the high quality of the proceedings. We would like to thank organizers of special sessions for their efforts to make this conference successful. We especially thank Syracuse University for their generous support of the conference. We thank our main sponsor, ISAI, as well as our cooperating organizations: Association for the Advancement of Artificial Intelligence (AAAI), Association for Computing Machinery (ACM/SIGART, SIGKDD), Austrian Association for Artificial Intelligence (OeGAI), British Computer Society Specialist Group on Artificial Intelligence (BCS SGAI), European Neural Network Society (ENNS), International Neural Network Society (INNS), Japanese Society for Artificial Intelligence (JSAI), Slovenian Artificial Intelligence Society (SLAIS), Spanish Society for Artificial Intelligence (AEPIA), Swiss Group for Artificial Intelligence and Cognitive Science (SGAICO), Taiwanese Association for Artificial Intelligence (TAAI), Taiwanese Association for Consumer Electronics (TACE), Texas State University-San Marcos. Finally, we cordially thank the organizers, invited speakers, and authors, whose efforts were critical for the success of the conference and the publication of these proceedings. Thanks are also due to many professionals who contributed to making the conference successful. April 2011

Kishan G. Mehrotra Chilukuri Mohan Jae C. Oh Pramod K. Varshney Moonis Ali

Organization

Program Committee

General Chair

Program Chairs

Moonis Ali, USA Kishan G. Mehrotra, USA Mohan Chilukuri, USA Jae C. Oh, USA Pramod K. Varshney, USA

Invited Session Chair

Sanjay Ranka, USA

Local Arrangements Chair

Thumrongsak Kosiyatrakul, USA

Program Committee Adam Jatowt, Japan Ah-Hwee Tan, Singapore Amruth Kumar, USA Andres Bustillo, Spain Anna Fensel, Austria Antonio Bahamonde, Spain Azizi Ab Aziz, The Netherlands B¨arbel Mertsching, Germany Bin-Yih Liao, Taiwan Bipin Indurkhya, India Bohdan Macukow, Poland Bora Kumova, Turkey C.W. Chan, Hong Kong Catholijn Jonker, The Netherlands Cecile Bothorel, France C´esar Garc´ıa-Osorio, Spain Changshui Zhang, Canada Chien-Chung Chan, USA Chih-Cheng Hung, USA Chilukuri K. Mohan, USA Chiung-Yao Fang, Taiwan Chunsheng Yang, Canada Chun-Yen Chang, Taiwan Colin Fyfe, UK Coral Del Val-Mu˜ noz, Spain

Dan Halperin, Israel Dan Tamir, USA Daniela Godoy, Argentina Dariusz Krol, Poland David Aha, USA Djamel Sadok, Brazil Domingo Ortiz-Boyer, Spain Don Potter, USA Don-Lin Yang, Taiwan Duco Ferro, The Netherlands Emilia Barakova, The Netherlands Enrique Frias-Martinez, Spain Enrique Herrera-Viedma, Spain Erik Blasch, USA Fevzi Belli, Germany Floriana Esposito, Italy Fran Campa G´ omez, Spain Francois Jacquenet, France Fred Freitas, Brazil Gerard Dreyfus, France Geun-Sik Jo, South Korea Gonzalo Aranda-Corral, Spain Gonzalo Cerruela-Garc´ıa, Spain Greg Lee, Taiwan Gregorio Sainz-Palmero, Spain

VIII

Organization

Guillen Quintana, Spain Guna Seetharaman, USA Gwo-Jen Hwang, Taiwan Hamido Fujita, USA Hans-Werner Guesgen, New Zealand Hasan Selim, Turkey Henri Prade, France Hiroshi Okuno, Japan Hisao Ishibuchi, Japan Huey-Ming Lee, Taiwan Humberto Bustince, Spain Iris Van De Kieft, The Netherlands Ishfaq Ahmad, USA Istenes Zolt´an, Hungary Jae Oh, USA Jamal Bentahar, Canada Jan Treur, The Netherlands Janusz Kacprzyk, Poland Jason J. Jung, South Korea Jean-Charles Lamirel, France Jeffrey Saltz, USA Jeng-Shyang Pan, Taiwan Jennifer Golbeck, USA Jes´ us Aguilar, Spain Jes´ us Maudes Raedo, Spain Jing Peng, USA John Dolan, USA Jorge Romeu, USA Jos´e Francisco Diez-Pastor, Spain Juan Jos´e Rodr´ıguez-D´ıez, Spain Judy Qiu, USA Jun Hakura, Japan Jyh-Horng Chou, Taiwan Kaikhah Khosrow, USA Kaoru Hirota, Japan Katarzyna Musial, UK Kazuhiko Suzuki, Japan Kishan Mehrotra, USA Krzysztof Juszczyszyn, Poland Kurosh Madani, France Kush Varshney, USA Laszlo Monostori, Hungary Lav Varshney, USA Leszek Borzemski, Poland Ling-Jyh Chen, Taiwan

Lin-Yu Tseng, Taiwan Lipo Wang, Singapore Longbing Cao, Australia Maciej Grzenda, Poland Man-Kwan Shan, Taiwan Manton Matthews, USA Marco Valtorta, USA Mario K¨ oppen, Japan Maritza Correa, Spain Mark Hoogendoorn, The Netherlands Masaki Kuremtsu, Japan Matthijs Pontier, The Netherlands Michal Lower, Poland Michele Folgheraiter, Germany Miquel S´ anchez-Marr´e, Spain Monika Lanzenberger, Austria Nancy McCraken, USA Natalie Van Der Wal, The Netherlands Ngoc-Thanh Nguyen, Germany Nicol´ as Garc´ıa-Pedrajas, Spain Niek Wijngaards, The Netherlands Nikolay Mirenkov, Japan Oshadi Alahakoon, Australia Pascal Wiggers, The Netherlands Patrick Br´ezillon, France Paul Chung, UK Philipp Baer, Germany Prabhat Mahanti, Canada Pramod Varshney, USA RadosLaw Katarzyniak, Poland Raja Velu, USA Rajmohan M., India Rianne Van Lambalgen, The Netherlands Riichiro Mizoguchi, Japan Robbert-Jan Beun, The Netherlands Rocio Romero, Spain Rodolfo Haber, Spain Rodrigo Ventura, Portugal Rung-Ching Chen, Taiwan Ruppa Thulasiram, Canada Shaheen Fatima, UK Shie-Jue Lee, Taiwan Shogo Okada, Japan Shusaku Tsumoto, Japan

Organization

Shyi-Ming Chen, Taiwan Simone Mainai, Italy Srini Ramaswamy, USA Stefano Ferilli, Italy Sung-Bae Cho, South Korea Takayuki Ito, Japan Tetsuo Kinoshita, Japan Tibor Bosse, The Netherlands Tim Hendtlass, Australia Tim Verwaart, The Netherlands Thumrongsak Kosiyatrakul, Thailand Tiranee Achalakul, Thailand Valery Tereshko, UK Victor Rayward-Smith, UK Victor Shen, Taiwan

Additional Reviewers Chein-I Chang, USA Chun-Nan Hsu, Taiwan John Henry, USA Jozsef Vancza, Hungary Michelle Hienkelwinder, USA

Vikt´ oria Zs´ ok, Hungary Vincent S. Tseng, Taiwan Vincenzo Loia, Italy Walter Potter, USA Wei-Shinn Ku, USA Wen-Juan Hou, Taiwan Wilco Verbeeten, Spain Yasser Mohammad, Egypt Ying Han, Spain Yo-Ping Huang, Taiwan Youngchul Bae, South Korea Yu-Bin Yang, Canada Yukio Ohsawa, Japan Zia Ul-Qayyum, Pakistan Zsolt-Janos Viharos, Hungary

IX

Table of Contents – Part II

Section 1: Cognitive Computing Facets in Intelligent Interaction Environmental Sound Recognition for Robot Audition Using Matching-Pursuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nobuhide Yamakawa, Toru Takahashi, Tetsuro Kitahara, Tetsuya Ogata, and Hiroshi G. Okuno Cognitive Aspects of Programming in Pictures . . . . . . . . . . . . . . . . . . . . . . . Yutaka Watanobe, Rentaro Yoshioka, and Nikolay Mirenkov

1

11

An Approach for Smoothly Recalling the Interrupted Tasks by Memorizing User Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kohei Sugawara and Hamido Fujita

21

Implementing an Efficient Causal Learning Mechanism in a Cognitive Tutoring Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Usef Faghihi, Philippe Fournier-Viger, and Roger Nkambou

27

Model Checking Commitment Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mohamed El-Menshawy, Jamal Bentahar, and Rachida Dssouli Mobile Architecture for Communication and Development of Applications Based on Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Luis M. Soria-Morillo, Juan A. Ortega-Ram´ırez, ´ Luis Gonz´ alez-Abril, and Juan A. Alvarez-Garc´ ıa A Simplified Human Cognitive Approach for Supporting Crowd Modeling in Tunnel Fires Emergency Simulation . . . . . . . . . . . . . . . . . . . . . Enrico Briano, Roberto Mosca, Roberto Revetria, and Alessandro Testa Model Checking Epistemic and Probabilistic Properties of Multi-agent Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wei Wan, Jamal Bentahar, and Abdessamad Ben Hamza Modeling Users of Crisis Training Environments by Integrating Psychological and Physiological Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gabriella Cortellessa, Rita D’Amico, Marco Pagani, Lorenza Tiberio, Riccardo De Benedictis, Giulio Bernardi, and Amedeo Cesta Personality Estimation Based on Weblog Text Classification . . . . . . . . . . . Atsunori Minamikawa and Hiroyuki Yokoyama

37

48

58

68

79

89

XII

Table of Contents – Part II

Design of an Optimal Automation System: Finding a Balance between a Human’s Task Engagement and Exhaustion . . . . . . . . . . . . . . . . . . . . . . . Michel Klein and Rianne van Lambalgen

98

A Cognitive Agent Model Using Inverse Mirroring for False Attribution of Own Actions to Other Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jan Treur and Muhammad Umair

109

Explaining Negotiation: Obtaining a Shared Mental Model of Preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Iris van de Kieft, Catholijn M. Jonker, and M. Birna van Riemsdijk

120

A Computational Model of Habit Learning to Enable Ambient Support for Lifestyle Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Michel C.A. Klein, Nataliya Mogles, Jan Treur, and Arlette van Wissen

130

Section 2: Applications of Intelligent Systems An Intelligent Method to Extract Characters in Color Document with Highlight Regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chun-Ming Tsai

143

Development of Technological System Structure for Threaded Connections Assembly under Conditions of Uncertainty . . . . . . . . . . . . . . . Roman Chumakov

153

Automatic Vehicle Identification by Plate Recognition for Intelligent Transportation System Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kaushik Deb, My Ha Le, Byung-Seok Woo, and Kang-Hyun Jo

163

Intelligent Page Recommender Agents: Real-Time Content Delivery for Articles and Pages Related to Similar Topics . . . . . . . . . . . . . . . . . . . . . . . . Robin M.E. Swezey, Shun Shiramatsu, Tadachika Ozono, and Toramatsu Shintani Meta-learning Based Optimization of Metabolic Pathway Data-Mining Inference System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tom´ as V. Arredondo, Wladimir O. Ormaz´ abal , Diego C. Candel, and Werner Creixell

173

183

Section 3: Optimization, Scheduling, and Routing Multiple Pickup and Delivery TSP with LIFO and Distance Constraints: A VNS Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiang Gao, Andrew Lim, Hu Qin, and Wenbin Zhu

193

Table of Contents – Part II

Distributed Learning with Biogeography-Based Optimization . . . . . . . . . . Carre Scheidegger, Arpit Shah, and Dan Simon Scheduling a Single Robot in a Job-Shop Environment through Precedence Constraint Posting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Daniel D´ıaz, M. Dolres R-Moreno, Amendo Cesta, Angelo Oddi, and Riccardo Rasconi

XIII

203

216

An Intelligent Framework to Online Bin Packing in a Just-In-Time Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sergey Polyakovskiy and Rym M’Hallah

226

A Greedy Heuristic for Airline Crew Rostering: Unique Challenges in a Large Airline in China . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qiao Chen, Andrew Lim, and Wenbin Zhu

237

Optimal Algorithms for Two-Dimensional Box Placement Problems . . . . Wenbin Zhu, Wee-Chong Oon, Yujian Weng, and Andrew Lim

246

An Algorithm for the Freight Allocation Problem with All-Units Quantity-Based Discount . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiang Gao, Andrew Lim, Wee-Chong Oon, and Hu Qin

256

A Distributed, Heterogeneous, Target-Optimized Operating System for a Multi-robot Search and Rescue Application . . . . . . . . . . . . . . . . . . . . . . . . Karl Muecke and Brian Powell

266

A Heuristic for the Multiple Container Loading Cost Minimization Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chan Hou Che, Weili Huang, Andrew Lim, and Wenbin Zhu

276

A Skyline-Based Heuristic for the 2D Rectangular Strip Packing Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lijun Wei, Andrew Lim, and Wenbin Zhu

286

Real-Time Resource Allocation Co-processor . . . . . . . . . . . . . . . . . . . . . . . . Stuart W. Card A Hybrid Search Strategy to Enhance Multiple Objective Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Li Ma and Babak Forouraghi

296

302

Section 4: Nature Inspired Optimization – Foundations and Applications Forest Planning Using Particle Swarm Optimization with a Priority Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Philip W. Brooks and Walter D. Potter

312

XIV

Table of Contents – Part II

Fuzzy Robot Controller Tuning with Biogeography-Based Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . George Thomas, Paul Lozovyy, and Dan Simon Development of a Genetic Fuzzy Controller for an Unmanned Aerial Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yan Qu, Swetha Pandhiti, Kalesha S. Bullard, Walter D. Potter, and Karl F. Fezer

319

328

Toward Evolving Self-organizing Software Systems: A Complex System Point of View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Liguo Yu, David Threm, and Srini Ramaswamy

336

Evolving Efficient Sensor Arrangement and Obstacle Avoidance Control Logic for a Miniature Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Muthukumaran Chandrasekaran, Karthik Nadig, and Khaled Rasheed

347

Section 5: Chemoinformatic and Bioinformatic Methods, Algorithms, and Applications Feature Selection for Translation Initiation Site Recognition . . . . . . . . . . . Aida de Haro-Garc´ıa, Javier P´erez-Rodr´ıguez, and Nicol´ as Garc´ıa-Pedrajas

357

DTP: Decision Tree-Based Predictor of Protein Contact Map . . . . . . . . . . Cosme Ernesto Santiesteban-Toca and Jesus Salvardor Aguilar-Ruiz

367

Translation Initiation Site Recognition by Means of Evolutionary Response Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rafael del Castillo-Gomariz and Nicol´ as Garc´ıa-Pedrajas An Evolutionary Algorithm for Gene Structure Prediction . . . . . . . . . . . . . Javier P´erez-Rodr´ıguez and Nicol´ as Garc´ıa-Pedrajas Prediction of Drug Activity Using Molecular Fragments-Based Representation and RFE Support Vector Machine Algorithm . . . . . . . . . . Gonzalo Cerruela Garc´ıa, Irene Luque Ruiz, and ´ Miguel Angel G´ omez-Nieto

376 386

396

Section 6: Neural Network, Classification, and Diagnosis A Hybrid Video Recommendation System Using a Graph-Based Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ¨ urk and Nihan Kesim Cicekli Gizem Ozt¨ A Diagnostic Reasoning Approach to Defect Prediction . . . . . . . . . . . . . . . Rui Abreu, Alberto Gonzalez-Sanchez, and Arjan J.C. van Gemund

406 416

Table of Contents – Part II

Multiple Source Phoneme Recognition Aided by Articulatory Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mark Kane and Julie Carson-Berndsen Plan Recommendation for Well Engineering . . . . . . . . . . . . . . . . . . . . . . . . . Richard Thomson, Stewart Massie, Susan Craw, Hatem Ahriz, and Ian Mills Lung Cancer Detection Using Labeled Sputum Sample: Multi Spectrum Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kesav Kancherla and Srinivas Mukkamala

XV

426 436

446

Section 7: Neural Network and Control Improvement of Building Automation System. . . . . . . . . . . . . . . . . . . . . . . . Mark Sh. Levin, Aliaksei Andrushevich, and Alexander Klapproth

459

Efficient Load Balancing Using the Bees Algorithm . . . . . . . . . . . . . . . . . . . Anabela Moreira Bernardino, Eug´enia Moreira Bernardino, Juan Manuel S´ anchez-P´erez, Juan Antonio G´ omez-Pulido, and Miguel Angel Vega-Rodr´ıguez

469

Predicting the Distribution of Thermal Comfort Votes . . . . . . . . . . . . . . . . Anika Schumann and Nic Wilson

480

Section 8: Agents, Game Theory, and Bidding Strategic Bidding Methodology for Electricity Markets Using Adaptive Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tiago Pinto, Zita Vale, F´ atima Rodrigues, Hugo Morais, and Isabel Pra¸ca Compromising Strategy Based on Estimated Maximum Utility for Automated Negotiation Agents Competition (ANAC-10) . . . . . . . . . . . . . . Shogo Kawaguchi, Katsuhide Fujita, and Takayuki Ito Negotiating Privacy Preferences in Video Surveillance Systems . . . . . . . . . Mukhtaj Singh Barhm, Nidal Qwasmi, Faisal Z. Qureshi, and Khalil el-Khatib

490

501 511

The Bayesian Pursuit Algorithm: A New Family of Estimator Learning Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xuan Zhang, Ole-Christoffer Granmo, and B. John Oommen

522

A Two-Armed Bandit Based Scheme for Accelerated Decentralized Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ole-Christoffer Granmo and Sondre Glimsdal

532

XVI

Table of Contents – Part II

Specification of Interlevel Relations for Agent Models in Multiple Abstraction Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jan Treur

542

Section 9: Cognition, Emotion, Psychology, and Beliefs An Argumentation Framework for Deriving Qualitative Risk Sensitive Preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wietske Visser, Koen V. Hindriks, and Catholijn M. Jonker Agent-Based Analysis of Patterns in Crowd Behaviour Involving Contagion of Mental States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tibor Bosse, Mark Hoogendoorn, Michel C.A. Klein, Jan Treur, and C. Natalie van der Wal Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

556

566

579

Table of Contents – Part I

Section 1: Incremental Clustering and Novelty Detection Techniques and Their Application to Intelligent Analysis of Time Varying Information Classification Model for Data Streams Based on Similarity . . . . . . . . . . . . Dayrelis Mena Torres, Jes´ us Aguilar Ruiz, and Yanet Rodr´ıguez Sarabia Comparison of Artificial Neural Networks and Dynamic Principal Component Analysis for Fault Diagnosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . Juan Carlos Tud´ on-Mart´ınez, Ruben Morales-Menendez, Luis Garza-Casta˜ n´ on, and Ricardo Ramirez-Mendoza Comparative Behaviour of Recent Incremental and Non-incremental Clustering Methods on Text: An Extended Study . . . . . . . . . . . . . . . . . . . . Jean-Charles Lamirel, Raghvendra Mall, and Mumtaz Ahmad

1

10

19

Section 2: Bayesian and Probabilistic Networks Fault Diagnosis in Power Networks with Hybrid Bayesian Networks and Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Luis Eduarda Garza Casta˜ n´ on, Deneb Robles Guill´en, and Ruben Morales-Menendez

29

Learning Temporal Bayesian Networks for Power Plant Diagnosis . . . . . . Pablo Hernandez-Leal, L. Enrique Sucar, Jesus A. Gonzalez, Eduardo F. Morales, and Pablo H. Ibarguengoytia

39

On the Fusion of Probabilistic Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . Salem Benferhat and Faiza Titouna

49

Section 3: Methodologies Basic Object Oriented Genetic Programming . . . . . . . . . . . . . . . . . . . . . . . . Tony White, Jinfei Fan, and Franz Oppacher

59

Inferring Border Crossing Intentions with Hidden Markov Models . . . . . . Gurmeet Singh, Kishan. G. Mehrotra, Chilukuri K. Mohan, and Thyagaraju Damarla

69

XVIII

Table of Contents – Part I

A Framework for Autonomous Search in the Ecli pse Solver . . . . . . . . . . . . Broderick Crawford, Ricardo Soto, Mauricio Montecinos, Carlos Castro, and Eric Monfroy

79

Multimodal Representations, Indexing, Unexpectedness and Proteins . . . Eric Paquet and Herna Lydia Viktor

85

A Generic Approach for Mining Indirect Association Rules in Data Streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wen-Yang Lin, You-En Wei, and Chun-Hao Chen Status Quo Bias in Configuration Systems . . . . . . . . . . . . . . . . . . . . . . . . . . Monika Mandl, Alexander Felfernig, Juha Tiihonen, and Klaus Isak Improvement and Estimation of Prediction Accuracy of Soft Sensor Models Based on Time Difference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hiromasa Kaneko and Kimito Funatsu Network Defense Strategies for Maximization of Network Survivability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Frank Yeong-Sung Lin, Hong-Hsu Yen, Pei-Yu Chen, and Ya-Fang Wen PryGuard: A Secure Distributed Authentication Protocol for Pervasive Computing Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chowdhury Hasan, Mohammad Adibuzzaman, Ferdaus Kawsar, Munirul Haque, and Sheikh Iqbal Ahamed

95 105

115

125

135

Section 4: Feature Extraction, Discretization, Clustering, Quantization, and Data Refinement A Global Unsupervised Data Discretization Algorithm Based on Collective Correlation Coefficient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . An Zeng, Qi-Gang Gao, and Dan Pan A Heuristic Data-Sanitization Approach Based on TF-IDF . . . . . . . . . . . . Tzung-Pei Hong, Chun-Wei Lin, Kuo-Tung Yang, and Shyue-Liang Wang Discovering Patterns for Prognostics: A Case Study in Prognostics of Train Wheels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chunsheng Yang and Sylvain L´etourneau

146 156

165

Section 5: Applications of Artificial Intelligence Automating the Selection of Stories for AI in the News . . . . . . . . . . . . . . . Liang Dong, Reid G. Smith, and Bruce G. Buchanan

176

Table of Contents – Part I

Diagnosability Study of Technological Systems . . . . . . . . . . . . . . . . . . . . . . . Michel Batteux, Philippe Dague, Nicolas Rapin, and Philippe Fiani Using Ensembles of Regression Trees to Monitor Lubricating Oil Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Andres Bustillo, Alberto Villar, Eneko Gorritxategi, Susana Ferreiro, and Juan J. Rodr´ıguez

XIX

186

199

Section 6: Image Processing and Other Applications Image Region Segmentation Based on Color Coherence Quantization . . . Guang-Nan He, Yu-Bin Yang, Yao Zhang, Yang Gao, and Lin Shang

207

Image Retrieval Algorithm Based on Enhanced Relational Graph . . . . . . Guang-Nan He, Yu-Bin Yang, Ning Li, and Yao Zhang

220

Prediction-Oriented Dimensionality Reduction of Industrial Data Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Maciej Grzenda Informative Sentence Retrieval for Domain Specific Terminologies . . . . . . Jia-Ling Koh and Chin-Wei Cho

232

242

Section 7: Intelligent Techniques for Document Processing Factoring Web Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David W. Embley, Mukkai Krishnamoorthy, George Nagy, and Sharad Seth

253

Document Analysis Research in the Year 2021 . . . . . . . . . . . . . . . . . . . . . . . Daniel Lopresti and Bart Lamiroy

264

Markov Logic Networks for Document Layout Correction . . . . . . . . . . . . . Stefano Ferilli, Teresa M.A. Basile, and Nicola Di Mauro

275

Extracting General Lists from Web Documents: A Hybrid Approach . . . . Fabio Fumarola, Tim Weninger, Rick Barber, Donato Malerba, and Jiawei Han

285

Section 8: Modeling and Support of Cognitive and Affective Human Processes Towards a Computational Model of the Self-attribution of Agency . . . . . Koen Hindriks, Pascal Wiggers, Catholijn Jonker, and Willem Haselager

295

XX

Table of Contents – Part I

An Agent Model for Computational Analysis of Mirroring Dysfunctioning in Autism Spectrum Disorders . . . . . . . . . . . . . . . . . . . . . . . Yara van der Laan and Jan Treur Multi-modal Biometric Emotion Recognition Using Classifier Ensembles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ludmila I. Kuncheva, Thomas Christy, Iestyn Pierce, and Sa’ad P. Mansoor Towards a Fully Computational Model of Web-Navigation . . . . . . . . . . . . . Saraschandra Karanam, Herre van Oostendorp, and Bipin Indurkhya

306

317

327

Section 9: Robotics and Navigation Stairway Detection Based on Single Camera by Motion Stereo . . . . . . . . . Danilo C´ aceres Hern´ andez, Taeho Kim, and Kang-Hyun Jo Robot with Two Ears Listens to More than Two Simultaneous Utterances by Exploiting Harmonic Structures . . . . . . . . . . . . . . . . . . . . . . . Yasuharu Hirasawa, Toru Takahashi, Tetsuya Ogata, and Hiroshi G. Okuno Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

338

348

359

Environmental Sound Recognition for Robot Audition Using Matching-Pursuit Nobuhide Yamakawa1 , Toru Takahashi1 , Tetsuro Kitahara2, Tetsuya Ogata1 , and Hiroshi G. Okuno1 1

Graduate School of Informatics, Kyoto University, Kyoto, Japan {nyamakaw,tall,ogata,okuno}@kuis.kyoto-u.ac.jp http://winnie.kuis.kyoto-u.ac.jp/members/nyamakaw 2 Department of Computer Science and System Analysis, College of Humanities and Sciences, Nihon University, Tokyo, Japan [email protected] http://virgo.cssa.chs.nihon-u.ac.jp/~ kitahara/index-e.html

Abstract. Our goal is to achieve a robot audition system that is capable of recognizing multiple environmental sounds and making use of them in human-robot interaction. The main problems in environmental sound recognition in robot audition are: (1) recognition under a large amount of background noise including the noise from the robot itself, and (2) the necessity of robust feature extraction against spectrum distortion due to separation of multiple sound sources. This paper presents the environmental recognition of two sound sources fired simultaneously using matching pursuit (MP) with the Gabor wavelet, which extracts salient audio features from a signal. The two environmental sounds come from different directions, and they are localized by multiple signal classification and, using their geometric information, separated by geometric source separation with the aid of measured head-related transfer functions. The experimental results show the noise-robustness of MP although the performance depends on the properties of the sound sources. Keywords: Environmental sound recognition, Matching pursuit, Robot audition, Computational auditory scene analysis.

1

Introduction

The physical control technology for robots has progressed remarkably in recent years. However, their perceptive ability to understand the real world, especially its auditory scenes, is still in its infancy. For this reason, a number of researchers have been developing the technologies, namely robot audition, that enable robots to understand real auditory scenes [1,2,3]. The most important feature in robot audition is for robots to be able to recognize auditory scenes with their own ears, that is, with the microphones installed on the robots themselves. Although various spoken dialogue robots have been developed, most of them receive sound only from speakers in a wearable headset in order to eliminate the influence of K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 1–10, 2011. c Springer-Verlag Berlin Heidelberg 2011 

2

N. Yamakawa et al.

background noise as much as possible [4,5,6]. In contrast, robot audition research aims at the use of microphones built into the body of robots so that they can detect the signals from unspecified sound sources. However, this feature makes the problem more difficult. First, a variety of sounds sometimes occur simultaneously. When multiple sounds are observed simultaneously, the characteristics of each sound often interfered with each other, so the recognition of these sounds is quite degraded. Second, there is a long distance between sound sources and the microphone in general. When sound sources are far from the microphone, the observed signals may become reverberant and have a low signal-to-noise ratio. Due to these difficulties, robot audition usually involves three elemental technologies: sound source localization, sound stream separation, and sound source recognition. Although these technologies were developed independently, attempts to integrate them have recently been made. For example, the robot audition software, HARK [7], has been developed for that purpose. Although the sounds presented in the world are obviously not limited to human speech, most robot audition studies have focused only on speech signals. When HARK detects non-speech sounds, for example, it ignores those sounds, and does not pass them to the recognition stage. Although rejecting non-speech sounds may be a possible solution for spoken dialogue robots, we believe it is not sufficient because such sounds may also contain relevant information. In human-human conversation, non-speech signals existing in scenes can indicate its meaningful sign strongly related to the environment. For example, if an alarm rings while people are chatting, one of the speakers might say “what is that disturbing sound?” and the other(s) would then identify the sound as coming from the alarm over there. Finally, one of them might stop the alarm and the chat would resumes. In order to achieve this kind of interaction between humans and robots, robots need to have the ability to recognize such non-speech sounds (environmental sounds). There are various studies on environmental sound recognition indirectly related to robot audition. Ikeda [8] and Jahns [9] developed a system to recognize psychological conditions of animals by means of audio features of their voices, e.g., linear predictive coding (LPC), and pattern recognition approaches, e.g., hidden-Markov model (HMM). To make mobile devices be capable of recognizing background sound and aware of the scene they are in, the research about audio context recognition has been carried out by several researchers [10,11]. Another main interest in environmental sound recognition is discriminating abnormal sounds from other environmental sounds. Here, the word “abnormal” refers to unknown sounds indicating unusual or emergency situations. Some security applications, e.g., home-security robots [12], are based on this classification. Although some of them are aimed at applications for mobile robots, those studies were not conducted on the assumption that in the real environment multiple sounds come from various directions with a large amount of background noise or reverberation. To achieve environmental sound recognition under such conditions, the ability to localize objective sound sources, separate them from background noise and identify them is required.

Environmental Sound Recognition for Robot Audition Using MP

3

To evaluate the functionality of the robot audition technologies in the situation where a robot has to recognize various sound events simultaneously under noisy environments, we conducted experiments on the identification of dual sound-events with an added background noise employing matching-pursuit (MP) and the sound-source localization and separation technique used in HARK. An eight-channel microphone array embedded on a semi-spherical robot head is used to localize sound sources and separate two environmental sounds occuring simultaneously. Each separated signal is then identified using MP with Gaussian mixture models (GMMs) or HMMs. The identification performance of MP is compared with mel-frequency cepstral coefficients (MFCCs), which is the most widely used speech recognition scheme. The contents of this paper are organized as follows: In Section 2, the problems in environmental sound recognition for robot audition systems are summarized, and the properties and theory of MP are explained briefly. In Section 3, the details of our experiments are described, and a conclusion and our planned future work are given in Section 4.

2

Environmental Sound Recognition by Robot Audition

There are two types of environmental sounds. The first type is what can be called sound events, that is, short sounds that indicate certain events or that something is occurring. For example, the sound of a doorknob indicates that someone is entering or leaving the room, and a chime indicates a certain time (e.g., the start of a class). A feature of this kind of sound is that durations are very short, and short durations may result in localization errors. The second type is relatively stationary background noise. Examples are the noise made by appliances such as air conditioners and the raucous atmosphere of a bar. Whereas the noise of air conditioners is nothing more than noise in general, the noise of a raucous atmosphere could be a cue for identifying where the robot is. The background noise also includes the noise produced by the robot itself. Because the two types of sounds usually occur simultaneously, robots have to identify these sounds after separating them. Based on the discussion above, we assume the following conditions in our environmental sound recognition task: 1. Sound events occur under some background noise with various signal-to-noise ratios (SNRs) (including 0 dB). 2. White noise is used as background noise to simulate the fan-noise of the robot itself. Ideally we should deal with various kinds of background noise including that in bars as discussed above, but we leave them as a future task. 3. The sounds are observed through the eight-channel microphone array installed in the body of the robot. 2.1

Issues in Feature Extraction of Environmental Sounds

In our task, observed sounds are mixtures of sound events and background noise. The difficulty of this task lies in the fact that both signals and noise are

4

N. Yamakawa et al.

environmental sounds. This means that the acoustical characteristics of signals cannot be exploited as cues in separating signals from noise. Whereas the harmonic structure of human speech is widely used as a cue in separating speech signals from noise, the harmonic structure cannot be assumed for environmental sounds. This means that sound source separation is extremely difficult, so we have to design a feature extraction method under the assumption that the signals and noise cannot be sufficiently separated in the source separation phase. From this point of view, mel-frequency ceptral coefficients (MFCCs), which are the most common audio features in the conventional sound recognition method, are considered unsuitable. This is because MFCCs approximate spectral envelopes of a signal with the same weights for all frequencies, and do not discriminate the signal and the noise. Compared to background noises, the energy of sound events tends to be concentrated locally in the time-frequency plane. In addition, sound events are expected to be presented more saliently compared to background noise because the sound events are meant to convey information to someone who hears them. A mechanism for extracting salient components from the time-frequency representation is therefore required for achieving noise-robust feature extraction. 2.2

Feature Extraction Using Matching Pursuit

We use matching pursuit (MP) as an audio feature extraction algorithm to spot salient components in a signal in the time-frequency domain. The algorithm extracts signal components from a given signal in order of the highest energy. The extracted components are expressed as atoms defined in an arbitrary base dictionary, and thus the signal is approximated as a linear sum of the atoms ordered according to energy. The number of extractions is also arbitrary; therefore, by selecting the number carefully, the algorithm can extract salient signal components in the time-frequency domain, and noise robustness for audio features is expected. We obtain an approximation of a given signal s as a linear sum of m atoms, φγ1 ...φγm , where m is an arbitrary number; then s can be expressed as: m  αγi φγi + R(m) . (1) s= i=1

Note that R represents a residual signal, and the atoms are selected from the dictionary D = {φγ1 ...φγm } which contains m (≥ m) atoms. The extraction procedure is as follows: 1. Calculate correlation of s and all the atoms in D. 2. Extract the atom with the maximum correlation φγ1 . from s together with the correlation coefficient αγ1 3. Apply the same procedure as 1. to R(1) = s−αγ1 φγ1 , and αγ2 φγ2 is obtained. 4. Iterate the procedure above until m atoms are decomposed. Each atom may have parameters such as width, frequency and amplitude, depending on the type of atom. Accordingly, these parameters can be used as

Environmental Sound Recognition for Robot Audition Using MP

White noise SNR = 0dB

5

Extraction MP/Gabor

Fig. 1. Signal extraction by matching-pursuit with the Gabor wavelet

feature vectors. The computational complexity of the extraction increases linearly with respect to m. Therefore, if the dictionary is designed effectively and the meaningful features are gained with a small number of m, the complexity does not noticeably increase. More theoretical details are described in [13]. Robustness against background noise. Figure 1 illustrates the MP extraction of high energy components (i.e., salient audio features) from an audio signal immersed in a Gaussian noise signal. The signal is a chime ringing sound which has a clear harmonic structure. Its energy is concentrated at around 3500 Hz. The left figure shows the spectrogram of the original signal, which is dry and contains no background noise (i.e., no reverberation or acoustical coloration). White noise with the SNR of 0 dB is added to the original signal, and its spectrogram is plotted in the figure at the center. MP with the Gabor wavelet extracts 48 Gabor atoms from the entire noisy signal (i.e., not a frame-wise analysis) and finally, the signal is reconstructed from the extracted atoms. The reconstructed signal is presented in the right figure. In the center figure, the spectra are almost masked by the noise at every analysis frame. If we extract audio features based on the Fourier transform from the noisy signal, the values of the feature differ substantially from those in the original signal. Therefore, audio feature extraction methods using short-time Fourier transform (STFT), which is a popular technique in the conventional sound recognition scheme, may perform poorly in the presence of such large background noise. The point here is that the reconstructed signal shown in the right figure contains only high energy components around 3500 Hz and no background noise. Thus, if the sustaining peak of spectrum power at 3500 Hz largely characterizes the signal, it can be said that MP succeeded in extracting audio features while eliminating the influence of background noise in this example. This property contributes to attaining noise-robust audio features.

3

Experiments

The experimental results of environmental sound identification are presented in this section. Practically, a robot surrounded by various sound sources and background noises needs to be capable of sound source localization and separation in order to recognize each sound source individually. The objective of our experiments is to investigate the effectiveness of applying MP with the Gabor wavelet to environmental sounds with effects of such signal processing technologies. A

6

N. Yamakawa et al.

robota head is used to create head-related transfer functions (HRTFs) for localization and separation so that the experiments involve robot audition under a more realistic condition. The procedure of our experiments was as follows: 1. make an eight-channel audio signal containing two directional environmental sounds with white-noise, 2. localize and separate the sounds using HARK and obtain two monaural sound files, 3. extract audio features of MFCCs and MP/Gabor, 4. learn multi-condition acoustical models using feature vectors of clean, noised and separated signals, 5. conduct 10-fold cross validation using the separated signals as test data. MP can be applied to the entire signal (a sound unit) or to each analytic frame of the sound. The difference in identification performance between unit-wise or frame-wise analysis is still unknown. We compare of both analysis types in the second experiment. 3.1

Setup

Head-related Transfer Functions of the Robot Head. For sound source localization and separation, we used HRTFs of a real robot with an eight-channel microphone array on its semi-sphere head. The HRTFs were made by the directional impulse responses recorded in an anechoic chamber. The responses were measured at points 150 cm away from the virtual sound sources at every 5 degrees (i.e., 72 HRTFs were made to cover 360◦ ). Localization and separation by HARK. We used the robot audition software HARK to conduct sound source localization and separation. The algorithms selected for localization and separation were multiple signal classification (MUSIC)[14] and geometric source separation (GSS)[15], respectively. Both are available as modules, LocalizeMUSIC and GHDSS, in HARK. To optimize the localization performance for environmental sounds, we set the parameter PERIOD, which represents the number of frames used in the calculation of correlation matrices, to 20, as environmental sounds include impulsive sounds, and hence require a short period of localization. Additionally, since the band width of environmental sounds is generally broader than that of speech signals, LOWER BOUND FREQUENCY and UPPER BOUND FREQUENCY were changed from “500 and 2800 Hz” to “125 and 7900 Hz” so that the system could handle a wider frequency-band range. Pilot tests indicated that the localization performance for environmental sounds was improved by this parameter setup for the modules. Specification of sound classes, audio features, and classifiers. Table 1 summarizes the specification of the audio features, classifiers, and sound classes as well as the training and testing combination. The audio signals used in the experiments were excerpted from the “Sound scene database in real acoustical environments” created by the Real World Computing Partnership (RWCP) [16].

Environmental Sound Recognition for Robot Audition Using MP

7

Table 1. Experimental setup (sound classes, audio features, classifiers) MFCC

Audio features

Classifiers Sound classes

MP/Gabor

MFCC(12) + ΔMFCC(12) + Δpower(1) Window width = 25 msec Shift rate = 10 msec GMM (4‐mix.) or HMMs (6 hidden states)

Frame

Unit

Window‐width = same as MFCC Base‐width = 2‐256 samples

Base‐width = 2‐4096 samples

time‐shift resolution = 16 samples # of extracted atoms = 48 GMM for both or HMMs only for "frame"

one at 0 degree (in front) and the other at 45 degree off chime, handclap, clock‐alarm, coin‐clinks, glass‐cups, doorknob, metal‐plate, phone‐beep, bell‐ringing, metal‐bin

Leaning set

Original signal (clean) + Noised signals (SNR = 0dB) + Noised separated signals = 900 samples

Test set

Separated signals = 100 samples

We selected 10 sound classes sampled at 16 kHz with a time length of 0.7 sec on average from the database. All classes were impulsive sounds (i.e., fast-decaying) except clock-alarm and phone-beep. The number of MP extractions and hidden states of HMM were chosen to be 48 and 6 as these numbers were found to be effective for environmental sounds in our previous study [17]. The training condition involved a combination of multi-condition learning consisting of clean signals (dry sources), the same signals with white noise (SNR = 0 dB) and the separated signals of the noised signals. Each condition contained 300 samples; therefore a training set has 900 samples in total. In the third condition, although there was only one sound source and no stream separation was necessary, we applied GSS to the samples and included them in training sets so that acoustic models could learn not only the sound classes’ features but also the filter-like property of the separation process. 3.2

Experiment 1: Identification Rate of MFCCs and MP

Figure 2 shows the result of the identification test performed with MFCCs and MP/Gabor with two classifier conditions (GMM and HMM of six hidden states). The vertical axis displays identification rates as percentages and the horizontal axis lists the names of the sound classes. This result is for the sounds coming from the 45◦ direction. Since the result does not dramatically differ from that of the 0◦ direction, we concluded that there was no directional dependency in the identification performance in our experiment set. Overall, our method, the frame-wise MP with the Gabor wavelet, outperformed the MFCCs, and generally, ones with HMM showed higher identification

8

N. Yamakawa et al.

/ĚĞŶƚŝĨŝĐĂƚŝŽŶƌĂƚĞ΀й΁ ƚŝŽŶƌĂƚĞ΀й΁

D&ͬ'DD

D&ͬ,DD

DW;&ƌĂŵĞͿͬ'DD

DW;ĨƌĂŵĞͿͬ,DD

ϭϬϬ ϵϬ ϴϬ ϳϬ ϲϬ ϱϬ ϰϬ ϯϬ ϮϬ ϭϬ Ϭ

Fig. 2. Identification rate of separated sound sources coming from 45◦ using MFCCs/GMM, MFCCs/HMMs, MP(frame)/GMM and MP(frame)/HMMs

rate except for doorknob and bell-ringing. The common property between these two classes is that they originally had peaks of spectrum power higher than 7000 Hz. Due to the LPF-like property of GSS, the power at higher frequency components is attenuated. If the original signal has large energy in the stop-band of the filter, a significant change can be seen in feature vectors created by MP. This is because for separated sound signals, the energy order of signal components can be changed from the original, and more atoms are extracted at the lower frequency region and fewer from the attenuated region. This results in a large difference between separated and non-separated signals in the MP feature space. Because GMM does not factor in the transition patterns of feature vectors, signal components that appear most frequently through all frames are consequently dominant features in the model. In the case of impulsive sounds generated by a collision of two materials, lower frequency components sustain longer while higher frequency ones decay faster. Because of this phenomenon, a GMM can intensively learn the lower frequency features that are commonly strong in nonseparated and separated signals without considering frame sequence like HMMs. Whereas some classes such as chime and glass-cups showed a high identification rate (nearly or above 90%), the others exhibited a rate around 50%. The sound groups with higher performance results have clear harmonic structures, and the states of the structures (i.e., pitch) do not vary significantly over the same sound class. The sound groups with poor performance results generally have unclear harmonic structures or variant pitch information. For example, the phone-beep sound samples consist of recorded beep sounds of various cell phones; hence, the pitch of the beeps has variations. In these cases MP features using frequency values are not consistent, and are therefore not effective audio features for these types of environmental sounds. 3.3

Experiment 2: Part or Unit?

Figure 3 presents the result of the identification test carried out by applying MP to each analytic frame or to a whole signal (i.e., a sound unit).

Environmental Sound Recognition for Robot Audition Using MP

/ĚĞŶƚŝĨŝĐĂƚŝŽŶƌĂƚĞ΀й΁

DW;ĨƌĂŵĞͿͬ'DD

DW;ĨƌĂŵĞͿͬ,DD

9

DW;ƵŶŝƚͿͬ'DD

ϭϬϬ ϵϬ ϴϬ ϳϬ ϲϬ ϱϬ ϰϬ ϯϬ ϮϬ ϭϬ Ϭ

Fig. 3. Comparison in identification rate of frame-wise and unit-wise MP features

In most cases, the unit-wise analysis performed similarly to the frame-wise with HMMs of six hidden states but failed to identify the phone-beep and the bell-ringing due to the reason discussed in Experiment 1. The unit-wise analysis can only generate one feature vector; therefore, it cannot be classified by HMM, so is classified by GMM. Because GMM models signals independent relationships between sequential frames, it can handle various sound classes more generally. It is remarkable that the unit-wise analysis with GMM shows almost the same performance as the frame-wise one in some sound classes. In other words, the unit-wise MP analysis can express temporal variation of a signal in a single feature vector.

4

Conclusion

This paper presented identification tests of two environmental sounds fired simultaneously by robot audition using matching pursuit with the Gabor wavelet. The sound source localization and separation were performed by the robot audition software, HARK, and its specialized configuration for environmental sound recognition was explored. The experimental results revealed the noise robustness of MP against MFCCs noting that not all properties of environmental sounds can be accurately identified by MP. It is also found that the unit-wise analysis of MP can model temporal variations as well as the harmonic structure of a signal. Our future objectives are planned as follows: first, it can be easily imagined that a speech signal and meaningful environmental sounds are recorded simultaneously in human-robot interaction. Identification experiments that separate a human voice and environmental sounds should be conducted to determine whether the proposed method is still valid for that condition. Second, as most furniture or electrical appliances usually stay in the same position in a home, it may be useful for a robot to refer to a map of their distribution when localizing and recognizing environmental sounds. For example, with the help of a map, the robot can determine whether the input signal comes from a familiar or unknown

10

N. Yamakawa et al.

sound source, and then judge the abnormality of a sound event or subsequently renew the map.

References 1. Rosenthal, D.F., Okuno, H.G.: Computational auditory scene analysis. L. Erlbaum Associates Inc., Mahwah (1998) 2. Brown, G., Cooke, M.: Computational auditory scene analysis. Computer Speech and Language 8(4), 297–336 (1994) 3. Okuno, H.G., Ogata, T., Komatani, K.: Computational Auditory Scene Analysis and Its Application to Robot Audition: Five Years Experience. In: ICKS 2007, pp. 69–76 (2007) 4. Matsusaka, Y., Tojo, T., Kuota, S., Furukawa, K., Tamiya, D., Hayata, K., Nakano, Y., Kobayashi, T.: Multi-person conversation via multi-modal interface — a robot who communicates with multi-user. In: EUROSPEECH 1999, pp. 1723–1726 (1999) 5. Nishimura, R., Uchida, T., Lee, A., Saruwatari, H., Shikano, K.: Aska: Receptionist robot with speech dialogue system. In: IROS 2002, pp. 1308–1313 (2002) 6. Brooks, R., Breazeal, C., Marjanovie, M., Scassellati, B., Williamson, M.: The cog project: Building a humanoid robot. In: Computation for Metaphors, Analogy, and Agents, pp. 52–87 (1999) 7. Nakadai, K., Takahashi, T., Okuno, H.G., Nakajima, H., Hasegawa, Y., Tsujino, H.: Design and Implementation of Robot Audition System’HARK’Open Source Software for Listening to Three Simultaneous Speakers. Advanced Robotics 24 5(6), 739–761 (2010) 8. Ikeda, Y., Jahns, G., Kowalczyk, W., Walter, K.: Acoustic Analysis to Recognize Individuals and Animal Conditions. In: The XIV Memorial CIGR World Congress, vol. 8206 (2000) 9. Jahns, G.: Call recognition to identify cow conditions–A call-recogniser translating calls to text. Computers and Electronics in Agriculture 62(1), 54–58 (2008) 10. Eronen, A.J., Peltonen, V.T., Tuomi, J.T., Klapuri, A.P., Fagerlund, S., Sorsa, T., Lorho, G., Huopaniemi, J.: Audio-based context recognition. IEEE TASLP 14(1), 321–329 (2005) 11. Chu, S., Narayanan, S., Kuo, C.: Environmental sound recognition with timefrequency audio features. IEEE TASL 17(6), 1142 (2009) 12. Ntalampiras, S., Potamitis, I., Fakotakis, N.: Sound classification based on temporal feature integration. ISCCSP 2010, 1–4 (2010) 13. Mallat, S.G., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE TSP 41(12), 3397–3415 (1993) 14. Schmidt, R.: Multiple emitter location and signal parameter estimation. IEEE TAP 34(3), 276–280 (1986) 15. Parra, L.C., Alvino, C.V.: Geometric source separation: Mergin convolutive source separation with geometric beamforming. IEEE TSALP 10(6), 352–362 (2002) 16. Real World Computing Partnership, Rwcp sound scene database in real acoustical environments, http://tosa.mri.co.jp/sounddb/indexe.htm 17. Yamakawa, N., Kitahara, T., Takahashi, T., Komatani, K., Ogata, T., Okuno, H.G.: Effects of modelling within-and between-frame temporal variations in power spectra on non-verbal sound recognition. In: INTERSPEECH 2010, pp. 2342–2345 (2010)

Cognitive Aspects of Programming in Pictures Yutaka Watanobe, Rentaro Yoshioka, and Nikolay Mirenkov University of Aizu, Aizu-wakamatsu, Fukushima 965-8580, Japan {yutaka,rentaro,nikmir}@u-aizu.ac.jp

Abstract. Programming in pictures is an approach supported by a special programming environment where pictures and moving pictures are systematically used for direct representation of features of computational algorithms and data structures. Within this approach some ”data space” is traversed by a ”front of computation” and necessary operations are performed during this traversal process. There are compound pictures to define algorithmic steps (called Algorithmic CyberFrames) and generic pictures to define the contents of compound pictures. Compound pictures are assembled into special series to represent some predefined algorithmic features. A number of the series is assembled into an Algorithmic CyberFilm. The environment helps to follow a cognitive model where success of users’ perception, comprehension and cognition depend on interaction with, at least, a few different but mutually supplementing features of a picture or a set of pictures. In this paper, we survey a number of techniques that are used with programming in pictures to enhance perception and cognition of people working on developing computational application models and corresponding algorithms. Keywords: programming in pictures, Algorithmic CyberFilm.

1

Introduction

The natural intelligence is related, in a great part, with people’s abilities to perform mental simulation, including thinking/reasoning, problem solving, and decision making. However, this mental simulation depends not only on the internal information processing mechanism of the brain, but also on the levels of abstraction connecting the physical world objects and corresponding abstract models. Syntax-semantics gap in the object-model representations is a fundamental basis for many problems related to the perception and cognition. This gap is predefined by existing systems of symbols, notations, languages, as well as by ambiguous terminologies, trans-disciplinary and cross-cultural contexts. It also depends on the representation of dynamical processes of modeling problems and the adaptive mechanisms of working systems. To bridge the syntaxsemantics gap and decrease burden of mental simulation, modern information systems are supported by special data/knowledge acquisitions, ontologies and high-level interfaces based on different mental strategies related to different decision functions [1-2]. In general, to enhance people’s ability in understanding meaning of objects and processes and in performing mental simulation, there is K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 11–20, 2011. c Springer-Verlag Berlin Heidelberg 2011 

12

Y. Watanobe, R. Yoshioka, and N. Mirenkov

a necessity not only to increase the levels of symbols, notations and languages, but also to update our current concept of abstraction. Some new programming languages (first of all related to developing human-computer interfaces) try to react to the above mentioned situation [3], but majority of others still keep the old level of cognitive features by being ”good for computers, but not so friendly for humans.” Here, among related papers we would like to mention cognitive aspects of software design [4], where authors try to find solutions by considering answers to the following questions: is it possible to bring together theoretically a computer program and a natural language text, and what are differences between programming languages and natural languages. Another paper of our interests is [5], where cognitive and computational features of diagrams are considered through another question: why a diagram is (sometimes) worth ten thousand words. In addition, we would like to mention approaches related to analogical representations of programs (including a semiotic analysis of comics)[6], selfassembling tilings (for the creation of the desired shapes)[7], and the long-term activity of researchers in Certec (Sweden) [8] (where the use of pictures as language for people with disabilities is promoted), as well as other variety of publications where the concept of putting human needs first is applied and where a necessity for uniting scientific and artistic sides to obtain a balance between usefulness and attractiveness of technologies is also considered [9]. Our motivation is to develop a programming environment where users can feel themselves comfortable in developing new models and corresponding algorithms, and in representing them in forms recognizable and understandable by other people. In this environment, the confidence of users in making decisions is based on access to previous decisions of others, on quick perception and understanding and on transformations leading to new models and algorithms. Such opportunities are supported by the use of an open set of very-high level algorithmic characters (super-characters), special constructs combining these characters to represent algorithmic steps, special assembly lines to define algorithmic scenes, and libraries (in fact, ontologies) to acquire such super-characters, constructs and algorithm representations. Pictures and moving pictures are applied as expressive forms of these super-characters, constructs, and lines. In many cases pictures and animation are much more attractive for understanding an algorithm (and its features) in comparison with conventional programming text. However, there are cases where pictures are still too abstract and animation is too boring. So, our approach is based on multiple views and multimodal formats to take into account different knowledge and abilities of users. Within this approach, we follow a cognitive model where success of user’s perception, comprehension and cognition depend on interaction with, at least, a few different but mutually supplementing features of a super-character, construct, or algorithm. In this paper we survey a number of techniques that are used with programming in pictures to enhance perception and cognition of people working on developing application models and corresponding algorithms. These techniques are based on special visual symbols and their compositions within compound constructs, on the use of background, foreground and auxiliary images, and on

Cognitive Aspects of Programming in Pictures

13

representing scalable forms of model spaces and attributes declarations, features of algorithmic steps, and multiple views of algorithm as a whole.

2

Programming in Pictures: A Brief Overview

As we have mentioned in the introduction, mental simulation depends not only on internal information processing mechanisms of the brain, but also on levels of abstraction connecting the physical world objects and corresponding abstract models. To make such simulation more efficient, it is necessary to decrease spending of people’s energy and time for recognition of semantics behind unknown terminology, for discovering associations between objects with syntax-semantics forms based on rote memorization, and for fighting with concepts and notations that have a trend to be over-abstracted. In other words, disorientation and cognitive overload are serious obstacles for people. They become tired after overcoming such obstacles and do not have enough energy and time for focusing on their own ideas and models, and on the use of their real knowledge and cognitive abilities. Programming in pictures is a special approach to help people in saving their energy and time. It is supported by a special environment where pictures and moving pictures are systematically used for more direct representation of features of computational algorithms and data structures. Within this approach some ”data space” is traversed by a ”front of computation” and necessary operations are performed during this traversal process. There are compound pictures to define algorithmic steps (called algorithmic CyberFrames) and generic pictures to define the contents of compound pictures. Compound pictures are assembled into special series to represent some predefined algorithmic features. A number of the series is assembled into an Algorithmic CyberFilm. Fig.1 depicts a structure scheme of the Algorithmic CyberFilm format. The first (leftmost) series of frames represents algorithmic skeletons that show space data structures and temporal schemes of computational flows on these structures. The second series shows variables and formulas (actions) that are attached to the space-time points of the algorithmic skeletons. The third series represents input/output operations that define the algorithmic interface with

D[ P[

D[

]= GR [ P[ ]= nod e( D[

4 Languages: Algorithmic Dynamics Algorithmic Commands Algorithmic interFace Integrated View

P[

D[ P[

GR, P ]= GR ]= nod

[

e(

]; )

GR, P T

]; T[

)

T[

GR, P ]= D[ ] + 1; ]= nod e( )

]= ] + T[

]

S -1

GR, P

-1 -1

]= D[ ] + 1; ]= nod e( )

-1

GR D[

]=

GR[

GR D[ P[

];

]= D[

] + 1; ]= nod e( )

Fig. 1. A structure scheme of the Algorithmic CyberFilm format

14

Y. Watanobe, R. Yoshioka, and N. Mirenkov

external world, as well as how a software component based on the algorithm should look from outside. Finally, the fourth series shows a compact combination of main features presented in the above mentioned groups of frames. For each series of frames there is its own visual language. As a result, the programming environment for making Algorithmic CyberFilms is a cluster of four mutually supplemented languages (and supporting systems) to define four different views of each software component. Each language uses its own set of pictures and super-characters. Within this environment, the application users usually do not create frames, algorithmic skeletons and related template programs, but take and assemble them from ontology type libraries. For these users, the programming is reduced to defining sizes of the space structures, declaring variables on these structures and specifying operations on traversal nodes. Filmification of methods has been applied to a great variety of algorithms including sequential and parallel matrix multiplications, solving algebraic and partial differential equations, cellular automation-like algorithms, as well as to algorithms on trees, pyramids, particles-in-cells, etc. (see, for example, [10-12]). A large set of algorithms on graphs and a corresponding library have been presented in [13]. In this paper we focus on some cognitive aspects of programming in pictures and how it is supported by the environment.

3

Integrated View of a Program in Pictures

To illustrate some of the above mentioned features of the Algorithmic CyberFilm technology, let us consider pictures of a program for the Jacobi relaxation algorithm for Laplace’s equation ∂ 2 A/∂x2 +∂ 2 A/∂y 2 = 0 on a 2D grid of N × M size with some values on the boundary of the domain [14]. Variable B is used for keeping the new iteration values and variables e and I are for checking the process of convergence. Variables A, B and I are attached (”affiliated”) to the 2D structure specialized on presenting computation distributed in space and e is attached to a special one-node structure specialized for presenting centralized computation. The picture below (Fig.2(a)) is an integrated view frame (compound picture) defining the algorithm. The first row in the frame is the declaration of the space structures and variables (including their types) attached to the structures. The computation includes three scenes for boundary values initialization, internal node values input and the convergence parameter initialization, and one scene for the main part of the algorithm and results visualization. Each scene has its own super-character (icon) with a ”caption” pointing operations which should be defined within the scene computation. The scene icons are depicted by the left column of the frame and request (from top-to-bottom) specifying 1, 1, 1, and 4 operations, respectively. The operations are defined by the expressions presented to the right of the icons. The first scene is scanning boundary nodes of the 2D grid and performing in the node of visit the rightly defined formula (a highlighted node with vertical/horizontal line means a number of row/column where the node of visit is located). Though this scene icon is intuitively understandable, the user can request its semantics at any moment, as shown by Fig.2(c). The similar case with

Cognitive Aspects of Programming in Pictures

15

(a) (b)

(c)

(d)

(e)

Fig. 2. A program in the pictures of the Jacobi relaxation algorithm for Laplace’s equation

the fourth scene icon is provided by Fig.2(e). In fact, in all these cases, frames of Algorithmic Dynamics view are shown. The embedded clarity support is applicable not only to scene icons, but also to other generic and compound pictures, including icons related to the variable declaration. Fig.2(b) depicts examples of physical meaning and measurement unit clarifications for variable A and B (representing heat in Celsius degrees), as well as unfolded forms of neighborhood (Fig.2(d)) where computational activity is performed. In conventional languages, declarations of variables are based on some abstract data constructs where elementary types of ”integer, real, double, character, etc.” play a fundamental role. Usually, essential efforts should be done to imagine and understand such a construct. For example, to represent a multi-grid (pyramid) construct, a number of 2D-arrays of real or double types representing different layers of the pyramid are declared. This declaration does not display a space structure image and does not provide any information about physical meaning of application attributes and units of their measurement. In our programming environment, special images are used as super-characters or as embedded clarity annotations which can be displayed on user’s demands. Fig.3 depicts a set of super-characters applying for the space structure declarations and Fig.4 depicts a set of supportive images applying for embedded clarity annotations. The supportive images of Fig.4 represent a physical attribute name, corresponding unit of measurements and an icon to hint on attribute meaning.

16

Y. Watanobe, R. Yoshioka, and N. Mirenkov L

H

N N

M

N

N

N

N M

N

N

N

Fig. 3. A set of super-characters applying for the space structure declarations

Time hour

Time min

Time sec

temp

Distance

Distance

Distance

volume

Frequency

Velocity

Hz

km/h

area m2

Resistnace

m3

R

Voltage

V

km

m

Mass

Mass kg

Mass g

Angle

Angle

Wavelength

Watt

rad

deg

m

w

Currency

Currency

Currency

Bandwidth

Pressure

euro

yen

Hz

Amount of substance

Current

dall

t

cm

temp

。 C

。 K

mol

A

Pa

Fig. 4. A set of supportive images applying for embedded clarity annotations

The super-characters represent some space structures (related to user’s model) which nodes and/or edges are used for allocation of application attributes (variables). In fact, the attribute elements obtain a ”space affiliation.” Through such structures the user can see more directly relations between his/her model and variables involved, and visually check these relations. The structures represent scalable spaces which sizes are defined by the users. This control of scalability they perform by assignment of necessary values to parameters presented by super-characters. For example, the leftmost super-character in Fig.3 is to declare a 3D space structure which size is defined by values assigned to M, N, and L. In a similar way, the second from the leftmost super-character is to declare a top-less pyramid space which size is defined by values assigned to N and H. For cases, where the structure spaces can be modified during the computation by applying special operations in appropriate nodes, for example, by cutting operations for removing some tree branches, special versions of the super-characters are used. These versions are to explicitly point about such features of modeling processes. Special cases of very dynamical processes are related to computation on particles. The rightmost super-character is to represent a 2D-model, where the number of particles involved is related to parameter N. For majority of the super-characters of the above mentioned type, there is a family of images to show ”natural” sub-structures of a structure; for example, for a tree structure, leaf nodes or a root node are considered as such substructures; for a 2D structure, the first row and the first column, as well as diagonals are also considered as the ”natural” sub-structures. These images are used to declare the variable allocation not on the whole structure, but on its parts. In addition to these images, there are also structure mask images which help in covering some substructures from computational activity and introducing various types of irregularities necessary for imitating complex processes of real world.

Cognitive Aspects of Programming in Pictures

4

17

Background, Foreground and Auxiliary Images

Algorithmic CyberFrames, especially related to dynamical process representations, show some space data structures, traversal schemes on these structures and high-level formulas on the structure nodes. Corresponding visual objects provide direct influence on the generation of executable codes. Examples of such frames are depicted by the first row of Fig.5. The first frame shows just a graph data structure, the second presents a tree with a node highlighted for some operations, and the third shows a grid with four highlighted nodes for identical operations to be performed in parallel. Such frames and ideas behind them are understandable much easier in comparison with corresponding representations in the conventional language texts. However, their perception and cognition features will be much higher if additional background images can be involved. Examples of such images are depicted by the second row of Fig.5. The first frame in this row shows not only a graph data structure, but also a hint that the model considered will be related to Fukushima prefecture territory. The second frame shows that the activity on the tree leaf node has ”a history” presented by a trace sub-tree, and the third frame depicts a hint about parallel computation in the past and possible computation in the future. The background images of these frames do not have any influence on the code generation, but are very important for helping people in understanding corresponding semantics and performing mental simulation. Another example of helping images is depicted by Fig.6, where different colors are used to show different sub-graphs

Fukushima

Fig. 5. Examples of Algorithmic CyberFrames and Background Images

Frame X

assign

Frame X + 1

to

where edge[

] is minimum

Fig. 6. Other Examples of Algorithmic CyberFrames and Background Images

18

Y. Watanobe, R. Yoshioka, and N. Mirenkov

involved and edges between them. In this case, the helping images and their colors are used not only for clarifying the meaning, but also for defining a halfhighlighted operation (that is for the influence on code generation). Frame X represents that some operation on selecting the node activity at the next frame should be done on a top-right half-highlighted node; for making this selection, data from contour highlighted nodes should be taken. Frame X+1 shows a result of the selection that is a node which was moved from dark-color sub-graph into light-color sub-graph and pointed by an arrow. This frame also shows that some operation should be defined for the top-right node based on data from all nodes. In addition to Frame X and Frame X+1, Fig.6 depicts the definition of the selecting operation mentioned with Frame X. This definition means ”assign a counter highlighted node of one sub-graph into another sub-graph where an edge between sub-graphs has a minimum value.” In general, the Algorithmic CyberFrame is a multilayer package of images (see Fig.7) including background, main ground and foreground slices. For example, the arrow in Frame X+1 at Fig.6 is represented in this package by a supportive image slice from the foreground layer. Other examples of CyberFrames with the foreground and background images can be found in [15]. The number of slices in the layers depends on semantics to be explained. For many CyberFrames main ground images are only used, but there are also cases of auxiliary frames where only supportive images are involved. The auxiliary frames are to help in comprehension of CyberScenes and CyberFilms meaning. They can be considered as cover frames, summary or documentation type annotations.

background

main ground

... layers

foreground

... layers

... layers

cyberFrame

Fig. 7. A scheme of multilayer Algorithmic CyberFrame

5

Case Studies: Libraries (Ontologies) of Super-Characters and Algorithms

Algorithmic CyberFilm programming technology is developed as environment based on a set of libraries where acquisition of super-characters, CyberFrames,

Cognitive Aspects of Programming in Pictures

19

CyberScenes and CyberFilms, as well as template programs is performed. Items of these libraries include a number of views presenting a number of different features through multimedia forms. Between different features inside one item and between different items there are a variety of links and associations that support users’ decisions on creating their models and for specifying corresponding algorithms, and for automatic synthesis of programs from CyberFilm specification. The libraries can be considered as ontologies because of their formal basis for searching items, items’ associations, necessary features and other actions related to automatic synthesis of programs and intelligent support of user’ activities. Usually, each library item obtains double ID to encode its features and a URLlike address. Encoding is based on classification of two types related to “what” is the item from external points of view and “how” is it constructed internally. For example, for a whole algorithm, a set of features of input/output data and their (allowed) relations represent “what” code and a set of space structures, traversal schemes on the structures and representative formulas represent “how” code. The search of items in relational database (saving the items) is performed on a combination of features which selection is organized by corresponding browser/editor subsystems. The interface panels of these subsystems help the users in making decision on which links to go and provide some images of model (information) subspaces and library item associations. This decreases chances for the users to be cognitively overloaded and disoriented and empower their cognition through the experience acquired in the libraries. The environment is a mental and computational system that guide, support and enhance the thinking processes of the users. In addition, it mediates the learning activities that stimulate the cognitive processes and engage learners in creating knowledge that represent their understanding and conception of the information. Continuing development of the libraries (by considering new algorithms and forms of their representations) and the use of them in real-life educational processes are considered as our permanent case studies and usability tests for this new environment. Within these studies and tests, not only understanding algorithms and developing programming skill are analyzed, but also cognitive adaptation of the users within their works on application models is considered. Such analysis and consideration are our guidelines for the environment development. The effectiveness of the approach is evaluated, first of all, by a variety of algorithms represented in the picture-based forms (please, see [10-13, 15] and their references). Practically, this variety does not have limitations because the environment is based on an open set of super-characters allowing the extension of the algorithmic alphabet [16], as well as on adding and editing the embedded clarity annotations.

6

Conclusion

A brief overview of programming in pictures has been presented and a number of techniques that are used to enhance perception and cognition of people working on developing computational application models and corresponding algorithms

20

Y. Watanobe, R. Yoshioka, and N. Mirenkov

have been provided. These techniques are based on special visual symbols and their compositions within compound constructs, on the use of background, foreground and auxiliary images, and on representing scalable forms of model spaces and attributes declarations, features of algorithmic steps, and multiple views of algorithm as a whole, as well as on special acquisition of them in ontology type libraries. The interface panels of the environment help the users in making decisions and in decreasing the chances for becoming cognitively overloaded and disoriented. They guide, support and enhance the thinking processes of the users.

References 1. Fensel, D.: Ontologies: A Silver Bullet for Knowledge Management and Electronic Commerce. Springer, Heidelberg (2001) 2. Rasmussen, J., Pejtersen, A.M., Goodstein, L.P.: Cognitive Systems Engineering. John Wiley & Sons, Inc., Chichester (1994) 3. Carroll, J. (ed.): Human-Computer Interaction in the New Millennium. AddisonWesley, Reading (2002) 4. Detienne, F., Bott, F.: Software design - cognitive aspects. Springer, Heidelberg (2002) 5. Larkin, J.H., Simon, H.A.: Why a Diagram Is (Sometimes) Worth Ten Thousand Words. In: Diagrammatic Reasoning: Cognitive and Computational Perspectives. AAAI Press, Menlo Park (1995) 6. Kindborg, M., McGee, K.: Visual programming with analogical representations: Inspirations from a semiotic analysis of comics. Journal of Visual Languages and Computing 18(2), 99–125 (2007) 7. Beckers, F.: Pictures worth a thousand tiles, a geometrical programming language for self-assembly. Theoretical Computer Science 410(16), 1495–1515 (2009) 8. Danielsson, H., Jonsson, B.: Pictures as Language. In: Proceedings of the International Conference on Language and Visualisation, Stockholm, Sweden (2001) 9. Shneiderman, B.: Leonardo’s laptop: human needs and the new computing technologies. MIT Press, Cambridge (2002) 10. Yoshioka, R., Mirenkov, N.: Visual Computing within Environment of Selfexplanatory Components. Soft. Computing Journal 7(1), 20–32 (2002) 11. Mirenkov, N., Vazhenin, A., Yoshioka, R., Ebihara, T., Hirotomi, T., Mirenkova, T.: Self-explanatory components: a new programming paradigm. Int. Journal of Software Engineering and Knowledge Engineering 11(1), 5–36 (2001) 12. Ebihara, T., Mirenkov, N., Nomoto, R., Nemoto, M.: Filmification of methods and an example of its applications. International Journal of Software Engineering and Knowledge Engineering 15(1), 87–115 (2005) 13. Watanobe, Y., Mirenkov, N., Yoshioka, R.: Algorithm Library based on Algorithmic CyberFilms. Knowledge-Based Systems 22(3), 195–208 (2009) 14. Relaxation method, http://en.wikipedia.org/wiki/Relaxation_method 15. Watanobe, Y., Yoshioka, R., Tsukumoto, S., Mirenkov, N.: Filmification of methods: Convex Hull algorithms. In: Proceedings of the 2009 IEEE TENCON Conference, Singapore, pp. 1–6 (2009) (CDROM) 16. Yoshioka, R., Watanobe, Y., Mirenkov, N.: Open set of Algorithmic Characters. In: Proceedings of 10th WSEAS Int. Conference on Applied Computer Science, Iwate, Japan, pp. 327–334 (2010)

An Approach for Smoothly Recalling the Interrupted Tasks by Memorizing User Tasks Kohei Sugawara and Hamido Fujita Iwate Prefectural University, Faculty of Software and Information Science, 152-52 Sugo, Takizawa, Iwate-gun, Iwate, 020-0173, Japan Sangikyo Corporation, 4509, Ikebe-cho, Tsuzuki-ku, Yokohama, 224-0053 Japan [email protected]

Abstract. An approach was previously proposed for assisting users on selecting the displayed multiple objects on their screen. This approach is related to have a useful GUI by dynamic configurable visualization of overlapping displayed objects provide by the user for executing a certain task. Our proposed GUI is associated with metaphor of finding a book of related task from the piled up books of similar tasks with different precision. We considered for improving the assist of user’s finding about issues of restarting interrupted work. This paper describes the proposed idea for resolving issues by modifying indication of the user’s behavior information on this GUI and by applirng the color gradation to the twenty four hour scale.

1 Introduction As internet and IT technologies are providing many resources to users to solve problems by navigating through different files. User opens many files and objects on their screen to manipulate them to execute a certain tasks or edit a document related to different knowledge among these collected from different windows. User may lose his/her way when a certain interruption or else obscures the user work. The connectivity among these overlapped objects would be loosen that would have a certain impact on user to strengthen this connectivity back. We need to provide a mechanism that makes the user always engaged to all objects on the workspace, in active fashion to have these windows participation in transitioning user work. We have defined a conceptual framework for designing an interaction based interface that can estimate user workflow using several parametric views related to user situation or else reflecting the user profiling aspects. The user profiles several parametric status (mental state, user age, gender rank and else) as those participate in the style user utilizing these object. This is needed to estimate the suitable or friendly systematic components that ensemble the interaction scenarios, fitting with a specific situation at hand. The situation evolvement can be reflected into another derivation of additional component resembling articulated type of user interface. Such sort of component articulation evolvement is functional of user situation for best performance in executing simultaneous overlapped object on same task force screen. In the office work, user’s K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 21–26, 2011. © Springer-Verlag Berlin Heidelberg 2011

22

K. Sugawara and H. Fujita

continuous computer operation is often interrupted by other task or own break. The interrupted operation is restarted by reselecting target objects with trusting user’s short-term memory. Since the content of the interrupted operation depends on user’s short-term memory in this situation, the time until recalling previous operation is slow if the interruption time is long. When the previous operation is forgotten, user needs to check all displayed objects. In order to give user the best performance, the GUI that can list the overlapping of the all displayed object window from horizontal view point with using user’s behavior was previously proposed by authors [1]. This proposed GUI has two features. One is the computer user can find out intended object window from hidden object window by other object windows. The other feature is this GUI can assist to restart the interrupted task or past work by giving information of important object window to the user.

2 Consideration of Interrupted Work The previously proposed GUI lists the all displayed object windows that is sorted in ascending order from front side layer to backward side on the screen. The active elapsed time of each displayed object window is memorized in this GUI. The proposed GUI indicates materials for deciding intended object window to user when the interrupted task is restarted because the information of overlapping objects are shown at a glance [2][3]. In order to give more tangible information to user, we consider existing interrupted work by assuming the office work. In the previous paper [4] an Optimization-Ware that is called Cyber Manual was presented as one of IT collaborative system for improving workers’ productivity. Cyber Manual is used as a functional core of the Optimization-Ware in Sangikyo Corporation. It is a database to accumulate the knowledge and know-how of a group or an entire company as well as share the information among division. Cyber Manual consists of assembled document. A document creation work on Cyber Manual is often interrupted by general work or co-related other tasks. This interruption is defined to classify three type cases that are a short-term interruption, a middle-term interruption and a long-term interruption. While a document is edited, this creating task is interrupted a short-term about few minutes by telephone calling or a break or e-mail checking. If a document creation could not be finished in a work day, this interrupted work is restarted after a middle-term that is next day or next week. When the coworker that is taking over the related project to the document from the creator of the document revises the document, the document creation have been interrupted a longterm (including a short term and a middle term) since last editing. A document creator needs to recall what object window was used in order to resume document creation because the creator often uses reference for making document. This paper proposes supporting human’s short-term memory in restarting from short or middle term interruption by indicating overlapping each object windows, active elapsed time and relation of simultaneously used object windows. There is an approach for improving user performance of visualization by using context history [5]. However, the proposed GUI cannot get the context history from used object windows for document creation due to the storing the context history on each application program. So, visualizing a day time is proposed for assisting restarting of an interrupted

An Approach for Smoothly Recalling the Interrupted Tasks by Memorizing User Tasks

23

document creation past day by logging desktop environment of displayed object windows as the history of simultaneously used object windows. In the previously proposed GUI, each object window’s activated time is stored by counting elapsed time while target object window is displayed most front side on the screen. With the longest elapsed time of displayed object as the base in this proposed GUI, the listed items of the displayed objects are colored from the left side by each elapsed time. Therefore, a user can know at a glance which object window how long was used in looking back from a short-term interruption to the document creation work. However, if there is a big difference between the longest active elapsed time of an object window and the other active elapsed time of object windows, the length of each shorter time selected object window becomes similar indication. Even if there is the clear time differences between each object windows, user cannot understand the difference in the large gap of elapsed time and recognize the related object windows.

3 Our Approach In order to assist user’s short-term memory in restarting document creation, the idea that is to indicate relation of each displayed object window in the same color by classifying each object window according to operated time slot is proposed. In Addition, we propose the method that is to revert to the past desktop environment by using the past history. 3.1 Proposed Relating Hue to Time Slot We define that there is relation in the activated each object window in same time slot. Because when the user operates plural object windows simultaneously, these target object windows are used by near time. Figure.1 shows applied hue circle (color wheel) to the proposed GUI. A day time as twenty-four hours is applied to twelve colors based on red, green and blue (RGB) color by two hours unit. The twelve colors

Red.Green.Blue Time slot 00.80.FF 0-2 from 22 o’clock 00.00.FF 2-4 22-24 to 24 o’clock 80.00.FF 00.80.FF 20-22 4-6 00.FF.FF 18-20 FF.00.FF 6-8 FF.00.80 00.FF.80 FF.00.00 00.FF.00 16-18 FF.80.00 80.FF.00 8-10 14-16

FF.FF.00 12-14

10-12

Fig. 1. Defined hue circle

24

K. Sugawara and H. Fujita

ware defined that the each gradation level of RGB is changed by half. The maximum gradation level is “FF”, the half gradation level is “80” and minimum gradation level is “00”. As this twelve color transition assumes the gradation of sky in a day, user can know the time from each color. 3.2 Improvement of Visual Information Figure 2 is described the proposed GUI that is applied new proposed ideas. The information of six displayed object windows is sorted in this GUI. The left side of this GUI in Figure.2 is indicating icons of the application program related to each displayed object window. The texts on the right side of proposed list are displaying the titles of each displayed object windows. There is the elapsed time bar of the selected object windows under the each object title. The nearest object window from front side layer on the screen is counted elapsed time as activated object window. In order to prevent count of idle time, the counting of the elapsed time is stopped when the cursor pointer is stopping for the ten minutes that is user-selectable smallest unit on this color scale. In addition the elapsed time is reduced for ten minutes. We defined for simplicity the time to be ten minutes period because the interrupted time is different according to interposing work. The reason of using the cursor pointer movement is that the main operation is mouse operation in document creation of Cyber Manual. The length of the elapsed time bar is decided as maximum length based on one hour. The elapsed time that exceeds the one hour is converted to numeric display with displaying minutes and seconds on the right side of the elapsed time bar. A color scale is displayed on the upper part of this GUI for distinguishing time slot. A check mark on the color scale means which color is now time slot. The issue that user cannot understand the time difference due to big gap of each elapsed time is resolved by deciding

0

12

24

Cyber Manual 56:20 1h

Mailer ‘D’ 28:35 0h

Text Editor ‘C’ 20:12 0h

Internet Browser ‘B’ 26:06 0h

Picture Viewer ‘A’ 15:10 0h

Fig. 2. Applied new proposed ideas to the previously proposed GUI

An Approach for Smoothly Recalling the Interrupted Tasks by Memorizing User Tasks

25

the maximum length of the elapsed time bar based on one hour. Therefore, even if there is the big gap between the object window of longest elapsed time and the other object windows, user can understand the time difference of activated each object windows with displaying total elapsed time. When an object window is selected by user, the background color of the related program icon becomes the color of the previously defined time slot color. Therefore, user can understand the relation of each displayed object window at a glance which object windows are related and when the object window is used. 3.3 Switching to Past Desktop Environment Function In order to look back to past situation, the viewing function of the activated the object window title and switching function are proposed. The proposed GUI stores the history of desktop environment that are top position, left position, height, width, date and time about the displayed object windows on the screen by time slot unit. When the cursor pointer moves to the target time slot in the color scale, the all each color of the color scale are changed to the detailed time slot color as Figure.3 shown. The color scale of time slot indicates the time based on separating two hours to twelve by using twelve color gradations. The titles of used object window at the time slot are listed under the cursor pointer after changing color scale. Therefore, a user can know the past used object windows by overlapping cursor pointer on the intended time slot. In addition, a user can select detail time slot on ten minutes unit for looking back to past situation by clicking each time slot. When the user click on the color scale of intended time slot, all displayed objects are minimized and only target objects are appeared in sequence. Therefore, the layout of displayed object windows are switched to around clicked time and the related windows are readied by just deciding the time slot that user want to restart. The symbol of the black circle in Figure.3 means that the object window has been displayed recent. Meanwhile, the other symbol of the uncolored circle means that the object window has already closed. As the symbols are displayed, user can know at a glance whether which the object windows are displaying resent.

10:00

10:30

11:00

11:30

12:00

٨Internet Browser ‘B’ ٤Text Editor ‘G’ ٨Picture Viewer ‘A’

Fig. 3. Color scale of 120 minutes view

4 Conclusion The functionality of the previously proposed GUI is improved by applying the proposed ideas. The type of interruption on time span is appeared by considering the situation of a document creation on Cyber Manual. Therefore, a document editor can

26

K. Sugawara and H. Fujita

know for restarting environment from this GUI in a short-term interruption or a middle-term interruption by using operation history and color gradation. If the history of the expert worker as efficient workflow applies to the novice worker, the document creation time of the novice worker may be shortened [6]. This paper proposed the approach that is assisting user's short-term memory in order to improve user’s work efficiency. To further improve operational efficiency, the computer needs to give effective information to worker by guessing the next appropriate task. However, the appropriate task is different by worker’s situation, skill level or its role. Therefore, the computer needs to understand the relationship of user profiles and tasks for reasoning appropriate task by using workflow, semantic network and so on. Future research should include the research of actual relationship between users profile and their work in order to give the suitable expression for the user perception to the worker.

Acknowledgment The authors appreciate their team members, Dr. R. Maruta and Dr. S. Hayashida and Ms. N. Taniguchi of Sangikyo Corporation for their useful comments and advices in the development of this work.

References 1. Sugawara, K., Fujita, H.: An Approach of Visualizing Objects Overlapping for Assisting User Cognition. In: Proceedings of the 10th WSEAS International Conference on Applied Computer Science, ACS 2010, pp. 461–464 (2010) 2. Sugawara, K.: Approach for Handling Multiple Objects Without Breaking the Continuity of User’s Thinking. In: Fujita, H. (ed.) New Trends in Software Methodologies, Tools and Techniques Proceedings of the 9th SoMeT 2010, pp. 218–225 (2010) 3. Sugawara, K., Maruta, R.: A New Windows Desktop Icon for Single Point-of-Control of the Application Window. In: Fujita, H., Marirk, V. (eds.) New Trends in Software Methodologies, Tools and Techniques Proceedings of the Eighth SoMeT 2009, pp. 483–491. IOS press, Amsterdam (2009) 4. Maruta, R., Hayashida, S., Taniguchi, N., Sugawara, K., Sengoku, M., Shoji, W.: Developing a Unique Collaborative System – “Optimization Ware” Using a Novel Massive Data Retrieval/Management Approach. In: Fujita, H., Zualkernan, I. (eds.) New Trends in Software Methodologies, Tools and Techniques –Proceedings of the Seventh SoMeT 2008, pp. 113–125 (2008) 5. Hong, J., Suh, E.-H., Kim, J., Kim, S.: Context-aware system for proactive personalized service based on context history. In: Expert Systems with Applications, pp. 7448–7457. Elsevier, Amsterdam (2009) 6. Madhusudan, T., Leon Zhao, J., Marshall, B.: A case-based reasoning framework for workflow model management. In: Data & Knowledge Engineering, pp. 87–115. Elsevier, Amsterdam (2004)

Implementing an Efficient Causal Learning Mechanism in a Cognitive Tutoring Agent Usef Faghihi1, Philippe Fournier-Viger2, and Roger Nkambou3 1

Department of Computer Science, University of Memphis, Tennessee, USA 2 Department of Computer Science and Information Engineering, National Cheng Kung University, Taiwan 3 Department of Computer Science, University of Quebec in Montreal, Montreal, Canada [email protected], [email protected], [email protected]

Abstract. An important research problem for developing complex cognitive agents is to provide them with human-like learning mechanisms. One important type of learning among episodic, emotional, and procedural learning is causal learning. In current cognitive agents, causal learning has up to now been implemented with techniques such as Bayesian networks that are not scalable for handling a large volume of data like a human does. In this paper, we address this problem of causal learning using a modified constraint based data mining algorithm that respects temporal ordering of the events. That is, from a huge amount of data, the algorithm filters the data to extract the most important information. We illustrate the application of this learning mechanism in a cognitive tutoring agent for the complex domain of teaching robotic arm manipulation. Keywords: Cognitive Tutoring Agent, Causal Learning, Data mining.

1 Introduction Reasoning is crucial to cognition [1-4]. One important type of reasoning is inductive reasoning, where either one tries to generalize rules from a set of examples or from a set of probable or inadequate premises, one decides the likeliness that a conclusion is true. The truth of a conclusion is likely when the premises give some evidence or support towards the conclusion. In inductive reasoning, when evidence is deemed to be absolute, significant, and generally persuasive, they are cogent. They may bring us to a true conclusion. When the evidence is not deemed absolute, significant and persuasive, then the evidence is non-cogent. In inductive reasoning, the relation between premise and conclusion is uncertain. After generalization, one cannot claim that all potential information about a situation has been collected and that there is no additional unseen information that could discredit the hypothesis. For instance consider the following example. From the statement “The football moves when kicked by a player,” we can infer “All footballs move when kicked by a player.” The inductive reasoning is ampliative – it extends our knowledge. One usually needs to examine prior knowledge, relation, set of examples and experiences in order to draw inductive K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 27–36, 2011. © Springer-Verlag Berlin Heidelberg 2011

28

U. Faghihi, P. Fournier-Viger, and R. Nkambou

conclusions from premises. However, all this information that one must examine to come to a conclusion from a set of premises makes it difficult for scientists to propose a universally accepted theory of inductive reasoning [5-7]. The specific memory to register particular events such as what, where and when, is known as episodic memory in human. Episodic memory allows human to adapt its behavior according to previous experiences in the face of a new environment or other agents. The human being systematically constructs its causal knowledge based on episodic memory. Given that episodic memory contains the memory of events their outcomes, we make inductive abstraction to construct relations between two events. Thus, in humans, causal memory is influenced by the information retained by episodic memory. Inversely, new experiences are influenced by causal memory [8-11]. Different methods are proposed for finding causal relations between events: scientific experiments, statistical relations, temporal order, prior knowledge, and so forth. [12]. Once we find a causal relation between two events, we can explain the cause of the second event if it occurs again. As human beings, we recall recent events better than older events [13, Page 475]. Most cognitive agents do not have any fully automated reasoning. The founders of CLARION have claimed they have implemented causal reasoning in it [14]; however, the causes and effects in this agent are set by hand [15]. In this paper, we first briefly explain the functioning of the Conscious Emotional Learning Tutoring System (CELTS), a cognitive agent that was created to provide assistance to learners in learning activities [15-18]. We then explain how we improved causal learning in CELTS by integrating the capability to handle temporal constraints and event constraints to the algorithm. This can improve CELTS’ memory construction and at the same time allow for better interactions with learners. We finally present an experimental evaluation of the agent behavior with learners and an evaluation of the performance of the algorithms with real life data mining datasets.

2 CELTS CELTS is a hybrid artificial intelligent tutor which is based on Baars’ [19] theory of consciousness. It performs through cognitive cycles. Cognitive cycles in CELTS start by perception and usually end by the execution of an action. CELTS uses its Behavior Network (BN) for action selection (Figure 1.B). The BN is implemented based on Maes’ Behaviour Net [20]. It is a network of partial plans that analyses the context to decide what to do and which type of behavior to set off (Figure 1.B). Given that CELTS is a tutor, an expert can define different solutions in the BN to help learners. Thus, BN’s nodes are messages, hints, demonstration, etc (Figure 1.D) to assist learners while they manipulate Canadarm2 in the virtual environment (Figure 1A). The virtual environment is a simulation of Canadarm2, the robotic telemanipulator attached to the International Space Station (ISS) to help astronauts learn how to manipulate Canadarm2 before going to space. The learners’ manipulations of the virtual world simulator, simulating Canadarm2 (Figure 1.A), constitute the interactions between them and CELTS. In particular, the virtual world simulator sends all manipulation data to CELTS, which, in turn, sends learners advices to improve their performance. Canadarm2 manipulation on the ISS is a very difficult task, because there

Implementing an Efficient Causal Learning Mechanism in a Cognitive Tutoring Agent

29

is a constant risk of collision. Our team has now added different types of learning in CELTS based on neurobiology and neuropsychological theory [21]. We have added and Emotional Mechanism [22], which simulates the peripheralcentral” theory of emotions. The peripheral-central approach takes into account both the short and long route of information processing and reactions, as in humans. Both the short and long routes perform in a parallel and complementary fashion in CELTS’ architecture. The Emotional Mechanism (EM) learns and at the same time contributes emotional valences (positive or negative) to the description of the situation. It also contributes to the decisions made and the learning achieved by the system. We have also added an Episodic Mechanism (EPM), which simulates the multipletrace theory of memory consolidation [21, 22]. The multiple-trace theory postulates that every time an event causes memory reactivation, a new trace for the activated memory is created in the hippocampus. Memory consolidation occurs through the reoccurring loops of episodic memory traces in the hippocampus and the construction of semantic memory traces in the cortex. Thus, the cortical neurons continue to rely on the hippocampus even after encoding. We used sequential pattern mining algorithms to simulate this behavior of memory consolidation in CELTS. To do so, every information broadcast in the system during a training session between CELTS and learners is assigned to a specific time. Thus, CELTS’ EPM extracts information from registered events in the system. Given a problem, EPM is capable of finding the best solution among different solutions conceived by the expert in its BN [21]. Finally, we have also added a Causal Mechanism (CLM), which is implemented by using a combination of sequential pattern mining algorithms and association rules respecting temporal ordering of events- statistical relations, temporal ordering of events, and prior knowledge (from conceived by the expert in the BN). Causal knowledge is generated in CELTS after a) the information is broadcast in the system; b) a decision is made about the ongoing problem, which c) is reinforced by future experiences while CELTS receives information from the virtual world and interacts with learners (Figure 1. A.D). Using these three mechanisms, CELTS can memorize learners’ errors and find the causes of the errors. CELTS is a tutor that interacts with learners and is faced with a huge amount of data. The algorithms previously implemented in CELTS generate lots of irrelevant rules. For this reason, CELTS does not always find relevant causes [15]. In this paper, we aim at generating only the rules that are relevant to CELTS’ current situation. To reach this goal, we have replaced our previously developed rule mining algorithm with an improved version that handles event constraints and a temporal constraint. By performing this modification and adapting CELTS’ BN, we are able to generate only relevant rules. As we will see, event constraints and the temporal constraint help CELTS to not only generate fewer and more reliable rules, but also make CELTS produce explanations for the errors caused by learners. As in the case of humans, our new algorithm also helps CELTS to better remember more recent events’ causes. For the remainder of this paper, we briefly review CELTS’ Episodic and Causal Learning Mechanisms. We also explain how we added event constraints and a temporal constraint to the algorithm.

30

U. Faghihi, P. Fournier-Viger, and R. Nkambou

Fig. 1. CELTS’ interface and Canadarm2

3 Episodic Learning Episodic Learning mechanism (EPL) extracts frequently occurring events from its past experiences [15, 21]. In our context, CELTS learns during astronauts’ training sessions for arm manipulation in the Canadarm2 simulator virtual world [23] (Figure 1.A). To construct CELTS’ Episodic Memory, a trace of what occurred in the system is recorded in CELTS’ different memories during consciousness broadcasts [21]. For instance, each event X=(ti,Ai) in CELTS represents what happened during a cognitive cycle. The timestamp ti of an event indicates the cognitive cycle number, whereas the set of items Ai of an event contains an item that represents the coalition of information (e.g., collision risk with ISS) that were broadcast during the cognitive cycle. For example, one partial sequence recorded during our experimentations was <(t=1, c2 e{-0.6}), (t=2, c4 e{-0.8})>. This sequence means that during cognitive cycle 1, the coalition c2 (indicating that learner forgot to adjust camera in the simulator, Figure 1.A) was broadcast, followed by the broadcast of c4 (indicating that learner brought about an imminent collision in the simulator, Figure 1.A. The e{-0.6} in the sequence, indicates that CELTS’ Emotional Mechanism assigned negative valance 0.6 to this information (see [22] for more details). However, e{-0.8} indicates more negative emotional valence which is the imminent collision-dangerous situation on the ISS. If this subsequence appears several times during learners’ interactions with CELTS, the following rule can be discovered: forgetting to adjust the cameras and not adjusting the camera parameters correctly is followed by a collision risk. {Forget-camera adjustment, Selected wrong joint} {Collision risk} This information later helps CELTS to remember learners’ errors and make better decision to help them. Below are a few examples: • Manipulating Canadarm2 close to the space station and not being aware of the distance can lead to a collision: {Canadarm2_NearISS, Not_aware_of_distance} {Collision }

Implementing an Efficient Causal Learning Mechanism in a Cognitive Tutoring Agent

31

• A learner manipulating Canadarm2 close to the space station, being aware of the distance and reaching the goal is an expert: {Canadarm2_NearISS, Aware_of_distance, Goal_attained} {Expert}. • If the learner is inactive, the learner lacks motivation: {learner_inactive} ¼{learner_lacks_motivation}

4 Causal Learning To extract causal rules, we previously designed a custom algorithm for CELTS [15]. However, we have noted in previous experiments that this algorithm can generate a large amount of irrelevant rules where the events are separated by too much time (e.g., several minutes). For example, in the following rule {user_did_not_adjust_cameras} {good_manipulation}. The rule indicates that each time a user forgets to adjust cameras before moving Canadarm2 in the virtual, he or she will manipulate the Canadarm2 well. In the contrary to the above rule, our experiences demonstrate that each time a user forgets to adjust the camera, he or she is likely to make collision risk in the virtual world (Figure 1.A). To improve CELTS performance we used TRuleGrowth [24]. The original TRuleGrowth algorithm takes a database of event sequences (as previously defined) and three parameters as inputssupport, confidence1 and, window_size, which is the maximum time length a rule occurs. In the context of CELTS, this parameter is very important as it allows CELTS to exclude rules between events that are separated by too much time. The window_size constraint is a global parameter that must be selected by an expert in CELTS. Using window_size, support and confidence and temporal ordering of the events, CELTS is capable of inductive reasoning [15]. This information can be interpreted as an estimate of the conditional probability P(Y | X) [25-27]. However, one problem that occurs when discovering rules applying TRuleGrowth in CELTS is that there can be up to several thousands of rules. This, degrades the performance of the data mining algorithm and also of CELTS, which has too many rules to consider for reasoning. To reduce the number of rules and to extract only the most precise and relevant ones, we have adapted the TRuleGrowth algorithm to add constraints on events that a rule can contain so that only the rules that are relevant to CELTS are extracted in any situation. 4.1 Adapting TRuleGrowth to Handle Events Constraints We have modified TRuleGrowth to allow specifying the following four types of event constraints on events that a rule can contain. The constraints are set when an expert, for a given problem, creates solutions in the BN. When the expert creates questions, he tags some CELTS Behavior Network (BN) nodes so that they respect some specific event constraints (Figure 1.B). After assigning constraints, the TRuleGrowth algorithm extracts the rule that respects only set constraints from interactions between 1

Support and confidence. The support of a rule is defined as the number of sequences that contain the rule. The confidence of a rule is defined as the ratio between the number of sequences where the rule appears and the number of sequences containing its left part.

32

U. Faghihi, P. Fournier-Viger, and R. Nkambou

CELTS and learners to help learners with only relevant hints, messages, demonstrations, and so forth. To assign the temporal constraint, CELTS uses its BN’s nodes that are tagged by the expert to respect event constraints. That is, when CELTS reaches the node tagged as C1, the temporal constraint timer for this solution starts automatically. The timer stops once the user makes mistakes. In what follows, we first present these constraints and then explain how they are useful for the reasoning of CELTS. The constraints are the following: • • • •

C1: the set of events that the left part of a rule can contain, C2: the set of events that the right part of a rule can contain, C3: the set of events that the left part of a rule has to contain, C4: the set of events that the right part of a rule has to contain.

We modified TRuleGrowth to ignore events that are excluded according to event constraints C1 and C3, or C2 and C4 when searching for events that can extend the left or right parts of a rule. Although this modification is simple, it can dramatically decrease the execution time and reduce the number of rules found, as we will show in the experimental results of this paper. How event constraints are used by CELTS to discover relevant rules We now explain how the four event constraints that we have defined are used to discover rules that are relevant for CELTS. The constraints can be used to discover rules that answer the following four questions: Q1: What may happen? Constraint C1 is used to force rules to contain only events that occurred in the left part of the rule. For instance, if the event {Forget-camera adjustment} and {Selected wrong arm joint} occurs, CELTS can put these two events in C1 and discover rules such as {Forget-camera adjustment, Selected wrong joint} {Collision risk}, which indicates that this combination of events could result in a collision risk. CELTS can then use this information to stop the learner or take any other remedial action. Q2: What was the cause of the event? Constraint C4 is used to force rules to contain only events that occurred in the right part of the rule. This allows us to create explanations of why these events occurred. For instance, in one situation, CELTS recorded that Collision risk (CR) occurred. By searching the right part of the rules containing constraint C4, CELTS was able to find that forgetting camera adjustments and choosing wrong joint(s) are the most likely causes of the problem. Q3: Given a specific situation, which events will take place? Using constraint C1 and/or C3, we make sure that occurred events will be registered in the left part of the rules, and by using constraints C2 and/or C4 we make sure that the prediction will be registered in the right part of the rules. By using this strategy, CELTS can obtain rules that predict the occurrence of some specific events in the future. For instance, if a learner moves the Canadarm2 too close to the ISS, CELTS can put this event in constraint C3 and the collision event in constraint C4, to know if it is likely that a collision with ISS will occurs. This information can then be used to take the appropriate action.

Implementing an Efficient Causal Learning Mechanism in a Cognitive Tutoring Agent

33

Q4: What may happen after given action of CELTS? Given a situation, constraint C1 is assigned to all registered events that have happened and constraint C3 is assigned to all registered action(s) that CELTS can take for the situation. By using these constraints, CELTS can discover rules that indicate the consequences of the possible actions to be taken. For instance, if while manipulating Canadarm2, the learner makes mistakes by choosing a bad joint in a low visibility situation, C1 is assigned to the situation and the remedial possible actions that CELTS can take are assigned to C3 (e.g., “Engage dialogue about joint selection”). Thus, using constraints C1 and C3 can allow discovering rules such as: {mistakes_with_joints, low_visibility,engage_dialogue } {learner_correct_mistake} Given that the situation described by the rule is similar, CELTS can then use the rule to take appropriate action. Finally, the previous four questions can be combined to achieve more complex reasoning. For instance, CELTS can detect why some learners don’t know which joint must be chosen to achieve a specific goal (the event “don’t_know_right_joint” and “goal#21”, which means putting Canadarm2 in the END point in Figure1.A). To do so, while CELTS interacts with learners, it seeks all the rules whose left part contains the following information: {don’t_know_right_joint, exercise_goal#21}. According to the information extracted by Episodic Learning Mechanism, CELTS knows that {not_perform_distance_evaluation, did_not_performed_camera_adjustement} {doesn’t_know_right_joint, goal#21}. This means that the cause may be that the learner forgot to make a distance evaluation or forgot to adjust camera. According to constraints C3, CELTS can then search the following information "What is the best camera for observing ISS" or "What is the distance between Canadarm2 and ISS" given the cause found and {don’t_know_right_joint,goal#21}. Asking these questions, CELTS can help learners to solve the problem by providing explanations to the learner. This helps CELTS to predict the results of its action and the learner’s response and helps it to choose the best action to help the learner. How CELTS uses found rules to takes decision Given that learners make mistakes, after several interactions with learners, CELTS must choose a rule to find the cause of the learners’ problems. By applying the data mining algorithm, there can be many candidate rules that match the event constraints determined by the current situation and the temporal constraint. The found rules may suggest different causes for the learners’ mistakes. For each rule, CELTS considers the left part of the rule as the cause and the right part as the effect. To choose a rule the following process is used. First, all the rules that do not match the current execution are discarded. Then, to choose a rule among the remaining rules, CELTS will compute the strength of each rule as follows [15]: Strength(rule) = confidence × support. The rule with the highest strength will be chosen to help the learner. Third, if by using the window_size parameter CELTS cannot find the cause of the learner’s error, the algorithm will search the BN by going backward from the point that the error

34

U. Faghihi, P. Fournier-Viger, and R. Nkambou

occurred until finding the cause. In other words, CELTS will search outside the window size time interval if it cannot find the cause within it. If CELTS cannot find any answer, the message “I am not capable of finding the cause of the problem” is shown.

5 Experimental Results To demonstrate how Q1, Q2, Q3 and Q4 can improve the quality of CELTS’ assistance to learners, learners were invited to perform arm manipulations using Canadarm2. To do so, we conducted an evaluation of the new version of CELTS with learners who had previously used the older versions of CELTS. The experiment simply consisted in letting each learner try the new version for one hour, observing them and interviewing them. All learners agreed that this new version is more complete and that the enhanced tutoring services were very helpful and helped CELTS to provide a more natural dialogue. Furthermore, to illustrate how our new modified algorithms improved CELTS performance, we applied it to our database of recorded interactions between learners and CELTS that comprises 20 000 sequences of events. Each sequence is a dialogue conducted by CELTS where sequences contain around 50 messages or questions depending on what the learner answers and the choices CELTS makes. We evaluated the performance of the modified data mining algorithm. Applying TRuleGrowth with minSup = 0.01 and minConf = 0.30 and window size = 20 took 524 seconds and generated 2,655,064 rules. After, we reapplied the same algorithm by activating constraints C1, C2, C3 and C4 during the scenario, while recording 100 more sequences. On average, approximately 64 rules were found each time the algorithm was applied and the execution took on average 0.5 seconds. In other words, a speed improvement of 100 times and a rule reduction of 41,485 times were respectively achieved by using event constraints. This shows that these constraints greatly increase the performance of CELTS, without sacrificing any relevant rules. Before making this improvement CELTS could just not handle such a large database of sequences.

6 Conclusion In this paper, by integrating temporal and event constraints with data mining algorithms in CELTS, we demonstrated that CELTS is capable of finding more reliable causal rules than its previous version. This helps CELTS to provide better assistance to the learners. As in the case of humans, the episodic and causal memories in CELTS mutually influence each other during interactions with learners. For instance, if the causes found by CELTS turn out to be false, it influences the support of the causal rules which in turn influences episodic memory, and increase or decrease the events supports. However, in humans, recent events are remembered better than older events. To simulate this we added temporal constraints to our algorithm. To our knowledge, researchers in artificial intelligence have, up to now, used Bayesian methods to study causal reasoning and causal learning model for cognitive agents. However, the Bayesian approach is not applicable when the agent is faced

Implementing an Efficient Causal Learning Mechanism in a Cognitive Tutoring Agent

35

with large amounts of data, like it is the case in CELTS. Another important issue with Bayesian Networks is that they generally require domain experts to specify conditional probabilities by hand, which is often a difficult and time-consuming task. For CELTS, we chose to use data mining algorithms instead of Bayesian Networks because we wanted to create a completely automatic approach that could acquire causal knowledge incrementally. For future works, we plan to improve the causal learning algorithms to make them incremental (make them reuse past results to further improve their performance). We are also working on improving the pedagogical strategies used by CELTS and other modules such as dialogue generation.

References 1. Gopnik, A., Schulz, L. (eds.): Causal Learning: Psychology, Philosophy, and Computation. Oxford University Press, USA (2007) 2. Sarma, V.V.S.: Decision making in complex systems. Springer, Netherlands (1993) 3. Leighton, J.P.: Defining and describing reason. In: Leighton, J.P., Sternberg, R.J. (eds.) The Nature of Reasoning. Cambridge University Press, Cambridge (2004) 4. Demetriou, A.: Cognitive development. In: Demetriou, A., Doise, W., van Lieshout, K.F.M. (eds.) Life-Span Developmental Psychology, pp. 179–269. Wiley, London (1998) 5. Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach, 2nd edn. Prentice Hall, Upper Saddle River (2003) 6. Feeney, A., Heit, E.: Inductive Reasoning: Experimental, Developmental, and Computational Approaches. Cambridge University Press, Cambridge (2007) 7. Sternberg, R.J., Mio, J.S.: Cognitive psychology. Wadsworth/Cengage Learning, Australia, Belmont, CA (2009) 8. Martin, C.B., Deutscher, M.: Remembering. Philosophical Review 75, 161–196 (1966) 9. Shoemaker, S.: Persons and their Pasts. American Philosophical Quarterly 7, 269–285 (1970) 10. Perner, J.: Memory and Theory of Mind. In: Tulving, E., Craik, F.I.M. (eds.) The Oxford Handbook of Memory, pp. 297–312. Oxford University Press, Oxford (2000) 11. Bernecker, S.: The Metaphysics of Memory. Springer, Berlin (2008) 12. Lagnado, D.A., Waldmann, M.R., Hagmayer, Y., Sloman, S.A.: Beyond Covariation: Cues to Causal Structure. In: gopnik, A.S., Laura (eds.) Causal learning: Psychology, Philosophy, and Computation, pp. 154–172. Oxford University Press, Inc., Oxford (2007) 13. Ezak, M.D., Howieson, D.B., Loring, D.W., Hannay, H.J., Fischer, J.S.: Neuropsychological Assessment, 4th edn. Oxford University Press, USA (2004) 14. Sun, R.: The CLARION cognitive architecture: Extending cognitive modeling to social simulation Cognition and Multi-Agent interaction. Cambridge University Press, New York (2006) 15. Faghihi, U., Fournier-Viger, P., Nkambou, R., Poirier, P.: A Generic Causal Learning Model for Cognitive Agent. In: The Twenty Third International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems, IEA-AIE 2010 (2010) 16. Dubois, D., Gaha, M., Nkambou, R., Poirier, P.: Cognitive Tutoring System with "Consciousness". In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 803–806. Springer, Heidelberg (2008)

36

U. Faghihi, P. Fournier-Viger, and R. Nkambou

17. Faghihi, U., Fournier-Viger, P., Nkambou, R., Poirier, P.: A generic episodic learning model implemented in a cognitive agent by means of temporal pattern mining. In: Chien, B.-C., Hong, T.-P., Chen, S.-M., Ali, M. (eds.) IEA/AIE 2009. LNCS, vol. 5579, pp. 545– 555. Springer, Heidelberg (2009) 18. Faghihi, U., Poirier, P., Dubois, D., Gaha, M.: A new emotional architecture for cognitive tutoring agents. In: Proc. FLAIRS 2008, pp. 445–446 (2008) 19. Baars, B.J.: In the Theater of Consciousness:The Workspace of the Mind. Oxford University Press, Oxford (1997) 20. Maes, P.: How to do the right thing. Connection Science 1, 291–323 (1989) 21. Faghihi, U., poirier, P., Fournier-Viger, P., Nkambou, R., Human-Like Learning in a Conscious Agent. Journal of Experimental & Theoretical Artificial Intelligence (2010) (in press) 22. Faghihi, U., Fournier-Viger, P., Nkambou, R., Poirier, P., Mayers, A.: How Emotional Mechanism Helps Episodic Learning in a Cognitive Agent. Proceedings of the 2009 IEEE Symposium on Intelligent Agents (2009) 23. Nkambou, R., Belghith, K., Kabanza, F.: An Approach to Intelligent Training on a Robotic Simulator Using an Innovative Path-Planner. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 645–654. Springer, Heidelberg (2006) 24. Fournier-Viger, P., Wu, C.W., Tseng, V.S.: TRuleGrowth: Mining Sequential Rules Common to Multiple Sequences with Window Size Constraint. In: Proceedings of FLAIRS 2011, 6 pages. AAAI press, Menlo Park (2011) (in press) 25. Hipp, J., Güntzer, U., Nakhaeizadeh, G.: Data Mining of Association Rules and the Process of Knowledge Discovery in Databases. In: Industrial Conference on Data Mining, pp. 15–36 (2002) 26. Deogun, J., Jiang, L.: Prediction Mining – An Approach to Mining Association Rules for Prediction, pp. 98–108. Springer, Heidelberg (2005) 27. Li, D., Deogun, J.S.: Discovering Partial Periodic Sequential Association Rules with Time Lag in Multiple Sequences for Prediction, pp. 332–341. Springer, Heidelberg (2005)

Model Checking Commitment Protocols Mohamed El-Menshawy, Jamal Bentahar, and Rachida Dssouli Concordia University, Faculty of Engineering and Computer Science, Canada [email protected], [email protected], [email protected]

Abstract. We investigate the problem of verifying commitment protocols that are widely used to regulate interactions among cognitive agents by means of model checking. We present a new logic-based language to specify commitment protocols, which is derived from extending CTL∗ with modalities for social commitments and associated actions. We report on the implementation of the NetBill protocol—a motivated and specified example in the proposed language—using three model checkers (MCMAS, NuSMV, and CWB-NC) and compare the experimental results obtained.

1

Introduction

A society mainly evolves through communication among cognitive participating entities. In such a society, people interact and exchange information with each other to satisfy their goals. Such communication requires languages and mechanisms to structure interactions among participants within dialogues. Correspondingly, a cognitive agent-based model for an artificial society should provide adequate support for communication languages by providing means to specify interaction protocols that regulate intelligent interactions among agents. It is also beneficial to equip this model with a formal verification method to eliminate errors in the sense that protocols comply with the given specifications and to increase confidence on the model in terms of safety and efficiency. Motivation. This paper introduces a new logic-based language to specify a special kind of interaction protocols, called commitment protocols [5,6,13,12]. This logic extends CTL∗ [4] with modalities for social commitments and associated actions. Social commitments usually represent the contractual obligations that associate the cognitive agents with one another in order to capture multi-agent intelligent interactions in this artificial society. In this context, social commitments model cognitive aspects of agent interactions by enforcing them to keep the fulfillment of their contracts and reducing cognitive dissonance in agent behaviors. Moreover, we adopt these commitment protocols as they are increasingly used in different applications such as modeling business processes [5,6,13] and developing web-based applications [11,2,7]. Conventionally, commitment protocols are defined in terms of creation and manipulation of social commitments that capture a rich variety of interactions among agents. Several approaches have K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 37–47, 2011. c Springer-Verlag Berlin Heidelberg 2011 

38

M. El-Menshawy, J. Bentahar, and R. Dssouli

been proposed to specify commitment protocols [6,13,12,11]. However, the languages used to specify those protocols are either not suitable for model checking (a formal verification method) or defined in computational logics having weak expressive power. Approach. Our approach begins with developing a new logic-based language by extending CTL∗ with modalities for social commitments and associated actions. The resulting logic, ACTL∗sc , is used to: 1) express well-formed formulae of commitments and their contents; 2) specify an abstract semantics of commitment actions; and 3) express desirable properties to be checked such as safety. Formulae of commitments and their contents New ACTL*sc logic

Commitment actions (Abstract semantics) Protocol properties (Formal specifications)

Specification language of commitment protocols

Translated into: ISPL, SMV, and CCS

Model checkers: MCMAS, NuSMV, and CWB-NC

Verification results

Fig. 1. A schematic view of our approach

By abstract semantics, we mean a semantics that captures the performance of the commitment actions without defining the meaning of all concrete action instances. Using an abstract action makes our approach more flexible as it allows for capturing not only standard commitment actions like those introduced in [10] such as Withdraw, but also other possible actions such as Inform. However, the concrete semantics of the commitment actions considered in this paper is completely given in our previous work [8]. Using social commitments and their actions, we define a new specification language of commitment protocols. We present experimental results of verifying the NetBill protocol taken from ebusiness domain and specified in our language against some properties. Finally, the implementation of this protocol is done using the MCMAS model checker [9] along with NuSMV [3] and CWB-NC [14] as benchmarks after translating the protocol into these model checkers input languages ISPL, SMV and CCS respectively (see Figure 1). Organization. Section 2 presents the syntax and semantics of ACTL∗sc logic. In Section 3, we use commitments and their actions to define a new specification language of commitment protocols. The translation and implementation of the NetBill protocol and its verification using the MCMAS, NuSMV and CWB-NC model checkers with experimental results are discussed in Section 4. In Section 5, we discuss relevant literature and future work.

2

Commitments and ACTL∗sc Logic

The introduction of a new branching time ACTL∗sc logic is motivated by the fact that the needed modal connectives for social commitments and associated

Model Checking Commitment Protocols

39

actions cannot be only expressed using CTL∗ [4]. Before introducing the syntax and semantics of our logic, we present the notion of social commitments. Notation 1. Social commitments are denoted by SC(i, j, ϕ), where i is the debtor, j the creditor and ϕ a well-formed formula (wff ) in the proposed logic representing the commitment content. SC(i, j, ϕ) means i socially (i.e., publicly) commits to j that ϕ holds [10,8]. Notation 2. Conditional commitments are denoted by τ → SC(i, j, ϕ), where “→” is the logical implication, i, j and ϕ have the above meanings and τ is a wff in the proposed logic representing the commitment condition [10,8]. We use SC c (i, j, τ, ϕ) as an abbreviation of τ → SC(i, j, ϕ). In this case, we have SC(i, j, ϕ)  SC c (i, j, , ϕ), where  is the constant for truth. Commitments can be manipulated using a set actions defined in [10]. We classify these actions into two and three party actions. The former ones need only two agents to be performed such as: Create(i, j, SC(i, j, ϕ)) to establish a new commitment, Withdraw(i, j, SC(i, j, ϕ)) to cancel an existing commitment, Fulfill(i, j, SC(i, j, ϕ)) to satisfy the commitment content, Violate(i, j, SC(i, j, ϕ)) to reflect there is no way to satisfy the commitment content, and Release(j, i, SC(i, j, ϕ)) to free a debtor from carrying out his commitment. The latter ones need an intermediate agent to be completed such as: Delegate(i, k, SC(i, j, ϕ)) to delegate an existing commitment to another debtor k to satisfy it on his behalf and Assign(j, k, SC(i, j, ϕ)) to assign an existing commitment to another creditor k. 2.1

Syntax of ACTL∗sc

We use Φp = {p, p1 , p2 , . . .} for a set of atomic propositions, AGT = {i, j, k, . . .} for a set of agent names and ACT = {θ, θ1 , θ2 , . . .} for a set of actions. Definition 1. The syntax of ACTL∗sc is given by the following BNF grammar: S ::= p | ¬S | S ∨ S | E P | C C ::= SC(Agt, Agt, P) P ::= S | ¬P | P ∨ P | XP | PUP | Create(Agt, Agt, C) | Θ(Agt, Agt, C) where Agt and Θ are non-terminals corresponding to AGT and ACT respectively. Formulae in ACTL∗sc are classified into state formulae S and path formulae P. The intuitive meanings of the most constructs of ACTL∗sc are straightforward from CTL∗ operators. The formula Aϕ (resp. Eϕ) means that ϕ holds along all (some) paths starting at the current state. The formula SC(i, j, ϕ) means agent i commits towards agent j to bring about a formula ϕ. The action formula θ(i, j, C) means an action θ is performed on the commitment C by i towards j. Furthermore, there are some useful abbreviations based on temporal operators: F ϕ  true Uϕ (sometimes in the future) and Gϕ  ¬F ¬ϕ (globally).

40

M. El-Menshawy, J. Bentahar, and R. Dssouli

2.2

Semantics of ACTL∗sc

The semantics of ACTL∗sc logic is interpreted with respect to the formal model M associated to the commitment protocol using a Kripke structure as follows: M = S, ACT, AGT, R, V, Rsc , L, AS, I, where: S is a finite set of states; AGT is a set of agents; ACT is a set of actions; R ⊆ S×AGT×ACT×S is a total labeled transition relation; V : Φp → 2S is a valuation function; Rsc : S × AGT × AGT → 2Π , where Π is the set of all paths, is the accessibility relation for social commitments; L : S → 2AGT×AGT is an agency function that associates to each state a set of two interacting agents in this state; AS ⊆ S is a set of acceptance states; and I ⊆ S is a set of initial states. Agl :θk Instead of (sk , Agl , θk , sk+1 ), the labeled transitions will be written as sk −−−−→ sk+1 . The paths that path formulae are interpreted over have the form π = Agl :θk

Agl+1 :θk+1

Agl :θk

sk −−−−→ sk+1 −−−−−−−→ sk+2 . . . such that ∀k, l ≥ 0, (sk −−−−→ sk+1 ) ∈ R. A path in M is then an infinite sequence of states and labeled transitions with agents and their actions. π(m) refers to the mth state in this sequence. When a state sm is a part of a path π, we write sm ∈ π. The set of all paths starting at sk is denoted by Π sk and sk , π refers to the path π starting at sk . A path π ∈ Rsc (sk , i, j) is an accessible path for the interacting agents i and j iff all states along this path are possible states for the two interacting agents. Formally: π ∈ Rsc (sk , i, j) iff sk = π(0) and ∀sm ∈ π we have (i, j) ∈ L(sm ). Excluding social commitments and action formulae, the semantics of ACTL∗sc state formulae is as usual (semantics of CTL∗ ). The notation M, sk  |= ϕ means that the model M satisfies ϕ at state sk and M, sk , π |= ϕ means that the model M satisfies ϕ along the path π starting at state sk (sk = π(0)). The state formula SC(i, j, ϕ) is satisfied in the model M at sk iff the content ϕ is true in every accessible path from this state using Rsc (sk , i, j). M, sk  |= SC(i, j, ϕ) iff ∀π ∈ Π sk s.t. π ∈ Rsc (sk , i, j) we have M, sk , π |= ϕ A path π emanating from sk satisfies Create(i, j, SC(i, j, ϕ)) in the model M iff i: Create is the label of the first transition on this path and SC(i, j, ϕ) holds in the next state sk+1 on this path. i:Create

M, sk , π |= Create(i, j, SC(i, j, ϕ)) iff sk −−−−→ sk+1 ∈ R and M, sk+1  |= SC(i, j, ϕ) A path π emanating from sk satisfies θ(Agx , Agy , SC(i, j, ϕ)) in the model M where Agx = Agy , x ∈ {1, 2} and y ∈ {1, 2, 3} iff Agx : θ is the label of the first transition on this path and the commitment has been created in the past. Agx :θ

M, sk , π |= θ(Agx , Agy , SC(i, j, ϕ)) iff sk −−−−→ sk+1 ∈ R and ∃l ≤ k s.t. M, sl , π ↓ sl  |= Create(i, j, SC(i, j, ϕ)) where π ↓ sl is the prefix of π starting at sl .

3

Specification of Commitment Protocols

Our specification language of commitment protocols is defined as a set of commitments capturing the business interactions among agents (or roles) at design

Model Checking Commitment Protocols

41

time. In addition to which messages can be exchanged and when, a protocol also specifies the meanings of the messages in terms of their effects on the commitments and each message can be mapped to an action on a commitment. The proposed specification language begins with the commitment COM, which is followed by a message MSG. This message is directly mapped into either commitment actions (captured by θ) or inform action. Specifically, MSG could either be Withdraw, Fulfill, Violate, Release, Assign, Delegate or Inform. The delegated (resp. the assigned) message is followed by create message that enables the delegatee (resp. the assignee) to create a new commitment. The inform message is an action performed by the debtor i to inform the creditor j that a domain proposition holds. It is not a commitment action, but indirectly affects commitments by causing transformation from conditional to unconditional commitments. The domain proposition Dom-Pro identifies the set of propositions related to the application domain of the protocol. The protocol terminates when the interacting agents do not have commitments to each other. The formal specification of commitment protocols is defined using BNF grammar with meta-symbols “::=” and “|” for the choice and “;” for action sequence as follows: Table 1. The formal specification of commitment protocols Protocol ::= COM ; MSG MSG ::= Withdraw(i, j, COM) |Fulfill(i, j, COM) | Violate(i, j, COM) | Release(j, i, COM) | (Assign(j, k, COM) | Delegate(i, k, COM)) ;  Create(i, j, COM) | Inform(i, j, Dom-Pro) ; MSG COM ::= SC c (i, j, Prop, Prop) | SC(i, j, Prop) Prop ::= A well−formed formula in ACTL∗sc Dom-Pro ::= Identify domain propositions

The above specification can either be used to reason about the actions at run time or compiled into finite state machines (FSMs) at design time. At run time, the agents can logically compute their execution paths that respect the given protocol specifications using some reasoning rules as axioms [13]. However, these rules are not enough to verify protocols against some properties when the system is complex. Moreover, the flexibility resulting from these reasoning rules can be expensive and may increase the code of the agents [13]. For these reasons, we adopt the second approach, which consists in compiling the commitment protocol into enhanced FSM wherein commitments hold on states and actions label transitions, and thereby the legal computations can be inferred. This enables the protocols to be enhanced with additional transitions to handle exceptions. 3.1

A Motivating Example

Let us consider the NetBill protocol used in [13,8,1] and taken from e-business domain to clarify our specification of commitment protocols. Figure 2 depicts

42

M. El-Menshawy, J. Bentahar, and R. Dssouli

our modeling of the NetBill protocol using enhanced FSM where S is the set of vertices, AGT and ACT the set of edge labels, I = s0 , AS = {s6 }, and R the set of edges. Our modeling begins by an inform message on a path starting at s0 . In this message, the customer (Cus) requests a quote for some goods. This request is followed by the merchant (M er) reply that consists in sending the quote as an offer, which means creating a commitment. The Cus agent could either reject this offer, which means releasing the offer and the protocol ends at the failure state s7 (as it is not in the set AS ), or accept the offer, which means creating a commitment at s3 . When the Cus agent accepts the received offer, then he has three choices: 1) to withdraw his commitment; 2) to delegate it to a financial company (say Bank1 ); or 3) to directly pay the M er agent. As in [12], the Bank1 agent can delegate this commitment to another bank (say Bank2 ), which delegates the commitment back to the Bank1 agent. The bank agents delegate the commitment back and forth infinitely often and this is presented by a transition loop at s11 . In a sound protocol, this behavior should be avoided (see Section 4.2). The M er agent, before delivering the goods to the Cus agent, can withdraw his offer and then move to the recovery state s9 after refunding the payment to the Cus agent. When the Cus agent pays for the requested goods and the M er agent delivers them, the M er agent fulfills his commitment at s5 and then moves to the acceptance state s6 after sending the receipt to the Cus agent. Conversely, the Cus agent can pay for the requested goods without being delivered by the M er agent. In this case, the merchant agent violates his commitment at s8 and then moves to the recovery state s9 after refunding the payment to the customer. Finally, the M er agent, for some reasons, can assign his commitment to another merchant (say M er1 ) at s12 in which a new commitment between Cus and M er1 is created as a new offer. As for the delegate scenario, the assign action can be repeated infinitely often among agents and this scenario, presented by a transition loop at s12 , is unwanted behavior in our protocol.

S0

Cus: Inform (Price-request)

S1

Mer: Create (Offer)

Mer1: Create

S2

Cus: Release (Reject)

S7

Cus: Withdraw

S12

(Offer)

Mer1: Assign (to another Merchant)

Mer: Assign (to Mer1) Cus: Create (Accept) S3

Cus: Fulfill (Pay)

Cus: Delegate (to Bank1) S11 Bank1: Delegate (to another Bank2)

Mer: Violate (Not Deliver)

S4

Bank1: Inform (Pay)

Mer: Fulfill (Deliver)

S8

Mer: Withdraw S10

Mer: Inform (Refund)

S9

Mer: Inform (Refund)

S5

S6

Mer: Inform (Send-Receipt)

Fig. 2. Enhanced FSM of NetBill protocol with commitments and their actions

Model Checking Commitment Protocols

4

43

Implementation

In a nutshell, given the model M representing the NetBill protocol and a formula ϕ, the problem of model checking is establishing whether or not the model M satisfies ϕ. Clarke et al. shown that the problem of CTL∗ model checking can be reduced to the problem of CTL and LTL model checking [4]. This paper follows similar approach by effectively reducing the problem of ACTL∗sc model checking to the problem of model checking ALTLsc and ACTLsc . ALTLsc and ACTLsc are LTL and CTL augmented with commitments and action formulae. 4.1

Translating Commitment Protocols

The first step in our implementation is to translate the NetBill protocol into: 1) ISPL (the input language of MCMAS); 2) SMV (the input language of NuSMV); and 3) CCS (the input language of CWB-NC). In ISPL, this process begins by translating a set of interacting agents (M er, Cus, M er1 and Bank1 ) into standard agents in the Agent section and commitments into local variables in the Vars section. Such variables are of enumeration type including all possible commitment states plus the acceptance, failure and recovery states. The actions on commitments are directly defined in the Actions section. The translation is completed by declaring a set I of initial states in the InitStates section and the formulae that need to be checked in the Formulae section. In SMV, the set of interacting agents are translated into isolated modules, which are instantiated in the main module. The latter module includes the definition of initial conditions using the INIT statement and the keyword SPEC to specify the formulae that need to be checked. The commitment states plus the successful, failure and recovery states are defined as SMV variables in the VAR statement. Such states with actions are used as reasoning rules to automatically evolve the states change. The labeled transition relation between commitment states is described using the TRANS statement. This statement uses the next and Case expressions to represent agent’s choices in a sequential manner. In CCS, each agent in our protocol is represented by a set of processes and each process is specified using the proc statement. The states of commitments are captured by the set of actions performed on these commitments. Each transition is represented by the action labeling this transition followed by another process. 4.2

Expressing Protocol Properties

To automatically verify commitment protocols, we need to express some desirable properties. Specifically, we formally express a rich class of protocol properties: fairness, safety, liveness, and reachability using the proposed logic. These properties include the properties defined in [2,5] and satisfy the same functionalities of the properties defined in [12,1] (see our discussion in Section 5). Fairness constraint property. It is needed to rule out unwanted behaviors of agents in our protocol. An example of fairness constraint is given by ϕ1 , which

44

M. El-Menshawy, J. Bentahar, and R. Dssouli

states that along all paths it is globally the case that on all emerging paths the Bank1 agent eventually does not delegate commitment. ϕ1 = AGAF ¬Delegate(Bank1 ) Safety property. This property means “something bad never happens”. For example, in our protocol a bad situation is: the Cus agent sends the payment, but the M er agent never delivers the requested goods: ϕ2 = AG(¬(P ay(Cus) ∧ AG¬Deliver(M er)) Liveness property. This property means “something good will eventually happen”. For example, in our protocol a good situation is: along all paths it is globally the case that if the Cus agent sends the payment, then in all computations in the future the M er agent will either 1) deliver the requested goods; 2) withdraw his commitment; or 3) violate his commitment: ϕ3 = AG(P ay(Cus) → AF (Deliver(M er) ∨ Withdraw(M er) ∨ (N otDeliver(M er)))

Reachability property. A particular situation can be reached from the initial state via some computation sequences. For example, along all paths it is eventually the case that there is a possibility in the future for the M er agent to deliver the requested goods to the Cus agent: ϕ4 = AF EF Deliver(M er) 4.3

Experimental Results

We implemented the translation tools on top of the three model checkers (MCMAS, NuSMV, and CWB-NC) and provided a thorough assessment of this translation on three experiments. These experiments were meant to compare the verification results of checking the NetBill protocol against some properties specified in ACTLsc (fragment of our logic) using MCMAS and NuSMV (symbolic-based techniques) and CWB-NC (an automata-based technique). We used two criteria (the model size and execution time) to perform these comparisons. The model size is defined as |M | = |S| + |R|, where |S| is the state space and |R| is the relation space. The state space is computed by multiplying the state space of each agent and the state space of the protocol. We can approximate |R| by 10x|S|2 , where 10 is the number of ACTL∗sc operators. So we have |M | ≈ 10x|S|2 . The model size of our experiments is shown in Table 2. Table 3 displays the full results of these three experiments and the execution times (in seconds) on a laptop running 32-bit Windows Vista with 2 GiB of memory and Intel Core 2 Duo clocked at 1.66 GHz. For instance, the execution time for MCMAS increases when augmenting the number of agents from 2 to 4 as the number of OBDD variables needed to encode agents increases with Table 2. The model size in the three experiments Exp.1 Model size 3.5E+7

Exp.2

Exp.3

9.73E+10

6.23E+11

Model Checking Commitment Protocols

45

the number of agents. This proves the fact that the time complexity of model checking is a function of the size of the input problem. The results also reveal that the number of OBDD variables (which reflect the model size) is greater in NuSMV than in MCMAS, but the execution time in NuSMV is better than in MCMAS. In CWB-NC, when the size of the model is small, the execution time is better than in MCMAS and NuSMV (see Table 3). However, when the state space increases (as in Exp.3), the execution time in MCMAS and NuSMV is going to be better than in CWB-NC. Notice that we put “–” in Table 3 because CWB-NC does not use OBDD variables. We can conclude that MCMAS and NuSMV are better than CWB-NC in terms of the execution time. Table 3. Verification results of the MCMAS, NuSMV and CWB-NC MCMAS NuSMV CWB-NC Exp.1 Exp.2 Exp.3 Exp.1 Exp.2 Exp.3 Exp.1 Exp.2 Exp.3 # OBDDs 24 39 49 33 53 67 – – – # Agents 2 3 4 2 3 4 2 3 4 Exec. Time (sec) 0.52 2 6 0.23 1.11 1.98 0.094 0.564 8.814

5

Related Literature and Conclusion

Desai et al. [6] used a modular action description language to specify commitment protocols. Their approach composes multiple protocols to simplify the development of business processes. Yolum and Singh [13] used commitment actions to show how to build and reason about commitment protocols using event calculus. Their approach only indirectly models the fulfillment of a commitment. Our approach belongs to the same line of research, but it complements the above frameworks by defining a more expressive logic-based language to specify commitment protocols, which is suitable for model checking. Venkatraman and Singh [11] presented an approach for testing whether the behavior of an agent complies with a commitment protocol specified in CTL. The proposed approach uses model checking to automatically verify the conformance of commitment protocols with specifications (properties). Cheng [2] and Desai et al. [5] used the automata-based model checker SPIN to verify commitment business protocols and their compositions expressed in LTL. Our language uses not only ALTLsc specification, but also ACTLsc specification. We also used the MCMAS and NuSMV tools, which are more efficient than SPIN (see Table 3). Bentahar et al. [1] proposed a framework where the interacting agentbased systems communicate by combing and reasoning about dialogue games. Our verification technique is based on encoding the protocol and formulae using symbolic model checkers instead of translating them into alternating B¨ uchi tableau automata. Yolum [12] presented the main generic properties needed to design commitment protocols. Our properties meet these generic properties in the sense that the reachability can be used to satisfy the same objective of the effectiveness property. The consistency property is achieved in our protocol by

46

M. El-Menshawy, J. Bentahar, and R. Dssouli

satisfying the safety property. Moreover, the robustness property is satisfied by considering liveness property and fairness paths. Conclusion. In this paper, we presented a new logic to define a specification language of commitment protocols, which excludes spurious aspects that plague existing approaches. We described a verification technique to automatically— instead of semi-automatically [12]—verify the compliance of these protocols with the given properties. In our implementation, we conducted 3 experiments, which demonstrate the following fact: three model checkers were able to verify a variety of complex formulae correctly and efficiently but the symbolic technique is better than the automata one. As future work, we plan to develop a new model checking algorithm for our logic to directly verify the proposed specification language. Acknowledgements. The authors would like to thank Natural Sciences and Engineering Research Council of Canada (NSERC) and Fond Qu´ebecois de la recherche sur la soci´et´e et la culture (FQRSC) for their financial support.

References 1. Bentahar, J., Meyer, J.-J.C., Wan, W.: Model Checking Agent Communication. In: Dastani, M., Hindriks, K.V., Meyer, J.-J.C. (eds.) Specification and Verification of Multi-Agent Systems, 1st edn., pp. 67–102. Springer, Heidelberg (2010) 2. Cheng, Z.: Verifying Commitment based Business Protocols and their Compositions: Model Checking using Promela and Spin. Ph.D. thesis, North Carolina State University (2006) 3. Cimatti, A., Clarke, E.M., Giunchiglia, E., Giunchiglia, F., Pistore, M., Roveri, M., Sebastiani, R., Tacchella, A.: NuSMV: An Open Source Tool for Symbolic Model Checking. In: Brinksma, E., Larsen, K.G. (eds.) CAV 2002. LNCS, vol. 2404, pp. 359–364. Springer, Heidelberg (2002) 4. Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. The MIT Press, Cambridge (1999) 5. Desai, N., Cheng, Z., Chopra, A.K., Singh, M.P.: Toward Verification of Commitment Protocols and their Compositions. In: Durfee, E.H., Yokoo, M., Huhns, M.N., Shehory, O. (eds.) AAMAS, pp. 144–146. IFAAMAS (2007) 6. Desai, N., Singh, M.P.: A Modular Action Description Language for Protocol Composition. In: Proc. of the 22th AAAI Conference on AI, pp. 962–967 (2007) 7. El-Menshawy, M., Bentahar, J., Dssouli, R.: Modeling and verifying business interactions via commitments and dialogue actions. In: Jedrzejowicz, P., Nguyen, N.T.,  Howlet, R.J., Jain, L.C. (eds.) KES-AMSTA 2010. LNCS, vol. 6071, pp. 11–21. Springer, Heidelberg (2010) 8. El-Menshawy, M., Bentahar, J., Dssouli, R.: Verifiable semantic model for agent interactions using social commitments. In: Dastani, M., El Fallah Segrouchni, A., Leite, J., Torroni, P. (eds.) LADS 2009. LNCS, vol. 6039, pp. 128–152. Springer, Heidelberg (2010) 9. Lomuscio, A., Qu, H., Raimondi, F.: MCMAS: A model checker for the verification of multi-agent systems. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 682–688. Springer, Heidelberg (2009)

Model Checking Commitment Protocols

47

10. Singh, M.P.: An Ontology for Commitments in Multi-Agent Systems: Toward a Unification of Normative Concepts. AI and Law 7(1), 97–113 (1999) 11. Venkatraman, M., Singh, M.P.: Verifying Compliance with Commitment Protocols: Enabling Open Web-based Multi-Agent Systems. Autonomous Agents and MultiAgent Systems 2(3), 217–236 (1999) 12. Yolum, P.: Design Time Analysis of Multi-Agent Protocols. Data and Knowladge Engineering 63(1), 137–154 (2007) 13. Yolum, P., Singh, M.P.: Flexible Protocol Specification and Execution: Applying Event Calculus Planning using Commitment. In: Proc. of the 1st International Joint Conference on AAMAS, pp. 527–534. ACM, New York (2002) 14. Zhang, D., Cleaveland, W.R., Stark, E.W.: The integrated CWB-nC/PIOATool for functional verification and performance analysis of concurrent systems. In: Garavel, H., Hatcliff, J. (eds.) TACAS 2003. LNCS, vol. 2619, pp. 431–436. Springer, Heidelberg (2003)

Mobile Architecture for Communication and Development of Applications Based on Context L.M. Soria-Morillo, J.A. Ortega-Ramírez, L. González-Abril, and J.A. Álvarez-García Computer Languages and Systems Dept University of Seville 41012, Seville {lsoria,jortega,luisgon,jalvarez}@us.es

Abstract. The arrival of ubiquitous computing and the increasing use of mobile devices can geta lot of information about the user. This information is used by some applications to adapt its functionality to the user itself and the surrounding environment. In this way the applications need to become more autonomous and less each time user interaction. However, the computational cost, battery consumption and the complex process of obtaining useful information from sensory data means that many applications do not use this information in a massive way. This research proposes a framework and a middleware for the development, implementation and communication of contextual applications. Using this architecture allows communication between applications so that they can share applications without contextual information should both generate. Thanks to definition of a SOA platform,subscription to services provided by other applications is possible through the middleware.

1 Introduction In the last few years, the Context Based Computing has undergone a major revolution, mainly due to three reasons: the integration of sensors into mobile devices, presence of increased generation of mobile phones in “the daily lives of users” and the massive connectivity between devices and Internet. On the one hand, technological progress has allowed the integration of a large number of sensors in next generation mobile devices. Years ago, a large infrastructure would be needed for assembly. Examples of these sensors are triaxial accelerometers, chips based on GPS positioning, wireless connectivity features like WiFi and Bluetooth as well as light sensors and more. Furthermore, the integration of mobile devices in the daily lives of users of a population, allows the development of useful applications for users that can be used anywhere thanks to the portability of devices where run on. We should note that, due to we are working with portable elements, energy savings must be present in any development takes place, as it will be necessary to maximize the usage time of devices. Thus the batteries time is a problem. In addition, it is necessary maximize battery time for the convenience of users of devices, since a high recharge rate can reach displeasing them. K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 48–57, 2011. © Springer-Verlag Berlin Heidelberg 2011

Mobile Architecture for Communication and Development of Applications

49

The applications launched by use of context, is a step forward in computing. In such applications, the device itself is which launches applications based on specific conditions of some environmental variables and state of the user who wears it. By obtaining, analyzing and processing data from sensors and minimizing the interaction between the user and the device itself is possible to speed up the use of the device.

2 Objectives The core motivation of this research is to solve the high degree of dispersion when developers create common implementation of applications based on context and no author to date has defined a framework that is fully adapted to the development of ubiquitous systems standard from a common view for all mobile applications. The creation of this framework would be a great advantage for applications to exploit the advantages of knowledge of the context surrounding the user to increase the functionality of applications. In addition to increased functionality, most important would be the total adaptation of content presented to the user and the state which is both the user and his/her environment at the time of execution. In addition to creating a standardized framework, the capabilities of the work environment must support the integration of all context recovery techniques in the most ubiquitous device known: the mobile. This should be done at any time maximizing the number of contexts recognized by the framework, so that the potential of the development of applications is as high as possible. Under this development framework and application structuring, is going to be developed a communication middleware between applications. Thus, based on our own ontology for contextual information systems, applications can share all the necessary information. This drastically reduces the complexity of building new contextual applications, because information could be provided by another application that already offers this functionality.

3 Scenario The following is a usage scenario of the development system based on context. We will analyze the interaction of Daniel, a user of next generation mobile device that has acquired an application developed with the exposed framework, and which uses the proposed middleware. Are 6:30PM and makes a beautiful summer night. The system detects that Daniel is at home, thanks to indoor positioning system and labeling of common sites. During all the night, the mobile phone has been in eco-battery mode, because thanks to the activity recognition module, the system knows mobile phone has been in the table without any use. The system detects that Daniel has left home, thanks to detection of exit to outdoors, and automatically the own system activates the GPS device. The exit detection module is not implemented by own application, but this one a request to the middleware to search if necessary information is available. After consulting the information, the middleware will provide it to the application, in this case if Daniel has left or not the location where he was.

50

L.M. Soria-Morillo et al.

It's Saturday, the day in which Daniel, with his girlfriend Mar often go to the cinema to see films released the previous day. Therefore, after getting into the car, the detection module determines that the frequent destinations most likely fate of Daniel is the home of his girlfriend Mar and activates the navigation system with the specific purpose to meet the traffic to the destination. Once together, Mar and Daniel go to the cinema. Already in the cinema, Mar meets her sister, but since they do not routinely uses the Bluetooth device, she has disabled it. This makes the subsystem responsible for detecting proximity using Bluetooth cannot find any matches. However, Mar starts talking to her sister about what they will do next weekend. Thanks to the module of detection by voiceprint, Daniel's device recognizes the voice of his sister-in-law Rosa and detects her proximity. Through a search process on the agenda, the device detects that Daniel has something yet to talk to Rosa, that is, has a to-do with her. Due to the system has detected that Daniel is close to her, shows an alert with this task. Because the film does not start until in half an hour, Daniel and Rosa have decided talk about, so the agenda of Daniel is released from the appointment with Rosa.

4 Related Works The first researches related to ubiquitous computing emerged in 1991 from the hand of Mark Weiser [2]. This work set a precedent in how to understand computing, and brought the benefits of information systems to the daily lives of users, i.e. what we now understand by ubiquitous computing. During the next years, the trend of ubiquitous systems went almost exclusively the calculation of the position and its applications. This was aided largely the launch of GPS (Global Position System), which reached full operational capability in 1995, and the creation of portable receivers for this technology. During the years 2000 and 2001 a series of frameworks for the management of ubiquitous systems were developed [6], [7] and [8]. In particular, the most widely used architecture of the systems of the time was ContextToolkit. This tool allowed integrating several sensors in one system so that the addition of a new sensor could be carried out in a fast and efficient without being necessary to reprogram the entire system. However, the main limitation of the system was that applications could not obtain context data from ContextToolkit, which was faced with the philosophy of applications based on context that years later adopted the scientific community. However, the real revolution of ubiquitous computing has been in the past three years due to the increase in integrated sensors on mobile devices. Since we mentioned earlier in the Introduction section, massive use of mobile phones has encouraged the development of applications based on context. A clear example of this type of application is the work of Jae Sik Lee et al [10]. It implements a music recommendation system based on the date, location, season, day of week, average temperature, maximum and minimum as well as weather conditions at the site and the present moment. Thus, the system is able to determine the area where the user is and play the music the user usually plays in a context similar to the current.

Mobile Architecture for Communication and Development of Applications

51

In the last years has appears a lot of framework mainly related with internet of things and collaborative environments [1]. However mobile phones middlewaresare not yet totally defined and there is no works related to this field in the references, at least in an integrated manner.

5 Context Technologies After entering the work we carry out, this section shall proceed to give the basic ideas on which we based the system to be developed. In this section we will make a clear distinction between the various technologies used to support the attainment of the context in which the user is at any time. These technologies will be determined by the device on which is executed the framework, so that not all devices support all functionality. However, the technologies that will be described throughout this section are included in the next-generation devices that are currently marketed. Any system that is based on the context to run needs to know a set of information to model the environment around the user. One hand, this model allows taking decisions of execution and on other hand, learning across decisions taken by the user in a given context as a basis for possible automatic decisions. By tradition, as presented above, it is necessary to know who performs the action, where it takes place, when it occurred and what action has been taken. The following technologies enable us to get answers to all these questions. Furthermore, the system will add new information that traditional systems have not taken into account. 5.1 Activity Recognition The activity carried out by the user also has great value to determine the current execution context. When we speak of activity, we refer to the action that is carried out the user from the analysis of the values obtained from accelerometer sensors. By learning and classification of a number of parameters obtained from the accelerometer device, the system is able to classify the activity that the user is performing. We do this out using a classification system based on probability tables and decision trees, and a second level, governed by a hidden Markov model (HMM). In this respect we have developed a detection method for classifying activities with a success rate of 95% the activity takes place. In addition, this system is possible to add new patterns of activity to the system without involving reprogramming of the platform. 5.2 Outdoor Location At present there are very precise location systems outdoors. The most widespread of these is the GPS tracking system. This system requires a direct view of satellites to function properly, so indoor, this system is invalid. Despite this drawback, global coverage and accuracy make this system the most widely used worldwide. However, the main drawback of this system is the high energy cost GPS receiver. This is especially important in embedded systems in mobile devices. Therefore, at this point, the work will focus on reducing energy consumption of this system in order to extend as far as possible, the battery useful time between charges.

52

L.M. Soria-Morillo et al.

A detection system to external inputs and outputs has been developed. This makes it possible to disconnect the GPS device when the user is indoors and reconnect it when the user goes out. This causes the batteries of mobile devices increasing use time by 230% between charges. 5.3 Indoor Location The current techniques for indoor positioning are useful in sensorized spaces, since most of them are based on RFID tags [3], bar-codes, QR-codes or proximity sensors to determine the user's position on a roof or the proximity to a particular object. However, this type of positioning techniques is uncomfortable since it requires an infrastructure for its operation. There are other projects that base their position in the triangulation of WiFi networks. The main problem of such solutions is that WiFi access points must be of high quality in order to eliminate fluctuations in signal strength. In addition to this we must bear in mind that in areas where the density of WiFi networks is low, the effectiveness of the system is not assured. The aim of this work in the field of indoor positioning is getting a new positioning techniques based solely on accelerometer, digital compass and other sensors included in next generation mobile devices. Thus, it is possible to obtain an approximate user's position with a margin of error as small as possible anywhere roofing. In contrast to RFID-based techniques, the solution to be proposed in this work can be used anywhere without the building is adapted for positioning. This is important when we speak of a ubiquitous system. Using this technique will be possible to positioning the user at any house, office or shopping center where he/she is. 5.4 People Detection Detection of people near the user opens a range of possibilities for context-based system and therefore the functionality of the framework that we are developing will increase. A detection system makes the application knows at all times who around the user. The system will be even capable of storing information of new people who had not previously interacted with the system's user. For detection of people close to the user we will use two different but complementary technologies, detection based on sound fingerprints[9] and detection based on Bluetooth signals. 5.5 Semantic Context Induction Mainly due to poor security of the traditional methods of storing information collected from the user's position, it is necessary to develop a new model. This model is based on the labeling of storage locations instead of latitude and longitude of the location of the user. The labeling of places introduces much semantic information in the storage system, so that, only the user and the individuals within their environment could understand the information.

Mobile Architecture for Communication and Development of Applications

53

6 Architecture Layers Once studied the various functional modules supported by the system, it is time to analyze the layers that make up the framework itself. Applications developed based on the layersproposals, will be able to perform all necessary tasks identified in the applications based on context. Furthermore, each layer consists of one or more modules, which contain specific functionality for specific software developed needs. Obviously, each module is interconnected with other modules in other layers of the architecture, so it should be abundantly clear the responsibilities of each. Figure 1 shown de architecture of the framework with its seven layers. As we shall see when each layer is look in depth, the main advantages of this architecture is on the one hand, the independence of the particular specifications of the sensors used and on the other hand, the ability to share information with other applications and even other information systems present on remote machines or devices via data exchange modules. On the other hand, we must pay attention to the architecture proposed in this chapter provides the ability to launch applications based on the certain characteristics of the environment, denoted as context-shot applications. As stated above, we then describe in more detail each of the modules in the architecture for the development of applications based on given context. 6.1 Sensor Adaptation Module (SAM) This part of the structure contains all the logic for adjusting the data from the sensors of the device or other applications through the middleware. Generally, the data must be processed prior to the application can proceed with the analysis, classification and use of such data. 6.2 Information Classification Module (ICM) The information from sensors or other system applications is not usually enough to provide functionality to the application to develop, but these data must be processed to generate new information. It is precisely this layer of the architecture which provides such access to developers. ICM layer is one of the main modules of the architecture; it is where information is processed by algorithms and processing methods. This layer will have support of SAM layer and storage ISM layer through which the application can obtain and store persistently all the information generated. 6.3 Information Storage Module (ISM) The purpose of this layer is to provide a method of persistent or temporary storage of all information generated by lowers layers of the application. This layer should provide methods to access, modify and delete information stored. On the other hand, we should avoid as much as possible to other layers of the architecture to access the ISM, except for the ICM and SSL, because usually the information stored in context has a high degree of privacy. It is therefore essential to prevent other applications or systems uncontrolled access to such information.

54

L.M. Soria-Morillo et al.

Fig. 1. Framework architecture diagram

6.4 System Security Layer (SSL) The security layer provides an effective system to prevent access to information by applications without permission to. In this section does not specify specific security methods to prevent access, although it is recommended that access to the main database of the application is done safely. In addition, whenever possible, they have to delegate the storage of information to the middleware, which has a specific module for this purpose. Thus it is possible to isolate and wean the application of data stored.In this way, the information can be accessed or stored using the functions that the middleware offers. 6.5 Share Information Module (SIM) As mentioned above, access to persistent or temporary information stored in the database should be limited to applications with sufficient privileges to do so. In addition, the information stored must be marked as available for a specific application profiles to provide greater security to the system. Furthermore, external applications residing in other systems may access information through web services, so as to facilitate the exchange of information with other systems. This task will be done by SIM module, which will enable the application designed to send and receive information from other systems via Internet. 6.6 Context Aware Service Module (CASM) This module allows storing information about the launch of applications based on specific context. In particular, the information stored will be the application to execute, the execution parameters and context that must be met for execution. Thus, the CASM module communicates directly with the middleware module sending said registration information of an automatic release. In addition, this module

Mobile Architecture for Communication and Development of Applications

55

provides a point of information exchange in case the current application should be executed because another application has registered the process of automatic execution.

7 Middleware In this research a middleware following SOA architecture is going to be generated, which allow applications to provide services to other applications. In addition, a certain application can subscribe to the services offered by another easily. The proposed middleware consists of five layers. In Figure 2 is shown a diagram explaining the different modules of the middleware architecture. ’’Ž‹…ƒ–‹‘

’’ŽŽ‹…ƒ–‹‘

’’Ž‹…ƒ–‹‘

 ’’Ž‹…ƒ–‹‘

ƒ–ƒ Šƒ”‹‰ ‘–‡š–– ƒƒ‰‡‡ –

”‹‰‰‡” ’’• ƒƒ ‰‡‡–

‡…—”‹–› › ƒƒ‰‡‡ –

ƒ–ƒ –‘”ƒ‰‡

‡”˜‹…‡ ‡ —„•…”‹’–‹‘ 

‡”˜‹…‡ ‹•…‘˜‡ ”› ƒ–ƒ † †ƒ’–‘”

‡•‘”

‡ •‘”

‡•‘”

‡•‘”

Fig. 2. Middleware architecture preview

The first of these is the adaptation layer of sensory data. It will package the information received from sensors and will be introduced to the middleware as a service, so that both the information generated by other applications such as collection from the sensors, will be accessible from any application that requires it. Secondly we have the data storage and security system. The first of them is responsible for storing all the necessary information in a persistently, so that information is available at any time for that application that need it. As has been seen, this module should implement a policy of deleting data, to prevent malfunction due to excessive memory. Security manager will be responsible for determining whether an application can do or not to use a particular service. The third layer is subscription services module and service discovery module. Although both are described later in detail, we must take in mind that these modules are responsible for managing all events generated by the middleware for sending and receiving context information between applications. The service discovery module is designed to provide applications a way through which indicates whether a particular service is or is not available in the system. Instead, the subscription to the service

56

L.M. Soria-Morillo et al.

module is used by applications to register the notification system in a particular service that has already been checked that is available. At the next level of middleware context manager appears. This module is the intermediary between applications and the middleware itself, providing an interface to access the system. In addition, process all requests coming from the applications themselves and delegate the service module that is appropriate. On the other hand, the Context Management unifies all communication logic modules, standardizing the ontology intended to be communicated. In the same level as this module is the launching applications module, which is responsible for receiving requests and performs automatic launching of applications. Finally appears the information exchange module, which aims to facilitate communication between applications that are not hosted on the device itself, using a channel of communication and external dissemination middleware. This module is an expansion point in the architecture, because it can extend the operation of the system to a collaborative environment in which applications are not always implemented on the device itself, but in a distributed environment.

8 Future Works Although middleware and architecture development stage is still early, some objectives can be defined to ensure proper adaptation of architecture to the realm of contextual and ubiquitous applications. In this way should be complete the identification of sources of information compatible with the architecture so that it succeeds in obtaining an ontology compatible with existing information sources and provide a general enough to support news data that have not been taken into account. On the other hand it should provide a set of optimal solutions for each of the requirements in the layers of architecture. This will get applications that use this architecture using compatible technology, which will facilitate the reuse of modules among several applications at the time of development.

9 Conclusions The result of this work will aim to solve the traditional problems of systems based on context and provide a common development environment for most devices. This ensures the independence of individual sensors and provides a development interface for applications. The framework will make the development of applications based on context is conducted in a much more efficient than they have done so far. This will ensure an increase in this type of application, with the comfort this means for the user. Finally, we must comment that the structure of the framework will make allow the developers to extend the functionality of them. On this way, the developer could adapt the framework to their own needs, thereby increasing the performance, quality and minimizing the time cost of application development based on context.

Mobile Architecture for Communication and Development of Applications

57

Acknowledgements This research is partially supported by the projects of the Ministry of Science and Innovation INCARE (TSI2006-13390-C02-02) and ARTEMISA (TIN2009-14378C02-01).

References [1] Song, Z., Cárdenas, A.A., Masuoka, R.: Semantic middleware for the Internet of Things. In: Proceeding of Internet of Things, IOT (2010) [2] Weiser, M.: The computer for the 21st century. Scientific American 265(3), 94–104 (1991) [3] Hameed, B., Khan, I., Dürr, F., Rothermel, K.: An RFID based consistency management framework for production monitoring in a smart real-time factory. In: Proceeding of Internet of Things, IOT (2010) [4] Schmidt, A., Beigl, M., Gellersen, H.-W.: There is more to Context than Location. Computers & Graphics Journal 23(6) (1999) [5] Schmidt, A., Forbess, J.: What GPS doesn’t tell you: determining one’s context with low-level sensors. In: Proceedings of the ICECS 1999, 6th IEEE International Conference on Electronics, Circuits and Systems, ICECS 1999 (1999) [6] Dey, A.K., Abowd, G.D.: The Context Toolkit: Aiding the Development of ContextAware Applications. In: Workshop on Software Engineering for Wearable and Pervasive Computing, Limerick, Ireland, June 6 (2000) [7] Dey, A.K., Abowd, G.D.: CybreMinder: A Context- Aware System for Supporting Reminders. In: Proceedings of the 2nd International Symposium on Handheld and Ubiquitous Computing (HUC2K), Bristol, UK, September 25-27, pp. 172–186 (2000) [8] Dey, A.K.: Understanding and Using Context. In: Personal and Ubiquitous Computing, vol. 5(1), pp. 4–7. Springer, Heidelberg (2001), http://www.personal-ubicomp.com/ ISSN: 0949-2054 [9] Lu, L., Zhang, H.-J. (eds.): Unsupervised speaker segmentation and tracking in real-time audio content analysis, vol. 14. Association for Computing Machinery, Inc, New York (2005) [10] Lee, J.S., Lee, J.C.: Context Awareness by Case-Based Reasoning in a Music Recommendation System. In: Ubiquitous Computing System, pp. 45–58 (2008) [11] Nord, J., Synnes, K., Parnes, P.: An Architecture for Location Aware Applications. In: Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS 2002), vol. 9 (2002)

A Simplified Human Cognitive Approach for Supporting Crowd Modeling in Tunnel Fires Emergency Simulation Enrico Briano, Roberto Mosca, Roberto Revetria, and Alessandro Testa DIPTEM – Department of Industrial Production, Technology, and Mathematical Models Via Opera Pia 15, Genoa, Italy {enrico.briano,roberto.revetria,alessandro.testa}@unige.it, [email protected],

Abstract. This work focuses on the human behavior during an evacuation of a motorway tunnel. First of all the technical prescriptions have been taken into account, according with the current legislation updated after the Mont Blanc Tunnel tragedy of March 1999. In order to study how people behave in case of emergency an innovative simplified approach has been applied based on a mix of Thomas Saaty’s Analytic Hierarchic Process (AHP) and Schmidt’s PECS (Physical Emotional Cognitive Social) Model. This methodology has been implemented in a simulation model implementing both 2D and 3D heat and smoke propagation in order to investigate crowd reaction during escape in tunnel fires emergencies. All the scenarios have been analyzed and evaluated using Analysis of Variance (ANOVA) and Response Surface Methodology (RSM). Keywords: Simulation, Human Cognitive Models, AHP, Human Behavior.

1 Introduction In the last decade safety in motorway tunnels, especially for those longer than 1000 m, has significantly increased its importance inside the European Union Regulation especially after the Mont Blanc tunnel disaster of 24TH March, 1999. The Alpine District countries (Italy, France, Switzerland, Austria, Germany) have the higher number of long tunnels among all European countries, and so national law poses particular attention to safety imposing strict rules over every technical aspect such as traffic lights, SOS, by-passes, escape signs and fire fighting devices. This excess of rules is resulting in a very “overload” environment where scared people get easily confused instead of being guided to safety places. This situation is even worst in Italy, the country that has the highest number of road tunnel in Europe (50% of the total). In a recent study (2005-2008) led by German Automobile Club (ADAC) in cooperation with other 11 Automobile Clubs, on a sample of 22 Italian tunnels examined 14 did not respect the minimum safety criteria presenting serious problems (emergency exits missing, fire fighting system inappropriate, etc.) and only 8 passed such examination. K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 58–67, 2011. © Springer-Verlag Berlin Heidelberg 2011

A Simplified Human Cognitive Approach for Supporting Crowd Modeling

59

Based on this research the authors focused on the development of simulation models able to provide more effective measures for managing emergency situations in order to identify critical situation and provide support to better applying the EU technical rules increasing the safety level quickly. This paper presents, after a general methodology, a case study focused on a real 4,5 km tunnel, considering some of the most important aspects of human behavior like physical, emotional, cognitive and social factors, which can have a serious impact on the management of the emergency with the final goal to design a better safety in tunnels.

2 A Simulation Based Methodology: PECS Reference Models In the search for a methodology suitable for studying the human behavioral aspects of the tunnel fire escape the author identified the Physical conditions, Emotional state, Cognitive capabilities and Social status reference model (PECS). PECS is devoted to simulate human behavior embedded in a social environment and according to this definition, human behavior presents a complex structure that can be identified in four different classes of factors, or state variables: physical, emotional, cognitive and social. The human being is consequently - according to Schmidt - perceived as a psychosomatic unit with cognitive capacities embedded in a social environment. The four classes of factors of PECS reference models play a significant role in the behavior control: the model of the human behavior has to take into account the following state variables, in order to be comprehensible and predictable: 1. Physical State Variables, in this case the ability to perceive, decide and eventually run away to safety if not poisoned or injured by smoke or fire. 2. Emotional State Variables, related to the level of fear, angry or courage experienced by the subject. 3. Cognitive State Variables, based on the capacity of correctly interpreter the meaning of signs or the evolution of the current situation. This point is very important since the higher casualty level is among the people that remain in the car too much before decide to escape. 4. Social State Variables, that are playing an important role in the first minutes where leaders emerges and literally coach the crowd to safe or to death. The modeling type of human behavior depends on the nature of the problem; it is not mandatory to model all the four classes of state variables and/or all the interactions among them but the considered PECS model considers all the four classes similarly to the one discussed by Schmidt: there are danger points to be avoided, lives and health to be preserved and the needs of using of cognitive capacity in order to find the better solution possible as well as the social component: people could significantly affect the individual behavior, in terms of cognitive capacity and fear, in fact, without an appropriate cultural basis; the individual is pushed to behave “as the others do”. Like in Schmidt’s Adam model the proposed approach considers different aspects: • Environment: Like Adam lives in a world that is dynamic and constantly changing the individual has to continuously deal with this environment. • PECS State Variables: Individual physical state is represented by the energy level, while the emotional aspect is described by fear that can turn into panic. Adam has

60

•

•

•

•

•

E. Briano et al.

also cognition of himself and his environment like the individual in the tunnel that has to preserve its life and control its fear. Changes of the internal variables: The state variables are subjected to continuous changes during the simulation time. Changes occur in two ways: autonomously and/or triggered by an input; for instance health or energy level is subjected to both changes, in fact it decreases slowly with time and, when Adam falls in a trap or in a danger points, the decrease is slighter. Vice versa, when Adam reaches a food source, his energy level increases, depending on the amount of food consumed, until it reaches a maximum value. Fear variable behaves in the same way: it decreases when Adam becomes more confident of himself and his world but slightly increases when Adam falls in trap. Deliberative and Reactive behavior: The first behavior type follows simple rules, driven for instance by leading motives like hunger drive or the need of thinking, while deliberative behavior needs a predetermined planning action to reach a determined goal. Internal and external actions: Adam’s actions repertoire contains external actions, which have impact on the surrounding environment like gathering a food source, but also internal ones, which only affect Adam himself, like the planning or the thinking action. Same approach to the individual in the tunnel he can struggle to protect his car from fire (like too many do) or find a way to run away to safety. Learning: Adam is characterized by cognition of himself and his world and, thanks to his experience, recognizes some processes of the world where he lives in, like the food regeneration process, knowing even better the time it takes for the food to grow again. However this cognitive process is not sudden, but it is a process in a continuous evolution, because Adam initially does not know the food growth speed, but he learns that gradually in order not to mistake. In such way the individual can rapidly learn that some ways are interrupted by fire and take more long route to finally get safe. Forgetting: Adam is capable to learn but is also subjected to forgetfulness because he is human and his memory capacity is finite. Since the total time spent by individuals in the tunnels is usually very little no forgetting was taken into account developing this model.

The model described in the following section has several points in common with Adam’s model and PECS reference models, because, like Adam, the individual in the tunnel has to preserve his life avoiding danger points (fire, smoke), reaching the by passes (instead of the food sources) using the cognition to recognize the right road signs. In addition however he has to deal with other people that can affect his choices with their fear and their own behavior.

3 Modeling Human Perception and Judgment Using AHP A critical situation like a tunnel evacuation is, involves several parameters that affect the individual choice; for this reason this situation can be addressed to a multi-criteria decisional problem.

A Simplified Human Cognitive Approach for Supporting Crowd Modeling

61

The individual, acting as a decision maker, detects a set of alternatives to be assessed on the basis of high-level criteria and relative sub criteria determined by empirical studies; each criterion has a percentage of importance and reports a score, which determines the impact of the criterion on the decision. The score of each decisional alternative is the weighted average of each criterion scores on the decision multiplied by the weight assigned to each criterion. In order to practically describe the methodology the authors presents a simple case applied to the evaluation of the attractiveness of signs, in this example the sign typologies have been considered in such way: • The IsFireFight type: referring to a target (i.e. fire extinguisher) that allows the individual to take active actions in order to limit the damage to the property and the life. • The IsExit type: denotes a path with the possibility to reach safety. • The IsPhone type: denotes a path with the possibility of establish a communication with the rescue service (i.e. 911). • The IsSOS type: allowing putting in denotes a path/place with the possibility of activation of a fire alarm. Each individual will evaluate every sign type based on its internal know how, this is the sum of its training level and its natural attitudes; in particular for the considered example, only three categories of people have been considered but many others can be subsequently added: 1. “Informed” Individual (or trained). Associated with the higher culture in safety matter, generally is a truck driver. This individual will tend to choose the alternative able to save his life. 2. “Uninformed” Individual (or not trained). Without specific technical knowledge is generally a car driver, he will tend to preserve its property and, subsequently, its life. 3. “Hero” Individual. It is making himself useful for the others, is generally a Medical Doctor, a military, a retired fire fighting men or other people usually involved in rescue activity. For each one of these typologies the importance of each sign type has been calculated with the AHP methodology. As an example, the procedure for Informed Type users is explained in detail; as a first step the behavior local importance matrix {Xij} is built considering the importance of the ith line respect to the jth, column, for instance, considering the line 1 (altruism) and the column 2 (property) the following question for an “informed” individual arise: how is the altruism important respect to the property? If the behavior importance displayed in the line (altruism) is greater than that in the column (property), the Xij element of the matrix is a number greater than 1 (on the contrary we would have put a number lower than 1). Pj =

∏X

ij

i

⎛

∑⎜ ∏ X j

⎝

i

ij

⎞ ⎟ ⎠

(1)

62

E. Briano et al.

∑ (Xj = ∑⎡⎣⎢∑ (Xj n

Wg j

h=1 n

h=1

k

jh

⋅ Ph )

kh

⎤ ⋅ Ph )⎥ ⎦

(2)

Multiplying the local matrix importance {Wj} the global importance or attractiveness of j-th artifact Wgj is obtained and is used by the simulation to attract the individual in one direction instead another other. In the same way the other two behavioral typologies have been analyzed, in particular for the Uninformed Type user, which represents the most common type of car driver, there is no sign more attractive than others, especially if the signs have the same appearance and choice is merely made by chance. For the Hero Type user the Fire Fighting related signs are the most attractive as expected. After determining the sign/target attractiveness for every type of individual involved in the scenario, it is necessary to specify their behavioral time evolution: this is the core of the simulation. This process could be divided in a sequence of logical processes, schematically we have: 1. The individual is “activated” (simulation start) in the post-event condition with a “normal” breathing rate without any Carbon Monoxide (CO) intoxication and is associated to a predefined kind of individual according statistics (i.e. informed, uninformed and hero). 2. The individual starts to take awareness of its situation (it is still inside the car/truck/bus). 3. After a stochastic period of time, it exits from the car/truck/bus and start to look around for a decision, in this process the exiting from inside the vehicle is modeled as a general m/m/1/gd/gd/n queue according to the classical Markovian classification, not all the people inside the car may exit simultaneously its depends of how many doors has the vehicle and how many individuals are using such doors, generally they exit one after the other. 4. Once exited from the car, the individual is in front of a series of possible choices, which determinate eventually its path to safety: 5. According to the physical conditions of the place where find himself (i.e. heat, smoke) the individual identifies a target subset; he determines an eventual path towards the chosen target by excluding from the alternative those that cannot be followed because of heat or obstacles. The choice is based on the instinct and the importance assessment calculated with the above-mentioned AHP multi-criteria analysis. 6. The individual, after its choice has to evaluate the distance to cover and, according to the physical and environmental conditions, begins to run towards his target rather than simply walk trying to reach its desired target. Along the way it may evaluates other eventual alternatives, considering the shortest distance among the possible safe paths (obstacles, heat). When it is close to its target it generally slows down and turns eventually his run into walking. 7. Reaching its target the individual will stochastically evaluate the situation resulting in a possible set of decisions: it may have reached a safe place, it may have reached a sign that push him to go towards another target or it may get confused starting

A Simplified Human Cognitive Approach for Supporting Crowd Modeling

63

another evaluation of possible targets. In this last case the individual will return to process 4 until the individual achieves its safety or, eventually, dies. While the time passes the individual has its physiological process (breathing, heartbeating, etc.) that will involve smoke and CO present in the tunnel resulting in possible intoxication with several notable effects (i.e. confusion, panic, reasoning difficulties, slowing, etc.). Extreme heat is present in the scenario and may produces damages: causing hyperthermia, pain and skin burns forcing the individuals to take extreme decisions that under normal circumstances will never be take into consideration (i.e. crossing a fire to get out of a blocking situation, runaway from smoke far from a sign marking a possible emergency exit, etc.). In such way the proposed model is either driven by rational reactive behavior or by the individual instinct, which can be affected by artifacts encountered in the escaping path (i.e. SOS, Fire Extinguisher, Phone, First Aid Kits, etc.). The model is also able to explain real life observed behavior often caused by instinct, fear and panic that could significantly affect the rationality of the choice. For instance an individual, at the beginning of the accident, is strongly torn because it can not decide between leaving the car reaching safety or stay in because he does not want to leave an expensive properties, like car generally is, burning and so it tries until the last moment to preserve its properties. This indecision can cause anxiety, increasing fear and panic level, inhibiting any rational choice especially for Uninformed Type people. All these behavior has been extensively observed during the experimental campaign: hours of real tunnel camera videotape were carefully studied by the authors and later simulated on two real life exercises. First exercise was the reconstruction of an accident occurred in a mono-directional tunnel (axial ventilating system) and the other was related to the scenario reconstruction of a car accident in a bi-directional tunnel (cross ventilating system). In these real-life simulation two different tunnels were closed by police at 2.00 AM, crashed cars were positioned and artificial smoke was created to re-create the scenario drama. In this situation Fire Fighters made their intervention and the authors were present in order to observe and study the behavior of the people involved. After this real life simulation a computer based model was create to reproduce the behavioral choices of the involved people. This modeling phase is explained in the next section.

4 Implementing a 2D Simulation Model: Methodology and Case Study The implemented evacuation model presents two complementary and synergic modules with a graphic user interface (GUI): • Data input and scenario creation module: it is implemented using bCAD™ software and it is able to map the tunnel, to position the signs/artifacts and the safety devices (fire fighting systems, SOS columns, etc…) along the carriageways, defining, at the same time, the artifacts characteristics (i.e. height, colors, brightness, dimensions, etc.). This module help to build the accident scenario by positioning the vehicles (cars, trucks and buses) involved as well as to specify the burning ma-

64

E. Briano et al.

terials (i.e. plastics, paper, wood, hydrocarbons, rubber, etc.). Data were validated against the American Society for Hazard and Risk Engineers Handbooks. • Evacuation detail module: This module devoted to simulate the behavior of individuals using the above presented methodology. The bCAD™ interface is similar to most of the commercial 2-D drawing tool like AutoCad™ where the authors have developed a specialized scenario creation toolbar Thanks to this application it is possible to localize the cars/trucks/buses involved in the accident, the artifacts and the safety systems as well as locate the fire starting point with its flash power (i.e. 30 MW in the considered scenario). The evacuation module allows the users to observe the fire and smoke evolution at different time steps. In particular it is traced: − the evacuation paths of the individuals involved in the accident; − the smoke concentration and the heat radiance and the heat. The case study reproduces an accident in a 4.5 km long tunnel triggering a fire of 30 MW power starting from a truck, the accident involvers 38 cars, 2 buses each with 60 people on board. The simulator outputs analyzed as KPIs (Key Performance Indicators) are: 1. the number of people not escaped from the tunnel at the end of the 7 minutes simulation time; 2. the number of people escaped after the simulation time; 3. the number of people injured or intoxicated by carbon monoxide (CO) at the end of the simulation time; 4. the minimum, maximum and average exiting time from the vehicles; it represents a sort of “reaction time” after the fire explosion; 5. the minimum, maximum and average escaping time from the tunnel bypass, which is calculated starting from the fire exploding until the escaping from the bypass. These outputs have been set as objective functions for Analysis of Variance (ANOVA) devoted to evaluate the effect of a new artifact layout in the tunnel (i.e. provide new road sign configuration, new ventilation procedure, etc.). The results of the analysis on the main objective functions are shown in the following table 1. Table 1. ANOVA results from experimental campaign Origin of Variation

Variance

DoF

Mean Squared (MS)

# People left in at simulation end time

48.17

1

48.17

Error

14.67

4

3.67

Total Variation

62.83

5

# People Exited Safely

37.50

1

37.50

Error

19.33

4

4.83

F

Ftab

13.14

7.71

7.76

7.71

A Simplified Human Cognitive Approach for Supporting Crowd Modeling

65

Table 1. (continued) Total Variation

56.83

5

Max Permanence Time

607.22

1

607.22

Error

179.68

4

44.92

Total Variation

786.90

5

13.92

7.71

The results obtained clearly show how the new artifact layout significantly affects some KPI: the maximum permanence time in the tunnel, the number of people remained in the tunnel at the end of simulation and, even in a shorter measure, the number of the safely escaped people. On the contrary the new artifact layout position does not affect the minimum and average permanence times as well as the number of injured people. The experimental campaign was than extended with a 23 Factorial Experimental Design in which these factors have been considered: 1. Smoke speed: expressed in m/s of smoke advances in the tunnel by effect of the ventilation, values ranging from 0.2 to 0.8 m/s. 2. Exit (green) sign radiance measured in lux adopted values ranging from 100 lux (minimal legal value) and 800 lux (maximum legal value). 3. Safe (red) sign radiance measured in lux adopted values ranging from 100 lux (minimal legal value) and 800 lux (maximum legal value). The results of such experimental campaign are discussed in the following along with an example of a Response Surface. The number of people remaining trapped in the tunnel at the end of the simulation is affected both by the smoke spreading speed and the sign luminous intensity. Particularly the smoke spreading speed is definitely the most significant factor. Number of Individuals @End Simulation Time (7 min) 800,0

722,2

30,0-40,0

566,7

20,0-30,0

488,9

10,0-20,0

411,1

0,0-10,0

333,3

Safe (red) Radiance [lux]

644,4

255,6

177,8

0,2

0,3

0,3

0,4

0,5 0,5 0,6 Smoke Speed [m/s]

0,7

0,7

100,0 0,8

Fig. 1. Example of the RSM obtained from experimental campaign

66

E. Briano et al.

The Fig. 1 points out as at the smoke spreading speed increasing, the sign luminous intensity becomes irrelevant on the other end in case of low smoke spreading speeds, at the sign luminosity increasing it follows an increase of the people who reach the safety. In other words, once the smoke invades the tunnel it does not exist a luminous intensity value, among those allowed by the enforced standards able to orient the people towards the exit way. The regressive model points out, moreover, as the Exit signal must prevail with respect to the other signs. The Safe signs create indeed a background “noise” worsening the tunnel safety (in terms of users managing to reach the exit). The waiting time inside the motor-vehicles, before to decide to run away is barely affected: it exists only a slight dependence from the smoke speed on the waiting average time. The more rapidly the users see the smoke coming towards them the more quickly they will decide to left the car and run away. As for the waiting time in the car it does not exist significant variations about the maximum and minimum time spent to reach the safety. By converse the average time spent in the tunnel to reach a safety exit is affected both by the smoke speed and by the sign luminous intensity. Particularly with the smoke speed increasing, the average stay time in the tunnel of the people who did not managed to reach the safety decreases. An individual has to manage to reach safety immediately waiting the smoke make the decision to escape too late exposing the individual to a certain death. In this way the abuse of traffic lights inside the tunnels, as suggested by many, will turn into a level of confusion to untrained people resulting in a higher risk of death and injuries (intoxication/burn) since the sign luminosity increases the permanence time in the tunnel. The authors also developed a 3-D model devoted to analyze the flows of the people escaping and, in particular, the smoke evolution dynamics inside the tunnel. The aim of this 3-D model is to provide a feedback about the evolution of the smoke propagation that the evacuation model is not able to give; in fact smoke propagates mainly on the vertical axis, so a 3-D model is required to study the phenomenon; The 3D model has been rendered and inserted in a virtual scenario in which a car can be driven by the user in order to inspect the situation. By now there is only one type of vehicle but it is possible, in the future, to model ambulances or fire department trucks in order to provide a credible environment for training purpose.

5 Conclusions The proposed model results compared with real life observation demonstrate the validity of the approach, in particular during validation of the model subject matter experts recognize emerging behaviors as very similar to the one observed in many real life situation. The EU 2004/54/CE -Dispositions concerning Road Tunnel Safety- mention the possibility of use computer simulation instead of real life simulation but the today available computer simulators are still far from be a credible alternative to practical training and, for such reason, Italian D.Lgs. 5/10/2006 n. 264 provide no support for computer based simulation despite the fact that is simple the translation of cited 2004/54/CE forcing road tunnel owners to adopt very expensive real life simulations.

A Simplified Human Cognitive Approach for Supporting Crowd Modeling

67

This approach could be regarded as a significant starting point for future research in the field of cognitive system simulation allowing the possibility to adopt computer simulation to ensure higher level of safety in nowadays tunnels at a fraction of the cost.

References Schmidt, B.: The Modelling of Human Behaviour. SCS Europe Press, Ghent (2000) ISBN 156555-211-3 Benevolo, P.: Tunnel sotto Esame. Onda Verde (119), 38–39 (2009) Urban, C.: PECS – A Reference Model for the Simulation of Multi-Agent Systems, Tools and Techniques for Social Science Simulation. Physica Verlag, Heidelberg (1999) Rao, A.S., Georgeff, M.P.: BDI-Agents: From Theory to Practice. In: Proceedings of the 1st International Conference on Multi-Agents Systems (ICMAS), San Francisco, CA, USA (1995) Goleman, D.: Emotional Intelligence. Bloomsbury Publishing Plc., London (1996) Schmidt, B., Touissant, A.: Referenzmodelle fur Strategien SiP. Heft 3, 8–15 (1996) Saaty, T.L.: Decision Making for Leaders: The Analytical Hierarchy Process for Decisions in a Complex World ISBN 5-3497959-9, Wadsworth, Paperback, RWS (1988) ISBN 09620317-0-4 Banerjee, B., Abukmail, A., Kraemer, L.: Advancing the Layered Approach to Agent-Based Crowd Simulation. In: Proceedings of PADS 2008, Rome, Italy, June 3-6 (2008) ISBN 9780-7695-3159-5 Lee, S., Son, Y.J.: Integrated Human Decision Making Model Under Belief-Desire-Intention Framenwork for Crowd Simulation. In: Proceedings of WSC 2008 (WinterSim Conference, Miami, FL, USA, December 7-10 (2008) Kaup, D.J., Clarke, T.L., Oleson, R., Malone, L., Jentsch, F.G.: Introducing Age-Based Parameters into Simulations of Crowd Dynamics. In: Proceedings of WSC08 (WinterSim Conference, Miami, FL, USA, December 7-10 (2008) Briano, E., Revetria, R., Testa, A.: Behavior Models for the Evacuation of a Motorway Tunnel. In: Proceedings of WSEAS ICOSSE 2009, Genoa, Italy, October 17-19 (2009) Briano, E., Revetria, R.: A Study of Crowd Behavior in Emergency Tunnel Procedures. International Journal of Mathematics and Computers in Simulation 2(1), 349–358 (2008)

Model Checking Epistemic and Probabilistic Properties of Multi-agent Systems Wei Wan1 , Jamal Bentahar2, and Abdessamad Ben Hamza2 1

2

Department of Electrical and Computer Engineering, Concordia Univeristy Concordia Institute for Information Systems Engineering, Concordia University w [email protected], {bentahar,hamza}@ciise.concordia.ca

Abstract. Model checking, a formal automatic verification method, has been widely used in multi-agent systems to verify specifications that contain qualitative properties (e.g safety and liveliness) and quantitative properties. Decision making processes based on inherent knowledge are necessary for agents to act appropriately, particularly in uncertain settings. In order to check epistemic (i.e knowledge) and measurable properties in multi-agent systems, we propose a new logic P CT LK, which uses probabilistic, epistemic, and temporal modal operators. We exploit Discrete-Time Markov Chains (DTMC), in which we are able to represent measurable properties with probability, to model uncertainty in multi-agent systems. We extend the formalism of interpreted systems by adding probabilistic features to suit DTMC models and to present the model checking algorithm for our logic. At the end of this paper, we simulate our algorithm using an example of online shopping. Keywords: Probabilistic model checking, Discrete-Time Markov Chains (DTMC), Epistemic logic, Probabilistic logic, Interpreted systems.

1

Introduction

Multi-agent systems are comprised of a set of intelligent agents interacting with each other. In such systems, reasoning with uncertainty is an important feature. Model checking, a well-designed formal technique for verifying if models satisfy specific properties, is widely used to verify the epistemic properties of these systems [9,10,13]. In the past two years, researchers in [3,4,7] have proposed methods to model and check multi-agent systems using Markov chains, which are stochastic processes behaving as transition systems extended by probabilistic choices among successor states. The reason why probabilistic model checking is gaining more and more interest is that classical model checking techniques focus only on the absolute guarantee of correctness and some assumptions are made world that systems run ideally. However, in practice concrete scenarios are characterized by a degree of uncertainty. For example, in multi-agent systems, because of the agents’ autonomous reactivity, their actions are based on observing the environment changes, but very often agents cannot observe the complete environment. Therefore, agents K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 68–78, 2011. c Springer-Verlag Berlin Heidelberg 2011 

Model Checking Epistemic and Probabilistic Properties

69

must make estimations through the observable world as a part of their decisionmaking processes. Considering probabilistic aspects when modeling the system allows providing such estimations by assessing the likelihood of different events. Two categories of properties exist [1]: immeasurable qualitative properties and measurable quantitative properties. For qualitative properties, Boolean variables (true or false) can be used to express whether a good event will happen or a bad event will never happen. However, sometimes the likelihood of certain events need to be considered. For example, we may want to know “after ordering, what percentage of items will be successfully delivered”. In this case, qualitative properties cannot provide the precise probability required. Quantitative properties are also necessary to accurately express the system requirements, and probabilistic model checking seems to be appropriate. Checking quantitative properties for knowledge-based multi-agent systems is the main motivation of this paper. Because the majority of existing multi-agent logics only consider certain knowledge, modeling and verifying uncertain knowledge still need more investigation. Uncertain knowledge can be represented using probabilities and the multi-agent system can be modeled as a probabilistic Kripke-like model. In this paper, we define a new logic called probabilistic temporal logic of knowledge P CT LK, that extends P CT L, the probabilistic computation tree logic proposed in [6], by the K operator of knowledge. We also extend the classical interpreted systems used to model multi-agent systems [5] by adding probabilistic transition functions. This extension allows us to model interpreted systems as Discrete-Time Markov Chains (DTMC), which are transition systems with probability distributions. Two reasons are behind the use of DTMC; first, DTMC is the underlying formal model of P CT L, and second, DTMC are widely used to model systems with uncertain information. In order to verify properties expressed with P CT LK, we put forward the model checking algorithm for this new logic. The structure of this paper is organized as follows. In Section 2, we explain how we extend the normal interpreted systems to be modeled using DTMC. In Section 3, we present the syntax and semantics of the P CT LK logic. The model checking problem of P CT LK is described in Section 4. We provide an example, customer online shopping, to simulate the model checking algorithm in Section 5. Finally, we summarize our work and list the future work in Section 6.

2

Interpreted Systems for DT M C

Probability is used in the design and analysis of an agent to measure the likelihood that some specific events will occur. There are several methods of systems modeling to express the probability attributes. One of the most popular operational probabilistic models is Markov chain [1], which is a transition system with probability distributions over the transitions. DTMC is one of three Markov chain models, in which a system is in a given state at each “step”, with the state changing randomly between steps.

70

W. Wan, J. Bentahar, and A.B. Hamza

In Markov chains, the process can only be in a finite set of states. Over a set of atomic propositions AP , a model of DTMC can be expressed as a tuple {S, P, Iinit , L}, where: • S is a nonempty and finite set of states. • P : S × S → [0, 1] is the  transition probability function, such that for every state s ∈ S, we have s ∈S P (s, s ) = 1. • I init : S → [0, 1] is the initial distribution such that for all states s ∈ S, s∈S Iinit (s) = 1. • L : S → 2AP is a state labeling function. For mathematics treatment purposes, the initial distribution Iinit can be viewed as an ordered list of row vector (Iinit (s))s∈S , in which the value of every row represents the initial probability from all states in the model. The transition probability function P : S ×S → [0, 1] is represented by the matrix (P (s, t))s,t∈S . The probabilities of state s to its successors t are shown on the row of the matrix, while the probabilities of entering state s from start t are shown on the column of the matrix. According to Fagin et al in [5], in order to model knowledge, we need a framework based on a number of other possible states of affairs besides the true state of affairs. The formalism of interpreted systems, which provides a well-defined semantics to reason about time and knowledge in multi-agent systems, is frequently used in epistemic model checkers, such as MCMAS [11] and MCK[13]. We extend Fagin et al.’s interpreted systems with probability attributes. We then introduce our probabilistic model checking algorithm. Let A = {1, . . . , n} be a set of n agents in the system. Every agent i ∈ A is associated with its local state set Li , and possible actions set Acti . Besides Li and Acti , there are also a set Le and a set Acte for the environment, a special agent for providing global variables and actions that all agents are able to access. Therefore, for the system, a set of global states G ⊆ L1 × . . . × Ln × Le is the set of all possible tuples (l1 , . . . , ln , le ), and each tuple represents a computational state for the whole system. For each agent i, we also use a set of protocols Pi : Li → 2Acti assigning a list of enabled actions to each local state. Associated with the environment is a protocol Pe : Le → 2Acte that represents the functioning behavior of the environment agent. The probabilistic transition function T for the system can be defined as T : G × Act × PAct × G → [0, 1], where Act is the set of joint actions Act ⊆ Act1 × . . . × Actn × Acte that are performed by all the agents and environment respectively. Each agent is associated with a local probabilistic transition function ti ⊆ T . PAct is a probability function of the set of joint  actions. For every global state g ∈ G, with transition to state g  ∈ G, g ∈G PAct (g, g ) = 1. PActi is the local probability function for each agent i. Given a global initial distribution Iinit , for all states s ∈ G,  Iinit (s) = 1. Given a set of atomic propositions AP and an interpretation V ⊆ G × AP , an interpreted system over probabilistic transition function is a tuple: IS =< (Li , Acti , ti , PActi , Pi )i∈A , Iinit , V > Now, we need to associate DTMC, which is a Kripke-like structure extended with probabilities, to our extended interpreted system in order to evaluate

Model Checking Epistemic and Probabilistic Properties

71

 epistemic probabilistic properties. The resulting model is MIS = (W, Pt , Iinit , ∼1  , . . . , ∼n , V ). The converting rules from Interpreted system IS to our DTMC interpreted system MIS are described as follow.

• W ⊆ G is the set of reachable states. The state is reachable if and only if the probability is greater than 0 from initiation distribution via T .  ⊆ Iinit is the initial distribution of the model. • Iinit • Pt : W × W → [0, 1] is a probability relation, which is obtained by protocols Pi and the probabilistic functions ti , which are subsets of the transition probability function T ; • ∼i is the epistemic relations, one for each agent. It is a subset of W × W for i ∈ A, such that for two global states, (l1 , . . . , ln ) ∼i (l1 , . . . , ln ) if and only if li = li ; • V  ⊆ V is the valuation function V  : W × AP → {true, f alse}.

3

P CT LK Logic

The language we will use to specify properties is called Probabilistic Computation Tree Logic of Knowledge or P CT LK. This branching-time logic extends Computation Tree Logic (CT L) proposed by Clark et al. in [2] by adding epistemic and probabilistic operators. Thus this language combines CT L logic, epistemic logic [5], and probabilistic logic [1,6]. 3.1

Syntax of P CT LK

P CT LK supports two kinds of formulae: state formulae φ and path formulae ψ. We use p, p1 , p2 , . . . to range over the set of atomic propositions Φp . Given a set of agents A = 1, . . . , n, the syntax of this logic is as follows: φ ::= true | p | φ ∧ φ | ¬φ | Ki φ | P rb (ψ) ψ ::= φ | φ ∪≤n φ | φ ∪ φ where b ∈ [0, 1] is an interval giving rational boundary for the path,  ∈ {<, ≤ , ≥, >} is a relational operator indicating the kind of relationship boundary of the probability, and n ∈ N is an integer indicating the maximum steps to achieve a specific state. There are no universal (∀) and existential (∃) path quantifiers in P CT LK. Instead, the linear temporal operators  (next) and ∪ (until) are required to follow the probabilistic operator P r immediately. By removing Ki φ from this logic we obtain P CT L [1,6] and by removing φ ∪≤n φ from P CT LK and introducing the ∃ quantifier we obtain CT LK [5]. Three fragments, a temporal logic, an epistemic logic, and a probabilistic logic, are defined for P CT LK in addition to the standard Boolean connectives. The propositional temporal fragment has the same meaning as in CT L. For example, the formula φ has the meaning of “next step φ holds”. φ1 ∪ φ2 means “φ1 holds until φ2 ”. A new step-bounded variant of φ1 ∪≤n φ2 is added, meaning that φ2 will hold within at most n steps while φ1 holds in all states before φ2

72

W. Wan, J. Bentahar, and A.B. Hamza

state has been reached. The step-bounded until is necessary in probabilistic logic because the probability for at most n steps to reach φ2 may be different from at most n + 1 steps to reach φ2 . As in CT L, temporal operators  and ∪ are required to be immediately preceded by a path quantifier; in P CT LK, they must follow the operator P r immediately. Other Boolean connectives and operators are derived in the same way as in CT L, for example, ♦φ = true ∪ φ (in the future φ). The step-bounded future can be similarly obtained by step-bounded until: ♦≤n φ = true ∪≤n φ. The always operator can be derived by formula: P r≤b (φ) = P r≥(1−b) (♦¬φ). The details can be found in [1]. The probabilistic operator P rb (ψ) expresses that “ψ is true with probability b”. For illustration, P r≥0.9 (message receive) asserts that “with probability at least 0.9, the next step message will be received”. The epistemic operator Ki is a S5 modality [10], which means knowledge is reflexive and Euclidean [12]. Ki φ represents “agent i knows that φ”. There are some important properties of the epistemic operator. For examples, Ki φ ∧ Ki (φ ⇒ ψ) ⇒ Ki ψ means when an agent knows that φ and knows that φ ⇒ ψ, the agent knows ψ as well. Agents also know what they know and what they do not know. These properties can be expressed as Ki φ ⇒ Ki Ki φ and ¬Ki φ ⇒ Ki ¬Ki φ. More properties and their proof can be found in [5]. We have also interesting properties combining knowledge and probabilities, for example: Ki (P r≥0.5  φ) ⇒ P r≥0.5  φ. 3.2

P CT LK Semantics

Let s ∈ S be a state, π a given path, a ∈ AP an atomic proposition, φ a P CT LK state formula, and ψ a P CT LK path formula. A path is an infinite sequence of states related by transitions, i.e., π = s0 , s1 , s2 , . . . The (i + 1)th state in π is denoted π(i) i.e., π(i) = si . s |= φ denotes “s satisfies φ” or “φ is true in s”. π |= ψ denotes “π satisfies ψ” or “ψ is true in π”. The semantics of P CT LK is as follows. • For a state s s |= a s |= φ1 ∧ φ2 s |= ¬φ

iff V (s, a) = true iff s |= φ1 and s |= φ2 iff s  φ

• For a path φ: π |= φ iff π(1) |= φ π |= φ1 ∪≤n φ2 iff ∃0 ≤ k ≤ n, π(k) |= φ2 and ∀0 ≤ i < k π(i) |= φ1 π |= φ1 ∪ φ2 iff ∃k ≥ 0, π(k) |= φ2 and ∀0 ≤ i < k π(i) |= φ1 • For the epistemic operator s |= Ki φ

iff ∀s ∈ W if s ∼i s then s |= φ

For the probabilistic operator, s |= Pb (ψ) means that “the probability, from state s that ψ is true for an outgoing path satisfies b”. For example, s |=

Model Checking Epistemic and Probabilistic Properties

73

P<0.25 (true ∪≤5 φ) asserts that “the probability that the system reaches a φ being true within 5 steps of outgoing paths from state s is less than 0.25”. Given P rop to represent the probability, semantics of the probabilistic operator P r is given as follows: • For probabilistic operator iff P rop(s |= ψ) is in the range of b s |= P rb (ψ) where: P rop(s |= ψ) = P rop{π ∈ P ath(s)| π |= ψ}

4 4.1

PCTLK Model Checking Overview

Modeling, specification, and verification are three steps in the model checking process. In our approach, DTMC are used to describe the system models; then we convert DTMC models into interpreted systems for DTMC. P CT LK is used as our specification language to state the properties that the system must satisfy. The model checking algorithm for P CT LK is introduced in this section. The structure of our model checking approach is shown in Fig 1.

Fig. 1. verification work flow for P CT LK

4.2

SAT-Based Model Checking

Given a DTMC model M , we can generate an extended interpreted system MIS . Given a state s in MIS and a P CT LK state formula φ, the model checking determines if s |= φ. Due to existing quantitative properties, the computing result of probabilities is also needed in some cases. In fact, our P CT LK language can be seen as a variant of CT LK [12] by replacing the path quantifiers ∃ and ∀ by the probabilistic operator P r. The MCMAS [11] model checker can be used to verify the properties expressed with CT LK. P CT LK provides possibility b for path formulas to specify the likelihood of the path. The critical values, 0 and 1, state two specific situations: P r≥1 (ψ) stands for the path formula ψ happens all the time; while P r≤0 (ψ) represents the negation of the path formula ψ. Therefore, when we constrain that the probabilistic relation can only be “≥ 1”

74

W. Wan, J. Bentahar, and A.B. Hamza

Fig. 2. Structure of P CT LK

or “> 0”, P CT LK turns into CT LK: P r≥1 is equivalent to ∀ and Pr>0 is equivalent to ∃. P CT LK also is an extension of P CT L [1,6] as it includes the epistemic operator Ki . The properties expressed using P CT L can be checked by the PRISM model checker [8]. Therefore, based on P CT LK syntax, a formula can be a CT L formula, a CT LK but not P CT L formula, a P CT L but not CT LK formula, a conjunction of CT LK and P CT L like Ki φ ∧ P r>0.5 (ψ), or a pure epistemic probabilistic formula under the form Ki (P rb (ψ)). The structure of P CT LK is shown in Fig. 2. In order to calculate the set of states Sat(φ) satisfying a formula φ in P CT LK, we first need to decompose down the formula φ into sub-formulae φ , and compute recursively Sat(φ ). This is done using a bottom-up traversal of the parse tree of φ. For example, the conjunction Ki φ ∧ P rb (ψ) is decomposed into a sub-formula of CT LK (Ki φ) and a sub-formula of P CT L (P rb (ψ)). Based  on an interpreted system MIS = (W, Pt , Iinit , ∼1 , . . . , ∼n , V  ), the algorithm for computing Sat(Φ) is shown in Table 1. Proof. The validity of case a through c is straightforward. The proof of cases d and e can be found in [10] and [1]. We only prove the case f . Proof of f consists of two parts: 1. Show that if s ∈ Sat(Ki (P rb (φ1 ))), s will satisfies Ki (P rb (φ1 ). s ∈ Sat(Ki (P rb (φ1 ))) ⇒ s ∈ Y ⇒ {s ∈ W |¬((∃s ∈ X) ∧ (s ∼i s ))} ⇒ {s ∈ W |¬∃s ∈ X ∨ ¬(s ∼i s )} ⇒ {s ∈ W |∀s ∈ X ∨ ¬(s ∼i s )} ⇒ {s ∈ W |∀s ∈ X} ⇒ {s ∈ W |∀s ∈ Sat(¬P rb (φ1 ))} ⇒ {s ∈ W |∀s ∈ Sat(P rb (φ1 )} ⇒ s  P rb (φ1 ) ⇒ s  Ki P rb (φ1 ) 2. Show that for any state s, properties φ1 is true and outgoing path satisfies b, it belongs to Sat(Ki (P rb (φ1 ))) set, s ∈ Sat(Ki (P rb (φ1 ))). s  Ki (P rb (φ1 )) ⇒ {s ∈ W |∀s ∈ W ((s ∼i s ) → s  (Ki P rb (φ1 ))} ⇒ {s ∈ W |∀s ∈ W (¬(s ∼i s ) ∨ s  (Ki P rb (φ1 )))} ⇒ s ∈ Y ⇒ s ∈ Sat(Ki (P rb (φ1 ))) The algorithm for the calculation of the set of states SatCT LK (φ) can be found in [10]. For P CT L, to determine if s ∈ SatP CT L (φ), where φ = P rb ψ, we need to compute the probability P rob(s |= ψ) for the event specified by ψ. Then the set SatP CT L (φ) can be calculated by Sat(P rb ψ) = {s ∈ W |P rob(s |= ψ)p} ( ∈ {<, ≤, ≥, >}). The details of the algorithm are discussed in [1]. To compute the probability for “next” operator (P rb [φ]), first we need  to compute Sat(φ), then we sum all the states in Sat(φ) : s ∈Sat(φ) Pt (s, s ).

Model Checking Epistemic and Probabilistic Properties

75

Table 1. Algorithm for P CT LK model checking Input:MIS = (W, Pt , Iinit , ∼1 , . . . , ∼n , V ), P CT LK formula φ Output: Sat(φ) set of states satisfying φ 1. Decompose the parse tree of formula φ 2. For all the sub-formula φ in φ { Case φ { a. φ is an atomic formula: return {g|V (g, a) = true}; b. φ is ¬φ1 : return W − Sat(φ1 ); c. φ is φ1 ∧ φ2 : return Sat(φ1 ) ∩ Sat(φ2 ); d. φ is in P CT L: return SatP CT L (φ ); e. φ is in CT LK: return SatCT LK (φ ); f. φ is Ki (P rb (φ1 )): { X = Sat(¬P rb (φ1 )); Y = {s ∈ W |∃s ∈ X and s ∼i s }; return W − Y ; } } }

→ − Therefore, we will obtain a vector by multiplying probability relation Pt with a → − bit vector bs for Sat(φ), in which bs = 1 when s ∈ Sat(φ), otherwise, bs = 0. In order to compute SatP CT L (P rb [φ1 ∪≤k φ2 ]), we first need to compute Sat(φ1 ) and Sat(φ2 ). Then we can identify the states that are for sure in the satisfaction set SIN = Sat(φ2 ), and for sure not in the satisfaction set SOUT = W \ (Sat(φ1 ) ∪ Sat(φ2 )). We also need to classify the uncertain states, which may or may not be in the satisfaction set, Sunclear = W \ (SIN ∪ SOUT ). After we divide the states, we can compute the probability solution with a recursive equation: ⎧ 1, ⎪ ⎪ ⎪ ⎨ 0, ≤k P rob(s, φ1 ∪ φ2 ) = ⎪ 0, ⎪ ⎪ ⎩

s ∈W

P (s, s ) · P rob(s, φ1 ∪≤k φ2 ),

if if if if

s ∈ SIN s ∈ SOU T s ∈ Sunclear and k = 0 s ∈ Sunclear and k = 1

For the “until” operator, we can consider “bounded until” with unlimited value. The probability ⎧ solution equation is then as follows: ⎪ if s ∈ SIN ⎨1, P rob(s, φ1 ∪ φ2 ) = 0, if s ∈ SOUT ⎪ ⎩   s ∈W P (s, s ) · P rob(s , φ1 ∪ φ2 ), otherwise If the formula is a pure P CT LK formula (Ki (P rb (φ1 ))), then we compute first the set X of states satisfying ¬P rb (φ1 ), then we return all the states in W except those that have access to the states in X.

76

W. Wan, J. Bentahar, and A.B. Hamza

Theorem 1. The complexity of P CT LK model checking in interpreted systems is PSPACE-complete. Proof. The model checking of P CT LK is composed of the model checking of: 1) CT LK; 2) P CT L; and 3) Ki (P rb (φ1 )). The complexity of 2) in interpreted systems is known to be PSPACE-complete, and the same proof can be used for 2). For 3), computing X needs polynomial space (complexity of P CT L), and computing Y needs only logarithmic space, as only tow states needed to be memorized. Thus, the whole complexity is PSPACE-complete.

5

Case Study

We use a simple online shopping example to simulate our algorithm. The customer requests a delivery, and the system will successfully deliver the goods in 95% of the cases, and will fail in 5% of the cases. The DTMC model is shown on Fig. 3.

Fig. 3. DTMC for online shopping

We have two agents in this system: a customer agent and a server agent. The local states for the customer agent is: {s0 }; while for the server is {s1 }. The environment agent states are {s2 , s3 }; Thus, the global state W = {s0 , s1 , s2 , s3 }. Initial distribution ⎤Iinit is [0, 1, 0, 0]. The probability relation Pt = ⎡ 0 1 0 0 ⎢0 0 0.95 0.05⎥ ⎥ ⎢ ⎣0 0 1 0 ⎦ 0 0 0 0 We would like to check that after a customer orders, he knows that at least 90% items will be successfully delivered, i.e. Kc (P≥0.9  (successf ul)) 1. Sat(succesf ul) = {S2 } 2. ⎡ P r((successf ul)⎤= ⎡ P •⎤P rop ⎤ ≥0.9 (successf ul)) = ⎡ Sat (P 0 0 0 1 0 0 ⎢0 0 0.95 0.05⎥ ⎢0⎥ ⎢0.95⎥ ⎥ ⎥·⎢ ⎥=⎢ ⎢ ⎣0 0 1 0 ⎦ ⎣1⎦ ⎣ 1 ⎦ 0 0 0 0 0 0 3. P r((successf ul)) = [0, 0.95, 1, 0], Sat(P≥0.9  (successf ul)) = {S1 , S2 } 4. Convert Kc (P≥0.9 (successf ul)) to checking P≥1  (P≥0.9  (successf ul)). This conversion reflects the semantics of Ki by representing the quantifier ∀ using P≥1 .

Model Checking Epistemic and Probabilistic Properties

77

5. ⎡ P≥1  (P≥0.9  (successf ⎡ = P⎤· P ropSat (P≥0.9 (successf ul)) = ⎤ ⎡ ⎤ ul)) 1 0 0 1 0 0 ⎢0 0 0.95 0.05⎥ ⎢1⎥ ⎢0.95⎥ ⎥ ⎥·⎢ ⎥=⎢ ⎢ ⎣0 0 1 0 ⎦ ⎣1⎦ ⎣ 1 ⎦ 0 0 0 0 0 0 6. P r((successf ul)) = [1, 0.95, 1, 0], Sat(P≥1  φ) = {S0 , S2 } S0 is a customer agent local state. Therefore, the formula is satisfied. This case study has been implemented in the PRISM and MCMAS model checkers.

6

Conclusion and Future Work

In the field of multi-agent systems, the properties of agents such as their knowledge, beliefs, and autonomous reactions and pro-actions are important aspects for modeling system. In this paper, we extended interpreted systems by adding probabilistic functions so that we are able to use them as DTMC and to model probabilistic multi-agent systems. We also introduced a new language (P CT LK) to cover epistemic and probabilistic features. we also designed an efficient model checking algorithm for our DTMC interpreted systems model and properties expressed using P CT LK. In our algorithm, there are three modules: propositional temporal module, probabilistic module, and epistemic module. These features allow us to use different model checking tools. We also proved that the complexity of our model checking is the same as for CT LK and P CT L. Delgado and Benevides[4] modeled multi-agent systems using DTMC with synchronization actions and defined K − P CT L logic to specify the properties. They use the PRISM probabilistic model checker to verify their work. They do not use or generate the interpreted systems for DTMC. Instead of using PRISM directly, we embed epistemic model checker package into it. In [7] Jamroga proposed Markov temporal logic M T L, an extension of the “Discounted CT L” (DCTL), which uses a discount factor to achieve the probabilistic factor. He used Markov Chains to model the multi-agent systems, but the main focus is on quantitative properties, not epistemic properties. Our probabilistic knowledge model checking is an ongoing project. In the future, we will focus on the following aspects: • Currently, our model checking algorithm is based on decomposing the parse tree of the formula. We are planning to use symbolic model checking methods, such as M T BDD-based (Multi-Terminal Binary Decision Diagrams) or OBDD-based algorithm. • our P CT LK language only includes the simple epistemic operator Ki . In the future, we expect to extended the logic to include more epistemic properties like common knowledge, distributed knowledge, and commitments, etc.

78

W. Wan, J. Bentahar, and A.B. Hamza

References 1. Baier, C., Katoen, J.: Principles of model checking. MIT Press, Cambridge (2008) 2. Clarke, E.M., Grumberg, O., Peled, D.: Model checking. MIT Press, Cambridge (1999) 3. Dekhtyar, M.I., Dikovsky, A.J., Valiev, M.K.: Temporal verification of probabilistic multi-agent systems. In: Anonymous Pillars of Computer Science, pp. 256–265. Springer, Heidelberg (2008) 4. Delgado, C., Benevides, M.: Verification of epistemic properties in probabilistic multi-agent systems. In: Braubach, L., van der Hoek, W., Petta, P., Pokahr, A. (eds.) MATES 2009. LNCS, vol. 5774, pp. 16–28. Springer, Heidelberg (2009) 5. Fagin, R., Halpern, J.Y., Moses, Y., Vardi, M.Y.: Reasoning about knowledge. MIT Press, Cambridge (1995) 6. Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects of Computing 6, 512–535 (1994) 7. Jamroga, W.: A Temporal Logic for Markov Chains. In: AAMAS 2008, Padgham, Parkes, May 12-16, pp. 697–704 (2008) 8. Kwiatkowska, M., Norman, G., Parker, D.: PRISM: probabilistic symbolic model checker. In: Anonymous Proceedings, April 14-17, pp. 200–204. Springer, Berlin (2002) 9. Lomuscio, A., Pecheur, C., Raimondi, F.: Automatic Verification of Knowledge and Time with NuSMV. In: IJCAI 2007, Hyderabadad, India, pp. 1384–1389 (2007) 10. Lomuscio, A., Penczek, W.: Symbolic model checking for temporal-epistemic logics. SIGACT News 38(3), 77–99 (2007) 11. Lomuscio, A., Raimondi, F.: MCMAS: a model checker for multi-agent systems. In: Anonymous Proceedings, March 25-April 2, pp. 450–454. Springer, Berlin (2006) 12. Penczek, W., Lomuscio, A.: Verifying epistemic properties of multi-agent systems via bounded model checking. Fundamenta Informaticae, 167–185 (May 2003) 13. Gammie, P., van der Meyden, R.: MCK: Model Checking the Logic of Knowledge. In: Alur, R., Peled, D.A. (eds.) CAV 2004. LNCS, vol. 3114, pp. 479–483. Springer, Heidelberg (2004) 14. Wooldridge, M.J.: An introduction to multi-agent systems. John Wiley & Sons, Chichester (2009)

Modeling Users of Crisis Training Environments by Integrating Psychological and Physiological Data Gabriella Cortellessa, Rita D’Amico, Marco Pagani, Lorenza Tiberio, Riccardo De Benedictis, Giulio Bernardi, and Amedeo Cesta CNR – Consiglio Nazionale delle Ricerche, ISTC Via S.Martino della Battaglia 44, I-00185 Rome, Italy [email protected]

Abstract. This paper describes aspects of a training environment for crisis decision makers who, notoriously, operate in highly stressful and unpredictable situations. Training such decision makers is the goal of PANDORA -B OX, a system which is able to teach a class of trainees representing different authorities that coordinate their interventions in critical situations. This paper dwells on the selection and modeling of the relevant human features that are shown to have an influence in decision making under crisis. The chosen features are used to create a trainee model on the basis of which the system adjusts the training exercises with the ultimate goal of maximizing the effectiveness of training. Trainees models are built by merging physiological and psychological data, and are represented by means of a timeline-based approach, a representation derived from planning technology. The infrastructure built for the trainee modeling constitutes the basis to assess the influence of specific variables (e.g., personality traits, self efficacy, stress and anxiety) on the performance of crisis managers during the training. Keywords: Cognitive Model, Training, Psychological and Physiological Data.

1 Introduction Public health and safety crises, such as hurricane, inundation, infectious disease pandemic are characterized by uncertainty, time pressure and high stakes, requiring good decisions made quickly by all those involved in the response effort, from the senior officials to the local personnel involved in the treatment of people affected by the crises [1]. During these crises there is a tremendous need of effective leaders. Research from a number of fields demonstrates that critical thinking and decision-making skills make it possible for people in at-risk situations to recognize pre-event factors that lead to injury events and to act promptly to prevent an injury event or to diminish its severity. Little research still exists about the specifics of how leaders effectively respond to a crisis (e.g., [2]) and how their personal characteristics can be evaluated in advance of a crisis occurring [3]. Nonetheless, it is very important to understand these processes in order to facilitate the development of capability-building interventions before crisis events occur and, in this way, try to prevent the negative consequences of the crisis itself. In this context it is evident how training plays an important function in the preparation of a crisis manager. A great amount of investment is being devoted to the development of training procedures to increase the ability of crisis managers to face emergency K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 79–88, 2011. c Springer-Verlag Berlin Heidelberg 2011 

80

G. Cortellessa et al.

situations. Two main modalities are used for training purpose: (a) the table top exercise (paper based exercises, leaded group discussions guided by a simulated disaster); (b) real world simulation exercise (field tests replicating emergency situations). Table top exercises are low cost and can be easily and frequently organized, but they cannot recreate the real atmosphere, in terms of stress, confusion and need to take critical decisions in very little time. On the other hand, crisis managers trained through simulation exercises on the field can be very effective and can gain valuable skills, but such simulations are very expensive and cannot be easily and quickly organized. Authors are working at the PANDORA project the aim of which is to try to bridge the gap between these two alternative modalities by providing an intelligent training environment, called the PANDORA -B OX, which will synthesize a realistic simulation able to reproduce the stressful factors and to adapt to different trainees. Some challenging aspects contribute to make the PANDORA -B OX an innovative tool in case of success: an “engaging” and realistic reconstruction of a training environment with evolving crisis scenarios to emulate realistic pressure and stress to introduce among trainees; a personalized environment that take into account specific aspects of each individual trainee and allow the personalization of training programs. The focus on the “affective state” of the crisis manager is given by maintaining knowledge of human behavior, in all phases of the emergency management class. It is worth underscoring how most of the current training environments target the operational level of the commanders, while PANDORA is completely devoted to the training of “strategic decision makers”, thus entailing the need to foster critical and creative decision making. This paper describes work done during the first year of the PANDORA project aiming at personalizing training according to trainees static and dynamic analysis. In what follows, Section 2 provides a brief description of the overall PANDORA -B OX architecture to give an idea of the building blocks of the training system, then Section 3 describes the main features that have been chosen to model the trainee’s affective states in connection with stress during a crisis. In particular the section describes the goal of obtaining a user model by integrating psychological and physiological data. The following Section 4 shows the current implementation of the identified features in the PANDORA -B OX by using planning technology. Some conclusions end the paper.

2

PANDORA -B OX: An Adaptable Tool for Crisis Training

The PANDORA -B OX system should act as an interactive training tool for groups of crisis managers. In particular the PANDORA User Requirements Analysis suggested to focus on classes of up to six people to be trained with sessions of about 4 hours supported by a human trainer. The specific scenario we chose for demonstrating the training is the “severe weather conditions”, that is particularly rich and any variant of it may imply problems for all the agencies responsible for the emergency management. The training content quality is ensured by the presence of the Cabinet Office Emergency Planning College (EPC) which exercises a wide range of senior decision makers working in collaboration to resolve emergency situations through leading experts in emergency planning and crisis management. The system will act as a sort of “intelligent

Modeling Users of Crisis Training Environments

81

mediator” between trainer and trainees following a mixed-initiative interaction style. Figure 1 summarizes the main architectural idea pursued within PANDORA. Specifically, a group of trainees, representative of different agencies involved in the resolution of a crisis (e.g., Civil Protection, Local Authorities, Health, Fire Rescue, Police, Transportation and so on) have access to the training system through their training work station. If some of the representative authorities are not present in the class but are considered needed for didactic reasons they can be simulated in the system through Non Player Characters (NPCs), in which case, features and decisions are synthesized by the trainer directly.

Fig. 1. The PANDORA loop around the trainees. Three main modules orchestrate their work to provide adaptable and effective training. An additional module ensures the trainer supervision.

The various participants in the training are characterized by different facets, in relation to both the components closely linked to their role and responsibility, and the particular affective states they may exhibit during the training experience in response to presented stimuli. Each trainee, by interacting with the system, feeds personal data to the PANDORA -B OX, which gathers this information to build a user model (Behavioral Model in the figure). Based on this model, the system synthesizes, using the Behavioral Planner, a personalized training path that meets the specific needs and status of each trainee. The output of this process is passed to a second module (the Crisis Planner), which on the basis of the Behavioral Module decisions, as well as the knowledge of the chosen guiding training scenario, synthesizes a sequence of stimuli appropriate for both the group (information shared among all trainees) and the individual trainees (information tailored to induce the “right level of stress to each trainee”). The plan synthesized by the crisis planner is then given as input to the module called Environment and Emotion Synthesizer which is responsible for an effective rendering of the training temporal plan. In practice such a module adds an additional level of “realism” to the stimuli,

82

G. Cortellessa et al.

by customizing the appropriate presentation mode (e. g., introducing noise on a phone call report) in order to achieve a high level of realism, stress and pressure. The use of advanced 3D scenario reproduction is also studied in the project. A separate module (Trainer Support Framework), allows the trainer to keep control of the training session and dynamically adjust the stimuli based on his/her experience as a trainer. Overall the PANDORA -B OX supports the loop trainer→training environment→trainee, encouraging the customization and adaptation based on the users feedback as well as on goals and suggestions from the trainer.

3 Collecting Relevant Data to Model Crisis Decision Makers The selection of the factors relevant to build the trainee model has been inspired by different sources among which the analysis of the state-of-the-art, frequent interviews to our expert trainer as well as participation in a simulation of training exercise as both trainees and observers. Specifically the analysis of the state-of-the-art has been particularly useful to identify the main “affective factors” that play a crucial role in the decision making under crisis, while the last two sources have contributed to ground the choice on a solid and concrete basis. The results of this multi-dimensional analysis is shown in Table 1. The variables taken into account are personality traits, leadership style, background experience, self-efficacy, stress and anxiety. In particular these variables are subdivided between (a) static features that do not change during training episodes, being mainly related to individual personality and (b) dynamic, which can be on the contrary related to both the context and the time, so they may vary during the training. Both dynamic and static variables are used to create the initial trainee profile, while the dynamic ones are also used to update the model. Similarly, some of the variables are used to derive rules to personalize training, and some of them are used for experimental purposes as we will explain later in the paper. In the next subsections, we will focus on some of the main variables (self-efficacy, stress and anxiety) as guiding examples and will give a more detailed justification for their inclusion in the model. These examples will be also used to explain the modeling and personalization aspect. 3.1 Psychological Data Measurement: Self-Efficacy and Anxiety Individuals belief in their capabilities to perform a certain task successfully has been defined by Bandura [8] as the Self-Efficacy. This belief is likely to differ depending on the activity to which it is related. Accordingly, evidences suggest that self-efficacy is not stable over time, but may vary throughout a training or a learning episode [10]. For example, in the educational context it has been found that self-efficacy may predict which problems and sub-problems a student will select to solve, how long s/he will persist and how much effort will expend. As regards leaders, their level of self-efficacy may be a relevant factor in predicting their performance under crisis situations [11]. Information assessment and decision making are two core behaviors that are critical for effective crisis leadership [2]. Nevertheless, the ambiguity, urgency and high risk of a crisis situation posit some constraints on the abilities of leaders. For example, given the severe time pressure on their behavior,

Modeling Users of Crisis Training Environments

83

Table 1. List of variables used to build the trainee model STATIC

Variable Description They do not change during the training episode

Personality Traits

According to [4], Personality refers to an “individuals characteristic patterns of thought, emotion, and behavior, together with the psychological mechanisms, hidden or not, behind those patterns.” To assess this variable we will use the Revised NEO Personality Inventory, or NEO-FFI (Five Factors Inventory), a psychological personality inventory measuring five main personality factors: neuroticism, extraversion, openness to experience, agreeableness and conscientiousness [5]. NEO-FF has been widely used to correlate behavior and internal personality traits and each of the “big five” has been associated to specific behaviors and/or predicted significantly various job performances, as well as the ability to cope with stress. Within PANDORA we are setting an experimental apparatus in order to investigate the relationship between the personality traits and the performance of crisis decision makers.

Leadership Style

In [6] leadership is defined as a “process whereby an individual influences a group of individuals to achieve a common goal”. In [7] a distinction is made between socio-emotional and task oriented. The “socio-emotional leader” takes into account feelings and moods of individuals, pays attention to the emotional aspects of interpersonal relationships. A leader focused on the task has as his constant concern the attainment by the group of its purposes. In order for a group to be “successful”, both a socio-emotional leadership and a “task leadership centered style” is necessary. The leadership style can obviously not been changed with a short term training, but the training can be personalized in order to let trainees understand the consequences of their leadership style and to encourage them to learn accordingly.

Background experience DYNAMIC

Another relevant variable for a crisis leader is obviously his/her past experience in managing crisis situation. We refer to this as background experience. A very short questionnaire will assess leaders socio-demographic information, their previous experiences with leading public health and safety crises and their level of success in doing it. They might change during the training session

Self-efficacy

Individuals belief in their capabilities to perform a certain task successfully has been defined by [8] as Self-Efficacy. It has been shown that this variable has influence of different aspects like managing stressful situation, increase the performance as well as receive benefits from training programs.

Stress and Anxiety

In [9] stress is defined as “a process by which certain work demands evoke an appraisal process in which perceived demands exceed resources and result in undesirable physiological, emotional, cognitive and social changes”. This definition is considered particularly relevant, since in an emergency situation a key factor is that demand often exceeds resources, both in the management of an emergency and in response options. As for the anxiety we distinguish between State Anxiety, which reflects a “transitory emotional state or condition of the human organism that is characterized by subjective, consciously perceived feelings of tension and apprehension, and heightened autonomic nervous system activity” and Trait anxiety that denotes “relatively stable individual differences in anxiety proneness and refers to a general tendency to respond with anxiety to perceived threats in the environment”. These two variables will be assessed both before and during the training with psychological and physiological measures and well be at the basis of the experiment to study the reaction of different people under stress

they have less time to acquire and process information effectively. As a consequence, they are required to assess information and making decisions under tremendous psychological and physical demands [12]. However, if leaders believes that they can successfully exert leadership by setting direction for work group, gaining the commitment of the group to change goals and working to overcome obstacles, they will more likely solve a crisis situation [13]. On the contrary, if leaders have doubts about themselves regarding their ability to respond aggressively to crisis situations, they will be burdened by the stress and anxiety associated with the ambiguity, urgency and high risk of a crisis [11]. Previous studies [8], indicate that individuals who believe they are not capable of handling difficult situations, more likely experience stress and anxiety, as compared to those who believe they are able to do it. Accordingly, we expect that the lower the level of self-efficacy, the higher the stress and anxiety experienced by leaders. In addition,

84

G. Cortellessa et al.

self-efficacy has the advantage of being open to influence (e.g. training) rather than a trait-like quality that will remain fixed [14]. As a consequence, self-efficacy is especially useful not only for assessing the quality of leadership in crisis, but also for improving it. In addition, if it is true that there is a negative relationship between self-efficacy and anxiety, by improving leaders’ self-efficacy the training is expected to reduce also their level of anxiety. Evidences suggest that under stress people tend to rely on their dominant responses in similar circumstances [15]. On this basis it is expected that leaders that have found themselves to lead in one or more crisis situations, including practicing crisis response protocols (e.g. developing plans) and being efficacious as a leader in general, will show more efficacy to perform leadership task in a crisis. All these considerations suggests that a valid goal for the PANDORA -B OX is to personalize training so as to increase and promote trainees’ level of self-efficacy. Scale to measure self-efficacy and anxiety. The Crisis Leader Efficacy in Assessing and Deciding (C-Lead) Scale [1] is used to measure self-efficacy within the PANDORA B OX. The Scale consists of 9 items; leaders will be classified according to 2 levels of self-efficacy (low: -1 SD) and high: + 1 SD), based on their responses on a scale ranging from 1 (Strongly Disagree) to 7 (Strongly Agree). Anxiety is assessed using the Spielberg’s state-trait anxiety inventory (STAI), which consists of 40 items: 20 designed to assess state anxiety (S-anxiety scale) and the other 20 aimed at evaluating trait anxiety (T-anxiety scale). The first one assesses how respondents feel “right now, at this moment”. The second one evaluates how people “generally feel”. Leaders anxiety will be compared with a reference sample before the start of the training in order to establish a baseline level. Subsequently this measure will serve as their own control for the rest of the training. Anxiety level will be assessed before and after their training in order to verify the effectiveness of the training and any variations in the relationship between self-efficacy level and anxiety. 3.2 Physiological Data Acquisition: Stress and Anxiety In order to enrich the trainee model within PANDORA as well as to adopt an orthogonal approach for the assessment of trainee variables, we also use neurophysiological measures. Specifically, stress and anxiety will be also assessed through these measurements. An Holter monitor is part of the PANDORA system and will be responsible for recording parasympathetic activity (i.e., Heart Rate, HR and Heart Rate Variability, HRV). Autonomic nervous system fires neurons through sympathetic and parasympathetic nervous system. The sympathetic nervous system (SNS) activates the cardioaccelerating center of the heart, increases the ventilatory rhythm and contracts arteries of non-striated muscles. Parasympathetic nervous system (PNS) innervates the cardiomoderator center of the heart, slows ventilatory rhythm and dilates the arteries of non-striated muscles. These systems function in opposition to dynamically modulate vital functions. Heart Rate Variability is a measure of the continuous antagonism between sympathetic and parasympathetic effects on HR yielding also information about the capability to regulate emotional responding. Sympathetic signals increase HR and parasympathetic activity lowers HR, causing shorter and longer interbeat intervals respectively. SNS activity, mediated by norepinephrine, operates slowly on HR through the stellar ganglion while PNS, mediated by acetylcholine, primarily controls in a faster way HR through the

Modeling Users of Crisis Training Environments

85

vagus nerve. HRV analysis results from the monitoring of a set of temporally ordered interbeat intervals, typically by electrocardiography (ECG) recording and commercially available software. The autonomic nervous system allows the body to maintain its balance when experiencing stress. Stressful circumstances are always accompanied by activation of the parasympathetic nervous system critical in regulating of emotions. Recent studies show in depression and anxiety disorders abnormal HRV compared with non-psychiatric controls indicating in such patients decreased vagal tone and elevated sympathetic activity and reflecting deficit in flexibility of emotional physiological mechanisms. Furthermore psychological states have a large impact on autonomic control of the heart causing standard psychological stressors reciprocal sympathetic activation and parasympathetic withdrawal. However response to psychological stress are usually large in the general population indicating individual response uniqueness. In this frame, assessments of users psychological characteristics and measurements of physiological parameters during the training sessions and crisis situations might allow to cross-correlate personality traits and SNS/PNS responses to cognitive and behavioral responses. It is worth highlighting how the physiological measure will be used both during the training to dynamically update the trainee model and after the training to correlate personality traits to physiological response, as well as to compare this response with the trainees’ self assessment.

4 A Timeline-Based Representation of Trainee Model In PANDORA we use a timeline-based approach for both the behavioral and crisis planners [16]. In general terms, a timeline can be seen as a stepwise constant function of time or more specifically as an ordered sequence of values holding on subsequent temporal intervals. Different kinds of values can be used with timelines but, for the current purposes, we will consider only predicate values. A predicate is a statement P (x, y, . . .) that depends on one or more arguments x, y, . . . each having its own domain. Each value is considered valid over a time interval identified by a start and an end timepoints. A set of timeline is used to describe the temporal evolution of the represented world, usually called the domain. To describe consistent temporal behaviors of a domain, the timeline-based approach makes use of the concept of compatibility (called causal pattern in PANDORA). A compatibility specifies requirements among set of values either on the same timeline or on different timelines. In particular it involves (a) a reference value and one or more target values, that are other values on the same or on other timelines (b) some relations, that are constraints limiting the domain of reference value or target value arguments (e.g., a relation can specify that reference and target values would be separated by a given temporal distance). We call a value v admissible if there exists a compatibility having v as reference value and all temporal requirements are satisfied. A value v can also be made admissible through unification with an already admissible value v  such that v = v  . Finally, a timeline-based domain is called admissible if all values of the domain are admissible and if, for each timeline, no different values overlap in time. Modeling Trainees with Timelines. In the current prototype of the PANDORA -B OX timelines are used to maintain information about trainees features. In particular, various variables have been modeled through different value types:

86

G. Cortellessa et al.

– personality traits are modeled through predicates of the form personalitytrait (x) where x is an integer ranging from 0 to 10; – leadership style through predicates of the form leadershipstyle (x) with x being a boolean variable assuming value 0 for socio-emotional and 1 for task oriented leadership style; – background experience through predicates of the form backgroundexperience (x) where x is an integer assuming values 0 for low experience, 1 for medium experience and 2 for high experience; – self efficacy through predicates of the form self ef f icacy (x) with x being an integer ranging from 0 to 10; – stress and anxiety through predicates respectively of the form stress (x) and anxiety (x) where x is an integer ranging from 0 to 100; Additionally a timeline called profile with predicates of the form prof ile (x) with x being an integer ranging from 0 to 100 model an aggregate classification of the current trainee status and is used to both define different aggregations of trainee variables that may be of interest (i.e., to create different trainee profiles that consider different variables) and consequently to trigger particular behavioral plans suitable for that particular profile. In particular the key connection between Behavioral and Crisis planners is represented by a timeline induced stress that is used by the crisis planner to modulate lesson stimuli toward trainees – see sketch in Fig. 2. Once allowed values are defined, we define causal patters for each value. Static variables do not change over time so compatibilities for their values are of the form Pred(x) { this duration [H, H]; }

where Pred is the corresponding variable predicate (e.g., personality trait, leadership style, etc...), this refers to the refer- Fig. 2. Causal reasoning in the PANDORA -B OX ence value, duration is a constraint on the duration of the reference value, H is the planning horizon. For each allowed value of each dynamic variable, a pattern of a different type is added: Pred(x) { pro profile(y); }

this during pro;

where the label pro identifies the target value profile(y) inside the compatibility and this during pro is a constraint imposing that the reference value must appear during the pro target value. These causal patterns imply that, whenever a new value is added to the corresponding dynamic variable, a profile value is also added to the profiles variable with a free y argument. Based on specific personalization rules, profile value compatibilities allows both the definition the current profile and the synthesis of suggestions for training customization.

Modeling Users of Crisis Training Environments

87

4.1 Toward a Personalization of Training Programs In the current demonstrator we have operationalized the causal connection between user features and the planning of crisis stimuli – Fig. 2. An example of personalization rule is given by the compatibility: profile(x) { se self_efficacy(1); be background_experience(3); st stress(3); this.x = 1; this contains se; this contains be; this equals st; }

We consider two variables from trainees models, namely the self efficacy and background experience. In the example, se and be are labels identifying respectively the target values for self efficacy and background experience while st is the label for the level of induced stress to generate. Additionally, this.x = 1 forces the value of the profile to 1, this contains se and this contains be are constraints imposing that the reference value must contain the se and be target values and this equals st is a constraint imposing that the reference value must appear at the same time of the st target value. Hence in the figure the two variables are used to create four profiles (P1, P2, P3 and P4) by combining all the possible values of the two considered variables (assuming 0, 1 as possible values). Once the profiles are defined, personalization rules specify strategies to personalize training through causal patterns. Specifically the compatibility associates the values of the two behavioral variables (self efficacy and background experience) to a profile and then a profile to an induced stress value. In this example the personalization is inspired by the idea of fostering trainees’ self-efficacy by presenting them crisis events that can be managed successfully (mastery experience), which are neither too difficult nor too easy. The indication on the level of induced stress produced by the Behavioral Planner is given as input to the Crisis Planner that adjusts the current plan (crisis events presented to trainees) according to the given induced stress value. It is worth highlighting how this approach allows to create different trainee profiles by combining different behavioral variables and consequently alternative personalization strategies.

5 Conclusions We have shown how a number of cognitive features have been included in the PANDORA B OX design to support the analysis of the user involvement as well as the level of training personalization. We have also described the complete cycle we have realized for modulating stimuli while dynamically monitoring selected psychological states of trainees. This part of the work is included in our environment and is currently being made more robust. It is worth observing how cognitive reasoning has been made operational in a relevant application field. In PANDORA, psychological features dynamically extracted from both trainees’ self assessment and physiological analysis are used to enrich the training personalization. The robust version of the system will enable future work devoted to a specific evaluation effort aimed at assessing the validity of the proposed approach with end users. Additionally we aim at enlarging the use of personalization strategies to prove the generality of the approach.

88

G. Cortellessa et al.

Acknowledgements. The PANDORA project is supported by EU FP7 (GA.225387) under the joint call ICT/Security and is monitored by REA (Research Executive Agency). Authors are particularly indebted to Keith Strickland from Emergency Planning College (York, UK) for providing his invaluable expertise to the project, and to all the partners for the nice work environment. The PANDORA web site is: http://www.pandoraproject.eu/.

References 1. Adler, C., Pittinsky, T., Sommer, S., Zhu, W.: Measuring the efficacy of leaders to assess information and make decisions in a crisis: The C-LEAD Scale. Technical report, HKS faculty research Working Paper Series (2009) 2. Wooten, L.P., James, E.H.: Linking crisis management and leadership competencies: The role of human resource development. Advances in Developing Human Resources 10(3), 352–379 (2008) 3. Schoemberg, A.: Do crisis plans matter: A new perspective on leading during a crisis. Public Relations Quarterly 22, 640–657 (2005) 4. Funder, D.C.: The personality puzzle, 4th edn. Norton, NewYork (2007) 5. Costa, P.T., McCrae, R.R.: The NEO Personality Inventory (revised) manual. Odessa, FL: Psychological Assessment Resources (1992) 6. Northouse, G.: Leadership theory and practice, 4th edn. Sage publication, Thousand Oaks (2007) 7. Bales, R.F.: Task roles and social roles in problem solving groups. In: Macoby, E.E., Newcomb, T.M. (eds.) Readings in Social Psychology, Holt, Rinehart and Winston, New York (1958) 8. Bandura, A.: Social foundations of thought and actions. A social cognitive theory. Prentice Hall, Englewood Cliffs (1986) 9. Salas, E., Driskell, J., Hughs, S.: The study of stress and human performance. In: Driskell, J., Salas, E. (eds.) Stress and Human Performance. Lawrence Erlbaum Associates, New Jersey (1996) 10. McQuiggan, S., Mott, B., Lester, J.C.: Modeling Self-Efficacy in Intelligent Tutoring Systems: An Inductive Approach. User Modeling and User-Adapted Interaction 18, 81–123 (2008) 11. Pittinsky, T., Hadley, C., Sommer, S.: Measuring the Efficacy of Leaders to Assess Information and Make Decisions in a Crisis: The C-LEAD Scale. Technical report, Center for Public Leadership (2009) 12. Klann, G.: Crisis Leadership. Center for Creative Leadership, Greensboro, NC (2003) 13. Paglis, L., Green, S.: Leadership Self-Efficacy and Managers’ Motivations for Leading Change. Journal of Organizational Behavior 23, 215–235 (2002) 14. Stajkovic, A., Luthans, F.: Self-efficacy and work related performance: A meta-analysis. Psychological Bulletin 124, 240–261 (1998) 15. Dickerson, S.S., Kemeny, M.E.: Acute stressors and cortisol responses: A theoretical integration and synthesis of laboratory research. Psychological Bulletin 130, 355–391 (2004) 16. Muscettola, N.: HSTS: Integrating Planning and Scheduling. In: Zweben, M., Fox, M.S. (eds.) Intelligent Scheduling. Morgan Kaufmann, San Francisco (1994)

Personality Estimation Based on Weblog Text Classification Atsunori Minamikawa and Hiroyuki Yokoyama KDDI R&D Laboratories, Inc., Saitama, Japan {at-minamikawa,yokoyama}@kddilabs.jp

Abstract. This paper describes the investigation of personality estimation from Japanese weblog text. Among various personality types, we focus on Egogram, which has been used in Transactional Analysis and is strongly related to the communicative behavior of individuals. Estimation is performed using the Multinomial Naïve Bayes classifier with some feature words that are selected based on the information gain. The validity of this approach was evaluated with real weblog text of 551 subjects. The results showed that our approach achieved 1225% improvement from baseline. The feature words selected for the estimation are strongly correlated with the characteristics of Egogram. Keywords: Computational Linguistics, Personality Estimation, Egogram.

1 Introduction Recent advancement in Internet technology and mobile devices enable users to record and restore their experience, thought, emotion or other personal information electronically. Vast amount of daily activity record, called Lifelog, are generated and exchanged in various social network services through Internet every day. User profiling using these Lifelog data has attracted attention as the key technology to achieve cognitive computing applications, such as intelligent user interface, personalized recommender system and so forth. Among various kinds of Lifelog, personal weblog is one of the most appropriate data to extract user’s traits since it is widely used. In personal weblogs, many users record their daily event, their thought, and their emotion with text subjectively and release them through Internet. Many researches in computational linguistics explore methods for automatic extraction of various traits from weblogs such as emotion [5, 7], mood [10], deception [16], sentiment or opinion [2]. Although these works provide successful contributions for recognizing the context of the user, it is still difficult to estimate user situation since these traits provides only temporal characteristics of the user. This work focuses on personality as another trait for the same purposes since it provides a longer term aspect of individuals [8]. User personality extracted from text could be useful in many interactive situations, for instance, as a cue to understanding the background of their action to search for the appropriate services which matches one’s personality. This paper describes our investigation of the validity of personality estimation from Japanese personal weblog text. Personal weblog is also appropriate for the extraction of K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 89–97, 2011. © Springer-Verlag Berlin Heidelberg 2011

90

A. Minamikawa and H. Yokoyama

long term characteristics of the author since it is the accumulated corpus of one’s expression for a certain period of time. Many works have investigated the relationship between written texts and personality. Most of these works focus on the personality parameter called Five Factor Model (FFM) or “Big Five” [3] which emerged from lexical research. Argamon et al. [1] especially focused on Extraversion and Neuroticism as the prediction targets of the personality, employing various feature sets, function words, and systemic functional grammar analysis. Mariesse and Walker [8, 9] used features derived by the top down approach using the Linguistic Inquiry and Word Count text analysis program [13]. Oberlander and Nowson [11, 12] investigated the personality prediction accuracy of the five factors by the data-driven approach. Gill et al [6] investigated the relation between blog topic and FFM. Our work focuses on another personality parameter called Egogram which was originally developed by Dusay [4]. Egogram has been developed in the field of Transactional Analysis (TA), which deals with how people communicate with each other from a social and psychological perspective. Compared to FFM which represents the task-related individual behaviors [9], the Egogram model focuses more on the communication aspect of the personality. It provides finer-grained characteristics for the communicative situation; Critical Parent (CP), Nurturing Parent (NP), Adult (A), Free Child (FC), and Adapted Child (AC). Applying Egogram model is much more appropriate for the interactive user interface which needs various kinds of communication, because a lot of researches have been done in TA with the Egogram. For example, which Egogram patterns are a good match, which Egogram pattern is the most appropriate for leaders in cooperative, which ego states cause confliction in certain communication scenarios and not, and so forth. These properties have much advantage over the other personality models. Furthermore, the Egogram has been already widely used especially in Japan for psychological analysis of cooperated work, and for self-analysis in the school teachings and even in the training seminars for new recruits in the company. There are a great store of knowledge and experiment which link the Egogram and cooperatedwork. These previous works gives us a firm basis of the transactional analysis, and our work will provide a new inspiration for the readers who are interested in the interactive communications.

2 Egogram Model Egogram represents the amount of psychological energy which a person distributes at one given time. Each of the five ego states indicates the role which is played in the interaction. In particular, CP and NP represent how one treats others, FC and AC represents how one represents oneself, and A indicates the objectivity in the communication and represents how one controls it. Table 1 shows five ego states and their associated behaviors. The personality of a subject is explained by the relative intensity among these five ego states. For instance, the subject whose CP intensity is much higher than the other ego states tends to be authoritarian and to restrict the freedom of others, and is labeled as “chauvinist” in [4].

Personality Estimation Based on Weblog Text Classification

91

In clinical use, the Egogram of a subject is extracted through counseling by a specialist or an answer sheet examination which has been developed by various organizations. Table 1. Traits of five ego states Ego States CP (Critical Parent) NP (Nurturing Parent) A (Adults) FC (Free Child) AC (Adapted Child)

Features paternal, idealism, responsible, conservative, critical maternal, nurturing, pampering, sympathetic objective, rational, logical, cold creative, active, selfish, abandon adaptable, obedient, cooperative, passive

3 Personality Estimation Method In order to estimate the Egogram from weblog, we use text classification method using machine learning techniques. The classification was performed based-on the appearance of feature words which are derived from statistical analysis of supervised data. Figure 1 illustrates the outline of our method. Details of our method are described in the following subsections. Learning Process

9GDNQI

…

'IQITCO

9GDNQI

(GCVWTG9QTFU 5VCVKUVKECN #PCN[UKU

/QTRJQNQIKECN #PCN[UKU

Estimation Process /QTRJQNQIKECN #PCN[UKU

(GCVWTG 'ZVTCEVKQP

%NCUUKHKECVKQP 0CïXG $C[GU

'IQITCO

Fig. 1. Outline of the personality estimation process. In the learning process, feature words are derived from the statistical analysis of supervised data. Classifiers are also modeled in this process.

3.2 Data Sets We gathered real personal weblog text of 551 Japanese subjects which were carefully selected from the several major Japanese weblog sites to avoid bias on gender and

92

A. Minamikawa and H. Yokoyama

generation (278 males and 273 females). The subjects also answered the personality examination, called TEG2 (Todai-shiki Egogram version 2) [15] which is one of the answer sheet examinations for Egogram. TEG2 consists of 50 items each of which corresponds to each Ego states and is most frequently used in clinical investigation in Japan. From the answers of TEG2 the intensities of the five ego states (CP, NP A, FC and AC) are finally scored using a 5-point scale. Blog Text Feature Words

Morphological Analysis

bag-of-words

Feature Vector

CP

(xCP1, xCP2 ,....,xCPn )

CP Classifier

CP Score

NP

( xNP1, xNP2 ,....,xNPn)

NP Classifier

NP Score

A

( x A1 , xA2 ,....,x An )

A Classifier

A Score

FC

( xFC1 , xFC2 ,....,xFCn )

FC Classifier

FC Score

AC

( xAC1 , xAC2 ,....,x ACn )

AC Classifier

AC Score

Fig. 2. Classification of the ego state score.

3.2 Classification In the estimation, we separately modeled each five ego state, and built a classifier for each state. Each classifier derives the score of each ego state which are transformed into the discrete bins as mentioned in the following section “Evaluation Scenario”. Before classifications, weblog text data are separated into bag-of-words using Sen [14] that is one of the Japanese morphological analysis tools. In next step, feature vector corresponding to each state is derived by counting the appearance of the feature words in the bag-of-words. These feature words were preliminarily selected by the statistical analysis for each ego state. After the feature vectors were derived, each classification was performed using Multinomial Naïve Bayes (MNB) classifier which has been often used for text classification. These MNB classifiers are implemented using the WEKA library [17]. In the evaluation, 10-fold cross validation was used. The original sample was randomly partitioned into 10 subsamples. A single subsample of them was retained as the validation data for testing the model, and the remaining 9 subsamples were used as training data. Then validation process was repeated 10 times with each of the 10 subsamples used once as the validation data. Classification accuracy was derived from the average of the 10 results.

Personality Estimation Based on Weblog Text Classification

93

3.3 Feature Selection Many potential features can be used for text classification. In this work we use wordbased features. The effective feature words are selected based on the information gain which is calculated in the following equations.

InfoGain ( w) = H (C ) − H (C | W )

(1)

H (C ) = −∑ Pr(c) log Pr (c )

(2)

H (C | W ) = − ∑ Pr(wd ) H (C | wd )

(3)

c∈C

wd ∈W

w is the word, W is the class where the , C is the class which corresponds to bins of each ego states, Pr(c) is the probability of the bin c in the supervised data, Pr(wd) is the probability of the word w appearance In other text classification cases, such as gender, age and resident area, nouns are extracted as the effective feature words. However nouns are not appropriate in this work, because 1) the usage of nouns depends on the topic and is not directly related to one’s personality and 2) it is difficult to prepare an adequate corpus which covers the vast amount of nouns. Especially for the second reason, nouns as classification feature words may cause overfitting in personality estimation. Therefore we did not include nouns in the feature words. Instead, we took into account the appearance of adjectives, adverbs, conjunctions, exclamations and some internet slangs and emoticons. In the classifications, we set the upper limits N for the number of feature words, and the words whose information gains are included in the top N are selected as the feature words for classification.

4 Evaluation In this section, we evaluate our method in some estimation scenario. Before the evaluations, we investigate the word class dependency of the egogram estimation. The F-measure of each estimation is calculated as the following Equation . F=

2 PR P+R

(4)

P and R denote the precision and the recall of the estimation, respectively. 4.1 Word Class Dependency In this subsection, we investigate the effect of each word class for the Egogram estimation. Classifier classifies the intensity of each ego state into 2classes, whether the TEG2 score of the subjects is more than the average score or not. Each classification

94

A. Minamikawa and H. Yokoyama

used the feature words which were selected from only one word class. The number of feature words used in each classification is set at 50. As table 2 illustrates, adjective, adverb and emoticon are effective for each ego state estimation. Especially for NP, A, and AC estimation, the improvements from baseline are relatively large. Table 2. Improvements of F-measures of the estimation using each word class compared to the baseline (%). The F-measures of the baseline were the percentage of the majority class. CP

NP

A

FC

AC

Adjective

1.8

9.7

7.3

2.3

4.6

Adverb

2.8

11.1

6.9

4.3

8.4

Conjunction

0.4

-0.6

4.2

-0.2

-1.0

Interjection

-1.8

1.9

2.0

1.1

1.4

Emoticon

5.2

11.1

10.7

1.1

7.2

Symbols

0.0

0.9

3.8

0.0

1.4

Pronoun

0.6

3.3

2.8

0.3

2.6

4.2 Estimation Evaluation In this subsection, we evaluate the classification accuracy according to the following three evaluation scenarios: • (E1) Classify into 2 classes, whether the TEG2 score of each ego state of the subject is more than the average score or not. • (E2) Classify into 3 classes, whether the TEG2 score is high, middle or low. Subjects whose score is 5 or 4 are classified as high; subjects whose score is 3 are classified as middle; the remaining are classified as low. • (E3) Classify into 5 classes, based on the original TEG2 score. Classifiers have been built for each evaluation scenario. Table 3 illustrates examples of the selected feature words translated into English for each ego state. It should be noted that emoticons are effective features for the classifications of four ego states. Especially the emoticon “orz” is effective for CP, FC and AC classifications. This emoticon is a familiar Japanese net slang which represents a man pounding his head on the floor. The o is the head, the r is the arms and the z is the legs. It is used to symbolize the emotion of frustration. Tables 4 shows the results of each ego state classification for three evaluation scenarios. The most accurate performance of the number of feature words is highlighted in bold. The binary classifications of scenario (E1) achieve almost 70-85% accuracy when the number of feature words is over 150. Although FC classification shows the highest accuracy among five, it is due to the high baseline score. The improvement for NP from baseline is the highest performance among the five ego states. For AC, the classification using 50 feature words achieve the highest performance. This indicates the possibility of overfitting when the number of feature words is over 100.

Personality Estimation Based on Weblog Text Classification

95

In multiple classification of scenarios (E2) and (E3), the classification accuracy decreases as the classification grain becomes finer. MNB classifier achieves 12-20% improvement for each ego state. Table 3. Exapmles of feature words selected in the experiment (translated into English) Positive

CP

NP

A

FC

Negative

immediately

orz (disappointed)

supposedly

Oh

actually

joyfull

great

handily

kind of

orz* (disappointed)

smart

point by point

probably

hey

on the contrary

・

philosophic

!

joyful

orz (disappointed)

ω・ (emoticon)

familiar ≧▽≦

primitive (emoticon)

orz (emoticon) AC

・∀・

warm respectable

(emoticon)

Oh

while either

Table 4. F-measures of the estimation in each evaluation scenario. Baseline is the percentage of the majority class

(E1)

Num of Features

CP

NP

A

FC

AC

Baseline

63.8 69.8 74.4

56.9 72.8 77.9

53.3 63.8 66.4

75.0 81.9 83.5

63.8 75.8 75.3

150

77.1

79.5

67.8

85.5

75.0

200

75.4

81.1

69.6

85.7

74.8

MNB

50 100

Baseline (E2)

MNB

39.8

43.1

46.7

47.7

42.2

50 100 150

55.3 59.6 59.8

59.1 64.4 67.0

61.6 64.4 68.4

59.4 62.2 63.4

200

64.0

68.0

58.1 62.0 64.4 64.2

69.8

65.4

39.8

36.2

41.2

47.7

42.2

47.5 51.9 54.3 55.3

48.9 54.1 53.5 55.9

50.7 52.5 52.5 54.5

55.3 57.3 61.4 62.0

51.1 56.3 58.1 58.9

Baseline (E3)

MNB

50 100 150 200

96

A. Minamikawa and H. Yokoyama

5 Discussion In the previous section, we evaluate the personality estimation method which considered the usage of adjectives, adverbs, conjunctions, exclamations and Internet slang in blog text. In the evaluation, subjects whose CP or A scores are higher, have tendency to use more formal expressions, while subjects whose AC scores are relatively higher, use more informal or colloquial expressions, such as net slang and emoticons. Similarly, subjects who tend to use exclamations have high AC scores, while their CP and A scores are low. These results indicate that the appearances of emotional expressions in blog text are strongly related to CP, A, and AC characteristics of the author. It was also seen in the experiment that conjunctions are relatively used by subjects whose CP and A scores are high. On the other hand, high NP subjects tend to use admirative words to others. These findings conform to the characteristics of each ego state illustrated in Table 1. In all evaluation scenarios, our approach achieved higher performance than the baseline. This means that Egogram estimation from weblog text using computational linguistic analysis is a convincing method. Here we discuss the effectiveness and limitation of our method from the aspect of its application. The advantage of our method is that it can preliminarily extract the personality of the user which affects the communicative behavior. Since the personality is the latent characteristics, it is difficult for instantaneous sensing system to detect it with short term observation. By detecting user’s personality, our method enables the system to predict the user reaction in each interactive session and to control its interface behavior as it matches user preference. It is also the advantage of our method that Egogram has the high compatibility with the interactive user interface since it models various communicative situations between people. On the other hand, the limitation of our method is that it cannot detect the user cognitive change during interaction. Therefore it should be combined with the other cognitive system, such as Fujita et al. [18].

6 Conclusion This paper describes the personality estimation method from weblog text classification techniques, in order to achieve interactive user interface in the cognitive computing. The Egogram model is appropriate for such purpose since it deals with the communication aspects of the users. The estimation was performed by Multinomial Naïve Bayes classifier with feature words which were selected based on the information gains. The method was evaluated using real weblog text of 551 subjects. The results show that our approach achieved 12-25% improvements from the baseline. Especially in the classification for the aspect of Nurturing Parent (NP), our approach performed the highest results in five ego states. The feature words selected for classifications are convincing from the aspect of the characteristics of each ego state. Therefore, our estimation method is successful at this stage. The Egogram estimation method could be useful for the interactive user interface of the cognitive applications. For example, when starting the interactive session,

Personality Estimation Based on Weblog Text Classification

97

system can preliminary grasp the characteristics of its users and can predict their reaction in each interactive stage. In future works, other features such as the semantics of the word will be investigated in order to improve the estimation accuracy. It is also important to apply our method to larger corpus for the validation.

References 1. Argamon, S., Dhawle, S., Koppel, M., Pennebaker, J.W.: Lexical Predictors of Personality Type. In: Proc. the 2005 Joint Annual Meeting of the Interface and the Classification Society of North America (2005) 2. Breck, E., Choi, Y., Cardie, C.: Identifying Expressions of Opinion in Context. In: Proc. IJACAI 2007 (2007) 3. Costa, P.T., McCrae, R.R., Revised, N.E.O.: Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI): Professional Manual. Psychological Assessment Resources, Odessa 4. Dusey, J.M.: Egograms and the “Constancy Hypothesis”. Transactional Analysis Journal 2, 37–42 5. Gill, A.J., Gergle, D., French, R.M., Oberlander, J.: Emotion Rating from Short Blog Texts. In: Proc. CHI 2008, pp. 1121–1124 (2008) 6. Gill, A.J., Nowson, S., Oberlander, J.: What Are They Blogging About? Personality, Topic and Motivation in Blogs. In: Proc. ICWSM 2009 (2009) 7. Hancock, J.T., Landrigan, C., Silver, C.: Expressing Emotion in Text-Based Communication. In: Proc. CHI 2007, pp. 929–932 (2007) 8. Mairesse, F., Walker, M.: Using Linguistic Cues for the Automatic Recognition of Personality in Conversation and Text. Journal of Artificial Intelligence Research 30, 457–500 (2007) 9. Mairesse, F., Walker, M.: Words Mark the Nerds: Computational Models of Personality Recognition through Language. In: Proc. CogSci 2006, pp. 543–548 (2006) 10. Mishne, G.: Experiments with Mood Classification in Blog Posts. In: Style Workshop, SIGIR 2005 (2005) 11. Nowson, S., Oberlander, J.: Identifying More Blogger: Towards Large Scale Personality Classification of Personal Weblogs. In: Proc. ICWSM 2007 (2007) 12. Oberlander, J., Nowson, S.: Whose Thumb is it Anyway? Classifying Author Personality from Weblog Text. In: Proc. ACL 2006, pp. 627–634 (2006) 13. Pennebaker, J., Francis, M.: Linguistic Inquiry and Word Count 2001. Lawrence Erlbaum Associates, Mahwah (2001) 14. Sen (in Japanese), http://ultimania.org/sen/ 15. TEG Research Group in the University Tokyo Medical school, TEG: Tokyo University Egogram (New Ver.), Kaneko Shobou (2000) 16. Toma, C.J., Hancock, J.T.: Reading between the Lines: Linguistic Cues to Deception in Online Dating Profiles. In: Proc. CSCW 2008 (2008) 17. Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005) 18. Fujita, H., Hakura, J., Kurematu, M.: Intelligent Human Interface Based on Mental Cloning-Based Software. Journal of Knowledge-Based Systems 22, 216–234 (2009)

Design of an Optimal Automation System: Finding a Balance between a Human’s Task Engagement and Exhaustion Michel Klein and Rianne van Lambalgen Vrije Universiteit Amsterdam, De Boelelaan 1081a, 1081HV Amsterdam {michel.klein,rm.van.lambalgen}@few.vu.nl

Abstract. In demanding tasks, human performance can seriously degrade as a consequence of increased workload and limited resources. In such tasks it is very important to maintain an optimal performance quality, therefore automation assistance is required. On the other hand, automation can also impose additional demands on a person monitoring the system and it will likely reduce the person’s awareness of the situation. Therefore, automation should be adapted to the human’s state in order to keep the human in the loop. In this paper an agent model is proposed that calculates a person’s workload and engagement to an automated task. Based on this agent model, an intelligent support system can provide different types of automation and aims at adapting to the human state such that an optimal balance is found between exhaustion and task engagement. Keywords: Agents, Cognitive Modeling, Adaptive Automation.

1 Introduction Human task performance can degrade over time when demanding tasks are performed. Such degradation can for instance be caused by available resources being exceeded [1]. However, high effectiveness and efficiency levels are of particular importance for critical tasks. In such cases, automated assistance to support humans in effective and efficient task execution is often required. However, while humans take advantage of such systems in that it reduces their workload, these systems can also have negative effects on human functioning. Automation induced complacency causes a person to rely on an automated system, even when that system is not always flawless [2]. In addition, automation reduces a person’s picture of the environment, often referred to as situation awareness [3]. A reduction in situation awareness may lead to human error, especially when automation does fail. Parasuraman defines levels and stages of automation that all have a different effect on a person’s situation awareness and human functioning [4]. To overcome disadvantages of automation, it is important to adapt automation to the state of the human [5]. Existing systems often adapt their level of automation throughout a specific task either to the person’s workload [6] or situation awareness [7]. Also, automation can be adapted by analysis of the human’s state through psychophysiology ([8], [9], [10]). For an extensive review on adaptive automation, see [11]. The system K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 98–108, 2011. © Springer-Verlag Berlin Heidelberg 2011

Design of an Optimal Automation System

99

that is proposed in this paper also aims at providing adaptive automation, but as opposed to previous systems it integrates both workload and situation awareness (referred to as task engagement) in the model of a human’s state. This way, advantages of automation (in reducing workload) as well as disadvantages of automation (in reducing awareness) are taken into account. The design of the system is based on the agentparadigm: the human and its behavior and states are explicitly described in a formal way, as a separate entity, as is the interaction between the human and its environment. This agent model is then used to decide on the effect of specific actions on the human. Within the agent model theories of automation ([4], [5], [6]) and operator functional state ([12], [13]) are used to calculate the effects of a specific level of automation on both a person’s engagement to the task and the human’s functional state (i.e. exhaustion and performance). This allows the support system to choose the optimal support level at each point in time and to find the balance between a reduction in exhaustion on the one hand and maintenance of a person’s engagement on the other hand. In this paper, first a theoretical background on human automation and human workload is given in Section 2. Section 3 describes the agent model and in Section 4 simulation results are presented for different types of automation. Section 5 explains how support can be given using the agent model, which is illustrated by an example. Finally, a discussion of the work is given in Section 6.

2 Theoretical Background Parasuraman ([4]) defines 4 possible types of automation that are applied to 4 stages of information processing: information acquisition, information analysis, decision and action selection and action implementation. Within those 4 types of automation, different levels can be distinguished, ranging from no automation to very high automation. Depending on the specific type and level of automation, each system has different consequences to a person’s workload, situation awareness and performance. Decision automation can decrease a human’s awareness, which impairs human performance in case the automation fails. Therefore, decision automation should be set at a low level, especially in demanding circumstances such as air traffic control. On the other hand, both automation of information acquisition and information analysis allow the human to stay involved with the task and thus are beneficial to a human performing a task. Other effects of the level of automation are that on a person’s workload. A high level of automation reduces the workload of the person performing the task. However, a high level of automation also induces monitoring demands, which in turn increases workload. Workload is influenced by the demands of the task, which determine the effort a person has to give to the task for an optimal performance. A high workload may lead to exhaustion, caused by a depletion of resources when the person generates effort to the task (see [13]). Such a mechanism is also common in sports and exercise, where each person has a critical point that determines a person’s basic level at which effort can be contributed without getting exhausted [14]. In [12] a model is presented that explores these relationships between demands, effort, exhaustion and critical point (also referred to as a human’s functional state). In addition to the type and level of automation, awareness of the operator can also be influenced by the level of self-agency. The concept of self-agency is found in

100

M. Klein and R. van Lambalgen

literature on consciousness and can be defined as ‘the feeling that one causes one’s own actions and their outcomes’ ([15], [16]). Normally, self-agency is present when a task is consciously performed. However, research points out that self-agency can also be evoked by priming of the outcome of an action just before the outcome itself emerges ([17]). This is especially useful when humans are supported by high level information. In this case, such priming leads to an increased awareness of the human to the task even though it is performed by the system.

3 Agent Model for Human’s Task Engagement and Exhaustion To design a system that can intelligently decide on the optimal form of automation, it is necessary to create a dynamic model that describes the interaction between the system, the environment, and the human agent. The overall agent model is depicted in Figure 1 and is composed of three main components: the system, the world and the human (or operator). Within the system component, the possible actions of an intelligent support system in order to provide support are represented. The world contains information on the environment and output of both the support system and the operator. Furthermore, in the operator component concepts and relations on operator states are defined. In this section, first the three components explained in more detail. Secondly, a computational specification of the model is provided, in which formulas are given for the calculation of the values for different concepts.

Fig. 1. Agent Model

3.1 Model Components System. The actions that the support system performs can be divided in two different sorts of actions: actions for automation and actions for agency. Actions for

Design of an Optimal Automation System

101

automation define the different levels and stages of automation the support system can adapt to provide support, based on [3], [4]. All actions for automation affect one of the 4 stages of information processing: sensing, perception, decision making or action. Furthermore, within each stage, the support system can perform a low automation action or a high automation action. Each automation action affects the strength of self-action, the strength of automation, the active task demands and the monitoring demands in the world. In addition, the agency action can be performed by the support system in order to increase task engagement. This function is based on literature on consciousness which states that support can be given to increase a person’s awareness of performing a specific task ([16], [17]). If task engagement is low given the automation actions of the system, agency action can be performed by the support system to overcome this decrease in task engagement. World. The world contains the characteristics of the environment as well as the output of the system and the operator. On the side of the operator, eye movements and task performance can be reflected in the world. On the side of the system, firstly active task demands and monitoring demands are distinguished, where the former represents the demands of performing the task that is left after automation and the latter represents the demands of monitoring the automated task. Both forms of demand influence the total effort generated by the operator at a specific point in time. Secondly, the strength of automation is determined by the automation action of the system. Finally, the system’s automation action also influences the strength of self action, which indicates the extent to which operators feel themselves to be responsible for performing the task. Both strength of automation and strength of self action influence the person’s task engagement. Operator. In the operator component, knowledge of concepts and relations is represented used by the support system to determine task engagement and exhaustion. The world component serves as input for the task engagement, the self agency and the total task effort generated by the operator. A simplified model of operator functional state based on ([12]) is used to determine the level of exhaustion with the total generated effort and a person’s critical point (a person’s basic cognitive abilities). Furthermore, the total task effort is split up into a monitoring effort level and active task effort level. Task engagement is determined by the strength of automation, strength of self-action and the monitoring effort. The output that can be measured based on the states of the operator are eye movements and task performance. 3.2 Formal Computational Model To determine the interaction of the human, the system and the world, a formal model had been developed that specifies the effect of specific types of support on the exhaustion and engagement of a human. In this formalization, all concepts have a value between 0 and 1. In order to distinguish between the different types of support, it is assumed that for each variant of automation it is defined what the effect is on: • • • •

the strength of automation (SoA) strength of self action (SSA) active task demands (AT_D) monitoring demands (M_D).

102

M. Klein and R. van Lambalgen

An example of such a specification is given in Table 1 in the next section, where a simulation based on the model is presented. The total strength of automation (SoA) and the total strength of self action (SSA) are determined by averaging the effect of all automation actions; to determine the total active task demands (AT_D) and the total monitoring task demands (M_D) the demands that result of the individual automation actions are summed up. Total task demands (TT_D) represent the sum of the active task demands and the monitoring demands. TT_D = AT_D + M_D (1) The demands of the task and the exhaustion of the operator together determine how much effort the operator generates in total. The parameter ex_speed determines the influence on exhaustion. TT_E= TT_D * (1-(Ex(t-1)*ex_speed))

(2)

The total effort is distributed between the active task and the monitoring task according to the priority that is given to either active task effort (AT_E) or monitoring effort (M_E) (note that prior_m should be 1-prior_at and the other way around). (3)

AT_E = 0 AT_D-((TT_D-TT_E)-M_D) AT_D-((TT_D-TT_E)*(1-prior_at))

if TT_E = 0 if (TT_D-TT_E)*(1-prior_m)>MD else

M_E = 0 M_D-((TT_D-TT_E)-ATD) M_D-(diff*(1-prior_m))

(4) if TTE = 0 if ((TT_D-TT_E)*(1-prior_at)>AT_D else

The engagement to the task (TE) depends on the Strength of Automation, the Self Agency and the effort that is given to the task. TE =

(5)

w1*(1-SoA)*SSA + w2*1 + w3*AT_E/AT_D w1*(1-SoA)*SSA + w2*M_E/M_D + w3*1 w1*(1-SoA)*SSA + w2*1 + w3*1 w1*(1-SoA)*SSA + w2*M_E/M_D + w3*AT_E/AT_D

if M_D=0 if AT_D=0 if AT_D=0 and M_D=0 else

The formula for exhaustion (Ex) is temporal as it takes into account the person’s exhaustion at the previous point in time. Ex_parameter determines the influence of effort (related to the critical point) on exhaustion. Ex(t+Δt) = Ex(t) + (Ex)* Δt *(TTE-CP)/CP * Ex_parameter Ex(t) + (1-Ex)* Δt *(TTE-CP)/TTE * Ex_parameter

(6) if TTECP

4 Simulations To illustrate the working of the model, a number of simulations have been performed in which the effect of different variants of automation on a person performing the

Design of an Optimal Automation System

103

same fictitious task is compared. In the simulations, the effect of a specific type of automation is investigated on two variables, namely exhaustion and engagement, assuming that the system would give continuous support for a long period of time (in this case 100 time steps, where Δt=0.5). In order to execute the model, it is required to specify the effect of different types of automation on the task demands and the strength of self-action and strength of automation. These values are task specific. In the experiments, four different types of support are considered, i.e. support on sensing, support on perception, support on decision making and finally support on action. The latter two are highly dependent on each other, as action support always requires decision support. Each of the automation actions can be off, medium active, or high active. In Table 1, it is specified what the effect of each support action is on the factors described before. Here, the assumption is that the effect of automation on Active Task Demands and Monitoring Demands is independent of the specific type. On the other hand, the effect of the automation action on strength of automation is strongly dependent on the specific type of automation; sensing and perception automation is relatively more ‘lightweight’ as compared to the decision making automation (e.g. [4]). Table 1. Values that determine effects of automation Automation Action Off Sensing

Perception

Decision Making

Action

Strength of Automation 0

Strength of Self Action 1

Active Task Demands 1

Monitoring Demands 0

Medium

0.1

0.5

0.5

0.5

High

0.4

0

0

1

Off

0

1

1

0

Medium

0.4

0.8

0.5

0.5

High

0.7

0

0

1

Off

0

1

1

0

Medium

0.8

1

0.5

0.5

High

1.0

0

0

1

Off

0

1

1

0

Medium

1.0

0

0.5

0.5

High

1.0

0

0

1

The parameter values in this simulation were as follows: ex_speed=0.2; ex_parameter=0.5; prior_at=0.8; prior_m=0.2. w1=0.7; w2=0.1; w3=0.2. The operator’s critical point was set to 4.4. The pattern of engagement and exhaustion over time are shown in Figure 2 a and b, for the case that there is no automation and the case of medium sense and medium perception automation respectively. In both figures it can be seen that the exhaustion and engagement reaches a stable situation after some time, however, with different value for the variables of interest. Since each specific type of automation leads to some stable values for exhaustion and engangement, it is also possible to compare these end-situations directly. This shows

104

M. Klein and R. van Lambalgen

the eventual effect of a specific type of support directly. Therefore, 10 different combinations of support were compared by simulation, in a somewhat increasing level of complexity; the different combinations are listed in Table 2. Figure 4 shows the person’s engagement and exhaustion for the different combinations of types and levels of automation listed in Table 2.

Fig. 2a, b. Exhaustion and Engagement over time in a system without automation (left) and a system with high sense and high perception automation (right) Table 2. Simulated combinations of automation Combination 1 2 3 4 5 6 7 8 9 10

Automation No Automation Only medium Sensing automation Only high Sensing automation Only medium Perception automation Only high Perception automation Only medium Decision automation High decision automation and high action automation Medium Sensing and Medium Perception automation Medium Sensing, Medium Perception and Medium Decision automation High sensing, high perception, high decision and high action automation

Fig. 3. Engagement and Exhaustion for 10 different combinations of automation

Design of an Optimal Automation System

105

As can be seen in Figure 3, exhaustion is highest in the system without any automation and lowest in the fully automated system. In addition, engagement is also relatively low for combination 7, which has a high decision automation and high action automation.

5 Support Framework This section presents how the model introduced in Section 3 can be used within a intelligent support system to reason about the most appropriate type of support for a human, taking both the engagement and the exhaustion into account. To do so, the support system uses the information presented in Table 1, to obtain the knowledge about the effect of different types of support on the task demands, the strength of automation and the self agency. In addition, the system makes use of a preference value for the optimal balance between exhaustion and engagement. This preference value is task specific, as more critical tasks might require relatively higher engagement than enduring tasks, for which a lower exhaustion is important. Based on this knowledge and the model, the support system can calculate the reduction in exhaustion and engagement for the upcoming period for all different types of support, compared with the scenario in which no support is given, i.e. Exdiff_x(t) = Exno_support(t) – Exsupport_x(t)

(7)

TEdiff_x(t) = TEno_support(t) – TEsupport_x(t) for all different types of support_x, where TEsupport_x is the task engagement when a specific variant of support is given (similar for Exsupport_x). To decide on the optimal balance between exhaustion and engagement, both values can be combined into a single value using a formula with a weighting mechanism, e.g. Fx(t) = w1 * TEdiff_x(t) / w2 * Exdiff_x(t)

(8)

In this function, a high task engagement and a low exhaustion both contribute positively to the combined value (w1 and w2 are weighting factors). Alternative formalizations are also possible, e.g. based on a weighted sum of the TE and (1 – Ex). The system will choose the most optimal type of support (i.e. with the highest value for the combination function) until the time point that it is predicted that a different type of support will result in a better balance. At that time point, a new simulation is started, in which it is predicted what the additional effect of all variants of support is, compared with the type of support given until then, i.e. Exdiff_y(t) = Exsupport_x(t) – Exsupport_y(t)

(9)

TEdiff_y(t) = TEsupport_x(t) – TEsupport_y(t) for all types of support_y, and support_x the current type of support. Example scenario To illustrate the approach, simulations are performed using the agent-model described in Section 3 and the reasoning in the support framework as described above. For the

106

M. Klein and R. van Lambalgen

example scenario, a task is considered for which the effects are described in Table 1. In addition, the engagement and exhaustion are assumed to be equally important: formula 8 is used with weights w1 = w2 = 1. All other variables in the simulations have the same value as in Section 4. First, the system compares the effect of a specific variant of support with giving no support. Figure 4 shows the value of the combination function for all different variants of support listed in Table 2. It can be seen that up to time point 22, the line with small dots (support variant 3, high sensing automation) is the most effective type of support. Figure 5 shows the same simulation up to time point 22, but from time point 23 on the support system compares all variants of support with support variant 3. It can be seen that until the end of the simulation the line with the thick dots is the highest. This represents variant 4, low perception automation. Consequently, the intelligent support system would generate high sensing support up to time point 23, and then would switch to low perception support. Note that in the example no “agency actions” (i.e. actions that would increase the self-agency) are considered. Using those actions, it might be possible to influence the engagement directly, without implementing a lower level of automation (which would increase the task demands).

Fig. 4. Simulated balance between reduction of exhaustion and engagement compared with no support (line with thin dots is support variant 3)

Fig. 5. Simulated balance between reduction of exhaustion and engagement compared with support variant 3 from timepoint 23

Design of an Optimal Automation System

107

6 Discussion In this paper, an agent model and a framework for an intelligent support system is presented to decide on the optimal type of automation, taking both the task engagement and the exhaustion into account. The model is generic and can be used for different tasks. This requires a specification of the effect of the support actions on both the task demands and the strength of automation and the self agency. The specification of these effects is not trivial because it requires quantification for the effect of all actions on different aspects, but part of this specification can be based on actual measurements of demands. The approach has been illustrated with an example scenario. In this scenario, no agency actions are assumed, and a constant balance between the importance of task engagement and exhaustion is assumed. However, it could be that the importance changes over time. The framework can cope with such situations as well. This will result in a more varied behavior of the support system. For example, if the task is very critical at a specific point in time and the operator is too exhausted to perform, automation should be kept constant. When the critical task is performed, the task should be given back to the operator such that the operator’s engagement can be kept optimal.

References 1. Posner, M.I., Boies, S.J.: Components of attention. Psychological Bulletin 78, 391–408 (1971) 2. Rovira, E., McGarry, K., Parasuraman, R.: Effects of imperfect automation on decision making in a simulated command and control task. Human Factors 49(1), 76–87 (2007) 3. Endsley, M.R.: Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics 42(3), 462–492 (1999) 4. Parasuraman, R.: Designing automation for human use: empirical studies and quantative models. Ergonomics 43(7), 931–951 (2000) 5. Parasuraman, R.: Humans and automation: Use, misuse, disuse, abuse. Human Factors 39, 230–253 (1997) 6. Kaber, D.B., Endsley, M.R.: The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task. Theoretical Issues in Ergonomics Science 5(2), 113–153 (2004) 7. Parasuraman, R., Barnes, M., Cosenzo, K.: Adaptive automation for human-robot teaming in future command and control systems. The International C2 Journal 1(2), 43–68 (2007) 8. Wilson, G.F., Russell, C.A.: Performance enhancement in an uninhabited air vehicle task using psychophysiologically determined adaptive aiding. Human Factors 49(6), 1005– 1018 (2007) 9. Byrne, E.A., Parasuraman, R.: Psychophysiology and adaptive automation. Biological Psychology 42(5), 249–268 (1997) 10. Prinzel, L.J., Freeman, F.G., Scerbo, M.W., Mikulka, P.J., Pope, A.J.: Effects of a psychophysiological system for adaptive automation on performance, workload, and the eventrelated potential P300 component (2003) 11. Parasuraman, R., Wickens, C.D.: Humans: still vital after all these years of automation. Human Factors 50(3), 511–520 (2008)

108

M. Klein and R. van Lambalgen

12. Bosse, T., Both, F., Lambalgen, R.: van, and Treur, J., An Agent Model for a Human’s Functional State and Performance. In: Jain, L., Gini, M., Faltings, B.B., Terano, T., Zhang, C., Cercone, N., Cao, L. (eds.) Proceedings of the 8th IEEE/WIC/ACM International 13. Robert, G., Hockey, J.: Compensatory control in the regulation of human performance under stress and high workload: a cognitive-energetical framework. Biological Psychology 45, 73–93 (1997) 14. Hill, D.W.: The critical power concept. Sports Medicine 16, 237–254 (1993) 15. Wegner, D.M.: Précis of The illusion of conscious will. Behavioral and Brain Sciences 27, 1–46 (2004) 16. Aarts, H., Custers, R., Marien, H.: Priming and authorship ascription: when nonconscious goals turn into conscious experiences of self-agency. Journal of Personality and Social Psychology 96(5), 967–979 (2009) 17. Custers, R., Aarts, H.: Positive affect as implicit motivator: on the nonconscious operation of behavioral goals. Journal of Personality and Social Psychology 89(2), 129–142 (2005)

A Cognitive Agent Model Using Inverse Mirroring for False Attribution of Own Actions to Other Agents Jan Treur1 and Muhammad Umair1,2 1

Vrije Universiteit Amsterdam, Department of Artificial Intelligence De Boelelaan 1081, 1081 HV Amsterdam, The Netherlands 2 COMSATS Institute of Information Technology, Lahore, Pakistan {j.treur,m.umair}@vu.nl, [email protected] http://www.few.vu.nl/~{treur,mumair}

Abstract. This paper presents a cognitive agent model capable of showing situations where self-generated actions are attributed to other agents, as, for example, for patients suffering from schizophrenia. The mechanism underlying the model involves inverse mirroring: mapping preparation states onto sensory representations of observed actions. It is shown how this mechanism can develop based on Hebbian learning. The model provides a basis for applications to human-like virtual agents in the context of for example, training of therapists or agent-based generation of virtual stories. Keywords: action attribution, cognitive agent model, schizophrenia.

1

Introduction

For the development of agent models as a basis for virtual agents in serious or nonserious gaming, an often used criterion is that they show realistic human-like behaviour. One of the ways to obtain such human-like agent models is to exploit the fast growing amount of neurological literature, so that models are developed that have biological plausibility. In addition, to obtain realistic virtual agents, not only ideally functioning persons should be considered but also persons with deviant behaviour, in order to cover larger parts of the variety in types of behaviour as occurring naturally in the overall human population. This paper addresses an agent model for such naturally occurring deviant behaviour in attribution of self-generated actions to other agents. False attribution of self-generated (e.g., manual or verbal) actions to other agents is a common symptom occurring in patients with schizophrenia. One explanation put forward for the phenomenon that self-generated actions are not attributed to oneself is that self-attribution depends on prediction and monitoring of effects of these actions, and this does not function well for persons with schizophrenia; see, for example [6], [7], [9], [10]. However, in other work it is debated whether this is an appropriate explanation. For example, in [8] experimental work is reported that indicates that differences in these respects between patients with schizophrenia and a control group are not very convincing. In [16] it is argued that a more important role is played by what is called ‘the sense of agency’ (which is at a more conscious, personal level) than action effect prediction and monitoring (which is at a unconscious, subpersonal level). K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 109–119, 2011. © Springer-Verlag Berlin Heidelberg 2011

110

J. Treur and M. Umair

Note that the issue of not attributing a self-generated action to oneself, as addressed in the literature as mentioned above, is not the same as attributing such an action to another agent, as in order to create a mental image of somebody else performing the action requires a shift from a representation of an action from a firstperson to a representation from a third-person perspective (mental rotation). Patients with schizophrenia do not only fail to attribute self-generated actions to themselves, they also attribute them to other agents (which can be real or imaginary). Using neurological literature on mirroring [4], [14], [15], [20] and self-other differentiation [17], in this paper this form of false attribution to other agents is addressed. In this paper, in Section 2 the ideas from the neurological literature are briefly discussed. In Section 3 the cognitive agent model is introduced. Section 4 presents a number of simulation results. In Section 5 a mathematical analysis is made. Finally, Section 6 is a discussion.

2 The Cognitive Agent Model for False Attribution of Actions In this section the cognitive agent model and its detailed specifications are presented. First the modelling some background knowledge is briefly discussed, next the format used is briefly introduced, and the example scenario used is described, and finally the agent model is addressed in detail. Background knowledge. One of the recent neurological findings concerns the mirroring function of certain neurons; e.g., [4], [14], [15], [20]. Mirror neurons are active not only when a person prepares for a specific action or body change, but also when the person observes somebody else intending or performing this action or body change. This includes expressing emotions in body states, such as facial expressions. The idea is that these neurons and the neural circuits in which they are embedded play an important role in social functioning and in (empathic) understanding of others; e.g., [4], [14], [15], [20]. Their discovery is often considered a crucial step for the further development of the discipline of social cognition; cf. [15]. When states of other persons are mirrored by some of the person’s own states that at the same time are connected via neural circuits to states that are crucial for the person’s own feelings and actions (shared circuits), then this provides an effective basic mechanism for how in a social context persons fundamentally affect each other’s actions and feelings; e.g. [14]. Mirroring involves a change of perspective from another agent (third person) to oneself (first person). This requires a nontrivial mental rotation transformation of the available representations (cf. [17]): sensory representations of observed actions of other agents are mapped onto representational structures for self-generated actions. Attribution a self-generated action to another agent is in fact a kind of reverse process. It requires a change of perspective from preparation for a self-generated action (first person) to another agent (third person) perspective, based on a reverse mental rotation transformation of the available representations. In fact this is inverse mirroring: the representational structures for self-generated actions are mapped onto sensory representations of observed actions of other agents, thus forming a mental image of somebody else performing the action. When it is assumed that such an inverse

A Cognitive Agent Model Using Inverse Mirroring

111

mapping is made, a self-generated action is perceived as observed from a third person perspective, and thus it provides a mechanism for the self-generated action to be attributed to another agent. A further question is how such a reverse mental rotation mapping can exist or develop in a neurological context. One possibility is that the mechanism is there initially, due to improper genetics. However, another possibility is that it is developed during lifetime. This is also analysed below, assuming a Hebbian learning principle. This is the principle that connected neurons that are frequently activated simultaneously strengthen their connecting synapse. The principle goes back to Hebb [12], but has recently gained enhanced interest by more extensive empirical support (e.g., [2]), and more advanced mathematical formulations (e.g., [11]). In the models a variant of this principle has been adopted to realise an inverse mirroring connection from preparation of an action to sensory representation of a similar observed action. Modelling Format Used. To formalise the agent model in an executable manner, the hybrid dynamic modelling language LEADSTO has been used; cf. [3]. Within LEADSTO the dynamic property or temporal relation a → →D b denotes that when a state property a occurs, then after a certain time delay (which for each relation instance can be specified as any positive real number D), state property b will occur. Below, this D will be taken as the time step Δt, and usually not be mentioned explicitly. Both logical and quantitative calculations can be specified, and a software environment is available to support specification and simulation. In most cases in the model below some form of combination function f is used. A typical example of an update rule in LEADSTO is SS(w1, V1) & SS(w1, V1) & SR(w3, V3)

→ → SR(w3, V3 + γ [ f(ω1V1, ω2V2) – V3 ] Δt)

which expresses that when the sensor states for w1 and w2 have values V1 and V2 respectively and the sensory representation of w3 has value V3, then after time duration Δt this sensory representation will have value V3 + γ [ f(ω1V1, ω2V2) – V3 ] Δt. Here ω1 and ω2 are the connection strengths from the sensor states to the sensory representation, respectively, and γ is an update speed factor. Moreover, f is a combination function, for which different choices can be made, for example, f(W1, W2) = β(1- (1-W1)(1-W2)) + (1 - β )W1W2

(0 ≤ β ≤1)

In simulations with the agent model, a combination function f(W1, …, Wn) based on a continuous logistic threshold function th(σ, τ, W) has been used of the form f(W1, …, Wn) = th(σ, τ, W1 +… + Wn) (where σ is a steepness and τ a threshold value) with: th(σ, τ, W) =

(

σ

τ

-

στ

) (1

στ

)

For higher values of στ this threshold function is approximated by the expression: th(σ, τ, W) =

1

σ

τ

Example Scenario. The designed agent model will be illustrated for the following scenario. A sensed stimulus s1 leads to a sensory representation of this stimulus, which in turn triggers preparation and execution of an action b as a response of the agent; see the causal chain in the lower part of Fig. 1. Moreover, when another agent

112

J. Treur and M. Umair

performs action b, this is represented as a stimulus s2 that can be sensed; see upper part of Fig. 1. The sensory representation srs(s2) indicates the mental image of another person performing the action. The agent’s mirroring capability is based on the assumption that an activated sensory representation of s2 will also activate the agent’s own preparation for b. This assumption has been confirmed in neurological literature such as [4], [14], [15], [20]. When this latter chain of events happens (i.e., whenever mirroring takes place) , for the model introduced here it is assumed that by Hebbian learning this will strengthen the connection from preparation of b to sensory representation of s2 (observed action), thus developing inverse mirroring capabilities. When such a learning process has achieved substantial connection strength, the agent’s response on stimulus s1 has changed. When s1 is sensed (in the absence of s2), not only will the agent trigger preparation and execution of action b as before, but in addition it will generate a mental image of another agent performing action b (the sensory representation srs(s2)), thus creating a third person perspective on the action.

LP1

ω0 world_state(s2)

LP5

ω1

ω

sensor_state(s2)

srs(s2)

ω4 LP1

ω0 world_state(s1)

LP2

LP3

ω2 sensor_state(s1)

LP4

ω5

ω3

preparation_state(b)

srs(s1)

effector_state(b)

Fig. 1. Overview of the cognitive agent model Table 1. State properties used notation description world state for s SS(s) sensor state for s SRS(s) sensory representation state of s WS(s)

notation description preparation state for b execution state for b cs(ω) strength of connection ω (from preparation of b to sensory representation of s2)

PS(b) ES(b)

Detailed Specification of the Agent Model. In the detailed specification, states have been formalised as shown in Table 1. Moreover, the dynamical relationships between these states are shown in Table 2. Note that each connection has a fixed strength, except the connection from preparation of b to sensory representation of s2, which is adapted over time by the Hebbian learning. More detailed specifications of the dynamical relationships are presented below. First it is shown how in a generic manner sensor states are generated from world states.

A Cognitive Agent Model Using Inverse Mirroring

113

Table 2. Overview of the connections and their weights from states

to state

WS(s)

SS(s)

SS(s1)

SRS(s1)

SRS(s1), SRS(s2)

PS(b)

weights

process

LP

sensing world state representing world state preparing action / mirroring action action execution representing observed action / inverse mirroring action Hebbian learning for inverse mirroring

LP1 LP2 LP3 LP4 LP5 LP6

PS(b)

ES(b)

SS(s2), PS(b)

SRS(s2)

ω0 ω2 ω3, ω4 ω5 ω1, ω

SRS(s2), PS(b)

cs(ω)

η, ζ

LP1 Generating a sensor state If world state s has level V1 and the sensor state for s has level V2 then after duration Δt the sensor state for s will have level V2 + γ [ f(ω0V1) – V2] Δt WS(s, V1) & SS(s, V2) → → SS(s, V2 + γ [ f(ω0V1) – V2] Δt)

Note that this applies to both s1 and s2. Activation for sensory representation for s1 has been modelled in a straightforward manner as shown in LP2. LP2 Sensory representation for a sensor state If property s1 is sensed with level V1 and the sensory representation of s1 has level V2. then after Δt the sensory representation of s will have level V2 + γ [ f(ω2V1) - V2 ] Δt. SS(s1, V1) & SRS(s1, V2) → → SRS(s1, V2 + γ [ f(ω2V1) – V2 ] Δt Next it is shown how an action preparation for b is generated. This is based on the sensory representation of stimulus s1, or based on the sensory representation of s2 representing an observed action of another agent; in the latter case mirroring takes place. Note that the two options are not exclusive. LP3 Preparing and mirroring an action If sensory representation of s1 has level V1, and sensory representation of s2 has level V2, and the preparation for b has level V3 then after Δt the preparation state for b will have level V3 + γ (f(ω3V1, ω4V2) - V3) Δt. SRS(s1,V1) & SRS(s2,V2) & PS(b, V3) → → PS(b, V3 + γ (f(ω3V1, ω4V2) - V3) Δt) Action execution has been modelled in a straightforward manner as shown in LP4. LP4 Action execution If preparation for b has level V1 and the action execution state for b has level V2 then after Δt the action execution state for b will have level V2 + γ (f( ω5V1) – V2) Δt. PS(b, V1) & ES(b, V2)

→ → ES(b, V2 + γ (f(ω5V1) – V2) Δt)

Next it is shown in LP5 how the sensory representation for s2 (a mental image of an observed action) is generated based on a sensor state for s2 or an action preparation for b (or a combination of both). In the latter case inverse mirroring take place. LP5 Representing a world state and inverse mirroring of an action If sensor state of s2 has level V1, and preparation of b has level V2, and the sensory representation of s2 has level V3

114

J. Treur and M. Umair

then after Δt the sensory representation of s2 will have level V3 + γ (f(ω1V1, ωV2) V3) Δt. SS(s2,V1) & PS(b,V2) & SRS(s2, V3) → → SRS(s2, V3 + γ (f(ω1V1, ωV2) - V3) Δt) Finally, it is shown in LP6 how the Hebbian learning process of the connection from preparation state for b to sensory representation s2 of an observed action was modelled. This takes place using the following Hebbian learning rule, with maximal connection strength 1, a learning rate η, and extinction rate ζ (usually taken small): Δω = γ [ ηV1V2 (1 - ω) - ζω ] Δt

Here V1 and V2 are (time-dependent) activation levels of the connected nodes, and

γ is an adaptation speed factor. In differential equation format it can be written as ω

= γ [ ηV1V2 (1 - ω) - ζω ] = γ [ ηV1V2 - (ηV1V2 + ζ) ω ]

A similar Hebbian learning rule can be found in [11], p. 406. By the factor (1 - ω) the learning rule keeps the level of ω bounded by 1. When the extinction rate is relatively low, the upward changes during learning are proportional to both V1 and V2 and maximal learning takes place when both are 1. Whenever one of them is close to 0, extinction takes over, and ω slowly decreases. This is specified as follows: LP6 Learning for inverse mirroring If the sensory representation of stimulus s2 has level V1,

and the preparation for b has level V2, and the connection weight from preparation for b to sensory representation of s2 has level W, then after duration Δt the connection weight from preparation for b to sensory representation of s2 will have level W + γ [ ηV1V2 (1 - W) - ζW ] Δt. → cs(ω, W + γ [ ηV1V2 (1 - W) - ζW ] Δt) SRS(s2, V1) & PS(b, V2) & cs(ω, W) →

3

Simulation Results

A number of simulations have been performed with the focus of simulating normal functioning and deviant functioning of the model. Normal functioning of the agent occurs by parameter settings in which stimulus s1 does not lead to high activation of sensory representation of s2 (i.e., no mental image of somebody else performing the action is created) in absence of the stimulus s2, although stimulus s2 has occurred time and time again in the past. In contrast, deviant functioning occurs by parameter setting in which stimulus s1 does lead to high activation of sensory representation of s2 under similar circumstances, so in this case a mental image of somebody else performing the action is created. In the simulations shown, time is on the horizontal axis and the activation level of the state properties is on the vertical axis. The connection strengths between different states were initialized with 1 (i.e., ω0 = ω1 = ω2 = ω3 = ω4 = ω5 = 1) and kept fixed throughout the simulation, except the connection strength ω which was initialized with 0 and adapted over time by the Hebbian learning rule given in LP6 in Section 2. Other parameters are set as: ∆t = 0.1, learning rate η = 0.3, extinction rate ζ = 0.2,

A Cognitive Agent Model Using Inverse Mirroring

WS(s2)

V a l u e

1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

WS(s1)

V a l u e

0

115

1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 Time Point

Fig. 1. World State for s2

10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 Time Point

Fig. 2. World State for s1

speed factor λ = 0.5, steepness σ = 5 and threshold τ = 0.5. For the initial duration of 45 time units the stimulus s2 occurs three times for 5 time units alternatively, i.e., for the first 5 time units world state for s2 has value 1 and for the next 15 time units value 0, and so on (see Fig. 1). During these 45 time units the world state for s1 was kept 0 (see Fig. 2). This represents the situation in which a person observes somebody else performing some action (or bodily change) and the mirroring function of the preparation neurons makes the person prepare for this action. The fluctuation in the activation level of the sensor state is repeating the same pattern between 0.1 to 0.9 as it only depends (via LP1) upon the world state for s2, which also is repetitive. PS(b) 1 0.9 0.8 V 0.7 a 0.6 l 0.5 u 0.4 e 0.3 0.2 0.1 0

CS(ω) 1 0.9 0.8 V 0.7 a 0.6 l 0.5 u 0.4 e 0.3 0.2 0.1 0

0

10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 Time Point

Fig. 3. Preparation State for b (Normal Functioning)

0

10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 Time Point

Fig. 4. Connection Strength ω (Normal Functioning)

Fig. 3 shows the activation of preparation state PS(b) resulting from the sensory representation pattern. The slight change in the strength of the connection (inverse mirroring) from preparation state PS(b) to sensory representation SRS(s2) (via LP6) shows a similar but slightly delayed fluctuating pattern; see Fig. 4. After that for about 30 time units both world states for s1 and s2 were kept 0, so that the effect of any stimulus on preparation state and execution state becomes zero, as reflected, for example, in Fig. 3. For the rest of the simulation, the world state for s2 is kept 0 while the world state for s1 is becoming 1 (see Figs 1 and 2). As soon as world state for s1 becomes 1, the values of sensor state and its sensory representation increase smoothly and become stable at a high value (of about 0.9). One of the interesting facts (in

116

J. Treur and M. Umair

comparison to the second scenario discussed below) is that, even though a link from preparation state to SRS(s2) develops, it is not strong enough to propagate the effect of SRS(s1) via PS(b) to SRS(s2), see Fig. 4. Hence SRS(s1) only has a positive effect on the activation levels of the preparation state and execution state of b (shown in Fig. 3), and not on SRS(s2). No activation is developed of a mental image SRS(s2) of another person performing action b; this shows normal functioning. To obtain the deviant behavior of the model, again all parameters were initialized with the same values as used for the normal behavior and mentioned earlier in current section, except the extinction rate which was taken lower this time: ζ = 0.01. In this case by Hebbian learning the connection from preparation state of b to sensory representation of s2 achieves a substantially higher connection strength (see Fig. 6) which also induces an upward trend in the fluctuating value of the preparation state for b (see Fig. 5). PS(b)

V a l u e

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

CS(ω)

V a l u e

0

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 Time Point

0

10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 Time Point

Fig. 6. Connection Strength ω (Deviant Functioning)

Fig. 5. Preparation State for b (Deviant Functioning)

This achieved connection strength is sufficient to change the impact of stimulus s1 on SRS(s2) (see Fig. 8) compared to the impact shown in Fig. 7. Even in the absence of the world state for s2, from time 75 onwards, if the world state for s1 occurs, it leads to high activation of SRS(s2) (see Fig. 8), which shows that the agent develops a mental image of somebody else performing action b. This contrasts the case of normal functioning in which case after time point 75 the level of SRS(s2) stays (close to) 0 (see Fig. 7). SRS(s2) 1 0.9 0.8 V 0.7 a 0.6 l 0.5 u 0.4 e 0.3 0.2 0.1 0

SRS(s2)

V a l u e

0

10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 Time Point

Fig. 7. Sensory Representation for s2 (Normal Functioning)

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 Time Point

Fig. 8. Sensory Representation for s2 (Deviant Functioning)

A Cognitive Agent Model Using Inverse Mirroring

117

4 Mathematical Analysis of the Model The equilibrium of the connection strength from preparation state of b to the sensory representation of s2 may be found by using the equation for Δω presented in Section 2 to determine the change in the connection strength after ∆t as follows. An equilibrium will occur if Δ ω = 0, so the equation can be rewritten as

γ [ ηV1V2 (1 - ω) - ζω ] Δt = 0

or

ηV1V2 (1 - ω) - ζω = 0

or

As V1V2 ≤ 1, the following is an upper bound for the equilibrium value: 1 1 This expression gives the relation describing the maximum connection strength that may be achieved for given values of learning and extinction rates. It shows that for a smaller value of the extinction rate compared to the learning rate the connection strength will be closer to 1. For the simulation results discussed in Section 3, for the second case (deviant functioning) the learning rate was η = 0.3 and the extinction rate ζ = 0.01 which means the maximum connection strength which may be achieved is 0.97. Indeed it was observed that the connection strength indeed becomes stabilized just below 0.97. Similarly, for the case of normal functioning, the learning rate was η = 0.3 and extinction rate ζ = 0.2. Indeed the equilibrium was below the 0.6 indicated by the above analysis. Note that these are upper bounds resulting from maximal values 1 continuously for V1 and V2. In practice this will not continuously happen, so the strength will stay lower.

5 Discussion One of the recent developments in Neuroscience concerns the notion of a mirror system and its functions; e.g., [4], [14], [15], [20]. Cognitive agent models have been designed using this notion as a point of departure, and showing its role in various high-level cognitive and social capabilities such as prediction, imagination, emotion reading, empathic understanding, imitation, and attribution of observed actions; see, for example [5], [13], [18]. Mirroring is a process from an observed action or body state of another person to the person’s own preparation states. As discussed in [17] this involves a mental rotation mapping sensory representations of observed actions of other agents onto the representational structures for self-generated actions. This realises a change of perspective from another agent (third-person) to perspective from oneself (first-person). Attribution a self-generated action to another agent proceeds in the opposite direction, realising a change of perspective from oneself (first-person) to another agent (third-person) perspective. This requires inverse mirroring: the representational structures for self-generated actions are mapped onto sensory representations of an observed action of another agent, thus creating a mental image of another agent performing the action. When such a mapping involving reverse mental rotation is made, a self-generated action is perceived as observed from a third person

118

J. Treur and M. Umair

perspective. The agent model presented in this paper addresses the issue of false attribution of self-generated actions using such a mechanism. In addition it is shown how the mechanism can develop based on Hebbian learning [12], [2], [11]. Note that due to the opposite direction, an inverse mirroring process is not covered by cognitive agent models based on mirroring, such as descibed in [5], [13], [18]. The modelling format used to formally specify the agent model is based on the executable hybrid dynamical modelling language LEADSTO [3]. This hybrid language combines executable temporal logical elements [1] and numerical dynamical system elements [19]. Although the model can also be specified well in a purely numerical format, an advantage of LEADSTO is the possibility to use a logical format to describe state properties. As a next step a validation study can be conducted to compare the agent model’s behaviour to real behaviours. The agent model obtained can be used as a basis for applications involving realistic, human-like virtual agents in the context of serious or nonserious gaming. In the area of virtual stories it can be used to create virtual characters that react in a less standard but realistic manner. Another possible application is to obtain virtual patients in the area of simulation-based training for psychotherapists.

References 1. Barringer, H., Fisher, M., Gabbay, D., Owens, R., Reynolds, M.: The Imperative Future: Principles of Executable Temporal Logic. John Wiley & Sons, Chichester (1996) 2. Bi, G.Q., Poo, M.M.: Synaptic Modifications by Correlated Activity: Hebb’s Postu-late Revisited. Ann. Rev. Neurosci. 24, 139–166 (2001) 3. Bosse, T., Jonker, C.M., Meij, L., van der Treur, J.: A Language and Environment for Analysis of Dynamics by Simulation. Intern. J. of AI Tools 16, 435–464 (2007) 4. Bosse, T., Memon, Z.A., Treur, J.: A Cognitive and Neural Model for Adaptive Emotion Reading by Mirroring Preparation States and Hebbian Learning. Cognitive Sys-tems Research Journal (in press, 2011) 5. Damasio, A., Meyer, K.: Behind the looking-glass. Nature 454, 167–168 (2008) 6. Farrer, C., Franck, N.: Self-monitoring in schizophrenia. Curr. Psychiatry Rev. 3, 243–251 (2007) 7. Feinberg, I.: Efference copy and corollary discharge: Implications for thinking and its disorders. Schizophrenia Bulletin 4, 636–640 (1978) 8. Fourneret, P., de Vignemont, F., Franck, N., Slachevsky, A., Dubois, B., Jeannerod, M.: Perception of self-generated action in schizophrenia. Cogn. Neuropsych. 7, 139–156 (2002) 9. Frith, C.D., Blakemore, S., Wolpert, D.: Explaining the symptoms of schizophrenia: Abnormalities in the awareness of action. Brain Research Rev. 31, 357–363 (2000) 10. Frith, C.D.: The cognitive neuropsychology of schizophrenia. Lawrence Erlbaum Associates Ltd., Hove (1992) 11. Gerstner, W., Kistler, W.M.: Mathematical formulations of Hebbian learning. Biol. Cybern. 87, 404–415 (2002) 12. Hebb, D.O.: The Organization of Behaviour. John Wiley & Sons, New York (1949) 13. Hendriks, M., Treur, J.: Modeling Super Mirroring Functionality in Action Execution, Imagination, Mirroring, and Imitation. In: Pan, J.-S., Chen, S.-M., Nguyen, N.T. (eds.) ICCCI 2010. LNCS, vol. 6421, pp. 330–342. Springer, Heidelberg (2010)

A Cognitive Agent Model Using Inverse Mirroring

119

14. Hurley, S.: The shared circuits model: How control, mirroring, and simulation can enable imitation, deliberation, and mind reading. Behavioral and Brain Sciences 31, 1–22 (2008) 15. Iacoboni, M.: Mirroring People: the New Science of How We Connect with Others. Farrar, Straus & Giroux, New York (2008) 16. Jeannerod, M.: The sense of agency and its disturbances in schizophrenia: a reappraisal. Exp. Brain Res. 192, 527–532 (2009) 17. Jeannerod, M., Anquetil, T.: Putting oneself in the perspective of the other: A frame-work for self-other differentiation. Social Neuroscience 3, 356–367 (2008) 18. Memon, Z.A., Treur, J.: Designing Social Agents with Empathic Understanding. In: Nguyen, N.T., Kowalczyk, R., Chen, S.-M. (eds.) ICCCI 2009. LNCS (LNAI), vol. 5796, pp. 279–293. Springer, Heidelberg (2009) 19. Port, R.F., van Gelder, T. (eds.): Mind as Motion: Explorations in the Dynamics of Cognition. MIT Press, Cambridge (1995) 20. Rizzolatti, G., Sinigaglia, C.: Mirrors in the Brain: How Our Minds Share Actions and Emotions. Oxford Univ. Press, Oxford (2008)

Explaining Negotiation: Obtaining a Shared Mental Model of Preferences Iris van de Kieft, Catholijn M. Jonker, and M. Birna van Riemsdijk Man Machine Interaction Group, Delft University of Technology, Mekelweg 4, 2628 CD Delft, The Netherlands

Abstract. Negotiation support systems (NSSs) aim to assist people during the complex process of negotiation. We argue that having a shared mental model of the negotiation task enables and enhances the collaboration between the human negotiator and the NSS. This paper presents an analysis of negotiation that results in a set of concepts that a shared mental model of the user and the NSS should contain. Discrepancies between the individual mental models can arise for various reasons, such as the constructive nature of preferences. Explanation can increase user understanding of the NSS’s reasoning, allowing the user to detect and resolve discrepancies. We therefore propose using explanation to achieve and maintain sharedness. We present a framework that provides a means to generate content for such explanations, where we focus on the mental models of user and opponent preferences.

1

Introduction

Negotiation is an interactive decision-making process between two or more parties. It is a complex process that involves emotions as well as computational complexity. As a result, even experienced human negotiators can fail to achieve efficient outcomes [11]. This has motivated the development of negotiation support systems (NSSs). These software systems assist a human negotiator (user) in negotiation by, for example, aiding communication, enhancing negotiation skills, and reducing cognitive task load. The Pocket Negotiator project, see [4], strives for synergy between NSS and the human negotiator. The NSS and the user should work together as a team in which their complementary skills are needed to achieve good outcomes. It is wellknown from the social psychology literature that performance of human teams is positively influenced by the team members having a shared understanding or shared mental model of the task and the team work involved ([5,7]). The concept of shared mental model is defined in [3] as: knowledge structures held by members of a team that enable them to form accurate explanations and expectations for the task, and, in turn, coordinate their actions and adapt their behavior to demands of the task and other team members. K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 120–129, 2011. c Springer-Verlag Berlin Heidelberg 2011 

Explaining Negotiation: Obtaining a Shared Mental Model of Preferences

121

We maintain that having a shared mental model is not only important in human teams, but also in human-agent teams. The representation that an automated agent has of a task can be viewed as its mental model. Discrepancies between the mental models of the NSS and the user may at best result in innocent misunderstandings but at its worst may result in a dysfunctional cooperation. This paper contributes to the technology to achieve sharedness between mental models of NSS and user. In Sect. 2, we first analyze negotiation and the interaction between user and NSS, to determine the essential components of such a shared mental model . Furthermore, the analysis reveals the possible causes of discrepancies between mental models, for example, the constructive nature of preferences. Based on our analysis, in Sect. 3 we identify explanation (see for example [12,13]), as a suitable technology for improving sharedness between mental models. Explanations can increase the transparency of the system, allowing the user to detect and resolve discrepancies. We provide a structured approach to generating such explanations. Due to space restrictions, we focus on one part of the shared mental model: user and opponent preferences. We describe different levels of content selection, for explanations of bids and their utilities. We leave the form (i.e., presentation) of the explanation for future work.

2

Bilateral Multi-issue Negotiation

Bilateral multi-issue negotiation is the process in which two parties try to reach an agreement about multiple issues. The following four major stages can be discerned in integrative negotiation: private preparation, joint exploration, bidding, and closing. Private preparation is about information gathering and reflection before meeting the other party. In joint exploration the negotiating parties talk to each other, but do not place bids on the table. During bidding, both negotiators exchange bids according to the agreed protocol, typically a turn-taking protocol. During the closing stage the outcome of the bidding stage is formalized and confirmed by both parties. We first present a basic negotiation framework, followed by an analysis of the negotiation process. This analysis discusses human weaknesses in negotiation and subsequently focuses on the interaction between user and NSS. 2.1

A Basic Negotiation Framework

Let p ∈ {s, o} represent the negotiating parties “Self” (s) and “Opponent” (o). Let I, with typical element i, denote the finite set of issues under negotiation. For example, in a job negotiation I might consist of the issues salary, car, vacation days. For convenience we assume issues to be numbered from 1 to n, where n = |I|, and henceforth, we will refer to issues by their respective numbers. For example, issue 1 is salary, issue 2 is car, issue 3 is vacation days. For each i ∈ I let Vi be the set of possible values that this issue may have. For example, the issue car may have {yes, no} as possible values. Let V = V1 × V2 × . . . × Vn , be the set of possible outcomes. A possible outcome v ∈ V is

122

I. van de Kieft, C.M. Jonker, and M.B. van Riemsdijk

thus an n-tuple v1 , v2 , . . . , vn . A possible outcome for the job domain can, for example, be 2000, no, 20. A negotiation domain is denoted by D = I, V . A bid in domain D is denoted by bp , with b ∈ V and p the party that proposed the bid. For simplicity, a bid may be denoted as b, if p is understood or not relevant. Each party p has a utility function up : V → [0, 1] which assigns a utility between 0 and 1 to possible outcomes and bids. A commonly used type of utility linear additive normalized function, defined by up (b) = n ifunction is the i i=1 up (b), where up (b) is the utility of issue value bi for party p. That function is defined in terms of a weight wpi and evaluation function eip for that issue and party: uip (b) = wpi eip (bi ). The weight is the relative importance that n party p assigns to issue i in such a way that i=1 wpi = 1, for all p ∈ {s, o}. The evaluation function eip : Vi → [0, 1] assigns a score between 0 and 1 to the possible values of issue i. We use the notation up(b) to represent the ntuple u1p (b), u2p (b), . . . , unp (b). Similarly, ep (b) = e1p (b1 ), . . . , enp (bn ) and w p = 1 n wp , . . . , wp . 2.2

The Weaknesses of the Human Negotiator

In this first part of our analysis, we discuss the problems humans have with negotiation, assuming there is no NSS support. There are two ways to categorize the problems humans have with negotiation: related to outcome, or related to the negotiation process. The outcome related pitfalls in negotiation are: leaving money on the table, settling for too little, rejecting a better offer than any other available option, and settling for terms worse than alternative options [1,11]. The outcome related pitfalls are caused by the problems people have during the negotiation process, which are related to the following (see [1,11] for more information): – Lack of training: Without training, humans often have difficulty in structuring negotiation problems and thinking creatively about such problems. – Lack of preparation: Preparation is insufficient when it leaves the negotiator unaware of an important part of the issues, underlying interests, the preferences and/or circumstances of the parties involved, see for example,. – Structural barriers to agreement: This refers to such problems as die-hard bargainers, a bad atmosphere, power imbalance [6], cultural and gender differences, disruptive or incommunicative people, and a lack of information. – Mental errors: Parties commit mental errors such as the escalation error, biased perception, irrational expectations, overconfidence, and unchecked emotions. – Satisficing: Due to uncertainty of the future, the costs of acquiring information, and the limitations of their computational capacities, people have only bounded rationality, forcing them to make decisions by satisficing, not by maximization. This difficulties indicate why it may be difficult for a human to have an accurate mental model of a negotiation.

Explaining Negotiation: Obtaining a Shared Mental Model of Preferences

2.3

123

The Interaction between Human Negotiator and NSS: What to Share?

In this section we analyze the interaction between user and NSS, to provide insight into their task division. This analysis helps determine the contents of the shared mental model that needs to be cultivated between user and NSS. Negotiation is a prime example of a task for which the human mind is only partially equipped, and for which artificial intelligence can only provide partial assistance. For example, the user has a wealth of knowledge about the world and about interacting with other humans, but need not be a specialist in negotiation. The NSS specializes in negotiation. It makes generic negotiation knowledge available to the human. Also, the user has limited working memory and limited computational power, i.e., bounded rationality. The NSS has better memory and can search much more quickly through much larger outcome spaces. This implies that tasks should be divided between user and NSS in a way that respects their complementary capabilities. This task division suggests they need not share all their knowledge. However, some shared information is necessary for cooperation, hence the need for a shared mental model. The information and knowledge exchange between these two team members is as follows: during the preparation and exploration stage the user needs to inform the NSS about the current negotiation, e.g., the Opponent, the set of issues I, and outcome space V , and the utility functions of Self and Opponent. These utility functions are a model of the actual preferences of Self and Opponent. The actual preferences may not be fully known, and may be subject to change. The NSS needs this user input in order to provide assistance during the bidding stage, when strategic bidding decisions have to be made. For this information exchange to be successful, the user must fully understand the process of negotiation and what is expected of him/her by the NSS, and what can be expected in return. This implies that during the negotiation stages, the NSS needs to provide the user (upon request) with generic negotiation information, but also current negotiation information regarding the Opponent, I, V , and utility functions, in as far as such information is available to the NSS. Thus, a shared mental model of a human negotiator and an NSS should at least contain submodels on: – domain knowledge D • I: set of issues • ∀i ∈ I: Vi the value range of issue i – knowledge about negotiating parties Self and Opponent. For each p ∈ {s, o}: • up : the utility function of p, in so far as known • the emotional status and coping style of p • the negotiation model of p – knowledge about the capabilities of the team members: Self and NSS

124

I. van de Kieft, C.M. Jonker, and M.B. van Riemsdijk

– bidding knowledge • bidding history: the sequence of bids that have been exchanged so far • the current bidding strategy for Self • the bidding protocol, including information about available time 2.4

Discrepancies between Mental Models

Based on the previous subsections, we identify what may cause lack of sharedness with respect to the elements of the shared mental model, or in other words, what may cause discrepancies between the mental models of the user and the NSS. We consider a discrepancy between mental models to exist when one model contains information regarding an element, and the other model contains either conflicting information regarding this element, or no information regarding this element. Once a discrepancy is detected, it can be resolved by adapting (one of) the mental models. One particular aspect that may lead to discrepancies is the constructiveness of domain and preference information. Even with proper preparation, information on the domain and preferences of Self and Opponent is often difficult to determine fully at the start of the negotiation. Humans have been found to discover this information along the way. Due to this constructiveness, the user may discover new knowledge during the negotiation that the NSS does not yet have, thus causing a discrepancy. Table 1 lists, for each team member, some possible causes for their mental model to lack (correct) information. Table 1. Causes for lack of (correct) information in mental models User mental model -lack of training -lack of preparation -constructive domain -bounded rationality -constructive preferences of Self and Other

3

NSS mental model -lack of user input -constructive domain -constructive preferences of Self and Other

Explanation Framework: Resolving Discrepancies

Explanation can serve various purposes, such as improving effectiveness (helping users make good decisions), increasing the users trust in the system and improving transparency of the system [12]. In this paper we are in particular interested in the latter, as this facilitates detection and resolving of discrepancies between mental models of NSS and user. Transparency means explaining how the system works, thus giving the user a better understanding of the NSS’s reasoning process. This allows the user to detect any discrepancies between the mental models, and subsequently to resolve these discrepancies by updating the mental models where necessary. Due to space restrictions, for the remainder of this paper, we focus on the mental models of preferences, as represented by the utility functions uS and uO . The different causes of discrepancies between the mental models (of preferences)

Explaining Negotiation: Obtaining a Shared Mental Model of Preferences

125

suggest that different types of preference information need to be made transparent in different situations. First, lack of training can result in difficulty in switching between one’s own perspective and that of the Opponent. The user may neglect the preferences of the Opponent, whereas the NSS often uses this information for bid calculations. This suggests that both user preferences and opponent preferences should be presented to the user, so that the user remains aware of both perspectives. This will make it easier for the user to understand the NSS’s bid recommendations. Second, due to bounded rationality, users may have difficulty calculating bid utilities. To make transparent why bids are assigned a certain utility value, more insight into the utility function needs to be available. If the NSS assigns a utility that the user believes is too high, it is not always immediately clear to the user why it is too high. In order to resolve the discrepancy, the user needs to be able to determine where the problem lies exactly (i.e., in which part of which mental model). As we assume the utility function is a linear additive function, we distinguish three levels of insight into this function. The first level is the overall utility, up (b). The second level concerns the building blocks of the first level, the issue utilities: uip (b). The third level concerns the building blocks of the second level: the weights and evaluation functions of the issues: wpi and eip (bi ). Third, also due to bounded rationality, users may have difficulty calculating utility differences between bids. This suggests that these differences should also be made transparent. The NSS may determine that bid b is better than bid b , however, the user may also want to know how much better. Explanation about utility differences reduces the chance of mental errors such as irrational expectations. In this section, we present a framework that can be used as the basis for generating explanations to increase transparency with respect to the NSS’s mental model of preferences. Research has distinguished between the content and form of explanations (e.g., [9]). The focus of the proposed framework is content selection; how it should be presented is an additional step that we only touch upon lightly in this paper. Further work will address this in more detail. The explanation content is selected from the so-called originator [10], which in our case is the original NSS without explanation capabilities. The content is selected by the so-called explainer [10], a component that is to be added to the NSS. The explainer is in charge of generating explanations. Given a bid b, we assume that the utility functions up (b) and their sub-parts uip (b), wpi and eip (bi ) are provided by the originator. When comparing two bids, additional content regarding differences in utility is necessary. This may not be readily available in the originator, we thus leave it to the explainer to perform these calculations. Any additional knowledge that should be present in the explainer will be described at the relevant points in the framework. Content selection is presented for each of the three levels of detail of the utility function described above. For each level, the framework provides the content to be selected when evaluating a single bid, as well as the additional content needed when comparing two bids.

126

3.1

I. van de Kieft, C.M. Jonker, and M.B. van Riemsdijk

Overall Utility Level

This level concerns the overall utility function: up (b). When evaluating a single bid b at this level, the content selection, referred to as CS1, consists of the bid itself and the overall utility up (b) for each p involved: CS1(b) = b, us (b), uo (b) When comparing two bids b and b , we first select the relevant content for a single bid (CS1). Additionally, the explainer calculates the difference in overall bid utility for each party. The difference in overall utility for party p between two bids b and b is defined as follows: Δup (b, b ) = up (b ) − up (b). Then, the content selection, referred to as CS2, is: CS2(b, b ) = CS1(b), CS1(b ), Δus (b, b ), Δuo (b, b ) This content could, for example, be presented to the user as follows: “Bid b has Δus (b, b ) more utility for you than bid b. For your opponent bid b has Δuo (b, b ) more utility than bid b.” A graphical display could supplement this text, listing the utilities per bid per party, such as in Table 2. Table 2. Content for overall utility level bid b b

3.2

utility self us (b) us (b ) Δus (b, b )

utility other uo (b) uo (b ) Δuo (b, b )

Issue Utility Level

This level concerns the utility functions per issue, uip (b). Here, more detail is provided as to how the utility of the bid was calculated, by showing how the utilities per issue together determine the overall utility. For a single bid b, we select the bid and the n-tuple of the utility values per issue, for each party: CS1(b) = b, us (b), uo (b) Furthermore, additional explanation knowledge is needed, to explain to the user how the utilities per issue are combined to get the overall utility. Because here we use an additive utility function, the additional content consists of the fact that the combination of the issue utilities is additive. When comparing two bids b, b at this level, we select the relevant content for each bid separately (CS1). Additionally, we provide content to show which issues have different values, and what the associated difference in utility is. We first define the difference in utility of issue i for party p, between bid b and b :

Explaining Negotiation: Obtaining a Shared Mental Model of Preferences

127

Δuip (b, b ) = uip (b ) − uip (b). The corresponding n-tuple of utility difference for  p (b, b ) The content selection is then defined as follows: each issue is: Δu  s (b, b ), Δu  o (b, b ) CS2(b, b ) = CS1(b), CS1(b ), Δu As this content consists of several n-tuples, a textual presentation does not seem suitable; a graphical presentation such as Fig. 1 seems more appropriate. In this figure, the utility differences are not presented as such, however, this information is used to highlight certain utility changes. In general, focus can be created by highlighting just the issues that have different values (DiffIssues), increased utility value (PosDiff ), decreased utility value (NegDiff ), or the same utility value (NoDiff ). This could be made into a dynamic feature for the user to interact with. We therefore define the following sets of issues: DiffIssues(b, b ) PosDiff p (b, b ) NegDiff p (b, b ) NoDiff p (b, b )

= {i|bi = bi } = {i|Δuip(b, b ) > 0} = {i|Δuip(b, b ) < 0} = {i|Δuip(b, b ) = 0}

In addition to highlighting purposes, these sets can be used to select a subset of the content. For example, if only one issue differs between bids b and b (i.e., |DiffIssues(b, b )| = 1), we might choose to only present the content related to that issue i ∈ DiffIssues(b, b ).

Fig. 1. Example presentation of explanation content for two bids, two parties, at issue utility level. The utilities highlighted in white indicate a decrease in utility, the utility highlighted in orange indicates an increase in utility, when comparing b and b

3.3

Level of Issue Weights and Evaluation Functions

This third level provides more detail as to why an issue has a certain utility value. This concerns the weights and evaluation functions of the issues, wpi and eip , which together determine the utility assigned to an issue value. The content when evaluating a single bid b consists of the issue weights and the evaluation function score of each issue value: CS1(b) = b, w  p , ep (b)

128

I. van de Kieft, C.M. Jonker, and M.B. van Riemsdijk

Additionally, the explanation should contain the information that the weight and evaluation function score are multiplied to give the issue utility. Several kinds of comparisons can be made at this level. For example, comparisons can take place for one issue across two bids, for two issues within one bid or for two issues across two bids. Due to the many possible comparisons, this information should only be presented to the user when requested. We define Δwp (i, j) = (wpj − wpi ) and Δep (bi , bj ) = ejp (bj ) − eip (bi ). The explainer can use these formulas together with CS1 to determine the content necessary for a requested comparison. An explanation may clarify why an issue i has Δuis (b, b ) more utility for the user in b than in b. Alternatively, an explanation may clarify why, for bid b, issue i is assigned more utility than issue j. For example, the explanation may be: “Although issue j has Δwp (i, j) more weight than issue i, issue value bi has a higher evaluation score (Δep (bi , bj )) than issue value bj . When multiplied, this results in a higher issue utility for value bi of issue i.”

4

Conclusion

We presented an approach in which we analyzed the cooperation between user and NSS from the perspective of shared mental models. This analysis let us determine the elements that should be part of the shared mental model between user and NSS. This analysis also served to determine possible causes of discrepancies between mental models. Following this analysis, we focused on resolving discrepancies for one part of the shared mental model: preferences. We proposed using explanation, as it is a means to increase user understanding of the reasoning of the NSS, which helps detect and resolve discrepancies. We then presented a framework that provides a formal description of content selection for explanaining preferences. We focused on bilateral multi-issue negotiation, nevertheless, as the general idea of explaining preferences also applies to other types of negotiation, such as multilateral negotiation, our work should be extendable to such other types. Future work includes investigating how to present the explanation content (i.e., visualization techniques), a procedure to determine when to present which explanation to the user, implementation of the framework and user tests to validate it, and extending this work to other aspects of the shared mental model. Formalizing our approach of analyzing the problem domain in the light of shared mental models would allow it to be applied to other decision support domains. Also, as integration between explanation and argumentation research has been proposed in [8], it would be interesting to investigate how we might integrate our explanation framework, which focuses on the user-NSS interaction, with argumentation-based negotiation (e.g.,[2]), which is used for the interaction between user and opponent. Acknowledgements. This research is supported by the Dutch Technology Foundation STW, applied science division of NWO and the Technology

Explaining Negotiation: Obtaining a Shared Mental Model of Preferences

129

Program of the Ministry of Economic Affairs. It is part of the Pocket Negotiator project with grant number VICI-project 08075.

References 1. Harvard Business Essentials: Negotiation. Harvard Business School Publishing Corporation, Boston (2003) 2. Amgoud, L., Dimopoulos, Y., Moraitis, P.: A unified and general framework for argumentation-based negotiation. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 1–8. ACM Press, New York (2007) 3. Cannon-Bowers, J., Salas, E., Converse, S.: Shared mental models in expert decision making teams. In: Current Issues in Individual and Group Decision Making, pp. 221–246 (1993) 4. Hindriks, K., Jonker, C.: Creating human-machine synergy in negotiation support systems: towards the pocket negotiator. In: Proceedings of the 1st International Working Conference on Human Factors and Computational Models in Negotiation, pp. 47–54. ACM, New York (2008) 5. Lim, B., Klein, K.: Team mental models and team performance: A field study of the effects of team mental model similarity and accuracy. Journal of Organizational Behavior 27(4), 403–418 (2006) 6. Mastenbroek, W.: Negotiating as emotion management. Theory, Culture & Society 16(4) (1999) 7. Mathieu, E., Heffner, T., Goodwin, G., Salas, E., Cannon-Bowers, J.: The influence of shared mental models on team process and performance. The Journal of Applied Psychology 85(2), 273–283 (2000) 8. Moulin, B., Irandoust, H., B´elanger, M., Desbordes, G.: Explanation and argumentation capabilities: Towards the creation of more persuasive agents. Artificial Intelligence Review 17(3), 169–222 (2002) 9. Pu, P., Chen, L.: Trust-inspiring explanation interfaces for recommender systems. Knowledge-Based Systems 20(6), 542–556 (2007) 10. Roth-Berghofer, T., Richter, M.: On explanation. K¨ unstliche Intelligenz 22(2), 5–7 (2008) 11. Thompson, L.: The mind and heart of the negotiator. Pearson Prentice Hall, NJ (2005) 12. Tintarev, N., Masthoff, J.: A survey of explanations in recommender systems. In: ICDE 2007 Workshop on Recommender Systems and Intelligent User Interfaces, pp. 801–810 (2007) 13. Ye, L., Johnson, P.: The impact of explanation facilities on user acceptance of expert systems advice. MIS Quarterly 157–172 (1995)

A Computational Model of Habit Learning to Enable Ambient Support for Lifestyle Change Michel C.A. Klein, Nataliya Mogles, Jan Treur, and Arlette van Wissen VU University Amsterdam, Agent Systems Research Group De Boelelaan 1081, 1081HV Amsterdam, The Netherlands {michel.klein,nm.mogles,treur,wissen}@few.vu.nl

Abstract. Agent-based applications have the potential to assist humans in their lifestyle change, for instance eliminating addictive behaviours or adopting new healthy behaviours. In order to provide adequate support, agents should take into consideration the main mechanisms underlying behaviour formation and change. Within this process habits play a crucial role: automatic behaviours that are developed unconsciously and may persist without the presence of any goals. Inspired by elements from neurological literature, a computational model of habit formation and change was developed as a basis for support agents able to assist humans in lifestyle and behaviour change. Simulations are presented showing that the model exhibits realistic human-like behaviour. Keywords: habit learning, computational agent model, lifestyle change support.

1 Ambient Support for Lifestyle Change In Western societies health policy is directed at the reduction of medical costs by switching more and more from the treatment of diseases resulting from unhealthy lifestyle to promotion of healthy lifestyle habits [18]. Lifestyle change may comprise eliminating bad habits, for example addictive behaviours (e.g., smoking, alcohol or drugs use), and may simultaneously be directed at adopting new healthy habits, such as dieting and increasing physical activity (e.g., [13], [20], [24], [29], [30]). Considering the fact that lifestyle change requires intensive support, monitoring and supervision (e.g., [25]), the potential of smart ambient applications that assist humans in their daily life is substantial, as they allow for constant monitoring and instant feedback. In order to provide adequate support for humans, these support applications should be able to reason about the main determinants of human behaviour and the mechanisms underlying behaviour change. Apart from conscious goals and decisions, human behaviour is often based on habits – automatic behaviours that can be developed and maintained unconsciously. Habits may persist without the presence of any clear and definite goals and are very difficult to overcome. The model for habit learning and change presented In this paper can be used as basis for ambient intelligence applications to support lifestyle change. Using the model, an ambient application can predict – given a certain context or cue – the behaviour of a person with already formed habits. In addition, the system can K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 130–142, 2011. © Springer-Verlag Berlin Heidelberg 2011

A Computational Model of Habit Learning to Enable Ambient Support

131

exploit the model to reason about required changes in the context or goals that need more attention in order to form new habits or get rid of old ones. The proposed computational model of habit learning was inspired by elements from the neurological literature on habit learning (e.g., [1], [8], [10], [31]), and neural plasticity, such as Hebbian learning (e.g., [3], [11]), and adopts such adaptive mechanisms. The model has been formally specified in an executable manner, in order to conduct experiments and to allow the model to be embedded in an intelligent software agent that can support humans in their lifestyle and behaviour change. This paper is organized as follows. Section 2 addresses some background information on habit learning and change and the neural mechanisms underlying these processes. In Section 3 the description of the model is presented, Section 4 demonstrates some simulation results. Automated verification of the model is presented in Section 5, and finally, Section 6 contains discussion on the topic.

2 Background on Habit Learning Habits are learned dispositions to repeat past responses (cf. [31], p. 843), which by themselves were goal-driven. Once habits have been acquired, they are triggered by the cues in a certain context that co-occurred frequently with past performance, and which activate habitual responses directly, without the mediation of goals. These cues can be locations, objects, sequence of actions or presence of particular persons during or preceding the action performance. Habits formation corresponds thus to a contextresponse learning that is acquired slowly with experience ([31], p. 844). Behaviourists described habits as behaviour as creation of connections between stimulus and a particular response (e.g., [26], [28]). The cognitivist perspective on human behaviour suggests the existence of a central executive controller of behaviour (e.g., [21]). Nowadays, neurological literature describes the mechanisms underlying habit formation, which explain the behaviourists’ stimulus-response-based learning phenomenon, and introduces the concept of neural plasticity. Learning occurs due to the change of the connections strengths, for example, based on a principle known as Hebbian learning (e.g., [3], [11], [16]). It states that if two or more neurons are coactivated, the connections between these neurons strengthen. For example, repeated action in a certain context results to the gradual strengthening of the connection between the context representation and this particular response. These associations are difficult to override, though it is possible to influence habits (indirectly) via the activation of new goals. Strong goals that aim to direct one’s behaviour are associated with activation in the prefrontal cortex. This activation can inhibit the activation of subcortical structures (e.g., basal ganglia and cerebellum), associated with habitual behavior (e.g., [1], [8], [19], [27], [28]). Thus, when habits and goals are both present to guide action, they interact such that under some circumstances humans respond habitually and under other they exert regulatory control to inhibit the cued response. Although a habit is no longer goal-mediated, it can be regulated by post-hoc goal inference or cue control, for example, by 1) inhibiting the performance of responses, 2) drawing one’s attention to the undesired behavior, 3) associating the learned context with multiple responses or 4) altering exposure to the cues in the context [31].

132

M.C.A. Klein et al.

Summarising, from neurological literature such as [1], [8], [10], [15], [19], [23], [27], [32], [36] the following characteristics of habit learning have been identified: 1. Under repeated occurrence of cues and under influence of goal-directed behaviour leading to satisfaction, habits are developed. 2. When a habit has developed, the behaviour will also occur without the presence of a goal, when the cue is present. 3. A developed habit will persist when the relevant goal is present, also in absence of the cue. 4. When a habit was developed based on a goal, and this goal is changed to another (competitive) goal, then the habit can change to a new habit. These patterns have served as requirements for the design of the adaptive computational model described in Section 3. The patterns themselves will be formalized in Section 4 and checked for simulation traces of the computational model.

3 The Computational Model for Habit Learning The structure of the computational model presented in this section is based on the literature described in the previous section. The model is at a cognitive level, which still reflects the underlying neurological concepts, but without taking into account too many neurological details. It uses temporal relationships to describe the mechanisms at work. An overview of the model is depicted in Fig. 1. It enables two alternative ways (paths) in which behaviour can be generated. The first is by the activation of a long term goal (e.g., loose weight), a short term goal corresponding to this long term goal (reduce consumption of high calorie food), generation of an intention (able to achieve the goals), and finally execution of this intended action. The second path goes directly via cue activation in a certain context to the activation of a particular intention that leads to the action execution. This path corresponds to the habit, which is learned over time: the connection between cue and intention changes dynamically after their simultaneous activation according to the Hebbian learning principle. In the model also the influence of feeling on the chosen action has been incorporated: frequent execution of a particular action provides a reinforcement by the feeling of satisfaction after the performed action, and this feeling leads in turn to the higher activation of the intention related to this action. For example, a positive feeling of satisfaction resulting from the consumption of delicious cookies will lead to higher activation of the intention of eating these cookies. The model allows for multiple goals and intentions that result in behaviour. In principle each long term goal has connections with different strengths to short term goals, and the same holds for cues. The dynamical relationships below describe the model in semi-formal form and in a formal temporal relation notation in LEADSTO (cf. [6]). Within LEADSTO a dy→ b denotes that when a state property a (or namic property or temporal relation a → conjunction thereof) occurs, then after a certain time delay, state property b will occur. Below, this delay will be taken as a uniform time step Δt. The first dynamic

A Computational Model of Habit Learning to Enable Ambient Support

133

relationship addresses the Hebbian learning principle applied for the connections between cues en intentions, as also described in ([11], p. 406). LP1 Cue-intention connection adaptation If and and then

relevant cue C with level V1 occurs and intention I has value V2 learning rate from cue C to intention I is η and extinction rate from cue C to intention I is ζ the connection strength between cue C and intention I is w1 after ∆t the connection from cue C to intention I will have strength w1 + (η*V1*V2( 1 - w1) - ξ *w1)* ∆t

cue(C, V1) & intention(I, V2) & learning_rate(C, I, η) & extinction_rate(C, I, ζ) & connection_strength(C, I, w1)

→ → connection_strength(C, I, w1 + (η*V1*V2( 1 - w1) - ζ *w1)* ∆t)

Fig. 1. Computational model for habit learning: overview

134

M.C.A. Klein et al.

The following relationship specifies how activations of short term goals are determined based on long term goals and cues. LP2 If and and and and

Short term goal from cue and long term goals relevant cue C with level V0 occurs, long term goal LG1 has value V1 … and long term goal LGn has value Vn the connection strength between cue C and short term goal SG is w0 the connection strength between long term goal LG1 and short term goal SG is w1 the connection strength between long term goal LG2 and short term goal SG is w2 … and the connection strength between long term goal LGn and short term goal SG is wn and short term goal SG1 has value V3 then short term goal SG1 after ∆t will have level V3 + α(g(σ1,τ1, V0, V1 , V2, …, Vn, w0, w1, w2, …, wn) – V3) ∆t cue(C, V0) & ltgoal(LG1, V1) & … ltgoal(LGn, Vn) & connection_strength(C, SG, w0) & connection_strength(LG1, SG, w1) & … connection_strength(LGn, SG, wn) & stgoal(SG1, V3)

→ → stgoal(SG1, V3 + α(g(σ1, τ1, V0, V1 , V2, Vn, w0, w1, w2, wn) – V3) ∆t)

Here α is a speed parameter that defines the impact of long term goals and context cues upon the new activation value of the short term goal. Moreover, g is a combination function for which various choices are possible; a logistic threshold function has been chosen: g(σ, τ, V0, …, Vn, w0, …, wn) = th(σ, τ, w0V0+ … + wnVn) with th(σ, τ, V) = 1/(1+ e - 4σ (V- τ)) Parameters σ and τ define steepness and threshold values of the function. The threshold function ensures that the value of the goal is most often either close to zero or close to one. Only when the input for the threshold function is close to the threshold value itself, the values of the goal are somewhere between 0 and 1. In all subsequent formulae the combination function g is always based on a threshold function of this form. The third relationship of the model describes how intentions are determined. Intentions depend on short term goals and cues, and the feelings of satisfaction for both short and long term goals. Moreover, different intentions also affect each other by a form of mutual inhibition. Note that for the sake of simplicity in LP3 only two long term and short term goals are considered. LP3 If and and and and and and and and and and and and and and

Intention dynamics short term goal satisfaction SGSAT1 has value V1 long term goal satisfaction LGSAT1 has value V2 short term goal satisfaction SGSAT2 has value V3 long term goal satisfaction LGSAT2 has value V4 relevant short term goal SG1 has value V5 relevant cue C has value V6 intention I1 that corresponds to these goals has value V7 intention In has value V8 the connection strength between intention I1 and intention In is w1 the connection strength between short term goal SG1 and intention I1 is w3 the connection strength between cue C and intention I1 is w4 the connection strength between short term goal satisfaction SGSAT1 and intention I1 is w5 the connection strength between long term goal satisfaction LGSAT1 and intention I1 is w6 the connection strength between short term goal satisfaction SGSAT2 and intention I1 is w7 the connection strength between long term goal satisfaction LGSAT2 and intention I1 is w8

A Computational Model of Habit Learning to Enable Ambient Support

135

then intention I1 that corresponds to these goals after ∆t will have value V7 + β (g((σ2,τ2, V8, V9, V1, V2 , V3, V4, V5, V6, w 3, w 4, w 5, w 6, w 7, w 8, w 1 ) – V7) * ∆t stg_satisfaction(SGSAT1, V1) & ltg_satisfaction(LGSAT1, V2) & stg_satisfaction(SGSAT2, V3) & ltg_satisfaction(LGSAT2, V4) & stgoal(SG1, V5) & cue(C, V6) & intention(I1, V7) & intention(In, V8) & connection_strength(I1, In, w1) & connection_strength(SG1, I1, w3) & connection_strength(C, I1, w4) & connection_strength(SGSAT1, I1, w5) & connection_strength(LGSAT1, I1, w6) & connection_strength(SGSAT2, I1, w7) & connection_strength(LGSAT2, I1, w8)

→ → intention(I1, V7 + β (g(σ2, τ2, V8, V9, V1, V2 , V3, V4, V5, V6, w3, w4, w5, w6, w7, w8, w1 ) – V7) ∆t)

Here β is a parameter that defines the impact of inhibition of other intentions, and the feeling of satisfaction from the performed actions upon the intention to perform new actions. Weight w1 is negative here as it defines inhibition from the alternative competing intention(s). It is assumed that different intentions are conflicting, in other words one cannot perform two behaviour simultaneously to satisfy different goals; for this reason the weights between the intentions are always negative, or inhibitory. The step from intention to behaviour has been kept simple: LP4 From intention to behaviour If intention I with level V occurs, and then behaviour with level V will occur intention(I, V) & V> threshold

V> threshold

→ → behaviour(B, V)

The feeling of satisfaction for a long term goal was modelled as follows: LP5 If and and and and then

Long term goal satisfaction intention I1 has value V2 behaviour B1 with level V1 occurs and long term goal LG corresponding to this behaviour has value V3 long term goal satisfaction LTSAT has value V4 connection strength from behaviour B1 to the long term goal satisfaction LTSAT is w1 connection strength from intention I1 to long term goal satisfaction LTSAT is w2 long term goal satisfaction LTSAT after ∆t will be V4 + θ (f((σ3,τ3, V3 , V1, V2, w1, w2) – V4) * ∆t

behaviour(B1, V1) & intention(I1, V2) & ltgoal(LG, V3) & ltg_satisfaction(LTSAT, V4) & connection_strength(B1, LTSAT, w1) & connection_strength(I1, LTSAT, w2)

→ → ltg_satisfaction(LTSAT, V4 + θ (f(σ3, τ3, V3 , V1, V2, w1, w2) – V4) * ∆t)

Here parameter θ defines the impact of a long term goal, behaviours and intentions upon the long term goal satisfaction. The feeling of satisfaction for a short term goal was modelled in a similar manner: LP6 If and and and and then

Short term goal satisfaction intention I1 has value V2 behaviour B1 with level V1 occurs and short term goal SG corresponding to this behaviour has value V3 short term goal satisfaction STSAT has value V4 connection strength from behaviour B1 to the short term goal satisfaction STSAT is w1 connection strength from intention I1 to short term goal satisfaction STSAT is w2 short term goal satisfaction STSAT after ∆t will be V4 + θ (f((σ4,τ4, V3 , V1, V2, w1, w2) – V4) ∆t

behaviour(B1, V1) & intention(I1, V2) & stgoal(SG, V3) & stg_satisfaction(STSAT, V4) & connection_strength(B1, STSAT, w1) & connection_strength(I1, STSAT, w2)

→ → stg_satisfaction(STSAT, V4 + θ (f(σ4, τ4, V3 , V1, V2, w1, w2) – V4) * ∆t)

136

M.C.A. Klein et al.

4 Simulation and Verification The cognitive computational model described in the previous section was implemented in the Matlab environment. A number of simulations of 50 and 200 time steps have been performed. For the sake of simplicity only two initial long term goals and the corresponding behaviours were assumed. In this section four example simulation runs of 50 time steps are presented. These simulations illustrate the ability of the computational model to exhibit important patterns of habit learning and change. In Table 1 the values are shown used for learning and extinction rate, steepness and threshold values, speed factors, and connection weights (note that weight values for interaction between two options are symmetric). In order to investigate whether the computational model indeed learns and behaves according to what is expected, some logical properties (requirements) have been identified, formalized, and verified against the simulation traces of the model (see also the characteristics informally described at the end of Section 2). In this section, first the language used to express such properties is briefly introduced, followed by the specification of the actual properties, presentation of an example trace illustrating the pattern, and the result of their verification. Table 1. Parameter and connection weight values used α η ζ 0.5 0.01 0.8 connection cue-intention1 initially cue-intention2 initially cue-stgoal1 cue-stgoal2 ltgoal1-stgoal1 ltgoal1-stgoal2 ltgoal2-stgoal1 ltgoal2-stgoal2 ltsat1- intention1 ltsat1- intention2 stsat1- intention1

σ1 15

weight 0.1 0.1 0.1 0.1 0.9 0.2 0.2 0.9 0.9 0.2 0.9

σ2 τ1 β 0.9 0.8 20

σ3 τ2 thr θ 0.5 0.5 0.8 15

connection stsat1- intention2 intention1-behaviour1 intention2-behaviour2 intention1-intention2 intention2-intention1 intention1-ltsat1 behaviour1-ltsat1 intention2-ltsat1 intention2-stsat1 intention1-ltsat2 intention1-stsat2

weight 0.2 1 1 -0.9 -0.9 0.9 0.9 0.1 0.1 0.1 0.1

τ3 σ4 0.2 15

τ4 0.6

connection intention1-stsat1 behaviour1-stsat1 intention2-ltsat1 behaviour2-ltsat1 intention2-stsat1 behaviour2-stsat1 behaviour1- ltsat2 behaviour1-stsat2 behaviour2- ltsat2 behaviour2- stsat2

weight 0.9 0.9 0.2 0.1 0.2 0.1 0.1 0.1 0.9 0.9

Formal specification of desired properties of the computational model enables automatic verification of them against simulation traces. This was performed using the hybrid language TTL and its software environment [5]. TTL is built on atoms referring to states of the world, time points and traces, i.e. trajectories of states over time. Dynamic properties are temporal statements formulated with respect to traces in the following manner. Given a trace γ, the state in γ at time point t is denoted by state(γ, t). These states are related to state properties via the infix predicate |=, where state(γ, t) |= p denotes that state property p holds in trace γ at time t. Based on these

A Computational Model of Habit Learning to Enable Ambient Support

137

statements, dynamic properties are formulated in a sorted first-order predicate logic, using quantifiers over time and traces and the usual first-order logical connectives such as ¬, ∧, ∨, ⇒, ∀, ∃. For more details on TTL, see [5]. Each of the three subsections addresses one scenario. In the figures that demonstrate the simulation results, time is depicted on the horizontal axis and the activation values of the variables of interest are depicted on the vertical axis.1 4.1 Habit Formation In this simulation a specific behaviour is generated by a strong long term goal related to this behaviour in the presence of a strong cue. As a result even after a decrease of the value of the goal corresponding to this behaviour after time point 24, the behaviour persists up to end of the simulation; see Fig. 2. The value of the second long term goal is kept low during the whole simulation; therefore the second type of behaviour that corresponds to this goal does not come to expression. To verify this pattern formally, it first has to be checked whether a specific behaviour results from the presence of a high-level goal and a cue: P0: Long-term goal and cue leads to behaviour If a cue and a high-level goal are present for a certain time duration MIN_DURATION, then at some later time the corresponding behaviour will be present. ∀γ:TRACE, t:TIME [ habit_learning_phase(γ:TRACE, t:TIME, MIN_DURATION, ACT_VALUE2) ⇒ ∃t2:TIME > t , R3:REAL

state(γ, t2) |= has_value(beh1,R3) & R3 > ACT_VALUE2 ]

Here (and in the other properties below) the following abbreviation is used: habit_learning_phase(γ:TRACE, t:TIME, MIN_DURATION:INTEGER, ACT_VALUE:REAL) ≡ ∀t2:TIME > t & t2 < t + MIN_DURATION [ ∃R1:REAL state(γ, t2) |= has_value(ltg1, R1) & R1 > ACT_VALUE2 & ∃R2:REAL state(γ, t2) |= has_value(cue1, R2) & R2 > ACT_VALUE2 ] 2

2

behavior 1 cue behavior 2 long term goal 1

1.8 1.6

1.6

1.4

1.4

1.2

1.2

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

0

5

10

15

20

25 time

30

35

40

45

Fig. 2. Habit formation and persistence

1

behavior 1 cue behavior 2 long term goal 1

1.8

50

0

0

5

10

15

20

25 time

30

35

40

45

50

Fig. 3. Habit formation and cue removal

Note that gradual values for intentions, cues and goals are assumed, which represent the strength of the presence of these variables as perceived by the subject.

138

M.C.A. Klein et al.

Property P0 corresponds to characteristic pattern 1 of habits as defined at the end of Seciton 2, and has been automatically checked and proven to be true for the following values of the constants: MIN_DUR = 2, MAX_LEVEL_P1 = 0.5, ACT_VALUE1 = 0.5, ACT_VALUE2 = 0.7. All properties described in the remainder of this section have been automatically verified and found satisfied for these values. The pattern of habit formation itself (characteristic pattern 2) was specified as follows: P1: Habit persistence If a cue and a high-level goal have been present for some time period MIN_DURATION, the behaviour will exist in the presence of a cue even if the goal is no longer present. ∀γ:TRACE, t:TIME [ habit_learning_phase(γ:TRACE, t:TIME, MIN_DURATION, ACT_VALUE2) & ∀t2:TIME > t + MIN_DURATION, R3: REAL [ state(γ, t2) |= has_value(ltg1, R3) ⇒ R3 < ACT_VALUE1 ]] ⇒ [∀t3:TIME > t2, R4: REAL ⇒

state(γ, t3) |= has_value(cue1, R4) & R4 > ACT_VALUE2

∃t4:TIME>t3, R5:REAL state(γ, t4) |= has_value(beh1, R5) & R5 > ACT_VALUE2 ]]

When in the scenario in Fig. 3, after time point 26 the value of the cue is substantially decreased, habitual behaviour is not performed anymore from time point 31. As expected, and shown in Fig. 3, the second behaviour (‘behaviour 2’) does not occur. Formally, the illustrated characteristic is specified as follows. P2: Habit and cue removal If a habit is formed and the cue and the goal are no longer present, the behaviour will after some time cease to exist. ∀γ:TRACE, t:TIME [ [habit_learning_phase(γ, t, MIN_DURATION, ACT_VALUE2) & ∀t2:TIME > t + MIN_DURATION, R1, R2: REAL [ state(γ, t2) |= has_value(cue1,R1) ⇒ R1 < ACT_VALUE1 & state(γ, t2) |= has_value(ltg1, R2) ⇒ R2 < ACT_VALUE1 ] ⇒ ∃t3:TIME > t2, R3: REAL state(γ, t3) |= has_value(beh1,R3) & R3 < ACT_VALUE1 ] ]

4.2 Influence of Long Term Goal on Behaviour This scenario demonstrates how behaviour is influenced by goals in the absence of the learned cue; see Fig. 4. In the beginning habitual behaviour is formed: a strong cue is present and a behaviour pattern that coincides with the first long term goal. The value of the goal remains the same during the whole run, but the cue almost disappears after time point 24. The low value of the cue does not prevent the behaviour to occur due to the strong influence of the long term goal. This corresponds to characteristic habit pattern 3 from Section 2. This was specified as follows. P3: Habit and cue removal in presence of strong goal

If a habit is formed, the behaviour will still exist if the cue is not present any more and the high-level goal is present. ∀γ:TRACE, t:TIME, [ [habit_learning_phase(γ, t, MIN_DURATION, ACT_VALUE2) & ∀t2:TIME > t + MIN_DURATION, R3: REAL

[state(γ, t2) |= has_value(cue1, R3) ⇒ R3 < ACT_VALUE1 ]

⇒ [∀t3:TIME > t2, R4: REAL state(γ, t3) |= has_value(ltg1, R4) & R4 > ACT_VALUE2 ⇒ ∃t4:TIME>t3,R5:REAL state(γ, t4) |= has_value(beh1, R5) & R5 > ACT_VALUE2 ]]

A Computational Model of Habit Learning to Enable Ambient Support 2

2

behavior 1 cue behavior 2 long term goal 1

1.8 1.6

1.6 1.4

1.2

1.2

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

5

10

15

20

25 time

30

35

40

behavior 1 cue behavior 2 long term goal 1 long term goal 2

1.8

1.4

0

139

45

0

0

10

20

30

40

50

60

time

Fig. 4. Influence of the goal on behaviour in Fig. 5. Behaviours resulting from the goal the absence of the original cue change

4.3 The Effect of Goal Change In this simulation the result of the switch from one goal to another is demonstrated in the presence of a strong cue. As shown in Fig. 5, the habitual behaviour (‘behaviour 1’ in the picture) does not disappear immediately after adopting a new goal, conflicting with the previous one. It takes a little time to perform new behaviour pattern after the new goal has been adopted. This simulation demonstrates how the old undesired habitual behaviour can be substituted with the new positive behaviour. The formal specification of this pattern is: P4: New goal results in new habit If a habit is formed for long term goal ltg1, which disappears, a new behaviour will be developed if another long term goal ltg2 is present ∀γ:TRACE, t:TIME _DURATION [ [habit_learning_phase(γ, t, MIN_DURATION, ACT_VALUE2) & ∀t2 > t+MIN_DURATION, R1, R2, REAL

[ state(γ, t2) |= has_value(ltg1,R1) ⇒ R1 < ACT_VALUE1 ] &

[ state(γ, t2) |= has_value(ltg2,R2) ⇒ R2 > ACT_VALUE2 ] ⇒ ∃t3:TIME > t2 , R3, R4, REAL [ state(γ, t3) |= has_value(beh1,R3) & R3 < ACT_VALUE1 & [ state(γ, t3) |= has_value(beh2,R4) & R4 > ACT_VALUE2 ] ]

Fig. 5 also shows the effects of P1, which demonstrate that the new behaviour results in a habit after some amount of time: the behaviour persists even after the corresponding long term goal is no longer present. Combined, P1 and P4 account for characteristic habit pattern 4.

5 Discussion and Conclusions The cognitive computational model presented above can form the basis of an intelligent ambient support application. To this end, an agent based approach for creating ambient intelligence applications can be used [4]. Within such a framework, the ambient system consists of components, i.e., agents, that have context awareness about human behaviours and states, and (re)acts on these accordingly. For this purpose, the behaviour of the subject of the system (a person taken care of) relevant to the support

140

M.C.A. Klein et al.

provided should be explicitly described, e.g., via a computational model. If this is the case, an ambient agent can (re)act by undertaking actions in a knowledgeable manner that improve the human’s wellbeing and performance. Reasoning using an explicit model of the behaviour of a process is called modelbased reasoning [22]. Basically, there are two ways in which model-based reasoning on habits can be used within an intelligent support application. First, predictions can be made of what will happen given certain cues /contexts, long term goals and short term goals. For example, if the system has identified a specific behaviour – such as the eating of cookies at work – several times in the past, and it has knowledge about the short-term and long-term goals, it can predict whether a person in the workcontext will again eat a cookie. These predictions capabilities allow a support application to take action before an undesired habit actually took place. Second, the model can be used to perform analysis of the causes of the undesired behaviour and the effect of interventions on the behaviour of a person [9]. Causes of behaviour can be determined by backward abductive reasoning. For example, if an undesired behaviour is taking place, the presented computational model can be used to find hypothetical causes for this behaviour, for example a short term goal that leads to the intentions for the undesired behaviours. Symmetrically, forward deductive model-based reasoning derives the effect of interventions on the behaviour. For example, determining the effect of a different or more important long term goal after some time. This can be used by the ambient intelligence application to explicitly change the situation, e.g. removing cues, generating additional intention for long term goals leading to different behaviour, or suggesting actions to create new (more desired) habits. Existing models of habit learning take either the perspective of behaviourism that does not follow the internal mechanisms underlying habit development (e.g., [7], [17]) or propose the description of habit learning in a very detailed manner at the lowest neurological level (e.g., [2], [7], [14]). The proposed computational model is at a cognitive level, between the neurological and behavioural level. The proposed way of modeling is a manner to exploit within the computational modeling area principles from the neurological literature, by lifting neurological knowledge to a mental (cognitive/affective) level. In order to successfully model more complex and human-like behaviour, for example incorporating mutual cognitive/affective interactions, and adaptive behaviour, the modeler has to consider such numerical modeling techniques; see also [23]. In future work, the model will be deployed on actual data and used to improve habit performance. Also, the model could be improved by taking into account the environment in which a person is embedded, which is currently limited to perceiving cues, but preferably also incorporates socio-environmental factors shown to play a role in habit formation development (e.g., [12]).

References 1. Ashby, F.G., Turner, B.O., Horvitz, J.C.: Cortical and basal ganglia contributions to habit learning and automaticity. Trends Cogn. Science 14, 208–215 (2010) 2. Baldassarre, G.: A modular neural-network model of the basal ganglia’s role in learning and selecting motor behaviours. Cogn. Systems Research 3, 5–13 (2002)

A Computational Model of Habit Learning to Enable Ambient Support

141

3. Bi, G., Poo, M.: Synaptic modification by correlated activity: Hebb’s postulate revisited. Annu. Rev. Neurosci. 24, 139–166 (2001) 4. Bosse, T., Hoogendoorn, M., Klein, M.C.A., Treur, J.: An Agent-Based Generic Model for Human-Like Ambience. In: Mühlhäuser, M., Ferscha, A., Aitenbichler, E. (eds.) Constructing Ambient Intelligence: AmI 2007 Workshops Proceedings. Communications in Computer and Information Science (CCIS), vol. 11, pp. 93–103. Springer, Heidelberg (2008) 5. Bosse, T., Jonker, C.M., Meij, L., van der Sharpanskykh, A., Treur, J.: Specification and Verification of Dynamics in Agent Models. International Journal of Cooperative Information Systems 18, 167–193 (2009) 6. Bosse, T., Jonker, C.M., Meij, L., van der Treur, J.: A Language and Environment for Analysis of Dynamics by Simulation. International Journal of Artificial Intelligence Tools 16, 435–464 (2007) 7. Cohen, M.X., Frank, M.J.: Neurocomputational models of basal ganglia function in learning, memory and choice. Behavioural Brain Research 199, 141–156 (2009) 8. De Wit, S., et al.: Differential engagement of the ventromedial prefrontal cortex by goaldirected and habitual behaviour toward food pictures in humans. J. Neurosci. 29, 11330– 11338 (2009) 9. Duell, R., Hoogendoorn, M., Klein, M.C.A., Treur, J.: An Ambient Intelligent Agent Model using Controlled Model-Based Reasoning to Determine Causes and Remedies for Monitored Problems. In: Li, Y., Pasi, G., Zhang, C., Cercone, N., Cao, L. (eds.) Proceedings of the Second International Workshop on Human Aspects in Ambient Intelligence, HAI 2008, pp. 489–494. IEEE Computer Society Press, Los Alamitos (2008) 10. Everitt, B.J., Belin, D., Economidou, D., Pelloux, Y., Dalley, J.W., Robbins, T.W.: Neural mechanisms underlying the vulnerability to develop compulsive drug-seeking habits and addiction. Philos. Trans. R Soc. Lond. B Biol. Sci. 363, 3125–3135 (2008) 11. Gerstner, W., Kistler, W.M.: Mathematical formulations of Hebbian learning. Biol. Cybern. 87, 404–415 (2002) 12. Giles-Corti, B., Donovan, R.J.: The relative influence of individual, social and physical environment determinants of physical activity. Social Science & Medicine 54, 1793–1812 (2002) 13. Gollwitzer, P.M., Sheeran, P.: Implementation intentions and goal achievement: A metaanalysis of effects and processes. Advances in Exp. Social Psychology 38, 69–120 (2006) 14. Gurney, K., Prescott, T.J., Wickens, J.R., Redgrave, P.: Computational models of the basal ganglia: from robots to membranes. Trends Neurosci. 27, 453–459 (2004) 15. Haigh, K.Z., Kiff, L.M., Myers, J., Guralnik, V., Geib, W.C., Phelps, J., Wagner, T.: The Independent LifeStyle Assistant (I.L.S.A.): AI Lessons Learned. In: Proc. of the 16th Conference on Innovative Applications of AI, San Jose, California, pp. 852–857 (2004) 16. Lymberis, A., De Rossi, D.E.: Wearable eHealth Systems for Personalised Health Management: State of the Art and Future Challenges. IOS Press, Amsterdam (2004) 17. Machado, A.: Learning the temporal Dynamics of Behaviour. Psychological Review 104, 241–265 (1997) 18. Miller, E.K., Cohen, J.D.: An Integrative Theory of prefrontal Cortex Function. Annual Review of Neuroscience 24, 167–202 (2001) 19. Mowrer, O.H.: Learning Theory and the Symbolic Processes. Wiley, New York (1960) 20. Neal, D.T., Wood, W.: Automaticity in situ: Direct context cuing of habits in daily life. In: Morsella, E., Bargh, J.A., Gollwiter, P. (eds.) Oxford Handbook of Human Action, pp. 442–457. Oxford University Press, New York (2009)

142

M.C.A. Klein et al.

21. Oulette, E.A., Wood, W.: Habit and Intention in everyday life: the Multiple Processes by Which Past Predicts Future Behaviour. Psychological Bulletin 124, 54–74 (1998) 22. Pearl, J., Verma, T.S.: A theory of inferred causation. In: Principles of Knowledge Representation and Reasoning: Proc. Second Int. Conference (KR 1991), pp. 441–452 (1991) 23. Port, R., van Gelder, T.J.: Mind as Motion: Explorations in the Dynamics of Cognition. MIT Press, Cambridge (1995) 24. Quinn, J.A., Pascoe, A., Wood, W., Neal, D.T.: Can’t control yourself? Monitor those bad habits. Pers. Soc. Psychol. Bull. 36, 499–511 (2010) 25. Skinner, B.F.: The Behaviour of Organisms: An Experimental Analysis. AppletonCentury, New York (1938) 26. Tang, C., Pawlak, A.P., Prokopenko, V., West, M.O.: Changes in activity of the striatum during formation of a motor habit. Eur. J. Neurosci. 25, 1212–1227 (2007) 27. Watson, J.B.: Psychology as Behaviourist Views It. Psy. Review 20, 158–177 (1913) 28. Webb, T.L., Sheeran, P., Luszczynska, A.: Planning to break unwanted habits: Habit strength moderates implementation intention effects on behaviour change. British Journal of Social Psychology 48, 507–523 (2009) 29. Webb, T.L., Sheeran, P.: Mechanisms of implementation intention effects: The role of goal intentions, self-efficacy, and accessibility of plan components. British Journal of Social Psychology 47, 373–395 (2008) 30. Wood, W., Neal, D.T.: A New Look at Habits and the Habit-Goal Interface. Psychological Review 114, 843–863 (2007) 31. Yin, H.H., Knowlton, B.J.: The role of Basal ganglia in habit formation. Nature Reviews Neuroscience 7, 464–476 (2006)

An Intelligent Method to Extract Characters in Color Document with Highlight Regions* Chun-Ming Tsai Department of Computer Science, Taipei Municipal University of Education, No. 1, Ai-Kuo W. Road, Taipei 100, Taiwan [email protected], [email protected]

Abstract. Most conventional characters extraction methods include binarization (background determination), region segmentation, and region identification. Incorrect binarization results adversely influence the segmentation and identification results. This can be a problem when color documents are printed with different background color regions as the binarization will not have effective threshold results and subsequent segmentation and identification steps will not work properly. Conventional region segmentation methods are time-consuming for large document images. Conventional region identification methods are applied for the preceding segmentation results, using a bottom-up method. This study presents an intelligent method to solve these problems, which integrates background determination, region segmentation, and region identification to extract characters in color documents with highlight regions. The results demonstrate that the proposed method is more effective and efficient than other methods in terms of binarization results, extraction results, and computational performance. Keywords: Intelligent methods, character extraction, highlight regions.

1 Introduction Digital libraries (DL) are emerging as a supplement to traditional libraries [1]. The two main goals of DLs are (1) providing digital document images for users who wish to look up the original documents quickly, and (2) providing the digital contents of the documents for searching, reading, and printing easily. Color document image analysis (CDIA) [2], [3], [4] has been studied to convert paper contents to digital contents. The CDIA method can help traditional libraries to become digital libraries. The conventional methods for analyzing color document images have three basic steps: binarization (background determination), region segmentation, and region classification. Binarization includes global, region-based, local, and hybrid methods. Global thresholding methods find a threshold from the entire image information. The Otsu [5] method is a representative work. These global methods neglect spatial relationships among pixels [6] and scan the entire image. Global methods cannot threshold an image with low, middle, gradually decaying, or highlight background distribution. Region-based thresholding divides the image into k × k regions and *

This paper is supported by the National Science Council, R.O.C., under Grants NSC 99-2221E-133-002-.

K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 143–152, 2011. © Springer-Verlag Berlin Heidelberg 2011

144

C.-M. Tsai

decides how to threshold each region. Chou et al. [7] proposed learning rules to threshold document images. They divided a document image into 3 × 3 regions. Each region is thresholded by decision rules. How to determine the k value intelligently is critical to this method. Local thresholding methods such as Niblack [8] determine an individual threshold for each pixel according to the intensities of itself and its neighbor. Local methods for processing large size color documents are very timeconsuming. Hybrid thresholding methods used by Tseng and Lee [9] combine both global and local methods. They proposed a document image binarization method, which is a two-layer block extraction method. First layer dark foregrounds are extracted by locating connected-components. Second layer background intensities are determined and removed Hybrid methods have better experimental results, but are slower than global methods. The region segmentation includes both top-down and bottom-up methods. Topdown methods segment the whole image into small blocks. The method of recursive X-Y cutting (RXYC) [10] is a representative work. This method recursively makes cuts on the valleys of the projection profiles, obtained by scanning the image in horizontal and vertical directions alternately. An image requires at least two scans to determine threshold cut points. Bottom-up approaches start from document image pixels, and iteratively group them into larger regions. Fletcher and Kasturi [11] used connected component (CC) labeling to extract graphic and text objects from document images. The advantage of this approach is that it can extract italics characters. When document images include complex objects, such as non-rectangular images and various character font sizes, the approach segments images correctly. The time complexity of the CC based approaches is higher than top-down approaches. If documents contain large graphics, after the background determination and CC labeling, many small CCs will be produced, and will need time to merge them into a large graphic. Most traditional region classification methods are bottom-up methods. Lee and Ryu [12] first used CC labeling to extract bounding boxes from input binary images. Subsequently, horizontal and vertical projection methods are used to estimate the periodicity of each region. If a region is not a single periodical region, it would be split into two regions, repeating until the region becomes a single periodical region. After confirmation of the segmentation results, heuristic rules were used to classify segmented maximal homogeneous regions as texts, images, tables, or ruling lines. Lee et al. [13] extracted the bounding boxes of the CCs, then they used the projection profile to determine thirty-six region classification rules, based on geometric characteristics (and forty-four threshold values) of layout objects to facilitate region identification. Most traditional methods first used binarization method to threshold the document images. Then, they used top-down or bottom-up segmentation methods to extract CCs and projection profile was used to group the CCs. Finally, features were extracted. The knowledge or heuristic rules would be used and parameters set to identify the grouped objects into non-text or text objects. However, these methods are dependant on the effectiveness of binarization results. The segmentation and identification results could be adversely influenced. Further, binarization, region segmentation, and region identification methods operated independently; no one helps another. In particular, color documents are printed with different regions having different background colors. These methods cannot have effective results. Subsequent segmentation and identification steps will not work properly. The conventional region segmentation

An Intelligent Method to Extract Characters in Color Document with Highlight Regions

145

methods are time-consuming. An A4 color document image scanned in 300 dpi true color is approximately 25 MBytes. These processing methods will not be timeefficient. Also, the conventional region identification methods are applied to the preceding segmentation results. These methods are of a bottom-up type, and used the document model created from the top-down information. Thus, this study will present an intelligent method designed to solve abovementioned problems, which integrates background determination, region segmentation, and region identification to extract characters in color documents having different background colors.

2 Integrated Method for Extraction Characters The proposal method includes the following steps. First, RGB color space is transformed to gray level. Second, a border background determination is applied to obtain the background color in the border of the document image. Third, the restricted projection method is used to extract the regions in the color document image. Fourth, the preceding segmentation information and boundary analysis are applied to identify the characters and highlight regions in the extracted regions. Fifth, the border background determination is applied again for the highlight regions. Then the highlight regions are converted into binary images, and the characters in the highlight region are extracted. 2.1 Border Background Determination The layout in an article document is arranged by the background space surrounding printed regions. That is, the foreground objects, including text and graphics, are placed at the center of the document image. If the document border can be effectively determined, the document image background distribution can be found. Then, the foreground and the background in the document image can be separated using the background distribution as a reference. Herein, it is assumed that the background distribution of the color document is unimodal Gaussian distribution or changing gradually. The algorithm of the border background determination is as follows: 1. The color document image is transformed into the luminance space. 2. The algorithm of the scan line color differentiation [6] is applied to obtain the left, right, top, and bottom boundaries of the color document. 3. The histograms of the top, bottom, left, and right boundaries are combined to form the total histogram of the border background. 4. The total border histogram is smoothed to remove unreliable peaks by using the Gaussian smoothing filter [2]. 5. Major peaks are determined using the average differences approach [2]. 6. Small peaks are removed and the significant peaks are reserved [2]. 7. If the peak number is one, the background distribution is obtained by the low and high valleys (VLow, VHigh). These values are obtained by using modified triangle method [2]. The low bound (THRLow) and the high bound (THRHigh) of the background distribution are determined by following equations:

THRLow = VLow − σ × k

(1)

146

C.-M. Tsai

THRHigh = VHigh + σ × k

(2)

Note: σ is the standard deviation computed from the border pixels. k is a background constant value, which is used to adjust the low bound and the high bound of the background distribution. Herein, k is set as 2.0 for the purpose of our experiments. 8. If the peak number is two, the peak with the maximum population is the background distribution. The low bound (THRLow) and the high bound (THRHigh) of the background distribution are determined by using the same equations. 9. If the peak number is greater than two, gradually decaying detection [6] is applied to detect if the background is gradually changing. 10.If the background is not gradually changing, the peak with the maximum population is the background distribution. Low bound (THRLow) and high bound (THRHigh) of the background distribution are determined by using the same equations. 11.If the background is gradually changing, the thresholding algorithm [6] for gradually changing background is applied. 2.2 Regions Segmentation The object of region segmentation is to segment the region of interest (ROI) automatically. Usually, the extracted regions are displayed as either a rectangular bounding box or a polygon [3]. The simple method to segment the ROI is to locate the left-top and right-bottom coordinates for the ROI. Herein, a restricted background projection (RBP) algorithm is used to extract the ROI. This RBP algorithm includes horizontal and vertical directions background projection. In each direction, the background pixels in the scanning line are checked until the first foreground pixel is touched. The checking background pixels are summed to obtain the projection profile. A straight-scan is done to extract the disjoint regions. The straight-scan is defined as the pixels in one scanning line which all belongs to the background distribution. The RBP algorithm is described in more detail as follows: 1. The background distribution of the color document is determined. 2. In regard to horizontal restricted background projection (HRBP), the leftward restricted background projection is used to obtain the left background projection profile (LBPP). This LBPP is determined by checking the pixels in the scanning line from left to right until the first foreground pixel is found. The location of the preceding background pixel is recorded in the LBPP. 3. The rightward restricted background projection is used to obtain the right background projection profile (RBPP). It is determined by checking the pixels in the scanning line from right to the corresponding LBPP location until the first foreground pixel is found. The location of the preceding background pixel is the RBPP. 4. After the background projections are completed, the vertical disjoint regions can be extracted by locating the top, the left, and the bottom coordinates from the LBPP and the right coordinates from the RBPP. The consecutive non-straight-scan lines from top to bottom between the two straight-scan lines are grouped as a strip region. The straight-scan lines before and after the consecutive non-straight-scan lines are the top and bottom coordinates. The minimum x coordinates between the

An Intelligent Method to Extract Characters in Color Document with Highlight Regions

147

consecutive non-straight-scan lines of the LBPP is denoted as the left coordinates. The maximum x coordinate from top to bottom of the RBPP is the right coordinate. 5. In regard to vertical restricted background projection (VRBP), each strip region has applied downward and upward restricted background projections to obtain the top background projection profile (TBPP) and the bottom background projection profile (BBPP), similar to the leftward and the rightward projection. After VRBP is applied, many regions can be extracted. The left, right, and top coordinates of each region are obtained from TBPP, the bottom coordinates from BBPP. 2.3 Regions Identification Background and foreground regions are extracted by this approach. Usually, the foreground regions include primitive and composite regions [3]. Primitive regions include noises, vertical lines, horizontal lines, text characters, photographs, and highlight regions. A photograph is defined as a primitive region with large width, large height, which can be bounded by a rectangle. The highlight is defined as a primitive region with homogenous or gradual background color. Composite regions include text lines, text blocks, and columns. A text line is defined as characters aggregated in the vertical or horizontal directions. A text block is defined as the text lines or characters aggregated in the vertical or horizontal directions. A column is defined as a union of regions that do not have vertical straight-scans. These features are used to indentify above-mentioned regions: Horizontal region number (HRN) is the number of the extracted horizontal regions after applying the HRBP. If HRN is greater than one, the region can be divided into HRN horizontal regions. Otherwise, the region cannot be split. Horizontal height (HH) is the height of the extracted horizontal regions after applying the HRBP. Vertical region number (VRN) is the number of the extracted vertical regions after applying the VRBP. If VRN is greater than one, the region can be divided into VRN vertical regions. Otherwise, it cannot be split. Vertical width (VW) is the width of extracted vertical regions after applying the VRBP. Left variance (LV) is the left boundary variance of the extracted region obtained from the LBPP. Right variance (RV) is the right boundary variance as obtained from the RBPP. Top variance (TV) is the top boundary variance of the region obtained from the TBPP. Bottom variance (BV) is the bottom boundary variance of the extracted region obtained from the BBPP. A simple document model from the top-down model is applied to show the proposed identification method which is based on the decision tree shown in Fig. 1. 1.

2. 3. 4. 5.

The HRBP is applied to the color document image. If the HRN is equal to one, the color document image cannot be split in a horizontal direction. The VRBP is applied to obtain the VRN. The complex document model is not considered. If the HRN is greater than one, the color document image can be decomposed into horizontal regions from the horizontal direction. If the height of the horizontal region is greater than threshold value (TH1), this horizontal region belongs to a composite region. Proceed to step 10. If the height of the horizontal region is greater than threshold value (TH2), this region belongs to horizontal text line or highlight text line. Proceed to step 7. If the width of the horizontal region is greater than threshold value (TW1), this region belongs to horizontal line. Otherwise, the horizontal region is noise.

148

C.-M. Tsai

Fig. 1. Decision tree-based regional identification method

6. Applying the VRBP to the height is smaller than and equal to the threshold value (TH1) and is greater than the threshold value (TH2) of the horizontal region. 7. If the VRN is equal to one, this horizontal region cannot be split in the vertical direction. This region is a horizontal highlight text line. 8. If the VRN is greater than one, this horizontal region can be decomposed into characters from the horizontal direction. This region is a horizontal text line. 9. Applying the VRBP to the height is greater than the threshold value (TH1) of the horizontal region. 10. If the VRN is greater than one, the horizontal region can be decomposed into vertical regions in the vertical direction. This horizontal region is a composite region. (To simplify the problem, the complex document model is not considered.) 11. If the VRN is equal to one, the horizontal region cannot be split in the vertical direction, as it is a horizontal highlight text block or a photo. Highlight detection is applied to identify whether this horizontal region is a horizontal highlight text block. The highlight detection is based on LV, RV, TV, and BV. If LV, RV, TV, and BV of this horizontal region are all smaller than a pre-learning threshold value, this horizontal region is a horizontal highlight text block. Otherwise, the horizontal region is a photo. The parameters are derived from a learning phase that inputs training document images. In pattern classification research, a support vector machine (SVM) approach is used to find a hyper-plane that splits the training samples into two classes with the widest margin between them. The parameter of the threshold value one for horizontal height (TH1) is obtained as follows:

An Intelligent Method to Extract Characters in Color Document with Highlight Regions

1. 2. 3. 4.

149

The HRBP is applied to the training color document image. Each horizontal region is labeled as text line or non-text line. Height for horizontal text line and non-text line are calculated as training features. LIBSVM [14] is invoked to construct the decision function.

The proposed identification method establishes an interesting application of the SVM approach in the area of document image analysis. 2.4 Characters Extraction from Highlight Region There are many characters or text lines in the highlight text line or highlight text block, respectively. To extract characters in the highlight text line, the border background determination (Sect. 2.1) is applied to determine the background distribution of the highlight text line. The VRBP (Sect. 2.2) is used to extract characters in the highlight text line. For the highlight text block, the border background determination (Sect. 2.1) is applied again to determine the background distribution of the highlight text block. The HRBP (Sect. 2.2) is first applied to split the text block into horizontal text lines in the highlight. For each horizontal text line, the VRBP (see Sect. 2.2) is then used to extract the characters in the horizontal text line.

3 Experimental Results and Discussion The intelligent method was implemented as a Microsoft Visual C# 2008 Windowsbased application on an Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz Notebook. All test documents were scanned at 300 dpi. Results were saved as true color images. To illustrate thresholding results, the foreground objects are shown using “black” color and the background is shown by “white” color. Different color documents have been used, which include name cards, magazines, textbooks, and color receipts. Figure 2 shows the example for the proposed intelligent method. The original color document image is shown in Fig. 2(a), printed with a red background color. Two highlight regions are designed. One is printed using a black background color, the other with a yellow background color. After applying the border background determination (Sect. 2.1), the HRBP (Sect. 2.2) is used to segment the color document image. The LBPP and RBPP are obtained by using the leftward and the rightward restricted background projection, as shown in Fig. 1(b). The LBPP is white, the RBPP is black. The coordinates of the first horizontal highlight text line are shown in Fig. 2(b). After the HRBP and the decision tree-based identification, six regions were extracted, shown in Fig. 2(c). Region 1, a horizontal line, is drawn with a yellow rectangle. Regions 2, 3, 4, and 6 are horizontal text lines or highlight text lines, drawn with a green rectangle. Region 5 is a composite region, drawn with a brown rectangle. To discriminate between horizontal text line and horizontal highlight text line, the VRBP (Sect. 2.2) is applied. The split results are shown in Fig. 2(d). Region 2 is a horizontal highlight text line, drawn with a green rectangle. Regions 3, 4, and 6 are horizontal text lines, split into characters and delimited by a blue rectangle to differentiate Region 5, the composite region. The VRBP and identification rule in the decision tree are applied. Region 5 is a horizontal highlight text block, drawn with a black

150

C.-M. Tsai

rectangle and is shown in Fig. 2(d). To extract characters from Region 2, the border determination and the VRBP are applied again. These extraction characters are drawn with a blue rectangle and are shown in Fig. 2(e). Similarly, the characters in Region 5 are extracted by using border background determination, HRBP, and VRBP method, drawn with a blue rectangle and shown in Fig. 2(f).

Fig. 2. Example for the proposed intelligent method

Our binarization result is compared with three other methods: Otsu’s [5] global thresholding method, Niblack’s [8] local thresholding method and our previous border thresholding method [6]. Figure 3 shows four thresholding results for Fig. 2(a). Figures 3(a) – 3(d) show the result using our method, Otsu’s method, Niblack’s method (3 x 3), and (d) our previous work. Otsu’s method binarized Region 5 (Fig. 2(c)) correctly, but other regions are inversed, thresholding the gray regions well. The other conditions are not thresholded properly. From Fig. 3(c), the Niblack’s method binarized only Region 5 (Fig. 2(c)) correctly. Our previous work only binarized Region 3, 4, and 6 correctly. The proposed method is a significant improvement. The character extraction results obtained using our method, our previous work [3], Tseng and Lee [9], and Lee and Ryu [12] are shown in Figure 4. Fig. 4(b) shows our previous work can extract the characters in Regions 3, 4, and 6, drawn with blue. Region 1, a horizontal line, is yellow. The characters extraction results by [9] are shown in Fig. 4(c). This result extracted only the characters in Region 5 (Fig. 2(c)). Lee and Ryu cannot extract the characters from Fig. 2(a) because this approach used the CC labeling to obtain the bounding boxes and used projection profile to project the bounding boxes in identifying the segment results. However, when processing the color document image such as Fig. 2(a), the Lee and Ryu’s projection profile method extracted only one object. The proposed method can extract characters correctly.

An Intelligent Method to Extract Characters in Color Document with Highlight Regions

151

Fig. 3. The thresholding results for Fig. 2(a). This image binarized by: (a) our method, (b) Otsu’s method [5], (c) Niblack’s method [6], and (d) our previous work [8].

Fig. 4. The character extraction results for Fig. 2(a) by using (a) our method, (b) our previous work [3], (c) Tseng and Lee [9], and (d) Lee and Ryu [12]. Table 1. The computational performance of our proposed method, our previous work [3], Tseng and Lee [9], and Lee and Ryu [12] for Fig. 2(a)

Methods Ours Our previous work [4] Tseng and Lee [12] Lee and Ryu [16]

Average execution times 18956 (uS) 20110 (uS) 53732 (uS) 81873 (uS)

152

C.-M. Tsai

The computational performance of the methods is shown in Table 1. The average execution time of the Lee and Ryu method is four times that of our proposed and previous methods. Tseng and Lee method is three times that of our proposed method. The proposal method is considerably faster than the compared methods.

4 Conclusions We have presented an intelligent method to extract characters in color document images, which uses border background determination to determine the background distribution of the color image. The restricted background projection and decision tree-based identification are combined to segment regions and identify them. When the highlight region is identified, the border background determination and the restricted background projection are applied again to extract characters. Comparison to other methods demonstrates that the proposed method is more effective and time-efficient.

References 1. Smith, E.B., Monn, D., Veeramachaneni, H., Kise, K., Malizia, A., Todoran, L., El-Nasan, A., Ingold, R.: Reports of the DAS02 working groups. IJDAR 6, 211–217 (2004) 2. Tsai, C.M., Lee, H.J.: Binarization of Color Document Images via Luminance and Saturation Color Features. IEEE Transactions on IP 11(4), 434–451 (2002) 3. Tsai, C.M., Lee, H.J.: Efficiently Extracting and Classifying Objects for Analyzing Color Documents. online version in Machine Vision and Applications (2009) 4. Chen, Y.L., Wu, B.F.: A Multi-Plane Approach for Text Segmentation of Complex Document Images. PR 42(7), 1419–1444 (2009) 5. Otsu, N.: A thresholding selection method from gray-scale histogram. IEEE Trans. Systems, Men, and Cybernetics 9, 62–66 (1979) 6. Tsai, C.M.: An Efficient and Effective Background Determination for Color Document Images. In: ICMLC 2009, Baoding, Hebei, China, vol. 5, pp. 2857–2862 (2009) 7. Chou, C.H., Lin, W.H., Chang, F.: A binarization method with learning-built rules for document images produced by cameras. PR 43(4), 1518–1530 (2010) 8. Niblack, W.: An Introduction to Digital Image Processing, pp. 115–116. Prentice Hall, Englewood Cliffs (1986) 9. Tseng, Y.H., Lee, H.J.: Document Image Binarization by Two-Stage Block Extraction and Background Intensity Determination. PAA 11, 33–44 (2008) 10. Nagy, G., Seth, S., Viswanathan, M.: A Prototype Document Image Analysis System for Technical Journals. IEEE Computer 25(7), 10–22 (1992) 11. Fletcher, L.A., Kasturi, R.: A Robust Algorithm for Text String Separation from Mixed Text/Graphics Images. IEEE Transactions on PAMI 10, 910–918 (1988) 12. Lee, S.W., Ryu, D.S.: Parameter-Free Geometric Document Layout Analysis. IEEE Transactions on PAMI 23(11), 1240–1256 (2001) 13. Lee, K.H., Choy, Y.C., Cho, S.B.: Geometric Structure Analysis of Document Images: A Knowledge-Based Approach. IEEE Transactions PAMI 2(11), 1224–1240 (2000) 14. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001), Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm

Development of Technological System Structure for Threaded Connections Assembly under Conditions of Uncertainty Roman Chumakov Production Processes Management Department Chita State University 30 Alekzavodskaya, Chita, 672039, Russia Tel./Fax: +7 3022 352738 [email protected]

Abstract. This article presents an intelligent method to synthesize a technological system structure for threaded connections assembly under conditions of uncertainty. To accomplish the task, a fuzzy neural network was used. In the process of the neural system training, rules for optimal selection of assembly equipment were obtained. The optimal structure of the assembly system provides for high quality characteristics of the assembly process, as well as of the resulting threaded connections. Keywords: assembly, fasteners, neural network, structure.

1 Introduction In the assembly of threaded connections, wide use is made of modern high-production equipment, automation and intensification aids, and control systems to determine operating parameters for the assembly process. In spite of considerable progress in the machine building industry, however, mechanical assembly operations are still highly laborintensive. Threaded connections make up nearly 70% of all mechanical connections in industries worldwide [4], and involve 25-35% of all assembly labor. A firm may, therefore, suffer great financial loss if it fails to choose an adequate assembly strategy [6]. One of the major problems that present day assembly structure designers face is the development of a technological system structure. The development process brings about serious challenges for the specialist due to the complexities involved. The wide range of mounting hardware, existent technologies and equipment structured in a multidimensional technological space should yield an optimal realization variant. This variant should optimally accompany threaded connection design requirements as well as feasibility indices. The initial information in this case is uncertain. Traditional structural system modeling involves a mathematical apparatus of set and graph theory [2]. Usually the relationship between set elements is described as a binary relationship. This approach has a certain limitation which amounts to the impossibility of describing the fuzzy properties of the assembly system elements. K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 153–162, 2011. © Springer-Verlag Berlin Heidelberg 2011

154

R. Chumakov

Generalized systems and training methods, based on examples, are widely used for designing information support systems. One promising trend in the field of intelligent systems involves the integration of fuzzy logic with artificial neural networks (Neural Fuzzy Systems) [7]. Neural systems are capable of being learning based on a given number of examples. Their drawback is the impossibility of analyzing the logical conclusion process, i.e. they are a “black box” for the user [1, 3, 8]. Unlike neural systems, fuzzy logic methods are clear for the user from the point of view of the results obtained, but no generalized system can be made on their basis [9]. The purpose of the given work is to develop an intelligent method to form an optimal technological system structure for threaded connections assembly.

2 Structural Analysis of Technological Process of Threaded Connections Assembly Fig. 1 presents the structural chart of the technological system. However, structural presentation alone can’t make the classification of technical objects quite full. It is necessary to consider the following factors: structural, functional, geometrical, physical and mechanical. Drive Fulfilling

Carrier system

Protecting Basic

Loadaccumulating Feeding

Initiating D E V I C E S

P A R T S

А Positioning and transporting B Initiating Basic Protecting Fulfilling

Feeding Loadaccumulating

Drive

Fig. 1. Structure of the assembly automation technological system

Development of Technological System Structure for Threaded Connections Assembly

155

The technological structure of an assembly system depends on the number of initial conditions and requirements relating to the assembly process (Table 1), many of which are hard to formalize. Nowadays fuzzy sets are used to describe such systems. Table 1. Sets of technological conditions Set name

Symbol

Elements or boundary values

Thread size

S1

Type of threaded connection

S2

Basic part material Availability of elements for preliminary fastener insertion into the hole Basic part dimension and weight Basic part surface configuration Capacity Power input (energy consumption) Reliability Cost Matching precision of assembled parts axes Technological system rigidity System controllability Assembly system complexity Assembly unit dimension and mass

S3 S4

Small М3-М5; Medium М6-М12; Large М14-М20 With a gap (gap b/w screw threading and basic part threading); With tightness (tight); Thread-forming Fragile; Plastic Available; Not available

S5 S6 S7 S8 S9 S10 S11

Small; Medium; Large Simple; Complex Low; High Low; High Low; High Low; High Low; High

S12 S13 S14 S15

Low; High Low; High Low; High Low; High

Fuzzy sets are described by a membership function. One of the standard ways is the description of fuzzy sets using a Gauss function: ⎡ ⎛ x − c ⎞2 ⎤ μ Si ( x) = exp ⎢− ⎜ ⎟ ⎥ ⎢⎣ ⎝ σ ⎠ ⎥⎦

(1)

where Si is a technological condition; x is the value of a function (condition); c is the value of set under which the membership function possesses the peak value; and is the function range factor. Thus, mathematical description of such vague requirements as “Matching precision of assembled parts axes should be high” becomes possible. Fig.2 presents a membership function for the “Basic part dimension and weight” fuzzy set.

156

R. Chumakov

μ ( S5 ) 1 0.5 0 Small

Large Medium

S5

Fig. 2. Membership function for “Basic part dimension and weight” fuzzy set

The technological system of an assembly machine consists of two subsystems: a fastener subsystem (the stud connection variant will be considered later) and a basic part subsystem. There are several ways to supply assembly parts to the assembly position (Table 2). Stud connection has a distinguishing feature including the possibility to supply a fastener to the assembly position in two methods: either supply of a stud from a tray or supply of a chuck inserted with a stud. Table 3 presents the supporting assembly equipment. For the stud subsystem two kinds of accumulators were selected, primary and secondary ones. This is due to the fact that a fastener tends to go a complex course from the primary position to the assembly position. Table 2. Methods of supplying studs and parts to the assembly position STUD SUBSYSTEM Supply of a stud

Supply of a chuck inserted with a stud

1. Into the chuck (through the chuck)

1. Into the spindle

2. Into the chuck (on the outside of the chuck) 2. Spindle supplied to the chuck with a stud 3. From the chuck to the stud 4. Into the hole of the part (if a special adjusting element is available on the stud or bush) PART SUBSYSTEM Supply of a part 1. A part is supplied to the machine table / assembly industrial robot 2. A part is supplied with a fixture pallet to the machine table / assembly industrial robot

Development of Technological System Structure for Threaded Connections Assembly

157

Table 3. Support assembly equipment STUD SUBSYSTEM Primary accumulators 1.Magazine accumulator 2. Conveyor 3. Storage place

Secondary accumulators 1. Bunker with a chute 2. Magazine with studs 3. Pallet with studs 4. Magazine with tools 5. Pallet with chucks

PART SUBSYSTEM Accumulators 1. Loader 2. Conveyor with parts 3. Conveyor with fixture pallets

Transfer and supply equipment 1. Feeder 2. Machine manipulator (tool head) 3. Separate manipulator 4. Industrial robot 5. Robocar (trolley) 6. Pallet-loading table 7. Machine spindle Transfer and supply equipment 1. Separate manipulator 2. Industrial robot 3. Pallet-loading table

a10

(a) b2

b3

z2

b1

b4

(b) Fig. 3. Assembly process graph: (a) Stud subsystem; (b) Part subsystem

y1 y2

u4

u2

u3

u1

v6 v7

a8

v5

a7

z1

a9

a6

v6

a5

a3

v3

a4

a2

v4

a1

v1 v2 v3 v4

The assembly process graph is presented in Fig.3. This figure shows the interconnections between the ways to supply parts and assembly equipment. This kind of interconnection description may be viewed as certain for depicting a relational data model [9].

158

R. Chumakov

Equipment: a1 – magazine accumulator; a2 – conveyor; a3 – storage place; a4 – bunker with a chute; a5 – magazine with studs; a6 – pallet with studs; a7 – magazine with tools; a8 – pallet with chucks; a9 – part; a10 – spindle. elements a4, a5 and a6 are secondary stud accumulators (indicated by a dotted line). elements a7 and a8 are secondary accumulators of chucks with studs (indicated by a chain line). b1 – part loader; b2 – conveyor with parts; b3 – conveyor with fixture pallets; b4 – machine/robot table. v1 – feeder; v2 – machine manipulator (tool head); v3 – separate manipulator; v4 – industrial robot; v5 – robocar (trolley); v6 – pallet-loading table; v7 – machine spindle. Methods of supplying studs: u1 – into the chuck (through the chuck); u2 – into the chuck (on the outside of the chuck); u3 – from the chuck to the stud; u4 – into the hole of the part. Supply of a chuck inserted with a stud: y1 – into the spindle; y2 – spindle supplied to the chuck with a stud. Methods of supplying parts: z1 – a part is supplied to the machine table; z2 – a part is supplied with a fixture pallet to the machine table

Taking into consideration the fact that parts to be assembled may be delivered to the assembly position via different ways with the aid of different equipment, the number of possible variants of assembly process implementation amounts to several thousands. The high number of variations is the reason classical methods have failed to assist in choosing the optimal variant.

3 Fuzzy Model and Algorithm to Choose Optimal Assembly Process Structure Proceeding with the objective of this study, we determine the ways to supply parts and equipment to use in forming an assembly process structure. The task is accomplished in the following order. 1. A table is constructed indicating which technological conditions are attributed to which ways of supplying a part and assembly equipment (for stud and part subsystems, respectively). This table enables identification of significant parameters, thus diminishing the caliber of the task. Table 4. Membership table P2 P3 B1 B2 B3 B4 P1 S1 a11 … … … … S14 … aij S1…S14 – set of technological conditions (Table 1). Ways of supplying studs and parts to the assembly position (Table 2): Stud subsystem: P1 – way of supplying a stud, P2 – way of supplying a chuck inserted with a stud; Part subsystem: P3 – way of supplying a part. Supporting assembly equipment (Table 3): Stud subsystem: B1 – primary accumulators, B2 – secondary accumulators, B3 – transfer and supply equipment; Part subsystem: B4 – accumulators, B5 – transfer and supply equipment. aij – a numerical indicator of membership degree, range [0…1]

Development of Technological System Structure for Threaded Connections Assembly

159

2. On the basis of the significant parameters determined in Table 4, a learning set of rules is formed, e.g. for the supporting assembly equipment: Rulei: IF xi is Si1 AND … AND xj is Sij AND… AND xm is Sim, THEN y is Bi 3. An adaptive network-based fuzzy inference system (ANFIS) [5] is created. The number of input signals is determined in accordance with the number of significant parameters. The neural network structure is depicted in Fig.4. The ANFIS fuzzy neural network a hybrid neurofuzzy network. Its structure is isomorphic to a fuzzy knowledge database. The ANFIS comprises 5 layers: Layer 1. is intended for the fuzzification of variables x1…xN. Membership coefficient μ ((Sk )) ( xi ) is determined in accordance with the fuzzification function (1) for each koutput rule. Parameters and c are adapted in the training process. Layer 2. M is the number of neurons in the second layer. Each neuron in that layer corresponds to one fuzzy rule. A neuron in the second layer is connected to those neurons in the first layer forming the antecedents of the corresponding rule. Thus, each neuron in the second layer can accept from 1 up to N input signals. Weight is for the x input array. defined in this layer as

μ ((1)S ) ( x1 )

w1 = μ((1)S ) ( x1 )

w1*

w1* y1 ( x)

…

(m) μ ((k)S ) ( xn ) wm = μ( S ) ( xn )

wm*

y(x)

…

…

x1

xN

Layer:

1

2

3

wm* ym ( x) 4

5

Fig. 4. Structure of ANFIS fuzzy neural network

Layer 3. M is the number of neurons in the third layer. Each neuron in that layer estimates relative fuzzy rule weight ∑ Layer 4. M is also the number of neurons in the fourth layer. Each neuron in the fourth layer is connected to one neuron in the third layer and also with all networks inputs (not illustrated in Fig.4). The function is evaluated in this layer. The values of the function are multiplied by the relative degree of the rule implementation from the previous layer. Layer 5. The only neuron of this layer summarizes all contributions of the rules. 4. The neural network is learns on the basis of the learning set created beforehand.

160

R. Chumakov

5. To check the neural network a test set established. It contains examples which are not part of the learning set. In this manner, the neural network is checked for ability to make a logical conclusion based on the data the network had not been exposed to beforehand.

4 Neural Network for Choosing Optimal Assembly System Structure Let us regard choosing support assembly equipment, namely, accumulators for a part subsystem, as a case in point. We make a membership table (Table 5) for the technological conditions and the part accumulators. The following values are used to indicate membership degrees: 1 indicates a major condition which determines the choice of the given equipment; 0.8 implies that this is always considered, but does not determine the choice in the decision making; 0.4 means that this condition is indirectly related to the equipment choice; and the value of 0 is disregarded in the decision making. Table 5. Membership of technological conditions to support assembly equipment (part accumulators) Technological conditions (Table 1) Part accumulators (Table 3)

S1

S2

S3

S4

S5

S6

S7

S8

S9

S10

S11

S12

S13

S14

0

0

0

1

1

1

0.8

0.4

0.4

0.4

0

0.8

0.8

0.4

Natural language rules are formed for parameters with membership degree ≥ 0.8, for example: IF the dimensions and weight of the basic part are large AND the basic part surface configuration is complex AND the power input is above medium AND the system controllability is medium AND the assembly system complexity is medium THEN the part accumulator is a conveyor with fixture pallets. Each condition is set from 0 to 1, while the type of equipment is code marked. Table 6 presents some training set values which determine the choice of the part accumulator. Table 6. Some training set values x4

x5

Input vector x6 x7

x12

x13

Output signal

0 0 1 0 1 0 1 1 0 1 0 0.5 0.5 0.8 0.2 0.7 0.3 0.7 0.5 0 0.5 0.5 0.4 0.6 Output signal marking (code): 0 – loader; 0.5 – conveyor with parts; 1 – conveyor with fixture pallets

0 1 1 0.5

Development of Technological System Structure for Threaded Connections Assembly

161

Fuzzy neural network modeling is performed with the MATLAB® NN toolbox. The resulting neural network structure and test results are given in Fig.5. Ninety-six rules for choosing part accumulators were obtained. The neural network test results show that the network identifies with confidence an input vector which was not part of the learning set. The rest of the assembly equipment, as well as the way of supplying a part, are determined in a similar manner. To complete the technological system structure, seven further neural networks were constructed to match the parameter numbers in Table 4 (P1 – P4 and B1 – B4).

(a)

(b) Fig. 5. Neural network structure (a) and test result (b)

5 Conclusion Traditional methods of decision making have been successfully applied only for a limited number of tasks; usually these are rather simple tasks with a minimal number of estimated alternatives. They include estimation of every single alternative in

162

R. Chumakov

accordance with a certain criterion, a successive binary selection method, based on comparing pairs of separately taken alternatives. These methods can be formalized only if simple and investigated tasks and processes are fully unambiguous. Designing an assembly process for threaded connections constitutes a system with complex relations which admit no formal solution methods. Nevertheless, well-trained and competent specialists often make optimal selection upon decision making. Thus, the methods which combine a formal approach with intuitive experts’ knowledge (expert systems, fuzzy logic and neural networks) have been widely used. Unlike classical systems, the given methods provide for assembly system synthesis under conditions of uncertainty. To this end, a fuzzy neural network is used. It combines the advantages of fuzzy logic and a neural network. The ability of a neural network to generalize permits reduction of the volume of a problem, as typical examples for equipment selection and ways to supply parts are used in the process of neural network training. Fuzzy neural network training resulted in generation of certain rules for technological system structure synthesis depending on initial conditions and assembly technology requirements.

References 1. De Jesús, O., Hagan, M.T.: Backpropagation Algorithms for a Broad Class of Dynamic Networks. IEEE Transactions on Neural Networks 18(1), 14–27 (2007) 2. Gross, J., Yellen, J.: Graph theory and its applications. Chapman & Hall/CRC (2006) 3. Haykin, S.: Neural Networks: A Comprehensive Foundation. Prentice Hall Press, NY (1998) 4. Hellmann, K.-H.: Entwicklung der Schraubtechnik. VDI Zeitschrift 131(5), s.89–s.93 (1989) 5. Jang, J.-S.R.: ANFIS: Adaptive Network-based Fuzzy Inference System. IEEE Trans. on Syst., Man and Cybernetics 23(3), 665–685 (1993) 6. John, R., Jones, S.: The economic Relevance of materials joining technology. Welding World 39(3), 145–153 (1997) 7. Lin, C.-T., Lee, C.-C.: Neural Fuzzy Systems. A Neuro-Fuzzy Synergism to Intelligent Systems. Prentice-Hall, NY (1996) 8. Warwick, K., Irwin, G.W., Hunt, K.J.: Neural Networks for Control and Systems. Stevenage, U.K (1992) 9. Zha, X.F., Lim, S.Y.E., Fok, S.C.: Development of Expert System for Concurrent Product Design and Planning for Assembly. International Journal of Advanced Manufacturing Technology 15, 153–162 (1999)

Automatic Vehicle Identification by Plate Recognition for Intelligent Transportation System Applications Kaushik Deb1 , My Ha Le2 , Byung-Seok Woo2 , and Kang-Hyun Jo2 1

Dept. of CSE, Chittagong University of Engineering & Technology, Chittagong-4349, Bangladesh 2 Dept. of EE and Information Systems, University of Ulsan, Daehak road 100, Nam-gu, 680-749 Ulsan, South Korea [email protected], {lemyha, woo}@islab.ulsan.ac.kr, [email protected]

Abstract. Automatic vehicle identification is a very crucial and inevitable task in intelligent traffic systems. In this paper, initially, a HueSaturation-Intensity (HSI) color model is adopted to select automatically statistical threshold value for detecting candidate regions. The proposed method focuses are on the implementation of a method to detect candidate regions when vehicle bodies and license plate (LP) have similar color based on characteristics of color. Tilt correction in horizontal direction by the least square fitting with perpendicular offsets (LSFPO) is proposed and implemented for estimating rotation angle of the LP region. Then the whole image is rotated for tilt correction in horizontal direction by this angle. Tilt correction in vertical direction by reorientation of the titled LP candidate through inverse affine transformation is proposed and implemented for removing shear from the LP candidates. Finally, statistical based template matching technique is used for recognition of Korean plate characters. Various LP images are used with a variety of conditions to test the proposed method and results are presented to prove its effectiveness. Keywords: HSI color model, Tilt correction, Least square fitting with perpendicular offsets (LSFPO), Affine transformation, and template matching.

1

Introduction

With the rapid development of highway and the wide use of vehicle, people have started to pay more and more attention on the advanced, efficient and accurate intelligent transportation systems (ITSs). The task of recognizing specific object in an image is one of the most difficult topics in the field of computer vision or digital image processing. The vehicle license plate recognition (VLPR) task is quite challenging from vehicle images due to the view point changes, when vehicle bodies and LP have similar color, multi-style plate formats, and the nonuniform K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 163–172, 2011. c Springer-Verlag Berlin Heidelberg 2011 

164

K. Deb et al.

outdoor illumination conditions during image acquisition. In addition, vehicle license plate detection (VLPD) system should operate fast enough to be real time to satisfy the needs of ITS and not miss a single interest object from the vehicle image. The VLPR is used widely for detecting speeding cars, security control in restricted areas, unattended parking zones, traffic law enforcement, and electronic toll collection, etc. Recently, the necessity of vehicle license plate recognition (VLPR) has increased significantly. One of the major problems in LP detection, tilt correction and recognition is determining LP systems. This system must guarantee robust detection and recognition under various weather and lighting conditions, independent of orientation and scale of the plate. As far as detection, tilt correction and recognition of the license plate region are concerned, researchers have found various methods of correcting tilt, locating and recognizing license plate. For example, Karhunen-Loeve (K-L) transformation method has been introduced for correcting a VLP tilt in [1]. However, no explanation of extracting LP region has been given in the paper. A region-based LP detection method has been presented in [3], which first applies a mean shift procedure in a spatial-range domain to segment a color vehicle image in order to get LP regions. Fuzzy logic has been applied in detecting license plates in [4]. A template matching application is described in [6]. Where the character recognizer extracts some feature points and uses template matching operators to get a racy solution under multiple acquisition conditions. The emphasis of this paper is on the tilt correction in vertical direction by reorientation of the titled LP candidate through inverse affine transformation is proposed and implemented for removing shear from the LP candidates. In addition, the consolidation of a method to detect candidate regions when vehicle bodies and LPs have similar color. A common drawback of color-based VLPD is the failure to detect the boundaries or border of LPs. This occurs when vehicle and LPs have similar colors. To overcome the common drawback, in this paper, we propose and implement a method named as modified recursive labeling algorithm. Furthermore, we focus on this paper in on the conducting an experiment using template matching technique with the use of normalized cross-correlation to perform for recognition of Korean plate characters.

2

Proposed Framework

In the author’s previous work [2], projection and least square fitting with perpendicular offsets based vehicle license plate tilt correction was presented. We propose in this section, an enhanced version of the framework for VLP tilt correction and recognition as shown in Figure 1. To improve the traditional LP detection method, as license plates can appear at many different angles to the camera’s optical axis, each rectangular candidate region is rotated until they are all aligned in the same way before the candidate decomposition. For correcting a VLP tilt in vertical direction by reorientation of the titled LP candidate through inverse affine transformation is proposed and implemented for removing

Automatic Vehicle Identification by Plate Recognition

165

Fig. 1. The proposed vehicle license plate recognition framework

shear from the LP candidates. A common drawback of color-based VLPD is the failure to detect the boundaries or border of LPs. This occurs when vehicle and LP have similar colors. It is important to mention here that some previous researches [3] and [4] doesn’t solve this problem and they assert leave these issues to be considered in future study. To overcome this common drawback, we propose and implement a method named as modified recursive labeling algorithm. Finally, conducting an experiment using template matching technique with the use of normalized cross-correlation to perform for recognition of Korean plate characters.

3

Vehicle License Plate Recognition Module

In this section, the five primary stages of the proposed VLP recognition framework, i.e., color segmentation, labeling and filtering, correcting tilt in horizontal and vertical direction, decomposing candidates and recognizing LP alphanumeric character have been discussed in details. Color arrangement of the Korean LPs are well classified. A More detailed explanation for color arrangement and outline of the Korean VLPs could be found in [2]. 3.1

Color Segmentation

Color is a distinctive feature because the law decides its usage of the vehicle according to color. Representation of plate color in an invariant manner is of main objectives for our color-based LP detection method. In the proposed framework, input vehicle images are converted into a Hue-Saturation-Intensity (HSI) color images. Then the candidate regions are found by using HSI color space on the basis of using hue, saturation and/or intensity. Many applications use the HSI color model. Machine vision uses HSI color model in identifying the color of different objects. Plate color information is used to detect candidate regions in our experiments, and shape properties of LP allow reducing number of LP-like candidates. A More detailed explanation could be found in [2] for detecting green, yellow, and white license plate pixels. Color segmentation parameters are very sensitive in order to detect as much candidates as possible. All false candidates

166

K. Deb et al.

Fig. 2. Successful license plate identification sequence in an unmoving vehicle: (a) an LP images, when vehicle and LP have similar colors and also with uneven illumination condition, (b) color segmentation results, (c) detected candidate after implementation of morphological closing operation and filtering, and (d) extracted candidate after tilt correction in horizontal direction

will be filtered out on the next stages. Examples of proposed color segmentation method is depicted in Figure 2. 3.2

Labeling and Filtering

In the proposed method, a recursive algorithm is implemented for connected component labeling operation. A common drawback of color-based VLPD is the failure to detect the boundaries or border of LPs. This occurs when vehicles and LPs have similar colors. To overcome this common drawback, we proposed and implemented a new method named modified recursive labeling algorithm. If we investigate carefully, when vehicle bodies and LPs have a similar color, we can find there is little color differences between LPs and vehicle color. Based on this idea, we overcome this problem, by trying to find those color difference parameters. To label connected pixels 8-neighbors recursive algorithm was used. Furthermore, connected pixels were grouped if distance in color space was less then predefined threshold Dmin . Two connected pixels are grouped if Dist(Ii, j , Im, n ) < Dmin . Color distance between two connected pixels Ii,j , Im,n is calculated using equation 1. Dist (Ii,j , Im,n ) =



 k  k  Ii,j − Im,n

(1)

k={R,G,B} k={R,G,B}

k={R,G,B}

and Im,n are pixel belongs to red, green, and blue comwhere Ii,j ponents of input image I with neighbor coordinate (i, j) and (m, n), respectively. In this step, we extract candidate regions that may include LP regions from the binary mask obtained in the previous step. During this step, main geometrical properties of LP candidate such as area, bounding box, and aspect ratio are computed. A more detailed explanation could be found in [2]. These parameters are used in the filtering operation to eliminate LP-like objects from candidate list. Figure 2 portrays the steps for LP segmentation.

Automatic Vehicle Identification by Plate Recognition

3.3

167

Correcting tilt

In order to facilitate character segmentation and recognition in a VLPR system, the tilted LPs in the located image should be corrected in both the horizontal and vertical directions. Following the successful filtering operation of the vehicle image, in this paper, according to the LSFPO, the VLP candidate region is fitted to a straight line. After the line slope is obtained, tilt or rotation angle is estimated. Figure 3(a, b) depicts rotation angle α between the principal axis X  of and the the horizontal axis X of the tilt VLP region. Then, the whole image is rotated for tilt correction in horizontal direction by this angle. Correcting horizontal tilt by straight line fitting method based on LSFPO: The least square method is a commonly used fitting method. Fitting requires a parametric model that relates the response data to the predictor data with one or more coefficients. To obtain the coefficient estimates, the least square method minimizes the summed square of residuals. A more detailed explanation could be found in [2]. Correcting horizontal tilt by straight line fitting method based on LSFVO: Given a set of data points. It is desired to find the best fitting line from a given set of data points. In principle, deviation between data and fitting line should be minimized. A more detailed explanation could be found in [2]. After getting the fitting slope a. Let tan α = a, and get the tilt angle α. Rotate the entire image with α from centroid rectangular candidates region of LP image and perform the tilt correction. Figures 5(c1-d1) and 5(c2-d2) portrays a sequence of successful horizontal tilt correction by LSFPO and LSFVO, respectively. Determining the rotation angle using principal axis: Following the successful filtering operation of the image, measurements such as center of area and the axis of least second moment are employed to solve the rotation adjustment problem. A more detailed explanation could be found in [2]. Figures 5(c3) and 5(d3) portrays a sequence of successful horizontal tilt correction by the principal axis (PA) method. Correcting vertical tilt. The purpose of correcting tilt in vertical direction is to correct VLP shear left and right in horizontal axis X as shown in Figure 3(c, d). Vertical tilt correction is also essential to facilitate character

Fig. 3. VLP tilt image in horizontal direction: (a) tilt angle α > 0 and (b) tilt angle α < 0. VLP tilt image in vertical direction: (c) tilt angle θ > 0 and (d) tilt angle θ < 0.

168

K. Deb et al.

Fig. 4. Vehicle tilt correction in vertical direction: (a) Reorientation of 4 points by using inverse general affine transformation, (b) an LP image with titled and (c) shear correction by using inverse general affine transformations

segmentation and to recognize VLP character accurately. Tilt correction in vertical direction by reorientation of the tilted LP candidate through inverse affine transformation is proposed and implemented for removing shear from the LP candidates. Shear correction through general affine transformation: A general affine transformation (AT) from 2D to 2D as in (2) requires  six parameters and can  be computed from only 3 matching pairs of points ([xj , yj ], [uj , vj ])j=1, 3 . ⎤⎡ ⎤ ⎡ ⎤ ⎡ x u a11 a12 a13 ⎣ v ⎦ = ⎣ a21 a22 a23 ⎦ ⎣ y ⎦ (2) 1 0 0 1 1 We use Hough transformation for getting 4 vertices from candidate LP regions. Hough line is estimated from boundary points of candidate LP (i.e. contour) and vertices are detected by intersection of hough line. After determining the 4 points (vertices) from LP candidate, the inverse general affine transformation (A−1 ) is applied for rectifying candidate LP region as shown in Figure 4. VLP tilt correction in vertical direction by reorientation of the tilted LP candidate through inverse affine transformation is applied for rectifying candidate LP region and is depicted in Figure 5(d). 3.4

Decomposing Candidate

Information is extracted from the image by intensity histograms that play a basic role in image processing, in areas such as enhancement, segmentation and description. In this section, verification and detection of the VLP region as well as character segmentation are considered and discussed. Once the candidate area is binarized the next step is to extract the information. At first, regions without interest such as border or some small noisy regions are eliminated; the checking is made by height comparison with other plate characters height. Figure 6 shows the results for verifying predetermined alphanumeric characters. 3.5

Recognizing LP Alphanumeric Characters

The objects are to recognize alphanumeric characters, both numeric numbers and Korean letters, written on the license plate. In this paper, template matching

Automatic Vehicle Identification by Plate Recognition

169

Fig. 5. Illustration of license plate segmentation: (a) an LP image, (b) detected candidate after filtering, (c) finding best fitting line through LSFPO(c1), LSFVO(c2), and principal axis (c3), (d) extracted candidate after tilt correction

Fig. 6. Steps for verifying predetermined alphanumeric characters: (a) extracting the candidate region, (b) vertical position histogram with LP border, (c) view of normalization candidate region after removing border and noisy area, (d) vertical position histogram (seven peaks for predetermined seven alphanumeric characters in LP region), and (e) equal-sized candidate alphanumeric characters after extraction

method with the use of normalized cross-correlation to perform recognition for different types of alphanumeric characters used in Korean vehicle LP. Before recognition scheme, the characters are normalized. Normalization is to refine the characters into a block containing no extra white spaces (i.e. pixels) in all four sides of the characters. Then each character is fit to equal size as shown in Figure 6(e). Fitting approach is necessary for template matching. For matching the candidate characters with the template, input candidate images must be equal sized with the template characters. Here the characters are fit to 40×25. The extracted characters are cut from license plate region and the characters on database are now equal sized. The Korean license plates contain Korean letters and numbers. For recognition of plate numbers and letters, we use the database where containing 10 prototypes for the Korean plate numbers and 30 prototypes for the

170

K. Deb et al.

Fig. 7. Template used for pattern matching: (a) 10 prototypes for the Korean plate numbers and (b) 30 prototypes for the Korean plate characters

Fig. 8. Illustration of LP segmentation and alphanumeric characters recognition: (a) an LP image, (b) extracted candidate region, (c) equal-sized candidate alphanumeric characters and (d) candidate alphanumeric characters measuring the similarity by NCC

Korean plate characters with the size of 40×25. The database formed is shown in Figure 7. To measure the similarity and find the best match, a statistical method correlation is used. Correlation is an effective technique for image recognition. This method measures the correlation coefficient between a number of known images with the same sized unknown images or part of an image with the highest correlation coefficient between the images producing the best match. In our case, to detect the presence of a candidate character into the template or database, a cross-correlation function is utilized between a sub-area of the normalized template and each candidate character. The candidate plate character recognition process is based on the value of the correlation coefficient. If the value of the correlation coefficient execeeds a threshold set by the user, then the similarity measure is large enough and the input character can be assumed to present. Finally, a box on the target character is plotted as shown in Figure 8(d).

4

Experimental Results and Conclusions

All experiments were done on Pentium-IV 2.4 GHz with 1 GB RAM under MATLAB environment. Images of size 640*480 and 320*240 pixels were used. The image database consists of 200 digital images from different two groups. In order

Automatic Vehicle Identification by Plate Recognition

171

Table 1. Comparison of detection rates Reference number

Detection rate

5 80.4% Proposed framework 100%

to evaluate the proposed framework, two groups of experiment were conducted. First group was used to compare the proposed framework with mathematical morphology (MM) method of [5]. A comparison between the proposed method and a well-reported method in the literature is given in Table 1. From Table 1 it can be seen that the proposed method outperforms the method repot in [5] from the detection rate points of view. The second group contains 175 images. All images in that group represents South Korean license plates from the natural scenes obtained in the nonuniform outdoor illumination conditions, multi-style and color of license plates, and various angles of vision. They were taken in distance of 3 to 15 m. Under these conditions, the success of LP detection has reached to more than 96%. In tilt correction experiments, we compare the tilt performance results of LSFPO with those of LSFVO and principal axis method. A more detailed explanation could be found in [2]. The recognition results of Korean alphanumeric characters (KAC) and average computational time for recognition of single character are shown in Tables 2 and 3, respectively. Table 2. Recognition results Stage

Occurrences of KAC Number of successes

Recognition of KAC 646

634

Success rate 98.15%

Table 3. Average computational time for single character Type

Average computational time (s)

Number 1 Korean word/character 3

In conclusion, a method is presented in this paper for automatic vehicle identification by the recognition of vehicle license plate. In this proposed method, initially, a Hue-Saturation-Intensity (HSI) color model is adopted to select automatically statistical threshold value for detecting candidate regions. These candidate regions may include LP regions; geometrical properties of LP are then used for classification. In addition, the emphasis of this paper is on the implementation of a method to detect candidate regions when vehicle bodies have similar

172

K. Deb et al.

color. In the vehicle horizontal tilt correction process, three correction methods are implemented for comparing the tilt performance results. Analysis and simulation results suggest that LSFPO and principal axis method tilt correction are more precise than LSFVO. However, LSFPO outperforms than principal axis because of faster computational time, easily implemented and more precise tilt correction. In this paper, tilt correction in vertical direction by inverse affine transformation is proposed for removing shear from the LP candidates. Finally, we focus on this paper in on the conducting an experiment using template matching technique with the use of normalized cross-correlation to perform for recognition of Korean plate characters. Color arrangement and predetermined alphanumeric character of the Korean license plate are important features for verification and detection of license plate regions. While conducting the experiments, different view point, illumination conditions, and varied distances between vehicle and camera often occurred. In such cases, confirmed the result is very effective when the proposed method is used. However, the proposed method is sensitive with motion blur in the input image. We leave these issues for consideration in future studies. Acknowledgments. This research was supported by the MKE(The Ministry of Knowledge Economy), Korea, under the Human Resources Development Program for Convergence Robot Specialists support program supervised by the NIPA(National IT Industry Promotion Agency) (NIPA-2010-C7000-1001-0007)

References 1. Pan, M.-S., Xiong, Q., Yan, J.-B.: A new method for correcting vehicle license plate tilt. Int. J. of Automation and Computing 6(2), 210–216 (2009) 2. Deb, K., Vavilin, A., Kim, J.-W., Kim, T., Jo, K.-H.: Projection and Least Square Fitting with Perpendicular Offsets based Vehicle License Plate Tilt Correction. In: Proceedings of the Society of Instrument and Control Engineers, pp. 3291–3298 (2010) 3. Jia, W., Zhang, H., He, X.: Region-based License Plate Detection. J. Network and Comput. Applications 30(4), 1324–1333 (2007) 4. Chang, S.-L., Chen, L.-S., Chung, Y.-C., Chen, S.-W.: Automatic license plate recognition. IEEE Trans. Intell. Transp. Syst. 5(1), 42–53 (2004) 5. Martin, F., Garcia, M., Alba, J.L.: New Methods for Automatic Reading of VLP’s (Vehicle License Plates). In: Proceedings of the IASTED Int. Conf. on SPPRA (2002) 6. Comelli, P., Ferragina, P., Granieri, M.N., Stabile, F.: Optical recognition of motor vehicle license plates. IEEE Trans. Veh. Technol. 44(4), 790–799 (1995)

Intelligent Page Recommender Agents: Real-Time Content Delivery for Articles and Pages Related to Similar Topics Robin M.E. Swezey, Shun Shiramatsu, Tadachika Ozono, and Toramatsu Shintani Graduate School of Engineering, Nagoya Institute of Technology Gokiso-cho, Showa-ku, Nagoya, Aichi, 466-8555 Japan {robin,siramatu,ozono,tora}@toralab.ics.nitech.ac.jp

Abstract. In this paper, we present an architecture and sample implementation of a system which allows us to push latest up-to-date related contents to any Web news article or page in real-time. The architecture makes use of page Agents which recommend the contents and are persistent as well as synchronized over all page instances in browsers. The Agents are easy to incorporate on any Web page and make use of state-of-the-art Web technology. In our sample implementation, we show how our Agents, coupled with a Complementary Naive Bayes classifier, can recommend latest contents related to 47 Japanese prefectures and over 1700 Japanese cities. We show performance results and conclude on further research to improve the affiliate and user experience on the Web.

1

Introduction

In a previous paper [1], we proposed an architecture for a Multi-Agent System which would assist bloggers and webmasters of topic-oriented content sites to find suited and fair affiliation proposals for their affiliate page slots. We aimed at reducing the load in the affiliate search process and helping the mutually winning strategy when exchanging respective links on websites. In another paper, we have presented as work in progress the system on which we rely for heavy data harvesting and on-the-fly web page modification with push-type technology [2]. This system allowed us to push content in real-time at precise locations on a web page, using a server-side scheduler. In this paper, we present a system which lets Web pages stay up-to-date by using Real-Time Affiliate Content Agents. The implications of this technology are broader, but in the case at hand and current paper we name them Recommender Agents. This system makes use of Page Agents, Agents nested inside Web Pages, which assist in improving the contents of the related articles section of a given article. More than helping find useful affiliates for shared sections of the site such as the blogroll [1], it actually selects the latest up-to-date related articles in real-time for the pages the agents run on. The set from which the related affiliate content can be open (all existing web content in the system) or closed (affiliate content only). K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 173–182, 2011. c Springer-Verlag Berlin Heidelberg 2011 

174

R.M.E. Swezey et al.

With a sample implementation as part of the development of a Web platform of which the objective is to involve citizens in regional debate in Japan, we show that our architecture is valid for real-world implementation, if realistic performance requirements are met with the classifier. In the sample implementation, the latest up-to-date news and articles can be fetched from the open Web, and then classified for each city of Japan, then pushed on articles and news related to the same city. Firstly, we will begin by stating the context of the research. Secondly, we detail our architecture and how our current system works. Thirdly, we will show the sample implementation of our system, which fetches latest affiliate news contents related to 47 Japanese prefectures and over 1700 Japanese cities. We detail technical challenges and show achievements and performance. We then conclude and discuss further research about the subject.

2 2.1

Context Affiliation

We call affiliation, the process of linking between websites, when this linking holds more meaning that of a simple reference. Besides friendship or common interest, affiliation seeks to share visitors, as well as raising site awareness mutually. Affiliation is also most commonly defined by commercial affiliation, i.e a generally unidirectional link from an editor (blog, website) to an announcer (commercial or promotion site), in a textual or graphic fashion. The first important dimensions in settling community or commercial affiliation, consists in the statistics of the editor. If this is a community affiliation, then it generally concerns both sites since they are both editors. Those statistics are considered more or less importantly in community affiliation, but they are vital in commercial affiliation, especially when there is no third-party broker between the editor and the announcer. They are also important for the webmaster to study the audience. This dimension was assessed using the Agents described in section 2.3. The second important dimension is context. Affiliate Content does not solely consists in community affiliate links or advertisement blocks, it can also be imported textual content such as news feed content. At any rate, we define affiliate content by contents which are related more or less to each page’s topic, or to the reader’s profile (targeted advertisement). Therefore, affiliate content that can be pushed dynamically depending on the context of the page is of high value to advertisers, or news/topics background and continuity investigators such as those concerned by our sample implementation. 2.2

Limitations of Recommended Contents Sections

For commercial affiliate contents: Most context-related affiliate content today is provided through a system known as sponsored search auctions [3]. This system however is flawed in the sense that any bidder with enough bidding power can win the auction and show totally unrelated content for any keyword, so long as its bid for it is high enough.

Intelligent Page Recommender Agents

175

For related topic contents: Often, related contents sections on Web articles will present flaws such as the following: – It is frozen in time: cached at the time of page generation, or used as a source at the time of writing. It does not recommend latest up-to-date content on the topic, if the article is too old but the topic itself still has ongoing events. – It is limited to the domain/site at hand and does not recommend content from other sources. On recommender systems and synchronous Javascript: Though little research seems to have been done on these issues, we can notice a general trend in real-time recommendation that is aimed at live information streams such as Twitter [4,5]. Our system has the same objective, only that it recommends related contents in real-time directly when users are browsing article pages relevant to a given topic. 2.3

Technology Developed Until Now

Using our previous research utilizing channelizing technology and server-side scheduling [2] (see Fig. 1), website or blog editors can measure the utility of each page slot on the web page, for the various affiliate content blocks in which contents can be recommended. This utility can be measured gathering statistics from various data, such as page scrolling, mouse movement, and so forth. Secondly, the event data is analyzed and interpreted to modelize reader behavior. Those functionalities are implemented using machine learning technology [6]. They can push contents in real-time at the location they want on their Web pages, at the time they wish. This Agent technology can be used independently on CMS-generated Web pages, such as typical blog pages or news sites, as well as on plain HTML pages.

Web Page

Channel

Defining Channels

Content

(Channelizing)

Server Webmasters

Web Browser Push Delivery

Time Table

Edit Contents

Wisdom Ad Balloon

Fig. 1. Push-type Content Delivery Mechanism on Wisdom Ad Balloon

176

R.M.E. Swezey et al.

One of the system’s strong points is that it requires only to modify the HTML page, or the layout template in case of a CMS. In the latter case, since modifying the layout template is generally much easier and do not require any advanced programming skill at all. We have further improved it in the research at hand, with Recommender Agents.

3

General Architecture

In this section, we talk about our new framework and its architecture. We introduce a new architecture for real-time recommendation of contents. Hence, in the rest of this paper, we assimilate the terms Page Agent and Recommender Agent. 3.1

Modules

Our system relies on the following modules: Table 1. Modules Feed Engine

Aggregates RSS feeds from various sources and process new articles Agent Server Server interface for the Recommender Agents Classifier Engine Classifies the new articles: can output a single class or a vector of classes for the articles (in case we want to recommend by cosine similarity) Database Known articles and their respective classes Recommender Agents Page Agents running on instances (Web pages opened by users) of articles and recommending content accordingly

3.2

Agent Interface

Using our previously implemented system and considering other research (see previous section), we have taken our Page Agent framework a step further, by allowing it to use synchronized Javascript objects over numerous instances of opened pages in many browsers. The Agent consists of two parts: – The Agent itself, which interacts with the page and replaces page slots with content by channelizing, and can store his own history and local data. – The shared memory, a synchronized Javascript object shared by the Agents over all article pages opened in browsers which relate to the same class (topic). We call this object a Wisdom Shared Object (WSO). As all operations on the synchronized socket objects are performed on the client side, it is also important to take into account the atomicity of some of these operations. Data corruption can occur if two or more browser agents (i.e execution

Intelligent Page Recommender Agents

177

threads in the general case) try to write into a shared property of each synchronized agent, or if one agent reads a shared property and begins an operation on it before another has finished writing into it. A trivial example being that of a counter incrementation, with the counter stored in a shared property. In the operation counter = counter + 1, run by agents A and B using the shared property counter, a save (write) can occur between the load (read) and save (write) of this operation. To address this synchronization issue, several traditional methods which can be adapted exist [7,8], but we choose a timestamp/revision number technique [9] which is more up-to-date and best suited for client-server handling of conflicts. 3.3

Workflow

Client-Server workflow 1. A user opens the article a in a Web browser. 2. The Agent located on a sends a’s URI and contents to the Agent Server when it is opened on the Web for the first time, for example by his author. This article can also be input by other sources such as a Feed Engine. 3. The Agent Server interrogates the Database to find out a’s topic class ca . In case no result is found, i.e a is new to the system, the Agent Server retrieves a class ca from the Classifier Engine and saves (a, ca ) to the Database. In the case of a vector implementation, instead of just ca we have a vector of pairs (ci , wi )a with wi,a the weight of the geographic class ci in relation to a. 4. In response to the Recommender’s Request, the Agent Server pushes ca , or the vector of pairs (ci , wi )a in the case of a vector implementation. 5. The Recommender fetches a Javascript shared object, building its identifier depending on ca or the vector (ci , wi )a . This object contains recommendations related to a’s geography, and can be fetched from the same Agent Server or another one. Recommender Agent workflow 1. The Agent builds automatically an Article Object out of the Web Page it runs on, by extracting HTML tags marked with specific ID’s. The final object has the same properties as a RSS feed item or HTML5 article: URI, title, article text, etc. 2. As seen on the previous workflow, the Agent sends the Article Object to the Agent Server in order to get either its most probable Topic Class as a scalar, or a Topic Class vector. 3. After having been given a Topic Class ID by the Agent Server, the Agent loads a Shared Object (see Sect. 3.2) associated with the ID of the Topic Class, which is a synchronized shared Javacript object between Agents running on similar topic pages. In the case of a vector implementation, the shared Javascript object is associated to the closest centroid vector of clustered articles dimensioned by the

178

R.M.E. Swezey et al.

geographic classes, which we can find with cosine similarity for the article at hand. 4. The Agent, as well as Agents using the same shared object, reorganizes the recommended contents in it. They can be extended to use their own heuristics: article’s date of publication, visitor counter for each article, and other heuristics depending on the Agent. 5. The Recommender Agent shows the recommended contents, and modifies it in real-time if necessary, in a related contents section of the Web Page.

4

Sample Implementation and Results

4.1

Aim and Original Project

We implemented a sample system based on this architecture, as part of the development of a Web platform of which the objective is to involve citizens in regional debate, based on [10]. This platform is itself a project funded by the Japanese Ministry of Internal Affairs and Communications, as part of the SCOPE1 competitive nationwide research development programme in Japan. In this sample implementation, we aim to fetch the latest up-to-date news and articles from the open Web, classify them for each city of Japan, and push them on articles and news related to the same city, to local keep citizens informed and aware. 4.2

Classifier Module

For the Classifier Module, we chose to use a Complementary Naive Bayes classifier (CNB) [11], because of the lack of data for certain classes, e.g cities and prefectures of Japan which are not popular and do not provide with an amount of training/testing data that can be compared against, for example, the wards of Tokyo metropolis. The data set for training and testing was built from the Japanese Wikipedia article base, by adding each article as a document for the prefecture/city/district class it relates to. For all 2024 classes2 , a total of 105018 documents was gathered, which amounts to 299889 documents when building the hierarchical set. Documents from each city are duplicated for the prefecture they belong to, and documents from each ward are as well duplicated for the city they belong to3 . We conducted a closed-test. To train and test the classifier, we used Apache Mahout4 over an Apache Hadoop5 cluster of 3 machines. Mahout implements CNB as a Transformed 1 2 3 4 5

SCOPE: Strategic Information and Communications Research & Development Promotion Programme. 47 prefectures, 1811 cities including the Tokyo metropolis wards, and 166 wards. During the hierarchical classification, the metropolises of Tokyo, Osaka, Kyoto count as prefectures, and their districts (wards) as independent cities. http://mahout.apache.org/ http://hadoop.apache.org/

Intelligent Page Recommender Agents

179

Weight-normalized Complement Naive Bayes (TWCNB) as described in [11] (steps 1-8). We modified the CNB algorithm of Mahout and added a few other features so that the classifier could be tested in two fashions: flatly and hierarchically. Because our geographic topical classes can be broken down into a threelevel class tree (prefectures/cities/districts), and because of the large amount of classes, we felt that implementing a hierarchical testing of documents [12,13], as opposed to a naive flat testing, would let us achieve better results in: – Accuracy: If the classes can be broken into a class tree, especially when the number of classes is important, we have more chances of achieving better accuracy if we can classify under one common/parent category (the prefecture) before looking for the right child one (the city), instead of directly looking for the city. – Performance: The first classification check is against 47 prefecture, and the second against 1810 ≈ 38 cities, which amounts to roughly 85 score 47 calculations for each document, if classes were equiprobable. If we did a flat classification directly over the 1810 cities, it would amount to 1810 score calculations, and result in a 2029% time cost increase. Table 2. Performance of Flat Classifier Classes Classes for flat training Correctly Classified Instances Incorrectly Classified Instances Total Classified Instances Avg. Time to Classify one Instance

1810 cities 1810 (cities only) 69944 (42.086%) 35074 (57.914%) 105018 522 ms

Table 3. Performance of Hierarchical Classifier Classes

2024: 47 prefectures 1810 cities 167 districts Correctly Classified Instances 69944 (66.602%) Incorrectly Classified Instances 35074 (33.398%) Total Classified Instances 105018 Avg. Time to Classify one Instance 38,6 ms

We can see here that the hierarchical classifier here outperforms the flat classifier for real-world application, notably in processing time performance. However, the accuracy of the hierarchical classifier can be improved, it will be the object of further research. We also intend on doing open tests as well as N-fold tests of the classifier.

180

4.3

R.M.E. Swezey et al.

Page Agent

The Page Agent Module was implemented using the WebSocket protocol of HTML5 (draft 766 ). After having followed the protocol described in Section 3.3 and fetched the shared synchronized Javascript object related to the Topic Class (here, a city) related to the article currently displayed, the Agent implements the following delegates of the object: Table 4. Javascript Wisdom Shared Object Delegates open()

Actions to perform when the shared object is opened on the current Web page (e.g populate the related contents list next to the article) close() Actions to perform when the shared object is closed on the current Web page update() Actions to perform when the shared object is modified by this or another Agent using the object, and its new state pushed to the Page error() Actions to perform when an error occurs

One of the interesting actions that the Recommender Agent can perform outside the real-time modification of the related contents section of the Web Page is the modification of the Page template itself, according to what is indicated for the Topic Class of the shared object it fetches. Extension to custom affiliate template modification: This type of affiliate content is usually more characteristic of commercial affiliation. It basically consists in modifying the appearance (background, colors, images), namely

Fig. 2. Differences between Regular Affiliate Content Blocks and Custom Affiliate Templates 6

http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-76

Intelligent Page Recommender Agents

181

the template, of the main page, or a specific subset of pages, depending on the contract (see Fig. 2). The resulting appearance helps the affiliate by raising its brand awareness, e.g showing the brand name or advertising about a product of the brand. One good example of this custom affiliate template strategy can often be seen on video game news sites, promoting an editor’s new game. One example application in the case at hand is, for example, when reading an article about a particular city of Japan, recommending the forthcoming matsuri 7 of this city by changing the appearance of the news site or blog being browsed. Another example is a more visible alert in case of localized disasters such as earthquakes (we have developed such a system, which we call Wisdom Alert).

5

Conclusion

In this paper, we have introduced a novel method for recommending the latest up-to-date contents in a set of related events or topics, dynamically and in realtime, on any article Web page that is related to the events or topics. We made use of persistent and synchronized recommender agents which can easily be extended to implement a lot of functionalities. We considered accuracy, processing time and ease of implementation in our approach. We have addressed each of these challenges in a example implementation of our architecture. In this implementation, we classify articles from more than 47 Japanese prefectures and over 1700 Japanese cities. By making use of hierarchical classification, we are able to categorize articles faster and more accurately, and demonstrate that such a system is scalable and usable in real-world applications. As for the recommender in the client Web pages, it is based on Javascript agents with synchronized shared memory as well as local memory, which require no server code, only a request handler or proxy on the same domain. The algorithm can also be used with traditional AJAX and long-polling to suppress this need. The challenges in research to come are the following: improve placement on the recommender slots not only according to, but through more subtle heuristics which can incorporate but are not limited to: CTR, impact factor of the article, page ranking, etc. We also need to improve the efficiency of our Complementary Naive Bayes classifier in the sample implementation. For the improvements, we plan on testing with bi-grams and tri-grams, and dealing with the classes holding the biggest amount of training documents (Tokyo, Osaka, Hokkaido). As for the testing of the classifier, we are planning to do 10-fold cross validation tests. Finally, in further papers, we will present new modules in our system for supporting large-scale participative debate.

7

Town Festival. Most Japanese cities have their own.

182

R.M.E. Swezey et al.

References 1. Swezey, R., Nakamura, M., Shiramatsu, S., Ozono, T., Shintani, T.: Architecture for Automated Search and Negotiation in Affiliation among Community Websites and Blogs. In: Proceedings of the Twenty Third International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA-AIE (2010) 2. Nakamura, M., Asami, S., Ozono, T., Shintani, T.: A Dynamic Rearrangement Mechanism of Web Page Layouts Using Web Agents. In: Proceedings of the 22nd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems: Next-Generation Applied Intelligence, p. 643. Springer, Heidelberg (2009) 3. Lahaie, S., Pennock, D.M., Saberi, A., Vohra, R.V.: Sponsored search auctions. Algorithmic Game Theory, 699–716 (2007) 4. Chen, J., Nairn, R., Nelson, L., Bernstein, M., Chi, E.: Short and tweet: experiments on recommending content from information streams. In: CHI 2010: Proceedings of the 28th International Conference on Human Factors in Computing Systems, pp. 1185–1194. ACM, New York (2010) 5. Phelan, O., McCarthy, K., Smyth, B.: Using twitter to recommend real-time topical news. In: RecSys 2009: Proceedings of the Third ACM Conference on Recommender Systems, pp. 385–388. ACM, New York (2009) 6. Velayathan, G., Yamada, S.: Behavior-based web page evaluation. In: Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, pp. 409–412. IEEE Computer Society, Los Alamitos (2006) 7. Lamport, L.: A new solution of dijkstra’s concurrent programming problem. Commun. ACM 17, 453–455 (1974) 8. Peterson, G.L.: Myths about the mutual exclusion problem. Information Processing Letters 12(3), 115–116 (1981) 9. Cannon, B., Wohlstadter, E.: Automated object persistence for javascript. In: Proceedings of the 19th International Conference on World Wide Web, WWW 2010, pp. 191–200. ACM, New York (2010) 10. Macintosh, A., Gordon, T.F., Renton, A.: Providing Argument Support for EParticipation. Journal of Information Technology and Politics 6(1), 43–59 (2009) 11. Rennie, J., Shih, L., Teevan, J., Karger, D.: Tackling the poor assumptions of naive bayes text classifiers. In: Machine Learning-International Workshop Then Conference, vol. 20, p. 616 (2003) 12. Demichelis, F., Magni, P., Piergiorgi, P., Rubin, M., Bellazzi, R.: A hierarchical naive bayes model for handling sample heterogeneity in classification problems: an application to tissue microarrays. BMC Bioinformatics 7(1), 514 (2006) 13. Langseth, H., Nielsen, T.D.: Classification using hierarchical naive bayes models. Mach. Learn. 63, 135–159 (2006)

Meta-learning Based Optimization of Metabolic Pathway Data-Mining Inference System Tom´as V. Arredondo1 , Wladimir O. Ormaz´ abal1, 2 1,3 Diego C. Candel , and Werner Creixell 1

3

Departamento de Electr´ onica [email protected] 2 Departamento de Inform´ atica, Universidad T´ecnica Federico Santa Mar´ıa Av. Espa˜ na 1680, Valpara´ıso, Chile CSIS The University of Tokyo, Tokyo, Japan

Abstract. This paper describes a novel meta-learning (MTL) based methodology used to optimize a neural network based inference system. The inference system being optimized is part of a bioinformatic application built to implement a systematic search scheme for the identification of genes which encode enzymes of metabolic pathways. Different MTL implementations are contrasted with manually optimized inference systems. The MTL based approach was found to be flexible and able to produce better results than manual optimization.

1

Introduction

A common problem in bioinformatic research consists of determining the genes that encode enzymes of a particular metabolic process of interest. Obtaining this information is essential to produce a functional map of the metabolic capabilities of the organism and can be a benefit in determining the advantages of using one specific microorganism in a particular application. In our research, we are interested in determining the genes which encode enzymes involved in the bacterial degradation of aromatic compounds that may be harmful to human health. Given that not all bacteria are capable of performing such degradation it is important to determine the genes that are responsible for said functions because it provides a fingerprint which can be used to asses and understand such capability in various bacteria. It is toward this goal, that we search for the genes that encode the enzymes (i.e. proteins) which participate in the process of aromatic compound degradation. With these genes, a search for other such organisms can take place using well known tools and databases such as BLAST [1] and Vector NTI [2]. These tools perform DNA sequence alignment and search in order to find a sequence segment (known as an Open Reading Frame or ORF) which codifies a protein (known as a coding sequence or CDS). The data sources used for said comparisons have DNA, protein sequences and metabolic pathway maps stored within them (MetaCyc [3], UM-BBD [4], KEGG [5], NCBI [6] amongst others). K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 183–192, 2011. c Springer-Verlag Berlin Heidelberg 2011 

184

T.V. Arredondo et al.

The integration of Bioinformatic data sources is a continuing problem given the complexity and skills required to utilize them in an efficient manner. With this in mind, we have produced two such applications: GeXpert ([7],[8]) and Phylogist [10]. Both of these systems attempt to improve and optimize the process of integrating data and applications from different public sources. Part of the workflow of these applications involves the selection of candidate genes which encode an enzyme of interest. In both of these applications, an inference system is used to classify the candidates from genes previously analyzed for an organism under study. Previous approaches toward metabolic network reconstruction have used various algorithmic methods such as name-matching and using EC-codes to link metabolic information to genes (e.g IdentiCS [9], metaSHARK [14]). The AUTOGRAPH (AUtomatic Transfer by Orthology of Gene Reaction Associations for Pathway Heuristics) method [15] uses manually curated metabolic networks, orthologs and their associated reactions to compare predicted gene-reaction associations. Our current approach in optimizing the inference system is applicable to any such data-mining application. In Section 2, we describe the bioinformatics application and the inference system used. In Section 3, we expose design details and the proposed implementation solution. We describe our experiments and results obtained in Section 4. Finally in Section 5, we present conclusions and outline ideas for future work.

2 2.1

Bioinformatic Inference Systems Integration Framework Process

Toward determining if certain genes participate in a metabolic pathway of interest (e.g. biodegrading of an aromatic compound), many current bioinformatic data-mining systems ([7]-[12]) utilize an iterative work flow. As in [12], an Artificial Neural Network (ANN) is used to estimate the quality of an alignment match of a DNA candidate sequence in terms of corresponding to the gene encoding a specific protein for a metabolic pathway of interest. This inference system, uses a selection of BLAST results as inputs and seeks to improve the selection of genes based on meta-optimization. In addition, the information provided by experts on previously researched genes must be scored in the organism under study and the inference system (i.e. the Learner) must learn from this information in order to generate future scorings for other genes 2.2

Inference System Workflow

The idea behind our proposed workflow as used in [13] is that once an alignment is performed, the results returned by the BLAST algorithm are passed to an ANN classifier which provides the user with an indicator of its goodness and also permits a constant updating of the training data with the objective of continually improving the neural network.

Meta-learning Based Optimization

185

The input parameters correspond to the BLAST sequence alignment scores, which are defined as follows: – – – – –

Identity: Coincidence percentage. Positives: Symbol similarity percentage. E-value: Random match result probability. Gaps: Missing sequence segment percentage. BitScore: Statistical significance measure.

The inference system output corresponds to an indicator of the goodness of the candidate gene alignment, which take continuous values in the interval [0, 1]. Values higher than 0.5 indicate a good candidate, otherwise it is a bad candidate.

3

Meta-learning Based Solution

It is possible to find diverse approaches that have been categorized as MTL [16,17], all these approaches eventually aim to obtain the best result from the learning process. Some have centered in the study of the nature of the data as in the number of characteristics to analyze, the classes being classified or the amount of training data amongst others [16,17] (e.g. Landmarking [18]); others have applied combinations of base learners (e.g. Stacked Generalization [19]). The majority of previous works [20,21,22], consider certain aspects of interest (e.g. learning parameters or learning model structural elements). To the best of our knowledge, only [23] approaches both parametric aspects and model configuration but in a manner specific to his context. Our current proposal is a general conceptual framework that generalizes the concept of MTL independent of the inference system (i.e. learner or weak learner) and the algorithm being used to optimize it (e.g. backpropagation, genetic algorithms, etc). 3.1

Training and Validation Data Definition

There are diverse possible data sets, given a determined context or problem of interest. Considering one of these possible sets, D, as a set of N elements: D = {d1 , d2 , . . . , dN } , |D| = N

(1)

Where: – – – –

di = (Ii , Oi ) , with 1 ≤ i ≤ N . Ii : Input values. Oi : Output values. |X|: Cardinality of the set X.

Following [24], it is possible to divide the set D in r disjoint subsets T : D = T1 ∪ T2 ... ∪ Tr , with Ti ∩ Tj = ∅, ∀i, j ∈ {1, 2, . . . , r} , i = j

(2)

186

T.V. Arredondo et al.

Starting from these r subsets, we can generate a training set (Si ) and a test set (Ti ). Ti corresponds to one of the r subsets, and Si is the union of the r − 1 remaining subsets. We thus define a training set Si as: Si = D − Ti , with D = {T1 , T2 , . . . , Ti , . . . , Tr }

(3)

Considering expression (2) we establish that: D = Ti ∪ Si , ∀ i ∈ {1, 2, . . . , r} 3.2

(4)

Training Methods and Inference System Model

Extending the idea of learner (i.e. inference system) training used in [24], we define Lk as a learner formed by the pair (learning model, learning method) as: Lk = (modi , trj ), with modi ∈ MOD, trj ∈ TR (5) Where MOD is the set of possible learners (e.g. neural network, fuzzy inference system, etc.) and TR is the set of possible learning methods that can be used to train those models (e.g. backpropagation, genetic algorithm, etc.). Consider L as the set of possible different learners, that is the valid combinations (modi , trj ) of learning models and training methods: L = {L1 , L2 , . . . , Lk , . . .} , k ∈ N

(6)

Considering expressions (5) and (6), as well as the aforementioned restriction it is possible to assume that: |MOD × TR| ≥ |L|

(7)

For every Lk , a vector p can be defined that contains all parameters of the training method being used by Lk . We let Pk be the set of all γ possible p vectors: ∀Lk ∃ a Pk = {pk,1 , pk,2 , . . . , pk,γ } (8) In a similar vein, we define the set Mk of all θ possible inference system model configurations, each one of these represented by a vector m: ∀Lk ∃ a Mk = {mk,1, mk,2 , . . . , mk,θ } 3.3

(9)

Hypothesis

We understand a hypothesis h, as an instance of an inference system type (e.g. a neural network learner) after undergoing through the learning process described in [24]. There is an infinite set of hypotheses h associated with each one of the

Meta-learning Based Optimization

187

learners in L. For example, HA is the set of possible hypotheses generated from learner LA : LA → H A (10) Finally, considering all factors previously mentioned it is possible to establish that a hypothesis h depends on the learner LA being used, on the training data (Si ), on the model being used (MA ), and on the parameter values of the training method being used (PA ). We define Hl,j A (pA,l , mA,j , D) as the set of all hypotheses generated using vectors pA,l and mA,j while trained with Si (1 ≤ i ≤ r) obtained from D: Hl,j A (pA,l , mA,j , D) =

 1  hA (pA,l , mA,j , S1 ), . . . , hrA (pA,l , mA,j , Sr )

(11)

We define hA,l,j,i as a hypothesis generated using vectors pA,l and mA,j , and the training set Si . Also, the output for hypothesis hA,l,j,i given an input Ix is defined as: hA,l,j,i (Ix ). As seen in Fig. 1, the MTL seeks to obtain the best possible values for the parameters vector (pA,l ), this process is denoted as tuning. It is also tasked with finding the best possible values for the model configuration (mA,j ), this process is known as fitting. For a particular template, the generated hypotheses allow us to obtain a measure of fitness for the template than instantiated them. The measure of fitness used to evaluate the diverse hypotheses generated is the mean error over r of the test set Ti applied to all hypotheses (i.e. EMT L ). This fitness measure allows the MTL to evaluate and select the best combination of pA,l and mA,j found for a specific problem. EMT L =

 r  1 Correctly classified 1− r 1 Total cases

Fig. 1. Inference System Meta-optimizer

(12)

188

4 4.1

T.V. Arredondo et al.

Experiments and Results Neural Networks

We use a multilayer feed forward artificial neural network with one hidden layer and backpropagation for weight updating: Δwij (n + 1) = η · δP j oP j + α · Δwij (n)

(13)

We applied two implementations of backpropagation, the first one (NN1) using with a fixed learning rate (η) and the second one (NN2) with a variable learning rate. According to [25], during each learning cycle the initial value of η decreases exponentially with the d factor until reaching the value ηlow . Upon reaching this value ηlow , η takes the value of ηhigh and this process repeats:

⎞ ⎛ low log ηηhigh ⎠ η(t) = η(t − 1) · exp ⎝ (14) d In Table 1, we show the values that are used for each variable. The chosen parameters are those that were considered to have the greatest possible impact upon hypothesis generation. Table 1. Variable Values and Ranges Meta-learner Ni Nh No η ηlow ηhigh α d NN1 4 1 - 30 1 0 - 0.1 N.A. N.A. 0 - 0.9 N.A. NN2 4 1 - 30 1 0.3 0.01 0.01 - 0.3 0 - 0.9 20 - 84

To avoid overtraining, we utilized Ti to monitor the change in error according to (15). After each training round, in case of an increase in error the process is stopped. We restricted the possible maximum number of training iterations to 20000 and the minimum number to 1000. Toward the selection of training and test sets, we applied cross validation random sub-sampling [26], in a 70% - 30% proportion respectively. Errori = 4.2

|Ti |−1 1  1 2 (Oj − hA,l,j,i (Ix )) |Ti | x=0 2

(15)

Meta-learning Algorithms

A genetic algorithm (GA) and Stochastic Hill Climbing (SHC) were used in order to implement the process of MTL. These modified the number of hidden layer neurons (Nh ) as well as backpropagation parameters according to the test case (fixed or dynamic learning rate).

Meta-learning Based Optimization

189

Genetic Algorithm. The performance of the GA is associated with a series of parameters and considerations which determine its behavior. Previously, researchers have determined certain reasonable ranges for GA parameters to be used in a variety of optimization problems [27,28]. From these, we have chosen the following values for GA settings and parameters: tournament selection, one Elite individual, 100 iterations, a 20 individual population, two point crossover, probability of crossover of 0.95, and probability of mutation of 0.01. In GA-NN1, a granularity of 5 bits was used for Nh , 14 bits for η, and 14 bits for the momentum (α). For GA-NN2 we used 5 bits for Nh , 6 bits for ηhigh , 12 bits for α and 6 bits for d. Stochastic Hill Climbing. Another algorithm that was utilized for MTL was SHC [29]. This algorithm modified the same NN parameters as the GA. We used an equivalent number of objective function evaluations in order to have a comparable effort with respect to the GA. The parameters used were 200 iterations and 10 restarts. In SHC-NN1, we used a granularity of 5 bits for Nh , 14 bits for η, and α was also 14 bits. For SHC-NN2 we used 5 bits for Nh , 6 bits for ηhigh , 12 bits for α and 6 bits for d. 4.3

Results

In Table 2, the total average correct % classification results reached by all classifiers is compared. GA-NNX corresponds to the GA based MTL training NNX and SHC-NNX to the SHC based MTL training NNX. Naive-NNX and ExpertNNX correspond to a superficial (3 hours) and a mored detailed (6 hours) manual parameter search respectively. Table 2. Final Total Average Correct % Classification Results

Naive-NN1 Expert-NN1 Naive-NN2 Expert-NN2 SHC-NN1 SHC-NN2 GA-NN1 GA-NN2

KT 0.9472 ± 0.0266 0.9608 ± 0.0148 0.9540 ± 0.0145 0.9581 ± 0.0134 0.9635 ± 0.0143 0.9581 ± 0.0085 0.9729 ± 0.0124 0.9729 ± 0.0099

JMP 0.9324 ± 0.0168 0.9513 ± 0.0300 0.9391 ± 0.0264 0.9513 ± 0.0264 0.9581 ± 0.0195 0.9567 ± 0.0182 0.9702 ± 0.0166 0.9608 ± 0.0227

LB 0.8878 ± 0.0299 0.9175 ± 0.0224 0.9013 ± 0.0235 0.9202 ± 0.0148 0.9256 ± 0.0182 0.9189 ± 0.0222 0.9229 ± 0.0231 0.9229 ± 0.0231

Table 3, presents the best parameter and model values obtained by the different methods considered for the KT, JMP and LB data respectively. In Table 4, we show cross validation results obtained when testing the learners obtained by GA-NN1 y GA-NN2 with all the data. The rows indicate the training data and the columns indicate the test data used.

190

T.V. Arredondo et al. Table 3. Best Parameter and Model Values Obtained MTL KT GA-NN1 GA-NN2 SHC-NN1 SHC-NN2 MTL JMP GA-NN1 GA-NN2 SHC-NN1 SHC-NN2 MTL LB GA-NN1 GA-NN2 SHC-NN1 SHC-NN2

Ni 4 4 4 4 Ni 4 4 4 4 Ni 4 4 4 4

Nh No 25 1 27 1 5 1 19 1 Nh No 27 1 30 1 25 1 20 1 Nh No 15 1 27 1 5 1 13 1

η 0.06256 0.3 0.0318 0.3 η 0.07543 0.3 0.0683 0.3 η 0.09980 0.3 0.0318 0.3

ηlow N.A. 0.01 N.A. 0.01 ηlow N.A. 0.01 N.A. 0.01 ηlow N.A. 0.01 N.A. 0.01

ηhigh N.A. 0.0471 N.A. 0.0685 ηhigh N.A. 0.0614 N.A. 0.0342 ηhigh N.A. 0.0971 N.A. 0.04

α 0.3357 0.4446 0.8692 0.4775 α 0.6649 0.0718 0.5249 0.7712 α 0.8825 0.4428 0.8692 0.6793

d N.A. 51 N.A. 53 d N.A. 25 N.A. 74 d N.A. 81 N.A. 68

Table 4. Cross Validation Results

KT KT 0.9729 ± 0.0124 JMP 0.9594 ± 0.0142 LB 0.9364 ± 0.0247 KT KT 0.9729 ± 0.0099 JMP 0.9635 ± 0.0143 LB 0.9418 ± 0.0191

5

GA-NN1 JMP 0.9621 ± 0.0188 0.9702 ± 0.0166 0.9094 ± 0.0278 GA-NN2 JMP 0.9472 ± 0.0224 0.9608 ± 0.0227 0.9243 ± 0.0271

LB 0.9270 ± 0.0222 0.9283 ± 0.0211 0.9229 ± 0.0231 LB 0.9202 ± 0.0224 0.9283 ± 0.0191 0.9229 ± 0.0231

Conclusions and Future Work

As seen by the test results, a clear benefit of using MTL can be observed. Better results are obtained than in the manual optimization results and the complete automation of the search by using well known search algorithms (GA and SHC) alleviates the user from a tedious and non trivial task. Also, the proposed conceptual model leaves open a wide variety of potential enhancements and experiments without requiring model modifications and considers aspects previously left out by other learner models. The MTL software developed for this optimization is open source and is freely available [30]. The range of obtained performance results demonstrates that there is an impact in varying selected parameter and model values in fixed and dynamic learning rate cases. The utilization of the backpropagation method with a dynamic

Meta-learning Based Optimization

191

learning rate does not improve obtained results, on the contrary it seems to worsen obtained results [25]. When comparing classification performance, we can observe that using a GA rather than SHC as the MTL produces better results. Even so, both GA and SHC obtain better average results than the manually optimized classifiers. Future work includes the utilization of other algorithms in the MTL, the verification of their performance on a wider variety of problems as well as testing different parameters and model values of different granularity.

Acknowledgment This research was partially financed by Universidad T´ecnica Federico Santa Mar´ıa DGIP Grants 231138 and 231021.

References 1. NCBI: Basic Local Alignment Search Tool, http://blast.ncbi.nlm.nih.gov/Blast.cgi 2. Invitrogen Corporation: Vector NTI Software, http://www.invitrogen.com/site/us/en/home/Applications/ Cloning/Vector-Design-Software.html 3. National Institute of General Medical Sciences: MetaCyc Encyclopedia of Metabolic Pathways, http://www.metacyc.org/ 4. University of Minnesota: University of Minnesota Biocatalysis/Biodegradation Database, http://umbbd.msi.umn.edu/ 5. Kanehisa Laboratories: KEGG Pathways Database, http://www.genome.ad.jp/kegg/pathway.html 6. NCBI: National Center for Biotechnology Information, http://www.ncbi.nlm.nih.gov/ 7. Arredondo, T., Seeger, M., Dombrovskaia, L., Avarias, J., Calder´ on, F., Candel, D., Mu˜ noz, F., Latorre, V., Agull´ o, L., Cordova, M., Gomez, L.: Bioinformatics integration framework for metabolic pathway data-mining. In: Ali, M., Dapoigny, R. (eds.) IEA/AIE 2006. LNCS (LNAI), vol. 4031, pp. 917–926. Springer, Heidelberg (2006) 8. GeXpert Open Source Project, Available in Sourceforge http://sourceforge.net/projects/gexpert 9. Sun, J., Zeng, A.P.: IdentiCS - Identification of coding sequence and in silico reconstruction of the metabolic network directly from unannotated low-coverage bacterial genome sequence. BMC Bioinformatics 5, 112 (2004) 10. Phylogist UTFSM Bioinformatics Project Website, http://www.feriadesoftware.cl/2007/phylogist/producto.html 11. Arredondo, T., V´ asquez, F., Candel, D., Dombrovskaia, L., Agull´ o, L., C´ ordova, M., Latorre-Reyes, V., Calder´ on, F., Seeger, M.: Dynamic Penalty Based GA for Inducing Fuzzy Inference Systems. In: Rueda, L., Mery, D., Kittler, J. (eds.) CIARP 2007. LNCS, vol. 4756, pp. 957–966. Springer, Heidelberg (2007) 12. Leiva, M., Arredondo, T., Candel, D., Dombrovskaia, L., Agull´ o, L., Seeger, M., V´ asquez, F.: Feed-Forward Artificial Neural Network Based Inference System Applied in Bioinformatics Data-Mining. In: Conference Paper IJCNN 2009, Atlanta, USA, June 14-19, pp. 1744–1749 (2009)

192

T.V. Arredondo et al.

13. Burkholderia xenovorans LB400 at the Joint Genome Institute, http://genome.jgi-psf.org/finished_microbes/burfu/burfu.home.html 14. Pinney, J., Shirley, M.W., McConkey, G.A., Westhead, D.R.: metaSHARK: software for automated metabolic network prediction from DNA sequence and its application to the genomes of Plasmodium falciparum and Eimeria tenella. Nucleic Acids Research 33(4), 1399–1409 (2005) 15. Notebaart, R., van Enckevort, F., Francke, C., Siezen, R., Teusink, B.: Accelerating the reconstruction of genome-scale metabolic networks. BMC Bioinformatics 7, 296 (2006) 16. Anderson, M.L., Oates, T.: A review of recent research in metareasoning and metalearning. AI Magazine (2007) 17. Vilalta, R., Drissi, Y.: A Perspective View and Survey of Meta-learning. Journal of Artificial Intelligence Review 18(2), 77–95 (2002) 18. Bensusan, H., Giraud-Carrier, C.: Casa Batlo in Passeig or landmarking the expertise space. In: Eleventh European Conference on Machine Learning, Workshop on Meta-learning: Building Automatic Advice Strategies for Model Selection and Method Combination, Barcelona, Spain (2000) 19. Wolpert, D.: Stacked Generalization. Neural Networks 5, 241–259 (1992) 20. Aran, O., Yildiz, T., Alpaydin, E.: An incremental framework based crossvalidation for estimating the architecture of a multilayer perceptron. International Journal of Pattern Recognition and Artificial Intelligence 23(2), 159–190 (2009) 21. Hong, T.P., Lin, W.T., Chen, C.H., Ouyang, C.S.: Learning Membership Functions in Takagi-Sugeno Fuzzy Systems by Genetic Algorithms. In: First Asian Conference on Intelligent Information and Database Systems, pp. 301–306 (2009) 22. Pedersen, M.E.H., Chipperfield, A.J.: Simplifying Particle Swarm Optimization. Applied Soft Computing (2009) 23. Abraham, A.: Meta-learning Evolutionary Artificial Neural Networks. Neurocomputing 56 (January 2004) 24. Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997) 25. Watkins, D.: Clementine’s Neural Networks Technical Overview. Technical Report (1997) 26. Bouckaert, R.R., Frank, E.: Evaluating the replicability of significance tests for comparing learning algorithms. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 3–12. Springer, Heidelberg (2004) 27. De Jong, K.A.: An Analysis of the Behavior of a Class of Genetic Adaptive Systems. Ph.D. Dissertation. University of Michigan, Ann Arbor, MI, USA. AAI7609381 (1975) 28. Grefenstette, J.: Optimization of Control Parameters for Genetic Algorithms. IEEE Transactions on Systems, Man, and Cybernetics in Systems, Man and Cybernetics 16(1) (1986) 29. Juels, A., Wattenberg, M.: Stochastic Hillclimbing as a Baseline Method for Evaluating Genetic Algorithms, Technical Report CSD-94-834, Department of Computer Science, University of California at Berkeley (1994) 30. SimMetaLib: Simple Meta-Learning Library, http://profesores.elo.utfsm.cl/~ tarredondo/simmetalib.html

Multiple Pickup and Delivery TSP with LIFO and Distance Constraints: A VNS Approach Xiang Gao1 , Andrew Lim1 , Hu Qin2,1, , and Wenbin Zhu3 1 2

Department of Management Sciences, City University of Hong Kong, Tat Chee Ave, Kowloon Tong, Hong Kong School of Management, Huazhong University of Science and Technology, No. 1037, Luoyu Road, Wuhan, China 3 Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong [email protected]

Abstract. The pickup and delivery traveling salesman problem with last-in-first-out loading (TSPPDL) requires that a set of pickup and delivery requests are fulfilled such that the items to be transported are loaded in LIFO order. We generalize the TSPPDL to consider multiple vehicles as well as a maximum distance constraint for each vehicle; a solution to this problem is a set of routes for each vehicle, and each route can be represented by a tree data structure. To solve this problem practically, we propose a variable neighborhood search (VNS) heuristic using several new search operators. Our VNS approach also includes a dynamic programming component for optimally solving TSPPDL instances. Keywords: traveling salesman problem; pickup and delivery; last-infirst-out loading; distance constraint; variable neighborhood search.

1

Introduction

The traveling salesman problem with pickup and delivery (TSPPD) is a wellstudied problem. We are given a set R = {1, . . . , n} of n requests, where each request requires a load to be transported from pickup vertex i+ to delivery vertex i− (1 ≤ i ≤ n). There is a single vehicle with unlimited capacity that starts from a depot vertex 0+ . Its task is to fulfill all requests by visiting each pickup vertex before its corresponding delivery vertex (known as the precedence constraint) and finally return to a depot vertex 0− while minimizing the total distance traveled. The TSPPD is defined on a complete and undirected graph G = (V, E, d), where V = {0+ , 1+ , . . . , n+ , 0− , 1− , . . . , n− } is the vertex set, E = {(x, y) : x, y ∈ V, x = y} is the edge set, and the edge weight d(x, y) is the non-negative distance between vertices x and y where the matrix [d(x, y)] satisfies the triangle inequality condition. 

Corresponding author.

K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 193–202, 2011. c Springer-Verlag Berlin Heidelberg 2011 

194

X. Gao et al.

A variant of this problem that has received significant attention requires that loading and unloading are done in a last-in-first-out (LIFO) manner, known as the TSPPD with LIFO loading (TSPPDL). This constraint naturally arises when the vehicle storage unit has only a single door located at the rear and works like a stack ; it is especially applicable when the cost of rearranging loaded items is much higher than that of the extra traveling distance caused by the LIFO constraint, i.e., the rearrangement cost dominates the traveling cost. Examples include the transportation of bulky, fragile or hazardous items. The TSPPDL was first mentioned by Ladany and Mehrez [7], who solved a real-life delivery problem in Israel by an enumerative approach. Two classes of solution approaches have been applied to the TSPPDL, namely exact algorithms [4] and heuristics [8,3,12]. The difficulty of the TSPPDL is evidenced by the fact that the largest instances that can be solved optimally contain only 25 requests [4]. Hence, large instances with hundreds of requests arising in practical applications are commonly handled by heuristics. Currently, the best and latest heuristic is the variable neighborhood search (VNS) heuristic proposed by Tu et al. [12] that uses trees to represent the feasible solutions of the TSPPDL. This paper introduces a useful and practical generalization of the TSPPDL by considering two additional factors. The first is to consider m > 1 vehicles, which reflects the fact that practical problems of this nature usually involve a fleet of vehicles rather than a single vehicle. The second is to require that the route length for each vehicle cannot exceed a predetermined limit L, which stems from regulations on drivers’ working time or driving time; this type of distance constraint has been widely applied to many vehicle routing problem (VRP) variants, e.g., [9,5]. We call the resultant problem the multiple pickup and delivery traveling salesman problem with LIFO and distance constraints (MTSPPD-LD). To the best of our knowledge, there is no existing research on the MTSPPDLD. Consequently, our test instances and results can be used as benchmarks by subsequent researchers. We use the tree representation for feasible TSPPDL solutions first proposed by Tu et al. [12] to devise a VNS heuristic for the MTSPPDLD. Furthermore, we devise a dynamic programming (DP) algorithm that is able to solve small instances of the TSPPDL very quickly; this DP algorithm is integrated into our proposed VNS heuristic as a subroutine.

2

Tree Representation of Feasible Solutions

A feasible solution of the MTSPPD-LD is a set of at most m routes and can be represented by a set of vertex sequences; Figure 1(a) shows a feasible solution of an MTSPPD-LD instance comprising two routes. Most heuristics that have been applied to vehicle routing and scheduling problems are based on edgeexchange operations [1], which involve the moving or exchanging of edges in routes. However, such heuristics cannot be directly applied to the MTSPPD-LD because the resultant routes usually violate the LIFO constraint. To resolve this issue, we represent the feasible solutions of the MTSPPD-LD by a set of ordered trees rooted at node 0, where each tree node corresponds to a request.

Multiple Pickup and Delivery TSP with LIFO and Distance Constraints

0+

1+

4+

4-

5+

5-

0-

3-

3+

2-

2+

1-

R oute 1

Tree 1

1+

4+

1-

4-

0+

0-

2+

2-

5+

Tree 2 6+

8+

8-

9+

9-

6-

0-

7-

11-

11+

10-

10+

7+

6+

3-

5-

0+

0+

3+

195

0-

7+

6-

7-

R oute 2

(a)

8+

8-

9+

9-

10+

10-

11+

11-

(b)

Fig. 1. (a) The sequence representation of a feasible solution. (b) The tree representation of a feasible solution.

For example, the tree representation of the solution corresponding to the sequence in Figure 1(a) is given in Figure 1(b). By following the dashed arrows (which is similar to a preorder traversal), we can derive the corresponding vertex sequence from the ordered tree. In addition, all sequences derived from ordered trees automatically respect the precedence and LIFO constraints, and there is a one-to-one correspondence between the set of all ordered trees of this type and the set of all feasible TSPPDL solutions [12]. For the MTSPPD-LD, we can use a separate tree for each vehicle, with the added restriction that the sum of all edge weights in each tree is not greater than the maximum route length L.

3

The VNS Solution Scheme

The Variable Neighborhood Search (VNS) approach [10] systematically performs local search using several neighborhood operators to arrive at a local optimum, and then uses a perturbation operator to move the search to a new area of the search space. It has previously been successfully applied to vehicle routing problems with precedence and loading constraints (e.g., [11,6]). Our VNS heuristic is summarized in Algorithm 1. We first generate an initial solution and regard it as the best solution Sbest . In each iteration, we first invoke a function local search, which performs five operations node-relocate, nodeexchange, subtree-relocate, subtree-exchange and tree-merge in order; whenever local search encounters an improving solution, it restarts from the first operation. Next, we attempt to improve the route represented by each tree in the current solution individually. If the tree involves 14 or more requests, we use the VNS heuristic for TSPPDL devised by Tu et al. [12]; otherwise, we find the optimal tree using a dynamic programming (DP) approach. Then, we update Sbest if the current solution is superior. Finally, the perturbation operator is used to diversify the search process. We repeat the above operations until max nonimproving consecutive non-improving iterations occurs.

196

X. Gao et al.

Algorithm 1. The VNS heuristic for the MTSPPD-LD 1: INPUT: Integer max nonimproving; 2: Scurrent ← generate initial solution; Sbest ← Scurrent ; 3: while i ≤ max nonimproving do 4: local search(Scurrent ); 5: for each tree in Scurrent do 6: if tree contains 14 or more nodes then 7: Optimize tree using TSPPDL VNS heuristic [12] 8: else 9: Optimize tree using DP 10: end if 11: end for 12: if Sbest < Scurrent then 13: Sbest ← Scurrent and i = 0; 14: else 15: i = i + 1; 16: end if 17: Perturbation(Scurrent ) 18: end while

4

Search Operators

In this section, we describe the five search operators and the perturbation operator used by our VNS heuristic; the DP approach to optimally solve the TSPPDL for small trees is presented in the next section. For ease of discourse, we specify that node refers to a tree node corresponding to a request in R, and vertex refers to a pickup or delivery vertex in V . Each solution of the MTSPPD-LD consists of m trees, and trees with only one node 0 (called empty trees) represent vehicles that do not carry out any request. All of the operators only consider moves that satisfy the distance constraint, i.e., the sum of the edge weights for the resultant trees are at most L. Other than tree-merge and perturbation, all of these operators perform the move that results in the greatest savings in total distance out of all possibilities. The node-relocate operator removes a node x ∈ R from tree Tp and inserts it as a child of node y in another tree Tq (Tq can be an empty tree); this moves the request from one vehicle to another. Consider the example in Figures 2(a, b) showing two trees T1 and T2 that form a feasible solution to an instance with 12 requests. First, node x is removed from T1 and its children are linked to its parent node such that a new tree T1 (Figure 2(c)) is created. Next, node x is inserted into T2 as a child of node y (Figure 2(d)); if y has cy children, then there are cy + 1 possible positions for this insertion. If node x now has α right siblings, we can create other neighboring solutions by relocating the 1, . . . , α right siblings of node x as the children of x (Figures 2(e, f)). If the maximum number of children for any node in all trees is w, then the complexity of this operator is O(n2 · w). The node-exchange operator selects two non-root nodes from different trees and exchanges their positions, thereby swapping two requests. This operator runs in O(n2 ) time. Let TSx denote the subtree of T rooted at node x ∈ R; the subtree-relocate operator selects a subtree TSx from one tree and relocates it as the child of a

Multiple Pickup and Delivery TSP with LIFO and Distance Constraints

0 1 3

4

8

y

x 6

5

1

7

8

10

9

11

3

6

2

4

5

(a)

(b)

(c)

0

0

0

y x

0

0

2

y

7 9

10

11

x

10

(e)

7 10

x

11 8

8

(d)

y

7 9

197

11

9

(f)

Fig. 2. (a) Tree T1 in the original solution. (b) Tree T2 in the original solution. (c) Tree T1 in the resulting solution. (d–f) Possible trees T2 in the resulting solution.

node y in another tree; this moves the route segment represented by TSx from one vehicle to another. If node y has cy children, then there will be cy +1 possibilities to place node x as a child of node y. This operator also requires O(n2 ) time. The subtree-exchange operator is analogous to the node-exchange operator except that the exchange is applied to subtrees (rooted at nodes in R) rather than nodes. This operator similarly runs in O(n2 ). The tree-merge operator is applied on a feasible solution and successively attempts to merge a pair of trees into one tree, thereby reducing the number of vehicles by one. The merging of two trees combines their root nodes into one node; there are two ways to do so (Figure 3). With the triangle inequality condition, this merging process always reduces the total distance. Our tree-merge operator is given by Algorithm 2. If the number of non-empty trees in the feasible solution is h ≤ m, the time complexity of this operator is O(h3 ). Algorithm 2. The tree-merge operator 1:

Set i = 1; sort non-empty trees by increasing traveling distance to generate a tree sequence T1 , T2 , . . ., Th ; 2: Check the resultant solutions of merging Ti with each of the trees Tj (i < j ≤ h); 3: if there exists a feasible resultant solution then 4: Perform the merge that results in the largest distance saving and return to Step 1; 5: else 6: if i + 1 < h then 7: i = i + 1 and return to Step 2; 8: else 9: Terminate the process. 10: end if 11: end if

Finally, the perturbation operator helps the search process escape from local optima. Given a feasible solution consisting of h non-empty trees, we randomly remove a subtree from each tree to obtain a set V  of nodes (Figure 4(a)). The nodes in V  are then randomly chosen in turn and reinserted into the remaining trees. The insertion process follows two rules: (1) choose the non-empty tree

198

X. Gao et al.

0

1

0

0 2

3

4

1

5

0 3

2

4 5

6

(a)

3

4

6

5

1

2

6

(b)

(c)

Fig. 3. (a) The initial solution. (b–c) The new solutions created by tree-merge. V'

0

0

y 1

7

2

0

0

x 1

7

2

6

x

11

y

6 3

4

5

8

9

11 10

3

4

5

10

8

9

(b)

(a)

Fig. 4. (a) The remaining trees and V  after subtrees TSx and TSy are removed. (b) New solution after node insertion.

with the smallest total traveling distance; (2) attempt to insert the node in a way similar to the node-relocate operator. If a node cannot be inserted into any non-empty tree, then we insert it into an empty tree.

5

Dynamic Program for TSPPDL

A feasible solution to the MTSPPD-LD consists of a number of trees, each of which is a TSPPDL solution. After performing the local search procedure, we attempt to improve each tree in the current solution. When the tree consists of more than 13 requests, we employ the VNS heuristic proposed by Tu et al. [12], while a DP algorithm is used for trees with 13 requests or fewer. We now describe our DP algorithm for optimally solving the TSPPDL. Let S be the set of requests fulfilled by a vehicle. We define function f (a, b, S) to be the length of the shortest route starting from the pickup vertex of request a ∈ S and a+

a-

a+

b-

a+

b+

c-

a-

a+

a-

c+

b-

a+

b+

b-

a-

a+

a-

b+

b-

(a)

(b)

Fig. 5. Route with the first pickup vertex and the last delivery vertex belonging to (a) the same request and (b) different requests

Multiple Pickup and Delivery TSP with LIFO and Distance Constraints

199

ending at the delivery vertex of request b ∈ S that fulfills all requests in S. The routes covering set S and complying with the LIFO and precedence constraints can be divided into two cases depending on whether the first pickup vertex and the last delivery vertex come from (a) the same or (b) different requests, and each case can be further divided into two groups. These groups are shown in Figure 5, where a, b and c are different requests (a dashed arrow between two vertices signifies a subroute between the vertices). Our DP algorithm is based on these groups. Let dis[x][y] denote the distance between vertices x and y; and let a, b, c, d denote different requests. The values of f (a, a, S) and f (a, b, S) for |S| > 1 can be calculated by: 

{dis[a+ ][b+ ] + f (b, c, S − {a}) + dis[c− ][a− ]},  min {dis[a+ ][b+ ] + f (b, b, S − {a}) + dis[b− ][a− ]} (1) ∀b∈S−{a}  f (a, b, S) = min min {f (a, a, A) + dis[a− ][c+ ] + f (c, b, B)}, ∀A:a∈A,∀B:b∈B ∀c∈B−{b}  min{f (a, a, A) + dis[a− ][b+ ] + f (b, b, B)} (2)

f (a, a, S) = min

min

∀b,c∈S−{a}

For the base case of S = {a}, we set f (a, a, S) = dis[a+ ][a− ]. When the set S contains more than one request, it can be partitioned into two non-empty sets A and B. Given two disjoint non-empty request sets A and B, the subroutine calc(A, B) given in Algorithm 3 updates the values of f (a, b, S), where S = A ∪ B and a, b ∈ S. Our DP algorithm considers all possible partitions of S into two non-empty sets A and B, and invokes calc(A, B) as appropriate (Algorithm 4).

Algorithm 3. The calc(A, B) subroutine 1: INPUTS: Sets A and B, where A ∪ B = S and A ∩ B = ∅; 2: for all i ∈ A do 3: for all j ∈ A do 4: for all k ∈ B do 5: if f (k, k, B) + dis[k− ][i+ ] + f (i, j, A) < f (k, j, S) then 6: f (k, j, S) ← f (k, k, B) + dis[k− ][i+ ] + f (i, j, A) 7: end if 8: if f (i, j, A) + dis[j − ][k+ ] + f (k, k, B) < f (i, k, S) then 9: f (i, k, S) ← f (i, j, A) + dis[j − ][k+ ] + f (k, k, B) 10: end if 11: if |B| = 1 then 12: if dis[k+ ][i+ ] + f (i, j, A) + dis[j − ][k+ ] < f (k, k, S) then 13: f (k, k, S) ← dis[k+ ][i+ ] + f (i, j, A) + dis[j − ][k+ ] 14: end if 15: end if 16: end for 17: end for 18: end for

200

X. Gao et al.

Algorithm 4. The DP Algorithm for TSPPDL 1: INPUT: Set S; 2: Initialize f (a, b, S  ) ← +∞ for all S  ⊆ S and a, b ∈ S  ; 3: Initialize best ← +∞ 4: Let S1 , S2 , . . ., Sg be the subsets of S sorted in non-increasing order of 5: for i = 0 to g do 6: if |Si | = 1 then 7: f (k, k, Si ) ← dis[k+ ][k− ], where k ∈ Si 8: else 9: for each B ⊂ Si do 10: if |S − B| ≥ |B| then 11: Invoke calc(S − B, B) 12: end if 13: end for 14: end if 15: end for 16: for all i ∈ S do 17: for all j ∈ S do 18: if dis[0+ ][i+ ] + f (i, j, S) + dis[j − ][0− ] < best then 19: best ← dis[0+ ][i+ ] + f (i, j, S) + dis[j − ][0− ] 20: end if 21: end for 22: end for 23: Return best;

6

cardinality;

Computational Experiments

To evaluate our VNS heuristic, we conducted experiments using the data set introduced by Carrabs et al. [2]. This data set was derived from six TSP instances fnl4461, brd14051, d15112, d18512, nrw1379, and pr1002 taken from TSPLIB. For each of these TSP instances, subsets of vertices were selected with 25, 51, 75, 101, 251, 501 and 751 vertices. One arbitrary vertex was designated as the depot and the rest were paired to form requests using a random matching process, resulting in 48 test instances. Due to space limitations, we only report the results for instances with 251 or more vertices. Let dmax = maxi∈R {d(0+ , i+) + d(i+ , i− ) + d(i− , 0− )} be the greatest distance for any route involving a single request; we solved each instance with the traveling distance limit L set to 1.5×dmax, 2.0×dmax and 2.5×dmax . The number of vehicles m is set to 20, which is sufficient to ensure that all instances have feasible solutions. The only parameter max nonimproving was set to 10. We executed our VNS heuristic ten times for each traveling distance limit and for each instance with different random seeds. The algorithm was coded in C++ and all experiments were run on a Linux server with 3 GB memory and Intel Xeon(R) 2.66 GHz processor. Computation times reported here are in CPU seconds on this machine. The initial solution was generated by a cheapest insertion heuristic. Beginning with an empty tree, we choose an unselected request whose insertion minimizes the length of the tour. If no request can be inserted into the current tree, we continue with another empty tree. The above process is repeated until all requests are fulfilled. This is a natural heuristic that approximates how a human scheduler would solve the problem. Since the MTSPPD-LD has not been previously investigated, we evaluate our VNS heuristic by comparing it with this natural heuristic.

Multiple Pickup and Delivery TSP with LIFO and Distance Constraints

201

The results are displayed in Table 1, where Cost is the total distance traveled by all vehicles in the final solution averaged over 10 runs, and Gap(%) is the percentage improvement over the cheapest insertion heuristic achieved by our VNS heuristic. The values show that our heuristic is able to improve on the solutions that a human scheduler can reasonably find by between 14.81% and 29.7% on average; the results are similar for instances with 101 vertices or fewer. Furthermore, our heuristic takes less than 15 minutes on average for all instances of up to 751 vertices. Table 1. Comparison of VNS with the cheapest insertion heuristic L = 1.5 × dmax Instance Size Cost Time Gap(%) fnl4461 251 31430.6 48.03 21.93 501 85867.1 230.87 20.92 751 141688.5 279.00 22.97 brd14051 251 28409.1 43.77 26.97 501 61394.7 252.48 29.77 751 100602.1 350.55 22.62 d15112 251 760769.2 7.06 16.48 501 1206575.8 69.69 23.69 751 1713090.8 180.04 18.46 d18512 251 29683.9 32.51 24.83 501 59973.3 223.28 28.94 751 98081.9 461.22 21.29 nrw1379 251 33117.3 20.53 25.18 501 71696.5 97.60 23.51 751 130130.3 133.39 15.02 pr1002 251 279505.2 25.28 19.20 501 630616.9 100.19 14.81 751 1068144.3 222.58 20.31

7

L = 2.0 × dmax L = 2.5 × dmax Cost Time Gap(%) Cost Time Gap(%) 30417.3 59.21 24.9 30594.4 135.99 18.5 81491.2 206.24 16.3 78692.5 424.81 18.4 134488.7 337.23 16.2 128827.3 644.37 16.0 27042.0 68.60 23.7 25379.1 121.10 23.5 57754.9 296.30 27.2 55291.1 384.01 20.8 96035.0 652.23 21.2 93645.8 759.53 21.4 681074.8 22.95 20.5 655343.1 32.01 22.9 1118088.9 97.88 19.4 1097025.4 102.34 16.2 1573359.9 207.23 19.5 1516195.1 319.93 16.8 27471.8 55.32 24.8 26044.7 82.38 21.9 56622.5 311.54 25.0 54884.1 364.42 23.8 90748.8 727.01 23.0 89432.3 858.64 21.6 30996.2 26.39 21.9 30034.3 44.26 16.6 67910.8 150.19 17.5 66061.3 216.92 17.8 121934.0 229.23 15.4 118357.5 326.78 16.5 238978.4 30.69 25.2 225086.1 35.13 20.4 578011.9 136.05 17.0 554952.5 124.42 15.8 987741.3 265.20 15.5 935861.1 359.73 15.8

Conclusions

This article describes a new and practical traveling salesman problem involving pickup and delivery, LIFO loading, multiple vehicles and distance constraints. We make use of a tree representation of feasible solutions to develop a VNS heuristic using several new operators, as well as a dynamic programming subroutine that solves small instances of the single-vehicle subproblem optimally. Our approach can be used as a benchmark for future researchers in this area. Our preliminary work assumes that the number of available vehicles m is sufficient to cater to all requests and a feasible solution can be easily generated. If the number of vehicles is small, a two-phase approach may be required such that a separate first phase finds a feasible solution before our VNS heuristic is employed in the second phase. The class of traveling salesman problems with LIFO loading can be further generalized to handle more practical constraints, e.g., limited capacity vehicles and time windows. We believe that a VNS approach using the tree representation may also be effective for these variants.

202

X. Gao et al.

References 1. Br¨ aysy, O., Gendreau, M.: Vehicle routing problem with time windows, part I: Route construction and local search algorithms. Transportation Science 39(1), 104– 118 (2005) 2. Carrabs, F., Cordeau, J.-F., Laporte, G.: Variable neighborhood search for the pickup and delivery traveling salesman problem with LIFO loading. INFORMS Journal on Computing 19(4), 618–632 (2007) 3. Cassani, L., Righini, G.: Heuristic algorithms for the TSP with rear-loading. In: 35th Annual Conference of the Italian Operational Research Society (AIRO XXXV), Lecce, Italy (September 2004) 4. Cordeau, J.-F., Iori, M., Laporte, G., Gonz´ alez, J.J.S.: A branch-and-cut algorithm for the pickup and delivery traveling salesman problem with LIFO loading. Networks 55(1), 46–59 (2010) 5. Erera, A.L., Morales, J.C., Savelsbergh, M.: The vehicle routing problem with stochastic demand and duration constraints. Transportation Science (2010) 6. Felipe, A., Ortu˜ no, M.T., Tirado, G.: The double traveling salesman problem with multiple stacks: A variable neighborhood search approach. Computers & OperationsResearch 36(11), 2983–2993 (2009) 7. Ladany, S.P., Mehrez, A.: Optimal routing of a single vehicle with loading and unloading constraints. Transportation Planning and Technology 8(4), 301–306 (1984) 8. Levitin, G., Abezgaouz, R.: Optimal routing of multiple-load AGV subject to LIFO loading constraints. Computer & Operations Research 30(3), 397–410 (2003) 9. Li, C.-L., Simchi-Levi, D., Desrochers, M.: On the distance constrained vehicle routing problem. Operations Research 40(4), 790–799 (1992) 10. Mladenovi´c, N., Hansen, P.: Variable neighborhood search. Computers & Operations Research 24(11), 1097–1100 (1997) 11. Tricoire, F., Doerner, K.F., Hartl, R.F., Iori, M.: Heuristic and exact algorithms for the multipile vehicle routing problem. OR Spectrum (2009) 12. Tu, D., Guo, S., Qin, H., Oon, W.-C., Lim, A.: The tree representation of feasible solutions for the tsp with pickup and delivery and LIFO loading. In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2010), Atlanta, Georgia, USA, pp. 191–196 (July 2010)

Distributed Learning with Biogeography-Based Optimization Carre Scheidegger, Arpit Shah, and Dan Simon Cleveland State University, Electrical and Computer Engineering

Abstract. We present hardware testing of an evolutionary algorithm known as biogeography-based optimization (BBO) and extend it to distributed learning. BBO is an evolutionary algorithm based on the theory of biogeography, which describes how nature geographically distributes organisms. We introduce a new BBO algorithm that does not use a centralized computer, and which we call distributed BBO. BBO and distributed BBO have been developed by mimicking nature to obtain an algorithm that optimizes solutions for different situations and problems. We use fourteen common benchmark functions to obtain results from BBO and distributed BBO, and we also use both algorithms to optimize robot control algorithms. We present not only simulation results, but also experimental results using BBO to optimize the control algorithms of mobile robots. The results show that centralized BBO generally gives better optimization results and would generally be a better choice than any of the newly proposed forms of distributed BBO. However, distributed BBO allows the user to find a less optimal solution to a problem while avoiding the need for centralized, coordinated control.

1 Introduction Biogeography is the study of the geographical distribution of plant and animal life. Alfred Wallace and Charles Darwin were some of the first to observe the patterns of biogeography and introduce the subject to the scientific world [1]. Biogeography did not evolve into the quantitative science that it is today until Robert MacArthur and Edward Wilson created models from their studies of island biogeography in the early 1960s [2]. Other scientists also contributed to the emergence of the theory of island biogeography, most notably Eugene Monroe in 1948 [12]. Biogeography has continued to develop after MacArthur and Wilson’s research and it has recently been used as the motivating framework for the development of an evolutionary algorithm (EA) called biogeography-based optimization (BBO) [3]. In this paper, we apply BBO to experimental mobile robotics control optimization. This paper gives an overview of the evolutionary algorithm called BBO, which is based on mathematical models of biogeography, and which has been developed to solve general optimization problems [3]. BBO has been applied to several real-world problems. In addition to experimental robot control tuning, as discussed in this paper and in [4], BBO has been applied to aircraft engine sensor selection [3], power system optimization [13, 14], groundwater detection [15], mechanical gear train design [16], satellite image classification [17], and neuro-fuzzy system training for biomedical K.G. Mehrotra et al. (Eds.): IEA/AIE 2011, Part II, LNAI 6704, pp. 203–215, 2011. © Springer-Verlag Berlin Heidelberg 2011

204

C. Scheidegger, A. Shah, and D. Simon

applications [8]. Recent research in the area of BBO has focused on putting it on a firm theoretical and mathematical foundation, including the derivation of Markov models [19, 20] and dynamic system models [21] that describe its behavior. In this paper, we use BBO in computer generated simulations and on experimental mobile robots to study its performance. This paper also develops BBO’s distributed counterpart, which is based on distributed learning and intelligence, and the simulation results gathered from the distributed algorithm. The distributed algorithm’s communication and control is distributed through various BBO individuals rather than coordinated by a central computer. The development of distributed BBO (DBBO) has been motivated by the confluence of centralized BBO and concepts from distributed learning. Distributed learning is a theory developed to explain how the human mind understands and learns [7] [9]. Human’s mental capabilities are often assumed to be centralized inside the brain, but research has shown that outside social interactions greatly affect how the brain learns [7]. Distributed learning is an example of this type of environmental influence, and is often seen in humans working in teams to solve a common problem or complete a task [5] [6]. Distributed learning has been used in recent years to study how automated technology can be taught to perform human-like tasks using teams of robots that function together with nearly the same effectiveness as a team of humans [6] [7] [10] [11]. This distributed learning or intelligence in robotics is the ability of numerous entities to solve problems, perform tasks, learn, and understand by communicating with other entities in a group, rather than under the control of a centralized coordinator [6] [9] [10] [11]. Each organism is considered an individual with governing logic that allows it to perform a particular task, and the way in which these distributed learning groups carry out a task is dependent on the mode of communication by each entity [5] [6] [8] [10]. The study of distributed interaction is helpful in discovering and researching different ways to make artificial intelligences communicate to perform like a group of human beings, and has increasingly been applied to different types of systems. The advantages of distributed interaction in robotics open doors to new types of systems that do not need a central processor to control the team of robots, and allow flexibility for change and improvement. This flexibility in robotic applications of distributed intelligence is increasingly being studied because it has advantages that centralized systems do not. These advantages include autonomy, fault tolerance, and robustness. Section 2 in this paper gives an outline of BBO. It also proposes a distributed extension of BBO, which is one of the primary contributions of this paper. Section 3 presents fourteen benchmark function simulation results. Section 4 discusses the mobile robot system used for an experimental application of BBO and DBBO, including its design, hardware, task description, simulation results, and experimental results. Section 5 concludes with some discussion and suggestions for future work.

2 Biogeography-Based Optimization (BBO) BBO is an evolutionary algorithm that is modeled on the theory of biogeography. These models of biogeography mathematically describe how species travel to and

Distributed Learning with Biogeography-Based Optimization

205

from environments based on different factors of the environment [2]. These environmental factors can be represented quantitatively and are called suitability index variables (SIVs) and determine the suitability of the area for habitation. Examples of natural SIVs seen often in habitats are the amount of rainfall, the diversity of vegetation, and the temperature range. An area that is highly habitable is considered to have a high habitat suitability index (HSI) [3]. Habitats can be observed by scientists to develop mathematical models of migration, speciation, and extinction. A high-HSI habitat is likely to have a large number of species. It has a high rate of emigration and a low rate of immigration due to its dense population. The opposite occurs in low-HSI habitats because of the habitat’s sparse population. Emigration and immigration rates in a given habitat are proportional to the number of species that reside in that habitat. This habitat suitability concept that biogeography quantifies is what makes biogeography applicable to optimization problems in engineering and other fields of study. An individual in an evolutionary algorithm that has a high HSI (performance, or fitness) represents a good solution, and that individual will emigrate successful features to other individuals. Individuals that receive features from successful candidate solutions tend to increase their own fitness. Quantifiably applying the emigration and immigration of specific features from one individual to another depending on the individuals’ HSI values generally creates better solutions. Biogeography-based optimization (BBO), which was introduced by Simon [3], is the implementation of this extension of biogeography to optimization. 2.1 Centralized BBO Centralized BBO is the original BBO algorithm created to optimize solutions based on the theory of biogeography, and it uses the migration of traits to create better generations of candidate solutions to an optimization problem. As explained previously, a habitat’s migration rates are dependent on the habitat’s HSI. A habitat is analogous to a problem solution, and its HSI is analogous to the fitness of the solution. A solution’s fitness determines its rates of immigration, λ, and emigration, μ, and is determined in a way that is similar to natural biogeography. BBO bases the migration rate of each candidate solution on the HSI of the solution, with high HSI giving a high emigration rate, and low