Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding - Volume 151 Frontiers in Artificial Intelligence and Applications

LEARNING BY EFFECTIVE UTILIZATION OF TECHNOLOGIES: FACILITATING INTERCULTURAL UNDERSTANDING Frontiers in Artificial I...

Author: R. Mizoguchi; P. Dillenbourg and Z. Zhu; Editors

34 downloads 1610 Views 10MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

LEARNING BY EFFECTIVE UTILIZATION OF TECHNOLOGIES: FACILITATING INTERCULTURAL UNDERSTANDING

Frontiers in Artificial Intelligence and Applications FAIA covers all aspects of theoretical and applied artificial intelligence research in the form of monographs, doctoral dissertations, textbooks, handbooks and proceedings volumes. The FAIA series contains several sub-series, including “Information Modelling and Knowledge Bases” and “Knowledge-Based Intelligent Engineering Systems”. It also includes the biennial ECAI, the European Conference on Artificial Intelligence, proceedings volumes, and other ECCAI – the European Coordinating Committee on Artificial Intelligence – sponsored publications. An editorial panel of internationally well-known scholars is appointed to provide a high quality selection. Series Editors: J. Breuker, R. Dieng-Kuntz, N. Guarino, J.N. Kok, J. Liu, R. López de Mántaras, R. Mizoguchi, M. Musen and N. Zhong

Volume 151

Recently published in this series Vol. 150. B. Bennett and C. Fellbaum (Eds.), Formal Ontology in Information Systems – Proceedings of the Fourth International Conference (FOIS 2006) Vol. 149. X.F. Zha and R.J. Howlett (Eds.), Integrated Intelligent Systems for Engineering Design Vol. 148. K. Kersting, An Inductive Logic Programming Approach to Statistical Relational Learning Vol. 147. H. Fujita and M. Mejri (Eds.), New Trends in Software Methodologies, Tools and Techniques – Proceedings of the fifth SoMeT_06 Vol. 146. M. Polit et al. (Eds.), Artificial Intelligence Research and Development Vol. 145. A.J. Knobbe, Multi-Relational Data Mining Vol. 144. P.E. Dunne and T.J.M. Bench-Capon (Eds.), Computational Models of Argument – Proceedings of COMMA 2006 Vol. 143. P. Ghodous et al. (Eds.), Leading the Web in Concurrent Engineering – Next Generation Concurrent Engineering Vol. 142. L. Penserini et al. (Eds.), STAIRS 2006 – Proceedings of the Third Starting AI Researchers’ Symposium Vol. 141. G. Brewka et al. (Eds.), ECAI 2006 – 17th European Conference on Artificial Intelligence Vol. 140. E. Tyugu and T. Yamaguchi (Eds.), Knowledge-Based Software Engineering – Proceedings of the Seventh Joint Conference on Knowledge-Based Software Engineering Vol. 139. A. Bundy and S. Wilson (Eds.), Rob Milne: A Tribute to a Pioneering AI Scientist, Entrepreneur and Mountaineer ISSN 0922-6389

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding Edited by

Riichiro Mizoguchi Osaka University, Japan

Pierre Dillenbourg

Swiss Federal Institute of Technology Lausanne, Switzerland

and

Zhiting Zhu

East China Normal University, China

Amsterdam • Berlin • Oxford • Tokyo • Washington, DC

© 2006 The authors. All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without prior written permission from the publisher. ISBN 1-58603-687-4 Library of Congress Control Number: 2006936136 Publisher IOS Press Nieuwe Hemweg 6B 1013 BG Amsterdam Netherlands fax: +31 20 687 0019 e-mail: [email protected] Distributor in the UK and Ireland Gazelle Books Services Ltd. White Cross Mills Hightown Lancaster LA1 4XS United Kingdom fax: +44 1524 63232 e-mail: [email protected]

Distributor in the USA and Canada IOS Press, Inc. 4502 Rachael Manor Drive Fairfax, VA 22032 USA fax: +1 703 323 3668 e-mail: [email protected]

LEGAL NOTICE The publisher is not responsible for the use which might be made of the following information. PRINTED IN THE NETHERLANDS

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

v

Preface Welcome to the 14th International Conference on Computers in Education (ICCE) hosted by Beijing Normal University (BNU) and supported by the Ministry of Education (MOE) and Chinese Association for Artificial Intelligence (CAAI). The series of conferences started in Taiwan in 1989. The three first editions of ICCE occurred there and then the conference moved across many countries in the Asia-Pacific region: China, Singapore, Malaysia, Japan, Korea, New Zealand, Hong Kong and Australia. The ICCE series is organized by the Asia-Pacific Society for Computers in Education (APSCE). The ICCE is an international event, with strong participation from researchers from Asia and Oceania. Since the beginning, there has also been strong involvement from researchers in Europe and North-America. These proceedings include the 45 full papers and 53 short papers that have been selected by an international board of scholars listed hereafter. All papers have been reviewed by three or at least two reviewers and double-checked by the Co-chairs of the programme committee. Altogether 254 papers were submitted, which represents a rate of acceptance of 18% if considering only long papers or 38% if including short papers. The content of this volume hence results from a severe selection process. In addition, 22 posters will be presented at the event. A major theme of this conference is the use of computers for supporting collaborative learning. This is not surprising since computer-supported collaborative learning has become both a widespread educational practice and a main domain of research. Moreover, collaborative learning has deep roots in Asian educational traditions. Given the large number of researchers within this field, its scope has become very broad. Under this umbrella, one finds a variety of more specific topics such as: interaction analysis, collaboration scripts (e.g. the Jigsaw script), communities of practice, sociocognitive conflict resolution, cognitive apprenticeship, various tools for argumentation, on-line discussion or collaborative drawing tools (whiteboards), collaborative writing and the role of facilitators. Most research work on collaborative learning focuses on interactions rather than on the contents of environments, which had been the focus in the previous decades of learning technology research. However, there is no reason to focus on one aspect to the detriment of the other. Hence, we are quite pleased that the selected papers also cover multiple issues related to the storage, representation and retrieval of knowledge: ontologies for learning environments and the semantic web, knowledge bases and data mining, meta-data and content management systems, and so forth. These proceedings also reveal a growing interest for non-verbal educational material, namely pictures and video materials, which are already central to new popular web-based applications. Interestingly, these proceeding include contributions that bridge both research tracks, the one focusing on interactions and the other on contents: the pedagogical use of digital portfolios, both for promoting individual reflections and for scaffolding group interactions. Another specificity of these proceedings, maybe due to regional policies, is the salience of language learning within the range of educational contents covered by learning technologies.

vi

Among the other research trends that appear in the set of contributions, we find the educational use of mobile technologies and the design of educational games. The use of mobile devices and games opens new ways to think about learning technologies, namely the fact that tools that do not a priori appear as learning tools have perhaps more chances to enter into schools. There are nonetheless a limited number of papers on these issues since new conferences emerged in our community, which are devoted to these two subsets of learning technologies. Finally, let us stress the truly interdisciplinary character of this volume. It contains contributions from the field of computer science, psychology and educational sciences, three fields that are sometimes bound together under the label of ‘learning sciences’. The very positive aspect is that computational, pedagogical, cognitive or social factors are not only treated by different papers, but, in many cases, tackled within the same paper. It is our great pleasure to have gathered these rich contributions within this volume. We hope all readers will share our enthusiasm for this exceptional event. Program Committee Co-Chairs Riichiro Mizoguchi, Osaka University, Japan Pierre Dillenbourg, Swiss Federal Institute of Technology Lausanne, Switzerland Zhiting Zhu, East China Normal University, China

vii

Program Committee Conference Chair Fong-Lok Lee, The Chinese University of Hong Kong, Hong Kong Program Co-Chairs Riichiro Mizoguchi, Osaka University Pierre Dillenbourg, Swiss Federal Institute of Technology Lausanne Zhiting Zhu, College of Educational Sciences, East China Normal University Committee Members Lora Aroyo, Eindhoven University of Technology, The Netherlands Nelson Baloia, Chille University, Chile Jacqueline Bourdeau, Tele-universite du Quebec, Canada Tak-Wai Chan, National Central University, Taiwan Yam San Chee, National Institute of Education, Singapore Jim Greer, University of Saskatchewan, Canada Paivi Hakkinen, University of Jyvaskyla, Finland Yusuke Hayashi, Osaka University, Japan Lyn Henderson, James Cook University, Australia Tsukasa Hirashima, Hiroshima University, Japan Chris Hoadley, Penn State University, USA Ulrich Hoppe, University Duisburg, Germany Ronghuai Huang, Beijing Normal University, China Mitsuru Ikeda, JAIST, Japan Michael Jacobson, Nanyang Technological University, Singapore Akihiro Kashihara, The University of Electro-Communications, Japan Yong-Se Kim, Sungkyunkwan University, Korea Gee Kin Yeo, National University of Singapore, Singapore Kinshuk, Massey University, New Zealand Paul Kirschner, University of Utrecht, The Netherlands Piet Kommers, University of Twente, The Netherlands Fong-Lok Lee, The Chinese University of Hong Kong, Hong Kong Insook Lee, Sejong University, Korea Mun Kew Leong, Institute for Infocomm Research, Singapore Robert Lewis, University of Lancaster, UK Chee-Kit Looi, National Institute of Education, Singapore Tatsunori Matsui, Waseda University, Japan Gordon McCalla, University of Saskatchewan, Canada Naomi Miyake, Chukyo University, Japan Kiyoshi Nakabayashi, NTT Resonant, Inc., Japan Eugenia Ng, Hong Kong Institute of Education, Hong Kong Hiroaki Ogata, University of Tokushima, Japan Ron Oliver, Edith Cowan University, Australia Michael Sharples, University of Nottingham, UK

viii

Kathy Sinitsa, International Research and Training Center of Information Technologies and Systems, Ukraine Raymund Sison, De La Salle University, Philippines Amy Soller, Institute for Defense Analyses, Italy Gerry Stahl, Drexel University, USA Dan Suthers, University of Hawai’i at Manoa, USA Seng Chee Tan, Nanyang Technological University, Singapore Pierre Tchounikine, Université du Maine, France Qiong Wang, Peking University, China Lu Wang, Department of Educational Technology, Capital Normal University, China Yoneo Yano, Tokushima University, Japan Shelley Shwu-Ching Young, National Tsing Hua University, Taiwan Fu-Yun Yu, National Cheng-Kung University, Taiwan Shengquan Yu, College of Educational Technology, Beijing Normal University, China

ix

Additional Reviewers TSE Wing-Cheung Alex Liesbeth Baartman Ryan S.J.D. Baker Vincent Barre Moushir Bishouty Lars Bollen Chris Brook Christopher Brooks Ole C. Brudvik Jan M. Van Bruggen Murat Cakir Hyun Jin Cha Sung-Bin Chang Elizabeth S. Charles Elizabeth Charles Yu-Fen Chen Nancy Yu-Chien Chen Weiqin Chen Zhi-Hong Chen Emily Ching Winnie Chow Catherine Cleder Steve Corich Charlie Cox Nathanael Ng Hsien Ern Stuart Garner Giulia Gelmini Bas Giesbers Sabine Graf Yoshiaki Hada Kim Hagen Andreas Harrer Elizabeth Hartnell-Young Shinobu Hasegawa Toshihiro Hayashi Tomoya Horiguchi Chris Houser Gan Li Hui Yang Hui Sebastien Iksal Wai-Hung Ip Hyunju Jeung

Morris S.Y. Jong Laltitha Jonnavithula Ilkka Jormanainen Marco M.A. Kalz Toshinobu Kasai Fengfeng Ke Tomoko Kojiri Siu Cheung Kong Oskar Y.M. Ku Hidenobu Kunichika Pierre Laforcade Richard Chih-Hung Lai Yiu Chi Lai Joey Lee Youngmin Lee Michael Hung-Liang Lee Joey J. Lee Kai Ming Li Yanyan Li Calvin Liao Taiyu Lin Yimei Lin Chiu-Pin Lin Weng-Jeng Liu Joe Luca Wai Wing Ada Ma Noriyuki Matsuda Noboru Matsuda Kenji Matsuura Mark Mcmahon Takahiko Mendori Carrie E. Miller Hiroyuki Mitsuhara Akiyoshi Miyatake Yasuo Miyoshi Tomiaki Morikawa Martin Muhlpfordt Manabu Nakamura Nathanael H.E. Ng Trang Nguyen Youji Ochi Ryo Okamoto

Shuji Okamura Josephine Pelleu-Tchetagni Dominique Py David Raymond Valerie Renault Raul Morales Salcedo Johann W. Sarmiento Hong Khai Seng Junjie Shang Fawaz Shareef Hajime Shirouzu Kati Maekitalo-Siegl Oyvind Smestad Hyo-Jeong So Chiann-Ru Song Soonshik Suh Kazumoto Tanaka Philippe Teutsch Patricia Thornton Satoshi Togawa Ramon Prudencio S. Toledo Yuen Tsui Michael Verhaart Jen-Hang Wang Eric Wang Astrid Wichmann Lung-Hsiang Wong Winston Wu Jingyu Yang Cheng-Yen Yeh Yau Yuen Yeung Chengjiu Yin Eric Yu Fu-Yun Yu Fei Yuan Alan Zemel Yuejun Zhang Zhenhong Zhang Yan Zhang Guoqing Zhao Nan Zhou

x

Organizing Committee Organizing Committee Chair Ronghuai Huang, Beijing Normal University, China Tutorials Chair Li Chen, Beijing Normal University, China Workshops Chair Tak-Wai Chan, Graduate Institute of Network Learning Technology, Taiwan Workshop Local Organization Chair Huanglingzi Liu, Beijing Normal University, China Doctor Student Consortium Chair Lora Aroyo, Eindhoven University of Technology, The Netherlands Exhibition Chair ShengQuan Yu, Beijing Normal University, China Organizing Committee Secretariat Zheng Chen, Beijing Normal University, China Yan Dong, Beijing Normal University, China Ping Li, Beijing Normal University, China Qian Li, Beijing Normal University, China Jiaoyang Guo, Beijing Normal University, China Jingbin Zhang, Beijing Normal University, China Lanqin Zheng, Beijing Normal University, China

Sponsors Ministry of Education, the People’s Republic of China Chinese Association for Artificial Intelligence

xi

Contents Preface Riichiro Mizoguchi, Pierre Dillenbourg and Zhiting Zhu Program Committee

Additional Reviewers

Organizing Committee

v vii ix

x

Keynote Speeches The Design of Effective Simulation-Based Inquiry Learning Environments Ton de Jong

E-Learning Evolution: From M-Learning to Educational Semantic Web and Beyond Cui Guangzuo, Yang Gongyi, Chen Hu, Fei Chen and Guo Jiuling Designed Collaboration as a Scaffold for Schematic Knowledge Integration Naomi Miyake

Technology Affordances for Intersubjective Meaning-Making Daniel Suthers

3 7 15 21

Modeling and Representation Design of an Environment for Developing Presentation Skills Kazuhisa Seta and Mitsuru Ikeda

Ontological Modeling Approach to Blending Theories for Instructional and Learning Design Yusuke Hayashi, Jacqueline Bourdeau and Riichiro Mizoguchi A Case of Blending Learning in Computer Teaching –– The Model and the Application Li Cuiling, Hong Wang and Huiyu Zhang A Combined Method for Extracting Rules with Improved Quality Fuyan Liu

29 37 45 49

Programming The Impact of CABLE on Teaching Computer Programming Ioana Chan Mow, Wing K. Au and Gregory C.R. Yates

Problem Solving Process Oriented Diagnosis in Logic Programming Nguyen-Thinh Le and Wolfgang Menzel

55 63

xii

Programming Teaching Support System Using Student Model KeunWoo Han, EunKyoung Lee and YoungJun Lee

A Method for Creating Teaching Materials of Practical Object-Oriented Methods Education Izuru Kume, Naoya Nitta and Yasuhiro Takemura

71 75

Science Education at School Science Net: Effects of an e-Learning System on Elementary School Students’ Self-Regulated Learning in Science Classes Takeshi Kitazawa, Masahiro Nagai, Hiroshi Kato and Kanji Akahori Using Satellite Resources for Scientific Inquiry Beaumie Kim, Manetta Calinger and Debbie Denise Reese

Experimental Researches on Development of Pupils’ Advanced Cognitions in PRIME Environments Zhou Yueliang, Lianghui Wang and Xiuqin Lin

81 89 97

ITS, et al. Teaching Chinese Handwriting by Automatic Feedback and Analysis for Incorrect Stroke Sequence and Stroke Production Errors Kai-Tai Tang and Howard Leung

Developing a Practical Domain Knowledge Base and Problem Solving System for Intelligent Educational System of High School Chemistry Nana Ishima, Toru Ueda, Tatsuhiro Konishi and Yukihiro Itoh

The COLAC Model: Collaborative Paper-Writing in the Humanities Guillaume Schiltz and Andreas Langlotz

Experimental Investigation and Implementation of Support for Problem Generation by Presenting Cases Kazuaki Kojima and Kazuhisa Miwa A Computer-Based Environment for Learning by Problem-Posing as Sentence-Integration Tsukasa Hirashima, Takuro Yokoyama, Masahiko Okamoto and Akira Takeuchi

107 115 119 123 127

Discussion The Impact of Structured Discussion on Students’ Attitudes and Dispositions Toward Argumentation Khai Seng Hong, Ole C. Brudvik and Yam San Chee

Assessing the Impact of a Structured Argumentation Board on the Quality of Students’ Argumentative Writing Skills Ole C. Brudvik, Khai Seng Hong, Yam San Chee and Libo Guo

133 141

xiii

Incorporating Online Discussion in Classroom Learning: A New Strategy Wenli Chen and Chee Kit Looi

Using Agents for Enhancing Learning Effects in an Advanced Discussion Forum Yuejun Zhang, Kinshuk, Øyvind Smestad, Jingyu Yang and Lynn Jeffery

149 157

Emotion & Personality Research on Personality Mining System in E-Learning by Using Improved Association Rules Luo Qi, Yanwen Wu, Liyong Wan and Ying Yu

Analysis on Relationships of Emotional Transmissions Between Participants and Their Emotional Aspects in Communication Using Bulletin Board System Shogo Kato, Yuuki Kato and Kanji Akahori

Learning Nonverbal Emotion Interaction in 3D Intelligent Virtual Environment for Children Zhen Liu Development of Know-How Information Sharing System in Care Planning Processes Kaoru Eto, Tatsunori Matsui and Yasuo Kabasawa

167 171 175 179

CSCL Student Learning and Team Formation in a Structured CSCL Environment Nobel Khandaker, Leen-Kiat Soh and Hong Jiang

An Integrated Framework for Fine-Grained Analysis and Design of Group Learning Activities Seiji Isotani and Riichiro Mizoguchi

The Development of a Grouping System in a Collaborative Learning Environment Pao-Ta Yu, Yen-Shou Lai, Chia-Ming Liu and Jenq-Muh Hsu

Students’ Understandings and Attitudes Toward Group Learning: An Empirical Study Jianhua Zhao and David McConnell

The Effectiveness of Knowledge Building Through Computer Supported Collaborative Learning Among Elementary Students: A Case Study Wing Cheung Alex Tse, Fong Lok Lee and Yong Ou

Computer-Supported Content Analysis for Collaborative Knowledge Building in CSCL Jian Liao, Yanyan Li, Ying Zhou, Ronghuai Huang and Jingjing Wang

A Novel Web-Based Collaborative Learning Supporting System with Navigation Function Tian Chenyuan, Zuoliang Chen and Shigayeshi Watanabe

185 193 201 205 209 217 225

xiv

Towards Auto-Coding of Collaborative Interaction Texts Based on Maximum Entropy Approach Jian Liao, Ronghuai Huang, Yanyan Li, Jingjing Wang and Jing Leng

Web Based Collaborative Environment for Engineering Graphics Education Lianguan Shen, Mujun Li, Xiaodong Wang, Wei Zhao and J.J. Zheng Time-Based Self-Learning Support Using Collaborative Learning Process Masahide Kakehi, Tomoko Kojiri and Toyohide Watanabe

The Design of a Collaborative Learning Environment in a Mobile Technology Supported Classroom: Concept of Fraction Equivalence Siu Cheung Kong

Scientific Modeling of Technology-Mediated Collaborative Learning Processes Yau-yuen Yeung Exploring the Learning Effect of a Web-Based Learning Community on EMBA Students I-Fan Liu, Meng Chang Chen and Yeali Sun Development of a Discussion Board System Designed for the Group Discussion That Includes Peer-Review Process Shigeru Sasaki and Hiroyoshi Watanabe

229 233 237 241 249 257 261

Interface Improving Creativity for Mathematical Problem Solving Using Web-Based Multimedia Whiteboard System Wu-Yuin Hwang, Nian-Shing Chen, Jian-Jie Dung and Yi-Lun Yang

Developing a VR-Based Projectile System Using Haptic Device for Learning Physics Atsushi Kanbe, Yukihiro Matsubara, Noriyuki Iwane and Kimiko Hirayama Proposal for Digital Partners Project Chris Davies and Jingjing Zhang

267 275 283

Assessment and Evaluation Development of Portfolio Assessment Support System Yasuhiko Morimoto, Isao Kikukawa, Maomi Ueno, Setsuo Yokoyama and Youzou Miyadera

Peer-Assessment in Web-Based ePortfolios System: An Experimental Study Youmei Wang Construction and Performance Evaluation of High Quality Curriculum Integrated with Information Technology YouRu Xie and Rui Yin

Effectiveness of WebQuest in the Teaching of STS in Secondary Biology Ka-leung Tse and Sai-wing Pun

289 297 305 309

xv

Mobile and Web-Based Learning Effects of Using Digital Contents Designed for PDA as a Teaching Aid in an Observational Learning of Planktons for Fieldworks on a Ship Hitoshi Miyata and Mitsuo Ishigami

A Study of Message Reading Efficiency of Color Screen Mobile Phones Xuemin Zhang, Bo Wang, Lilin Rao, Bin Yang, Yongna Li and Xueming Lu

The Design of a Web-Based Learning Platform: A Case Study in Taiwan I-Fan Liu, Meng Chang Chen and Yeali Sun

A Study of Implementing Web-Based Learning Systems to Enhance Learning for the Supply Chain Management (SCM) Course in Higher Education I-Fan Liu and Shelley S.-C. Young

What Is Expected of a Facilitator in a Virtual Learning Environment? Ni Chang

315 323 327 331 335

Social Networking & Blog Proactivity, Autonomy & Social Networking: Transitional Environments for the Japanese Educational Context Deborah C. Turk and John W. Brine

341

Effects of Peers Interactivity and Self-Regulated Learning Strategies on Learning Art Appreciation Through Weblog Sau Hung Cheung and Percy Lai Yin Kwok

349

Understanding Asynchronous Teaching and Learning Dialogues – An Integrative Approach E. Vass, F. Concannon, M. LeVoi, K. Littleton and D. Miell

357

An Application of Social Network Analysis in Evaluation of CSCL Yonggu Wang and Kedong Li

353

Cultural Issues Participatory Agent-Based Gaming Methodology in Cross-Cultural Education: Exploring Efficient and Sustainable Civil Society and Community Reiko Hishiyama and Toru Ishida Internet for Senior Citizens in China: Survey and Proposal Wei Zhou, Takami Yasuda and Shigeki Yokoi

Why Do Students Engage in e-Learning: A Chinese Perspective Zhenhong Zhang and Ronghuai Huang

Enabling a Multilateral Distance Class Between China, Korea and Japan: Effective Utilization of Networking Technologies Yuri Nishihori, Keizo Nagaoka, Nozomu Nishinaga, Kenji Tanaka, Yuichi Yamamoto, Haruhiko Sato, Masahiro Harada, Ruimin Shen, Jinjin Feng and Myunghee Ju Kang

363 371 379 383

xvi

Content and Knowledge Management Digital Video Database: Supporting Student Teachers’ Learning About Teaching During Teaching Practice Winnie So, Vincent Hung and Walker Yip

Building and Evaluation of a Semantic Web System That Provides Teachers with Lesson Plans Toshinobu Kasai, Haruhisa Yamaguchi, Kazuo Nagano and Riichiro Mizoguchi An Improved Learning Content Management System Framework Liyong Wan, Chengling Zhao, Ming Zhao, Luo Qi and Libing Jiang Resource Based Solution to Teachers’ Knowledge Management Sun Hongtao, Lu Wang and Hongwei Dai

Collaborative Building of Japanese Kanji Pronunciation Database for Learning Japanese by Chinese Fei Yuan, Jing Yuan, Rong Wang, Hiroyuki Mitsuhara, Kazuhide Kanenishi and Yoneo Yano

Using IT to Power and Support Problem-Based Engaged Learning Wee-Meng Hoe and Irene Tan

389 397 405 409 413 417

IT at School & Teacher Training A Study of Innovative Uses of ICT in Primary Education Shelley S.-C. Young and Hsin-Ho Ku

Impacts of Grade 7-9 Students’ Computer Usage After School on Academic Achievement: A School Case Study Yiu Fai Wong and Percy Lai Yin Kwok Using Interactive Whiteboards (IWB) to Enhance Learning and Teaching in Hong Kong Schools Fong-lok Lee, Sai-wing Pun, Sandy Siu-cheung Li, Siu-cheung Kong and Wai-hung Ip

Conditions Facilitating the Implementation of Information Communication Technology Integration in Malaysian Smart School Wan Zah Wan Ali, Hajar Mohd Nor, Azimi Hamzah and Nor Hayati Alwi A Study of the Present Status of IT Teachers Training in Microsoft’s ‘Partners in Learning’ Project –– A Content Analysis Approach GuiJing Huang, Yong Xu, JinBao Zhang and XiaoYuan Wang

inPD: An Emerging Theoretical Framework for Educational Professional Development in the Information Age Simon Hughes

423 431 439

443

447 451

xvii

Reflection and Self-Directed Learning Automated Mentoring for Reflection in an Eportfolio Tzemin Chung, Mun Kew Leong and Joel P.L. Loo

Guided Map for Scaffolding Navigation Planning as Meta-Cognitive Activity in Hyperspace Akihiro Kashihara, Mitsuyoshi Nakaya and Koichi Ota Self-Directed Learning in Technology Supported Project Work Allan H.K. Yuen and Liping Deng

Multi-Step Annotation to Promote Reflective Learning with a Mobile Phone N. Gotoda, K. Matsuura, K. Kanenishi, K. Niki and Y. Yano

457 465 473 477

Game and Edutainment Property Exchange Method for Automatic Generation of Computer-Based Learning Games Takanobu Umetsu, Tsukasa Hirashima and Akira Takeuchi Development and Validation of an Animation-Based Test in the Area of Earth Sciences Huang-Ching Wu and Chun-Yen Chang

Weaving Pedagogy into Gaming: Learning Design Principles for Developers Yam San Chee, Yi Liu and Khai Seng Hong VR Edutainment Material Interlude for Dynamics Experiment and the Development Platform Prelude Yuma Hanafusa, Takashi Inoue, Hiroyuki Tominaga, Toshihiro Hayashi and Toshinori Yamasaki Using the “Record-Replay” Function for Elaboration of Knowledge in Educational Games Junjie Shang, Morris S.Y. Jong, Fong-Lok Lee, Jimmy H.M. Lee, Marti K.H. Wong, Eric T.H. Luk and Kevin K.F. Cheung

483 491 495 499

503

Participation/Attitude Toward Learning Investigating Learner Autonomy Toward e-Learning Shu-Sheng Liaw and Hsiu-Mei Huang

Moderating Role of Online Self-Efficacy in Relation Between Learning Strategy and Online Performance Huamao Peng, Ying Wang and Ronghuai Huang

Understanding E-Learners’ Characteristics and Performance in Online Courses Rowena Santiago and Minoru Nakayama An Exploratory Study on Teachers’ Perceptions of Game-Based Situated Learning Morris S.Y. Jong, Junjie Shang, Fong-Lok Lee, Jimmy H.M. Lee and Huk-Yuen Law

509 517 521 525

xviii

Visualization Visualizing Errors for Self-Correcting Discrepancy Between Thinking and Writing Hidenobu Kunichika, Tsukasa Hirashima and Akira Takeuchi

Using Systematic Animation to Teach Dynamic Science Concepts Othman Talib, Christirani Azhar Shah and Nabila Abdullah

Creating Animations in SVG Format for Visualizing Program Execution Koji Kagawa

Effects of the Voice Recognition on the Writing of Students with Learning Disabilities Hu Lailin

535 543 551 555

Curriculum Designing a Teacher Professional Knowledge Base and Its Operation Model Based on School-Based Curriculum Development Yih-Ruey Juang, Tak-Wai Chan and Tzu-Chien Liu

561

Development of a Photo Management System in Schools Which Ensures Students Appear Equally Kyoko Umeda, Shinsuke Takito, Tetsuro Ejima and Hironari Nozaki

577

Dynamic Composition of Curriculum for Personalized E-Learning Yanyan Li and Ronghuai Huang

569

Sense-Making and Facilitation Probing Technology as Affordances for Negotiating Meaning in the Elementary Science Classroom –– A Participation Perspective Fei-Ching Chen, Huo-Ming Jiang, Jie-Chi Yang and Yu-Wei Lee

Analysis of Meaning Making in Online Learning Daniel Suthers, Nathan Dwyer, Richard Medina and Ravi Vatrapu

Facilitating Knowledge Construction by Providing Individualized Services Weidong Pan and Igor Hawryszkiewycz

The Role of On-Line Facilitators: Types of Collaborative Skills for Effective E-Learning Activities Tengku Putri Norishah Tengku Shariman and Habibah Abdul Jalil

587 595 603 611

PBL and Test Fostering Project-Based, Active Learning Through Use of Technology Teresa Bader and Teh-yuan Wan Framework for Problem-Solving Based Learning in Nursing Domain – A Practical Study – Yukie Majima, Yoichiro So and Kazuhisa Seta

621 625

xix

A Polytomous Computerized-Adaptive Testing That Rewards Partial Knowledge Yung-Chin Yen, Rong-Guey Ho and Li-Ju Chen

Research on Algorithm of Computer Adaptive Test Using Optimized MDPLTM Jin-Ling Li, Feng-lin Wang and Wang-Xiu Li

629 637

Doctor Student Consortium (DSC) Papers Cognitive Maps-Based Student Model Alejandro Peña, Humberto Sossa and Agustín Gutiérrez

643

Improving Quality of Online Forum Interactions in Distance Higher Education Zhenhong Zhang and Ronghuai Huang

645

Annotation in Information Research for Decision Making Robert Charles and David Amos

649

Applying Weighted Learning Object to Build Adaptive Course in E-Learning Anh Nguyen Viet and Dam Ho Si

Effects of the Use of Graphic Calculators on Cognitive and Metacognitive Domains in Teaching and Learning of Mathematics N. Mohd. Tajudin, R. Ahmad Tarmizi, W.Z. Wan Ali and M.M. Konting Improve Effectiveness of Dialogue in Learning Communities Jingyu Yang and Kinshuk

Validation of the Mathematics Courseware Usefulness Evaluation Instrument S.A. Noraidah, A.G. Abdul Azim, S. Hasan and M.Y. dan Aida Suraya Expert Tutoring and Natural Language Feedback in Intelligent Tutoring Systems Xin Lu Learning Environment for Designing Physics Experiment: DEEP Takahito Toumoto, Tomoya Horiguchi, Tsukasa Hirashima and Akira Takeuchi

647

651 653 655 657 659

Standards, Adaptation & Pedagogy: Quality Assessment in e-Learning Silvia Sanz-Santamaría, Julián Gutiérrez Serrano and José A. Vadillo Zorita

661

Author Index

663

This page intentionally left blank

Keynote Speeches

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

3

The Design of Effective Simulation-Based Inquiry Learning Environments Ton de Jong Faculty of Behaviorial Sciences, University of Twente, The Netherlands [email protected] Abstract: Modern theories on learning and instruction call attention to learning environments that create constructivistic, situated, and collaborative learning experiences. Simulations offer specific features that enable self-directed, highly autonomous, high interaction learning. First, learning in these environments differs from learning in expository environments in that it puts a higher emphasis on inquiry processes such as hypothesis generation and testing and on regulative processes such as planning and monitoring. Second, these environments offer specific opportunities to situate learning in realistic settings, but they also offer the possibility to adapt reality to support learning. Third, inquiry learning presents opportunities for negotiation and collaboration. This presentation will set out characteristics of simulations discuss characteristic inquiry processes and associated problems, and examine what is needed to design effective inquiry learning environments.

Introduction New types of (on-line) learning environments are becoming available for use in the actual classroom rapidly. Development that nowadays dominate the field of learning and instruction are constructivism, situationism, and collaborative learning. More specifically, we can say that the new view on learning entails that students are encouraged to be active in construcingt their own knowledge, in realistic situations, together with others. Technology plays a major role in implementing these new developments in education. Constructivism is supported by computer environments such as hypertexts, concept mapping, simulation, and modeling tools (see de Jong & van Joolingen, 1998), realistic situations can be brought into the classroom by means of for example video or even virtual reality, and collaborative learning is supported in Internet based learning environments such as Co-Lab (van Joolingen, de Jong, Lazonder, Savelsbergh, & Manlove, 2005). In this presentation I will focus on the riole of computer simulations in relation to inquiry learning.

1. Characteristics of computer simulations Computer simulations are programs that hold a computable model of some kind of reality. Students can manipulate the simulated reality by changing values of input variables, and the output is usually displayed in several formats (e.g., animations, graphical displays, and numbers). These characteristics make simulations well suited for implementing the trends above. First, simulations elicit a learning process that is called inquiry learning. In inquiry learning, a domain is not directly offered to students; rather students have to induce characteristics of the domain from experiences or examples. Inquiry learning can be defined as an approach to learning that involves a process of exploration, that leads to asking

4

T. de Jong / The Design of Effective Simulation-Based Inquiry Learning Environments

questions and making discoveries in the search for new understandings (National Science Foundation, 2000). This is a learning approach that is in line with contructivistic principles. Second, simulations can easily be used to introduce realities in the classroom; interfaces of simulations may mimic any reality. Third, inquiry learning is very appropriate for collaborative learning, as is real, scientific, inquiry since it requires students to make decisions along the way (e.g., which hypothesis to test) that are good anchor points for knowledge exchange and negotiation (Gijlers & de Jong, 2005).

2. Inquiry learning Though simulations seem to be able to take a central role in realizing the above mentioned trends in education, students do have considerable trouble in realizing an effective inquiry process. Knowledge about the inquiry process and the related problems students experience may help to design adequate cognitive scaffolds.

2.1 Inquiry learning processes Although there may be some variations, for example in the way data are gathered (e.g., from experimentation or from data sets), and variations in the complexity of the experimentation, there is a fair consensus about which processes basically comprise inquiry learning. The different classifications in the literature differ mainly in their granularity, ranging from very detailed to rather broad, but basically do not differ in the processes that are distinguished. In de Jong (2006b), I introduced a set of learning processes that form a suitable basis for describing inquiry learning: In orientation, the student makes a broad analysis of the domain; in hypothesis generation, a specific statement (or a set of statements, for example, in the form of a model) about the domain is chosen for consideration; in experimentation, a test to investigate the validity of this hypothesis or model is designed and performed, predictions are made and outcomes of the experiments are interpreted; in conclusion, a conclusion about the validity of the hypothesis is drawn or new ideas are formed; and, finally, in evaluation, a reflection on the learning process and the domain knowledge acquired is made.

2.2 Problems that students experience In a review of research, de Jong and van Joolingen (1998) concluded that students may have serious problems with all of the above mentioned inquiry learning processes. In general, students may have trouble stating hypotheses, designing experiments, and interpreting data; they often do not engage in overall planning and do not adequately monitor what they have been doing (de Jong & van Joolingen, 1998). These inquiry process problems may be associated with wrong mental models of systems in general (Kanari & Millar, 2004; Kuhn, Black, Keselman, & Kaplan, 2000) and it may lead to a misinterpretation of experimental outcomes from the experiments that were performed in the inquiry (Chinn & Brewer, 1993). On this basis and also based on overall research in inquiry learning many researchers, therefore, conclude that students need guidance in the discovery process (de Jong, 2006a; Mayer, 2004).

T. de Jong / The Design of Effective Simulation-Based Inquiry Learning Environments

5

2.3 Providing students with cognitive scaffolds In the software students can be supported with cognitive tools or cognitive scaffolds to ensure an effective inquiry learning process. An example of this is providing students with assignments. These assignments help to students in their planning activities and they help to focus on relevant aspects of the simulation (van Joolingen & de Jong, 2003). Another example is a monitoring tool. A monitoring tool helps students to save all their experiments, to re-order, and to replay them. A hypothesis scratchpad offers students elements (variables, relations, conditions) for composing hypotheses. Students can also be provided with hints on how to experiment, or on how to reflect over the knowledge that is acquired. Another way to support students is to offer them just-time background information or explanations. Finally, the inquiry process can be subdivide in several phases and for every phase students can be offered a specific structure to work in. Extensive overviews of cognitive tools or cognitive scaffolds can be found in Quintana et al. (2004) and de Jong (2006b).

3. Conclusion Large scale environments show that inquiry learning based on simulations can be an effective learning process (Hickey & Zuiker, 2003; Ketelhut, Dede, Clarke, & Nelson, 2006; White & Frederiksen, 1998) . However, the inquiry process needs to be scaffolded to reach these results. This scaffolding can be offered in the software, but could also be offered by the teacher or a co-learner. Combined with more realistic interfaces simulations can indeed offer educational opportunities that combine contructivist, collaborative, and situational characteristics. For sure, to give inquiry learning a place in the curriculum a balance needs to be found between inquiry learning and other ways of learning and instruction so that an integrated, attractive, and effective curriculum results.

References Chinn, C. A., & Brewer, W. F. (1993). The role of anomalous data in knowledge acquisition: A theoretical framework and implications for science instruction. Review of Educational Research, 63, 1-51. Cognition and Technology Group at Vanderbilt. (1997). The Jasper project; Lessons in curriculum, instruction, assessment, and professional development. Hillsdale (NJ): Lawrence Erlbaum Associates. de Jong, T. (2006a). Computer simulations - Technological advances in inquiry learning. Science, 312, 532-533. de Jong, T. (2006b). Scaffolds for computer simulation based scientific discovery learning. In J. Elen & R. E. Clark (Eds.), Dealing with complexity in learning environments (pp. 107-128). London: Elsevier Science Publishers. de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68, 179-202. Gijlers, H., & de Jong, T. (2005). The relation between prior knowledge and students’ collaborative discovery learning processes. Journal of Research in Science Teaching, 42, 264-282. Hickey, D. T., & Zuiker, S. (2003). A new perspective for evaluating innovative science learning environments. Science Education, 87, 539-563. Kanari, Z., & Millar, R. (2004). Reasoning from data: How students collect and interpret data in science investigations. Journal of Research in Science Teaching, 41, 748-769. Ketelhut, D. J., Dede, C., Clarke, J., & Nelson, B. (2006). A multi-user virtual environment for building higher order inquiry skills in science. Paper presented at the American Educational Research Association, San Francisco. Kuhn, D., Black, J., Keselman, A., & Kaplan, D. (2000). The development of cognitive skills to support inquiry learning. Cognition and Instruction, 18, 495-523. Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery learning? American Psychologist, 59, 14-19.

6

T. de Jong / The Design of Effective Simulation-Based Inquiry Learning Environments

National Science Foundation. (2000). An introduction to inquiry. In Foundations. Inquiry: Thoughts, views and strategies for the K-5 classroom. (Vol. 2, pp. 1-5). Quintana, C., Reiser, B. J., Davis, E. A., Krajcik, J., Fretz, E., Duncan, R. G., et al. (2004). A scaffolding design framework for software to support science inquiry. The Journal of the Learning Sciences, 13, 337-387. van Joolingen, W. R., & de Jong, T. (2003). SimQuest: Authoring educational simulations. In T. Murray, S. Blessing & S. Ainsworth (Eds.), Authoring tools for advanced technology educational software: Toward cost-effective production of adaptive, interactive, and intelligent educational software (pp. 1-31). Dordrecht: Kluwer Academic Publishers. van Joolingen, W. R., de Jong, T., Lazonder, A. W., Savelsbergh, E., & Manlove, S. (2005). Co-Lab: Research and development of an on-line learning environment for collaborative scientific discovery learning. Computers in Human Behavior, 21, 671-688. Vreman-de Olde, C., & de Jong, T. (2006). Scaffolding the design of assignments for a computer simulation. Journal of Computer Assisted Learning, 22, 63-74. White, B. Y., & Frederiksen, J. (1998). Inquiry, modelling, and metacognition: making science accessible to all students. Cognition and Instruction, 16, 3-118.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

7

E-learning Evolution: From M-learning to Educational Semantic Web and Beyond Cui Guangzuo, Yang Gongyi, Chen Hu, Chen Fei, Guo Jiuling Modern Education Technology Center, Peking University, China [email protected] Abstract: An e-learning model with ICT technology (ELM) is proposed in this paper. With this model, some education technologies and e-learning evolution are interpreted, such as network education, mobile education, ubiquitous education, educational semantic web, and etc. In the meantime, the way of how to combine new ICT technologies into education is also demonstrated. After discussing the convenience and challenge of various education technologies, a new model called intelligent education is introduced and some recent research results are presented. At last, the author looks ahead the future of information technology and human related disciplines and their effects on education. Keywords: E-learning, M-learning, Educational Semantic Web, Intelligent Education

1. E-learning Model with ICT Technology E-learning means the education technology enhanced by ICT technology. With the development of new ICT technologies, e-learning has been improved continuously, and as a result, new education technologies emerged and named after the corresponding ICT technology, such as network education, mobile education, ubiquitous education and educational semantic web. What will the ICT technology tend to and what will be the next education technology? To deal with this, a new model for e-learning technology is proposed, depicted as figure1. In ELM model, ICT technology enhances education in the following way: information technology is used to design and present course content, and communication Figure1 E-learning Technology Model technology is used as an interaction method among educate, Educatee and educational agent. With the information technology’s evolution from data model to information model and now to knowledge representation, and communication technology’s evolution from wire to wireless and now to mobile communication, the e-learning technology evolutes from CAI, network education, mobile education, ubiquitous education to educational semantic web and so on. In the following, these e-learning technologies are demonstrated.

8

C. Guangzuo et al. / E-Learning Evolution: From M-Learning to Educational Semantic Web

2. From M-learning to Educational Semantic Web Network Education: with fixed wired network and information model. Mobile Education (also named m-learning) [1-6]: Courseware is designed with data model and information model; communication with mobile network. So, the interaction can be completed anywhere and anytime with text message. On the other hand, the courseware can not be accessed by some devices with different contexts, such as different screen size. So, in m-learning, not all the devices can access the published course content. Ubiquitous Education (also called u-learning): is almost the same as m-learning except Figure2 Portal for PC Screen that in u-learning universal information access (UIA) is required [19]. UIA means that the published courseware can be accessed anywhere, anytime, any network and by any device. UIA has been one unresolved key problem in ubiquitous computing which is also called device independence. The related factors which affect UIA include input device, output device, network type, device screen, OS type, application tool, human preference, dynamic environment, unpredictable action, and etc. To demonstrate the difficulty of UIA, we give an explanation. Figure2 is an education portal designed for PC which looks very well with PC screen. But what about is it when accessed with PDA or Hand Hold? It is terrible. So what kind of resource should course content be for UIA? Let’s look at an example of figure3.

Figure3 Adaptation of SCORM Resource

Figure3 represents a part of courseware resource with SCORM standard. What to present when a device accesses this course at first time? The well solution is that the content of first screen should depend on, at least, the screen size. Figure4 Adaptation with Resource Tree That is, with different screen size, the presented content should match with that size. The above problem is called content adaptation. How to implement content adaptation? This can be demonstrated in figure4 [7]. From figure4, we can see that with resource tree representation, the proper content size can be selected according to the screen size. But the candidate resource sets maybe more than one, which node set should be selected? At the worst case, the selected content may be has none meaning. So, to produce the proper content, other constraints should be

C. Guangzuo et al. / E-Learning Evolution: From M-Learning to Educational Semantic Web

9

confined, that is, the selected resource node set should be a meaningful block in logical. How to guarantee that? It should refer to educational semantic web. Educational Semantic Web: Educational Semantic Web (ESW) [9-12] is a new education technology enabled by semantic web technology [8]. In ESW, the education content and activity should be annotated with well-defined meaning which is understandable to human and machine. The well-defined meaning is represented with a special data model, called education ontology [14-16]. In education ontology, the concepts and their relations in course content and education activity are defined in a formal method called description logics. With these logical defined educational resources, intelligent education application can be developed by reasoning with logic programming and rules. The educational semantic web diagram is depicted as figure5 and a draft educational ontology is as figure6 (proposed by author’s group).

3. Intelligent Education With the research and development of ICT, new technologies will come out, and where should education technology go? From the above statement, we can see that e-learning technologies are developed for education practice, that is, only for educate and educatee. It is natural to think that ICT technology should not be confined only for education practice. The problem is: how does ICT technology provide support to education research (theory and method)? Is it possible? And a more challenge problem is: are there any other technologies or theory, except for ICT, that can be used to enhance education? Of course, these are human related disciplines, such as brain science, cognition science, knowledge science, logics, philosophy, and etc. The evolution of education technology with human related disciplines and ICT technology is depicted as figure7. In figure7, D-Com stands for data communication, D-Process stands for data process and K-reasoning stands for knowledge reasoning. From figure7, we see that the

10

C. Guangzuo et al. / E-Learning Evolution: From M-Learning to Educational Semantic Web

development trend of information technology is knowledge representation and reasoning (KRR). With KRR, intelligent applications can be developed. On the other hand, from the viewpoint of application, domain users can also model and develop applications without relying on software engineers. In other words, education domain users include educate, educate, researcher, and other ones who are engaged in education. So, as the development of technology, not only education practice, but also education theory can also be enhanced by ICT in an intelligent way. At the meantime, evolution roadmap of ICT is very like the cognition process for human being to learn knowledge, depicted as figure8. As figure8 indicates, the cognitive process is as follows: collect large amount of data; select useful data (information) from large data; extract knowledge about a concept from synthesizing multiple aspects of information and setting up connection with other concepts; the knowledge is kept in long-term store. At the meantime, data selection and information synthesis are controlled by thinking and reasoning with knowledge originated from motivation. On the other hand, with the information model’s evolution from data model, information model to knowledge model, the course model changes from digital course, multimedia course to knowledge representation course. All of these reflect that the development process of information technology is almost consistent with the cognition process. Figure7 also indicates that, besides ICT, human related disciplines are also an important force to improve education. From viewpoint of history of thousands of years, human discipline plays an important role to education. Especially, in recent years, with the development of human related disciplines, we understand human mind more and more deeply, such as how to understand, how to learn, how to think, and etc. As a result, the human related achievements provide human a better way to understand themselves, to guide human to learn and to educate.

Figure9 Intelligent Education Architecture

In a word, with the rapid development ICT technology and human related disciplines, education will be enhanced in an intelligent way! Figure9 demonstrates more details of this intelligent way which called Intelligent Education Architecture (IEA) [17]. Figure9 indicates that IEA include five parts, they are Human Related Disciplines (HRD), Intelligent ICT (IICT, include Information Science, AI, Knowledge Engineering), Education Theory Research (ETR, includes Education Method), Education Practice and Intelligent Education Domain Engineering. Relations among these five parts are as follows:

C. Guangzuo et al. / E-Learning Evolution: From M-Learning to Educational Semantic Web

11

(1) Education Theory Research is supported by Human Related Disciplines, Intelligent ICT and Intelligent Education Domain Engineering. (2) Intelligent Education Domain Engineering is developed with the guide of Education Theory Research and Intelligent ICT. (3) Optimized Education Practice can make teaching process more efficient and the Human Related Disciplines can progress more rapidly. (4) All education activities are supported by Intelligent Education Domain Engineering. In the proposed Intelligent Education, all parts of education will form an ecosystem where every part in education will be enhanced by others. More details refer to [17].

4. Some Researches in Knowledge Science and Intelligent Education Laboratory at Peking University Knowledge Science and Intelligent Education Laboratory at Peking University (KSIE) aim to build an education ecosystem with Intelligent Education Architecture [17]. Such a system will provide scientific education research method and scientific education practice˄includes teaching and learning˅. With this system, the efficiency of education theory and method will be predicted by intelligent simulation or formal verification, not like the nowadays experiment with real person. The goal is good, but implementing such a system depends on a great progress of Human Related Disciplines and Intelligent ICT where the learning principle is understood and the proper model of learning nature is constructed. Indeed, it needs a lot of work and a long time, but we are doing something to proceed to the goal. Some researches on this work in recent years are presented as below.

4.1 Education Software Automation based on Education Ontology This means that, with educational ontology, we can develop an infrastructure platform to generate application system in terms of user requirements. Such an infrastructure is proposed as figure10, more details refer to [17].

4.2 Educational Semantic Web Service Composition Model Semantic Web Service (SWS) technology [13, 22, 23, 24] provides a good choice to realize education software automation. With SWS, every education service is annotated with well-defined meaning which can be understood by

12

C. Guangzuo et al. / E-Learning Evolution: From M-Learning to Educational Semantic Web

human and machine [18, 21]. With large amount of educational semantic web services available on www, we can compose some of them according to requirements and generate a realistic system. The above process is called SWS composition [18, 21, 25]. Figure11 is a proposed SWS composition model for education by our group which is called OntoComposer [21]. Figure12 is an implementation and its operation interface of OntoComposer, the labeled numbers in figure12 represent the process to compose a new service with available web services, details refers to [26].

4.3 Ontology-based Search Engine Search engine is an efficient tool to retrieve information from large amount of web resources. With semantic web, web resources are annotated with well-defined meaning understandable to human and machine. From figure5, with educational semantic web, educational ontology provides formal semantic description for educational resources which can be understood by machine. In this way, search engine can understand web resource and select information more efficiently (depicted as figure13), more details refer to [20].

4.4 Ontology-based M-learning Platform One problem in m-learning is context aware application. That is, the presentation of courseware content should be adaptive in terms of the user context. The user context includes device feature and user preference. When user accesses education server, the server selects proper content and style, and returns it to client. In this way, the courseware could be accessed by any device. Figure14 is a proposed design for m-learning with web service. The context-aware adaptation works as following: (1) at sever end, the e-learning

C. Guangzuo et al. / E-Learning Evolution: From M-Learning to Educational Semantic Web

13

system is composed of simple services which include operation and resource individually. (2) When server receives access request, it groups some simple services into one page in terms of user context automatically and returns it to client. The adaptive presentations are depicted as figure15. More details refer to [6].

5. Conclusions The advance of ICT would enhance education continuously. E-learning evolution with ICT indicates that M-learning provides a convenient interaction, educational semantic web provides a machine understandable description of education resource, and ontology model provides a knowledge representation model of course content, which will improve cognitive process for learn content in nature. Besides ICT, human related disciplines have become an important force to enhance education recently. A kind of education technology to combine ICT and human related disciplines into education and form a scientific ecosystem education system will be the next generation technology, which is called Intelligent Education.

References [1] Cui Guangzuo, etc. Mobile Education: A New Direction of Education Technology. Ceta Annual Conference of China. 2001.12. [2] Cui Guangzuo, etc. Mobile Virtual Campus: Design and Implementation. China Distance Education. 2002.12. [3] Cui Guangzuo, etc. Study and Implementation of SMS-based Mobile Education Architecture. GCCCE2002, Beijing. [4] Cui Guangzuo, etc. MVClass: Mobile Virtual Class for Open and Distance Education.International Conference of Distance Education " 2nd AEARU Workshop on Network Education”, Taiwan.2003.12 [5] Dong Sheqin, Yue Weining, Cui Guangzuo And Wang Guoping. A Educational Information Platform based on Mobile Device. Computer Application of Chine. 2004.V24.11, Pp142-145. [6] Cui Guangzuo, etc. A Study of Concept and Key Technologies of Mobile Education. China Distance Education. 2005.9. [7] Cui Guangzuo. WebUnify: An Ontology-based Web Site Organization and Publication Platform for Device Adaptation. SNPD2004 International Conference, Beijing. 2004.7. [8] Tim Berners-Lee, James Hendler and Ora Lassila. The Semantic Web. Scientific American 2001.5. [9] Lora Aroyo, Darina Dicheva. The New Challenges for E-learning: The Educational Semantic Web. IEEE Journal of Educational Technology & Society,2004.10.Pages 59-69. [10] Emanuela Moreale, Maria Vargas-Vera. Semantic Services in e-Learning: an Argumentation Case Study. IEEE Journal of Educational Technology & Society,2004.10.Pages 112-128. [11] Terry Anderson and Denise Whitelock The Educational Semantic Web: Visioning and Practicing the Future of Education. Journal of Interactive Media in Education, UK, 2004 (1). [12] Kendall Clark, Bijan Parsia and Jim Hendler Will the Semantic Web Change Education? Journal of Interactive Media in Education, UK, 2004 (1). [13] Sheila A. McIlraith, Tran Cao Son, and Honglei Zeng. Semantic Web Services. IEEE Intelligent Systems, MARCH/APRIL 2001. Pp46-53. [14] Cui Guangzuo. OntoEdu: Ontology based Education Grid System for e-learning. Fifth International Agricultural Ontology Service (AOS) Workshop, Invited report.2004.5. [15] Cui Guangzuo, etc. OntoEdu: Ontology based Education Grid System for e-learning. GCCCE2004,HongKong

14

C. Guangzuo et al. / E-Learning Evolution: From M-Learning to Educational Semantic Web

[16] Cui Guangzuo, etc.OntoEdu: A Case Study of Ontology-based Education Grid System for e-Learning . The Official Journal of Global Chinese Society FOR Computers in Education,2004,pp59-72 [17] Cui Guangzuo, etc. Architecture Study of Intelligent Education. Keynote Speaker. GCCCE2006, Beijing. [18] Yang Li-na. A Kind of Task Composition Method Based on Education Ontology. Master Thesis, Peking University, 2004.7. [19] Cui Guangzuo. Application and Challenge of education under Ubiquitous Environment. Ubiquitous Forum of Information Industry Ministry in China. 204.11. [20] Xiao Hong. Study on the Semantic of SCORM based on Knowledge Organization & Retrieval. Master Thesis, Peking University, 2006.7. [21] Liu Yang. A Kind of Ontology based Hybrid Semantic Web Service Composition: Mechinsm and Implementation. Master Thesis, Peking University. 2006.7. [22] Sheila A. McIlraith, Tran Cao Son, and Honglei Zeng,Semantic Web Services. IEEE Intelligent Systems.MARCH/APRIL 2001.pp46-54. [23] Sheila A. McIlraith,David L. Martin. Bringing Semantics to Web Services. IEEE Intelligent System,JANUARY/FEBRUARY 2003.pp90-94. [24] Massimo Paolucci.Autonomous Semantic Web Services. IEEE Intelligent Computing. SEPTEMBER • OCTOBER 2003. pp34-42. [25] Brahim Medjahed, Athman Bouguettaya, A Multilevel Composability Model for Semantic Web Services. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 17, NO. 7, JULY 2005.pp954-969. [26] Cui Guangzuo. OntoComposer: Development Model and Implementation of Semantic Web Service based on Domain Ontology. To be published.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

15

Designed Collaboration as a Scaffold for Schematic Knowledge Integration Naomi Miyake School of Information System and Technology, Chukyo University, Toyota, JAPAN [email protected] Abstract: Rapid change in modern society requires higher levels of learning such as the acquisition of adaptive or “schematic” knowledge. Rather than the efficiency of simply applying what one has learned, the schematic knowledge acquisition emphasizes portability, sustainability, and dependability of learning outcomes. Schematic knowledge is expected to allow the learners to apply them to solve the wider scope of similar problems, as well as to identify new problems and create new solutions. We have been developing and testing college level learning environments to enhance the acquisition of such schematic knowledge in the domain of cognitive science, by heavily relying on understandings of how and why collaborative reflection benefits learning. In the two-year curricular we have developed, the students are first introduced to the notion of schematic learning by experiencing their own formation of schemata, and then are guided to reflect upon the process, through carefully designed collaborative activities. They will also be encouraged to integrate their experiences to technical literature through collaborative discussion in a dynamically arranged jigsaw variations. I will report on the theoretical bases of our practice, concrete learning activities, technological supports, and some results of the evaluative analyses of the learning processes and the outcomes. Keywords: College level collaborative learning, schema formation, knowledge integration

Introduction: Emerging new learning goals Studies of how people could effectively learn have a long tradition because human beings have constantly been required to re-structure old experiences to accommodate new situations. However, in the current age, where rapid changes are the norm, the degree to which this requirement occurs has increased more than ever. Students, for example, are required not only to acquire routine knowledge but also to apply what they learn to new situations, outside of school and in a distant future. The outcomes are expected to be ‘portable’ so that the learners can bring them to apply problems outside of classrooms; the learning outcomes are also expected to be ‘sustainable’ so that the learners can build on them to keep strengthening their intellectual skills; the outcomes also have to be ‘dependable’ in the sense they are usable whenever and wherever need arises. To meet such requirements, new learning studies with new research methods have been carried out, which differ from those used in laboratory-based, conventional learning studies [1][2] , starting to yield some substantial reports [3][4].

1. Theories behind collaborative learning One of the major changes introduced by learning sciences is a change in the perspective on learning: learning sciences view learning as a collaborative act, where people mutually

16

N. Miyake / Designed Collaboration as a Scaffold for Schematic Knowledge Integration

enhance performance by influencing each other. There are two reasons for this change. One is that through many studies, collaboration has been proven to be effective in enhancing learning outcomes, in many disciplines and in different age groups [5][6][7][8][9][10][11]. The other reason is that collaborative situations help promote in-depth, process-oriented research in learning. The current rapid development of information recording technologies is playing a large role in research, by increasing the likelihood of learning studies achieving their potential of bringing significant change to actual education and progressing our understanding of the learning processes themselves. It has been widely acknowledged that a collaborative situation promotes motivation for comprehension [12]. Collaboration has also been a topic of research to see whether “Two heads are better than one,” and if so, how. It has also been repeatedly demonstrated that collaborative learning leads to adaptive acquisition of knowledge [13]. In order to understand the underlying mechanism for such results, some research has focused on the dynamic interaction between individual courses of understanding and their collaboration, while keeping the basic unit of analyses on the pair, by using tasks like jointly comprehending how a sawing machine saws [8] or what would be the area of a square sheet of paper to get the three-forth of its two-thirds, or the two-thirds of its three-forth [10]. On the sewing machine problem, even knowledgeable pairs could spend three to four hours easily to understand the hidden mechanism of how the internal bobbin works, through which each participant eventually constructed his or her own understanding, solidly different from the other partners’ comprehension. The detailed protocol analyses of such interaction also demonstrated that when caught in the “cal-de-sack” it was often the person who was monitoring the situation, not the person who was taking the initiative of the problem solving at the moment, who came up with an idea to radically change the situation to “save” them. These findings suggest that even during highly collaborative comprehension activities, social sharing of the situation does not impede each participant from pursuing individualistic knowledge construction. Rather, the interactive process supports each to realize different perspectives to check and modify their own understandings by making explicit the different perspectives, which are not within their individual repertoire [8]. On the square sheet area problems, paired participants, not the solos, could take alternating perspective-shifts from the local, task oriented view to more abstract perspective, and this alternation of the perspectives helped each participant to explicitly talk about their solutions in more abstracted forms toward the latter part of the interaction, thus enhancing the formation of abstracted solution [10]. This mechanism offers explanation why there could be different learning outcomes in each of the paired learners. It is because the abstraction level for each participant depends on the degree of the integration of the shared task-doing and the monitoring, during and at the end of the interaction. This also explains why a more knowledgeable participant, or the person who preceded the other in his/her understanding, could still benefit from the interaction. This model suggests that for the design of collaborative learning environments it is important to encourage role exchange and to secure ample chances for each individual learner to reflect upon the shared resources. 2. Designed Collaborative Reflection for the schematic knowledge integration In the project to test the theory described above at real classroom situations we have been developing a collaborative undergraduate curriculum to teach cognitive science for nearly 7 years [14][15][16][17][18][19][20][21]. The learning objective is broad because we believe knowledge about cognitive science has pragmatic value for most of what we do in our everyday lives. Cognitive science explains how human beings solve problems, make judgments, memorize events and schematize them, and create new ideas. Knowing how

N. Miyake / Designed Collaboration as a Scaffold for Schematic Knowledge Integration

17

people engage in such cognitive processes, with the associated strengths and weaknesses, helps in the development of a reflective, metacognitive viewpoint, which can be utilized in everyday practices of cognition. 2.1. Overall description of our curriculum and classroom activities The curriculum described here is for undergraduate students and covers two semesters per year, taking four years to complete. In the first year, hands-on experiences of simple cognitive tasks are completed and analyzed by the students, first individually and then collectively, in the class. They are then guided to reflect of their experiences to take metacognitive perspectives, to understand how people solve problems. The curriculum toward the end of the first year encourages them to tie their reflection to what is written in the textbooks, to integrate their personal knowledge to the theories of human problem solving. During their second year, these experience-based techniques are gradually meshed into more active reading of technical materials, to help students gain a deeper level of comprehension as well as to grasp the breadth of research in cognitive science. In the third to fourth years they are encouraged to engage in more inquiry-oriented, project-based learning, learning to do their research toward graduation theses. 2.2. Variations of the jigsaw method Throughout the curriculum, we use the various forms of the jigsaw method to enhance collaboration. A social psychologist, E. Aronson, devised this method in order to facilitate cultural merging in classrooms [22]. In his original design, a text may be divided into six parts each of which is read by a group of members, each of whom is responsible for different parts(this is called an “expert” group activity, because each is expected to be an expert of this assigned part). Then one member from each of those six groups gets together, for a group of six, to answer questions covering all six parts,requiring their equal participation to succeed(this is called the “jigsaw” group activity). The method produces a natural setting to explain what one understands to others, often motivating students to further examine their assigned parts. In our curriculum, students are introduced to a simple jigsaw having only two to three parts, gradually guided on to a more structured and dynamic jigsaw that covers twenty to thirty texts explaining different research findings. The jigsaw method is highly flexible, modifiable to facilitate many different types of collaboration. Suppose a complex and highly structured course work needs to be delivered among a class. One form we have developed in our curricula is a “matrix” jigsaw, wherein college students move in and out of hierarchically structured “expert” groups and “jigsaw” groups. Suppose you could prepare the learning materials in an n by m matrix, of perspectives A and B. Assign a student, say S1, to a cell Ai by Bj in the matrix. By doing this, S1 automatically becomes responsible of owing a particular viewpoint Ai on dimension A, as well as of another viewpoint Bj on dimension B. The course work can then be structured to require her to work in one of the expert groups on perspective A at one time, while at other times she is required to be an expert on B. A more dynamically expansive Jigsaw has been devised in our course to enhance collaborative understanding of 20 to 30 learning materials, each representing classic research in three different domains of e-learning research. We expect the students to collaboratively read, explain, exchange, and discuss the materials to integrate them, to the level where each participating student would come out of the course with the ability of verbally explain what the entire field is, with many evidential pieces of knowledge they could cite by identifying relevant research[18][19][20].

18

N. Miyake / Designed Collaboration as a Scaffold for Schematic Knowledge Integration

This dynamic jigsaw models common activities of real world researchers. Professional researchers take different responsibilities to study a common theme and exchange their interpretations of existing work and new findings from their own community. They most often do this repeatedly, explaining their thoughts to different audiences so that they can examine them from different perspectives for different integration possibilities. This series of activities help them achieve a coherent comprehension, which could dynamically evolve over time. The dynamic jigsaw models this process and requires each participating student to first become an expert of his/her core material(s), then repeatedly explain the core to different audiences. While they receive information from others, they are also required to expand their explanations to include the new, and relate them to their own older knowledge structure. They are then to integrate these explanations to form a new explanation, to improve understanding. The dynamic jigsaw class replicates this more systematically. In the first phase, a student becomes an expert of one piece of the learning materials. S/he exchanges its explanation with one other student, to form an explanation of two pieces of the materials. Then, they are ready to exchange their two-piece explanations with a neighboring pair, who also is ready to give her two-piece worth. At the end of this phase, each student is expected to know four pieces, two of his/her own plus two more explained by another student. They can now exchange four to four, to cover eight research pieces, enough to cover a sub-domain. Toward the end of the term, such domain experts share by exchanging explanations of the domains, or some eight piece worth of information. 2.3. A concept mapping tool to support collaborative reflection Note-sharing software in the form of a concept-mapping tool could aid the students to express their ideas, integrate them among themselves as well as with others. This kind of technological support works well with the course structure, to promote the students knowledge integration [14]. Students are encouraged to externalize their initial ideas at early stages of their understandings of the learning materials, because it becomes easier to “talk about” or “collaboratively reflect on” externalized forms than to do so on their “ideas”

N. Miyake / Designed Collaboration as a Scaffold for Schematic Knowledge Integration

19

in the head. It is also easy to expand he initial concept maps by adding new cards, by connecting once completed map with other maps. The concept mapping tool we devised is called ReCoNote (Reflective Collaboration Note), some of its displays during the course is shown in Fig. 1. We have also devised an evaluation tool to follow their development. The right column on Fig. 1 shows one student’s three concept maps drawn at different stages of a course in year 2004, put together to display the entire map so that both the students and the teaching team can 3. Major findings Some assessment analyses of their performance accumulated over the last 6 years reveal 1) fair amount of retention of the learning materials 4 to 6 months after the end of the course, 2) explicit knowledge integration surrounding each students’ personal needs, and 3) some conscious learning of learning skills such as asking specific, content-driven questions [15][16][18][19][20]. One index we have devised is the degree of integration, or structural coherence, of the final concept maps the students created after the course, before the deadline of the term papers. The maps were categorized into four groups according to their structural coherence and given scores ranging from 1 to 4, with 4 indicating the highest coherence. The term papers are also evaluated in terms of the conciseness and the correctness of the descriptions of the research findings they reported. We also examine the relationship between the implications they drew from what they had learned and their descriptions of possible usage in everyday life. We called this measure “extendibility.” We found that the integrity measures of the concept maps were generally high. Forty-two percent of the final concept maps were categorized as achieving high integrity, close to the performance of novice graduate students. In classes in recent years, all the learning materials were covered in 80 to 85% of the term papers, out of which some 50 to 60% being “concise descriptions” with the necessary components in an expected order. This indicated that the majority of the students had learned both the basic contents of the learning materials as well as how to give concise summaries. The extendibility measure, or the degree to which the students could connect what they learned to their daily experiences, was found to be positively correlated with the quality of concept maps, suggesting that the learning activity of externalizing their integration efforts had a positive effect of fostering their thinking toward applications of what they had learned [18]. The protocol analysis of the students’ conversations during the class revealed that the students’ explanations became more concise in terms of both the amount of time used and the content covered. To take three students as a representative example: their first explanation of one research paper took 400 to 500 utterances on average, which decreased to 20 to 30 utterances toward the end of the term, without losing any necessary components. Their first explanation attempts were closer in wording to the texts of the learning materials than their later explanations, yet the first explanations involved more incorrect, vague, or confused statements. Such confusions about the meanings of the learning materials tended to be resolved during the discussions that occurred while the students were integrating their materials with those of other students’ [19][20]. We also found some cases where the students, in their junior years, talked about their experiences of the dynamic jigsaw as a source of acquiring various research skills (e.g., taking notes, writing reports, and questioning) [15]. These are encouraging signs for further exploration of the conditions that make collaborative learning situations more productive and beneficial to those involved.

20

N. Miyake / Designed Collaboration as a Scaffold for Schematic Knowledge Integration

Acknowledgments This series of research has been supported by SORST/JSP 2005-2007 and JSPS Grant-in-Aid 15200020 (2004-2006). References [1] Brown, A. (1992) Design experiments: Theoretical and methodological challenges in creating complex interventions. Journal of the Learning Sciences, 2(x), 141-178. [2] Collins, A. (1992) Toward a design science of education. E. Scanlon and T. O’Shea (eds.), New directions in educational technology, Springer-Verlag, Berlin. [3] Bransford, J., D., Brown, A. L., and Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school (Expanded edition). National Academy Press, Washington, D.C.. [4] Bransford, J. D., & Donovan, S. (2005). How students learn. National Academy Press, Washington, D.C.. [5] Chi, M.T.H., Siler, S., Jeong, H., Yamauchi, T., & Hausmann, R.G. (2001). Learning from tutoring. Cognitive Science, 25:471-533. [6] Linn, M. C., & Hsi, S. (2000). Computers, teachers, peers: Science learning partners. Erlbaum. [7] Linn, M. C., Davis, B. A., & Bell, P. (Eds.) (2004). Internet environments for science education. Erlbaum, [8] Miyake, N. 1986. Constructive interaction and the iterative processes of understanding. Cognitive Science, 10(2), 151-177. [9] Scardamalia, M., & Bereiter, C., (1991). Higher-levels of agency for children in knowledge building: A challenge for the design of new knowledge media. The Journal of the Learning Sciences, 1(1), 37-68. [10] Shirouzu, H., Miyake, N., & Masukawa, H. (2002). Cognitively active externalization for situated reflection. Cognitive Science, 26(4), 469-501. [11] Roschelle, J. 1992. Learning by collaborating: convergent conceptual change. The Journal of the Learning Sciences, 2, 235-276. [12] Hatano, G., and Inagaki, K. (1991) Sharing cognition through collective comprehension activity. L. Resnick, J. Levin, and S. D. Teasley (eds.), Perspectives on socially shared cognition, American Psychological Association, Washington, D.C., 331-348. [13] Hatano, G., & Inagaki, K. (1986). Two courses of expertise. In H. Stevenson, H. Azuma & K. Hakuta (Eds.), Child development and education in Japan. Freeman & Co., 263-272. [14] Miyake, N., (2001), “Collaboration, technology and the science of learning: Teaching cognitive science to undergraduates,” The Annual Report of Educational Psychology in Japan, 40, 218-228. [15] Miyake, N. (2005a) Multifaceted Outcome of Collaborative Learning: Call for Divergent Evaluation. Paper presented at the meeting of the 13th International Conference on Computers in Education (ICCE2005), Singapore. [16] Miyake, N. (2005b) How can Asian educational psychologists contribute to the advancement of learning sciences?. Invited talk at the meeting of the Korean Society of Educational Psychology 2005 International Conference, Seoul, Korea. [17] Miyake, N., & Shirouzu, H. (2004) Learning from lectures for comprehension. Paper presented at the meeting of International Conference of the Learning Sciences 2004, Los Angeles, CA. [18] Miyake, N. and Shirouzu, H. (2006) A collaborative approach to teach cognitive science to undergraduates: The learning sciences as a means to study and enhance college student learning. Psychologia, 49, 2, 101-113. [19] Miyake, N., Shirouzu, H., & Chukyo Learning Science Group. (2005) Interactive learning cycles to foster knowledge integration. Paper presented at the meeting of the Germany-Japan Joint Workshop 2005, Tokyo. [20] Miyake, N., Shirouzu, H., and Chukyo Learning Science Group. (2005) The dynamic jigsaw: repeated explanation support for collaborative learning of cognitive science. Paper presented at the meeting of the 27th annual meeting of the Cognitive Science Society, Stresa, Italy. [21] Shirouzu, H, & Miyake, N. (2005) “CSCL for lecture comprehension and question asking: Commentable Movie Sheet on BBS”, Computer Supported Collaborative Learning 2005, Taipei, Taiwan. [22] Aronson, E., and Patnoe, S, (1997). The jigsaw classroom: Building cooperation in the classroom. Longman, New York.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

21

Technology Affordances for Intersubjective Meaning-Making Daniel SUTHERS Department of Information and Computer Sciences, University of Hawai`i, USA [email protected]

Abstract: The broad field of “computers in education” includes a diversity of approaches to using computers for learning. Each approach is based on an epistemology: a theory of how knowledge is gained. In this presentation, I will characterize the uses of technology and their corresponding epistemologies. I will single out intersubjective epistemologies as timely for research and practice, and call for development of technologies that offer social affordances and resources for meaning-making. The study of intersubjective meaning-making requires interactional analyses, but in new forms that transcend some of the assumptions and limitations of microanalysis and that can be coupled with other methodologies. The presentation illustrates these ideas with my research program on representational affordances for collaborative learning. Keywords: epistemologies of learning, representational guidance, technology affordances, interactional construction of meaning, research agenda

1

Introduction

The broad field of "computers in education" includes a diversity of approaches to using computers for learning. For example, we can find technology used as a publication medium, to present information or problems; as task-oriented tools for aiding performance, keeping track of information and organizing the learner's activities; as conceptual tools for relating features of problem instances to useful abstractions or expressing and testing the learner’s own ideas; as a communication medium through and with which learners engage with each other in peer tutoring, argumentation, or collaboration in making sense of a situation; and as a proxy for the teacher, selecting the next problem or activity, selecting learning partners, giving hints or correcting errors during performance, and confirming or correcting learners’ solutions. Each of these approaches is based on assumptions concerning learning and how technology can support it. These assumptions should be identified and used to guide design in a dialogue between theory and practice [26]. New forms of technology-mediated learning are possible if we re-examine our beliefs about learning and the roles of media in learning. My keynote presentation will provide an overview of my own quest. This extended abstract outlines the ideas to be covered and provides a bibliography for those who wish to pursue some of these ideas further. I first summarize relevant theories of how knowledge is gained, called epistemologies. I then single out intersubjective epistemologies as most timely for research and practice, and suggest lines of investigation into social affordances through which technology media can serve intersubjective meaning-making at various scales. The reader is referred to [30] for a more developed account of the material of this presentation, focused on the field of computer-supported collaborative learning (CSCL).

22

2

D. Suthers / Technology Affordances for Intersubjective Meaning-Making

Epistemologies of Learning

When the actual practices of our field are examined, we find that a great deal of work is based on a knowledge-communication epistemology. Knowledge communication is “the ability to cause and/or support the acquisition of one’s knowledge by someone else, via a restricted set of communication operations” [39]. Research conducted under this epistemology examines how to more effectively generate or facilitate communications that “cause and/or support” the desired acquisition of knowledge. The best work in this paradigm (e.g., [2]) eschews a simplistic view of learning as the transfer of information from outside to inside the learner's head, and treats knowledge communication in the context of constructivist and interactional stances, considered below. A constructivist epistemology [21, 37] emphasizes the agency of the individual learner in the learning process. Learning can only happen through the learner’s efforts at making sense of the world, although a mentor might arrange for the learner to have challenging experiences in order to accelerate the change process. Most researchers do not take constructivism to its solipsistic extreme, but instead view social interaction as helpful and even essential. Interactional epistemologies are diverse, and include accounts that emphasize both individual and social agency. With individual agency, the individual is the unit and agent of learning yet this learning can be enhanced through social interaction. Examples include cognitive dissonance theory [8] and socio-cognitive conflict theory [6]. Contribution theory [3] is interactional in its account of the construction of “common ground,” but is yet based on an individual epistemology as it does not explain how knowledge that did not predate the communication is jointly constructed within the communication process. At the boundary of individual and social agency, we find Vygotsky’s [38] oft-cited observation that developmental learning through social interaction can be understood as the internalization of interpersonal processes as intrapersonal processes. Intersubjective epistemologies are interactional epistemologies with social agency: they locate meaning-making and even learning at the group level. In a distributed or group cognition account, the group and its cultural/technological artifacts collectively constitute the proper unit of analysis [11, 28]. Knowledge and meaning can be understood as jointly created through interaction: learning consists of this interaction [16]. An intersubjective epistemology is distinguished from grounding in that interpretations emerge within the interaction, and so are shared from the outset. Learning is also conceived of as a community level phenomenon. A participatory epistemology sees learning as a process of increasing participation in the practices of a community [19], constructing personal and collective identity [40]. Another community level epistemology is knowledge building [25], the enterprise in which a community intentionally expands its cultural capital by reflecting on limits of understanding and choosing actions that address these limitations. 3

Intersubjective Meaning-Making

In my own analysis of CSCL [30], I single out intersubjective epistemologies as those that we most need to understand, at both the interpersonal and community levels. Given the pervasive social nature of learning, I maintain that this emphasis is of importance for other research communities such as the ICCE community. Intersubjective epistemologies lead to challenging unanswered questions. How is it possible for learning, usually conceived of as a cognitive function, to be distributed across people and artifacts [24]? What is the relationship of the change process we call “individual learning” to that individual’s participation in socially accomplished learning? The study of intersubjective learning is

D. Suthers / Technology Affordances for Intersubjective Meaning-Making

23

needed because we already have a substantial body of work on individual learning and on how the cognitive processes of participants are influenced by social interaction, while intersubjective learning is currently not prominent as a topic of study in our field (notable exceptions include [1, 16, 23, 27]). An intersubjective perspective will help designers understand how technologies can function as mediating resources in learning. In [30] I argue that “learning” is a judgment we make about the consequences of an activity, and to understand this accomplishment we must necessarily study the practices (the activity itself) of intersubjective meaning-making: how people in groups make sense of situations and of each other. 4

Implications for a Research Agenda on Social Affordances of Technology

In [30] I identify two distinct ways in which technology is applied to support collaborative learning—as a communication medium and as constraint (see also [10].) Both paradigms are limiting from an intersubjective meaning-making perspective, but both can contribute to a synthesis. Richer communication media are needed, particularly with respect to supporting the indexical nature of human communication [20]. Guidance for a learning agenda is needed for both discipline-specific practices and learning trajectories and for processes of intersubjective meaning-making, but without limiting creativity by excessively rigid scripting of action. In order to achieve advancements in these forms of support, we need to better understand the ways in which practices of meaning-making in the context of joint activity are mediated through designed artifacts [15] and apply this understanding to design fundamentally social technologies that are informed by the affordances and limitations of those technologies for mediating intersubjective meaning-making. The remainder of this paper identifies some unique social affordances of computational technology for intersubjective meaning-making, suggesting lines of investigation in research and design. Negotiation Potentials. Any medium offers certain potentials for action. Participants may feel an obligation to obtain agreement on modifications to shared workspaces. The potentials for action offered by the medium can therefore guide interactions towards ideas associated with the afforded actions [34]. If we would like users of our technology medium to focus on particular aspects of a problem, how can the medium be designed to prompt for actions that require negotiation of these aspects? Referential Resource. Jointly constructed representations become imbued with meanings for the participants by virtue of having been produced through a process of negotiation. These representational constituents then become a rich referential resource for conversation [33], facilitating elaboration on previous conceptions. Rather than being vehicles for communicating expert knowledge, representations become objects about which learners engage in sense-making conversations [23] and can be designed to lead to productive conversation. How can we make salient that which learners would productively interpret, elaborate on and relate to new information or ideas? Integration. The computational medium can leave a persistent record of activity [5]. How can traces of interaction and collaboration be designed to foster appropriate awareness of prior conceptions and the means to reference these in subsequent interactions so that they may be integrated with new information and ideas? (Im)mutable Mobiles. The mobility of digital inscriptions provides opportunities for recruitment of partners in the sense-making process [18] and supports continued engagement in that process. How can we exploit this property of technology for its potential to make new social alignments and their interactions possible?

24

D. Suthers / Technology Affordances for Intersubjective Meaning-Making

Reflector of Subjectivity. Computational media can be designed to foster group awareness [17], visualize conflict or agreement between members [12], or project representations of self into a social representation [14]. In what ways can we design technology to mediate intersubjectivity by reflecting activity, subjectivity, and identity? Trajectories of Participation. Technologies offer social affordances for patterns of participation over larger spans of time and collections of actors [22]. How can we encourage productive entanglement of multiple individual trajectories of participation by selectively making their contributions salient and hence available for subsequent interpretation by others? My colleagues and I have been engaged in work on social affordances of technology since we first realized that the visualizations and coach of Belvedere had significance primarily (if at all) in how they affected peer interaction [36]. This began a line of work on representational guidance [31] that investigated negotiation potentials [34], referential resources, and integration [33]. More recently, we have brought this work to an asynchronous paradigm [35], and have begun a new line of work examining the fundamental practices by which people appropriate the affordances of certain media for written communication [7]. 5

Analysis of Intersubjective Meaning-Making

Although some of our prior work has been in a quantitative experimental paradigm, we have found that the study of intersubjective meaning-making requires coordinated use of qualitative interactional analyses [4]. Quantitative methods aggregate over many sessions, obscuring the actual procedures by which participants accomplish learning through the affordances of online media [16]. Methods for studying the interactional construction of meaning are available [13, 9], but have largely been developed for brief episodes of face-to-face data, and do not scale well to online learning where media resources, time scale, and synchronicity all differ. This analytic tradeoff between scalability and fidelity must be resolved in order to inform the design of improved online learning environments and participation structures that engage participants more deeply in intersubjective meaning-making during collaborative inquiry. My research group has been working on this problem for several years now [29]. As our current progress is reported elsewhere in this proceeding [32], it will not be detailed here. The short-term objective of the work reported in [32] is to scale up sequential and interactional analysis to distributed and asynchronous interactions while remaining grounded in participants' use of media. The long-term objective of the entire enterprise discussed in this extended abstract is to obtain a deep understanding of how learning is accomplished interactionally in technology-mediated setting, and then to offer learners environments that provide the resources and guidance they need for engaged learning. Acknowledgments This work was supported by the National Science Foundation under award 0093505. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation.

References [1]

[2]

M. Baker, Computer-mediated argumentative interactions for the co-elaboration of scientific learning tasks., in Andriessen, Baker and Suthers, eds., Arguing to Learn: Confronting Cognitions in Computer-Supported Collaborative Learning Environments., Kluwer, Dordrecht, 2003, pp. 47-78. R. Bromme, R. Jucks and A. Runde, Barriers and biases in computer-mediated expert-layperson communication: An overview and insights into the field of medical advice, in R. Bromme, F. W. Hesse

D. Suthers / Technology Affordances for Intersubjective Meaning-Making

[3]

[4] [5]

[6] [7]

[8] [9] [10]

[11] [12]

[13] [14]

[15] [16]

[17]

[18] [19] [20] [21] [22] [23]

[24] [25]

[26] [27] [28]

25

and H. Spada, eds., Barriers and Biases in Comptuer-Mediated Knowledge Communication -- And How They May Be Overcome, Springer, New York, 2005, pp. 89-118. H. H. Clark and S. E. Brennan, Grounding in communication, in L. B. Resnick, J. M. Levine and S. D. Teasley, eds., Perspectives on Socially Shared Cognition, American Psychological Association, 1991, pp. 127-149. J. W. Cresswell, Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, Sage Publications, 2003. P. Dillenbourg, Designing biases that augment socio-cognitive interactions, in R. Bromme, F. W. Hesse and H. Spada, eds., Barriers and Biases in Computer-Mediated Knowledge Communication—and How They May Be Overcome, Springer, New York, NY, 2005, pp. 243-264. W. Doise and G. Mugny, The Social Development of the Intellect, International Series in Experimental Scoial Pscychology, Pergamon Press, 1984. N. Dwyer and D. D. Suthers, A Study of the foundations of artifact-mediated collaboration, in T. Koschmann, D. D. Suthers and T.-W. Chan, eds., Computer Supported Collaborative Learning 2005: The Next 10 Years!, Lawrence Erlbaum Associates., Mahwah, NJ, 2005, pp. 135-144. L. Festinger, A Theory of Cognitive Dissonance, Stanford University Press, Stanford, 1957. C. Goodwin and J. Heritage, Conversation Analysis, Annual Review of Anthropology, 19 (1990), pp. 283-307. T. Hansen, L. Dirckinck-Holmfeld, R. Lewis and J. Rugelj, Using telematics for collaborative knowledge construction, in P. Dillenbourg, ed., Collaborative Learning: Cognitive and Computational Approaches, Elsevier, Amsterdam, 1999, pp. 169-196. E. Hutchins, Cognition in the Wild, The MIT Press, Cambridge, Massachusets, 1995. P. Jermann and P. Dillenbourg, Elaborating new arguments through a CSCL script, in Andriessen, Baker and Suthers, eds., Arguing to Learn: Confronting Cognitions in Computer-Supported Collaborative Learning Environments, Kluwer, Dordrecht, 2003, pp. 205-226. B. Jordan and A. Henderson, Interaction Analysis: Foundations and practice, The Journal of the Learning Sciences, 4 (1995), pp. 39-103. J. Kaput and S. Hegedus, Exploring classroom connectivity by aggregating student constructions to create new learning opportunities, in A. D. Cockburn and E. Nardi, eds., 26th Annual Conference of the International Group for the Psychology of Mathematics Education, UK, 2002. T. Koschmann, Dewey's contribution to the foundations of CSCL research, Proc. Computer Supported Collaborative Learning 2002, Boulder, 2002, pp. 17-22. T. Koschmann, A. Zemel, M. Conlee-Stevens, N. Young, J. Robbs and A. Barnhart, How do people learn: Member's methods and communicative mediation, in R. Bromme, F. W. Hesse and H. Spada, eds., Barriers and Biases in Computer-Mediated Knowledge Communication (and how they may be overcome), Kluwer Academic Press, Amsterdam, 2005, pp. 265-294. K. Kreijns and P. A. Kirschner, Designing sociable CSCL environments, in J. W. Strijbos, P. A. Kirschner and R. L. Martens, eds., What We Know About CSCL and Implementing it in Higher Education, Kluwer, Dordrecht, 2004, pp. 221-243. B. Latour, Drawing things together, in M. Lynch and S. Woolgar, eds., Representation in Scientific Practice, The MIT Press, 1990, pp. 19-67. J. Lave and E. Wenger, Situated Learning: Legitimate Peripheral Participation, Cambridge University Press, Cambridge, 1991. G. Nunberg, Indexicality and deixis, Linguistics and Philosophy, 16 (1993). J. Piaget, The Grasp of Consciousness: Action and Concept in the Young Child, Harvard University Press, Cambridge, MA, 1976. P. Resnick, Beyond Bowling Together: SocioTechnical Capital, in J. M. Carroll, ed., Human-Computer Interaction in the New Millennium, ACM Press, Upper Saddle River, NJ, 2002, pp. 647-672. J. Roschelle, Designing for cognitive communication: Epistemic fidelity or mediating collaborating inquiry, in D. L. Day and D. K. Kovacs, eds., Computers, Communication & Mental Models, Taylor & Francis, London, 1996, pp. 13-25. G. Salomon, ed., Distributed Cognitions: Psychological and Educational Considerations, Cambridge University Press., Cambridge, 1993. M. Scardamalia and C. Bereiter, Knowledge Building Environments: Extending the Limits of the Possible in Education and Knowledge Work, Encyclopedia of Distributed Learning, Sage Publications Thousand Oaks, CA, 2003. D. A. Schön, The Reflective Practitioner, Basic Books, New York, 1983. G. Stahl, Group cognition in computer-assisted collaborative learning, Journal of Computer Assisted Learning, 21 (2005), pp. 79-90. G. Stahl, Group Cognition: Computer Support for Collaborative Knowledge Building, MIT Press, Cambridge, MA, 2006.

26

D. Suthers / Technology Affordances for Intersubjective Meaning-Making

[29] D. D. Suthers, A qualitative analysis of collaborative knowledge construction through shared representations Research and Practice in Technology Enhanced Learning 1(2006), pp. 1-28. [30] D. D. Suthers, Technology affordances for intersubjective meaning-making: A research agenda for CSCL, International Journal of Computers Supported Collaborative Learning, 1 (2006), pp. (in press). [31] D. D. Suthers, Towards a systematic study of representational guidance for collaborative learning discourse, Journal of Universal Computer Science, 7 (2001). [32] D. D. Suthers, N. Dwyer, R. Medina and R. Vatrapu, Analysis of Meaning Making in Online Learning, International Conference for Computers in Education, APSCE, Bejing, 2006. [33] D. D. Suthers, L. Girardeau and C. Hundhausen, Deictic roles of external representations in face-to-face and online collaboration, in B. Wasson, S. Ludvigsen and U. Hoppe, eds., International Conference on Computer Support for Collaborative Learning 2003, Kluwer Academic Publishers, Dordrecht, 2003, pp. 173-182. [34] D. D. Suthers and C. Hundhausen, An experimental study of the effects of representational guidance on collaborative learning, Journal of the Learning Sciences, 12 (2003), pp. 183-219. [35] D. D. Suthers, R. Vatrapu, R. Medina, S. Joseph and N. Dwyer, Beyond threaded discussion: Representational guidance in asynchronous collaborative learning environments, Computers & Education (to appear). [36] D. D. Suthers and A. Weiner, Groupware for developing critical discussion skills, First International Conference on Computer Support for Cooperative Learning, Bloomington, IN, 1995. [37] E. Von Glasersfeld, A Constructivist Approach to Teaching. , in L. S. J. Gale, ed., Constructivism in Education, Lawrence Erlbaum Associates, Inc., New Jersey, 1995, pp. 3-16. [38] L. S. Vygotsky, Mind in society, Harvard University Press, Cambridge, MA, 1978. [39] E. Wenger, Artificial Intelligence and Tutoring Systems: Computational and Cognitive Approaches to the Communication of Knowledge, Morgan Kaufmann, Los Altos, 1987. [40] E. Wenger, Communities of Practice: Learning, Meaning and Identity, Cambridge University Press, Cambridge, 1998.

Modeling and Representation

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

29

Design of an Environment for Developing Presentation Skills a

Kazuhisa Setaa and Mitsuru Ikedab Graduate School of Sciences, Osaka Prefecture University, Japan b School of Knowledge Science, JAIST, Japan [email protected], [email protected]

Abstract: Presentation plays an important role in transmitting one’s opinion and encouraging collaborative knowledge creation and decision-making processes. A key to producing persuasive presentation materials is to perform meta-cognitive activities well, but that is difficult for novice learners of presentation tasks. Furthermore, from the system-development viewpoint, it is difficult to develop a learning system with which learners can develop their presentation skills effectively because the cognitive activities in presentation task are typically not clarified. In this paper, we first present a cognitive model of the user who performs presentation task. We then overview two kinds of designed environments based on that model: one is for producing presentation materials that encourage learners to perform meta-cognitive activities; the other is a collaborative learning environment in presentation rehearsal, which encourages the transfer of context-dependent meta-cognitive knowledge among learning partners. Keywords: meta-cognition, scaffolding, presentation task, presentation skill, cognitive model

1. Introduction Presentation plays important roles in transmitting one’s opinions and encouraging collaborative knowledge creation and decision-making processes. In business scenarios, results of presentations often influence important administrative decision-making. It is not too much to say that the presentation skills that enable a person to organize their opinions and transmit them adequately are necessary skills for various applications in business, research, learning, and so on. In western countries, lessons related to rhetoric, speech, and presentation that enable a person to transmit one’s opinion adequately are provided [3][8]. Recently, such lessons have been reintroduced gradually into the university curriculum; numerous books on the subject are published even in Japan. These phenomena reflect the need for fostering well-developed presentation skills. Presentation processes, in general, consist of many activities such as understanding of audiences’ requirements for the presentation, setting presentation objectives and goals, and presentation design. Know-how of these activities can be partially externalized, i.e., general requirements satisfied by the result of each activity and what kind of (sub-)actions should be undertaken by the presenter are described explicitly in a document. The user can therefore acquire such fundamental knowledge. Nevertheless, the know-how strongly depends on one’s presentation scene: the way to build a consistent presentation logic, translate it to presentation sequences in an easily acceptable format for the audience, and build accurate audience models cannot be described explicitly in a document. Therefore, the user must develop such skills through experience.

30

K. Seta and M. Ikeda / Design of an Environment for Developing Presentation Skills

Document media that are written in natural language are low-cost and high-performance media for knowledge transmission and spreading a viewpoint. Fundamental rules for presentations are actually written in the document media. However, the describable principle of know-how in a document is essentially generic. Consequently, (1) it is often difficult to specify and adapt to the individual specific scene. In contrast, the described case, which is dependent on know-how in a document, is fundamentally specific. For those reasons, (2) it is often difficult to generalize and transfer it to other individual-specific scenes. Furthermore, (3) it is difficult to determine the descriptions even when appropriate descriptions should be referred. Moreover, because one’s constructed presentation know-how is rooted in experiences and its externalization is strongly influenced by the author’s subjective intentions and viewpoint, (4) the described explicit know-how is often subjective. It is therefore necessary (a) to analyze and clarify the principle knowledge for making high-quality presentations and (b) to develop a learning environment based on them whereby a learner can develop presentation skills through the user’s presentation experiences. Our research goal is to clarify and systematize presentation skills as a presentation skill ontology and to build a support environment with which a novice learner of presentations can produce persuasive presentation materials and develop the required skills. In this paper, we describe a design of a CSCL environment for producing presentation materials as a first step towards achieving our goal. 2. Cognitive Model in Presentation Task Through our surveys of presentation know-how, we have inferred that performing meta-cognitive activities [1][2] such as self-monitoring, self-evaluation, and reflection, plays an important role to produce consistent and persuasive presentation materials and to develop presentation skills continuously. Figure 1 shows a cognitive model that incorporates detailed working processes of a learner that we adopt as a reference model for our system design. We capture presentation tasks as a form of problem-solving oriented learning a learning style in which a learner must not only accumulate sufficient understanding of a subject domain but also acquire the capacity for constructing effective problem-solving and learning processes according to a sophisticated strategy [5][6][7]. In the presentation task, processes of creating presentation materials correspond to problem-solving processes. Figures 1(i) and 1(iii) correspond to the planning process of the presentation plan and learning plan in Fig. 1, respectively, and 1(viii) and 1(x) respectively correspond to presentation rehearsal processes and learning processes in Fig. 1. Figures 1(v), 1(vi) and 1(xi) correspond to the monitoring process. Figure 1(xii) represents performing processes before the audience. We have presented a subject of presentation, say, ‘make a presentation for explaining the role of UML in software development processes’. Two virtual persons in the user, a presentation planner and learning process planner, respectively function in the roles of planning, monitoring and controlling problem-solving, and learning processes. Through the presentation task, a learner first defines a presentation goal with audience models and refines it to sub-goals that contribute to achieving goal G (Fig. 1(i), 1(vii)). They are refined to feasible presentation plans (Fig. 1(ii)); thereafter, the learner performs them to achieve presentation goals (Fig. 1(viii)).

K. Seta and M. Ikeda / Design of an Environment for Developing Presentation Skills

31

Fig. 1. Overview of Cognitive Model in Presentation Task The user generates an adequate learning goal (LG) to gain knowledge (Fig. 1(iii)) if the user recognizes a lack of knowledge in sub-goaling and rehearsing presentation plans. The user then might refine it to learning process plans (Fig. 1(iv)). In learning processes (Fig. 1(x)), that learner constructs knowledge (Fig. 1(iv)) that will be required for planning and performs the presentation process. Based on constructed knowledge, the user specifies presentation plans with audience models, creates presentation materials, and rehearses the presentation processes (Fig. 1(viii)) that alter the audience world (Pseudo Audience, Fig. 1(vii)). The user assesses gaps among goal states (GS), current goal states (CGS) of presentation process plans, and the current state (c-state) of the pseudo-audience (Fig. 1(v)) and those among learning goal states (LGS), current learning goal states (CLGS) of learning process plans, and the understanding state (Fig. 1(vi)). The user iterates these processes until the c-state of the real world satisfies the GS of problem solving. It is noteworthy that learners in presentation tasks must produce and execute not only plans for making presentations; they must also learn plans in the process. Furthermore, it is important for them to monitor audience model changes by rehearsing presentation processes and to monitor their own understanding states by performing learning processes and analyzing whether states of the pseudo audience satisfy the defined goal states (Figs. 1(v) and 1(vi)). The gap between current states and goal states dissolve the definition of new goals. Consequently, there exist two types of meta-cognition that a learner must perform in the presentation task. One is self-monitoring and (re-)construction of one’s own understanding states. Performing these meta-cognitive activities encourages building of a deep understanding of the target and clear logic of the presentation. It therefore plays an important role in building the foundation of a consistent and persuasive presentation. The other is self-monitoring and (re-)constructing the audience model by referring to one’s own understanding states (cognition of other people). Performing these

32

K. Seta and M. Ikeda / Design of an Environment for Developing Presentation Skills

meta-cognitive activities adequately encourages the setting of reasonable goals, scope, and a presentation grain size. It therefore plays an important role in the design of sophisticated presentations that are acceptable to the audiences. Presenters must perform these meta-cognitive activities accurately to create consistent and persuasive presentation materials; therefore, presentation tasks compel a learner to perform complicated tasks with heavy cognitive loads. A learner must manage and allocate attention resources adequately because of limited human attentional capacity. This explains why a novice learner tends to fall into confusion and learn ineffectively. Novice learners of a presentation task tend to allocate much attention to cognitive activities, especially to material-making processes or representations of materials in general. 3. System Design to Scaffold the Development of Presentation Skills Many authors describe the necessity and importance of performing these meta-cognitive activities in their books, but they do not include the term or concept of meta-cognition. Nevertheless, it is difficult for a novice presenter to learn when, how, and what kind of meta-cognitive activities should be performed in actual processes of presentation-material preparation because meta-cognitive activity is essentially tacit and latent; performing these meta-cognitive activities requires expertise of causal relations among the order of explanation, the presentation grain size, audience characteristics, and how the presentation affects the audience members’ thinking. It is therefore necessary to develop a learning environment in which a learner can improve working knowledge of performing meta-cognitive activities through practice while grasping fundamental principles of presentation through traditional lessons or books. 3.1 Design Principle Our system is designed to realize a learning environment (a) that encourages learners’ spontaneous meta-cognition required to make high-quality presentation materials and, in which (b) a learner can use presentation-material preparation experiences as valuable learning resources, and in which (c) a learner can include expertise of other experienced learners effectively. Based on the cognitive model presented in the previous section, we designed three kinds of learning environment: a learning environment for presentation-material preparation phase, presentation rehearsal phase and post-presentation (reflection) phase. Thus, the user can learn through all presentation processes. We describe the learning environment for the presentation-material preparation phase and the presentation rehearsal phase as follows. 3.2 Learning Environment for Material Preparation Phase Figure 2 shows a learning environment for the presentation-material preparation phase. The environment consists of five interface windows. Figure 2(iii) is a window of a presentation preparation computer program (Power Point, PPT; Microsoft Corp.), which is familiar to many people throughout the world; other windows are designed to function together with the PPT. For example, when the user selects a PPT slide, then domain relations, logical relations, presentation goals, and knowledge resources that are related to the slide are shown and highlighted in each window. Figure 2(i) is named the domain structure view, in which a learner can describe the relations of domain concepts that the user learned in relation to the presentation target. For

K. Seta and M. Ikeda / Design of an Environment for Developing Presentation Skills

33

example, the user can describe that “UML plays an important role in enhancing communication among humans who attend the software lifecycle” in the view of using graphical representations (nodes and arcs) and natural language. Furthermore, the user can color each concept (node) in white when the user sufficiently understands the concept. Figure 2(ii) is named a logical relation view in which the user can derive and arrange the logical relations of arguments hierarchically by referring to concept relations described in the domain structure view. For example, the user can describe logical relations that the reason of “UML plays an important role in the software lifecycle” are that “UML provides a common language that is needed throughout the entire software lifecycle” and “it encourages smooth communication among humans who attend the lifecycle” in the view. Figure 2(iv) is named the intention view, in which the user can design a rational presentation structure that is acceptable to audiences to achieve presentation goals by referring to the described domain relations, logical relations, and presentation goal vocabulary, which are shown in the left part of the view. For example, the user can describe the presentation goal of “make audiences understand the roles of UML in software lifecycle” and then specify it to detailed sub-goals, as “make audiences understand the overview of software lifecycle by introducing the concept of software lifecycle” then “make audience understand the roles of UML in each process of software lifecycle by showing an example scenario.” In this case, domain-independent and presentation-task-specific exemplary scenarios that introduce, reject, or overview can be used for describing the presentation goals. They are specified in the presentation goal vocabulary described above. We specify them by surveying popular books and on-line resources related to presentation know-how [3][8][9][10] and by referring to the learning support systems’ goal ontology proposed by Hayashi et al. [4]. Furthermore, the user can specify intentional effects to audiences in greater detail using graphical representations in lower parts of the view.

Fig. 2: Designed learning environment for presentation-material design

34

K. Seta and M. Ikeda / Design of an Environment for Developing Presentation Skills

Figure 2(v) portrays knowledge resources. Because the user can bookmark them by connecting with related domain concepts in (i), arguments in (ii), and the slides in (iii), the user can refer to them easily by clicking the nodes or slides in respective windows. The salient advantage of this learning environment from the viewpoint of the improvement of presentation quality and developing her presentation skill is expected to be that the environment encourages learners’ spontaneous meta-cognition during the presentation-material preparation processes. Visually externalized information in (i) and (ii) encourages self-monitoring and confirms processes of the user’s own understanding and self-control processes. Furthermore, visually externalized information in (iv) encourages self-monitoring and confirmation processes of rationality between a goal and its sub-goals and motivates the user to be conscious of constructing an accurate audience model. The described information represents users’ intentions for the presentation. Consequently, it is used as a valuable learning resource for subsequent collaborative learning in the presentation rehearsal phase. 3.3 CSCL Environment for Presentation Rehearsal It is difficult for novice learners to be aware of mistakes embedded in their presentation materials because of the lack of other persons’ viewpoints about it. Furthermore, it is hard for another person to give appropriate instructions to the user because of the difficulties in inferring the creator’s design intentions. It is also difficult to teach knowledge of making sophisticated presentation materials in a general context that includes a variety of meta-cognitive knowledge: it is quite tacit, latent, and highly context-dependent. For those reasons, we designed a collaborative learning environment in which the user and the instructor can share not only presentation material but also learners’ intentions (rich contexts) behind the presentation materials, which encourages well-focused communications among learners. Consequently, it encourages smooth knowledge transfer. Figure 3 shows a collaborative learning environment for presentation rehearsal and reflection phase. Both the presenter and learning partners (playing the audience role) can share the same information shown in the figure. In this environment, the presenter can examine each component of the presentation materials collaboratively or conduct a rehearsal using text-based or speech-based chat. The environment includes three windows. Figure 3(i) is designated as the presentation view based on a PPT slide show. Both the presenter and learning partners (LPs) can advance the slides. Figure 3(ii) is an intention view, in which LPs can refer and understand the presenter’s intentions of the slides set in the presentation-material preparation phase: roles of the slides from the viewpoint of achieving the presentation goals described above, subjects of the slide, what kinds of reactions the presenter expects from audiences, and so on. By clicking a presentation goal node, participants can advance and review a series of slides that are intended to achieve the clicked presentation goal. Furthermore, LPs can evaluate each presentation material by checking the items shown in the lower part of the view. These items are constructed automatically using the system based on the described presentation goal hierarchy. Figure 3(iii) is a text-based chat window with which participants can communicate with each other. Generally speaking, it is difficult to evaluate the presentation materials because of the difficulties in understanding the presenter’s intentions. The major advantage of this collaborative learning environment is that the participants can refer to the presenter’s presentation goals and intentions in addition to presentation materials. This encourages smooth discussions that specifically address whether the presenter’s intentions are valid,

K. Seta and M. Ikeda / Design of an Environment for Developing Presentation Skills

35

Fig. 3: Designed collaborative learning environment for presentation rehearsal whether presentation materials adequately reflect the described intentions and so forth, because LPs can have a viewpoint for evaluation and discussion. For example, a presenter’s intention of the slide shown in Fig. 3(i) is described in the upper right part of Fig. 3(ii) in natural language as “understanding the overview of roles and merits of using UML in the software lifecycle.” Furthermore, the user’s intentions of achieving this presentation goal contribute to realization of the upper presentation goal of “explaining the roles of UML in software lifecycle” are represented graphically in the presentation goal hierarchy. Based on the goal hierarchy, the system can construct items for evaluation: “The presenter intended to explain an overview of roles and merits of using UML in software lifecycle. Can you understand?”, “Is this slide valid from the viewpoint of introducing the UML overview?” and so on. The LPs can evaluate the slide according to the evaluation items. The evaluation results are recorded and sent to the presenter. Furthermore, participants can discuss and send their suggestions: “I think it is better to explain the difficulties of each process in greater detail to achieve your goal. Because the audience can understand the roles of UML in greater detail based on their deep understanding of the necessities of UML” and so on. This kind of well-focused discussion can be inspired by referring to presenter’s intentions and items for evaluation. By providing the CSCL environment proposed here, presenters can use experiences of presentation-material preparation as valuable learning resources for developing presentation skills.

36

K. Seta and M. Ikeda / Design of an Environment for Developing Presentation Skills

4. Conclusions and Future work This paper presented a description of the outline of our learning support environment for developing presentation skills. The major advantage of our system is that it is designed to encourage the presenter to perform meta-cognitive activities in presentation-material design and import expertise of other experienced learners in presentation rehearsal based on the cognitive model. It is important to produce the model assumed in the system design that is explicit not only for sophisticated learning environment but also for its continuous improvement. We were unable to describe the learning environment for review after performing a presentation, which would allow the user to analyze their presentations collaboratively. This issue will be examined in another paper. We hope to improve the presentation goal vocabulary and build presentation skill ontologies in our future work. Acknowledgments We are very thankful to Mr. Takashi Kajino and Masayoshi Toyokura for their kind help and efforts. This research was supported in part by a grant from the Matsushita Education Foundation. References

[1] Brown, A. L., Bransford, J. D., Ferrara, R. A., and Campione, J. C. (1983) Learning, Remembering, and Understanding, In: E. M. Markman and J. H. Flavell. (Eds.), Handbook of child psychology (4th ed.) Cognitive Development, Vol. 3, Wiley, 515–529. [2] Flavell, J. H. (1976) Metacognitive Aspects of Problem Solving, in L. Resnick (Ed.), The nature of intelligence. Lawrence Erlbaum Associates: Hillsdale, NJ, 231–235. [3] Harvard University Writing Center: http://writing2.richmond.edu/writing/wweb.html [4] Hayashi, Y., Ikeda, M., and Mizoguchi, R. (2004) A Design Environment to Articulate Design Intention of Learning Contents, International Journal of Continuing Engineering Education and Life Long Learning, 14, 3, 276-296. [5] Seta, K., Tachibana, K., Umano, M., and Ikeda M. (2003) Basic consideration on reflection in problem-solving oriented learning, Proc. of the International Conference on Computers in Education (ICCE-03), Hong Kong, China, 160-168. [6] Seta, K., Tachibana, K., Fujisawa I., and Umano M. (2004) An ontological approach to interactive navigation for problem-solving oriented learning processes, International Journal of Interactive Technology and Smart Education, 1, 3, 185-193. [7] Seta, K., Tachibana, K., Umano., M., and Ikeda, M. (2005) Human Factor Modeling for Development of Learning Systems Facilitating Meta-Cognition, in Chee-Kit Looi, David Jonassen, Mitsuru Ikeda (Eds): Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences, Frontiers in Artificial Intelligence and Applications, Vol. 133, IOS Press, 396-403, (also Proc. of the International Conference on Computers in Education (ICCE-05), Singapore). [8] Stanford University Writing Center: http://swc.stanford.edu/library.htm . [9] Takada, T. (2004) Logical Presentation, Eiji Press (in Japanese). [10] Teruya, H., and Okada. K. (2001) Logical Thinking, Toyo Keizai Inc (in Japanese).

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

37

Ontological Modeling Approach to Blending Theories for Instructional and Learning Design Yusuke HAYASHIa, Jacqueline BOURDEAUb, Riichiro MIZOGUCHIa a ISIR, Osaka University, 8-1 Mihogaoka, Ibaraki, Osaka, 567-0047 Japan b LICEF, Télé-université, 100 Sherbrooke W., Montréal, (QC) H2X 3P2 Canada [email protected] Abstract: This paper proposes a modeling framework for learning and instructional design from the viewpoint of ontological engineering. One of the characteristics of this framework is a theory/paradigm-independent ontology for modeling learning/instruction. This paper discusses how our modeling framework with the theory/paradigm-independent ontology contributes to modeling learning and instruction from a comprehensive viewpoint of various educational theories. Keywords: Instructional design, Learning design, Ontology, Theory-aware, Design support

Introduction Sharing and reuse of information about not only resources but also their structure such as design intention have been brought to public attention as recent development of IT standards in the areas of learning, education, and training. Representative specifications include IEEE LTSC Learning Object Metadata (LOM) standard [9] and IMS Learning Design (LD) specification [10]. These bring about global benefits of exchange format for sharing and reuse of information about learning contents. However, a problem still remains: how to build “good” design for education using the standards. Of course, considerable achievement has been made in instructional and learning sciences. This being said, even though some educational theories (learning, instructional and instructional design theories) prescribe optimal/desirable methods of learning and instruction, many of the theories are not sufficiently articulated for the use of designers. Such theories allow for diverse interpretation and therefore may be difficult to use in practice. One of the reasons why these problems come up is that the description of educational theories is made in natural language, using different terminology. As Reigeluth points out, although many theories prescribe the same method for the same situation, these are described in different terminology [18]. This leads to a diversity of theories that are all open to interpretation. Even for experts, it is sometimes difficult to appropriately use theories while having a clear understanding of the similarities and differences between them. Consequently, we must first establish a common basis for understanding the theories at a conceptual level, along with organized concepts and vocabulary. The goal of this study is to build an engineering infrastructure that enables designers to select instructional and learning theories and blend them into an instructional/learning design. This paper thus proposes a modeling framework for learning and instructional design from the view point of ontological engineering [14], based on the results of previous

38

Y. Hayashi et al. / Ontological Modeling Approach to Blending Theories

research in this respect [1][2][16] . In addition, the purpose of this study is not to expose a scientifically valid basis for organizing educational theories nor to reconstruct them on this basis, but rather to find an engineering approximation that allows the building of an engineering infrastructure that enables practitioners to utilize educational theories. This paper is organized as follows. The next section presents our perspective on modeling of learning and instruction and proposes our resulting modeling framework for them. Section 2 illustrates an example of a model of theoretical knowledge for education based on this modeling framework. Section 3 discusses an application of this framework to design support system for learning scenarios or activities. The final section concludes this paper and states the issues to be examined further.

1. Ontological modeling framework for instructional/learning design This study adapts the ontological modeling framework of functional design knowledge in the world of manufacturing industry by Kitamura et al. [12] to model educational theories. The main characteristic of this framework is independent conceptualization of what is achieved and how to achieve the change in the target things. The former is organized as ontologies of functionality, and the latter is as ways of function achievement. The function of artifacts (devices) is modeled as hierarchical structure of component functions linked with ways of function achievement. This model is called “function decomposition tree”. Kitamura et al. have confirmed the effectiveness of the framework in: (1) affording a better understanding of the functionality of devices, (2) facilitating the sharing of design rationales of devices, (3) supporting the improvement of functionalities, and so on. Although the domain is different from educational knowledge, we believe that it is applicable to the systematization of theoretical knowledge for instructional/learning design. For the educational domain, we have developed an ontology of learning, instruction and instructional design (L/I/ID) [1][2][16] . This study proposes an ontological modeling framework for education based both on the L/I/ID ontology and Kitamura’s framework [8]. It aims at facilitating (1) the sharing of a model of instruction/learning and (2) the application of educational theories appropriately to the modeling. IMS Learning Design (LD) specification [10] has recently focused on the sharing aspect, and we believe that our approach will contribute to make IMS LD specification work with educational theories. Roughly speaking, the L/I/ID ontology is composed of five major concepts: concepts related to Common, Learning, Instructional and Instructional design worlds and Educational event in the learning and the instructional worlds [8]. This ontology is developed in an effort to model learning and instruction in any paradigm - Behaviorism, Cognitivism and Constructivism -. Ertmer and Newby [6] assert that although each of the paradigms has many unique features, each describes the same phenomena (learning). In a similar line of thought, one of the notable features of this ontology is the conceptualization of relation between learning and instruction. The meaning of an instruction is defined by the change of learner state as achieved or by the intended result of learning. However, instruction is defined independently of the change of learner state. That is because an instructional action may have different effects on the change depending on the context. Independence is the key to allow for a variety of combinations of instructional actions and effects and to compare strategies provided by educational theories. The Learning/instruction process is modeled based on our ontological modeling framework for education. A unit of learning/instruction is described as I_L event, which is a sub-class of Educational event. An I_L event is defined as a combination of instructional action and change of learner state caused by a learning action. This definition allows to describe the relation between instruction and learning in a learning/instruction process. The

Y. Hayashi et al. / Ontological Modeling Approach to Blending Theories

39

(A) Preparing the learner for learning / Being ready to learn

Macro -I_L event

Way of achievement Preinstruction

(B)

is-achieved by

(C) Motivate / Motivated

Stimulate recall of prior learning / Recalling prior learning

I_L event decomposition

Micro -I_L events

Fig. 1 An example of I_L event decomposition whole process is modeled as a function decomposition tree. In our framework, this model is a hierarchical structure of I_L event to achieve a certain change of a learner state. Thus, it is called “I_L event decomposition tree”. In an I_L event decomposition tree, an upper (macro) I_L event is connected with the lower (micro) ones by way of achievement of change of a learner state (referred to just as “Way” hereafter). For example, consider a situation where an instructor wants a learner to be ready to learn and it is necessary that the learner is motivated into the learning and then that he recalls prior learning. Fig.1 illustrates this case as I_L event decomposition. An oval node represents an I_L event. Fig.1 (A) indicates the macro-I_L event, in which “Preparing the learner for learning” is the instructional action and “Being ready to learn” is the change of learner state. A conceivable process to achieve this is to motivate the learner (Fig. 1 (B)) and then to stimulate recall of prior learning (Fig. 1 (C)). The former instructional action brings about the learner state “Motivated” (Fig. 1 (B)), and the latter does “Recalling prior learning” (Fig. 1 (C)). A way is a description of relationship of such a decomposition of the required change into the detailed changes and actions to achieve them. A ‘Way’ has two sorts of interpretations. One, so-called bottom-up manner, is that the sum of the changes of learner state in micro-I_L event promotes the changes of learner state of the macro-I_L event. This manner, which concentrates on states, is descriptive. It describes which outcome is produced by a sequence of changes of learner state. The other, so-called top-down manner, is that an instructional action of a macro-I_L event is decomposed into detailed/concrete instructional actions of micro-I_L events. This manner, which concentrates on action, is prescriptive. It prescribes which sequence of instructional actions is required for performing the intended instructional action. Following the top-down interpretation, this study proposes a method to systematize theoretical knowledge. The theories prescribe strategies for planning instructional and learning process according to supposed situations. These strategies may superficially vary from theory to theory but, as discussed in the introduction, some of the differences just come from the difference in terminology that each theory uses. We assume that some essentials of theories characterizing themselves are disclosed if their strategies are resolved conceptually. In our framework we propose modeling learning/instructional strategies as Way using L/I/ID ontology. A set of Ways derived from a theory characterizes the theory from the prescriptive aspect. In addition, modeling strategies on a common basis, which is L/I/ID ontology, is expected to disclose not only characteristics of each theory but also commonality among theories. We call Ways derived from theories ‘Way-knowledge’. Such Ways can be applied to various instructional/learning designs if the supposed situation is matched. This study aims to support designers blending theories into their own instructional/learning design by

40

Y. Hayashi et al. / Ontological Modeling Approach to Blending Theories

providing an engineering infrastructure for accumulating and sharing variety of Way-knowledge derived from theories.

2. An example of comprehensive model of educational theories In our ontological modeling framework for education with the L/I/ID ontology, educational theories are modeled as Way-knowledge. In this section, we will discuss how Way-knowledge describes educational theories comprehensively. Fig. 2 shows an example of an I_L event decomposition tree. This tree represents a decomposition of the I_L event, “Facilitate learning / Knowledge state”. An oval node represents I_L event. Its label expresses combination of instructional action and change of learner state in the form of “instructional action/change of learner state”. The links between I_L events found below and above a square represent a Way. I_L events joined to a square and located below it represent an AND relationship. This means that, according to a Way, an upper I_L event is achieved when all of the lower I_L events are achieved. If more than one Way leads to an I_L event, it represents OR relationship among Ways, that is to say, there are more than one method to achieve the change of learner state. This relationship provides alternatives to make more detailed instructional/learning design for designers. Such an I_L event decomposition tree with OR relationship is called “General I_L event decomposition tree”. The general I_L event decomposition tree shown in Fig. 2 covers the whole process of instruction/learning - from preinstruction to assessment -. The foundation of this process is the five major learning components by Dick et al [5]. They are a summary of Gagne’s nine events of instruction [7]. These theories are basically considered to be based on cognitivism approach but to be somewhat eclectic in view of paradigms - behaviorism, cognitivism and constructivism - [17]. This model is intended to be a comprehensive model of instructional/learning theories by giving a shape to Dick’s components based on various theories in different paradigms. Now, we take a close look at the decomposition below the I_L event, “Exercise the learner / Absorbed (Fig. 2 (B))”, because this decomposition includes both cognitivism and constructivism approaches with OR relationship of Ways. Ways included in this decomposition are based on cognitive approach - Gagne’s nine events of instruction [7] and Merrill’s component display theory [13] - and constructivist approach - Collins’s cognitive apprenticeship [4] and Jonassen’s model for designing constructivist learning environments [11] -. The point of selecting principles from multiple theories is the change of learner state. According to Carey [3] and Ertmer and Newby [6], aspects of constructivist approach can be compatible with aspects of prescriptive approach for specified types of learners and learning outcomes [5]. From this viewpoint, Dick relates constructivist strategy to the five major learning components. In the similar line of these thought, we proposes a modeling method for organizing any theories in terms of what and how their strategies intend to achieve. This study sets up the working hypothesis that the idea of states in the learning process is common while the assumed mechanism of developing knowledge is different for each paradigm 1 . With this consideration, we have organized learner states intended to be shared among different educational theories. “Being ready to learn (Fig.2 (A))” and “Self-aware (Fig.2 (D))” are examples of such learning states. Making attempts to organize common concepts and to interpret (supposed) intention of theories as the change of learner state will afford a better understanding and utilization of educational theories. 1

Note that we are not saying all the learning theories share the same learning states.

Y. Hayashi et al. / Ontological Modeling Approach to Blending Theories

41

Let us examine blending of these approaches in more detail. The I_L event “Exercise the learner / Absorbed (Fig. 2 (B))” is decomposed into two sub-I_L events: “Eliciting performance / Absorbed (Fig. 2 (C))” and “Providing informative feedback / Self-aware (Fig. 2 (D))”. This decomposition is derived from Gagne’s events. To put it briefly in terms of what to be achieved, the former objective is to develop what is learned through practice and the latter one is to become self-aware in order to make the practice more efficient. These I_L events can be decomposed by both of cognitivist and constructivist approaches. For example, there are two Ways to decompose “Provide informative feedback / Self-aware”. Cognitivist approach (Fig. 2 (10)) gives relatively explanatory feedback, e.g. giving correct answer or help. On the other hand, constructivist approach (Fig. 2 (11)) gives relatively inquiring feedback, e.g. providing hint or assisting learners to articulate. Constructivist approach also has other characteristics such as authenticity and self-construction of knowledge, though this is not shown in Fig. 2 because of presenting a contrast to explanatory methods. These characteristics can be described in a more detailed model but are too complicated to explain in detail here. The key point of the blending of these approaches is, as discussed in the previous section, to take particular note of the change of learner state. These theories indeed have differences in the method. However, at the same time, the objective (intended change of learner state) can be described as common in becoming self-aware in this example. From this viewpoint, we set both the cognitivist and constructivist approaches as a Way to decompose “Providing informative feedback / Self-aware (Fig. 2 (D))”. In the same manner as this, other cognitivist and constructivist methods are also organized as Way knowledge shown in Fig. 2 (1) – (13).

3. Toward a theory-aware instructional/learning design support system One of the remarkable applications of our ontological modeling framework for education is theory-aware design support system for learning contents [8]. By “Theory-aware” [14], we mean the capability of information systems which can support the activities of users based on the understanding of relevant theories. Nkambou et al. [15] discuss the benefits of accessing theories and required functionalities of theory-aware ITS authoring environment. The benefits are: 1) make decisions (macro, micro) after reflection and reasoning, 2) communicate about or explain their design decisions, 3) check consistency among design decisions, intra-theory and inter-theories, 4) produce scrutable learning environments, 5) have heuristic knowledge grounded in theoretical knowledge. The required functionalities are that authors can: 1) ask the system what theories apply best to this or that learning situation/goal, 2) ask the system to show examples, 3) ask the system for advice on whether this element of a theory can be combined to an element from another theory, what is the risk associated to doing so, any preferable other solution, etc. Referring to their argument, this study focuses on the following two points as the requirement for realization of theory-awareness in an instructional/learning design support system: The system can (1) help designers to build theoretically valid model of learning/instruction, and (2) explain the help with the theoretical justification to designers. We now discuss how our ontological modeling framework for education contributes to these two advantages. As discussed in the previous sections, learning/instructional strategies included in educational theories are described as Way-knowledge in our modeling framework. Way-knowledge is a relationship between macro-I_L event and some micro ones with desired change of learner state (goal) and necessary condition for the change (situation). In

42

Y. Hayashi et al. / Ontological Modeling Approach to Blending Theories

addition, Way-knowledge is described with not each theory’s own terminology but our L/I/ID ontology. This ontology is designed to accept terminological difference among educational theories and paradigms on a conceptual level as mentioned before. Based on this ontology, Way-knowledge is not only human-readable but also machine-readable. This enables systems to expound educational theories described as Way-knowledge and to interpret models of learning/instruction built by learning/ instructional designers founded on educational theories. These will be quite helpful to learning/instructional designers for in-depth understanding of theories and for communicating about their design decisions and products. Another characteristic of our modeling framework is that a theory is described as a set of Way-knowledge. In other words, a theory is split into pieces of strategy and each strategy is described as a Way-knowledge. For example, in fig. 2, Way-knowledge (1), (2), (4), and (6) is derived from Gagne and Briggs’s theory, and (8) and (12) derives from Merrill’s component display theory. Such modeling of theories helps learning/instructional designers to make a model of learning/instruction from various viewpoints such as those that follow. One viewpoint is that a support system can pick and choose applicable strategies within a theory according to the grain size of process. If a designer wants to decompose complete learning/instructional process by him/herself, Way knowledge (1) is recommended. On the other hand, if he/she wants to do more detailed process, Way knowledge (4) or (6) are recommended. The other is that a support system can also suggest whether an element of a process derived from a theory can be exchanged by an element derived from another theory. Way-knowledge (10) and (11) illustrated below the I_L event “Provide informative feedback /Self-aware (Fig. 2 (D))” is a good example to illustrate the possibility of exchange. Such guidance becomes possible with inter-theory comparison in terms of strategies included in theories or derived from them.

4. Conclusion We have discussed a modeling framework for learning/instructional design based on ontological engineering. The characteristics of this framework include: 1) a theory/ paradigm-independent ontology for modeling learning/instruction, 2) compatibility between prescriptive and descriptive models derived from educational theories, 3) theory-awareness brought out by an ontological modeling framework. In this paper, we have concentrated on how the theory/paradigm-independent ontology contributes to modeling learning/instruction from a comprehensive viewpoint of various educational theories. Our modeling framework based on this ontology will be helpful to blend of educational theories as discussed in section 2, and to enhance the quality of support by information systems for learning/instructional design as discussed in section 3. The following issues to be examined further still remain: 1) investigating the theory/paradigm-independence of the L/I/ID ontology through organizing much more educational theories in our modeling framework, 2) considering the relation between learning objects and abstract design of learning contents described as an I_L event decomposition tree, 3) implementation of a theory-aware design support system for learning contents, and 4) compliance with standards for semantic web and e-learning.

Y. Hayashi et al. / Ontological Modeling Approach to Blending Theories

43

Acknowledgments We would like to thank to Dr. Daniele Allard for her helpful comments. References [1] Bourdeau, J. and Mizoguchi, R.: “Collaborative Ontological Engineering of Instructional Design Knowledge for an ITS”, Proc. of ITS2002, pp.399-409, 2002. [2] Bourdeau, J. and Mizoguchi, R.: “Selecting theories in an ontology-based ITS authoring environment”, Proc. of ITS’2004, pp. 150-161, 2004. [3] Carey, J. O.: “Library skills, information skills, and information literacy: Implications for teaching and learning”, School Library Media Quarterly Online, Vol. 1, 1998 Available: http://www.ala.org/aasl/SLMQ/skills.html [4] Collins, A., Brown, J. S., and Newman, S. E.: “Cognitive apprenticeship: Teaching the crafts of reading, writing and mathematics”, In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser. Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 453-494, 1989. [5] Dick, W., Carey, L., and Carey, J. O.: The systematic design of instruction, Sixth edition, Addison-Wesley Educational Publisher Inc., 2004. [6] Ertmer, P. A., and Newby, T. J.: “Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective”, Performance Improvement Quarterly, 6 (4), 50-70, 1993. [7] Gagne, R. M. and Briggs, L. J.: Principles of Instructional Design (2nd Ed.). Holt, Rinehart and Winston, New York, 1979. [8] Hayashi, Y., Bourdeau, J. and Mizoguchi, R.: “Ontological Support for a Theory-Eclectic Approach to Instructional and Learning Design”, Proc. of EC-TEL 2006, (to appear), 2006. [9] IEEE LTSC, The Learning Object Metadata standard. Available: http://ieeeltsc.org/wg12LOM/lomDescription [10] IMS Global Learning Consortium, Inc.: IMS Learning Design. Version 1.0 Final Specification, 2003. Available: http://www.imsglobal.org/learningdesign/ [11] Jonassen, D.: Designing constructivist learning environment, In Reigeluth, C. M. (Eds.): Instructional-design theories and models A new paradigm of instructional theory, Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc., pp. 215-239, 1999. [12] Kitamura, Y., Kashiwase, M., Fuse, M., Mizoguchi, R.: “Deployment of an Ontological Framework of Functional Design Knowledge”, Advanced Engineering Informatics, Vol. 18, Issue 2, pp. 115-127, 2004. [13] Merrill: “Component display theory”, In Reigeluth, C. M. (Ed.), Instructional-design theories and models: An overview of their current status. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc., pp. 279-333, 1983. [14] Mizoguchi, R. and Bourdeau, J.: “Using Ontological Engineering to Overcome Common AI-ED Problems”, International Journal of Artificial Intelligence in Education, Vol.11, No.2, pp.107-121, 2000. [15] Nkambou, R., Frasson, C., and Gauthier, G.: “CREAM-Tools : An Authoring Environment for Knowledge Engineering in Intelligent Tutoring Systems”. In: Murray, T., Blessing, S. and Ainsworth, S. (Eds): Authoring Tools for Advanced Technology Learning Environments: Toward cost-effective adaptative, interactive, and intelligent educational software. pp. 93-138. Kluwer Publishers, 2003. [16] Psyche, V., Bourdeau, J., Nkambou, R., and Mizoguchi, R.: “Making Learning Design Standards Work with an Ontology of Educational Theories”, Proc. of AIED2005, pp. 539-546, 2005. [17] Snelbecker, G. E.: “Is instructional theory alive and well ?”, In Reigeluth, C. M. (Ed.), Instructional-design theories and models: An overview of their current status. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc., pp. 437-472, 1983. [18] Reigeluth, C. M.: “Instructional-design: What is it and why is it?” In Reigeluth, C. M. (Ed.), Instructional-design theories and models: An overview of their current status. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc., 1983.

44

Neutral (1)

Neutral

Constructivism

(2)

Cognitivism

Neutral

Constructivism (4)

(5)

(3)

(B)

(6)

(C)

Constructivism (D)

(7)

Cognitivism

Constructivism

Constructivism Cognitivism (8)

(9)

(11)

(10)

Constructivism Cognitivism (12)

Fig. 2 A general function decomposition tree

(13)

Y. Hayashi et al. / Ontological Modeling Approach to Blending Theories

(A)

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

45

A Case of Blending Learning in Computer Teaching üüThe Model and the Application a

Li Cuiling a, Wang Hong b, Zhang Huiyu a School of Computer and Information Technology, Beijing Jiaotong University, China b Higher Education Press, China [email protected]

Abstract: In this paper, firstly we introduce four key elements of application of blending learning in teaching, Dnd then we talk about the application of blending learning in the field of computer teaching based on the practical case of curriculum of ‘Fundamentals of Computer for University’. Keywords: Blending Learning, model, application

1. What is Blending Learning? Simply put, blending learning can be described as a learning mode that mixes different delivery modalities with the objective of combining the advantage of traditional learning method and that of e-learning. That is, the instructor should lead, enlighten and watch the learning course, and also inspire students of initiative, enthusiasm and creativity during the learning course.

2. The Model --Four Radical Elements of a Blending Learning Process Based on the experience of deploying blending learning and the control over the quality of the process, four key ingredients emerge as follows: x Arrange before class. x Strategy analysis. x Performance support service. x Evaluations.

3. The Application of a Case of Blending Learning The curriculum of ‘Fundamental of Computer for University’ is made up of a web station, books, e-learning materials , etc. The web is made up of ‘Curriculum Summary’, ‘Curriculum Content’, ‘Exercises’ and ‘Schoolwork’, ‘Courseware’, ‘Practical Teaching’, and ‘Test Online’, etc. It is the platform to implement the network teaching.

46

L. Cuiling et al. / A Case of Blending Learning in Computer Teaching

3.1Preparation before class 3.1.1 Identifying the instruction goals and summary Adding the summary before the learning content can make students understand the whole learning target, and at the mean time help teachers control the learning process and pace of learning. 3.1.2 Analyzing the learners The objects of the curriculum are freshmen, and the starting levels vary widely with the differences among them in the aspect of the mastering of computer knowledge, just as the table shows.

percentage year 2002

Table 1 the Students’ level of Mastering Computer Knowledge Know some Know computer Online and Use Use simple before and can send e-mail Word Excel knowledge operate simply 43ˁ 32ˁ 23ˁ 28ˁ 18ˁ

2003

54ˁ

51ˁ

41ˁ

30ˁ

23ˁ

2004

66ˁ

62%

54ˁ

42ˁ

27ˁ

2005

72ˁ

71%

68ˁ

50ˁ

35ˁ

We found that the general level of the degree that students mastering computer knowledge increased as the time goes. It is very good for our network teaching, but we should recognize the differences among different areas and different individuals. 3.1.3 Dividing students into groups In order to carry out the process of study successfully, we divided them into groups. We considered of the elements such as the characteristics of students, original level of knowledge mastering and skills, and sex, etc. And also we encouraged them to study in cooperation. In this practice, we divided 35 students into 7 groups. We also let them vote one leader in each group. Communication inner groups and between groups were encouraged. In order to know the knowledge basis of the students, teachers conducted a pre-assessment by giving a corresponding test using the system of ‘test online’ on the web. 3.2 Implementing the process With the goals of training students of abilities of self-study, innovation and cooperation, we used the web of ‘Fundamentals of Computer for University’ and adopted different kinds of instruction strategies to develop the practice of teaching of computer for university based on the mode of blending learning. Therefore, we adopted 3 strategies as follows: class instruction, self-study and cooperative study. x Face-to-Face Instruction. According to the approach of classifying teaching results by Gagne, we classifyed instructional goals in the following domains: verbal

L. Cuiling et al. / A Case of Blending Learning in Computer Teaching

47

information, intellectual skill, psychomotor skill, and attitude. Different contents should be presented by different way, as follows:

clarifying verbal information intellectual skill psychomotor skill attitude

Table 2 Clarifying and Method of Learning Method for learning Face-to-face, self- learning, cooperative learning, etc Face-to-face, cooperative learning, discussion learning, exploring learning, research learning, etc Operation exercise, case learning cooperative learning, Face-to-face

x

Self-study. In the design of the curriculum, we considered adequately that we should train them to study by themselves, to analyze and solve problems by themselves. x Cooperative Learning. We have put forward the plan to divide students to small groups. During the learning course, small-group learning was feasible and very effective. Firstly, we assessed the former situation of groups, allowing the adjustment of group numbers and outer-group numbers, in order to avoid the disadvantage of the method of group that teachers decided. The next task was to divide the work to each number in a group. After inner-group communications, the group leader could complete this task. Then a discussion was followed to decide everyone’s task. Secondly, we chose the corresponding learning task, and developed the group learning using three different methods: Inner-group Communication, Inner-group Cooperation and Communications among Groups. At last, a simple assessment concluding an assessment to the whole fruit for every group and each member in a group was done. During the learning course, we used the tool of the web of ‘Fundamentals of Computer for University’ to develop cooperative learning, providing a new means. 3.3 Performance support materials During the practical teaching course, to provide students with good service and support for study guarantees the completion of the learning task. Considering of the characteristic of blending learning, we should do this as follows. x Providing Comprehensive Resources. It should contain teaching materials and learning materials. Supported by the web of ‘Fundamentals of computer for university’, it also should contain some digital resources, for example, powerpoint lectures and other complementary resources, etc. x Tutoring and Answering. We arranged face-to-face tutoring once a week. The teacher and teaching fellow were responsible to answer the questions, correct students’ assigned work, tutor their experiment and psychology, train them use the platform to study, and tutor their learning style. They also deployed discussion areas online to encourage students to communicate and study. x Holding lectures. Holding lectures could not only increase students’ knowledge, but also arouse their interests.

48

L. Cuiling et al. / A Case of Blending Learning in Computer Teaching

4. Evaluation A good mechanism of evaluation is a very useful support to teaching course. The evaluation of the mode of blending learning is classified into two categories: formative evaluation and summative evaluation, just as the following table shows. Table 3 Item, Content and Methods of Evaluation Content of evaluation Methods of evaluation Evaluation to students Usual behavior and the mastering Formative evaluation, degree of content and skills summative evaluation Evaluation to teachers The feasibility and effects of the Formative evaluation, instructional design summative evaluation Evaluation to learning The learning style and other Formative evaluation strategy elements during the learning course Item

Formative evaluation is the process of collecting data and information in order to improve the effectiveness instruction. It contains evaluation to the behavior and learning effects of students, the organization and service of teachers and various strategies that are adopted during the teaching activity. Its purpose is to find problems and solve them in time to guarantee the teaching effects. Summative evaluation is the process of collecting data and information in order to make decisions about the acquisition or continued use of some instruction. In the summative evaluation of this term, generally we adopted the form of final examination. Students’ usual behavior accounted for 60%, homework accounted for 20% and final exam accounted for 20%. Acknowledgments We thank the people, Wang Hong and Zhang Huiyu, who helped me a lot during the course.

References [1] Kekang He, 6ee the 1ew 'evelopment in the 7heory from %lending /earning. http://www.etc.edu.cn/academist/hkk/blending.htm. [2] Ronghuai Huang, Jinbao Zhang and Yan Dong,The Discussion about the Four Key Ingredients during the Course of Network Teaching. http://www.vschool.net.cn/elr/zhiliang/zl0015.htm. [3] McAteer, E., Tolmie, A., Du.y,C., Computer-mediated &ommunication as a /earning 5esource. Journal of Computer Associated Learning,(1997).

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

49

A Combined Method for Extracting Rules with Improved Quality ˄

1

Fuyan Liu ʳ Hangzhou Dianzi University, Hangzhou, P.R.China, 310008 [email protected]

Abstract: This paper presents an effective approach, which combines association rules and clustering algorithm for extracting rules on the web. The advantage of the method is that it can provide more precise and detailed rules compared with traditional algorithms. The paper analyzed some issues in previous approaches first. Then the proposed method is presented. Experiments are performed to evaluate the performance of the combined approach. Experimental results show that the proposed method is more effective and efficient. Keywords: Clustering, association, inference, user profile, confidence

Introduction Compared to traditional techniques, one distinguished capability of the web is the ability to perform continuous unobtrusive observations on user browsing behavior. A web server can collect the information on the decision process rather than only the decision result. Based on these data, we can capture a more comprehensive user profile, which can be used as foundation for serving users. Usually there are two types of user service: individual-based and group-based. Most previous studies focused on individual-based inference. However, to new users, the individual-based rules may not make sense, even to old users, may be still difficult to adapt to their changing preferences. Thus, it is better to use a group-based inference mechanism. In this paper some concepts such as user profile are introduced first. Then some issues in previous approaches are analyzed. In the following section an effective combined approach to infer rules with improved quality is proposed. In addition, experiments are performed to evaluate the performance of the proposed approach. The experimental results show that the combined method can provide more effective and efficient performance.

1. Some Concepts There are different views about the user profile, for example, paper [1] defined the user profile as a set of keywords, which describes the information the user is interested in; paper [2] defined the user profile as user’s likes and dislikes. We adopt a broader view of the user profile as a database, which maintains all information about users [3]. For information from inside the web site, one source is provided by customers on purpose, such as data from registration form, online questionnaire, online orders, and message board, etc. These data can be collected through CGI or Cookies. Another way to collect information from inside the web site is to observe users’ behavior without the users’ awareness, e.g. online browsing behavior can be collected by ASP, Cookies or from the log files of the server. Information from outside the web site might be collected by

50

F. Liu / A Combined Method for Extracting Rules with Improved Quality

search engine from virtual community such as bbs, newsgroups or personal homepage, etc. User data can be divided into raw data and refined data. Raw data is the original data, such as a user’s ID, age, sex, browsing behavior and purchasing records. Refined data refers to data such as usage status, habit, affinity group, and preference evaluation, etc. It is extracted, hybrid or inferred from raw data on purpose to have a better description of users. Refined data can consist of individual related data as well as group related data. It is better to classify inference foundations into two types: individual-based and group-based. The individual-based inference is based on individual data. The group-based inference is based on affinity group data. Both can be applied to infer a user profile as well as association rules. Our paper will focus on group-based inference.

2. Considerations of the Proposed Method Most of previous studies focused on individual-based inference. There are some unique problems that stem from the special characteristics when neural networks or induction learning techniques are used, e.g. a stochastic problem or large number of attributes, infinite number of states, low response rate, and difficulty in explaining results etc. may be faced. Therefore, in this paper we use association rules instead. An association rule is an expression: XY, where XI, YI, XY= I, and I={il, i2,..., im} is a set of items. Association rules can avoid the stochastic problem, since they calculate the proportion of the specific item sets in the database to identify rules rather than learn from each user [4]. Also association rules have no problems about large number of attributes and infinite number of states because the analysis unit is item sets such as transactions or browsing on the web not user’s attributes. Besides, the low response rate does not cause much trouble because browsing data can be collected from the web. In order to find association rules Apriori is the most essential algorithm. But the algorithm only explores the association between items. It is important to consider categories of different subjects in the data set. So we use Algorithm Basic, which takes account of categories to explore association rules [5]. Previous studies explore association rules based on all transactions, where they didn’t consider the concept of segmentation. Thus the extracted association rules are too general to reflect different user’s needs. But users of different affinity groups have different desires, geographical locations, preference and behavior, etc. It is better to cluster all users into different groups before exploring association rules. The main goal of cluster analysis is to identify clusters existing in the data. There are many clustering algorithms developed for different applications and k-medoid methods are more popular due to their robustness, efficiency and being independent of the order in which the objects are selected during operations. Among k-medoid methods, PAM is a simpler and well-accepted one [6]. However, PAM works quite well only for small data sets. Its computing complexity is O(k(n-k))2 for one iteration. When the number of objects n and the number of clusters k are quite large, then it is too costly. In this paper we adopt CLARA for clustering [7]. Unlike PAM which will deal with the entire data set, CLARA draws a sample of the data set first, then applies PAM on the sample. Its computing complexity is O(ks2 +k(n-k)), where s is the sample size and it is quite smaller than n.

3. Framework of the Method We propose a combined method to infer association rules, which is shown in Fig.1. Clustering is implemented first, which is based on user’s demographic, geographic, and

F. Liu / A Combined Method for Extracting Rules with Improved Quality

51

behavioral data stored in the raw data, while using CLARA to identify the affinity group. The k-medoids and the individual user clusters are recorded in the refined data set of the user profile for the purpose of identifying the affinity groups that users belong to. Then for each cluster, Algorithm Basic is used to Inference engine learn the associations of relevant subjects Association Clustering such as browsing patterns and transactions rule algorithm algorithm to find association rules, which are valid for (A-BASIC) (CLARA) a particular cluster only. These rules retrieve transactions or browsing patterns from the raw data set. For the description of Output Row Site site contents and advertisements, it retrieves Refined rule base related data from the site database. The data set data set database User profile results of group-based association rules are stored in output rule base. Fig.1. Diagram of the combined method 4. Experiments and Conclusions In order to evaluate the proposed method, experiments were performed. In the experiments, a user profile was built first by the method and then the performance of the method was evaluated. The data used are collected from experiments of online assignment in the course of the Management Information Systems (MIS), which involves five categories: Fundamental Techniques (FT), Strategic Planning and Development Approaches (SP&DA), System Analysis (SA), System Design (SD) and System Implementation (SI). There are 198 students of five classes participating in the experiments over a period of two weeks in our computer lab. The experiments began with a registration form that required students to fill in their personal data and then submit to the lab server. We also used log files in the lab server to collect browsing behaviors of students during the specified period. In the experiments, students are firstly classified by CLARA of a k-medoid method before exploring the association rules. The collected students’ demographic and behavioral data are used for clustering. The registration form is used to collect students’ sex, age, class, daily hours of browsing, daily hours of online doing assignment, date of finishing assignment and total hours spent on assignment. It is assumed that the students may fall into one of the three groups according to our experience and observation: Ɣ Group 1: They visit the web site regularly, almost every day, and spend more time on the assignment during the first week of the specified period and they finish the assignment in advance. Ɣ Group 2: They visit the web site randomly and spend less time on the assignment during the first week of the specified period. But they spend much more time during the second week and finish the assignment on time. Ɣ Group 3: They visit the web site randomly but almost do nothing on the assignment each day. And they rush into the assignment on the last one or two days during the specified period and can not finish assignment on time. As stated earlier, in this paper we use CLARA algorithm to perform clustering. For accuracy, the quality of the clustering is measured by the average dissimilarity of all objects in the entire data set, although CLARA draws a sample of the data set. Then we apply PAM on the sample. Here we set the number of samples to 3 and set the size of each sample to 60. The predefined 3 clusters are classified as shown in Table 1. It can be seen from Table 1 that cluster 1 has more female students and above 89.7% students of cluster 1 visited the web regularly every day and finished the assignment in

52

F. Liu / A Combined Method for Extracting Rules with Improved Quality

advance. Above a half of students spent less than 40 hours for finishing the assignment. The other two clusters in Table 1 can be analyzed similarly. After clustering, the association rules for browsing behavior were extracted with Algorithm Basic based on all students with and without clustering. As a result, the numbers of association rules are 55 and 13 respectively and the group-based approach can provide more detailed rules. Table 1. Result of clustering Cluster attributes Sex Age Total hours spent on assignment Date of finishing assignment Daily online hours

Male Female Average 30 hours 30~40 hours >40 hours In advance On the last day Delay Regularly Randomly

Cluster 1 10.6% 13.6% 21.4 5.1% 7.6% 11.6% 24.2% 0.0% 0.0% 21.7% 2.5%

Cluster 2 35.9% 24.7% 22.6 0.0% 0.0% 60.6% 35.4% 25.3% 0.00% 25.3% 35.4%

Cluster 3 9.1% 6.1% 20.9 14.1% 1.1% 0.0% 0.0% 0.0% 15.2% 0.0% 15.2%

Now we compare the performance using the association rules and two measures: support and confidence for the non-clustering entire data set and clusters separately. Table 2 shows the comparison of clusters with the non-clustering entire data set for each measure and each association rule. All rules in cluster 1, cluster 2 and cluster 3 have higher support and higher confidence than that of the non-clustering data set. Table 2. Measure comparison on browsing NonClustering Clustering Cluster 1 Cluster 2 Cluster 3 Association rules S C S C S C S C (C1,C2) 0.13 0.35 0.33 0.49 0.28 0.45 0.15 0.38 (C2,C3) 0.15 0.38 0.40 0.52 0.34 0.48 0.17 0.40 (C3,C4) 0.15 0.40 0.43 0.54 0.30 0.56 0.16 0.43 (A,C4) 0.12 0.26 0.18 0.31 0.20 0.32 0.12 0.28 (A,C5) 0.11 0.22 0.15 0.28 0.16 0.28 0.12 0.22 Note: (1) the value of support is set to 0.1; S, C represent degrees of support and confidence respectively. (2) A and C1, C2, C3, C4, C5 represent the assignment and five categories: FT, SP&DA, SA, SD, SI.

In this paper a combined approach is proposed to infer rules of user browsing on the web, which combines clustering and association rules to avoid the troubles such as the stochastic problem, the interpretation difficulty, too many attributes and a low response rate etc. Experimental results show that the combined approach increases the degree of both support and confidence of association rules and it confirms that the proposed approach can produce more detailed and precise rules.

References [1] John Davies, Richard Weeks, Mike Revett. Jasper: Communicating information agents for WWW. http:// www.w3.org/Conferences/WWW4/Papers/180/. [2] George Leaks, George M. Giggles (2004) A Lifestyle-based approach for delivering personalized advertisements in digital interactive television. Journal of Computer-Mediated Communication, 9, 2, 23-45. [3] Lai H., and Yang, T. C. (1998) A system architecture of intelligent-guided browsing on the Web. Proceedings of the Thirty-First Annual Hawaii International Conference on System Sciences, 4, 423-432. [4] Hsiangchu Lai and Tzyy-Ching Yang (2000) A group-based inference approach to customized marketing on the Web-integrating clustering and association rules techniques. Proceedings of the 33rd Hawaii International Conference on System Sciences, 6054-6063. [5] Shi Zhong-zhi (2002) Knowledge Discovery. Tsinghua University Press, Beijing. [6] Carey V. J, Kollekolle. http://www.biostat.ku. dk/~pd/bioC-2003/vclec4.pdf. [7] Shu-Chuan Chu, John F. Roddick, Tsong-Yi Chen and Jeng-Shyang Pan (2002) Efficient search approaches for K-medoids-based algorithms. Proceedings of TENCON '02, 712-715.

Programming

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

55

The Impact of CABLE on Teaching Computer Programming Ioana CHAN MOW, Wing K. AU & Gregory C. R. YATES Dean of Science, National University of Samoa Program Director & Senior Lecturer, University of South Australia Senior Lecturer, University of South Australia [email protected] Abstract: This paper reports the second of two studies on the impact of a Cognitive Apprenticeship-Based Learning Environment (CABLE) in the teaching of computer programming. The pedagogical model used in this study employs a combination of instructional strategies including directive support, responsive cognitive apprenticeship, collaborative learning, stimulating metacognition, using technologies via the use of teleapprenticeship and online discussion. In an earlier study, students who participated within the CABLE project scored more highly on test scores, relative to comparable students who did not participate within CABLE, but these effects were found to be restricted to highability students. In the present study, students who participated within CABLE scored more highly than those participating within the non-CABLE group. However, with an enhanced CABLE environment, the benefits of CABLE were now evident in both ability groups, with the effects being more prominent within the low-ability group. Keywords: Cognitive metacognition

apprenticeship,

collaborative

learning,

tele-apprenticeship,

1 Introduction Computer programming is a difficult and challenging subject area which places a heavy cognitive load on students [1, 2]. After two years of learning programming, most novice programmers are still struggling to be proficient [3, 2]. From an examination of current research in this field, it can be postulated, that one reason computer programming instruction seldom results in the successful transfer of problem solving skills, lies in a lack of understanding about good instructional approaches in this direction [4, 5]. This paper reports the second of two studies on the impact of a Cognitive Apprenticeship-Based Learning Environment (CABLE) in the teaching of computer programming that would promote problem-solving skills of university students in Samoa. The main instructional approach used in this study is described as cognitive apprenticeship. The notion of apprenticeship stems from Vygotskian psychology, but we based our approach after the work of cognitive psychologists such as Collins, Brown and Newman [6] and others. Cognitive apprenticeship places emphasis upon reflective thinking as a metacognitive experience. The approach encourages metacognitive thinking, the use of directed teacher instruction to guide problem solving, and the use of scaffolding. The pedagogical model researched in the two studies represents our attempt to devise a viable instructional model based around the construct of apprenticeship. The aim was to achieve this goal through using a combination of instructional strategies working in concert to

56

I. Chan Mow et al. / The Impact of CABLE on Teaching Computer Programming

provide the learners with a highly demanding but responsive instructional experience that differed from those derived within more traditional learning environments. Cognitive apprenticeship is a model of instruction that involves the effective communication of domain knowledge in such a way that the students become aware of the thought processes involved in knowledge construction within that domain. As an instructional approach it is directed at teaching processes that experts use to handle complex tasks, characterized by a number of teaching methods [6]. The first of these, is modelling where the teacher models his or her thought processes in solving problems within a domain. The second of these methods is guided practice or coaching where the student attempts to solve the problems for themselves with the support of the teacher to answer specific queries. A third method is scaffolding, where the teacher assists students to manage complex task performance and then gradually withdraws support from the student (fading). Other key components of this approach include articulation, where the student attempts to articulate their problem solving strategies; reflection, where the students are encouraged to reflect on how they approached tasks and solved problems, possibly by discussion with other students and, finally, exploration which is intended to encourage learner autonomy and problem formulation by students. Another instructional strategy that is gaining prominence as an effective teaching method and is integrated into the learning environment being trialled is collaborative learning. There are many approaches to collaborative learning but all have the following characteristics in common [7]. It is a learning activity suitable for group work; it is small group based (usually 2-5); it has tasks which encourage cooperative behaviour; it is characterized by student interdependence; individual student accountability and responsibility for task completion. Our starting assumption is that the computer can be used as a tool assisting in both cognitive apprenticeship and collaborative learning. An important element of the CABLE learning environment is its use of computer-mediated communication techniques for implementing aspects of cognitive apprenticeship such as scaffolding, coaching, feedback and modelling. Computer-mediated communication refers to communication using a computer. Examples of computer-mediated communication include email, computer conferencing and electronic bulletin boards. How these techniques are utilized to support the learning process is referred to as electronic apprenticeship or tele-apprenticeship. Teleapprenticeship or online apprenticeship refers to the use of computer mediated communication techniques, for the implementation of cognitive apprenticeship [8]. Computer mediated communication techniques used in implementing tele-apprenticeship in the current study include email, online notes, interactive tests and a bulletin board. As mentioned earlier, the term used to refer to this pedagogical model, is the cognitive apprenticeship-based learning environment or CABLE, because although it is based on the cognitive apprenticeship approach, the approach used in this study, is further enhanced by the incorporation of elements of metacognition and collaborative learning, instructional elements, which studies have proven to be conducive to effective learning. Furthermore some elements of this cognitive apprenticeship based approach will be implemented electronically by means of email, bulletin board, online notes and interactive quizzes. This is referred to as tele-apprenticeship and a subsidiary aim of this research was to investigate the impact of computer-mediated communication in enhancing student learning. CABLE is a hybrid learning environment and it is implemented both in face-toface and online mode. This paper reports the findings of our second study on the impact of CABLE. In the initial study, results showed that students who participated in CABLE scored more highly on a post test measure of mastery of Java programming, relative to comparable

I. Chan Mow et al. / The Impact of CABLE on Teaching Computer Programming

57

students who did not participate within CABLE. However, these effects were found to be restricted to high-ability students. These results were disappointing in that the hope was to develop instructional procedures that would provide a richly motivating and responsive educational context that would appeal to low-ability students. One goal of this program lies within reducing variance within student achievement levels. From these findings our team re-evaluated our procedures and modified CABLE to include more scaffolding and more online resources. The effectiveness of this enhanced CABLE model is evaluated in the present study.

2 Methodology 2.1 Research design

Students participating in University Foundation Computer Studies class at the National University of Samoa were allocated to either the CABLE treatment or were taught in accordance with the traditional university model of teaching (i.e. Non-CABLE methods). This division was achieved through students being enrolled within different class times. Although the initial total enrolment was 80 students, complete data were able to be collected from 39 students within the CABLE group, and 33 students from the traditional group. Participants were students within their first year of studies from the Foundation (64%) and non-Foundation (36%) programs. The Foundation program was for students who entered at a higher GPA and were preparing for studies in overseas universities whereas the non-Foundation program was for those who entered at a lower GPA and intending to pursue study locally. The approximate average age of the participants was 19 years. Student records were available from previous computing courses in the form of test scores. The design of the present study was very similar to the first study. After six weeks of exposure to the treatments, all students completed the final test paper, and also completed a questionnaire intended to tap into their evaluations of their course experience. The post-test consisted of six questions which evaluated their knowledge of Java commands and also practical questions which tested the application of these Java skills and knowledge to solving a problem. Students were given program codes to explain, to troubleshoot and to predict program output. Furthermore, the assumption made here was that computer programming was a form of problem solving and hence these scores represented the problem-solving ability of these students. The questionnaire consisted of Likert items which gauged students’ attitudes towards the learning environment, effectiveness of feedback, effectiveness of collaborative learning, the effects on self-confidence, and students’ love of learning. A diary of students’ interviews was kept in order to provide some qualitative, narrative and descriptive data on the study. It is important to point out that there was considerable overlap in the instructional approaches in the two treatments. Students in both the cognitive apprenticeship and traditional groups were given the same set of notes and exercises on JAVA programming. Both groups were exposed to elements of the cognitive apprenticeship based approach such as feedback and coaching. In both groups, the teacher modelled computer programming theory and JAVA programming concepts using worked examples and reallife examples. A main difference between the two groups was in the provision of feedback to the students. In the control group, as was characteristic of didactic instruction, feedback was

58

I. Chan Mow et al. / The Impact of CABLE on Teaching Computer Programming

student initiated. With the experimental group, feedback was more structured and was given on a weekly basis. Feedback was provided by an online system where the lecturer provided individualised feedback via individualised emails sent to and received from each student. On a weekly basis, the students were expected to send an email to the lecturer, answering several questions. The first question required them to describe what activities and topics they had done during the week, and to indicate how they felt about their progress. The second and third question required them to describe any areas they are having problems with. The last question required the student to reflect upon what they have learnt and how useful they thought what they had learnt would be to them. The lecturer would then respond to each student via email by way of feedback and encouragement. From the individualised feedback, the lecturer could gauge areas most students were having problems with and then use their information to post some frequently asked questions (FAQs) and solutions on the class web-site, providing further feedback and guidance to students in the class. It was assumed that this feedback system would provide students with the opportunity to articulate their thoughts and ideas and also to reflect upon their work and their progress in class. A second differentiating factor between the two groups was metacognition. A key component of the cognitive apprenticeship based approach was the provision of a rich metacognitive experience to the learner, thus facilitating them to learn. This was facilitated within the cognitive group by encouraging students to reflect on their progress, problems encountered, what they had learnt, the usefulness of what they had learnt and also by the articulation of their thinking processes in the form of “think-alouds”. The third differentiating factor between the two groups was the incorporation of elements of collaborative learning. Within the cognitive apprenticeship based group, coaching and mediation would also be provided by a more capable peer as the students were be paired, with the more capable student collaborating with the weaker student in carrying out their programming tasks in class. 3 Results and Discussion 3.1 Achievement Test Scores The two treatment groups were found to be similar on non-Java prior test scores from earlier completed course units but diverged on the Java post-test scores. Results showed that there was an effect for CABLE treatment on post-test scores. That is, people who participated in CABLE scored more highly on the Java post-test than those who were in the traditional treatment. Statistical ANOVA procedures were used and showed a significant overall effect for treatment, F (1,53) = 8.48, p = .005. For further investigation, the two treatment groups were then split into two groups on the basis of the prior test scores. The prior test scores were taken as a measure of ability level. The median value of 50 was used, to generate a classification of high-ability and low-ability students. From an inspection of the data from the two ability levels separately, it was apparent that although the CABLE treatment had benefited both ability groups, the effect was more prominent in the less able group. The treatment effect was stronger in the less able group, with the F ratio increasing to 9.13 (p =.006). Inspection of effect-sizes (Cohen’s procedure) yielded d = 1.22 for low-ability and d = 0.66 for the more able group [The coefficient d is expressed in standard deviation units]. In other words, although the CABLE treatment benefited both ability groups, the effect was stronger for the less able group, as indexed upon through their prior achievement scores. This interaction effect is shown in Figure 1

I. Chan Mow et al. / The Impact of CABLE on Teaching Computer Programming

59

18

16

Java Posttest Means

14

12

TREAT

10

Trad 8

CABLE Low Ability

High Ability

ᴾ

Figure 1. Mean scores on post-test for students of low and high ability

3.2 Questionnaire responses A measure of general course affect (or ‘liking’) was generated through summing the responses to 11 items on the questionnaire. It was found that on this measure, the two treatment groups exhibited similar levels. Out of a possible score of 44, the actual mean was 34.1 (SD of 3.96), which indicated a very high level of course approval (Note: the natural midpoint on the scale was 27.5). In short, students from both treatment groups evidenced very high levels of liking for their course experience. The levels of course affect was found to be independent of either treatment mode or ability status. ᴾ

3.2.1 Responses to the Unstructured questions Responses to Question xviii: Probe: Problems encountered (if any) during treatment

For the traditional treatment, 6 out of 24 students reported having no problems in using this learning mode. The main problem students identified was the difficulty in understanding Java programming (12 out of 24 students). Other problems reported included (a) not enough time to do work, (b) difficulty in accessing the lecturer, and (c) difficulty in specific Java concepts such as declaring types, running the program. In the CABLE treatment, 4 out of 24 students reported having no problems with the learning environment. As in the traditional group the main problems students reported were (a) difficulty in understanding Java, and (b) too many Java terms to learn. Responses to Question xix Probe: List reasons why you think this is an effective or ineffective form of learning environment

Fifteen out of 24 students in the traditional treatment agreed it was an effective learning mode. The following reasons were given: (a) it kept students alert, occupied and provided experience of working under pressure, (b) it prepared students for programming independent of the lecturer, (c) improved student understanding of Java, (d) the provision

60

I. Chan Mow et al. / The Impact of CABLE on Teaching Computer Programming

of a balance of both practical and theory improved understanding of Java, (e) provided encouragement to learn, and (f) provided more computing knowledge and skills. Five students found the learning environment ineffective. One claimed it was because it was the first time they had studied computers, while another resented disruptions from other students. For the CABLE treatment, 15 of 24 students reported the learning mode as effective. The main reasons given were (a) the use of both practicals and lectures facilitated a better understanding of Java, (b) increased motivation and improved confidence in learning Java, (c) improved understanding of Java and, (d) the use of email. Two students found the learning environment ineffective, one of them giving the reason as insufficient coverage time for topics. 3.2.2 Effectiveness of Online learning To gauge the effectiveness of online learning, three data sources were used, (a) the responses to the questionnaire questions 12 – 17 administered only during the cognitive apprenticeship based treatment, (b) responses from personal student interviews, conducted in weeks 6 and weeks 12 of the study, and (c) an analysis of products, processes and perceptions based on the Triple Framework approach for evaluating online environments [9]. Questionnaire questions and interviews with students focused on four main issues: (a) The effectiveness of email for feedback (b) The effectiveness of online notes on class website (c) The usefulness of posting sample test solutions online (d) Whether students like working in pairs (a) The effectiveness of email for feedback The majority of the students interviewed agreed on the usefulness and effectiveness of email as a means of feedback. The main source of frustration was technical problems preventing effective access to email. Most of the students regarded email as very helpful as it gave useful and immediate feedback. (b) The effectiveness of online notes on class website All of the students interviewed liked online notes, giving the main reason for liking, as the ease of access and also that they could access the online notes at any time. Students also found the interactive self tests useful as a means of reviewing before the Java test. Again, the main complaints were technical. (c) The usefulness of posting sample test solutions online All of the students interviewed liked the idea of sample solutions online as they said they could i) access them any time, ii) useful for revision and iii) were useful for doing test corrections. (d) Whether students liked working in pairs All of the interviewees liked working in groups as they could help each other sharing ideas, especially when there were some things that the other group members knew, that they had no knowledge of. Another important source of data was the log of student email. According to Salomon as cited in [9], there are 5 stages in the development of an online learning

I. Chan Mow et al. / The Impact of CABLE on Teaching Computer Programming

61

community. The first stage is when community members develop the motivation to access and use the web environment proficiently. In the second stage, members are able to establish online identities and take the initiative to socialise with others online. Stage three is characterised by participants initiating the process of assisting and providing mutual support in information exchange. The process develops to stage four when course related group discussions eventuate and there is increased collaboration amongst members of the online community as they devise various means of collaborating in online work. Finally, the last stage, stage five, is characterised by members of the online community showing how the online learning has facilitated the achievement of personal goals and an ability to reflect on the learning process. Inspection of the log of student emails indicated that from the perceptions, processes and products of online learning using the Triple P Framework as developed by Ryba, Selby and Mentis [9], showed that the online community in Project 2, had progressed to stage three of the five stage model where students were involved in information exchange using email and the discussion forum. In terms of processes, the students were not only proficient in using the online environment, but were also using the online environment for receiving coaching, feedback and scaffolding. Students were also using the email facility for online discussion forums and for a few of them, the ability to use it for reflection on their work. In terms of products, student participation in email included: x Technical messages relating to the website or managing of the helpdesk. x Questions related to course work Lecturer participation was in the form of providing encouragement to the students, providing feedback on student queries and bringing to the notice of students valuable features of the online environment. Hence the effectiveness of the online learning environment is suggested from student interview responses, an examination of the perceptions of students, an analysis of the content of student email messages and the processes students engaged in.

4 Conclusion In essence it was found that: x CABLE, as a viable model of conceiving and delivering a high quality instructional aid system, receives a measure of strong positive support from the present results. The results showed that on the overall, students were advantaged through their participation within this program. However, at the group level, significant achievement effects for the CABLE treatment were more prominent in the case of students in the less able group as indexed on prior achievement scores. x In terms of positive attitudes towards the learning environment, results of the study indicate that, all the participants showed strong positive feelings towards their allocated treatment. x There is positive evidence for the effectiveness of online learning in all of the student interview and questionnaire responses and also from the analysis of email content and the processes students were engaged in. x The results indicate the effectiveness of CABLE as a viable instructional model for teaching Computer programming. However the results of the two studies differed in terms of which groups had benefited. In our initial study the high-ability students were clearly advantaged by CABLE. However in the current study, the low-ability students appeared to benefit slightly more than the high-ability students, although both ability groups showed clear advantages in the CABLE. Why might two similar studies yield

62

I. Chan Mow et al. / The Impact of CABLE on Teaching Computer Programming

slightly different findings? The solution we favour is that our CABLE model continues undergoing development firstly in terms of improvement and familiarity with procedures. Also in the second study the lecturing team placed relatively greater emphasis upon individualised feedback. It can be noted that the two studies differed slightly in the composition of the participant groups. In the first study, all the participants were from the Foundation program. These were generally the more able students, entering with high GPA’s. In the second study, 64% of the participants were from the Foundation program. However there were also 36% from the non-Foundation program, generally entering with low GPA’s. Despite the difference in ability levels of these two groups, there was no interaction between program and treatment, as CABLE had a positive effect on achievement levels of participants from both programs. Hence, the results of the present study have boosted the confidence of our team in the viability of CABLE. It has also confirmed that the provision of excellent instructional procedures in a richly motivating and responsive educational context can have appeal and positive effects on students who may not otherwise perform to a high level. In this respect CABLE can enable us to achieve the ultimate goal of reducing variance within student achievement levels. This is encouraging, especially within the context of teaching computer programming as the subject area is very challenging and cognitively demanding. Hence an effective instructional model would certainly improve the quality of instruction of such an inherently demanding subject and ultimately result in improved achievement levels of Computer programming students within the university.

References [1] Astrachan, O., Selby, T., & Unger, J. (1996). An object-oriented, apprenticeship approach to data structures using simulation: Proceedings of FIE '96, Frontiers in Education (pp. 130 – 134). Retrieved June 25th 2004 from [http://www.cs.duke.edu/~ola/papers/fie96.html]. [2] Garner, S. (2000). Cognitive load reduction in problem solving domains, Perth, Australia: Edith Cowan University. [3] AECT. (2001). The handbook of research for educational communications and technology. Retrieved June 12, 2003 from http://www.aect.org/intranet/publications/edtech/24/24-05.html [4] Au, W. K. (1992). Logo programming: Instructional methods and problem-solving. Unpublished doctoral dissertation. Palmerston North, New Zealand: Massey University. [5] Blair, A., & Hume, T. (1994). An exploration of the application of constructive learning techniques to software development using object orientation as a Vehicle. Paper presented at CTI Annual Conference, 1994. Retrieved March 12, 2003, from http://www.ulst.ac.uk/cticomp/therhume.html [6] Collins, A. (1989). Cognitive apprenticeship and instructional technology (Technical Report No. 474). BBN Laboratories, Cambridge, MA: Centre for the Study of Reading, University of Illinois. [7] Cooper, J., Prescott. S, Cook, L, Smith., L, Mueck, R., & Cuseo, J. (1990). Cooperative learning and college instruction: Effective use of student learning teams, California State University Foundation, Long Beach. CA. [8] Levin, J., & Waugh, M. (1998). Teaching tele-apprenticeships: Electronic network-based educational frameworks for improving teacher education. Journal of Interactive Learning Environments, 6(1-2), 39-58. [9] Ryba, K., Selby, L., & Mentis, M. (2001). Analysing the effectiveness of on-line learning found in communities. [Electronic version]. Retrieved September 25, 2003, from http:// www.ecu.edu.au/conferences/herdsa/main/papers/nonref/pdf/KenRyba.pdf.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

63

Problem Solving Process Oriented Diagnosis in Logic Programming Nguyen-Thinh Le, Wolfgang Menzel Department of Informatics, University of Hamburg, Germany {le,menzel}@informatik.uni-hamburg.de Abstract. In this paper, we present the evaluation result of our constraint-based tutoring system for logic programming from which we derive the conclusion that students need diagnostic information and remedial hints corresponding to the stage of the problem solving process where they are stuck. For this reason, we propose a three steps diagnosis approach which consists of: diagnosis at the task analysis stage, diagnosis at the solution design stage and diagnosis at the implementation stage. Our diagnosis approach should not only help students learn logic programming, but also master the skills of task analysis and solution design. Keywords: Constraint-based modeling, cognitive diagnosis, tutoring systems, logic programming, problem-solving process, instructional process.

Introduction Error diagnosis plays an important role in an Intelligent Tutoring System (ITS) because diagnostic information is essential for modelling the state of student’s knowledge and for initiating appropriate instructional actions. Currently, several programming tutoring systems apply a rather simple diagnostic approach by presenting a problem to the student and providing the possibility to submit a solution by choosing from several options or filling in a template. Diagnostic approaches supported by this type of solution submission might be used in tutoring systems aiming at helping students to become familiar with basic concepts of a programming language. In problem solving, however, students often make errors because they have difficulties with task analysis or solution design. Thus, at that point, diagnostic information about semantic or syntactic errors is not relevant. Rather it is important to consider the stage of the problem solving process, where the student becomes stuck: task analysis, solution design or implementation. We have developed a web-based tutoring system for logic programming applying the constraint-based modeling (CBM) approach. In this paper, first, we introduce the constraint-based approach briefly. In the second section, we present the result of the preliminary evaluation of our system. From the analysis of the evaluation result, we claim that diagnostic information is only useful for students if it matches the stage of the problem solving process where the student has difficulties. In the third section we outline related work. Our diagnosis approach is introduced in the fourth section. The current state and further directions of our research are summarized in the last section. 1. Constraint-based Modeling The CBM approach proposed in [1] can be applied to model general principles of a domain as a set of constraints. A constraint is represented as an ordered pair consisting of

64

N.-T. Le and W. Menzel / Problem Solving Process Oriented Diagnosis in Logic Programming

a relevance part and a satisfaction part: Constraint C =
satisfaction part>

The relevance part represents circumstances under which the constraint applies, and the satisfaction part represents a condition that needs to be fulfilled for the constraint to be satisfied. Constraints can be used to describe facts, principles or conditions which must hold for every solution contributed by the student. In addition, constraints can also be used to specify requirements of a task. Using the relevance part, constraints can be tailored according to an ideal solution, which represents the requirements of the given task. Requirements, which have to be satisfied in that specific situation, can be specified in the satisfaction part. More about the application of the CBM approach to model problem solving in logic programming can be found in [3, 4]. A constraint is evaluated by matching its relevance part to the solution. If the matching is successful then the solution should also fulfill the satisfaction part. Otherwise, the solution is considered to be incorrect with respect to the constraint that has been evaluated. If a constraint is violated, it indicates that the student solution does not obey principles of the domain or does not meet the requirements of the given task. We developed a tutoring system (INCOM) [3] for logic programming applying the CBM technique. The diagnosis approach of our current system consists of two steps. It starts by hypothesising the Prolog pattern the student solution is based on. A Prolog pattern represents a solution strategy for a programming problem [4]. For a given programming problem, there are usually several appropriate patterns which can be applied to solve it. The pattern selection is carried out heuristically. The second step of the diagnosis examines whether the student solution satisfies the task requirements. In the evaluated version, we use constraints to model task requirements. If a constraint is violated, a programming technique has been applied incorrectly or a task requirement is not fulfilled. 2. Evaluation and the Problem of Remedial Hints 2.1 Evaluation results We have conducted a preliminary evaluation for INCOM during the winter term 2004/05 at the University of Hamburg. We provided students with four exercise assignments: 1. Define a predicate which specifies the relationship between a list and its prefix. 2. Write a function to convert Peano numbers to integer numbers. 3. Write a predicate which defines an even Peano number. 4. Write a function to compute the sum of compound interest for a given amount, an interest rate and a duration in years. Students have been requested to consult our system via a web interface when experiencing difficulties in solving those four exercise assignments. On our server machine, we registered 261 log files created by 99 users. Table 1 number of false and correct trials for each task trials for a correct solution task solved

Task

trials/user

task not solved

1

6.07

4.33

11

7

2

6.21

6.54

22

23

3

5.72

6.83

27

17

4

6.21

74.5

1

24

The first goal of our evaluation was to find out which task is challenging for students. Table 1 provides the results of problem solving for each task. The 2nd and 3rd

N.-T. Le and W. Menzel / Problem Solving Process Oriented Diagnosis in Logic Programming

65

columns show, how many trials in average a user carried out, and how many trials he/she needed to reach a correct solution. The last two columns tell us, how many log files contained correct solutions and how many did not. From the last two columns we can identify that most students could solve Task 1 and 3. For Task 2, the numbers of successful and unsuccessful attempts are almost the same. We also noticed that most users could not solve task 4. Analyzing the log files, we found two main reasons for these frequent failures: 1. Task 4 is more complicated than the other tasks. It includes the concept of recursion, arithmetic expressions, and arithmetic computation; 2. Many users were not able to derive a correct formula for the computation of a compound interest. The second goal of our evaluation was to identify the problems of students and where they are usually stuck. By investigating the log files, we recognized that most errors have been detected by the first step of the diagnosis - the pattern identification process. Most users had one of the following problems: • Users were not familiar with the data structure of Peano numbers. Some of them simply input “Peano” as arguments and expected that to be a Peano number. • The arithmetic evaluation mechanism in Prolog poses considerable problems for many users. Some of them placed an arithmetic expression at an argument position and expected a functional evaluation. Others used “=” instead of “is” for arithmetic evaluation, as it is common in mathematical notations. • Users called auxiliary predicates without defining them in the hope that they are builtin predicates. Or, they used arbitrary material at an argument or subgoal position and expected that the system is able to provide helpful hints. Through errors detected during the second step of the diagnosis - the constraint evaluation process, we noticed that users had the following problems: • Many users applied arithmetic expressions without making sure that the arguments are sufficiently instantiated. Sometimes, they transposed the positions of operands and result arguments or used operands not correctly. • Instead of decomposing an input argument, many novice programmers composed it in recursive subgoals. Or, they decomposed an input argument and processed it, but then, they did not know how to return the result of the processed input value. This indicates that Prolog novices are not familiar with composition and decomposition. The third goal of our evaluation was to determine the efficacy of the system. Our system provides three levels of feedback. If the system detects an error in the user’s solution, first, it notifies that the solution is incorrect; second, upon request, it shows the problem location, gives an explanation and third, it provides suggestions to remove the error. We evaluate the efficacy of the system by determining if errors disappeared after users have seen the error location or a remedial hint. A remedial hint includes an error explanation and a correction proposal. In 75.6 % of a total of 632 false trials, students were interested in system feedback. In 60.5% of 478 feedback requests, after seeing the error location, they requested more detailed error explanation and remedial hints. That indicates that most students are interested in receiving feedback from the system in order to improve their solutions. In 68.3% of cases after seeing the error location without requesting remedial hint, users were able to remove the error. In 75.8% of cases after requesting remedial hints, the error was eliminated. As expected, the efficacy of remedial hints is higher than that of error location because remedial hints give more information. In general the system is helpful for students. However, the efficacy of our current system does not satisfy our ambitions. We investigated the log files to trace back how students have corrected their solutions after having read the feedback. We noticed that students could not correct their solutions according to some remedial hints provided by our system. This can be attributed 1) to the

66

N.-T. Le and W. Menzel / Problem Solving Process Oriented Diagnosis in Logic Programming

incoherence of the remedial hints which are specified for isolated constraints [5, 6] and 2) to a mismatch between the feedback and the stage of the problem solving process where the student’s difficulty occurred. In this paper, we mainly address the second problem. 2.2 The problem of solution remediation We illustrate the problem of providing students with appropriate feedback with the following example. The third exercise assignment requests students to define a predicate which specifies the relationship between a list and its prefix. Our system expects a correct solution like IP1 or IP2 which uses an auxiliary predicate append. Task: define a predicate to examine the relationship between a list and its prefix Solution IP1: prefix([], _). prefix([H|R], [H|T]):-prefix(R,T).

Solution IP2: IP2: prefix(L1, L2):-append(L1, Rest, L2).

A student submitted the following solution SP1 for the task above. Our system hypothesizes that the student decided to apply the strategy IP1, it then evaluates the relevant constraints and returns the corresponding diagnostic information. Student solution SP1:

prefix(List, List). prefix([], List).

Remedial hints: Error1: a base case in your solution is superfluous. Error2: a recursive case in your solution is missing.

The solution indicates that the student is in a position to specify a base case “prefix([], List).” but not able to specify a recursive case. Perhaps the student wanted to specify the type restriction for the argument positions by giving the clause “prefix(List, List)”. More likely, however, it is that the student does not know how to specify a list data structure which is required for both predicate arguments. That means he/she is not able to fully analyze the task and to specify the arguments correctly. Therefore, remedial hints concerning solution design are not helpful for the author of the solution above. In this case, we need to help the student analyze the task requirements. The task analysis includes questions like: Which information should be represented as an argument? What kind of data structures should be specified for predicate arguments? Which mode should an argument have? The following student’s solution SP2 indicates that he/she has succeeded with the task analysis, but is now struggling with designing a solution for the given task. Student solution SP2:

prefix([X], [X]). prefix(L, [X|Rest]):- append(H, Rest, [X|Rest]),prefix(H, Rest).

Remedial hints: Error1: the subgoal prefix(H, Rest) is superfluous. Error2: the clause prefix([X], [X]) is superfluous. Error3: L should be unified with H. Error4: the argument [X|Rest] in the head of 2nd clause should be represented as a variable.

The system hypothesizes that the student was following strategy IP2 and evaluates the corresponding constraints. While the first two feedback messages concern the solution design corresponding to the strategy IP2, the last two consider the erroneous implementation of arguments. This might have caused the student to be confused because she/he is currently having problems with designing the solution, not with the

N.-T. Le and W. Menzel / Problem Solving Process Oriented Diagnosis in Logic Programming

67

implementation. At this stage, the student has to deal with the questions: what kind of clauses and subgoals are required to construct a solution according to the intended design strategy, and how these clauses and subgoals have to be arranged? Table 2 tells us the proportion of false attempts due to errors in task analysis and in solution design. The second and the third columns indicate the absolute number of students’ attempts to solve the tasks. We notice that students made most errors (70%) at the stage of analyzing Task 3. That means, they were not able to specify a Peano number correctly. We also see that students had difficulties with designing solutions for Task 4. 42% of their attempts for Task 4 were not successful at finding appropriate clauses or subgoals. Table 2 Proportion of false attempts due to errors in task analysis and in solution design. Task Total attempts False attempts Errors in task analysis Errors in solution design 1

91

70

7%

20%

2

242

205

25%

18%

3

246

210

70%

17%

4

149

147

9%

42%

From the evaluation result and the analysis of student’s programs above, we can derive the need for a diagnosis approach which is able to provide diagnostic information corresponding to the stage of the problem solving process. 3. Related works Available tutoring systems are able to detect semantic or syntactic errors in a program. However, this kind of diagnostic information is not useful for students who already have difficulties in the early phases of problem solving. The problem is to determine which level of understanding the student has and how to guide him/her to correct his/her solution in a way he/she is supposed to do. Various attempts have been developed in this direction, but none of them is really able to provide diagnostic information tailored to the stage where the difficulties occurred. The Pascal tutoring system [7] is able to infer the student’s intention and to diagnose errors by mapping a student program to programming plans. This system focuses the diagnosis mainly on the solution design applying programming plans and misses the diagnosis at the task analysis stage. A model-tracing tutor [8, 9] follows the student’s intention by forcing the student to act as an expert would do. Hence, a model-tracing tutor always pretends to know the student’s intention. However, model tracing does not guarantee that student errors can always be corrected. When a student performs an act, which is neither on a correct path nor on a anticipated incorrect one, model tracing has nothing to say other than that is probably incorrect [10]. ELM-PE provides a syntax-based structure editor, which guides the student filling in appropriate insertions into predefined LISP statement slots, such that only valid LISP expressions may be constructed [11]. The diagnostic approaches mentioned restrict students’ creativity and do not support them to improve the problem solving skill. Some other approaches introduce different abstraction levels of errors made by students. The approach in [12] represents student’s actions and errors in terms of knowledge applied in a learning context. Two levels of knowledge are differentiated. The micro-level contains elements describing problems, operators, and control structures and the macro-level describes conceptions. The micro-level represents the way a conception may be revealed by a student, whereas the macro-level represents conceptions in terms of knowledge. The diagnosis approach is driven by taking into account student’s actions related to a particular task and the system provides explanations on the student’s reasoning

68

N.-T. Le and W. Menzel / Problem Solving Process Oriented Diagnosis in Logic Programming

by recognizing sub-jacent knowledge. According to [12], an environment that intends to provide personalized feedback must be able to interpret student’s actions in terms of knowledge. The approach in [13] distinguishes the surface level student model from a deeper level student model. The former one represents the scheduled problem solving plans and applied procedural knowledge. The authors of [13] argue that just diagnosing problem solving knowledge applied by the student is not sufficient, because the sequence of the procedures the student has used may reflect his or her belief in the domain axioms. Therefore, it is necessary to build a deeper level student model which consists of diagnostic hypotheses explaining the procedural operations of a student in terms of the domain axioms. Both approaches [12, 13] introduce different levels of knowledge which can be inferred from the student’s input. However, they do not provide diagnostic information along the process of problem solving. We propose a diagnostic approach which not only enables students to input a solution for a given task in free form. It also supports the students at all three stages of the problem solving process1: task analysis, solution design and implementation. 4. Three Steps Diagnosis in Logic Programming 4.1 Diagnosis at the task analysis stage To create a logic program, first, it is necessary to know how many arguments are required to solve the given task. Normally the number of required arguments can be inferred from the task specification. The student should be able to understand the functionality of every argument which is used to define a predicate. If an argument does not have any function, it is considered to be superfluous. If information from the given task has not been modeled as an argument in the predicate definition, then the student has missed a necessary argument to solve the given task. The second step is to determine the argument modes. In logic programming, an argument can have input mode, output mode or both. Students are requested to specify a mode for each argument of the predicate to be defined according to the given task. The last step of task analysis is to define appropriate data structures for the argument positions. A data structure for an argument in logic programming can be an atom, a number, a list or a special kind of term (i.e. Peano number). We request students to input the information for predicate declaration, i.e. argument list, argument mode and data structure, before they submit a complete solution. The system examines their inputs of the task analysis as the following example shows: Task description: please define a predicate which specifies the relationship between a list and its prefix. You can use the built-in predicate append if necessary. System: please, input the predicate name and the list of arguments with the appropriate modes. Student: prefix(?List1, -List2) System: Error location: 2nd argument; Explanation: wrong mode; Suggestion: check the task specification and choose the appropriate mode. Student: prefix(?List1, ?List2) System: The declaration for the predicate is correct.

1

Our term “problem solving process” differs from the definition in [13] where a sequence of operations applied by the student in solving a given problem is called a problem solving process.

N.-T. Le and W. Menzel / Problem Solving Process Oriented Diagnosis in Logic Programming

69

4.2 Diagnosis at the solution design stage Now, the student has analyzed the task correctly. The system allows the student to put in a solution. A template prefix(List1, List2) according to the predicate declaration is given to the student. After the student has submitted her/his solution, the system hypothesizes the design strategy of the student [4] and returns appropriate feedback. System: please, input your solution for the task above. Student: prefix([X],[X]).

prefix(L,[X|Rest]):-append(H,Rest,[X|Rest]),prefix(H,Rest).

System: we hypothesize that you apply the built-in predicate append to concatenate the first list with an arbitrary second list to get the third list. Please, consider the following feedback: Error1: Location: clause prefix([X],[X]) Explanation: this clause is superfluous. It is not necessary for this design strategy. Remedial hint: please remove it. Error2: Location: prefix(H,Rest) Explanation: this subgoal is superfluous. It is not required for this design strategy. Suggestion: please remove it. Student: prefix(L,[X|Rest]):-append(H,Rest,[X|Rest]).

4.3 Diagnosis at the solution implementation stage Once, the student’s solution conforms with the design strategy the system hypothesized, the system moves the diagnosis to the implementation level. This step deals with the questions: How is a necessary subgoal implemented? How are the arguments in the student’s solution implemented? Our system replies to the last student’s input in the example above as follows: System: please, consider the following feedback Error 1: Location: L, H Explanation: We need to process L. Therefore L and H have to be co-referenced. Remedial hint: write a subgoal L=H, or replace H by L, or replace L by H. Error 2:Location: [X|Rest] in the head of clause 1 Explanation: This argument position needs a variable which represents a list. Suggestion: choose a variable name. Error 3:Location: [X|Rest] in subgoal 1, clause 1 Explanation: This argument needs to represent the concatenation of two lists. Suggestion: choose a variable name instead using a list representation

Figure 1: The three steps of the diagnosis process

70

N.-T. Le and W. Menzel / Problem Solving Process Oriented Diagnosis in Logic Programming

A subgoal can be: unification, de/composition, calculation, an arithmetic test, binding, relation, recursion or user defined. An argument can be a variable, anonymous variable, list, atom (letter), number (float, int), Peano number, arithmetic expression or arbitrary term. For each subgoal, there are different implementation techniques. For example, to implement an arithmetic test subgoal “less then”, we have three possibilities: Y=Y or Y>X. The whole diagnosis process is illustrated by Figure 1. 5. Conclusion and Future Work We have presented the evaluation of our current tutoring system for logic programming. The diagnosis component of this system is developed applying the constraint-based modeling approach. The evaluation result indicated that students have not only difficulties on the on the implementation level, but also on the task analysis level (e.g. data structure for Peano numbers) and the solution design level (e.g. de/composition of lists). The evaluation result has shown that it is necessary to devise a diagnosis approach which is able to deliver diagnostic information corresponding to the stage of the problem solving process where the student is stuck. For this purpose, we have proposed a three step diagnosis approach: 1) diagnosis at the problem analysis stage, 2) diagnosis at the solution design stage and 3) diagnosis at the implementation stage. The three step diagnosis approach can serve several educational purposes. First, it helps students master analysis skills. Second, it supports students in solving programming problems by using design strategies and lastly, students become familiar with the semantics of logic programming. The diagnostic component of our system is under restructuring. We plan to launch and to evaluate a second version of our tutoring system during the winter term 2006/2007. References [1] Ohlsson, S. (1994). Constraint-based Student Modeling. In J. E. Greer, G.I. McCalla, Student Modelling: The Key to Indivi-dualized Knowledge-based Instruction, 167-189. Berlin. [2] Suraweera, P., Mitrovic, A., and Martin, B. (2005) A Knowledge Acquisition System for Constraintbased Intelligent Tutoring Systems. http://www.cosc.canterbury.ac.nz/tanja.mitrovic/SuraweeraAIED05.pdf [3] Le, N.T., and Menzel, W. (2005) Constraint-based Error Diagnosis in Logic Programming. In Proceedings of the 13th International Conference on Computers in Education. [4] Le, N.T. (2006) Using Prolog Design Patterns to Support Constraint-Based Error Diagnosis in Logic Programming. ITS workshop on Intelligent Tutoring Systems for Ill-Defined Domains. [5] Kodaganallur, V., Weitz, R.R., and Rosenthal, D. (2005) A Comparison of Model-Tracing and Constraintbased Intelligent Tutoring Paradigms. In International Journal of Artificial Intelligence in Education, Vol. 15, 117-144. [6] Menzel, W. (2006) Constraint-based Modeling and Ambiguity. In International Journal of Artificial Intelligence in Education, Vol. 16, Nr. 1. [7] Johnson, W. (1986) Intention-based Diagnosis of Novice Programming Errors. Morgan Kaufmann. [8] Anderson, J.R., and Reiser, B. (1985) The LISP Tutor. Byte, 10, 159-175. [9] Anderson, J.R., Conrad, F.G., and Corbett, A.T. (1989) Skill Acquisition and the LISP Tutor. Cognitive Science 13, 467-505. [10] Martin, B. (2001) Intelligent Tutoring Systems: The Practical Implementation of Constraint-based Modelling. PhD thesis, University of Canterbury. [11] Weber, G., and Möllenberg, A. (1995) ELM Programming Environment: a Tutoring System for LISP Beginners. In Cognition and Computer programming, 373-408. [12] Webber, C. (2004) From Errors to Conceptions – an Approach to Student Diagnosis. In Proceedings of the 7th International Conference on Intelligent Tutoring Systems. [13] Matsuda, N. and Okamoto, T. (1992) Student Model Diagnosis for Adaptive Instruction in ITS. In Proceedings of the 2nd International Conference on Intelligent Tutoring Systems., 467-474.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

71

Programming Teaching Support System Using Student Model KeunWoo Han, EunKyoung Lee, YoungJun Lee, Korea National University of Education, Korea [email protected] Abstract: This paper describes the CTi system that supports teachers in C programming courses. The CTi system provides teachers with student’s model, various learning information and inductive instruction tools. Teachers using the system can provide effective feedback to students. Bayesian Belief Networks are used to represent a student’s model and relationship between the structure of programming language and the student’s knowledge. An empirical study showed the CTi system had positive effects on students’ programming performance. Keywords: Programming education, student modeling, teaching support tool, inductive instructional method

1. Introduction Computer programming is a difficult skill for many students. New methods and techniques to help novices to learn programming are needed. Programming is difficult because of the abstract concepts. Students have problems in different issues related to program construction [6]. Therefore, it is necessary to support teaching and learning with various learning strategies such as project based learning [5], self-regulated learning [12], pair programming [2]. There were, however, few systems that support teachers to teach with various learning strategies. We developed a C programming teaching support system. It helps teachers to understand their students’ programming ability and support their teaching activities. We have used Bayesian Belief Networks to represent a student’s model and proposed a teaching support system using the inductive instructional method. Many teachers taught their class using the deductive instructional method. However, a programming language is similar to a secondary foreign language. It is better to provide students with various examples than to provide few examples. With various examples, students are not only able to experience with more the programming codes but also learn some new concepts. A good practice in computer programming teaching is to teach general concepts first and then progressively teaches more detailed grammars [9]. An inductive teaching is better for long-term retention and transference of concepts. The developed CTi system supports the inductive instructional strategy and provides various tools to support teachers. 2. The CTi System The CTi system is a C programming Teaching support system using the inductive instructional method. Students or teachers can connect to the system via a web browser. They can access many practice examples, tools and the student model on the web system. The system records all learning activities and is able to infer students’ knowledge about the

72

K. Han et al. / Programming Teaching Support System Using Student Model

concepts of the programming language. 2.1 Teaching Support tools We need to consider the inductive learning strategy in programming teaching environment [7]. It is necessary to provide students with good example codes and references such as manual. It is, also, necessary to provide teachers with theirs students’ models. It helps them to give effective feedback to their students. In the following, we illustrate the tools that are used to support teaching in the inductive instruction environment. x Annotation: Annotation can provide learning information such as learning covered and assessment. The annotations are useful for effective navigation support [11]. A screenshot of the CTi system interface is shown Figure 1. Each subjects’ icons are presented the covered material in the class and the result of assessment. x Good examples: Good example codes are needed to learn a programming language [9]. Especially in the inductive learning environment, it is important to spend time in teaching students about how to program and code. The CTi system provides students with good examples to learn programming concepts (see in Figure 1). x Visualization: A teacher can present various codes in a programming course and can control the font size of an example code on the system. The CTi system offers the highlight function that enables teachers to emphasize some codes using colored characters. The various teaching support tools are shown in Figure 2. To show example codes and change the font size of the code, teachers can use Tool 1. x Dictionary: A dictionary tool can help students to organize their knowledge about the programming language. Students’ knowledge organization directly affects their programming learning [13]. Teachers or Students are able to search a grammar, keywords and other codes (see Tool 2 in Figure 2). The result of the search is shown in Figure 2 (see Window 1 in Figure 2). Tool 1 Tool 2 Annotation Window 1

Figure 1. The CTi interface

Figure 2. The support tools in the CTi

2.2 Bayesian Student Modeling In a programming course, students learn programming by a topic. Each topic requires some prerequisite concepts. This situation can be represented as a cause-effect relation [10]. This relationship shows prerequisite concepts and learning order. Students can gradually learn contents of each topic in the inductive instructional environment. We developed a dynamic Bayesian Network student model based on a Corbett and Anderson’ probability model [4] and Reye’s paper [10]. Each topic is represented as a node and each node has a probability value of a student’s knowledge about the topic.

K. Han et al. / Programming Teaching Support System Using Student Model

73

We represented the relationship between a structure of programming language and students’ knowledge. Since the probability of a node affects the probability of other nodes, we used Dynamic Belief Networks. We implemented Bayesian Student Models for C programming course (BSMC), in which a student model is represented a Direct Acyclic Graph (DAG). As shown in Figure 3 and Figure 4, students can access their learning activities and models on the knowledge about the programming language. An internal reflective thinking is required in a programming course [1]. The student models provide students with opportunity to have internal reflective thinking.

Figure 3. BSMC DAG ScreenG

Figure 4. The Learning Information

3. The Study 3.1 Methodology To evaluate the effectiveness of the CTi system, we had done a study with 39 university students. The study was done in an introductory C programming course. The students are taught about condition statements using the inductive method. The study participants were divided into an experimental and a control group. In the experimental group, 19 students were taught a programming learning with the CTi system. In the control group, 20 students were taught without the CTi system. Before the study, students were given a pre-test. After two hour class, they took a post-test. Students in experimental group were asked to complete a short questionnaire about the CTi system. During the study, all students’ interaction with the system was logged.

3.2 Results An independent sample t-test was conducted to compare the pre-test scores of the two groups. The difference between the two scores was not statistically significant (t=-1.011, p=.319). This indicates that there were no significant differences between the two groups. To compare the learning gain scores of the two groups, an independent sample t-test was conducted. The t-test result revealed significant differences between the learning gain scores of an experimental group and that of a control group. As shown in Table 1, the experimental group had a statistically higher learning gain scores than the control group (p<.05). Students who used the CTi system perform better in programming than students who didn’t use the system.

74

K. Han et al. / Programming Teaching Support System Using Student Model

{GXUGzGGGGOTGTGTPG

Experimental group Control group

Mean .9842 .6700

Std. Dev. .55001 .37431

t

Sig.

2.095

.043

We examined the correlation between the learning gains and the BSMC visit count that records how many times students has viewed their knowledge about the programming language. We found a marginally significant correlation (Pearson r=.444, p=.057). This means that students who viewed BSMC achieve more learning gain. That is, BSMC can help students to learn programming language. The post-study questionnaire showed that students thought the system was useful in the programming course. 4. Conclusion and Future Work We implemented the CTi system that supports teachers in programming courses. The system represents the student model related to a programming language. The learning strategy of our system is the inductive instructional method. It is to provide students with many good examples, assessments and supporting tools. We found that the system helps students to improve their programming performance. In the future, we will study on student modeling using various learning activities and other learning strategies.

References [1] Angela Carbone, John Hurst, Ian Mitchell and Dick Gunstone (2001) Characteristics of Programming Exercises that lead to Poor Learning Tendencies:Part II. Innovation and Technology in Computer Science Education. [2] Charlie McDowell and Linda Werner (2003) The Impact of Pair Programming on Student Performance, Perception and Persistence. Innovation and Technology in Computer Science Education. [3] Conati C., Gertner A. and VanLehn K. (2002) Using Bayesian Networks to Manage Uncertainty in Student Modeling. Journal of User Modeling and User-Adapted Interaction, 12, 4, 371-417. [4] Corbett, A.T., and Anderson, J.R. (1992) Student modeling and mastery learning in a computer-based programming tutor. Intelligent Tutoring Systems, 413-420. [5] David Davenport(2000) Experience Using a Project-Based Approach in an Introductory Programming Course, IEEE TRANSACTIONS ON EDUCATION, 43, 4, 443-448. [6] Essi Lahtinen, Kirsti Ala-Mutka, and Hannu-Matti Jarvinen (2005) A Study of the Difficulties of Novice Programmers. Innovation and Technology in Computer Science Education. [7] Michael J. Prince and Richard M. Felder(2006) Inductive Teaching and Learning Methods: Definitions, Comparisons, and Research Bases. Journal of Engineering Education, 1-16. [8] Nicolas Guibert, Partrick Girard (2003) Teaching and Learning Programming with a Programming by Example System. International Symposium on End User Development. [9] Nunan, David (1991) Language Teaching Methodology. Prentice Hall. [10] Reye, J. (1996) A belief net backbone for student modelling. Intelligent Tutoring Systems, Proceeding of the Third International Conference, Intelligent Tutoring Systems, 596-604. [11] Rosta Farzan and Peter Brusilovsky (2005) Social Navigation Support through Annotation-Based Group Modeling. User Modeling, 463-472. [12] Susan Bergin, Ronan Reilly, and Desmond Traynor (2005) Examining the Role of Self-Regulated Learning on Introductory Programming Performance. International Computing Education Research. [13] Susan Wiedenbeck (2005) Factors Affecting the Success of Non-Majors in Learning to Program. International Computing Education Research.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

75

A Method for Creating Teaching Materials of Practical Object-Oriented Methods Education Izuru Kumea, Naoya Nittab, and Yasuhiro Takemura c Graduate School of Information Science, Nara Institute of Science and Technology, Japan b Faculty of Science and Engineering, Konan University, Japan c Faculty of Arts, Osaka University of Arts, Japan [email protected]

a

Abstract: We propose a teaching method that yields similar results to actual development projects instead of coordinating actual projects which require much labor and expense. Moreover, we propose a method of efficiently editing teaching materials from a real scale open source software product. According to this method, students can have experiences to make decisions with foresight in a realistic development process, and teachers can create teaching materials efficiently and make repeated use of them. Our method is suited to students who have acquired an understanding of object modeling and object-oriented programming but who have no experience of software development on a realistic scale. Keywords: Real Scale Teaching Object-Oriented Software Design

Materials,

Software

Engineering

Education,

Introduction Professional software developers are required to make decisions with foresight on software design, that is, a skill to foresee future changes and to apply the object-oriented principles and techniques they learned to their design. Experiences of developing a real scale software system are effective to learn such a skill. However, a real project must succeed, which will make it unacceptable to adopt an educational attitude towards tolerating mistakes. In addition, to introduce a real scale software project in higher education may require too much cost and labor compared to its educational benefits. In this paper we propose a practical training method which uses a real scale teaching material so that the training process can simulate various practical software changes in a real scale software development project. Our practical training enables inexperienced students to witness the serious results of their poor design, which are not tolerated in a real project, to makes students realize the importance and the benefits of decisions with foresight in a real scale software development project, and cultivate their ability to make such decisions. We also propose an approach to help teachers edit a teaching material. Our idea is to use a tagged open source software product [6] so that teachers can easily find real examples of decision with foresight and extract them to use as exercises. The method is aimed at students who have acquired the basic concepts of object modeling and techniques of object-oriented programming but who do not have experience developing real software.

1. Scheme of Practical Training A training process of our practical training consists oftraining cycles each of which

76

I. Kume et al. / A Method for Creating Teaching Materials

simulates a development phase of a real project. At the end of a training cycle teachers bring about software changes. Students can witness the robustness or vulnerability of their design and implementation against such changes, and can evaluate their decision. We aim that students can acquire the ability to make decisions with foresight and to apply proper object-oriented techniques through training cycles. A cycle consists of three steps, (1) briefing, (2) trial, and (3) evaluation. In the following we show an example cycle which simulates a development phase of an interactive diagram editor. The example cycle starts with a briefing about "the current progress" of the project to develop the diagram editing software. The system has several editing modes each of which translates user events. It can remember users' selection and can redraw presented diagrams according to users' operation. The design and implementation are assumed to be finished except for an editing mode. Figure 1 shows primary classes whose developments have already finished. A class LayerManager contains the information of all figures presented in diagrams. A class ModeSelect is designed to implement an editing mode to process mouse operation to select one or more figure.If the current editing mode is the selection mode, then a class Editor accepts a mouse event and the event is passed to ModeSelect. ModeSelect defines event handling methods which perform examination of a figure under the mouse, selecting of the figures specified by the mouse, and redrawing of the diagram to present selected figures. In this example cycle, the design and implementation of ModeSelect are assumed incomplete. More specifically, the redrawing process of ModeSelect is hidden for the students. Students are suggested that the drawing specification and design will possibly be changed in future. At the trial step, the students are given a task to complete the design and implementation of ModeSelect. The redrawing process of ModeSelect requires the functions provided by LayerManager. Careful students can notice a possibility that the design around LayerManger will change in future, which is suggested at the briefing step. The trial step finishes when the students complete the design and implementation of ModeSelect, and the class passed tests. At the start of the evaluation step, the teacher of the practical training performs several patterns of changes. A change impact around LayerManager reaches ModeSelect if it has a careless design. The teacher reveals the hidden parts of the implementation of ModeSelect as an example of robust design, and explains its designer's decision with foresight, the object-oriented technique and the way of thinking behind of the technique, the law of Demeter [2], for example. The students evaluate not only their design but also their design decision at the trial step. The witness of the change influence and the evaluation motivate the students to learn object-oriented design techniques and way of thinking.

Figure 1. Architecture of Event Handling Framework

I. Kume et al. / A Method for Creating Teaching Materials

77

2. Expected Educational Effects Software changes at evaluation steps in our practical training show students their design vulnerability. Students can experience design mistakes which are important from the viewpoint of software engineering education but are not permissible in a real software development project. Experiences make students consider change impacts from software components developed by others and understand the importance of communication in a development team. Students try to find object-oriented design techniques applicable to realistic problems instead of being given such techniques with examples. It helps students’ understanding from a practical viewpoint. Introducing reality into training cycles is important from the educational viewpoint. Real scale software project has frequent changes. Coping with changes and influence of design mistakes are harder in larger software projects. To experience realistic problem and difficulty requires not only designs and programs by students but also a large amount of artifacts developed by "other members in the development team". It is desirable to use real scale teaching materials that include a large amount of program code and artifacts which assumed to be developed by "other members".

3. Patterns for Materials Editing This section proposes an efficient method for editing teaching materials from an existing real scale software product, which we call a raw material. Teachers must construct the practical training scenario in such a way that a training cycle induces issues on a later change, as described in the section entitled “Scheme of Practical Training”. In order to form these inductions it is therefore necessary to extract related artifacts from the raw materials. It is further necessary to express the decisions taken during the creation of these artifacts in object-oriented terminology so that the teacher can explain them in the evaluation step. In addition, the teachers must identify the parts within raw materials which reflect the subject they wish to teach. For example, in editing the teaching materials presented in the section entitled “Scheme of Practical Training” the areas where implementation was simplified by the introduction of Demeter’s law (i.e., ModeSelect) had to be specified within the interactive diagram editor. It is often the case that these types of information are not explicitly expressed in the given software. Therefore, in order to make teaching materials suited to the objectives of practical training, teachers are required to explore large artifacts and huge amounts of source code. To reduce this burden on the teachers we propose a method of placing tags at various points in the raw materials, explicitly expressing information related to the implicit influence of decisions within the raw materials. A tag placed in a certain artifact X can represent information such as the object-oriented principles and methods used to construct X, and other artifacts influenced by the construction of X. By the tags, teachers can easily reverse engineer artifacts to identify instances of teaching subjects, and edit the practical training scenario following the relationships between artifacts. In this way the construction of teaching materials can be considered straightforward. The pattern of content specified in a tag consists of six pieces of information: (1) the name of a principle or technique (teaching subject), (2) the general explanation of the subject, (3) the general application of the subject, (4) a specific instance of the subject in the artifacts, (5) the reason for using the subject in the instance, (6) a specific area of another artifact that benefits from the instance of the subject. For example, to construct a training cycle presented in the section entitled “Scheme of Practical Training”, the following tag is helpful: (1) the law of Demeter, (2) "don't talk to strangers", (3) reducing the number of

78

I. Kume et al. / A Method for Creating Teaching Materials

dependency relationships among classes, (4) Editor, (5) because Editor hides LayerManager, (6) ModeSelect. Given this information, the teachers can easily discover that a task tocomplete ModeSelect would be beneficial to learn Demeter’s law and that it would be necessary to explain Editor, LayerManager, etc., in the briefing step.

4. Related Work Many researchers have observed the importance of engendering a sense of the nature of real scale software in software engineering education.(e.g. [3]) We hope to go further. A practical training should be more than an experience of successful software development. Teachers should design a training plan and prepare teaching materials in order to accomplish their educational purposes.Using a blanked source code of a real system as a teaching material, as seen in our practical training, is not new. For example, Tanennbaum and Woodhul [5] used MINIX, a real operating system as a teaching material.Our practical training differs at the point that it requires students not only to fill blanks but also to be aware of impacts of future changes. As a result it cultivates a skill to make a decision with foresight and motivate students to learn object-oriented design techniques listed in design patterns [1]. Our idea to put tags on a raw material comes from the concept of traceability that links designs to decisions behind them [4], although the research in [4] deals with a broader range of software development concepts such as stakeholders, assumptions behind a requirement specification, and alternatives discussed but not selected.

5. Conclusion A practical training method which makes effective use of real scale software was proposed. Students can witness the impact of software changes in a real scale software system, and acquire the ability to decision with foresight. Teachers can easily edit a teaching material which enables students to experience software changes and their difficulty. Hereafter, it is desired to incorporate the method into an actual curriculum in higher education and concretely investigate its benefits and problems.

Acknowledgments This study was partly supported by Grants-in-Aid for Scientific Research of Scientific Research (C) (No.17500662), from Japan Society for the Promotion of Science.

References [1] Gamma, E., et al. (1994) Design Pattern: Elements of reusable object-oriented software, Addison-Wesley. [2] Lieberherr, K., etal. (1988) Object-oriented programming: An objective sense of style, Proc. ACM OOPSLA’88, 323-334. [3] Lincke, S. J. (2005) Work in progress - Motivating students for software engineering, ASEE/IEEE Frontiers in Education Conference. [4] Ramesh, B., and Jarke, M. (2001) Toward reference models for requirements traceability, IEEE Trans. on Software Engineering, Vol. 27, No. 1, 58-93. [5] Tanenbaum, A. S., et al. (2006) Operating Systems Design and Implementation, 3/E, Prentice Hall. [6] Tigris.org, (2006) ArgoUML, http://argouml.tigris.org/ and GEF, http://gef.tigris.org/ .

Science Education at School

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

81

Science Net: Effects of an e-Learning System on Elementary School Students’ Self-Regulated Learning in Science Classes Takeshi KITAZAWA a, Masahiro NAGAI b, Hiroshi KATO c, Kanji AKAHORI d a Tokyo Institute of Technology, Tokyo Jogakkan Primary School, Japan b Tokyo Metropolitan University, Japan c National Institute of Multimedia Education, The Graduate University for Advanced Studies, Japan d Tokyo Institute of Technology, Japan [email protected] Abstract: This paper describes the evaluation of our science e-learning site ‘Science Net’ which elementary school students have used it for one year. First, the findings show that it is not only being used it for checking lesson content but also for asking questions, lesson revision, and searching for science information via a bulletin board system (BBS). Second, the users who have used it for checking lessons tend to score higher in the self-regulated learning strategies than the non users from the results of the Motivated Strategies for Learning Questionnaire (Pintrich࡮De Groot 1990), in terms of 2 of the cognitive strategies (When I study for a test, I try to put together the information from class and from the book, and so on), and 4 of the self-regulations (I work on practice exercises and answer end of chapter questions even when I don’t have to; when I’m reading I stop once in a while and go over what I have read, and so on.). Keywords: Elementary School, Science Education, e-Learning, Web, Self-Regulated Learning

Introduction Trends in International Mathematics and Science Study 2003 (TIMSS2003) of International Association for the Evaluation of Educational Achievement (IEA) indicated science ability of Japanese elementary school students has been declining since 1995. Additionally, the students tend to hardly ever study at home (PISA 2000). Therefore, Japanese educators need to improve not only students’ science ability but also their motivation and interest of science. E-learning is one way which students tend to study at home because it can be used whenever and wherever via the Internet. Therefore, we have developed and evaluated science e-learning site ‘Science Net’ for elementary school students (Kitazawa et al., 2005; Kitazawa et al., 2006). The benefit for students was that they could check lessons when they were absent, downloading work sheets and ask the teacher questions with BBS. What seems to be lacking, however is not clear, because of the uncertainty of how students have used the site over long periods. And then, the usability of ‘Science Net’ is not clear. The first purpose of this paper is to determine how the students used ‘Science Net’ over the coarse of a year, and also to analyze what contents of the site are not used easily by the users. Kougo & Nojima (2004a) suggest when students learn via e-learning, they need to

82

T. Kitazawa et al. / Science Net: Effects of an e-Learning System

apply self-regulated learning (SRL). Self-regulated learning is defined as a learner's intentional efforts to manage and direct complex learning activities (Zimmerman, 1998). The effects of self-regulated learning is more efficient for students’ grades than the elaborated epistemic learning model (Pintrich࡮De Groot, 1990; Zimmerman࡮MartinezPons, 1990). Accordingly, the second purpose will be to investigate students’ self-regulated learning strategies with the Motivated Strategies for Learning Questionnaire (Pintrich࡮De Groot, 1990).

1. Overview of ‘Science Net’ 1.1 Outline of ‘Science Net’ The e-learning site ‘Science Net’ is for elementary school students and is Web based because students can access anywhere (Kitazawa et al., 2005). The contents of the site are 1) checking lessons, 2) asking science questions via BBS (using free software ‘Kent Web’ (http://www.kent- web.com/) (see figure1)), and much more. The site was re-launched at the beginning of the 1st semester (Apr. 2005), and used by elementary school students from 3rd to 6th grade for one year (Mar. 2006). At first, Students learnt how to use ‘Science Net’ in the PC room. After that, they were given their ID. Then, they could freely access it at school or at home. If they didn’t want to use ‘Science Net’, they weren’t forced to use it.

Writing Area Submit Button Student’s Question Teacher’s Answer

Figure 1 BBS

Schedule Icon

Unit Icon of Each Grade

BBS Reference

Figure 2 Schedules of Lessons (3rd Grade)

Contents Icon

Classes

83

T. Kitazawa et al. / Science Net: Effects of an e-Learning System

Classes Documents Unite

Keywords

Digital Materials

Homework

Total of Lessons

Figure 3 Contents of Lessons (3rd Grade)

Table 1 Number of Science Net Users and Non Users Number of Students Grade 3rd 4th 5th 6th Total

Assessment of ‘Science Net’ (on Mar. 2006) 16 19 15 13 63

Investigation of Self-Regulated Learning Strategies (on Apr. 2006) 77㧔included 16 users㧕 80㧔included 19 users㧕 78㧔included 15 users㧕 235㧔included 50 users㧕

1.2 Improvement of ‘Science Net’ One of the points of improvement is adding 3rd grade’s contents so they can check their lessons (figures 2 & 3). Another point of improvement is to check previous lessons which they did 1 year ago because the previous lessons are saved as a reference for the users. In addition, the questions which were asked by prior students are also saving in the database of BBS as a reference.

2. Method 2.1 Assessment of ‘Science Net’ First, this paper aims to analyze how the students use ‘Science Net’ for a year. 63 students who have used ‘Science Net’ for a year did a questionnaire about the site at the end of the science lesson (on Mar. 2006). The questionnaire about ‘Science Net’ has 16 inquiries items, it measures from 1.strongly disagree to 5.strongly agree. 2.2. Investigation of Self-Regulated Learning Strategies Second, this paper aims to investigate the students who use ‘Science Net’ and have self-regulated learning strategies. At the beginning of the science lesson (on Apr. 2006) 50 users and 185 non users did the self-regulated learning strategies section of Motivated Strategies for Learning Questionnaire (Pintrich ࡮ De Groot 1990), it measures from 1.strongly disagree to 7.strongly agree (see table 1).

84

T. Kitazawa et al. / Science Net: Effects of an e-Learning System Table 2 Result of Questionnaire (Assessment of ‘Science Net’, 5 choices) 㪠㫋㪼㫄㫊㩷

㪘㫍㪼㫉㪸㪾㪼㩷

㪪㫋㪸㫅㪻㪸㫉㪻㩷㪛㪼㫍㫀㪸㫋㫀㫆㫅㩷

㪨䋱㩷㪠㩷㪺㪿㪼㪺㫂㪼㪻㩷㪸㩷㫃㪼㫊㫊㫆㫅㩷㫎㫀㫋㪿㩷㵬㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㵭㩷㫎㪿㪼㫅㩷㪠㩷㫎㪸㫊㩷㪸㪹㫊㪼㫅㫋㩷㪽㫉㫆㫄㩷㪸㩷㫃㪼㫊㫊㫆㫅㪅㩷

㪊㪅㪈㪊㩷㩷

㪈㪅㪉㪎㩷㩷

㪨䋲㩷㪠㩷㫌㫊㪼㪻㩷㩾㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㩾㩷㪽㫆㫉㩷㫇㫉㪼㫇㪸㫉㪸㫋㫀㫆㫅㩷

㪊㪅㪊㪌㩷㩷

㪈㪅㪈㪎㩷㩷

㪨䋳㩷㪠㩷㫌㫊㪼㪻㩷㵬㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㵭㪽㫆㫉㩷㫉㪼㫍㫀㪼㫎㪅㩷

㪊㪅㪋㪇㩷㩷

㪈㪅㪈㪏㩷㩷

㪨䋴㩷㪠㩷㫌㫊㪼㪻㩷㵬㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㵭㩷㪽㫆㫉㩷㫊㫋㫌㪻㫐㫀㫅㪾㩷㪸㫅㫆㫋㪿㪼㫉㩷㪾㫉㪸㪻㪼㫊㵭㩷㪺㫆㫅㫋㪼㫅㫋㫊㪅㩷

㪉㪅㪍㪏㩷㩷

㪈㪅㪉㪉㩷㩷

㪨䋵㩷㪠㩷㫃㫀㫂㪼㩷㫊㪺㫀㪼㫅㪺㪼㪅㩷

㪋㪅㪉㪊㩷㩷

㪈㪅㪇㪈㩷㩷

㪨䋶㩷㪠㩷㪻㫆㫅㩾㫋㩷㫋㪿㫀㫅㫂㩷㪠㩷㫎㫀㫃㫃㩷㪹㪼㩷㫀㫅㫋㪼㫉㪼㫊㫋㪼㪻㩷㫀㫅㩷㫊㪺㫀㪼㫅㪺㪼㩷㫋㪿㪸㫅㫂㫊㩷㫋㫆㩷㵭㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㵭㵭㪅㩷

㪈㪅㪏㪍㩷㩷

㪇㪅㪐㪍㩷㩷

㪨䋷㩷㪠㩷㫌㫊㪼㪻㩷㩾㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㩾㩷㫋㪿㪼㩷㫊㪸㫄㪼㩷㪻㪸㫐㩷㪼㫍㪼㫉㫐㩷㫎㪼㪼㫂㪅㩷

㪈㪅㪎㪈㩷㩷

㪈㪅㪇㪌㩷㩷

㪨䋸㩷㪠㩷㪻㫀㪻㫅㵭㫋㩷㫌㫊㪼㩷㩾㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㩾㩷㫋㪿㪼㩷㫊㪸㫄㪼㩷㫋㫀㫄㪼㩷㪼㫍㪼㫉㫐㪅㩷

㪊㪅㪋㪌㩷㩷

㪈㪅㪌㪈㩷㩷

㪨䋹㩷㪠㩷㫌㫊㪼㪻㩷㩾㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㩾㩷㫋㫆㩷㪺㪿㪼㪺㫂㩷㪽㫆㫉㩷㫆㫋㪿㪼㫉㫊㩾㩷㫈㫌㪼㫊㫋㫀㫆㫅㫊㩷㫎㫀㫋㪿㩷㪙㪙㪪㪅㩷

㪊㪅㪌㪉㩷㩷

㪈㪅㪊㪇㩷㩷

㪨㪈㪇㩷㪠㩷㫌㫊㪼㪻㩷㩾㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㩾㩷㫋㫆㩷㪸㫊㫂㩷㫈㫌㪼㫊㫋㫀㫆㫅㫊㩷㫎㫀㫋㪿㩷㪙㪙㪪㪅㩷

㪊㪅㪋㪍㩷㩷

㪈㪅㪌㪇㩷㩷

㪨㪈㪈㩷㪠㫋㩷㫎㪸㫊㩷㪻㫀㪽㪽㫀㪺㫌㫃㫋㩷㪽㫆㫉㩷㫄㪼㩷㫋㫆㩷㫉㪼㫋㫉㫀㪼㫍㪼㩷㫈㫌㪼㫊㫋㫀㫆㫅㫊㩷㫎㪿㫀㪺㪿㩷㪠㩷㫎㪸㫅㫋㪅㩷

㪉㪅㪐㪌㩷㩷

㪈㪅㪊㪊㩷㩷

㪨㪈㪉㩷㪮㪿㪼㫅㩷㪠㩷㪸㫊㫂㪼㪻㩷㪸㩷㫈㫌㪼㫊㫋㫀㫆㫅㪃㩷㫋㪿㪼㩷㫈㫌㪼㫊㫋㫀㫆㫅㩷㫎㪸㫊㩷㫋㪿㪼㩷㫊㪸㫄㪼㩷㪸㫊㩷㫊㫆㫄㪼㫆㫅㪼㩷㪼㫃㫊㪼㵭㫊㩷㫈㫌㪼㫊㫋㫀㫆㫅㪅

㪉㪅㪇㪇㩷㩷

㪈㪅㪊㪊㩷㩷

㪨㪈㪊㩷㪠㩷㪺㪿㪼㪺㫂㪼㪻㩷㫊㫆㫄㪼㩷㫈㫌㪼㫊㫋㫀㫆㫅㫊㩷㪹㪼㪽㫆㫉㪼㩷㪠㩷㪸㫊㫂㪼㪻㩷㪸㩷㫈㫌㪼㫊㫋㫀㫆㫅㪅㩷

㪉㪅㪐㪎㩷㩷

㪈㪅㪋㪊㩷㩷

㪨㪈㪋㩷㪠㫋㩷㫎㪸㫊㩷㪻㫀㪽㪽㫀㪺㫌㫃㫋㩷㪽㫆㫉㩷㫄㪼㩷㫋㫆㩷㫉㪼㫋㫉㫀㪼㫍㪼㩷㪸㩷㫈㫌㪼㫊㫋㫀㫆㫅㩷㫎㪿㫀㪺㪿㩷㪠㩷㪸㫃㫊㫆㩷㫎㪸㫅㫋㪼㪻㩷㫋㫆㩷㪸㫊㫂㪅㩷

㪊㪅㪊㪍㩷㩷

㪈㪅㪊㪋㩷㩷

㪨㪈㪌㩷㪠㩷㫊㫋㫌㪻㫀㪼㪻㩷㫊㪺㫀㪼㫅㪺㪼㩷㫄㫆㫉㪼㩷㫋㪿㪸㫅㫂㫊㩷㫋㫆㩷㵬㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㵭㪅㩷

㪊㪅㪋㪈㩷㩷

㪈㪅㪉㪎㩷㩷

㪨㪈㪍㩷㪠㩷㪾㫆㫋㩷㪸㩷㪿㫀㪾㪿㩷㪾㫉㪸㪻㪼㩷㫇㫆㫀㫅㫋㩷㪽㫆㫉㩷㫊㪺㫀㪼㫅㪺㪼㩷㫋㪿㪸㫅㫂㫊㩷㫋㫆㩷㵬㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㵭㪅㩷

㪊㪅㪈㪇㩷㩷

㪈㪅㪇㪎㩷㩷

2.3. Performance of Science Students tested for a year. The tests calculated an average score per students (national average was 82 points.). The results were compared between the users and the non users.

3. Result 3.1 Assessment of ‘Science Net’ 3.1.1 Result of Average

Table 3 Result of Factor Analysis 㪠㫋㪼㫄㫊㩷

㪝㪸㪺㫋㫆㫉㩷㪈

㪝㪸㪺㫋㫆㫉㩷㪉㩷

㪝㪸㪺㫋㫆㫉㩷㪊㩷

㪺㫆㫄㫄㫆㫅㩷㪾㫉㫆㫌㫅㪻㪅㪌㪎㪊㩷

㪨㪈㩷

㪅㪎㪊㪈㩷

㪅㪇㪐㪋㩷

㪅㪈㪎㪉㩷

㪨㪈㪈㩷

㪄㪅㪍㪋㪉㩷

㪅㪇㪐㪋㩷

㪅㪈㪏㪋㩷

㪅㪋㪌㪌㩷

㪨㪊㩷

㪅㪌㪏㪋㩷

㪅㪈㪈㪈㩷

㪄㪅㪈㪋㪌㩷

㪅㪊㪎㪌㩷

㪨㪉㩷

㪅㪋㪎㪈㩷

㪅㪊㪊㪇㩷

㪄㪅㪇㪇㪏㩷

㪅㪊㪊㪈㩷

㪨㪈㪋㩷

㪄㪅㪋㪎㪇㩷

㪄㪅㪇㪐㪇㩷

㪅㪉㪏㪈㩷

㪅㪊㪇㪏㩷

㪨㪏㩷

㪄㪅㪋㪊㪎㩷

㪄㪅㪈㪈㪌㩷

㪄㪅㪇㪈㪉㩷

㪅㪉㪇㪋㩷

㪨㪎㩷

㪅㪊㪍㪇㩷

㪅㪈㪊㪈㩷

㪅㪇㪈㪌㩷

㪅㪈㪋㪎㩷

㪨㪌㩷

㪅㪇㪌㪏㩷

㪅㪏㪊㪍㩷

㪅㪇㪎㪋㩷

㪅㪎㪇㪏㩷

㪨㪍㩷

㪄㪅㪇㪌㪈㩷

㪄㪅㪎㪇㪎㩷

㪅㪇㪏㪇㩷

㪅㪌㪇㪐㩷

㪨㪈㪌㩷

㪅㪋㪏㪉㩷

㪅㪌㪉㪎㩷

㪄㪅㪇㪉㪌㩷

㪅㪌㪈㪈㩷

㪨㪈㪍㩷

㪅㪉㪎㪐㩷

㪅㪋㪇㪎㩷

㪅㪈㪎㪇㩷

㪅㪉㪎㪉㩷

㪅㪐㪉㪉㩷㪅㪏㪌㪐㩷㪨㪈㪇㩷㪄㪅㪇㪎㪎㩷㪄㪅㪇㪌㪍㩷 Table 2 indicates the results of the 㪨㪐㩷㪄㪅㪇㪋㪎㩷㪅㪈㪈㪏㩷㪅㪎㪍㪋㩷㪅㪍㪇㪇㩷 questionnaire from the assessment of 㪚㫌㫄㫌㫃㪸㫋㫀㫍㪼㩷㪈㪏㪅㪉㪏㩷㪊㪉㪅㪋㪈㩷㪋㪌㪅㪇㪉㩷㩷㩷㩷 ‘Science Net’ (5 choices). First, the 㪚㫆㫅㫋㫉㫀㪹㫌㫋㫀㫅㪾㩷㪩㪸㫋㪼㩷㩿㩼㪀㩷 finding is the users tend to like science according to ‘Q5 I like science (average is 4.23).’ In addition, the site influence interest of science according to ‘Q6 I don't think I will be interested in science thanks to ‘Science Net’ (average is 1.71).’ The users used the system randomly rather than on a regular basis (see results of Q7 and Q8). ‘Q1 I checked a lesson with ‘Science Net’ when I was absent from a lesson.’, ‘Q2 I used 'Science Net' for preparation’, closely indicates the intermediate value (3). Therefore, it seems that they didn’t always use the site for preparation or review. Consequentially, we need to specifically analyze their awareness of these options.

3.1.2 Result of Factor Analysis First, we analyzed 16 items using factor analysis (principal factor analysis, varimax rotation) because we inquired about the users’ awareness of using ‘Science Net’. Second, we got 3 items out of the factor analysis because the factors loaded were less than 0.30. After that we did a factor analysis again. The number of the factor was defined by the result

T. Kitazawa et al. / Science Net: Effects of an e-Learning System

85

of screeplot. Table 3 shows the result of the factor analysis. We got 3 factors (cumulative contributing rate is 45.02%) which defined the first factor as ‘the use of checking lessons and retrieving questions’, the second factor as ‘the interest of science study’, and the third factor as ‘the use of BBS’. The finding that ‘Science Net’ has two uses one as ‘the use of checking lessons and retrieving questions’ and second ‘the use of BBS’. Therefore, results show that some users have used the site to check and retrieve questions, and others have used it for the BBS. It is clear that the interest of science correlates with the use of ‘Science Net’ according to the second factor ‘the interest of science study’. In addition, we need to analyze whether the use of ‘Science Net’ will bring an interest to science for students. ‘Science Net’ has had several improvements because of students’ comments. Results from students’ comments indicate that a retrieval system in the BBS is necessary. If we refer to table 4 this point becomes more evident. 3.2 Investigation of Self-Regulated Learning Strategies From the questionnaire results we are able to see how the students used ‘Science Net’. Result from the factor analysis showed that there are two groups of users. The first group of users is those who failed to check the lesson type. The second group of users is those who checked the BBS. We went on to further analyze individual students. From the results of Q2, Q3, Q9, and Q10, we analyzed the different types of individual users. We found that 16 students used the system only for checking lessons because they chose 4 (agree) or 5 (strongly agree) for Q2 or Q3, and for Q9 and Q10 their answers ranged from 1 (strongly disagree) to 3 (neutral). We defined students in this group as those who used Science Net ‘only for checking lessons’. Secondly, we found 19 students used Science Net for checking lessons as well as checking the BBS. We concluded this from their answers from Q2, Q3, Q9, and Q10. For all students chose 4 or 5 on the questionnaire. We defined these students as those who used Science Net for ‘checking lessons and the BBS’. Thirdly, 11 students used it only for checking the BBS because they chose 4 or 5 for Q9 or Q10, and for Q2 and Q3 their answers ranged from 1 to 3. We defined these students as those who used Science Net for ‘only checking the BBS’. Lastly, 4 students used it for other purposes because for Table 4 Students’ Comments which They Want to Improve Contents of ‘Science Net’ Grade 3rd

4th

Students’ comments I want you to tell me about the interest things of science. I want you to change the BBS so that I can check previous questions easier. I think it’s a good thing that questions are categorized according to grade. I will access everyday if there are some games about science on ‘Science Net’. I would like you to make a section in BBS for previous questions only. I would also like you to categorize the questions. For example, ‘plant’, ‘animal’, ‘celestial body’. I want to be able to check my friends’ questions easier䋮 If everyone doesn’t use ‘Science Net’ at home, then I suggest we decide on the day to use the site in class. When I asked a question, the question was asked already, so I didn’t ask my question. Also I would like to get some worksheets for tests. I want to be able to input my password easily.

5th

I want an independent research site, so I can do experiment at home. When I ask a question, it was the same as someone else’s question. Could you possibly improve that? I would like games on the site. I sometimes forgot my password, so I think everyone should be able to check their passwords easily.

6th

I couldn’t find my question because many questions were saved after my question. It’s terrible! It is difficult for me to retrieve a question that I want from the database.

86

T. Kitazawa et al. / Science Net: Effects of an e-Learning System

㪈

㪉

㪊

㪋

㪌

㪍

㪨㪈㩷㪮㪿㪼㫅㩷㪠㩷㫊㫋㫌㪻㫐㩷㪽㫆㫉㩷㪸㩷㫋㪼㫊㫋㪃㩷㪠㩷㫋㫉㫐㩷㫋㫆㩷㫇㫌㫋㩷㫋㫆㪾㪼㫋㪿㪼㫉㩷㫋㪿㪼㩷㫀㫅㪽㫆㫉㫄㪸㫋㫀㫆㫅㩷㪽㫉㫆㫄㪺㫃㪸㫊㫊㩷㪸㫅㪻㩷㪽㫉㫆㫄㩷㫋㪿㪼㩷㪹㫆㫆㫂䋮

㪎㪌㪅㪍㪍㩿㪈㪅㪌㪌㪀㪋㪅㪐㪊㩿㪈㪅㪐㪈㪀㪌㪅㪇㪉㩿㪈㪅㪌㪈㪀

㪂

㪨㪉㩷㪮㪿㪼㫅㩷㪠㩷㪻㫆㩷㪿㫆㫄㪼㫎㫆㫉㫂㪃㩷㪠㩷㫋㫉㫐㩷㫋㫆㩷㫉㪼㫄㪼㫄㪹㪼㫉㩷㫎㪿㪸㫋㩷㫋㪿㪼㩷㫋㪼㪸㪺㪿㪼㫉㩷㫊㪸㫀㪻㩷㫀㫅㩷㪺㫃㪸㫊㫊㫊㫆㩷㪠㩷㪺㪸㫅㩷㪸㫅㫊㫎㪼㫉㩷㫋㪿㪼㩷㫈㫌㪼㫊㫋㫀㫆㫅㫊㩷㪺㫆㫉㫉㪼㪺㫋㫃㫐㪅

㪌㪅㪏㪇㩿㪇㪅㪐㪊㪀㪌㪅㪉㪇㩿㪈㪅㪋㪉㪀㪌㪅㪈㪏㩿㪈㪅㪈㪎㪀

㪁

㪨㪊㩷㪠㫋㩷㫀㫊㩷㪿㪸㫉㪻㩷㪽㫆㫉㩷㫄㪼㩷㫋㫆㩷㪻㪼㪺㫀㪻㪼㩷㫎㪿㪸㫋㩷㫋㪿㪼㩷㫄㪸㫀㫅㩷㫀㪻㪼㪸㫊㩷㪸㫉㪼㩷㫀㫅㩷㫎㪿㪸㫋㩷㪠㩷㫉㪼㪸㪻㪅

㪉㪅㪏㪊㩿㪈㪅㪌㪉㪀㪊㪅㪈㪊㩿㪈㪅㪌㪈㪀㪊㪅㪈㪏㩿㪈㪅㪍㪉㪀

㪨㪋㩷㪮㪿㪼㫅㩷㪠㩷㫊㫋㫌㪻㫐㩷㪠㩷㫇㫌㫋㩷㫀㫄㫇㫆㫉㫋㪸㫅㫋㩷㫀㪻㪼㪸㫊㩷㫀㫅㫋㫆㩷㫄㫐㩷㫆㫎㫅㩷㫎㫆㫉㪻㫊㪅

㪌㪅㪋㪋㩿㪈㪅㪍㪎㪀㪌㪅㪉㪇㩿㪈㪅㪋㪎㪀㪋㪅㪐㪍㩿㪈㪅㪍㪐㪀

㪨㪌㩷㪠㩷㪸㫃㫎㪸㫐㫊㩷㫋㫉㫐㩷㫋㫆㩷㫌㫅㪻㪼㫉㫊㫋㪸㫅㪻㩷㫎㪿㪸㫋㩷㫋㪿㪼㩷㫋㪼㪸㪺㪿㪼㫉㩷㫀㫊㩷㫊㪸㫐㫀㫅㪾㩷㪼㫍㪼㫅㩷㫀㪽㩷㫀㫋㪻㫆㪼㫊㫅㩾㫋㩷㫄㪸㫂㪼㩷㫊㪼㫅㫊㪼㪅

㪌㪅㪉㪐㩿㪈㪅㪍㪍㪀㪌㪅㪈㪊㩿㪈㪅㪐㪉㪀㪋㪅㪏㪋㩿㪈㪅㪍㪏㪀

㪨㪍㩷㪮㪿㪼㫅㩷㪠㩷㫊㫋㫌㪻㫐㩷㪽㫆㫉㩷㪸㩷㫋㪼㫊㫋㩷㪠㩷㫋㫉㫐㩷㫋㫆㩷㫉㪼㫄㪼㫄㪹㪼㫉㩷㪸㫊㩷㫄㪸㫅㫐㩷㪽㪸㪺㫋㫊㩷㪸㫊㩷㪠㩷㪺㪸㫅㪅

㪌㪅㪍㪉㩿㪈㪅㪌㪏㪀㪋㪅㪏㪇㩿㪉㪅㪉㪈㪀㪌㪅㪇㪋㩿㪈㪅㪌㪈㪀

㪨㪎㩷㪮㪿㪼㫅㩷㫊㫋㫌㪻㫐㫀㫅㪾㪃㩷㪠㩷㪺㫆㫇㫐㩷㫄㫐㩷㫅㫆㫋㪼㫊㩷㫆㫍㪼㫉㩷㫋㫆㩷㪿㪼㫃㫇㩷㫄㪼㩷㫉㪼㫄㪼㫄㪹㪼㫉㩷㫄㪸㫋㪼㫉㫀㪸㫃㪅

㪌㪅㪐㪋㩿㪈㪅㪋㪊㪀㪌㪅㪇㪊㩿㪈㪅㪐㪋㪀㪌㪅㪈㪐㩿㪈㪅㪌㪏㪀

㪨㪏㩷㪮㪿㪼㫅㩷㪠㩷㫊㫋㫌㪻㫐㩷㪽㫆㫉㩷㪸㩷㫋㪼㫊㫋㩷㪠㩷㫇㫉㪸㪺㫋㫀㪺㪼㩷㫊㪸㫐㫀㫅㪾㩷㫋㪿㪼㩷㫀㫄㫇㫆㫉㫋㪸㫅㫋㩷㪽㪸㪺㫋㫊㩷㫆㫍㪼㫉㩷㪸㫅㪻㫆㫍㪼㫉㩷㫋㫆㩷㫄㫐㫊㪼㫃㪽㪅

㪋㪅㪉㪊㩿㪈㪅㪐㪎㪀㪊㪅㪌㪊㩿㪉㪅㪇㪇㪀㪊㪅㪐㪋㩿㪈㪅㪎㪈㪀

㪬㫊㪼㫉㫊㩷㫎㪿㫆㩷㪿㪸㫍㪼㩷㫌㫊㪼㪻㪽㫆㫉㩷㪺㪿㪼㪺㫂㫀㫅㪾㩷㫃㪼㫊㫊㫆㫅㫊㪬㫊㪼㫉㫊㩷㫎㪿㫆㩷㪿㪸㫍㪼㫅㵭㫋㩷㫌㫊㪼㪻㪽㫆㫉㩷㪺㪿㪼㪺㫂㫀㫅㪾㩷㫃㪼㫊㫊㫆㫅㫊㪥㫆㫅㩷㪬㫊㪼㫉㫊

㪨㪐㩷㪠㩷㫌㫊㪼㩷㫎㪿㪸㫋㩷㪠㩷㪿㪸㫍㪼㩷㫃㪼㪸㫉㫅㪼㪻㩷㪽㫉㫆㫄㩷㫆㫃㪻㩷㪿㫆㫄㪼㫎㫆㫉㫂㩷㪸㫊㫊㫀㪾㫅㫄㪼㫅㫋㫊㩷㪸㫅㪻㩷㫋㪿㪼㫋㪼㫏㫋㪹㫆㫆㫂㩷㫋㫆㩷㪻㫆㩷㫅㪼㫎㩷㪸㫊㫊㫀㪾㫅㫄㪼㫅㫋㫊㪅

㪌㪅㪏㪌㩿㪈㪅㪍㪍㪀㪌㪅㪋㪇㩿㪈㪅㪍㪏㪀㪌㪅㪈㪏㩿㪈㪅㪍㪏㪀

㪨㪈㪇㩷㪮㪿㪼㫅㩷㪠㩷㪸㫄㩷㫊㫋㫌㪻㫐㫀㫅㪾㩷㪸㩷㫋㫆㫇㫀㪺㪃㩷㪠㩷㫋㫉㫐㩷㫋㫆㩷㫄㪸㫂㪼㩷㪼㫍㪼㫉㫐㫋㪿㫀㫅㪾㩷㪽㫀㫋㩷㫋㫆㪾㪼㫋㪿㪼㫉㪅

㪋㪅㪋㪐㩿㪈㪅㪏㪇㪀㪋㪅㪋㪎㩿㪈㪅㪌㪌㪀㪋㪅㪈㪉㩿㪈㪅㪋㪏㪀

㪨㪈㪈㩷㪮㪿㪼㫅㩷㪠㩷㫉㪼㪸㪻㩷㫄㪸㫋㪼㫉㫀㪸㫃㩷㪽㫆㫉㩷㫋㪿㫀㫊㩷㪺㫃㪸㫊㫊㪃㩷㪠㩷㫊㪸㫐㩷㫋㪿㪼㩷㫎㫆㫉㪻㫊㩷㫆㫍㪼㫉㩷㪸㫅㪻㩷㫆㫍㪼㫉㩷㫋㫆㫄㫐㫊㪼㫃㪽㩷㫋㫆㩷㪿㪼㫃㫇㩷㫄㪼㩷㫉㪼㫄㪼㫄㪹㪼㫉㪅

㪋㪅㪍㪍㩿㪈㪅㪍㪍㪀㪊㪅㪍㪎㩿㪉㪅㪇㪍㪀㪋㪅㪇㪊㩿㪈㪅㪎㪌㪀

㪨㪈㪉㩷㪠㩷㫆㫌㫋㫃㫀㫅㪼㩷㫋㪿㪼㩷㪺㪿㪸㫇㫋㪼㫉㫊㩷㫀㫅㩷㫄㫐㩷㪹㫆㫆㫂㩷㫋㫆㩷㪿㪼㫃㫇㩷㫄㪼㩷㫊㫋㫌㪻㫐㪅

㪋㪅㪍㪊㩿㪈㪅㪏㪉㪀㪊㪅㪍㪇㩿㪉㪅㪇㪍㪀㪋㪅㪈㪍㩿㪈㪅㪏㪌㪀

㪨㪈㪊㩷㪮㪿㪼㫅㩷㫉㪼㪸㪻㫀㫅㪾㩷㪠㩷㫋㫉㫐㩷㫋㫆㩷㪺㫆㫅㫅㪼㪺㫋㩷㫋㪿㪼㩷㫋㪿㫀㫅㪾㫊㩷㪠㩷㪸㫄㩷㫉㪼㪸㪻㫀㫅㪾㩷㪸㪹㫆㫌㫋㩷㫎㫀㫋㪿㫎㪿㪸㫋㩷㪠㩷㪸㫃㫉㪼㪸㪻㫐㩷㫂㫅㫆㫎㪅

㪋㪅㪏㪏㩿㪈㪅㪏㪌㪀㪋㪅㪌㪊㩿㪈㪅㪌㪌㪀㪋㪅㪋㪍㩿㪈㪅㪋㪍㪀

㪁㫇㪓㪅㪇㪌㪂㫇㪓㪅㪈

Figure 4 Result of Self-Regulated Learning Strategies (Cognitive Strategy Use: Numbers are average, number in parenthesis is standard deviation) 㪈

㪉

㪊

㪋

㪨㪈㪋㩷㪠㩷㪸㫊㫂㩷㫄㫐㫊㪼㫃㪽㩷㫈㫌㪼㫊㫋㫀㫆㫅㫊㩷㫋㫆㩷㫄㪸㫂㪼㩷㫊㫌㫉㪼㩷㪠㩷㫂㫅㫆㫎㩷㫋㪿㪼㩷㫄㪸㫋㪼㫉㫀㪸㫃㩷㪠㪿㪸㫍㪼㩷㪹㪼㪼㫅㩷㫊㫋㫌㪻㫐㫀㫅㪾㪅

㪌

㪍

㪎㪊㪅㪎㪐㩿㪈㪅㪎㪈㪀㪊㪅㪋㪎㩿㪉㪅㪊㪊㪀㪊㪅㪇㪏㩿㪈㪅㪍㪏㪀

㪂

㪨㪈㪌㩷㪮㪿㪼㫅㩷㫎㫆㫉㫂㩷㫀㫊㩷㪿㪸㫉㪻㩷㪠㩷㪼㫀㫋㪿㪼㫉㩷㪾㫀㫍㪼㩷㫌㫇㩷㫆㫉㩷㫊㫋㫌㪻㫐㩷㫆㫅㫃㫐㩷㫋㪿㪼㩷㪼㪸㫊㫐㫇㪸㫉㫋㫊㪅

㪈㪅㪐㪎㩿㪈㪅㪌㪊㪀㪉㪅㪇㪇㩿㪈㪅㪈㪏㪀㪉㪅㪈㪎㩿㪈㪅㪊㪈㪀

㪨㪈㪍㩷㩷㪠㩷㫎㫆㫉㫂㩷㫆㫅㩷㫇㫉㪸㪺㫋㫀㪺㪼㩷㪼㫏㪼㫉㪺㫀㫊㪼㫊㩷㪸㫅㪻㩷㪸㫅㫊㫎㪼㫉㩷㪼㫅㪻㩷㫆㪽㩷㪺㪿㪸㫇㫋㪼㫉㫈㫌㪼㫊㫋㫀㫆㫅㫊㩷㪼㫍㪼㫅㩷㫎㪿㪼㫅㩷㪠㩷㪻㫆㫅㩾㫋㩷㪿㪸㫍㪼㩷㫋㫆㪅

㪁

㪌㪅㪏㪍㩿㪈㪅㪋㪇㪀㪌㪅㪋㪎㩿㪈㪅㪎㪎㪀㪌㪅㪈㪊㩿㪈㪅㪌㪈㪀

㪨㪈㪎㩷㪜㫍㪼㫅㩷㫎㪿㪼㫅㩷㫊㫋㫌㪻㫐㩷㫄㪸㫋㪼㫉㫀㪸㫃㫊㩷㪸㫉㪼㩷㪻㫌㫃㫃㩷㪸㫅㪻㩷㫌㫅㫀㫅㫋㪼㫉㪼㫊㫋㫀㫅㪾㪃㩷㪠㫂㪼㪼㫇㩷㫎㫆㫉㫂㫀㫅㪾㩷㫌㫅㫋㫀㫃㩷㪠㩷㪽㫀㫅㫀㫊㪿㪅

㪁

㪌㪅㪏㪏㩿㪈㪅㪉㪌㪀㪌㪅㪇㪇㩿㪈㪅㪎㪊㪀㪌㪅㪉㪊㩿㪈㪅㪋㪇㪀

㪨㪈㪏㩷㪙㪼㪽㫆㫉㪼㩷㪠㩷㪹㪼㪾㫀㫅㩷㫊㫋㫌㪻㫐㫀㫅㪾㩷㪠㩷㫋㪿㫀㫅㫂㩷㪸㪹㫆㫌㫋㩷㫋㪿㪼㩷㫋㪿㫀㫅㪾㫊㩷㪠㩷㫎㫀㫃㫃㩷㫅㪼㪼㪻㫋㫆㩷㪻㫆㩷㫋㫆㩷㫃㪼㪸㫉㫅㪅

㪋㪅㪉㪊㩿㪉㪅㪇㪋㪀㪋㪅㪋㪎㩿㪈㪅㪏㪌㪀㪊㪅㪍㪎㩿㪈㪅㪌㪌㪀

㪨㪈㪐㩷㩷㪠㩷㫆㪽㫋㪼㫅㩷㪽㫀㫅㪻㩷㫋㪿㪸㫋㩷㪠㩷㪿㪸㫍㪼㩷㪹㪼㪼㫅㩷㫉㪼㪸㪻㫀㫅㪾㩷㪽㫆㫉㩷㪺㫃㪸㫊㫊㩷㪹㫌㫋㩷㪻㫆㫅㩾㫋㫂㫅㫆㫎㩷㫎㪿㪸㫋㩷㫀㫋㩷㫀㫊㩷㪸㫃㫃㩷㪸㪹㫆㫌㫋㪅

㪈㪅㪍㪍㩿㪇㪅㪐㪎㪀㪉㪅㪋㪇㩿㪈㪅㪌㪌㪀㪉㪅㪈㪇㩿㪈㪅㪉㪎㪀

㪨㪉㪇㩷㪠㩷㪽㫀㫅㪻㩷㫋㪿㪸㫋㩷㫎㪿㪼㫅㩷㫋㪿㪼㩷㫋㪼㪸㪺㪿㪼㫉㩷㫀㫊㩷㫋㪸㫃㫂㫀㫅㪾㩷㪠㩷㫋㪿㫀㫅㫂㩷㫆㪽㩷㫆㫋㪿㪼㫉㫋㪿㫀㫅㪾㫊㩷㪸㫅㪻㩷㪻㫆㫅㩾㫋㩷㫉㪼㪸㫃㫃㫐㩷㫃㫀㫊㫋㪼㫅㩷㫋㫆㩷㫎㪿㪸㫋㩷㫀㫊㩷㪹㪼㫀㫅㪾㩷㫊㪸㫀㪻㪅

㪈㪅㪌㪉㩿㪈㪅㪇㪍㪀㪈㪅㪏㪇㩿㪈㪅㪇㪈㪀㪈㪅㪐㪈㩿㪈㪅㪉㪉㪀

㪨㪉㪈㩷㪮㪿㪼㫅㩷㪠㩾㫄㩷㫉㪼㪸㪻㫀㫅㪾㩷㪠㩷㫊㫋㫆㫇㩷㫆㫅㪺㪼㩷㫀㫅㩷㪸㩷㫎㪿㫀㫃㪼㩷㪸㫅㪻㩷㪾㫆㩷㫆㫍㪼㫉㩷㫎㪿㪸㫋㩷㪠㪿㪸㫍㪼㩷㫉㪼㪸㪻㪅

㪁

㪨㪉㪉㩷㪠㩷㫎㫆㫉㫂㩷㪿㪸㫉㪻㩷㫋㫆㩷㪾㪼㫋㩷㪸㩷㪾㫆㫆㪻㩷㪾㫉㪸㪻㪼㩷㪼㫍㪼㫅㩷㫎㪿㪼㫅㩷㪠㩷㪻㫆㫅㩾㫋㩷㫃㫀㫂㪼㩷㪸㪺㫃㪸㫊㫊㪅

㪬㫊㪼㫉㫊㩷㫎㪿㫆㩷㪿㪸㫍㪼㩷㫌㫊㪼㪻㪽㫆㫉㩷㪺㪿㪼㪺㫂㫀㫅㪾㩷㫃㪼㫊㫊㫆㫅㫊㪬㫊㪼㫉㫊㩷㫎㪿㫆㩷㪿㪸㫍㪼㫅㵭㫋㫌㫊㪼㪻㩷㪽㫆㫉㩷㪺㪿㪼㪺㫂㫀㫅㪾㫃㪼㫊㫊㫆㫅㫊㪥㫆㫅㩷㪬㫊㪼㫉㫊

㪋㪅㪏㪐㩿㪈㪅㪏㪏㪀㪋㪅㪎㪊㩿㪈㪅㪉㪏㪀㪋㪅㪈㪇㩿㪈㪅㪎㪍㪀

㪌㪅㪉㪐㩿㪈㪅㪌㪏㪀㪌㪅㪊㪊㩿㪈㪅㪈㪈㪀㪌㪅㪉㪌㩿㪈㪅㪌㪍㪀

㪁㫇㪓㪅㪇㪌㪂㫇㪓㪅㪈

Figure 5 Result of Self-Regulated Learning Strategies (Self-Rgulation: Numbers are average, number in parenthesis is standard deviation) Table 5 Results of Performance Test 㪪㫋㫌㪻㪼㫅㫋㫊㩷㪬㫊㪼㫉㫊㩷㫎㪿㫆㩷㪿㪸㫍㪼㩷㫌㫊㪼㪻㩷㵬㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㵭㩷㫆㫅㫃㫐㩷㪽㫆㫉㩷㪺㪿㪼㪺㫂㫀㫅㪾㩷㫃㪼㫊㫊㫆㫅㫊㩷䋨㫅㪔㪊㪌䋩㩷

㪘㫍㪼㫉㪸㪾㪼㩷

㪪㫋㪸㫅㪻㪸㫉㪻㩷㪛㪼㫍㫀㪸㫋㫀㫆㫅

㪐㪉㪅㪎㩷

㪌㪅㪎㪐㩷

㪬㫊㪼㫉㫊㩷㫎㪿㫆㩷㪿㪸㫍㪼㫅㵭㫋㩷㫌㫊㪼㪻㩷㵬㪪㪺㫀㪼㫅㪺㪼㩷㪥㪼㫋㵭㩷㫆㫅㫃㫐㩷㪽㫆㫉㩷㪺㪿㪼㪺㫂㫀㫅㪾㩷㫃㪼㫊㫊㫆㫅㫊㩷䋨㫅㪔㪈㪌䋩㩷

㪐㪈㪅㪎㩷

㪋㪅㪊㪇㩷

㪥㫆㫅㩷㪬㫊㪼㫉㫊㩷䋨㫅㪔㪉㪊㪌䋩㩷

㪐㪈㪅㪈㩷㩷

㪌㪅㪏㪈㩷㩷

㩷㫅㪅㫊㪅

Q2, Q3, Q9, and Q10 they chose 1, 2 or 3 on the questionnaire. We defined this last group

T. Kitazawa et al. / Science Net: Effects of an e-Learning System

87

of users as those who used Science Net for ‘other purposes’. From the above results it indicated there might be a difference between the students’ purpose for using Science Net and the students’ self-regulated learning strategies. Therefore, we compared the different self-regulated learning strategies with the Motivated Strategies for Learning Questionnaire (Pintrich࡮De Groot 1990), however it is not significantly different between 4 groups by ANOVA. But it is considered that the students who have used the system for checking lessons have higher self-regulated learning strategies such as ‘rehearsing and memorizing’ or ‘reviewing records’ (Zimmerman 1989) than the students who have not used it for checking lessons because they checked the lessons constantly. Therefore, we compared the different self-regulated learning strategies between 35 students who have used it for checking lessons (the users who used it only for checking lessons and the users who used it for checking lessons and the BBS), 15 students who haven’t used it for checking lessons (the users who used it only for checking the BBS and the users who used it for other purposes), and 235 non users. Figures 4 & 5 indicate the results of the self-regulated learning strategies section of Motivated Strategies for Learning Questionnaire (7 choices). 2 cognitive strategy use items tested by ANOVA show significant difference or significant tendency as below; ‘Q1 When I study for a test, I try to put together the information from class and from the book (F(2,232)=2.61, p<.1).’, ‘Q2 When I do homework, I try to remember what the teacher said in class so I can answer the questions correctly (F(2,232)=-4.22, p<.05).’ 5 self-regulation items tested by ANOVA show significant difference or significant tendency as below; ‘Q14 I ask myself questions to make sure I know the material I have been studying (F(2,229)=2.51, p<.1).’, ‘Q16 I work on practice exercises and answer end of chapter questions even when I don't have to (F(2,231)=3.54, p<.05).’, ‘Q17 Even when study materials are dull and uninteresting, I keep working until I finish. (F(2,231)=3.54 p<.05).’, ‘Q18 Before I begin studying I think about the things I will need to do to learn (F(2,228)=2.97 p<.1).’, ‘Q21 When I'm reading I stop once in a while and go over what I have read (F(2,230)=3.50, p<.01).’ These enquiry items were analyzed multiple comparison of Tukey. The findings are ‘Q1 (p<.1)’, ‘Q2 (p<.05)’, ‘Q14 (p<.1)’, ‘Q16 (p<.05)’, ‘Q17 (p<.05)’, ‘Q21 (p<.05)’ had significant difference or significant tendency between the users who have used it for checking lessons and the non users. It had no significant different between the users who haven’t used it for checking lessons and the non users. It is not too far from the truth to say that the ‘Science Net’ users who have used it for checking lessons tend to try to make everything fit together more than the non users from these results. It is clear that when the users who have used it for checking lessons study science, they can do their best whether they like science or not. Additionally, they tried to read their textbooks again and again more than the non users from the results of the self-regulation inquire items. Therefore, it is clear that the students who have used the system for checking lessons have higher self-regulated learning strategies than the students who have not used it. It is unsure if the students who used Science Net already had high self-regulated strategy or the students who used the system improved their self-regulated strategy. Therefore, we want to carry out a thorough investigation of that in the future. 3.3 Performance of Science Table 5 indicates the results of the performance test for a year. The average of the users who have used it for checking lessons is 92.7, the average of the users who haven’t used it for checking lessons is 91.7, and the non users’ average is 91.1. We analyzed them by ANOVA, however, they have no significance because the averages of the students’ tests were high-scoring. Therefore when we analyze their performance of science, we need to

88

T. Kitazawa et al. / Science Net: Effects of an e-Learning System

check not only performance tests but also other materials such as worksheets, homework, and so on.

4. Conclusion This paper describes the assessment of our science e-learning site ‘Science Net’ which elementary school students have used it for one year. First, the findings show that it is not only being used it for checking lesson content but also for asking questions, lesson revision, and searching for science information via a bulletin board system (BBS). Second, the users who have used it for checking lessons tend to score higher in the self-regulated learning strategies than the non users from the results of the Motivated Strategies for Learning Questionnaire (Pintrich࡮De Groot 1990), in terms of 2 of the cognitive strategies (When I study for a test, I try to put together the information from class and from the book, and so on), and 4 of the self-regulations (I work on practice exercises and answer end of chapter questions even when I don’t have to; when I’m reading I stop once in a while and go over what I have read, and so on.). But, when we analyze their performance of science, we need to check not only performance tests but also other materials such as worksheets, homework, and so on. And then, it is unsure if the students who used Science Net already had high self-regulated strategy or the students who used the system improved their self-regulated strategy. It is important that the students have self-regulated learning strategies when they study with e-learning because they have the potential to dropout if they don’t have the strategies (Kougo & Nojima 2004b). Snowman (1986) suggested that the learning cycle which the students would have the self-regulated learning strategies as below; 1) setting up a goal, and analyzing the learning strategies, 2) designing the learning strategies, 3) implementing the learning strategies, 4) reflecting their learning outcome, 5) improving their learning strategies, and 6) checking all learning steps with their meta-cognitive strategies. We hope that the learning cycle will be considered for use in science classes with ‘Science Net’, and all elementary school students will use it for checking the science lessons in the future. Reference Kitazawa,T., Kato,H., Akahori,K. (2006) A Study of How Elementary School Students Acknowledged our Science e-Learning Site: The Effectiveness of the ‘Science Net’ as a form of Blended Learning, Journal of Science Education in Japan,30(1): pp.37-47 Kitazawa,T., Nagai,M., Kato,H., Akahori,K. (2005) Development and Assessment of the ‘Science Net’ e-Learning System for Science Education at the Elementary School Level, Proceedings of International Conference on Computers in Education 2005: pp.740-743 Kogo, C., Nojima, E. (2004a) Self-Regulated Learning of eLearning, Proceedings of the Japan Psychological Association the 68th Annual Meeting: p.1157 Kogo, C., Nojima, E. (2004b) Student Dropout in e-learning and its Symptom, Proceedings of the 20th Annual Conference of Japan Society for Educational Technology: pp.997-998 Pintrich & De Groot (1990) Motivational and Self-Regulated Learning Components of Classroom Academic Performance㧘 Journal of Educational Psychology, 82(1): pp.33-40 Programme for International Student Assessment (PISA 2000), Organization for Economic Cooperation and Development (OECD), http://www.pisa.oecd.org/ Snowman, J.: Learning tactics and strategies. In G.D. Phye & T. Andre (Eds.), Cognitive classroom learning: Understanding, thinking, and problem solving. Orlando,FL: Academic Press, 1986. Trends in International Mathematics and Science Study 2003 (TIMSS2003) of International Association for the Evaluation of Educational Achievement (IEA)(2003) Zimmerman,B.J. (1989) A Social Cognitive View of Self-Regulated Academic Learning, Journal of Educational Psychology, 81(3): pp329-339 Zimmerman,B.J.(1998) Developing self-fulfilling cycles of academic regulation: an analysis of exemplary instructional models. in D.H. Shunk & B.J.Zimmerman (Eds.), Self-regulated leaning, .New York: The Guilford press Zimmerman & Martinez-Pons (1990) Student Differences in Self-Regulated Learning: Relating Grade, Sex, and Giftedness to Self-Efficacy and Strategy Use, Journal of Educational Psychology, 82(1): pp.51-59

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

89

Using Satellite Resources for Scientific Inquiry Beaumie Kima, Manetta Calingerb, Debbie Denise Reeseb National Institute of Education, Nanyang Technological University, Singapore b Center for Educational Technologies, Wheeling Jesuit University, U. S. A. [email protected]

a

Abstract: NASA Learning Technologies is a development effort to create learning tools that provide access to NASA content and data in an engaging and dynamic manner. The present study is the initiation of the design research for one of those developed tools, which provides access to the satellite imagery and scientific visualizations on a 3D globe. NASA education’s challenge is not the incorporation of authentic high-tech resources. Rather, it is the preparation for educational use with well-defined science topics. The results of this study suggest that using this tool within a motivating context and an appropriate learning activity could provide positive impacts on science understanding and learning experiences. Keywords: Science education, inquiry, visualization, 3D, evaluation

Introduction The science education community is placing increasing emphasis on classroom practices of authentic scientific inquiry [e.g., 1]. Inquiry as a pedagogical strategy is adopted for learners to experience important aspects of science practice, which includes scientists’ use of modern science technology [2]. As one of the U.S. government’s most influential scientific organizations, the National Aeronautics and Space Administration (NASA) has an important mission of inspiring the next generation in science, technology, engineering, and mathematics careers. With a wealth of research resources and world-class facilities, NASA aims to provide unique learning opportunities of modern scientific inquiry to students and educators [3]. NASA’s Learning Technologies initiative is specifically focused on making such scientific resources usable and freely available to students and educators. This study initiated the design research for one of NASA’s advanced learning technology applications. 1. Perspectives: Doing and Understanding Science 1.1 Visualizations in Science Education What does it mean to do science and be a scientist? Despite the emphasis on enhancing students’ conceptions of the nature of science, research shows that K-12 students as well as their teachers do not have a complete understanding about them [4]. Learning environments with authentic inquiry activities support construction of students’ understanding of science practice and knowledge. The science technologies, such as scientific visualizations, have great potential to provide learners with meaningful experiences in science practice. As the use of technology in science is rapidly increasing, educational researchers also give attention to the potential affordances of emerging technology for inquiry-based pedagogies [5, 2]. Many science areas now use computational devices for their research to facilitate the collection and interpretation of massive data sets [6]. To provide learners with authentic experience as novice scientists, there is enterprise to develop learning tools that

90

B. Kim et al. / Using Satellite Resources for Scientific Inquiry

imitate the tools of scientists. This tool development is based on the belief that the learning of scientific content should be grounded in an understanding of the scientific investigation processes and an application to real problems [7]. The Learning Through Collaborative Visualization (CoVis) project, which incorporated weather visualizations, went through the iterations of research and development [8]. They studied the implementation of a six-week global warming curriculum during which middle/high school students prepared briefings for a fictitious international conference. The study showed that the CoVis project provided a coherent motivating context, but with the issue of time commitment for in-depth inquiry [2]. 1.2 Finding Meaningfulness Through Science Practice with NASA Learning Technologies Learners often establish relations within the learning context in regards to the personal importance or meaningfulness (e.g., the athletically talented may regard it important to excel in every physical education activity). Wenger [9] interprets learning as social participation. The participants initially learn by taking a secondary role in the practices; they gain meanings of their practices through the learning experience. The social learning context in the classroom and the recognition of learners’ roles as participants are significant for the performance of the learners. However, the sense of belonging beyond the local context is what makes the learning context even more authentic. The learning experience provides learners with images of the world, with a view of past and future possibilities and with a better perspective of their position in relation to the world [9]. NASA Learning Technologies is a research and development effort to create learning tools that provide access to NASA resources in an engaging and dynamic manner. NASA has created a set of time-series satellite imagery and data visualizations, called SVS (Scientific Visualization Studio) and an application that lets you dynamically browse, download, and display them on 3D globe, called World Wind. Spatial representations transform the world into images by symbolizing, changing scales, and taking perspectives (i.e., reference frames) [10]. SVS/World Wind not only incorporates composites of spatial properties, i.e., satellite imagery and scientific visualizations, in scientifically meaningful manner but also potentially makes them personally meaningful to the learners. 1.3 The Potentials and Problems Dewey [11] argued that the gist of thinking is what we suggest to others based on what we see in the world. An understanding about space helps us “see” spatial properties as evidence with which we can suggest certain patterns, changes, and/or possible solutions to problems. Some problem-based learning (PBL) approaches direct learners to scientific visualizations from scientific organizations to guide authentic inquiry (e.g., Exploring the Environment, ETE). PBL, however, does not always provide a good interface to examine visualizations. SVS/World Wind could provide learners opportunities to 1) engage in authentic inquiry with scientific visualizations in an interactive interface, 2) find global meanings of their classroom practice by using NASA science/technology and dynamically communicating with NASA data servers, and 3) gain meaningful understanding about spatial properties through dynamic 3D Earth perspectives. The purpose of this study was to examine the potential of this new way of presenting scientific visualizations for a positive impact on science understanding and learning experience. The four guiding questions are: 1. How well can SVS/World Wind support learning as a learner investigation tool? 2. What experience (troubles/successes) do learners have during activities? 3. What tradeoffs, if any, are there for using SVS/World Wind? Are they worthwhile? 4. How can SVS/World Wind better support learning (directions)?

91

B. Kim et al. / Using Satellite Resources for Scientific Inquiry

2. Methods 2.1 Conditions and Design The El Niño unit of the ETE was modified into two sets of five one-hour lessons. Both sets provided potentially motivating context for learners, learner-appropriate activities and data, and the support from the same instructor [6]. The scaffolding interface was the condition. Context: The El Niño 2005. Predicting the upcoming El Niño event addresses the real situation facing scientists at present and provides a motivating context for learners: In the summer 2005, El Niño may develop in the Pacific. Early spring indications showed it may bring the same detrimental effects as the 1997-98 El Niño. The Australia Predictive Ocean Atmospheric Model issued warnings on this possibility. The US National Oceanic and Atmospheric Administration (NOAA) and NASA experts do not expect such a severity.

Learners play the roles of NASA (treatment) or NOAA (control) scientists to investigate the El Niño data and to either confirm or refute the Australian prediction. Learner-Appropriate Activities and Data. To understand the patterns in climate conditions during a severe El Niño, learners investigated data from 1997-98 (sea surface temperature/height anomalies). They then examined the recent data up to early July 2005 to make their predictions on a possible 2005 El Niño. Scaffolding Interface. The treatment group used World Wind and its data browser to observe visualizations on 3D globe (Figure 1). The data browser displays time-series data as sequenced animated frames or individual frames. While viewing the data, learners can zoom in/out of places and rotate the Earth. The control group used the NOAA and other websites for data visualizations of sea surface temperature/height anomalies (Figure 2).

Figure 1. World Wind data visualization

Figure 2. Web-displayed 2D visualization

2.2 Instruments and Procedures The information from multiple sources addresses the four questions (see Table 1). The analysis of pre/post tests and learner products mainly informs the learner performance. Surveys and observations address the learners’ experience and engagement. The data would indicate the tradeoffs of and directions for using SVS/World Wind for learning. Learning and Performance. Pre/post tests had identical 18 multiple-choice items: the definition (2 items) and effects (6 items) of El Niño; background (4 items) and practical knowledge (3 items); and understanding scientific visualizations (3 items). Learners presented their predictions with evidence from visualizations. We examined the differences between two groups in 1) knowledge gains, and 2) the quality of their presentations. Table 1. Alignment Between Data Sources and Questions Pre-/Posttests Learner products Surveys Observations

1. Learning u u u u

2. Experience u u u

3. Tradeoffs u u u u

4. Directions u u u u

Experience and Engagement. The pre-survey included items on learners’ previous science learning experience, computer experience, and expectations about this activity. The

92

B. Kim et al. / Using Satellite Resources for Scientific Inquiry

post-survey asked about their science learning experience with technology compared to their expectations. They also filled out a daily student evaluation answering what they most and least liked about the day and one thing they learned. We examined 1) learners’ engagements, 2) partner interactions, and 3) the role of technology for their activities. Procedures. The participants were recruited from a five-day camp, in which students in grades 5-9 learned about space travel, extreme weather, and robotics. All 26 campers agreed to participate and were randomly assigned to the treatment (SVS, NASA) and the control (Webpage, NOAA) group 1 . One instructor administered both groups’ activities to control for teaching abilities. When one group was doing the El Niño activity, the other group had a different camp activity for an hour. Then the two groups switched activities. 3. Results 3.1 Overall Gains and Performance Knowledge Gains. A 2×2 mixed model repeated measures (condition×time) examined the effectiveness on knowledge gains. Four of the 26 participants did not complete pretest or posttest (two in each condition). The mean and standard deviation for the tests are in Table 2. The analysis showed a significant effect for time, F (1, 20) = 36.714, p < .001, Șpartial2 = .647. Neither the main effect nor the interaction (condition×time) was significant. We could not determine the differences in effectiveness on knowledge gain. The average gain for SVS group was 3.3, ranging from -1 to 7. The average for the webpage was 3.3, from -1 to 6. Table 2. Descriptive Statistics of Test Measures Pretest SVS (NASA) Webpage (NOAA)

Posttest

N

X

SD

X

SD

11 11

9.09 9.45

2.66 1.86

12.36 12.72

2.01 1.61

100%

Pre SVS

90%

Pre Webpage

80%

Post SVS Post Webpage

% Correct

70% 60% 50% 40% 30% 20% 10% 0% Definition

Effects

Background

Practical

Visualizations

Figure 3. Percentage of correct answers for categories of items A similar result was found for the item categories (see, Figure 3). Using the 2×2 repeated measures design, three categories showed significant effects for time: El Niño effects (6 items), F (1, 20) = 10.198, p = .005, Șpartial2 = .338; background knowledge (4 items), F (1, 20) = 18.404, p < .001, Șpartial2 = .479; and practical knowledge (3 items), F (1, 20) = 11.575, p = .003, Șpartial2 = .367. The interactions (condition×time) were not significant and there is no evidence for the differences in gains in any category between the two groups. 1

The groups were called NASA and NOAA to provide a realistic/motivating context. They will be referred as SVS and Webpage in discussion of the findings as the results do not indicate differences in the two organizations.

B. Kim et al. / Using Satellite Resources for Scientific Inquiry

93

The gains on the definition of El Niño (2 items) and analyzing visualizations (3 items) were not significant for either group. Although statistically insignificant, Webpage group might have gained more understanding for the background knowledge category. We expected the Webpage group might benefit from the more focused activities. They may have had less distraction within the interface and more involvement in reading through the websites. Naïve Exploration with 3D Visualizations. The gains seem to be positively related to the participants’ control of the visualizations. The Webpage participants’ gain scores were highly consistent with observation. Persons leading the team or looked enthusiastic gained more. The four participants from the Webpage group who had less control over and/or were less engaged in their activities gained only between -1 and 1 from the pretest to the posttest. Some SVS participants showed extreme exceptions. A participant (Rob) who was always quietly engaged in the activity had -1 gain score, and his partner (Carl) gained only 2 points. A participant (Alan) who expressed his excitement of using the software and looked most engaged in the investigation had -1 gain score. These cases could not be explained by our remote observations, but their presentations revealed how they went awry with their investigations. The data visualizations of 1997-98 El Niño indicators provided only the slice of data from the Pacific Ocean between 20 degrees north and south (see, Figure 1). This focused the learners’ attention to the area that was relevant to El Niño patterns and effects. When examining 2005 data, which had data for the entire Earth, these two teams seemed to have gotten lost. Rob and Carl claimed that Figure 4 demonstrated a similar abnormality of sea surface temperature as that of 1997-98 data, in which data from the Atlantic Ocean was excluded. Likewise, Alan and Luke explained their prediction of the severity of El Niño by showing the abnormal sea surface temperature of the North Atlantic Ocean (Figure 5).

Figure 4. Screen capture by Rob and Carl

Figure 5. Screen capture by Alan and Luke

Reporting Scientific Explanations. There were qualitative differences between the two groups in reporting their observations. SVS data visualizations are accessed as events, such as El Niño 1997-98, and presented as animations. The Webpage group used a website with static visualizations, with which they had to find a certain day of a month one by one to examine them. Resulting from this difference, most of the SVS group participants broadly discussed data visualizations, using the phrases as the previous El Niño, 1997-98 El Niño, current conditions, this year, etc. Webpage participants, however, provided at least the month and the year for the visualization, which showed on the downloaded graphics. Another difference was in how the background information about El Niño and its effects were reported. A majority of Webpage participants talked about why people worry about having a severe El Niño. This information comes from the handouts/websites, not from the data. One team concluded its presentation with a list of expected El Niño effects without any claim and evidence from the data. SVS participants, however, placed less stress on the additional background and more on the observed patterns of the visualizations. 3.2 Learner Experience The SVS group gave more positive responses for post-survey. Examining the sum of 17 items (out of 29) directly related to learner contentment (alpha = .94), there was a significant difference between the two: t(20) = 2.41, p= .026. The learning with SVS ( X =62.82) provided more positive learning experience than static visualization ( X =49.55).

94

B. Kim et al. / Using Satellite Resources for Scientific Inquiry

A. Overall Learning Activity. Both groups had somewhat positive experience, and some participants talked about this activity with their parents. However, not many of them learned more about El Niño outside of the camp. SVS group felt as if having authentic experience more than Webpage group. The differences were significant for item A4, “I felt as if I was a scientist while participating in the El Niño activity,” t (19) = 2.41, p = .027, and item A9, “I enjoyed presenting our discoveries of our research to others,” t (19) = 2.10, p = .049. Both groups seemed to enjoy predicting the next El Niño and feel like real scientists. B. Use of Scientific Visualizations. Regarding the use of scientific visualizations, the SVS group responded more positively. A significant difference was for item B1, “The science/technology helped me understand about El Niño better,” t (20) = 2.11, p = .049. SVS participants seemed to think their learning experience was enriched and exciting more than Webpage participants. When asked what they liked most about the learning activity, one SVS participant said, “I liked spinning the globe, using it to see the patterns of El Niño,” and another said, “It was different from what I had experienced before, and I enjoyed working in a different way.” Another significant difference was identified by item B3, “Looking at the visualizations, I argued with my partner to agree on what was going on,” t (20) = 2.50, p = .021. This result was consistent with our observations. For the SVS group there were discussions about visualizations among partners within each team as well as among different teams. There was only one Webpage team in which one member was vocal and provoked more conversations among the team members. C. Working with Others. Both groups of participants seemed to enjoy their experience with their partners. There, however, was a stronger pull toward working alone by the SVS group as exposed in item C6, “I prefer working by myself,” t (20) = 3.28, p < .01. The SVS participants who were engaged in and learned more from the activities seemed to consider their partner as a distraction. In the open-ended items, SVS group had more complaints about the lack of partners’ cooperation. Some of the SVS participants still preferred working by themselves even without any problem with partners. With an additional test on the knowledge gain and the item C6, “I prefer working by myself,” the significant correlation was found for the SVS group: F (1, 9) = 7.586, p = .022. There was no significance on this test for Webpage group. Interestingly, SVS group still enjoyed some pair-work. Item C3, “I enjoyed preparing the presentation with others,” showed a significant difference between the two groups: t (20) = 2.81, p = .011. In the open-ended survey items, eight SVS participants expressed a preference for presentations, and none mentioned a dislike. Six Webpage participants mentioned their enjoyment in the presentations, and three least enjoyed this aspect of the activity. Both groups wanted more preparation time. 4. Discussion The results suggest that using SVS/World Wind within a motivating context and an appropriate learning activity could provide positive impacts on science understanding and learning experience. There seem to be some tradeoffs in adopting this learning technology. Tradeoff 1: Naïve Explorations. The participants’ lack of foundation in geography and Earth science knowledge seemed to have a greater impact when there was more opportunity to explore. While investigating El Niño indicators, several participants of both groups had trouble locating South America. Some Webpage group participants could find the area only when they could see the expanded red on the map for sea temperatures during the 1997-98 El Niño. This was not a problem with SVS group for the 1997-98 data because only the relevant area was shown on the visualization. When examining the development pattern of 2005 El Niño, SVS participants were exposed to the data for the entire Earth. In the cases of Rob, Carl, Alan, and Luke in the SVS group, they looked for highly anomalous data from any place on Earth for the indicator of the El Niño development. This tendency of

B. Kim et al. / Using Satellite Resources for Scientific Inquiry

95

SVS participants could be easily detected based on how they presented the evidence with SVS visualizations (see, Figures 4 and 5). This would not have been revealed from Webpage participants’ presentations because they do not decide their visualization views. The lack of geography knowledge led some SVS participants to provide incorrect evidence. However, SVS group participants seemed to take more ownership of what they discovered from those data sets. At the same time, World Wind’s 3D globe interface made the participants’ need of geography knowledge more explicit to themselves, their peers (the presentation audience pointed this out), and the teacher. This should give learners more opportunities to address such needs when used the software for an extended period. Tradeoff 2: Reporting Scientific Explanations. By examining the animated sequence of data visualizations, SVS participants seemed to get a better grasp of its changing pattern than Webpage participants who had to examine one page at a time. In their presentations, however, SVS participants used a capture of one day’s data visualization but treating it as a representation of the whole period of the 1997-98 El Niño or the first half of 2005. Even if the instructor specifically pointed out several times how to find the date information, they seemed to affix to the idea that they were examining a period. This is definitely a tradeoff between two conditions in terms of getting the bigger picture versus paying attention to the data details. Both insights are essential for scientific investigations. There seems to be a relationship among the Webpage participants’ reporting of background El Niño information, their trend of more gain in background knowledge category, and time constraints of this study. Most SVS participants focused on exploring with SVS El Niño indicators and then had enough time to collect appropriate visualizations for their claims. Webpage participants seemed to have time to look up some information. A SVS team who took time to discuss other information, however, could not finish its investigation with the second dataset. Tradeoff 3: Engagement vs. Distraction. SVS participants felt more engaged in authentic science practice. One of the SVS participants commented about SVS/World Wind, “I think it would help meteorologists predict weather better in the future.” The 3D globe interface, however, is not the typical way that scientists work with data visualizations. The initial excitement of the participants could be the novelty effect with a 3D globe interface, but their feeling of authentic engagement could stem more from manipulating their view, controlling the animated sequences, and therefore, gaining more ownership over the colleted data. This is somewhat evidenced by the significant difference in the item B3, “Looking at the visualization, I argued with my partner to agree on what was going on,” in that each team member perhaps found his or her own observations meaningful to them. This provides a possible explanation for this item and the relevant observations (i.e., only one team with a vocal member in Webpage group had active conversations): Opportunities to explore with 3D globe provoked more learner discussions for the SVS group. This leaves us with an exciting potential for supporting authentic scientific process. Scholars of science education see argumentation as a core scientist activity [e.g., 12, 13, 14, 15]. With the current implementation, we cannot say if learners were engaged in any scientific argumentation. The future implementation should consider this scientific practice. Distracted behaviors were shown by both groups (i.e., playing games, changing background, etc.). For the SVS participants, the 3D interface provided another source of distraction, such as zooming in/out on familiar locations, spining the globe, or exploring other SVS data. Such behaviors seemed to be related to their attitude toward the learning. The engaged participants paid some attention to such behaviors of others, follow it for a short period of time, and then return to their tasks. 5. Conclusions Visualizations available from scientific organizations offer great potential for learning.

96

B. Kim et al. / Using Satellite Resources for Scientific Inquiry

SVS/World Wind has the capacity to enable learners to meaningfully interpret science resources. The challenge is not in the incorporation of authentic, high-tech resources but in the preparation for educational use with well-defined topics. U.S. Department of Education [16] pointed out that teachers are now left with an abundance of resources without sufficient understanding of how to use them effectively to enhance learning. Without providing any guides to teachers, the great potential afforded by learning technologies might be wasted. With enhancements, SVS could strongly support such lesson plans for science education. For example, with baseline data (e.g., sea surface temperature anomaly for mild El Niño years) for comparison to event-based data (e.g., sea surface temperature anomaly during 1997-98 severe El Niño), learners would better understand the meaning of data. It is also essential to focus the categories of data sets to the topics that are most essential for K-12 science education and provide constant updates and additions to such collections. This study was intended to be formative and small-scale as we conducted an initial exploration of the instructional potential of SVS/World Wind. We could not find any difference in the effectiveness in participants’ knowledge gain between using SVS/World Wind and using the simple web interface. We expect better outcomes for future studies on long-term effects with enhanced learning technology and appropriate curricula. References [1] [2] [3] [4]

[5]

[6] [7]

[8]

[9] [10]

[11] [12] [13] [14] [15] [16]

National Research Council. (1996). National science education standards. Washington, DC: National Academies Press. Edelson, D. C., Gordin, D. N., & Pea, R. D. (1999). Addressing the challenges of inquiry-based learning through technology and curriculum design. Journal of the Learning Sciences, 8(3/4), 391-450. National Aeronautics and Space Administration. (2003). Education enterprise strategy (No. NP-2003-10-322-HQ). Washington, DC: NASA Headquarters. Lederman, N. G., Abd-El-Khalick, F., Bell, R. L., & Schwartz, R. S. (2002). Views of nature of science questionnaire: Toward valid and meaningful assessment of learners' conceptions of nature of science. Journal of Research in Science Teaching, 39(9), 497-521. Brown, A. L., Ash, D., Rutherford, M., Nakagawa, K., Gordon, A., & Campione, J. C. (1993). Distributed expertise in the classroom. In G. Salomon (Ed.), Distributed cognitions (pp. 188-228). New York: Cambridge University Press. Edelson, D. C., & Gordin, D. (1998). Visualization for learners: A framework for adapting scientists' tools. Computers & Geosciences, 24(7), 607-616. Loh, B., Reiser, B. J., Radinsky, J., Edelson, D. C., Gomez, L. M., & Marshall, S. (2001). Developing reflective inquiry practices. In K. Crowley, C. D. Schunn & T. Okada (Eds.), Designing for science (pp. 279-323). Mahwah, NJ: Lawrence Erlbaum. Edelson, D. C., Pea, R. D., & Gomez., L. (1996). Constructivism in the collaboratory. In B. G. Wilson (Ed.), Constructivist learning environments: Case studies in instructional design (pp. 151-164). Englewood Cliffs, NJ: Educational Technology Publications. Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge, MA: Cambridge University Press. Liben, L. S., & Downs, R. M. (1991). The role of graphic representations in understanding the world. In R. M. Downs, L. S. Liben & D. S. Palermo (Eds.), Visions of aesthetics, the environment & development (pp. 139-180). Hillsdale, NJ: Lawrence Erlbaum. Dewey, J. (1910/1997). How we think. Mineola, NY: Dove. Bell, P., & Linn, M. C. (2000). Scientific arguments as learning artifacts: Designing for learning from the web with kie. International Journal of Science Education, 22(8), 797-817. Driver, R., Newton, P., & Osborne, J. (2000). Establishing the norms of scientific argumentation in classrooms. Science Education, 84(3), 287-312. Kuhn, D. (1993). Science argument: Implications for teaching and learning scientific thinking. Science Education, 77(3), 319-337. Nussbaum, E. M. (2005). The effect of goal instructions and need for cognition on interactive argumentation. Contemporary Educational Psychology, 30(3), 286-313. U.S. Department of Education. (2004). Toward a new golden age in American education (National Education Technology Plan 2004). Jessup, MD: U.S. Department of Education.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

97

Experimental Researches on Development of Pupils’ Advanced Cognitions in PRIME Environments Zhou Yueliang1, Wang Lianghui1, Lin Xiuqin2 School of education, Zhejiang Normal University, China 2 Knowledge Science and Engineering Institute, Beijing Normal University, China 1

Abstract: ICT application in education has become one of the most attractive topic discussed in modern educational theory and practice. This paper try to verify the effect of ICT application in education, especially in development of students’ advanced cognitive abilities, by confirmative experiments in online learning support system named PRIME. After data analysis, the paper gets the conclusions that the PRIME improves students’ learning achievement by affect the students’ advanced cognitive abilities, and the students’ advanced cognitive abilities, as intermediate variables, play very important roles on the performances of students, where PRIME affects their learning. Keywords: Advanced Cognition; PRIME; Virtual learning Environments (VLEs);

subject learning strategy (SLS). 1. Introduction Recently, the application of ICT (Information and Communication Technologies) in education has become one of the most attractive topic discussed in modren educational theory and practice, and the countries all over the world have paid much attentions to it because it brings many new features of learning and teaching. As researches show, the mastery over the skill of online learning has close relationship with the advanced cognition, especially the self-cognition ability (Hong, McGee & Howard, 2000). Also, learning under ICT environments requires the learners of more independence and self-control, being called them advanced cognitive abilities. In fact, the relationships between ICT and the development of advanced cognition was paid much attention in late 1990s. Some people focused on the influence of the advanced cognition on learning in ICT environment, e.g. Hill & Hannafin˄1997˅, Hill(1999) , Jegede(1999) , Aleven & Koedinger(2002). Some studied on the cultivation of advanced cognition by ICT, e.g. Collins & Brown(1991), Herrington & Oliver(1999,2000), Lin(2001). Tools aimed at advanced cognition had also been developed, e.g. CSILE/KnowledgeForumˈSpeakEasy (Hsi & Hoadley, 1997), Belvedere (Cavalli-Sforza, Weiner, et al, 1994), Convince Me (Schank, 1995), Euclid (Smolensky, Fox, et al, 1988)ˈand SenseMake˄Bellˈ1997˅etc. All these researches mentioned that the support of the society is very important to the construction of learners’ knowledge, especially the acquisition of advacned cognition. In the paper, a VLEs named PRIME of cultivating the student’s advanced cognition is introduced. It has five key functions (PRIME is the abbreviation from all the functions’ first letter): a) the Process and Production of teaching and learning; b) the plan, summary,

98

Z. Yueliang et al. / Experimental Researches on Development of Pupils’ Advanced Cognitions

and Reflection of the learning and teaching; c) Interaction between the learning and teaching; d) Multilevel collaboration; e) Educational administration function. Advanced cognition is the most important variable in this research, but the concept of advanced cognition is very vague. In The Modularity of Mind, Fodor (1983) pointed out that the process of mind has two kinds: the advanced and the lowed, corresponding input system and central system in human’s information process. The central system referred to advanced cognitive processes includes the form of a belief, thought, reasoning, problem solving etc. Apparently, this research cannot include all advanced cognitive abilities. So we mainly focus on some important and typical advanced cognitions such as cognitive strategy ( or problem-solving strategies), self-regulatation (self-control) and reflection. They have a large impact on learning in ICT environments. Further, because ICT is the medium in the case, the student ICT literacy is included as a part of advanced cognition. 2. Research methods 2.1 Participants All the Participants are students from two elementary schools. One is in Shunde, Guangdong province and the other is from Zhuji, Zhejiang province. Each school participants are divided into two groups. One group students learn on the PRIME and their records of online activities will be analysed as independent variables. The students will use the VLEs and the relationship between advanced cognition and utilization of PRIME without the interferance of teaching will be explored. The other group is the same with the group except for they were interfered by the teaching and experiment. There are more than 1200 Participants altogether. The structures of the paticipants are showed in Table1. Table 1. The distribution and constitution of the participants school

Shunde

Zhuji

Types of class

Grade 3

Grade 4

Grade 5

Boy

Girl

Boy

Girl

Boy

Girl

Experimental

30

15

46

36

43

Comparative

26

19

22

16

Experimental

48

26

23

Comparative

33

34

22

Grade 6 Boy

Girl

Total Boy

Girl

35

119

86

25

17

73

54

15

25

15

17

19

113

75

18

44

35

50

23

149

110

2.2 Research Design The research is focus on two problems delt with by two experiments: experiment 1 (E1) and experiment 2 (E2). The E1 is focus on whether PRIME will facilitate the development of advanced cognition with no teaching interference. It takes the records of students’ online activities as the independent variables and the advanced cognitions as dependent variables. The E2 mainly focuses on whether the new teaching model using PRIME improve the students’ learning strategies. It takes the teaching model as the independent variables and the learning strategies as dependent variables The two experiments lasts for 2 years and proceding seperately. Concerning about the launch time of expriments, the E1 is half a year earlier than the E2. During the half year, paticipants were instructed to be familiar with PRIME and acquire ICT performances and it can make E2 smooth. Learners’ achievements (reading & writing, mathmatics, science and information technology (IT))

Z. Yueliang et al. / Experimental Researches on Development of Pupils’ Advanced Cognitions

99

are served as indexes of development of advanced cognition and effects of PRIME. All students’ ICT operating abilities are enough to accomplish the experiment, avoiding bias due to different abilities ICT. Participants in E2 was showed in Table 2. Table 2. Disposition of Experimental Group (E2) School

Shunde

Dis of exp Reading Writing

Math

Zhuji Science

IT

Reading Writing

Math

IT

Grade3(5) Grade5(1) Grade4(1) Grade4(1) Grade3(5) Grade3(8) Grade5(6) Grade3(8) Grade3(8) Grade5(1) Grade5(2) Grade4(2) Grade4(2) Grade4(1) Grade3(9) Grade4(7) Grade3(9) Grade3(9)

classes

Grade4(2) Grade4(6)

Grade4(6) Grade4(6)

Grade5(1)

Grade5(6) Grade5(6)

Grade5(2)

Grade6(7) Grade6(7)

2.3 Research Procedure The experiments in two schools are proceeding seperately with same start and finish time. The E1 and E2’s process of research are differed. E1 began with the official launch of PRIME and lasted for one year. Students were asked to do various activities on PRIME such as writing Blog, sharing resources, commenting works of each other, and etc. These activities were recorded in the form of E-portfolio. Compared with the E1, E2 procedure is more complex and strict. Four phases and control means are included: (1) Exploration of new teaching models: Researchers and teachers together discuss how to utilize PRIME in classroom and finalize some new teaching models based on PRIME. Every model is made certain by four process: design, teaching trial, amending and rechecking. Reading and writing teaching model will be discussed in this part. (2)Pre-test: including tests of all scores of the subjects, GOALS and questionnaires of students’ learning strategies. The pre-test ensure rough equal of learning, cognition and metacogtnition between the experimental class and comparative class. (3) Formal experiment: Teachers use the model to teach designed in the first phrase and the time lasts for 2 months. (4) Post-test: things that tested in this period is the same with the privious test. At last, all data are processed using SPSS11.5. Reading is very important abilities in elementary school. In the designed reading model within PRIME, the teacher analyse the task of a reading and decide what materials and questions to present to the students. After all the materials needed are prepared by teacher and students, teacher will use PRIME tools to unify and refine the materials and make them in hyper-text. Then teacher uploads all the things together on the internet and studets can download and browse them. The details of the model can be found in Figure 1.

Figure 1. Reading teaching on PRIME

Figure 2. Writing teaching on PRIME

Writing is a difficulty for elementary school teaching. Students have limited living circle and few experiences. Many of them are in defect of headspring and desire for

100

Z. Yueliang et al. / Experimental Researches on Development of Pupils’ Advanced Cognitions

writing. PRIME gives them a good place exchanging everyone’s ideas andexperiences and inspire students’ interestings using web publishing. The details can be seen in Figure 2. 2.4 Testing Tools Experimental results evaluated mainly from learning achievement, cognitive and meta-cognitive strategies, motivation and goal setting strategies and subject learning strategies (SLS). Learning achievement adopts the means of school examination and evaluation of homework, here mainly introduce tools to test the other variable. GOALS is abbreviation of the Goal Orientation and Learning Strategies Survey, created by Martin Dowson in his dissertation. GOALS’ structure and subscales as follows (Dowson,2003) : a) academic goals; b) social goals; c) cognitive strategies; d) cognitive strategy; e) additional questionnaires. The reliability of origin GOALS subscales is between 0.72~0.92 and Cronbach Į coefficient of Chinese version is 0.920. That’s to say, the revised questionnaire has good adaptation to Chinese schools and students. Four subjects are included in the experiments: Chinese(reading & writing), mathematics, science and ICT. Subject learning strategy (SLS) questionnaire is mainly used to examine advanced cognitions, such as reading skills, writing skills, math problem-solving strategies and ICT literacy. It has five subscales. With the exception of four subjects subscales, it also contains a special subscale of meta-cognition. Its Cronbach Į coefficient is 0.903. 3. Results and Discuss Two kinds of experiments brought forth two kinds of data. One is online activities records of participants on PRIME, which also can be called active indexes. Active indexes shows students’ visible behaviors on PRIME, such as online time, messages posted, resource or works uploaded, and so on. By analyzing the relationship between active indexes and development of advanced cognition, which community behaviors helping to students’ cognition can be recognized. It can help us design experiments to inspire students to do more contributing community behaviors, which can accelerate their academic performance. The other is the comparative data acquired by teaching interference with the pilot group and its counterpart, such as academic achievement, GOALS scores, result of pre-test and post-test of SLS. By analyzing them, we can understand effects of teaching model on PRIME, also examining the functions of PRIME to advanced cognitive and academic performance. 3.1 Relationships between Active Indexes and Advanced Cognitions The active indexes come into being from March 28th, 2004 to April 1st, 2005. They shows students’ activities in PRIME during this time. This analysis on active indexes prescinds from quality of class which students belong to. Its purpose is to get a general understanding about students’ using of PRIME. With a view to disbenefit impact on statistics result, we made a necessary revise on ultra data of active indexes. Table 3 is about the correlations between active indexes and GOALS post-test. Login times, accumulative online time and the two indexes of sociality which is called social approval and social status all have come forth negtive correlation. Besides them, the number of messages has a obvious relationship with general cognition ,and the number of post also has a obvious relationship with cognition of social status, general cognition and

Z. Yueliang et al. / Experimental Researches on Development of Pupils’ Advanced Cognitions

101

academic support. Table 3: Correlations between active indexes and items in GOALS post-test

Table 4 shows except two active indexes called online time and comments, the rest five active indexes have significant relationships with the total scores of SLS. All seven indexes have significant relationship with ICT, and the number of messages has significant relationship with the scores of reading and writing. Oppositely, accumulative online time has negtive correlation with SLS. This negtive correlation reflects people’s about students’ learning freely online indirectly, which makes tremendous contrast with positive effects caused by the development of advanced cognitive ability. It tells that people’s opinions of ICT should be altered urgently! Generally speaking, active indexes has close relationship with GOALS’ scores and SLS scores. Besides, correlation between active indexes and ICT literacy is very remarkable. Table 4: Correlations between active indexes and SLS

3.2 Relations between Teaching methods in PRIME and Advanced Cognition Developing 3.2.1

Relation between the Teaching Intervention in PRIME and GOALS Scores

Using the multivariate analysis of variance methods to compare GOALS pre-test and post-test scores of control group with experiment group, on GOALS total scores, quality of class (experiment or control group) main affect(F=5.507ˈp=0.021˅ˈgender main affect(F=7.939ˈp=0.006˅ˈclass and gender interaction˄F=4.298ˈp=0.041˅are all significant. It shows that PRIME applications in subjects teaching, gender and their interaction make prominent effects on GOALS total scores. But on subscale scores, these three variables manifest different effects: class quality makes notable effects on mastery

102

Z. Yueliang et al. / Experimental Researches on Development of Pupils’ Advanced Cognitions

goals, society conformity, social affiliation, sense of belonging, while notable main effect on work avoidance, social responsibility, social concern, common strategy˄F=7.933ˈ p=0.006˅, modulation ˄F=4.466ˈp=0.037˅, general cognitive˄F=6.746ˈp=0.011˅, common motivation, sense of belonging. They indicate that PRIME has different effects on boys and girls. It seems that by dint of PRIME to teach students can enhance schoolboys’ cognitive ability more efficiently. Certainly, application of PRIME in teaching has prominent effects on students’ general advanced cognitions, even brings more prominent social effect, such as sense of belonging, motive, etc. The result hints that cultivating methods of advanced cognition by PRIME may highly depend on social communication and community online. It is identical with the cultivation methods of metacognitition mentioned in Lin’s studies (Linˈ2001). 3.2.2

Relation between Teaching Intervention in PRIME and Development of SLS

To SLS, only Grade 4 pupils are involved in experiment. The result manifests that the total score of SLS make significant affects in both group˄F=8.044ˈp=.005˅, but pretest-posttest difference is insignificant; class quality has a prominent main affect on the interaction of pretest-posttest, but has little effect on subscales, which may due to the limited time btween pretest and posttest. The result of another same kind of experiment adopting pretest-posttest experimental design shows that sex difference has little effect on the SLS, but grade difference and class quality make very prominent effect on every development of SLS. Significant level of most items are lower than 0.01 (Table 5). Hence teaching models based on PRIME can promote students’ SLS efficiently. Table 5: Multivariate analysis of variance of SLS

3.2.3

Relations among PRIME, Advanced Cognition, and Students’ Performances

This part will discuss relationships among the application of PRIME, advanced cognition and learning performances so we can have a better understanding of PRIME, in which ICT affects performances. Table 6 shows the correlations between SLS and achievements of participants. From the scores students achieved in two midsemester on a whole, students from both classes can achieve a higher score as long as they have a better learning skill, and also some other

Z. Yueliang et al. / Experimental Researches on Development of Pupils’ Advanced Cognitions

103

factors, such ICT literacy, there being a close relationship between teaching and the development of strategies within PRIME. Anyway, there is still something left for us to explore. For example, in the first semester in 2004, the scores students achieved in Chinese are irrelevant to reading and coordinating skills, and some conditions exist in Math examination for the same semester. Table 6 :The correlation between achievement and SLS

Table 7 adopts multianalysis to analyze the relationships between active indexes and the final exams of the students in Zhuji Experimental School in the first semester, 2004. The result shows that grade, sex, experimental intervention cause different effects on different subjects. It manifests the factors which influence the students’ learning are very complicated in the process of the application of PRIME. Table 7: Analysis on the effect of the application of PRIME on achievement

4. Conclusion This research is according to the theory and practice related to the training of advanced cognition, and adopt education experiment to explore the education application of PRIME and the relation between learning achievement based on PRIME and the primary pupil’s advanced cognition, and also verify the effect of PRIME.

104

Z. Yueliang et al. / Experimental Researches on Development of Pupils’ Advanced Cognitions

From the data analysis, there is significant correlation between students using PRIME and their advanced cognitive ability, which include general metacognition and planning ability, SLS and ICT literacy. But the negative relative between some society indexes and use of PRIME shows that some improper conception of ICT may affect the use of PRIME, and more attentions should be paid on those problems. Although, We cann’t reach the conclusion from the analysis of statistics results that the PRIME can improve students’ learning achievement by affect the students’ advanced cognitive ability. It is evident that there are distinct relationships among the learning achievement, scores of GOALS and SLS, and PRIME plays an active role in the development of the SLS and the GOALS as well. Therefore, we can accept what we have presumed in the very beginning of this paper, that is, advanced cognitions, as an intermediate variable, plays an important role in a system where PRIME affects learning. Of Course, in order to collect more convinced data, further researches should be carried out.

References [1] American Association of School Librarians ˈ Association for Educational Communications and Technology. (1998) Information Literacy Standards For Student Learning: Standards And Indicators. [2] Aleven,V.A.W.M.M. &. Koedinger, K.R. (2002) An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science, 26: 147–179. [3] Bell, P. (1997) Using argument representations to make thinking visible for individuals and groups. In Proc. Computer Supported Collaborative Learning ‘97, University of Toronto˖10-19. [4] Bell, P., Davis, E. A., & Linn, M. C. (1995) The knowledge integration environment: Theory and design. In Proceedings of the Computer Supported Collaborative Learning Conference ‘95. Mahwah, NJ: LEA: 14-21. [5] Collins, A., Brown, J. S., & Holum, A. (1991) Cognitive apprenticeship: Making thinking visible. American Educator, 15 (3), 6-11, 38-46. [6] DowsonˈM.(2003) Relations Between Students' Academic Motivation, Cognition and Achievement in School Settings. Doctorial DissertationˈUniversity of Western Sydney. [7] Herrington, J., & Oliver, R. (1999) Using Situated Learning and Multimedia to Investigate Higher-Order Thinking. Journal of Interactive Learning Research, 10(1): 3-24. [8] Herrington, J. & Oliver, R. (2000) An instructional design framework for authentic learning environments. Educational Technology Research and Development, 48(3), 23-48. [9] Hill, J. R. & Hannafin, M. J. (1997) Cognitive strategies and learning from the World Wide Web. Educational Technology, Research and Development, 45(4): 37~64. [10] Hill, J. R. (1999) A Conceptual Framework for Understanding Information Seeking in Open-Ended Information Systems. Educational Technology, Research and Development, 47(1): 5~27. [11] Hong, N. S., McGee, S. & Howard, B. C. (2000) The Effect of Multimedia Learning Environments On Well-Structured and Ill-Structured Problem-Solving Skills. American Educational Research Association 2000, New Orleans. [12] International Society for Technology in Education. (ISTE, 1998) National Educational Technology Standards for Students. International Society for Technology in Education (ISTE), NETS Project. Available at http://www.iste.org. (Accessed 21 Nov, 2004) [13] Jegede, O., Taplin, M., Fan, R. K.; Chan, M. C. & Yum, J. (1999) Differences between low and high achieving distance learners in locus of control and Metacognition. Distance Education, 20 (2): 255-273. [14] LinˈX. D., Hmelo, C., Kinzer, C. K. & Secules, T. J. (1999) Designing technology to support reflection, Educational Technology, Research and Development; 47(3):43~62. [15] Waxman, H. C., Lin, M. F. & Michko, G. M. (2003) A Meta-Analysis of the Effectivenesss of Teaching and Learning With Technology on Student Outcomes. Learning Point Associates. [16] Pi Liansheng. (2000). Instruction DesignüüTheory and Technology of Psychology (In Chinese). Beijing: Higher Education Publishing House: 43. [17] Wang zhenhongˈLiu Ping. (2000) MotivationǃLearning Strategy and Intelligent Level Affect Students’ Achievement (In Chinese). Acta Psychologica Sinica, 32(1): 65-69.

ITS, et al.

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

107

Teaching Chinese Handwriting by Automatic Feedback and Analysis for Incorrect Stroke Sequence and Stroke Production Errors Kai-Tai Tang and Howard Leung Department of Computer Science, City University of Hong Kong, Hong Kong SAR [email protected] Abstract: Writing Chinese character is not trivial and people often commit stroke sequence error and stroke production errors. In this paper, we propose a web-based education system which allows users to practice Chinese handwriting freely. An Automatic Feedback and Analysis (AFA) tool is introduced to the system which can automatically check both the stroke sequence errors and stroke production errors by analyzing the online data of the learner’s input character. Feedback will be provided if the learner commits errors in the stroke sequence and/or during stroke production. Experimental results demonstrated that our method can check multiple handwriting errors with encouraging accuracy. User studies showed that the learning time is shorter if our proposed system is used. Keywords: Chinese handwriting education, stroke production, stroke sequence, stroke matching, web-based application, pattern recognition, stroke analysis, automatic feedback

Introduction Chinese character is a kind of ideogram composed of strokes with different lengths, shapes and orientations. The generally accepted stroke sequence can be looked up from the Chinese dictionary. Shimomura [1] suggested that the standard stroke sequence was designed for saving energy in writing and easy memorization. Law et al [2] have studied the handwriting ability of junior primary school students and they found that the children often fail to master the stroke sequence and stroke production. Stroke sequence error is committed if one writes a character with the wrong order of strokes. Stroke production error is committed if the handwriting contains (1) concatenated stroke, (2) broken stroke, (3) missing stroke, and/or (4) extra stroke. In a Chinese dictionary, the characters are indexed by either radicals (components of a character) or number of strokes. In order to learn Chinese efficiently, students should correct their handwriting errors first. Otherwise, they may wrongly recognize a radical or realize the wrong number of strokes of a character hence they are unable to lookup a character from the Chinese dictionary. Teacher assessment and student self-correction are two possible ways to correct Chinese handwriting errors. Teachers can point out the mistakes in the students’ handwritings and give them appropriate advices. Student can also compare his/her handwriting with the template to verify that there is no handwriting error. Caroline [8] emphasized that regular feedback can motivate the student in language learning. However, it is impossible for a teacher to be available for every student in a time-limited lesson and a student may not be able to spot his/her own errors through self-correction. There is thus a need to develop an intelligent tool that can automatically verify students’ handwriting and

108

K.-T. Tang and H. Leung / Teaching Chinese Handwriting by Automatic Feedback

give feedbacks to them in order to enhance their learning. This will also ease the workload of the teachers. There are three main streams of Chinese handwriting education systems. The systems in the first stream teach students to write properly [3][4][5][6][7]. Systems in [3][4][5] teach stroke sequence while [6][7] can verify whether the input character and the template character look alike. The second stream focuses on training users to write Chinese beautifully [6][8]. The third stream aims to improve the graphomotor skills such as writing speed and control of hands [7][9]. There are some drawbacks with existing systems. The system proposed in [3] is read-only and users cannot practice handwriting through the system. The systems proposed in [4] and [7] allow users to practice handwriting in a guided manner so they are not able to write freely. Tsang and Leung [5] proposed a PDA education tool that allows users to write freely and can verify the input stroke sequence. The system proposed by Tan et al. [6] also allows students to write freely and it can verify more handwriting problems such as number of strokes and relative stroke lengths. However, it is not flexible and generic enough because it is character dependent such that every time a new character is added, specific threshold values for that character should be set. The system proposed in [6] can only give feedback on one kind of handwriting error even though the learner has committed multiple handwriting errors. To relax the restrictions and limitations of existing systems, it motivates us to propose a system that allows users to practice handwriting freely, and checks both the stroke sequence error and stroke production errors in the input handwriting simultaneously. In our proposed system, users are allowed to practice handwriting freely using a tablet. Unlike handwriting recognition systems [10][11][12][13][14], we perform a detailed matching between each input character with a known template character rather than a set of candidate characters. In our proposed system, we have two levels of matching: stroke-level and character-level matching. The stroke correspondence between the input character and the template is considered in both matching levels. The problematic strokes, which are suspected to contain stroke production errors, are identified. We have introduced the “miss-and-merge” stroke operation that can derive new character instances for both sample and template characters by either throwing away or merging problematic strokes. Each character instance contains information about a potential production error, and the overall production error is determined by the matching the character instances. The effectiveness and accuracy of our system are evaluated with challenging samples that contain multiple handwriting errors. This paper is organized as follows. The proposed method is described in section 1. The experiments, user studies and results are presented in section 2. Conclusions and future work are provided in section 3.

1. Proposed Method Our proposed system consists of two parts: the user interface and the Automatic Feedback and Analysis (AFA) tool. Fig. 1(a) shows the user interface for the learners to practice handwriting. The user has to follow the template character on the “Example Box” and then write on the “Exercise Box” using a tablet to provide the input character. By clicking the “Analysis” button, the AFA tool is invoked to analyze the input character. If there are any errors, feedback will be provided by highlighting the erroneous parts by red circles or shading the erroneous strokes with different colors. The flow of the proposed AFA algorithm is shown in Fig. 1(b). The online demo of our handwriting education system is available from the following link: http://vache.cs.cityu.edu.hk/ccls/.

K.-T. Tang and H. Leung / Teaching Chinese Handwriting by Automatic Feedback

Example Box

109

Exercise Box

Click “analysis” button for Automatic Feedback and Analysis (a) The user interface

(b) Flowchart of our proposed algorithm Fig. 1. The system description.

1.1 Data Collection The online handwriting data is collected by using a tablet. In our system, two kinds of characters will be considered: the template and sample characters. The template characters are skillful handwritings given by a teacher and they contain information about the correct stroke sequence and stroke production. The sample characters are student input handwritings that might contain various kinds of handwriting errors. 1.2 Stroke Correspondence As the first step of the proposed method, we find the stroke correspondence that represents a matching between the strokes in the sample and template characters. The matching cost between each stroke from the sample character and each stroke from the template character needs to be computed forming a cost matrix. The stroke correspondence can then be determined from the cost matrix. 1.2.1 Stroke matching costs The stroke matching cost is defined between a stroke in the sample character and a stroke in the template character. Both of these strokes are resampled to contain n data points. Assume that the data points of the sample stroke are denoted by ( xiS , yiS ) and the data points of the

template strokes are denoted by ( xiT , yiT ) , where i = 1,2,…,n. For the stroke matching cost, we consider the Euclidean distance (distance cost) and the direction difference (direction cost) between these data points. These two component costs are then combined to form the final stroke matching cost. (a) Distance Cost The distance cost is the average Euclidean distance between every data points on a sample-template stroke pair. There are two possible writing direction combinations for the stroke pair. The Euclidean distances for the stroke pair for these two cases are computed,

xiS xiT yiS yiT and dic xiS xnTi1 yiS ynTi 1 . The distance cost i.e., d i CDist is defined as the minimum between these two values as shown in equation (1): 2

2

2

2

110

K.-T. Tang and H. Leung / Teaching Chinese Handwriting by Automatic Feedback

C Dist

1 n · §1 n min¨ ¦ d i , ¦ d ic ¸ ni1 ¹ ©n i 1

(1)

(b) Direction Cost The direction cost is the average sine values of the angle difference between every stroke segments on a sample-template stroke pair. The angle of a sample segment can be computed § yS yS · as T iS arctan¨¨ iS1 iS ¸¸ and the angle of a template segment can be computed © xi 1 xi ¹ § yT yT · arctan¨¨ iT1 Ti ¸¸ , where i = 1,2,…,n-1. Taking account of the two writing direction © xi 1 xi ¹ combinations, the direction cost CDir is defined in equation (2):

as T iT

C Dir

1 n 1 § 1 n 1 · min¨ sin T iS T iT , sin T iS T nTi ¸ ¦ ¦ n n 1 1 i 1 i 1 © ¹

(2)

(c) Combining the costs We combine the component costs CDist and CDir by a linear weighting method, and their weights wDist and wDir are determined based on the statistical analysis of the cost distributions for the matched and non-matched stroke pairs. Matched stroke pairs are the corresponding strokes between two handwriting samples of the same character and we obtain the handwriting samples by asking several users to write the same set of characters. Non-matched strokes pairs are those strokes that are not corresponding strokes between the two handwriting samples. By plotting the FMR (false matched rate) and FNMR (false non-matched rate) curves as shown in Fig. 2(a) and Fig. 2(b) from the cost distributions, we can determine how well each component cost can distinguish between matched and non-matched pairs through their equal error rates (EER) where FMR and FMNR converge to have minimum equal error. The lower the EER, the better is the component cost. The EER Dir EER Dist weights are computed as wDist and wDir EER Dist EER Dir EER Dist EER Dir respectively, and the EERs in the weights are used to boost the better component cost and suppress the poorer one. The combined cost is shown in equation (3): CCombined wDist C Dist wDir C Dir (3)

Fig. 2. Error rates at different stroke matching costs.

1.2.2 Stroke Matching The stroke matching cost defined in the previous section is computed between each stroke in the sample character and each stroke in the template character. A cost matrix is then formed.

111

K.-T. Tang and H. Leung / Teaching Chinese Handwriting by Automatic Feedback

From the cost matrix, the stroke correspondence can be determined by minimizing the total cost of the matched stroke pairs using the Hungarian method [15]. 1.3 Problematic strokes Problematic strokes are strokes that are suspected to contain stroke production errors and they may appear in sample and/or template characters. Fig. 3 illustrates some samples that contain production error. When the numbers of stroke of sample and template characters are different, it is trivial that the additional strokes are problematic. Examples in these cases are shown in Fig. 3(a) and Fig. 3(b). On the other hand, it is more challenging to determine problematic strokes from matched strokes as shown in Fig. 3(c). It can be done by applying the result in Fig. 2(c) for classifying between matched and non-matched stroke pair. Non-matched strokes are likely to be problematic since those strokes are not similar in geometry as shown in Fig. 3(c), but they can still be paired up if no better pairs can be formed. Hence we define a threshold matching cost T that can clearly distinguish matched (normal) and non-matched (problematic) stroke pairs. If the threshold is defined to be too low, then many strokes may be wrongly labeled as problematic strokes. If the threshold is defined to be too high, then many problematic strokes may not be identified. From the result in Fig. 2(c), we choose the matching cost with EER as our threshold T (0.65 in our case). Both the template and sample strokes in any paired strokes with matching cost higher than T are problematic. Template Sample

1

2

3

--

A

B

C

D

Template Sample

1

2

3

A

B

--

Cost

0.4

0.5

0.3

Problematic?

No

No

No

Template Sample

1

2

3

A

B

--

--

Cost

0.4

0.5

Yes

Problematic?

No

No

--

Cost

0.4

0.8

--

Yes

Problematic?

No

Yes

Yes

Fig. 3. The problematic strokes are determined by stroke correspondence and stroke matching cost.

1.4 Character instance creation Some problematic strokes may contain production errors while others may not. There may be many problematic strokes and we have to investigate which problematic strokes in fact lead to stroke production error. To solve this problem, new character instances are derived from either the sample or template characters for representing a particular case of stroke production error. For creating the character instances, we introduce an operation called “Miss and merge”. “Miss” operation means taking away particular problematic strokes from the character, while “Merge” operation means merging two or more problematic strokes together. With these operations, two sets of character instances are formed: one set derived from the sample character and another set derived from the template character. The potential type of production error represented by a character instance can be inferred by the operation used to form that character instance. The explicit meaning of the character instances produced by each kind of stroke operations is summarized in Table 1. Derived character instances are given beside each operation applied on the problematic strokes (highlighted). The template strokes are labeled as 1, 2, 3, etc., while the sample strokes are labeled as A, B, C, etc.

112

K.-T. Tang and H. Leung / Teaching Chinese Handwriting by Automatic Feedback

Table 1. Different meanings of character instances of sample and template.

1.5 Character-level matching We perform a character-level matching that identifies the most similar sample-template character instance pair. The character matching cost is calculated from the average stroke matching cost of the matched strokes. The operations used to form the matched pair indicate the type of production error in the handwriting. We have introduced some measures to speed up our algorithm. The “Merge” operation is only applied to merge neighboring problematic strokes to form new character instances since the broken stroke errors often happen in a turning point of a stroke. In the character-level matching phase, only the character instances with the same number of strokes are matched thus filtering out many character instance combinations. The average time needed for creating a character instance and matching a pair of character instances are 0.5ms and 18ms respectively. On average, we need about 66ms to determine whether a character has stroke production errors. It shows our proposed method is quite efficient. 1.6 Automatic Feedback on handwriting errors

Fig. 4. Automatic feedbacks on handwriting errors.

After analyzing the input handwriting, a feedback will be given to the student to state whether his/her handwriting has handwriting errors and where the errors are. Fig. 4(a) illustrated a sample feedback when both stroke sequence error and stroke production error are committed by the student. In this case, the system first notifies the user what and where the production errors are. For production errors, missing and extra strokes are highlighted and broken or concatenated strokes are located by a small red circle. After the user has corrected the production error, the system can animate the stroke sequence in groups of correct sub-sequences by a minimum feedback method [5] as shown in Fig. 4(b).

K.-T. Tang and H. Leung / Teaching Chinese Handwriting by Automatic Feedback

113

2. Experiments and results We invited 65 public users to write some Chinese characters shown in Fig. 5(a) and a total of 920 handwriting characters are collected. One-tenth (92 samples) is used in training the stroke matching cost as mentioned in section 1.2.1. The rest (828 samples) is used as test data in our experiment. Fig. 5(b) shows the handwritings written by different people. Although variation is large, our system is still robust enough to check handwriting errors.

(a) Characters used in the dataset (b) Handwriting variations Fig. 5. Characters used in the dataset and sample handwritings.

Fig. 6 shows the types and counts of the handwriting errors in our testing data. To evaluate the performance of our system, we consider the Type I and Type II errors of each handwriting error. For example, if a sample character contains stroke production error only (Region 3 in Fig. 6) but the system wrongly identifies it as stroke sequence error (Region 2, not Region 3), it results in “Type I stroke production error” and “Type II stroke sequence error”. Experimental result in Table 2 shows that our proposed method can identify nearly all stroke sequence errors (0.74%). The error rates for identifying stroke production errors are also low. Relatively more correct samples (i.e. without error) are wrongly identified as stroke production error because some input strokes are badly written (e.g. too tilt) and the system often considers them as problematic strokes. Fig. 6. The types and counts of handwriting errors in our dataset.

Table 2. Performance evaluation of our method. Types of handwriting error Stroke sequence error Stroke production errors Stroke production errors with sequence error

Type I Type II Type I Type II

Error rate (%) of our Proposed Method

Type I

0.74% 0.58% 1.68% 2.62% 0.49%

Type II

0.48%

In addition, we have performed some user studies to evaluate the effectiveness of our proposed system for handwriting education. Sixteen university students are being invited as users and we randomly divide them into “Test” and “Control” groups. We have prepared an exercise with five Chinese characters “ᓳ”, “ᰗ”, “⬋”, “႑” and “㟙” which are often written incorrectly. The users are required to write each of these characters until the characters are written correctly. In the mean time, they can click the “Analysis” button for advice. The users in different group receive different kinds of feedback. In the “Test” group, the users can receive full feedback provided by our proposed method. In the control group, the users can just receive either “correct” or “wrong” with no extra guidelines.

Fig. 7. Learning performance evaluation.

The number of clicks of “Analysis” button and the time elapsed for all characters to be written correctly is recorded. Before the experiment, we record the time taken for normal writing and it is used for normalization to account for the writing speed difference among users. Fig. 7 shows the evaluation result. It is quite contrasting that the test group using our propose system can learn faster (shorter normalized exercise time) and they require less feedback from the system (fewer clicks).

114

K.-T. Tang and H. Leung / Teaching Chinese Handwriting by Automatic Feedback

3. Conclusion and Future work We proposed a new method that can check stroke sequence errors and stroke production errors simultaneously. In our proposed method, users can write freely without the writing sequence restriction as imposed by existing methods. Using the problematic strokes, we introduced miss-and-merge stroke operations to create character instances to identify potential production errors. Experimental results showed that our proposed method can check handwriting errors with an encouraging speed and accuracy. The learning performance of the students has been evaluated and it is found that they learn faster in Chinese handwriting using our system. As future work, we will study other kinds of handwriting errors such as the stroke reversal problem and develop methods to evaluate the beautifulness of the student handwritings automatically. Acknowledgments The work described in this paper was partially supported by a grant from City University of Hong Kong (Project No. 9360092). References [1] Takeshi Shimomura, “Informatics: input and output: Science of the stroke sequence of Kanji”, Proc. of the 8th Intl. Conf. on Computational Linguistics, pp. 270–273, 1980. [2] Nancy Law, W.W. Ki, A.L.S. Chung, P.Y. Ko and H.C. Lam. “Children’s stroke sequence errors in writing Chinese characters”, Reading and Writing: An Interdisciplinary Journal, Kluwer Academic Publishers, 10: 267-292, 1998. [3] Jianguo Li and Xiaozhen Zhang, “The design and implementation of multimedia intelligent tutoring system for Chinese characters”, IEEE Proc. of the 1st Intl. Conf. on Multi-Media Engineering Education, pp.459–463, July 1994. [4] Chun-Hung Tzeng, Leon Hsu, Chien-Ping Chen and C. Uema, “A multimedia project in teaching Chinese and Japanese at Ball State University”, IEEE Intl. Conf. on Multi Media Engineering Education, pp.445 – 452, July 1996. [5] Kerry Tsang and Howard Leung, “Teaching Stroke Order for Chinese Characters by Using Minimal Feedback”, Intl. Conf. on Web-based Learning (ICWL 2005), Hong Kong, August 2005. [6] Chwee Keng Tan, “An algorithm for online strokes verification of Chinese characters using discrete features”, 8th Intl. Workshop on Frontiers in Handwriting Recognition, pp.339 – 344, 2002. [7] C.L. Teo, E. Burdet, H.P. Lim, “A robotic teacher of Chinese handwriting”, HAPTICS 2002, pp.335 – 341, 2002. [8] Caroline Coit, “Peer review in an online college writing course”, IEEE Intl. Conf. on Advanced Learning Technologies, pp.902 - 903, 2004. [9] Carl Frélicot, Céline Rémi and Pierre Courtellemont, “School Level Recognition from Children's Drawings and Writings”, Intl. Conf. on Pattern Recognition 2002, pp.493-496, 2002. [10]Y. Tonouchi and A. Kawamura, “An On-Line Japanese Character Recognition Method Using Length-Based Stroke Correspondence Algorithm,” Proc. of the 4th Intl. Conf. on Analysis and Recognition, Vol.2, pp.633-636, 1997. [11]Y.T. Tsay and W.H. Tsai, “Attributed String Matching by Split-and-Merge for On-line Chinese Character Recognition,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.5, No.2, pp.180-185, 1993. [12]W. H. Tsai and S. S. Yu, “Attributed string matching with merging for shape recognition,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.7, pp.453-462, July 1985. [13]T. Wakahara, A. Suzuki, N. Nakajima, S. Miyahara and K. Odaka, “On-line cursive Kanji character recognition as stroke correspondence problem”, ICDAR 1995, pp.1059-1064, 1995. [14]Toru Wakahara and Kazumi Odaka, "On-Line Cursive Kanji Character Recognition Using Stroke-Based Affine Transformation", IEEE Trans. on Pattern Analysis And Machine Intelligence, vol. 19, no. 12, December 1997. [15]R.R. Burkard and E. Cela, “Linear Assignment Problems and Extensions”, P.M. Pardalos and D.-Z. Du, editors, Handbooks of Combinatorial Optimization, Dordreck: Kluwer Academic Publishers, pp. 75-149, 1999.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

115

Developing a Practical Domain Knowledge Base and Problem Solving System for Intelligent Educational System of High School Chemistry Nana Ishima, Toru Ueda, Tatsuhiro Konishi, Yukihiro Itoh Faculty of Information, Shizuoka University, Japan E-mail: {cs1055, gs06008}@s.inf.shizuoka.ac.jp {konishi, itoh}@inf.shizuoka.ac.jp Abstract: In this paper, we describe the development of our intelligent educational system for high school chemistry. Our goal is to put our system in practical use. To achieve our goal, the system must be usable for at least half a year or more. So we focus on the scale of a domain knowledge base (DKB) and a problem data base. We examine the framework of the chemical world model, and describe problem representation and knowledge representation. We tried to expand the DKB and the problem data base. We include an example of problem solving process using the DKB. Keywords: Intelligent Educational System, Domain knowledge representation, Problem solving, practical educational system

Introduction Intelligent Educational Systems (IES) have been developed in recent years; few of these are yet sufficiently developed for practical classroom use. Most of the proposed systems are prototypes that focus on specific techniques or theories. We aim to develop an IES supporting exercise for practical use. There are many Conventional Educational Systems (CES) supporting study in the educational field. For example, most Learning Management Systems (LMS) have a simple testing environment [1][2]. Teachers and learners can look over results item by item and apply it to the next learning. However, CES can only accumulate a learning history arranged according to problem. On the other hand, an IES can accumulate learning history, not only by problem, but also by knowledge, because the IES automatically understands the relations between the domain knowledge and problems by solving problems by itself. Therefore, if our system is introduced in the educational fields, teachers and learners can better diagnose issues. In the longer term the system is used, such advantage of IES becomes the more important. In our previous research, we have been developing the IES for high school chemistry [3][4]. In this paper, we reconstruct the IES supporting exercise in order to make our system more practical. To put the system in practical use, the system must be usable for long-term. In order to make it possible, we must finish the following tasks: expand a Domain Knowledge Base (DKB) and problem database; develop a problem solving system based on the DKB, an assessment system of learner’s answers, an explanation generating system, and a management system of learning history. In this paper, we discuss a framework of a chemical world model, and describe the problem representation and the knowledge representation. We also explain the problem solving process using the DKB.

116

N. Ishima et al. / Developing a Practical Domain Knowledge Base and Problem Solving System

1. Chemical problems the system deals with Based on discussions with high-school teachers, we determined that the minimum scale of DKB must cover all chapters of inorganic chemistry in the Japanese standard textbooks for first grade of high-school. Our current plan is to construct a DKB covering the following chapters: “Structures of materials and combination of particles”, “Variation of materials” and “Characters of materials (inorganic)”.Our system deals with the following five types of problems; they cover about 85% of problems in chapter 1 to 3 of standard workbook [6]. (1) Simulate a chemical phenomenon; a part of simulated result is the answer. Ex. What is the name of the salt formed, when neutralization of HCl and NaOH takes place? (2) Set up a material attribute value using numerical relation knowledge. Ex. Find the mol of 3.2g CH4? (3) Find the chemical equation’s coefficient. Ex. aH2 + bO2 ψ cH2O (4) Combine (1) and (2) or combine (1) and (3). Ex. How much caustic soda will be needed to neutralize 100ml of 2.0mol/l acetic acid? (5) The answer is an item in DKB. Ex. What is hydrogen’s RMA? 2. Knowledge Representation and Problem Representation At first, we describe the basic framework of the chemical world model. In general, we regard physical phenomena as continuous. However in high school chemistry, it is relatively rare to recognize chemical reactions as continuous phenomena, but we usually isolate discrete ones. Indeed, a chemical reaction formula represents only materials existing before a reaction and after it, without the details of changing state. As a result, we adopt the following framework to represent the chemical world model: the state of the world is represented by “static states”, “dynamic states” and their “causal relations”. The “static state” means a state in which all chemical materials are stable, such as before or after a chemical reaction. The “dynamic state” means a state in which some change is happening. The “causal relation” means a relation that triggers a change from a static state to a dynamic state or the reverse. We design our knowledge representation based on the framework. The DKB is composed of general knowledge of chemistry which is used when the system solves the five problem types. Table1 shows the classification of general knowledge of chemistry, and explanations of each knowledge. We tried to establish DKB on knowledge representation, but we found a technical problem. It is that a concept cannot be written uniquely, but sometimes variously. We propose a resolution for the problem using labels. Some attributes of a concept can have multiple values. Each value has a label indicating the situation in which the value should be selected. We explain the further details Fig 1: Representation of material conception knowledge , about the issue and resolution in [7]. attribute conception knowledge and numerical relation knowledge Next we discuss problem representation. As mentioned in section 1, our system deals with five problem types. Though problem solving algorithms are different in each type, a framework of problem representation is desirable to be coherent. Thus we defined a syntax of problem representationwhich does not depend on problem types. We call chemical problems composed by the syntax “query”. We explain the further details Fig 2: Representation of chemical phenomenon knowledge, about problem representation in [7]. Material conception know ledge

Numerical relation know ledge ID=6000

formula=>

ID=100

name=hydrogen molecule

dens ity=m ass /volum e

Chemical formula=H2

Attribute conception know ledge

MW =2

Density=ID3000

ID=3000

V olume=ID3001

name=Dens ity

Mass=ID3002

Numerical relation=ID6000

dominant conception =m olecule

Chemical phenomenon kn ow ledge

Chemical state know ledge ID=30000

ID=10000

name=neutralization

Contents =ID101(acid) ,ID102(bas e)

condition=ID30000

Process=ID30000,ID30001 ID30002

Chemical state know ledge

Numerical relation=ID4000

ID=30001

Neutralize(ID101,ID102)

Numerical relation know ledge ID=4000

Generate(ID103(water)

formula=acid_M㬍acid_Volum e 㬍acid_Valency=base_M㬍bas e_ Volum e㬍bas e_Valency

Cons ume(ID101,ID102)

ID104(salt))

chemical state knowledge and numerical relation knowledge

N. Ishima et al. / Developing a Practical Domain Knowledge Base and Problem Solving System

117

Table1. Knowledge representation [Knowledge on material] ˉMaterial conception knowledge (represents individual attribute information of a material and conception class to which the material belongs˅ ˉ name slot [The name of the material ˉ attribute slot [*1] ˉ dominant conception slot [An ID of material class to which the material belongs (superclass)] ˉ lower conception slot [An ID of subclass of the material] ˉ construction slot[Constituent particles and the construction of a material] ˉAttribute conception knowledge (has pointers to numerical relation knowledge which include the attribute’s definitional equation.) ˉ name slot [The name of attribute] ˉ numerical relation slot [IDs of numerical relation knowledge representing relative equations with the attribute] ˉ definitional equation [A definitional equation (For example, a density is defined as mass per unit volume, and the formula is “density= mass/volume“.)] [Knowledge on phenomenon] ˉChemical phenomenon knowledge (has causal connection between a condition in which the dynamic phenomenon occurs and process of the dynamic phenomenon, and works as a production rule; initial condition slot is the condition part, and process slot is the action part of the rule.) ˉname slot [The name of phenomenon] ˉinitial condition slot [An ID of chemical state knowledge] ˉprocess slot [IDs of chemical state knowledge of initial state and of changing state] ˉnumerical relation [An ID of numerical relation knowledge which is applicable under the dynamic phenomenon] ˉChemical state knowledge (represents a state in the chemical world model. It indicates existing materials, changes in the chemical world model.) [Numerical relation knowledge] ˉType1. Applicable under a particular phenomenon ˉType2. Applicable for an attribute of a particular material class ˉType3. Applicable for an attribute of any material class (Each type of numerical relation knowledge is linked from chemical phenomenon knowledge, material conception knowledge, and attribute conception knowledge respectively.) *1 Attributes slot is classified into two types. One is that the attribute value is decided uniquely when deciding material name and attribute name (ex. RMA, atomic number).In this case, the attribute value is written in the slot.The other is that the attribute value is not decided uniquely when deciding material name and attribute name (ex. volume, mole, mass).In this case, an ID number of attribute conception knowledge is written.

3. Problem Solving System (PSS) The PSS has the following five units: Control routine, Simulator, Query processing routine, Relation determination routine, and Equation routine. The system has also DKB [Fig 3]. We explain the details of PSS behavior in [7].The problems of type 3 in Table 1 can be solved in the following way(Fig.4) : (1)The system checks information in the problem representation called “query” (Volume of CH4 is asked, and weight of CH4 is 3.2g). (2)The system searches a useful equation to specify asked value, and sets terms in the equation as subgoals (The equation 1 may be useful to specify the volume. Substance quantity and Mole volume become a subgoal, but the latter is achieved immediately because the value is found in the DKB). (3)The system tries to solve all the subgoals in the same way as (2) recursively (Equation 2(Fig.4) may be useful to specify the Substance quantity. Weight and MW become subgoals and achieved soon, as both of them are found in the DKB or the query). (4)If all of the subgoals are achieved, the system calculates the asked value by the equations (the system specifies the substance quantity of CH4 by Equation 2, and the volume of CH4 by Equation 1). Type 3 problems include Type 1 and Type 2. As for Type 4, we must add a routine for

118

N. Ishima et al. / Developing a Practical Domain Knowledge Base and Problem Solving System

solving for an undetermined coefficient to Equation routine. As for Type 5, the system does not have to infer because answers are written in DKB directly. The system records problem solving process including used knowledge and the order of applying the knowledge. We can use this record for making detailed learning history. And we also design Explanatory Text Generator which converts problem solving process to explanation.

Fig 3: Overview of Problem Solving System

4. Implementation We have implemented DKB using XML. We analyzed a textbook [5] and workbook [6], and counted the items of knowledge. The number of Material conception knowledge is 176, the number of Attribute concepts is 30, the number of Chemical phenomena is 100, the number of Chemical states is 160, and the number of Numerical relations is 72. We have already implemented half of them. We also have Fig 4: An example of solving a problem by our system implemented most of PSS and designing Simulator using Java (Fig 4). Now our system can solve 70 problems of Type 2, 4, 5. 5. Conclusion We have been developing the IES which can be used long-term, focusing on practical use. If our system is completed, we think the system can deal with learning chemistry from half to one year. This is conducive to practical use of IES. We prepare a DKB and the basic problem data base of a comparable scale with a standard workbook in advance. However, teachers in high school would want to edit problem data based on their experiences. In order to satisfy their requests, we will have to design authoring tools which help teachers edit problem data base by themselves in the future. As the future tasks of this study, we will finish developing PSS and also developing an interactive system which dialogues with learners on a problem solving process. References [1] http://moodle.org/ [2] http://www.xu.edu/its/elearning/blackboard.cfm [3] I.Takahashi, T.Konishi, Y.Itoh:"On a Framework of an Educational System which Acts both as ITS and Micro-World", Proc. of Int̉l Conf. On Comp. in Edu. ,Vol.1,pp.856-859. (1999) . [4] T.Konishi and Y.Itoh : "Domain World Models Represented from Variable Viewpoints for ICAI Systems of High-school Chemistry", Lecture Notes in Artificial Intelligence 1114 (PRICAI'96: Topics in Artificial Intelligence), pp.495-509, Springer-Verlag (1996). [5]“Koutou gakkou KagakuΣ”, Suuken Syuppan (2002) (in Japanese) [6] “Kihon mondaishu Kagaku IB”, Shinkou shuppansha Keirinkan (in Japanese) [7] N.Ishima, T.Ueda, T.Konishi, Y.Itoh: ”Developing Problem Representation Knowledge Representation and Problem Solving System for Intelligent Educational System of High School Chemistry” , The workshop 5 at ICCE 2006: Problem-Authoring, -Generation and -Posing in a Computer-Based Learning Environment, 2006(to appear).

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

119

The COLAC Model: Collaborative PaperWriting in the Humanities Guillaume Schiltz, Andreas Langlotz Department of English, University of Basel, Switzerland [email protected] Abstract: In this paper we introduce a pedagogical model that trains writing skills in a blended learning environment (online/in-class). The model has been successfully implemented in several university courses. The results of a formative evaluation support the effectiveness of the model but also point to problems related to the unfamiliar collaborative learning methods. Keywords: pedagogical models, collaborative writing

Introduction At the moment all European universities are going through a fundamental change of their curricular system; this is probably the most far-reaching structural reform in recent times. Termed as the Bologna Process [1], the reform mainly aims to bring degrees into line with each other and to make higher education more transparent as well as uniform. Besides the changes at the structural level, the Bologna Process also aims at pedagogical changes which include a clear shift from teacher-centered to learner-centered methods and the promotion of essential soft skills, such as social and media competence. These secondary aims support the integration of e-learning scenarios, even in fields and subjects that have so far remained fairly reluctant to technical innovation, such as the humanities. Writing academic research papers, i.e. articulating personal thoughts and arguing according to academic principles is one of the most important operational skills promoted in the humanities. Traditionally, the process of ‘how to write an academic paper’ has mainly been imparted by providing students with the necessary tools (reference lists, research inquiries) and by teaching the formal layout of a paper (structure, style sheet). However, writing itself should be seen as a joint communicative process: thoughts and ideas have to be transmitted from the author to the reader. This transmission is subject to principles of audience design and has to follow the basic rules of the genre ‘academic paper’. At university level these communicative aspects of paper writing often are only conveyed implicitly in the coaching between the student and the instructor. Therefore, the emergent ability of writing academic papers mostly results from the continuous practical experience of individual students and the corresponding individual communication with the instructors. In the past a great number of studies have focused on the cognitive and practical principles that guide students in writing papers (e.g. [2]). The results of these studies were even transferred to technical environments [3] to support the learning of writing. A pedagogical model, however, that teaches the more communicative aspects of academic paper writing in a general way and that is easy to implement has been missing so far.

120

G. Schiltz and A. Langlotz / The COLAC Model: Collaborative Paper-Writing in the Humanities

The COLAC model (Colac referring to a sleepy town in Australia) introduced here aims to link traditional classroom-teaching methods with the advantages of computermediated communication. Thus, it is an attempt to implement the basic pedagogical goals of the Bologna Process. The model is based on the following pedagogical approaches: x It is designed as a blended learning component in a location-based teaching environment according to Kerres’ 3C-model [4]. x It implements the three major elements of cooperative learning: social interdependence, positive relationship and promotive interaction [5]. x It relies on the activity theory within communities of practice [6].

1. The COLAC Model The basic setting of the COLAC model (fig. 1) simulates the process of an academic symposium, i.e. sharing knowledge among a peer community, discussing one’s research results and thus building up a reputation of expertise. Often, experts work together in teams and thus the model also relies on group work.

Figure 1: the COLAC model At the beginning, groups have to be formed with every participant being the member of one group. Each group has to choose one topic from a list of basic research questions. In the model each group performs two distinct roles. First, it acts as an author group by working out a paper on the chosen topic. Second, it has to review the paper of another group and present this paper in a simulated conference in the classroom. The peer-review is the fundamental part of the model and consists of a thorough discussion between the author group and the review group. Based on the outcome of this discussion the author group has to refine its paper continually and must eradicate all

G. Schiltz and A. Langlotz / The COLAC Model: Collaborative Paper-Writing in the Humanities

121

uncertainties, flaws, and weaknesses. Both writing and reviewing are realized online, preferably with the support of asynchronous communication technology. After the review is finished, the revised paper is made available to all course participants. The students should have enough time to read the paper thoroughly and to prepare questions for the presentation. The in-class paper presentation is split between a short oral summary of the paper and a subsequent discussion of the paper topic. During the presentation the authors are not involved but take notice about how their paper is received. At the end the review group is assessed depending on the expertise its members manifested during the presentation. The instructor also adopts a dual role in the overall model. On the one hand, he/she has to provide the organizational framework (time management, course structure) in a clear and comprehensive way. On the other hand the instructor has to act as an expert and as a coach by supplying help whenever specialist or social support is needed. Fig. 1 depicts the model in detail. It is divided into two distinct phases (paper writing, paper presentation), including four action groups (author group, review group, course participants and instructor), each one performing different actions (write, revise, summarize, discuss, read, ask, assess). Although the formal goal of the model is defined by writing and presenting an academic research paper, the actions of each group involved include a number of secondary learning aims such as teamwork organization, discursive argumentation and conflict resolution. Computer-mediated collaboration is essential for all actions during the paper writing phase, as well as within the action groups (authors have to elaborate the paper, reviewers have to organize the review), as between the action groups (authors and reviewers have to discuss the paper). Furthermore, the computer is used as a central repository for the paper and enables collaborative writing and revising facilities through wiki or similar technologies.

2. Actual Implementation and Outlook A pilot implementation with 44 undergraduate students was successfully conducted as part of an e-learning course in 2004 [7]. Since then we have been improving the model and transferred it to a Moodle platform. Now in 2006 we are running the model again with a total of 68 students in a joint course between two universities. Some of the action groups being geographically remote, the presentation phase this time will be partly realized as a joint video conference. Since 2006 the model is part of the national e-learning initiative Swiss Virtual Campus (http://www.virtualcampus.ch/) and it is planned that starting from 2007 it will be implemented simultaneously at three different universities in Switzerland.

3. Evaluation Results As a result of the formative evaluation carried out after the course, the great majority of students claimed that they had achieved the overall course aims (fig. 2). This self-appraisal can be supported by the instructor’s assessment (quality of the papers, active participation, final test). When asked for more differentiation, again the great majority of students answered that they judged the paper review as informative (‘I learned a lot’). The results for motivation (‘I liked it’) and for appropriateness (‘It should be adopted by other courses’), however, significantly tend to a less positive judging. Informal interviews gave evidence that the main cause of this divergence is due to the fact that all students were unfamiliar with collaborative learning. While working in the model they had to revise and

122

G. Schiltz and A. Langlotz / The COLAC Model: Collaborative Paper-Writing in the Humanities

adapt their learning strategies, which caused them to feel insecure and hesitant about the benefits of such an approach. In the meantime, however, we are observing that collaborative learning skills are becoming more developed among students. acquisition of course aims (student's judgment)

teamwork competence

1

e-learning skills

factual knowledge

20%

11

30

2

25

3

14

25

0%

20%

1

2

15

40%

60%

80%

100%

students poor

average

good

excellent

Figure 2: student’s judgment on the course aims rating of the paper review

inform.

motivat.

approp.

50%

6

19

19

17

25%

11

26

19

0%

25%

5

7

50%

75%

100%

students negative

average

positive

Figure 3: student’s rating of the paper review

References [1] Reichert, S., and Tauch, C. (2005) Trends IV: European Universities Implementing Bologna. EUA Report, Brussels. (http://www.eua.be/eua/jsp/en/upload/TrendsIV_FINAL.1117012084971.pdf) [last visited on May 20th, 2006]. [2] Flower, L. (1988) The Construction of Purpose in Writing and Reading. College English, 50, 5, 528550. [3] Neuwirth, C.M., and Wojahn, P. (1996) Learning to write: Computer support for a cooperative process. In: Koschmann, T. (Ed.) CSCL: Theory and practice of an emerging paradigm. Lawrence Erlbaum, Mahwah, NJ, 147–170. [4] Kerres, M., and de Witt, C. (2003) A didactical framework for the design of blended learning arrangements. Journal of Educational Media, 28, 101-114. [5] Johnson, D.W., Johnson, R., and Holubec, E. (2002) Circles Of Learning: Cooperation in the classroom, (5th Edition). Interaction Book Company, Edina, MN. [6] Hewitt, J. (2004) An exploration of community in a Knowledge Forum classroom: An activity system analysis. In: Barab, S., Kling, R., and Gray, J. (Eds.) Designing virtual communities in the service of learning. Cambridge University Press, Cambridge, MA, 210-238. [7] Schiltz, G., and Langlotz, A. (2004) eHistling - Integrating Communication and Cooperation in the Humanities. In: McKay, E. (Ed.) International Conference on Computers in Education 2004. Book of Abstracts. Common Ground, Altona, VIC, 149 (full presentation on CD-ROM).

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

123

Experimental Investigation and Implementation of Support for Problem Generation by Presenting Cases Kazuaki Kojima and Kazuhisa Miwa Graduate School of Information Science, Nagoya University, Japan [email protected]

Abstract: The usefulness of cases in creative generation has been widely discussed and well recognized. However, it has been also experimentally indicated that introducing cases can limit human creative generation. In this study, we investigate the effects of case presentation in the task domain of generating mathematical word problems. Mathematical word problems have two essential attributes: surface problem situations and the mathematical structures of solutions, the control of which is recognized as an important issue in learning mathematics. In response, we present problems as cases to participants while controlling those attributes, with the results indicating that such presentation actually provides different constraining effects based on the control to their problem generation. Based on the results, we propose a support system for generating word problems, which presents problems as cases by controlling similarities. Keywords: Creative generation, case presentation, word problems, problem generation

Introduction People often utilize past experiences or existing examples as cases in creative generation. It has been well explored that cases are useful during many processes of creative generation (e.g., see [1]). The presentation of cases to support the creative generation has been proposed and implemented in a variety of tasks. However, introducing cases can also limit human creative generation. It has been experimentally indicated that human creative generation tends to conform to examples given prior to starting the generation tasks [7]. Such a constraining effect by examples is somewhat undesirable, because the essential factor in creative generation is the production of novel output. On the other hand, providing appropriate constraints on creative generation is important to support concept formation and enhance creativity [4,5]; therefore is not necessarily wise to remove all constraints. Therefore, it is necessary to investigate the relation between cases and their constraining effects in creative generation in order to implement creative generation support by presenting cases. In this study, we experimentally investigate the effects of case presentation in the task domain of generating mathematical word problems. Then we implement a computer-based system that supports problem generation by humans by presenting problems as cases. Mathematical word problems have two attributes that are essential in problem solving; surface problem situations denoting contextual settings expressed in texts such as "purchase of goods," and the mathematical structures of solutions. In the context of mathematics education, it has been recognized as an important issue to present students with various problems while controlling the similarities [2]. Moreover, in the context of problem-posing education, it has been pointed out as important but also difficult to generate a broader range of problems from

124

K. Kojima and K. Miwa / Experimental Investigation and Implementation of Support

formal symbolism and informal situations [3]. According to the background, it is also beneficial in the learning of mathematics to investigate and implement support for creative generation by presenting problems as cases.

1. Experimental Investigation In our investigation, participants were asked to generate mathematical word problems. In the task of problem generation, they were presented problems as cases that were produced by controlling similarities in problem situations and solutions. We verified the effects of case presentation based on the relation between participants' problems and the presented cases.

1.1 Method One hundred and forty-five undergraduates from a class on psychology for design participated in our investigation. The experimental procedures were as follows. 1.

2. 3.

Generating a problem (first) Every participant was presented with an example of a word problem and asked to generate a new problem from the example. Presenting cases The participants were presented with three problems as cases. Generating a problem (second) Every participant was asked to generate another problem from the example. In the second round of problem generation, the participants could refer to the problems presented in Procedure 2.

Each participant was randomly assigned to one of six experimental groups. Every group was given a different set of problems as cases in Procedure 2. The groups and the set of problems for each group were as follows. II1: problems whose situations and solutions were identical to the example II2: problems whose situations and solutions were identical to the example, but some of parameters in the texts were different from the example problem DI: problems whose solutions were identical to and whose situations were different from the example ID: problems whose situations were identical to and whose solutions were different from the example DD: problems whose situations and solutions were different from the example Control Group: geometry problems that had no attributes similar to the example identical

Solutions

different

Ex Problem situations

A

C

B

D

different

Figure 1. Categories for evaluating the participants' problems

Problems generated by the participants were evaluated according to categories. The categories were specified based on similarities in problem situations and solutions between the example problem and the problems generated by the participants. The categories were as follows. A: problems whose situations and solutions were identical to the example B: problems whose solutions were identical to and whose situations were different from the example

K. Kojima and K. Miwa / Experimental Investigation and Implementation of Support

125

C: problems whose situations were identical to and whose solutions were different from the example D: problems whose situations and solutions were different from the example

Figure 1 shows the categories for evaluating the participants' problems.

1.2 Results In the first round of problem generation, the proportions of participants who generated problems of category-A, B, C, and D were 57.1%, 21.4%, 16.1%, and 5.4%. Thus, these results indicate that almost all participants generated problems whose situations and solutions were identical to the example problem when they had not seen any cases.

Figure 2. Proportions of participants who generated problems in each category (second round)

Figure 2 shows the proportions of participants who generated problems in each category in the second round of problem generation where the participants had already examined cases. Similar to the first round, more than half generated category-A problems in Groups II1, II2 and the Control Group in the second round. On the other hand, the proportion of category-B problems increased in Group DI, that of C in Group DI, and that of D in Group DD in the second round. These results indicate that case presentation influenced problem generation, and that the presentation of different cases had different effects on problem generation.

1.3 Discussion The results described above confirmed that presentation of different cases had different effects on problem generation. In Groups DI, ID, and DD, the similarities between the example problem and problems, increasing in the second round of generation, were consistent with those between the example problem and the presented cases in each group. Thus, case presentation might have had an effect that allowed the participants to learn the relation between the example problem and the cases, even though they were not told to do so. According to the results of our investigation, we propose a support system for problem generation that presents cases by controlling similarities. Since the aim of our system is to assist users in generating a broader range of problems, we have implemented functions in our system to evaluate the users’ problems and to present various cases while controlling similarities based on the evaluation.

2. Implementation of the Problem Generation Support System In this study, we propose a system that supports users in generating word problems, which presents problems as cases by controlling similarities in problem situations and solutions. In this study, our focus is on the aspect of problem generation as a creative generation task. Our system gives users a task to generate new problems from an example problem, and assists them

126

K. Kojima and K. Miwa / Experimental Investigation and Implementation of Support

in appropriately generating various problems. Our system supports the users by evaluating their problems based on similarities in two attributes of problem situations and solutions, and by presenting problems as cases that have various features in these two attributes. Since our system requires a variety of problems to function properly, it is assumed that our system stores problems propagated in advance by the problem-generation system [6]. In our system, a user is first given an example problem and prompted to generate a new from it. The user generates and inputs the problem into the system. In this phase, the system in turn requires (1) objects to appear in the problem text (such as apples or pencils), (2) numeric values to be included in the text for solving the problem, (3) equations for solving the problem, and (4) the problem text itself. The system analyzes the numeric values and the equations to check whether the solution can be solved, thus preventing any problems that have inappropriate solutions from being accepted. The system then estimates the problem situations. It uses situation-estimating models, each of which is constructed from nouns and verbs in the texts of problems in the casebase comprising identical situation. The system then indicates evaluation of the user's problem, which is expressed by showing similarities between the example problem and his/her problem, using the same representation as in Fig. 1. Simultaneously, it retrieves and presents some problems as cases. Our system can present various cases by controlling the similarities in problem situations and solutions, such as presenting problems whose solutions are identical to and whose situations are different from the user's problem. As described above, our system repeatedly evaluates problems devised by users and presents cases based on the evaluations. The intention is to provide various constraints to the users’ problem generation. We believe that the constraints allow the users to generate a broader range of problems.

3. Conclusions In this study, we experimentally investigated the effects of case presentation in the task of generating mathematical word problems. According to the results, we proposed a support system for generating word problems, which presents cases to users by controlling similarities.

References [1] Domeshek, E.A., Kolodner, J.L., and Zimring, C. (1994) The Design of a Tool Kit for Case-Based Design Aids, In Proceedings of International Conference on Artificial Intelligence in Design, pp. 109-126. [2] English, L.D. (1997) Children's reasoning processes in classifying and solving computational word problems. Mathematical reasoning: Analogies, metaphors, and images, pp. 191-220, Mahwah, NJ: Lawrence Erlbaum Associates. [3] English, L.D. (1998) Children's Problem Posing within Formal and Informal Contexts, Journal for Research in Mathematics Education, vol. 29, no. 1 pp. 83-106. [4] Finke, R.A., Ward, T.B., and Smith, S.M. (1992) Creative cognition: theory, research, and applications, MIT Press. [5] Hori, K. (1994) A Hypothesis for Discussing the Effect of the Systems for Aiding Creative Concept Formation, Transactions of Information Processing Society of Japan, vol. 35, no. 10, pp. 1998-2008. [6] Kojima, K., and Miwa, K. (2006) Evaluation of a System that Generates Word Problems through Interactions with a User, Lecture Notes in Computer Science, Springer-Verlag, vol.4053, pp.124-133 (2006). [7] Smith, S.M., Ward, T.B., and Schumacher, J.S. (1993) Constraining Effects of Examples in a Creative Generation Task, Memory & Cognition, vol. 21, no. 6, pp. 837-845.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

127

A Computer-Based Environment for Learning by Problem-Posing as Sentence-Integration Tsukasa Hirashimaa, Takuro Yokoyamab, Masahiko Okamotoc, Akira Takeuchib a Hiroshima University, Japan b Kyushu Institute of Technology, Japan c Osaka Prefecture University, Japan [email protected] Abstract: Learning by problem-posing is a well recognized as an important way to learn mathematics. In this paper, we describe a computer-based learning environment for learning by problem-posing as sentence-integration. The learning environment was practically used in arithmetic classes of the second and third grade students in several elementary schools. The definition of “learning by problem-posing as sentence-integration” and the framework of the learning environment are described in this paper. Keywords: Problem-Posing, Interactive Learning Environment, Sentence-Integration

Introduction Learning by problem-posing is a well recognized as an important way to learn mathematics [1,2]. We have investigated computer-based learning environment for interactive problem-posing that is composed of (1) problem-posing interface, (2) problem diagnosis function, and (3) supportive feedback for correcting or completing the posed problems, and confirmed that learning with these environments is useful to improve student’s problem-solving abilities [3-5]. However, the problem-posing is often difficult for lower grade students. In this paper, we described a learning environment for learning by problem-posing as sentence-integration, in order to keep the worth of problem-posing as a learning method and also to simplify the task of problem-posing. In the problem-posing as sentence integration, several simple sentences are provided to the students. The students, then, select necessary sentences and arrange them in an appropriate order. Although the process to integrate simple sentences remains as the characteristics of problem-posing, the process to make simple sentences is simplified to the process to understand them. Therefore, the "problem-posing as sentence integration" is a promising approach to satisfy both (I) simplification of problem-posing task for lower grade students in elementary schools, and (II) keeping the worth of problem-posing as a learning method. In Section 1, learning by problem-posing is discussed in more details. A learning environment for problem-posing as sentence-integration is described in Section 2.

1. Learning by Problem-Posing 1.1 Categorization of Problem-Posing In our research of "Learning by Problem-Posing", the "problem" is defined as follows. Problem = “Given-Information” + “Required-Information”

128

T. Hirashima et al. / A Computer-Based Environment for Learning by Problem-Posing

"Problem-Solving", is an activity to derive required-information from the given-information and then the problem solving should be able to complete deductively. "Solution Method" is, then, the method that is used in the problem solving. For example, when an arithmetical word problem is composed of the following three sentences {(I) Tom had five pencils. (II) Ken received three pencils from Tom. (III) How many pencils does Tom have?}, Sentence-I and –II are the given-information and Sentence-III is the required information. The solution method, then, is “5-3”. These definitions cover most of the problems used in problem exercises in arithmetic, mathematics, physics, and so on. Based on this definition, problem-posing can be completed by adequately combining the following three elements: (1) given-information, (2) required-information, and (3) solution method. These elements, then, are used as constraints that a student has to satisfy in problem-posing. In this subsection, forms of problem-posing are categorized from the viewpoint of the three elements. The following three forms are popular as problem-posing forms, namely, (A) to pose a problem that can be solved by a specific solution method, (B) to pose a problem by modifying a specific problem, (C) to pose a problem based on a specific story or picture. In Form-A, a solution method is used as constraint of problem-posing. We call this problem-posing form as "solution-based problem posing". In Form-B, given-information and required-information are used as constraint of problem-posing this form is called as "problem-based problem-posing". In Form-C, given-information, that is, story, is used as constraint of problem-posing; this form is called as "story-based problem-posing". Problem-posing that dealt with in our research is solution-based problem-posing. In the next subsection, several ways to realize solution-based problem-posing are described. 1.2 Solution-Based Problem-Posing In solution-based problem-posing, a learner is required to make a problem and the problem should be composed of given-information and required-information that can be solved by a specified solution method. To realize this exercise, it is important to provide the way to let a learner to make the composition of given-information and required-information. In a normal classroom, problems are posed by natural language, but such problems written by students often include errors as language. Because to deal with such erroneous sentences is difficult with current natural language processing techniques, to diagnose a problem composed of such sentences is not realistic. In the learning of arithmetical word problem, it is assumed that a learner has already had the ability to understand the meaning of each sentence in a problem and it is expected that the learner acquires the ability to understand the problem mathematically. In learning from problem-posing, therefore, it isn't indispensable to pose a problem by using natural language written by a learner. Based on these considerations, we have proposed three types of problem-posing methods as follows: (A) sentence template method [3], (B) problem template method [4, 5], and (C) sentence card method. In this subsection, because of page limitation, the outline of only the sentence card method has been explained. In the sentence card method, a learner is provided with several cards with a sentence in each card. The learner, then, is required to select some of them and to sort it out in a proper order. Although the process to make a sentence by combining concepts in other methods is substituted for the process to understand the sentences on the cards, the process to integrate simple sentences is same with the other problem posing methods. Therefore, the template card method is an approach to realize (1) to keep the worth of problem-posing as a learning method and (2) to simplify the task of problem-posing. A typical process model of problem-solving of word problems [6] is shown in Figure 1. The processes of transformation and integration are characteristics process of

T. Hirashima et al. / A Computer-Based Environment for Learning by Problem-Posing

129

word problem. In the transformation process, natural language is interpreted linguistically. A linguistic interpretation of a natural language sentence is corresponds to “sentence” in Figure 1. In the integration process, they are integrated mathematically. The structure made by the integration process is often called “problem structure” and it is describes the understanding of the problem. Adequate solution method is selected by using the structure. From the viewpoint of learning of mathematics, the integration process is far more important than the transformation process in the model. In the problem-posing by the sentence template method and the problem template method, transformation and integration are intermingled. Because the problem-posing with sentence card method is the problem-posing focused on the integration process, we call this type as "problem-posing as sentence integration". In the next section, a learning environment for interactive problem-posing as sentence integration, named MONSAKUN, is described.

Figure 1. A Process Model of Problem-Solving of Arithmetical Word Problems.

2. Learning Environment for Problem-Posing: MONSAKUN 2.1 Interface for Problem-Posing The interface of problem-posing in MONSAKUN is shown in Figure 2. The area in left side, imaged blackboard, is "problem-composition area". At the top of the area, a calculation expression is presented. A learner is required to pose a problem that is able to solve by the calculation expression. Here, the expression is the solution-method. Although the expression itself is easy, the learner has to consider the combination of not only a number but also a subject, object and predicate in each sentence. The three blanks in the area are the ones to put sentence cards. Sentence cards are presented at right side of the interface. A learner can move the card by drag&drop method freely in the interface. When a learner pushes "diagnosis button" under the problem-composition area, the system diagnoses the combination of sentences. The results of the diagnosis and message to help the learner's problem-posing is presented by Figure 2. Problem-Posing Interface of MONSAKUN. another window. 2.2 Diagnosis of a Posed Problem A problem posed by a learner is diagnosed by the following three viewpoints: (1) problem

130

T. Hirashima et al. / A Computer-Based Environment for Learning by Problem-Posing

type, (2) relations of concepts, and (3) numerical relation. The problem type is diagnosed by the combination of sentence type. For example, in the change problem, the first sentence has to describe a number of something exists, and the second sentence has to describe a change of the number. By this diagnosis, the type of problem the learner tried to pose can be detected. If the type can not be detected, that is, the combination of sentences is wrong, the system gives advises that what kind of sentence should be put in each blank. When the problem type is detected, the relations of the concepts in the sentences can be diagnosed. Depending on each problem type, the conditions that the relations have to satisfy are different. For example, in the combination problem, the numbers should belong to the concepts that are able to add mutually, like the number of apples and the number of oranges. In the change problems, the object of the change in the second sentence should be appeared in the first and the third sentence. By using such conditions, the relations of concepts are diagnosed. When the problem satisfies the conditions, a calculation expression can be made from the problem. If the problem doesn't satisfy a condition of the problem type, the system judge the problem is wrong. Then, the system gives advises that what kind of condition should be specified. In the diagnosis of numerical relation, a calculation expression is derived from the problem and then compared with the one that is the target of the problem posing. When the two expressions are the same, the posed problem is correct. If the derived expression is different only in the number, the system indicates the difference. When the expression is different in the operation, the system indicates that the posed problem is solved by the difference operation. When the calculation expression derives a minus number, the system indicates that the calculation is impossible because minus number has not been taught in elementary school. Since this case is happened only in subtraction, the system indicates that "it is impossible to subtract this number from that number".

3. Conclusion Remarks The learning environment was practically used in arithmetic classes of the second and third grade students in several elementary schools. As the results, we confirmed that teachers and students accepted it as a useful tool for learning. Besides, analysis of scores of the pretest and posttest as a function of condition and group suggested that the interactive learning environment could certainly improve the children’s problem solving performance. The details of the practical use and the analysis was omitted in this paper, they will be reported in another paper.

References [1] Silver, E.A., CAI, J.: An Analysis of Arithmetic Problem Posing by Middle School Students, Journal for Research in Mathematics Education, Vol.27, No.5, pp.521-539, 1996. [2] English, L.D.: Children’s Problem Posing Within Formal and Informal Contexts, Journal for Research in Mathematics Education, Vol.29, No.1, pp.83-106, 1998. [3] A.Nakano, T.Hirashima, A.Takeuchi: Problem-Making Practice to Master Solution-Methods in Intelligent Learning Environment, Proc. of ICCE'99, pp.891-898(1999) [4] A. Nakano, T. Hirashima, A. Takeuchi: A Learning Environment for Problem Posing, Proc. of ICCE2000, pp. 91-99 [5] A. Nakano, T. Hirashima, A. Takeuchi: An Evaluation of Intelligent Learning Environment for Problem Posing, Proc. of ITS2002, pp.861-872ʳ (2002). [6] Kintsch, W., Greeno, J.G.: Understanding and Solving Word Arithmetic Problem. Psychological Review, 92-1:109-129ʳ (1985)

Discussion

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

133

The Impact of Structured Discussion on Students’ Attitudes and Dispositions toward Argumentation Khai Seng HONG, Ole C BRUDVIK, Yam San CHEE National Institute of Education, Nanyang Technological University, Singapore [email protected] Abstract: Argumentation skills are highly valued in both education and business. As a process, participating in argumentation helps a person to develop their meta-cognitive and higher-order thinking abilities. This paper reports on empirical results on middle-school students’ changes in attitudes towards argumentation as part of an ongoing design-based research study. Past attempts by researchers to foster students’ argumentation skills have met with mixed results. General Web-based discussion boards often do not provide the structures and process scaffolds to help students acquire the target skill. In this study, a web-based structured argumentation board with sentence openers as scaffolds was designed to support students’ engagement with argumentation over a four week long intervention. Two questionnaires, a pre-post and a post-then-pre, were designed to measure students’ attitudes towards argumentation. The two questionnaires were used to identify any treatment dependent “response-shift bias”. Statistical results showed an improvement in students’ attitudes toward argumentation. Qualitative analysis of student essays was also carried out and will be reported separately. Keywords: argumentation, CSCA, student attitudes, critical thinking

1. Introduction Argumentation is a higher-order thinking ability highly valued in both education and business problem solving circles. However, the quality of people’s argumentation skills, in general, has been found to be low. Kuhn’s [1] study of argument-based reasoning found that, in everyday reasoning, people readily make assertions about the causes of various social phenomena (e.g., what causes a return to crime). However, they have difficulty providing convincing reasons for the phenomena, or they are unable to provide sound evidence for reasons they provide. It thus appears that unless educators pay special attention to nurturing students’ argumentation skills, the development of such skills will not occur. Argumentation that occurs in a group context has the decided advantage of helping participants understand their position better as they begin to re-evaluate it in the broader, fleshed out information space of counterarguments, rebuttals, and alternative assessments of evidence. Thus we have the saying that “the difficult part in an argument is not to defend one’s opinion but rather to know it” [2]. Argumentation that occurs in a collaborative learning context has the additional advantage of getting students to engage in the processes of collaborative meaning making and the co-construction of knowledge as teams of students engage in debate about important issues. Ramage, Bean, & Johnson [3] have identified six types of arguments. Our analysis of these six types reveals that they can be categorized into three main categories that we call debate, causal argumentation, and problem solving. In terms of educational usefulness, the

134

K.S. Hong et al. / The Impact of Structured Discussion on Students’ Attitudes

latter two stand out. In the current study, causal argumentation was chosen as the basis of our research on students’ acquisition of argumentation skills. Causal arguments try to show how one event brings about another and require close analysis of phenomena. A key purpose in exposing students to the process of causal argumentation is to help them understand the nature of scientific knowledge: that it is not simply a bundle of “discovered” or “proven” facts [4]. Rather, in education, students should be helped to understand that received knowledge involves a social process of “knowledge making” that is centrally grounded in argumentation. In the current study, a class of middle-school students engages in causal argumentation over a period of four weeks. A combination of online and offline learning activities was designed to introduce the students to causal argumentation. In this paper we investigate the students’ change in attitudes towards the causal argumentation learning activities as they engage in a web-based structured argumentation board and offline lessons. The paper proceeds with a review of relevant previous work and a description of the design of the research program and methods used for the current paper. Finally, we present the quantitative findings on the students’ attitudes. Qualitative analysis was carried out through the use of pre-post student essays, analysis of the forum messages and interviews. However, it will be reported separately and will not be discussed in this paper.

2. Previous Work There has been considerable work in the field of CSCL (Computer Supported Collaborative Learning) that focuses on student “conversation” in electronic discussion boards. Such discussions are typically threaded, allowing students to respond using free-form text to any particular message within a thread of messages. In such discussion boards, student dialog can be extremely rich. However, due to the lack of structure (other than threading), the dialog often ends up being difficult to follow and to analyze [5]. Two broad approaches have been used to try to improve on the quality of students’ electronic discourse. The first approach seeks to enhance students’ articulations through the use of sentence openers and prior classification of a student’s response. These features, evident in CaMILE [6] and CSILE [7] can encourage more critical and reflective thinking on the part of students despite the fact that some students will feel restricted by these features. A second approach seeks to enculturate students into a particular way of thinking by imposing “constraints” on what students can choose to do (or say) at any point in time. Belvedere [8] is an exemplar of such a system, although it is a diagrammatic representation tool rather than a textual one. Cho & Jonassen [9] were able to use Belvedere to explore students’ argumentation based on the constraints (i) hypothesis, (ii) data, (iii) principles, and (iv) other, and using the relations (i) for, (ii) against, and (iii) and. In subsequent work, Jonassen & Remidez [10] describe initial efforts in building a structured discussion board that implements arguments at four levels: (i) problem, (ii) proposal, (iii) warrant, and (iv) evidence. This scheme does not actually implement the complete Toulmin Argument Pattern which includes the elements datum, claim, qualifier, warrant, backing, and rebuttal. Research on constraint-based approaches directed at supporting the acquisition of argumentation skills is still in its infancy. Weinberger, Fischer, & Stegmann [11], for instance, get students to use the interface that uses an incomplete form of Toulmin’s Argument Pattern (like Jonassen & Remidez, [10]). The interface also does not separate ground from warrant; thereby introducing analytical difficulties when teachers (or researchers) try to determine what is the student’s intended ground and what is the intended

K.S. Hong et al. / The Impact of Structured Discussion on Students’ Attitudes

135

warrant. In addition, the representation is potentially confusing. While it adheres to the forward reasoning scheme leading to a claim that Toulmin describes, this scheme does not naturally support causal argumentation which starts with claims which are then debated (unlike the legal reasoning process that ends with a claim: a declaration of guilt or otherwise). Research to date on argumentation systems appear to suffer from at least two limitations. The first limitation relates to partial or selective implementation of the Toulmin Argument Pattern, together with sometimes confusing use of terms. Following Ramage et al. [3], a ground, strictly speaking, refers to the evidence brought to bear to support a justification for a claim, not the justification itself. The second limitations relates to the confusing way in which the elements of argumentation are often represented. There is a third area of concern. Research findings to date have not always yielded expected results. Cho & Jonassen [9], for instance, cite a study by Tan [12] showing that students using a constraint-based argumentation tool (QuestMap™) performed significantly better in stating “grounds” in their argumentation but this had no significant effect on students’ problem solving skills. Furthermore, findings from Jonassen & Remidez [10] suggest that students were not always choosing argument elements reliably and accurately.

3. Research Objectives Our analysis of the previous body of research led us to the belief that promoting the practice of argumentation requires the development of appropriate pedagogical strategies and materials that offer practical guidance to the teachers. Furthermore, due to the importance of engaging the students in cooperative and collaborative dialogical group argumentation we decided to develop a Web-based argumentation tool. Therefore our main research objectives are: 1. Develop understanding of and principles for classroom interventions related to fostering students development of critical thinking and argumentation skills using design-based research. 2. Develop and enhance a Web-based argumentation tool for group argumentation. 3. Investigate methods of effective student assessment using discourse-based qualitative methods. 4. Develop collaborating teachers’ abilities to design their own lesson plans for continued use of the Web-based argumentation tool. The focus of the study reported here is, however, principally on the first area of interest. We investigate the changes in students’ attitudes towards argumentation before and after they engage in causal argumentation learning activities.

4. Theoretical Framework In the current study the underlying argumentation framework is that of Toulmin’s Argument Pattern [13]. According to Toulmin [14], an individual argument consists of a statement or claim which can be supported by grounds or data while a warrant can be used as justification for the claim. Toulmin also recognizes three other elements that may be present in an argument backing, qualifier, and rebuttal. Backing provides credibility for the warrant and comes in the form of evidence and data. A qualifier indicates the degree of force or certainty that a

136

K.S. Hong et al. / The Impact of Structured Discussion on Students’ Attitudes

claim possesses and is sometimes implicit in the structure of an argument by the use of the words like “maybe” or “might”. A rebuttal represents certain conditions or exceptions under which the claim will fail [13]. Ramage et al [3] added another element called reason to link the grounds to the claim while a warrant identifies the underlying assumptions behind making the claim and/or reasons supporting it. We adopted an instructional approach that scaffolds the learning process through the use of modeling the argumentation steps and sentence openers for students to compose their arguments. The argument is made that, over prolonged use, the scaffolds will be internalized by students as a “cognitive residue” [15] so that when the scaffolds are removed, students’ habits of thinking with respect to argumentation and its structure will remain. Addressing the confusing use of terms, the intervention was designed to introduce these terms to students before they were used in the online structured argumentation board. This reduced chances of students selecting an inappropriate argumentative move due to their misunderstanding of the terms used. Differing from Weinberger et al [11], where grounds are combined with the warrant as one element, the online tool will represent Toulmin’s elements individually. This provides students with a clearer definition of Toulmin’s Argument Pattern and aids in clearer post-intervention analysis. For example, if grounds and warrants were combined as one entry, it might be hard to find out whether the student understood the difference between grounds and warrants through their responses using such an argumentation tool.

5. Research Design The current study takes the form of a design experiment with two cycles. Design experiments, as a research methodology, emphasize the detailed implementation and study of interventions with evolving pedagogical goals in rich, authentic settings. It acknowledges the complexities of classroom teaching and enlightens both practitioners and researchers by leading to the development of theoretical ideas grounded in contexts of practice. At the current stage, the study is in the first intervention which consists of a combination of classroom activities and activities using the structured argumentation board. The study follows the students’ development and the activities in the learning environment. 40 fourteen-year old students participated in the intervention which was infused into the regular English language curriculum. Before the intervention started, the students were tasked to write an essay on a causal argumentation topic. A survey that measured their self-reported attitudes towards argumentation was also administered. The essay and survey constituted the pre-intervention comparison data. They went through a series of classroom lessons introducing them to elements of Toulmin’s argument pattern. Due to the limited curriculum time, the warrant and backing elements are planned to be introduced to the students at a later intervention cycle. The students were given a topic to discuss about and went through tasks to (i) develop claims of their own, (ii) support their claim with reasons, (iii) provide evidence for their reasons and (iv) to develop rebuttals against another student’s claim. Throughout the classroom activities, they were exposed to the use of appropriate sentence openers for each Toulmin element through the use of structured worksheets and the teacher modeled the argumentation process. The sentence openers also appear as scaffolds in the online structured argumentation board. After a week and a half of classroom activities, the students used the online structured argumentation board to discuss about the topic they covered in class to get themselves familiarized with the online environment. A new discussion topic was given and the class

K.S. Hong et al. / The Impact of Structured Discussion on Students’ Attitudes

137

was split into teams. A jigsaw activity structure was used. Teams were seeded with different views on the discussion topic and each team member was assigned one of five discussion forums to participate in. Teacher facilitation was faded back to allow the students to engage in argumentation without support. After the intervention, the students were tasked to write another essay on a causal argumentation topic that was similar to the pre-intervention essay. The post self-report attitude survey was also administered to the students. A second survey was also done one day after the post survey. In this survey, the items were identical to the pre and post surveys. However, students were asked to respond twice to each item on the second survey. The first response required students to report their behavior or understanding as a result of the intervention (“after”). The second required participants to report their behavior before the program (“before” rating). This second survey was administered due to the possibility of a “response shift bias” phenomenon [16],[17],[18]. In using self-report measures, researchers assume that a person’s standard for measurement of the scale being used will not change from pretest to posttest. If the standard of measurement were to change, however, the posttest ratings would reflect this shift in addition to the actual changes in the person’s level of functioning. For example, the students might feel at pretest that they are “average” in terms of their attitudes and understanding of argumentation. As the intervention progresses, their understanding of the skills involved in argumentation improve. Finally, at the end of the intervention they realize that their level of understanding was really below average at the pretest. Consequently, comparisons of pretest with posttest ratings would be confounded by this distortion of the internalized scale and would encounter a lack of findings of significant differences between pre and posttest measurements.

6. Results The statistical results of the first survey (see Table 1) showed a general increase, from preto post-intervention, in the means across the five items used to measure students’ attitudes towards argumentative practice. With the use of univariate ANOVA procedure, it is found that only the second item, Attitude 2, showed significant increase from pre- to post-intervention, with its F-ratio value as 6.127 and p-value as .016. In the second set of items used to measure how much students value the skills of argumentation, none showed a significant increase from pre- to post-intervention even though all the means have increased except for the item Value 3 where the means remained constant throughout the intervention. Next, three of the items measuring the argumentative disposition of students to provide reasons and evidence showed a general increase from pre- to post-intervention. Item Evidence 1 remained constant while item Evidence 4 decreased insignificantly over the intervention. None of the items in this category showed significant difference over the intervention, as can be seen from the F-ratio values and p-values. Lastly, the only items measuring how much students enjoy learning the skills of argumentation that showed significant increments over the intervention are Enjoy 1 and Enjoy 2, with F-ratio values (and p-values) of 7.769 (p = .007) and 6.686 (p = .012). The other two items Enjoy 3 and Enjoy 4 also showed slight increase from pre- to post-intervention, while the last item Enjoy 5 dipped slightly.

138

K.S. Hong et al. / The Impact of Structured Discussion on Students’ Attitudes

Item Attitude 1 Attitude 2 Attitude 3 Attitude 4 Attitude 5 Value 1 Value 2 Value 3 Value 4 Value 5 Evidence 1 Evidence 2 Evidence 3 Evidence 4 Evidence 5 Enjoy 1 Enjoy 2 Enjoy 3 Enjoy 4 Enjoy 5

Table 1: Statistical Results for First Survey Pre-Intervention Post-Intervention n M SD n M SD 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36

3.81 4.33 4.42 4.64 5.06 4.36 4.72 4.61 3.94 4.64 4.89 4.31 4.61 4.25 4.22 3.81 3.33 4.06 4.39 4.83

1.009 .828 .806 .867 .532 .931 .849 .903 .715 .762 .820 .889 .934 .806 .760 .856 .956 .826 .871 .697

38 38 38 38 38 38 38 38 38 38 38 38 38 38 38 38 38 38 38 38

4.05 4.74 4.68 4.87 5.18 4.37 4.82 4.61 4.21 4.76 4.89 4.39 4.71 4.11 4.39 4.42 3.92 4.37 4.47 4.76

.899 .554 .662 .665 .730 .751 .766 .823 1.018 .852 .649 .718 .802 .764 .916 1.030 .997 1.149 .762 .852

ANOVA F p 1.240 6.127* 2.445 1.644 .744 .001 .248 .001 1.677 .436 .001 .227 .242 .629 .772 7.769** 6.686* 1.791 .199 .149

.269 .016 .122 .204 .391 .970 .620 .977 .200 .511 .973 .636 .624 .430 .382 .007 .012 .185 .657 .700

Note. *p < .05, **p < .01

The statistical results of the second survey (see Table 2) showed that the students improved in their attitudes towards argumentation based on the five items used, as shown in the increase in the means. The univariate ANOVA results also showed that all five items had a significant increase. For the scale of valuing the skills of argumentation, there was a significant increase in the rating of all but one item, Value 2. All the items measuring the argumentative disposition of students in finding reasons and evidence to support their arguments also showed a significant increase. Similarly in the next category of items measuring the extent to which students enjoy learning the skills of argumentation, the ratings before and after the project showed significant increments. Lastly, the students also rated significant increments in their attitudes towards the project, based on the three items used in this category.

Item Attitude1 Attitude2 Attitude3 Attitude4R Attitude5 Value1 Value2R Value3 Value4R Value5

Table 2: Statistical Results for Second Survey Before After n M SD n M SD 40 40 40 40 40 40 40 40 40 39

3.20 3.55 3.95 3.63 4.08 3.40 4.05 3.90 3.40 4.21

1.043 1.239 .932 1.148 1.185 1.105 1.377 .871 1.355 .978

40 40 40 40 40 40 40 40 40 39

4.30 4.23 4.95 4.65 5.18 4.63 4.45 4.98 4.10 5.08

.966 1.423 .783 1.167 .675 .705 1.339 .733 1.297 .839

ANOVA F p

23.954** 5.118* 26.990** 15.688** 26.018** 33.778** 1.528 33.992** 5.184* 17.842**

.000 .026 .000 .000 .000 .000 .220 .000 .026 .000

139

K.S. Hong et al. / The Impact of Structured Discussion on Students’ Attitudes

Evidence1 Evidence2R Evidence3 Evidence4R Evidence5 Enjoy1 Enjoy2R Enjoy3 Enjoy4R Enjoy5 Project1 Project2 Project3

40 40 40 40 40 39 40 40 40 40 40 40 40

3.75 3.35 3.95 3.23 3.50 3.72 2.98 3.38 3.65 3.78 3.53 3.18 3.40

1.127 1.145 1.037 1.165 1.038 1.099 1.230 1.213 1.312 1.230 1.012 .958 1.236

40 40 40 40 40 39 40 40 40 40 40 40 40

4.95 4.43 5.03 4.15 4.23 4.69 4.10 4.68 4.60 4.98 4.70 4.28 4.95

.959 1.217 .891 1.424 1.165 1.080 1.297 1.023 1.081 .733 .939 1.109 .959

26.304** 16.558** 24.738** 10.106** 8.634** 15.597** 17.821** 25.457** 10.253** 27.468** 28.958** 22.539** 39.245**

.000 .000 .000 .002 .004 .000 .000 .000 .002 .000 .000 .000 .000

Note. The items with an ‘R’ have been negatively phrased compared to the original survey. *p < .05, **p < .01

7. Discussion The statistical results from the two surveys indicate a “response-shift bias” in the first survey. The first survey shows a general increase in the means on most items even though most of these items are not shown as significant in the analysis of variance. In the second survey there is also a general increase in means, however, the analysis of variance shows a stronger effect than the first survey. Response-shift bias explains this as the students have a different understanding of the meaning of the questions in the questionnaire before participating in the study as compared to after they have participated in the study. In order to investigate further, 2 focus groups were held with a sample of 12 students, 6 students in each group. The focus group responses showed that the students valued learning the structure and process of writing a causal argumentation. They explained that learning and understanding this structure and process enabled them to better understand the process and views when they encounter argumentation in the media or other places. They also felt that learning the structure and the peculiar process of causal argumentation helped them to write better in other school work which involved a structured organization. The students’ understood the need for evidence but experienced difficulty in using and finding evidence to support their own arguments. This was despite students being provided with a ready selection of relevant evidence. This indicates that the students do understand the importance of evidence, but find it to be a major obstacle in their writing of arguments. The students expressed positive attitudes, values and enjoyment towards using an online structured argumentation board as compared to an offline face-to-face classroom argumentation activity like a verbal debate. They explained that they were able to concentrate better on developing a good argument in the online asynchronous environment. Noise in the classroom and the immediate “on-the-spot” responses required in a verbal debate impeded their ability to concentrate and construct better arguments. The researchers’ observations were congruent to the students’ views. When the students worked offline in the class, the noise level in the class was extremely high. However, when the class was engaged in a discussion using the online argumentation tool, the class was very quiet and the individual students were highly focused on the task. The students also liked the ability to follow other students’ arguments online as they can see other students’ work and learn from it. The students also mentioned that the online argumentation environment provided a more comfortable environment as compared to

140

K.S. Hong et al. / The Impact of Structured Discussion on Students’ Attitudes

“face-to-face”. This could be due to the emotional tension felt when students are expressing conflicting viewpoints against each other “face-to-face”. The students’ responses did not indicate any difficulties in using the online structured argumentation board. They found it easy to learn, taking one to five minutes to understand the interface. The sentence-openers and pre-set argumentation post selections (claim, reasons, evidence, rebuttal etc.) were also easily understood.

8. Conclusion A four week intervention on causal argumentation with a class of middle-school students was reported. The students’ showed changes in attitude toward the process of argumentation after accounting for response shift bias. We attribute this effect to the students going through the intervention and developing a deeper sense of what argumentation is about. Interviews with students showed that they enjoyed the web-based structured argumentation board as well as the intervention in general. Qualitative analysis of student essays is still ongoing and will be reported separately.

References [1] Kuhn, D. (1991). The Skills of Argument. Cambridge University Press. [2] Kirschner, P. A., Buckingham Shum, S. J., & Carr, C. (eds.) (2003). Visualizing Argumentation: Software Tools for Collaborative and Educational Sense-Making. Springer. [3] Ramage, J. D., Bean, J. C., & Johnson, J. (2004). Writing Arguments: A Rhetoric with Readings (6th edn.). Pearson Longman. [4] Driver, R., Newton, P., & Osborne, J. (2000). Establishing the norms of scientific argumentation in classrooms. Science Education, 84 (3), 287–313. [5] Tay, M. H., Hooi, C. M., & Chee, Y. S. (2002). Discourse-based learning using a multimedia discussion forum. In Proceedings of the Tenth International Conference on Computers in Education, Auckland, New Zealand, pp. 93–94. IEEE Computer Society. [6] Guzdial, M. & Turns, J. (2000). Computer-supported collaborative learning in engineering: The challenge of scaling-up assessment. In M. J. Jacobson & R. B. Kozma (eds.), Innovations in Science and Mathematics Education, pp. 227–257. Lawrence Erlbaum. [7] Scardamalia, M. & Bereiter, C. (1994). Computer support for knowledge-building communities. Journal of the Learning Sciences, 3 (3), 265–283. [8] Suthers, D. (1998). Representations for scaffolding collaborative inquiry on ill-structured problems. Paper presented at the 1998 AERA Annual Meeting, San Diego, CA. [9] Cho, K. L. & Jonassen, D. H. (2002). The effects of argumentation scaffolds on argumentation and problem solving. ETR&D, 50 (3), 5–22. [10] Jonassen, D. & Remidez, H. Jr. (2005). Mapping alternative discourse structures onto computer conferences. Int. J. Knowledge and Learning, 1 (1/2), 113–129. [11] Weinberger, A., Fischer, F., & Stegmann, K. (2005). Computer-supported collaborative learning in higher education: Scripts for argumentative knowledge construction in distributed groups. In Proceedings of CSCL 2005. [12] Tan, S. C. (2000). Supporting collaborative problem solving through computer-supported collaborative argumentation. Unpublished doctoral dissertation, Pennsylvania State University. [13] Toulmin, S. (1958/2003). The Uses of Argument (updated edn.). Cambridge University Press. [14] Toulmin, S., Rieke, R. and Janik, A. (1984). An Introduction to Reasoning. New York: Macmillan. [15] Zellermayer, M., Salomon, G., Globerson, T., & Givon, H. (1991). Enhancing writing-related metacognitions through a computerized writing partner. American Educational Research Journal, 28, 373–391. [16] Howard, G. S. (1980). Response shift bias - a problem in evaluating interventions with pre/post self-reports. Evaluation Review. 4(1), 93-106. [17] Rockwell, S. K. & Kohn, H. (1989). Post-then pre evaluation. Journal of Extension. 27(2), 19-21. [18] Rohs, F.R. & Langone, CA. (1997). Increased Accuracy in Measuring Leadership Impacts. Journal of Leadership Studies. 4(1), 150-159.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

141

Assessing the Impact of A Structured Argumentation Board on the Quality of Students' Argumentative Writing Skills Ole C BRUDVIK, Khai Seng HONG, Yam San CHEE, Libo GUO National Institute of Education, Nanyang Technological University, Singapore [email protected] Abstract: This paper focuses on design of qualitative discourse-based methods for effective assessment of students’ quality of argumentation. We advance a method for qualitative assessment of students’ argument which we through discourse-based methods evaluate the speaker’s stance in the argument, the speaker’s thematic development of the content and criteria for layout. Our research program takes a design-based research approach focusing on the design and evaluation of Web-based learning environments that support the teaching and development of student’s critical thinking skills and disposition through development of the skills of sound argumentation. The purpose of the research is to develop materials and strategies to support argumentation in the classroom with a Web-based tool for group argumentation, and to support the teachers’ developing skills for teaching argumentation. Data was collected pre and post and during a four week intervention. Students causal argumentation essay were collected both pre and post. The paper presents the method with an analysis of one student’s causal argumentation essays before and after the intervention. Keywords: Critical thinking, argumentation, qualitative assessment, ICT, structured argumentation board

Introduction A key skill for critical thinking is that of argumentation. To argue effectively, students must first be able to articulate their thinking, and then also be able to reflect on that thinking. Kuhn’s [1] study of argument-based reasoning across the lifespan found that, I everyday reasoning, people readily make assertions about what causes of various social phenomena (e.g., what causes students failure in school). However, they experience difficulty providing cogent reasons for the phenomena or they find themselves unable to provide sound evidence for the reasons they give. In her more recent book, Kuhn [2] further argues that the most enduring and valuable skills that educators can impart to school children are the skills of inquiry and argument. Unless educators pay special attention to nurturing students’ argumentation and reasoning skills, it appears that the development of such skills will not occur. Traditional methods for assessment of students argumentation has mainly focused on the criteria for layout of the argument, evidence and explanations [3] and [4], in this paper we advance a method for qualitative assessment of students’ argument which we through discourse-based methods evaluate the speaker’s stance in the argument, the speaker’s thematic development of the content, in addition to criteria for layout. In the current study, a class of middle-school students engaged in causal argumentation over a period of four weeks. A combination of online and offline learning activities was designed to introduce the students to causal argumentation. A web-based

142

O.C. Brudvik et al. / Assessing the Impact of a Structured Argumentation Board

structured argumentation board was used for the online argumentation. The paper proceeds with a review of relevant previous work critical thinking and argumentation, a brief overview of the research program, our theoretical framework and analysis of one student’s causal argumentation essays before and after the intervention. 1. Critical Thinking and Argumentation While there is a long-standing literature on thinking [5], critical thinking [6] and critical reasoning [7], traditional approaches to the subject have focused on critical thinking in a logico-deductive framework that emphasises the importance of argum`ent validity and the need to detect and avoid fallacies in reasoning (e.g., Munson, Conway & Black, 2004). All too often, students end up learning about critical thinking rather than acquiring the skills of critical thinking. As Kuhn [2] emphasises, however, “[t]he concept of thinking skills adopted here contrast sharply with [the] traditional one. Thinking is something people do, most often collaboratively, while they are engaged in pursuing the activities and goals that fill their daily lives” (original emphasis; p. 13). Dialogue is not simply talk or the sharing of ideas. It is a structured, extended process leading to new insights and deep knowledge and understanding and, ultimately, better practice. There is a strategic orientation implicit in dialogue aimed at advancing beyond participants’ initial stages of knowledge and belief. Argumentation that occurs in group context has the decided advantage of helping participants understand their position in a critical light as they begin to re-evaluate it in a broader, fleshed-out information space of counterarguments, rebuttals, and alternative assessment of evidence. Students engaged in the process of argument are given the opportunity to understand that “the difficult part in an argument is not to defend one’s opinion but rather to know it” Andre Maurois, 1885-1967, cited [8]. Argumentation that occurs in a collaborative learning context has the additional advantage of getting students engaged in the process of collaborative meaning-making and co-construction of knowledge as teams of students engage in dialogue and debate. Through this process, students are helped to understand that “any true understanding is dialogical in nature” [9] as they appropriate the multiple voices participating in the dialogic space. In addition, students also come to “know themselves” and construct a sense of personal identity [10] through the practice of argumentative discourse. 2. Research Programme The current study takes the form of a design experiment with two cycles. Design experiments, as a research methodology, emphasize the detailed implementation and study of interventions with evolving pedagogical goals in rich, authentic settings. It acknowledges the complexities of classroom teaching and enlightens both practitioners and researchers by leading to the development of theoretical ideas grounded in contexts of practice. 2.1 Research Objectives Our analysis of the previous body of research led us to the belief that promoting the practice of argumentation requires the development of appropriate pedagogical strategies and materials that offer practical guidance to the teachers. Furthermore, due to the importance of engaging the students in cooperative and collaborative dialogical group argumentation we decided to develop a Web-based argumentation tool. Therefore our main research objectives

O.C. Brudvik et al. / Assessing the Impact of a Structured Argumentation Board

143

are: 1. Develop understanding of and principles for classroom interventions related to fostering students development of critical thinking and argumentation skills using design-based research. 2. Develop and enhance a Web-based argumentation tool for group argumentation. 3. Investigate methods of effective student assessment using discourse-based qualitative methods. 4. Develop collaborating teachers’ abilities to design their own lesson plans for continued use of the Web-based argumentation tool. The focus of the study reported here is, however, principally on the third area of interest. 2.2 Intervention 40 middle-school students participated in the intervention which was infused into the regular English language curriculum. Before and after the intervention, the students were asked to write an essay on a causal argumentation topic. Classroom lessons introduced them to elements of Toulmin’s argument pattern [11] & [12]. Due to the limited curriculum time, the warrant and backing elements are planned to be introduced to the students at a later intervention cycle. The students were given a topic to discuss about and went through tasks to (i) develop claims of their own, (ii) support their claim with reasons, (iii) provide evidence for their reasons, and (iv) to develop rebuttals against another student’s claim. Throughout the classroom activities, they were exposed to the use of appropriate sentence openers for each Toulmin element through the use of structured worksheets and the teacher modeled the argumentation process. The sentence openers also appear as scaffolds in the online structured argumentation board. After a week and a half of classroom activities, the students used the online structured argumentation board to discuss about the topic they covered in class to get themselves familiarized with the online environment. A new discussion topic was given and the class was split into teams. 3. Theoretical Framework for Assessing the Quality of Students Argumentation Drawing upon previous work at the analysis of students’ argumentation [11] and [12], we propose a framework which involves an examination of the stance the student takes toward the content presented, the depth and strategies of the development of the content, and the layout of the argument. Each of these is described briefly below. In order to assess the speaker’s stance in the argument we draw upon Halliday and Matthiessen’s [13] modality system. The first element is that of polarity (‘yes’ and ‘no’). For example, “I liked Yen” (positive) and “I didn’t like Yen” (negative). When a text consists only of such sentences, the author takes an authoritative stance toward the content of the text and tends to close off the opportunity for negotiation and dialogue with the reader. Such a text does not encourage alternative points of view and is likely to become a target for attack on its tendency to over-generalize. The second element is the space between ‘yes’ and ‘no’, the Modality [13]. There are four types of modality. These are explained below. (i) Probability: where the speaker expresses judgments as to the likelihood or probability of something happening or being, ‘maybe yes or maybe no’. For example,

144

O.C. Brudvik et al. / Assessing the Impact of a Structured Argumentation Board

‘The bridge was possibly built in the 1920s’, ‘The bridge must have been built in the 1920s’. In both examples, the author is tentative and allows alternative judgments to be given by the reader; (ii) Usuality: where the speaker expresses judgments as to the frequency with which something happens or is, ‘sometimes yes sometimes no’. For example, ‘he usually sits there all day’; (iii) Obligation: where one gets other people to do things in a less direct way, through using a way other than the imperative ‘Get out of here!’, for example. Examples: ‘We must read “The Bostonians”’, ‘You are obliged to read Henry James!’; and (iv) Inclination (& Ability): willing / able to do something (for someone else). For example, ‘I’d like to lend you “The Bostonians”’. For the analysis of the speaker’s strategy for developing the thematic content and underlying ideology we draw upon Lemke’s [14] semiotic thematic system analysis. Semiotics describes social actions in terms of semiotic resources and semiotic formations. Semiotic resource systems matches the kinds of meanings you can make (semantic functions) with the actions (such as words) needed to make those meanings in a particular community [14]. Semiotic formation is an actual pattern of meaningful action, using semiotic resources that is repeatedly performed and recognized in a community [14]. Activity structures and thematic patterns (more properly called thematic formations) are examples of semiotic formations. A record of social action, whether piece of writing, video etc., is a semiotic text. The actual events constitute a semiotic production [14]. In the current study, we are analyzing a semiotic text in terms of the thematic pattern of the content. A thematic pattern is a way of picturing the network of relationships among the meanings of key terms in the language of a particular subject. Those terms and their synonyms amount to ways of saying the thematic items of the pattern. The grammar and rhetorical forms used in speaking or writing provide the means for expressing the semantic relationships among these items. For criteria for layout we draw upon Toulmin [11]. According to Toulmin [11] and [12], an individual argument consists of a statement or claim which can be supported by grounds or data. A warrant is used as a justification for the claim. In addition, backing, qualifier, and rebuttal are optional elements that make up an argument. Backing provides evidence and data for the warrant while a qualifier indicates the degree of force or certainty that a claim possesses. Qualifiers are similar in nature to Halliday’s modality system [13]. A rebuttal represents certain conditions or exceptions under which the claim will fail [11]. Ramage et al [15] adapted Toulmin by added another element named reason to link the grounds to the claim and re-defining the warrant as the underlying assumption behind making the claim and/or the reasons supporting it. 4. Data Analysis – Speakers Stance The two topics ‘what causes students to make friends more easily in your school?’ and ‘what causes teenagers in Singapore to stay up till past mid-night on a regular weekday?’ invite students to identify and hypothesize the causes for some phenomena and justify their hypotheses. Given the complexity of student and adolescent lives and the limitation of individual’s observation, the students will of necessity adjust the contents they put forward, unless they decide to include (a) some accounts about themselves, their siblings and other close relationships, and (b) some accounts that they believe they can present as facts, that do not need a modification or qualification. By examining the modal choices that a student makes we can judge their awareness of the audience and hence also their own selves, and their facility with these interpersonal resources.

O.C. Brudvik et al. / Assessing the Impact of a Structured Argumentation Board

145

4.1 Analytical procedure Following Halliday and Matthiessen [13], we first segmented the text into ranking clauses and then established whether or not each clause is polarized or takes a modal element. If the clause is of the latter type, we further identified the type of modal element presented briefly above. We also included as instances of modal elements separate clauses (interpersonal metaphor, [13]) and group or group complexes, in accord with the observation that interpersonal meaning is prosodic and may appear in various places and in various forms. In what follows we describe how one student employs modal resources to present his arguments. 4.2 Analysis of one pair of essays Within the 32 ranking clauses in the pre-intervention essay, 15 modalities were identified (will, it is known, usually, may, we also know, usually, definitely, will, definitely, able to, we also know, will, probably, may, can). This results in a 46.9% (15/32) ratio of modality to ranking clauses. Overall, the student took care to modify his statements through the use of modal elements, such as ‘may’ and ‘probably’. He was well aware of the presence of alternative points of view. For example, ‘Usually, people of our ages may make friends easily as we are more outspoken and more daring to approach’. The use of ‘usually’ and ‘may’ suggests that the student has taken a balanced, non-dogmatic view of the content of the clauses. Also, the student is able to make conscious selections of the modal resources to achieve the specific effects that he intends. For example, in ‘we students will definitely meet up everyday, and therefore…’, ‘will definitely’ brings about a specific effect and suits the content of the clause well. In the post-intervention essay, 36 ranking clauses were found. Within these clauses, there were 23 modalities (definitely, it is known, definitely, from my own experience, will, always, in my opinion, unable to, can, from what I see, unable to, really, will, from my personal experience, always, it is known, personally, can, from what I think personally, will, have to, will, the conclusion). This results in a 63.9% (23/36) ratio of modality to ranking clauses. Compared with the pre-intervention essay, the student attempted more modal elements and exhibited a larger variety of them as well. More significantly, the student included more mention of ‘my personal experience’ as a way to both beef up the validity of the content of his clauses and make his statements less prone to attacks from critical readers. ‘My personal experience’ was a sentence opener scaffold introduced to the students in the web-based argumentation board. However, it would take more practice for him to be equally skilled in both form and function. For instance, the structure for ‘in my opinion of staying till midnight’ is incorrect. 5. Data Analysis – Thematic Development 5.1 Analytical procedure In our analysis of the thematic pattern of the content we draw upon Lemke’s [14] semantic relations for thematic analysis. In thematic analysis, a semantic relation describes how the meanings of two words or phrases (thematic items) are related when they are used together in taking about a particular topic. Thus the thematic pattern shows how the semiotic text strategically develops its meanings.

146

O.C. Brudvik et al. / Assessing the Impact of a Structured Argumentation Board

5.2 Analysis of one pair of essays In the pre-essay the speaker sets up two thematic formations. The two thematic formations become clear when the speaker states: “These are some time and place factors which allows making friend in …easier”. In the first thematic formation the speaker refers to teenager’s stage in the life cycle of human being as “outspoken”, “attractive”, “daring to approach” and seem to imply that girls and boys are eager to know each other. In the second thematic formation the speaker refer to place as the school. The speaker develops the second thematic formation with the activities in the school (outings, field-trips, CCA, sports), shared religion and activities with the primary school. In the post-essay the speaker sets up two thematic formations and takes an evaluative stance to the content of the two thematic formations. The two thematic formations become clear when the speaker states: “Though these things can be controlled, there are stuff which makes one not sleeping early”. The first thematic formation is the activities teenagers themselves choose to do which causes them to stay up late (online chatting, watch television). The second thematic formation is the uncontrollable factors which cause one not to be able to sleep. The writer seems to refer to biological causes with the words “energetic” and “hyperactive” and “it is known that if a person does his workout late in the night, it affects his sleep”. The writer takes an evaluative stance to the content of the two thematic formations by constructing an alliance between them [16] the speaker normalizes and warrants the two thematic formations [16] as he develops his argument. 5.3 Comparison Compared to the pre-essay, the speaker better develops the thematic content in the post-essay. The speaker is able to develop the two thematic formations by constructing an alliance between them. In the pre-essay, the speaker does not explicitly construct an evaluative or attitudinal stance [16] to the content of both thematic formations. The speaker develops the second thematic formation (place: school) where as the first thematic formation is not developed. 6. Data Analysis – Criteria for Layout 6.1 Analytical Procedure The text is first segmented before being coded as one of the elements in Figure 1 based on its overall relation to the text as a whole. A portion of text will be used as an example: “Definitely it means less sleep each day due to the fact that teenagers are in secondary school. It is known that secondary schools definitely gives homework and projects to students. From my personal experience, homework will always be dragged till late midnight.” Analyzing the first sentence “Definitely it means less sleep each day due to the fact that teenagers are in secondary school”, the student claims that having less sleep each day is due to school. The student then continues on to support his claim by providing a reason, “school gives homework and projects”, of why he believes it is so. An additional elaboration is given in the next sentence in the form of personal experience, which acts as grounds for the student’s earlier reason.

O.C. Brudvik et al. / Assessing the Impact of a Structured Argumentation Board

147

6.2 Analysis of one pair of essays In the student’s pre-intervention essay (Figure 2), 9 claims were made, with only 5 being supported by reasons. No other major Toulmin Argument Pattern elements were found other than qualifiers. Even though a lot of claims were made, the student only gave supporting reasons for 5 of them. Even then, the reasons were not elaborated on and were no longer than one sentence each. In the post-intervention essay (Figure 3), the student made 5 claims and provided reasons for all of them. The reasons provided were also better elaborated on as compared to the pre-essay. Two counts of personal experiences were coded as evidence in the form of grounds. Two self-rebuttals were also coded in the essay. In summary, the post-essay was richer in terms of Toulmin’s Argument Pattern due to the display of a more dialogic tone through the self-rebuttals and having better elaborated reasons. The student also understood the need to back up his claims with grounds and offered two of his own personal experiences as evidence. 7. Conclusion The method for analyzing student argumentation described in this paper provides for a richer and more complete analysis in that it takes into consideration the speaker’s stance and the speaker’s strategy for developing the content in the argumentation as well as the criteria for the layout of the argument. It is our purpose to further develop this framework into an analytical tool for assessing students’ argumentation. Due to space limitations here, an exhaustive analysis at this stage is not possible. This will be provided in the presentation format. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

Kuhn, D. (1991). The Skills of Argument. Cambridge University Press. Kuhn, D. (2005). Education For Thinking. Harvard University Press. Osbourne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argumentation in school science. Journal of Research in Science Teaching, 41(10), 994-1020. Cho, K. L. & Jonassen, D. H. (2002). The effects of argumentation scaffolds on argumentation and problem solving. ETR&D, 50 (3), 5–22. Dewey, J. 1991. (original publication 1910). How we Think. Prometheus Books. Buffalo, NY. (Originally published by: D.C. Heath, Lexington, MA) Ennis, R. H. (1996). Critical Thinking. New Jersey: Prentice Hall. Cederblom, J., & Paulsen, D. W. (2001). Critical Reasoning. Understanding and Criticising Arguments and Theories. Belmont, CA: Wadsworth/Thomson Learning. Kirschner, P. A., Buckingham Shum, S. J., & Carr, C. S. (2003). Visualising argumentation: software tools for collaborative and educational sense-making. London: Springer Verlag. Bakhtin, M. M. (1986). Speech genres and other late essays. C. Emerson, & M. Holquist (Eds.); V. W. Mc Gee, Trans.; pp 259-422. Austin: University of Texas Press. Gee, J. P. (2001). Language in the science classroom: Academic social languages as the school based literacy. Paper presented at the Science Literacy Connections Conference, Baltimore, MD. Toulmin, S. (1958). The uses of argument. Cambridge, UK: Cambridge University Press. Toulmin, S., Rieke, R., & Janik, A. (1984). An Introduction to Reasoning. New York: Macmillan. Halliday, M. A. K., & Matthiessen, Christian M. I. M. (2004). An Introduction to Functional Grammar. New York: Oxford University Press. Lemke, J. L. (1990). Talking science: Language, learning and values. Norwood, NJ: Ablex. Ramage J., D., Bean, J.,C., and Johnson, J. (2004). Writing arguments: A rhetoric with readings. Pearson Education Inc. Lemke, J. L. (1998). Analysing Verbal Data: Principles, Methods, and Problems. In K Tobin & B Fraser, (Eds). International Handbook of Science Education (Kluwer)

148

O.C. Brudvik et al. / Assessing the Impact of a Structured Argumentation Board

Appendix – Student Essays

Claim Claim

Claim

Reason

Claim Claim Claim Claim Claim Claim

Reason Reason

Reason Reason

Figure 1: Pre-intervention Essay Toulmin Argument Pattern Analysis

Claim

Reason Ground

Claim Rebutt

Claim

Reason

Reason Ground

Claim Rebutt

Claim

Reason

Reason

Figure 2: Post-intervention essay Toulmin Argument Pattern Analysis

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

149

Incorporating Online Discussion in Classroom Learning: A New Strategy Wenli Chena, Chee Kit Looia Learning Sciences Laboratory, National Institute of Education, Singapore [email protected] [email protected]

a

Abstract: This paper explored the advantages and disadvantages of in-class online discussion by implementing online discussion in both in- and off-classroom settings for a professional development course, and by evaluating the strengths and weaknesses of such practices. Our results showed that the advantages of in-class online discussion are providing more perspectives, equalizing the participation of discussion, promoting cognitive thinking skills and in-depth information processing, and the ease of circulating and archiving files and documents. The disadvantages with in-class online discussion include lack of face-to-face interaction, the necessity to use more time, and the inefficiency accruing from the realization that not all learners will read the online postings. Keywords: online discussion, in-class discussion, face-to-face instruction

Introduction Research on online discussion in learning and education has proliferated recently, many of which documented the advantages of online discussion in teaching and learning: 1) increasing flexibility by removing time and space restrictions of the typical classroom setting (Curtis & Lawson, 2001; Harasim et al., 1995; Henri, 1992). 2) promoting discussion and collaboration among learners (Gay, Sturgill, Martin, & Huttenlocher, 1999; Pena-Pérez, 2000), 3) equalizing participation of learners (Hsi & Hoadley, 1997), 4) allowing learners to have more time to think “deeply” before posting messages in a discussion area (Moore, 2002), 5) providing students with an opportunity to see from different perspectives which may foster new meaning construction (Heller & Kearsley, 1996; Ruberg, Moore, & Taylor, 1996), and 6) recording learners’ s thoughts, reflection and debate by automatically saving the messages posted in the discussions (Hara, Bonk & Angeli, 1998). Despite the extensive research about online discussion in teaching and learning, most of them focus on exploring the use of online discussion in an off-classroom setting, as a main instruction method in distance learning, or as a supplementary teaching and learning method to face-to-face instruction. There is hardly any practice documented and research conducted on online discussion in an in-classroom setting. Is online discussion in-class a useful new teaching and learning strategy, or an impractical idea? What are the advantages and disadvantages? This study tries to answer these questions by implementing online discussion in both in- and off-classroom setting in a professional development course, and evaluating the strength and weakness of the practice towards the possibility of in-class application.

150

W. Chen and C.K. Looi / Incorporating Online Discussion in Classroom Learning

1. Instructional Design and Description of Online Discussions The online discussion group examined in this study consists of 16 Heads of Departments of Information Technology (IT HoDs) from a few Singapore schools who attended the professional development course at National Institute of Education, Singapore. Eight face-toface sessions of the course were conducted once a week, each lasting 3 hours. Online discussions were integrated both in-class and off-class. A typical teaching and learning cycle is the following: 1. Instructor briefly introduce topic at the end of previous session Within Class F2F 2. Elicit learners’ previous knowledge of target topic on XMail Off class Online 3. Instructor’s PowerPoint presentation which synthesizes the Within class F2F various ideas of the learners manifested in their online messages 4. Learners individually or form groups to post reflections on issues raised by the instructor (e.g. how their understanding of the Within class Online issue can be translated to practical implications in their schools) 5. Group presentation by learners Within class F2F 6. Instructor introduces the new topic of next session and ask the Within class F2F learners to post reflections on the new topic 7. Learners continue to discuss the old topic as well post their Off Class Online ideas on the new topic The software used in creating the online discussion group is XMail, an email-based tool that is easy to use and possesses additional capabilities of being able to automatically organize and make available information generated by group interactions. The online discussion group was created on January 2006 before the course started. Members included 16 learners and 4 instructors of the course. The messages analyzed in this study were posted during Jan 25 and Mar 20, eight weeks time overlapping with the face-to-face lectures. Overall there were 269 messages posted during the above-mentioned period. Among them about half (136) were posted during face-to-face lectures and the other half off-class (133). On average, there were 25 messages in each subgroup.

2. Methodology To examine several dimensions of incorporating online discussion in classroom, three research methods were applied in this study: a content analysis to analyze the nature and quality of the in-classroom discussion, classroom observations to gather information about the learners’ classroom performance which may affect their online discussion behavior, and an indepth interview to get the learners’ perceptions, opinions, and suggestions on the new strategy. These three methods complement each other to get a full picture of the participation pattern and effectiveness of in-class online discussion. For the content analysis, Henri’s (1992) framework was adapted. The framework analyzes data in five broad dimensions: participation, social, interaction, cognitive and metacognitive aspects. The unit of data for analysis was each “idea” as denoted in a message, instead of the message as a whole. The inter-rater reliability test result showed that the percent of agreement between the two coders was 100% on participation dimension, 69% on

151

W. Chen and C.K. Looi / Incorporating Online Discussion in Classroom Learning

interactive dimension, 79% on social dimension, 63% on the five-level cognitive dimension, 84% in the depth of information processing and 83% on metacognitive dimension. When examining the advantages and disadvantages of in-class online discussion, we compare it with its two counterparts - off-class discussion as well as in-class face-to-face discussion. The results from different methods were presented when and where appropriate.

3. Advantage of In-Class Online Discussion 3.1 The participation of discussion is equalized in class by online discussion In this study we compared the number of messages posted by different learners in class and off class, which was an indicator of chances the learners got to express themselves. As shown in Figure 1, all learners participated in the online discussion in-class, and the number of postings by each learner was similar. On the other hand, the difference in the number of postings off-class by different learners was quite apparent. One learner posted 29 messages off-class whereas two learners never posted off-class. In the contrast, during the face-to-face class discussion time, when the instructor asked learners to share their ideas, three to five learners really spoke in the face-to-face classes. Furthermore, during the 3-hour session, normally less than 10 learners expressed their ideas to their peers. The researcher found that “struggle the turn” to speak out was not an issue. Each time the oral discussion ended because no more learners would like to share. The possible reason is that some learners were “reticent” to speak in front of others, while some others were inactive to think when their peers were sharing. class

30

In-class Off-class

25

Count

20

15

10

5

0

e rg eo G a ili ec C ria lo G on ils W es m Ja e ol ic N s ni en D e in la E es m Ja an us S a el ng A

sa Li e hi op S y nn Je

x le A ny en K

Individual Learner

Figure 1. Number of postings by each learner, in-class and off-class Apparently in-class online discussions can equalize learners’ participation to a great extent, when most of them were task-oriented so that learners felt more obliged to post and they knew the instructors would synthesize their ideas later. The equalization of in-class online discussion may: 1) benefit those who are “reticent” to speak in front of others, 2) push those who are inactive in thinking to think and post more frequently.

152

W. Chen and C.K. Looi / Incorporating Online Discussion in Classroom Learning

The equalization may lie in another aspect that learners are more likely to judge their peers from the knowledge reflected in the postings instead of other factors which may be linked with their face-to-face speaking. The evidence of this statement is that the three project leaders were all active contributors of in-class discussion, and their postings were read and replied by other learners more frequently, which is an indicator of good quality of the postings. 3.2 Learners engage in extensive writing in online discussion during class In each face-to-face class, learners were given 15-30 minutes to post their thinking and reflection in the online discussion area. They were also encouraged to post even after class. The extensiveness of writing can be measured by the length of messages, which was operationalized by the number of sentences and lines (there are about 25 words per line). Table 1. Independent-Sample t Test of Length of postings and time of Postings (N=223) Number of Sentences Number of Lines

In-class Off-class In-class Off-class

N 129 94 129 94

Mean 10.64 8.14 12.40 8.68

t 1.973* 1.986*

Note: *p<.05.

Independent-sample t test was conducted to compare the length of messages posted inclass and off-class (Table 1). The result showed that the in-class messages have 2.5 more sentences than off-class messages (t = 1.973, p < .05), and 3.7 more lines (t = 1.986, p<.05). The learners engaged in more extensive writing in-class than off-class. 3.3 In-class online-discussion provides more perspectives of ideas When the in-class online discussion and the in-class oral discussion were examined in detail, it was observed that the online discussion provided more perspectives than oral discussions. For example, during oral discussion, it was very rare that learners had heterogeneous opinions towards certain issues. In one session there was an oral discussion on how Ministry of Education (MOE) could help schools do technology planning. When the instructors and MOE staffs sought oral opinions from the learners, only four learners shared their opinions, mainly focusing on the workable guide of technology planning, incubators schools as models to follow, and more sharing and communication needed. However, in online discussions, almost every learner stated their opinions, and the perspectives were expanded a lot to the financial matters and the need for experienced MOE staffs. As shared by one of the learner, Dennis, “You can see the different types of reflections online. And it’s really a variety. It’s important that we can see or hear everyone’s perspective. We need not accept everybody’s views. But it’s important that we can get a sense.” The heterogeneous perspectives of in-class online discussion may be due to the following two reasons: 1) in oral discussion, learners may be affected by other out-spoken ideas into homogeneity which may discourage them to express their ideas, and if she feels others had already expressed the same or similar ideas as her, then she may not want to use up

153

W. Chen and C.K. Looi / Incorporating Online Discussion in Classroom Learning

time to make a similar point, and 2) the Asian culture does not encourage people to openly challenge others as such practice is considered an impolite and sometimes inappropriate. 3.4 In-class online discussion contains more in-depth clarification and inference skills Scholars have long investigated the connections between writing and thinking (e.g., Vygotsky, 1962) - it is through the actual process of writing our thoughts and working them over that we really come to understand. The written record allows for in-depth clarification, revision and encourages self reflection and inference and these are important learning strategies for developing an understanding of new concepts. This study has the similar finding that online discussion, when compared to oral discussion, contains high-order cognitive thinking. In the in-depth interview, one of the online discussion participants of the study, Dennis, shared his perception towards online discussion, “I think online discussion will improve the cognitive development. Because when we type, tendency is that they have a bit more time to think through. …And when you type, you will tend to be more conscious of what you type. It sounds like you have to think twice.” A Chi-square test was conducted to examine the relationship between the cognitive skills manifested in the postings and time of postings (Table 2). The results indicated that paragraphs posted in class demonstrated more in-depth clarification (43%) than posted offclass (31%), and the in-class postings also contains higher percentage of inference skills (20%) than off-class postings (6%). The percentage of elementary clarification in paragraphs posted off-class (40%) was much higher than that of the paragraphs posted in-class (14%). Table 2. Chi-Square Analysis between Cognitive Skills vs. Time of Posting (N=644)

In-Class Off-Class

Elementary clarification 68 14.3% 77 40.5%

Category of Cognitive Skills In-depth Inference Judgment clarification 205 96 51 43.2% 20.3% 10.8% 59 12 19 31.1% 6.3% 10.0%

Strategy 54 11.4% 23 12.1%

ChiSquare

63.977**

Note: ** p< .01.

3.5 In-class online discussion contains in-depth information process The existing studies showed that one of the advantages of online discussion over face-to-face oral discussion is that it encourages a deeper level of thinking (Jusri & Lim, 2003). Information processing is closely related to cognitive skills. In the content analysis, this measure was used to classify the responses into the categories of superficial and in depth processing. Messages classified as evidence of surface level processing involved mostly examples where participants contributed information about extra resources without elaboration. Messages demonstrated deeper levels of processing involved relating new information to their experiences, critically evaluating ideas, and exploring strategies. Chi-Square test was employed to compare of level of information processing between in-class postings and off-class postings (Table 3). The analysis revealed that that paragraphs

154

W. Chen and C.K. Looi / Incorporating Online Discussion in Classroom Learning

posted within class had 20% more in-depth info processing than those posted between classes (Pearson Chi-Square = 28.003, p < .01). Table 3. Chi-Square Analysis between Depth of Info Processing vs. Time of Posting (N=644)

Within Class Between Class

Level of Info Processing Surface Processing In-depth Processing 126 348 26.6% 73.4% 91 99 47.9% 52.1%

Chi-Square

28.003**

Note: ** p< .01.

In addition, comparing with oral discussion in class, the learners felt that the online discussion is more in-depth. As shared by Gloria, “I think that online discussion will be more in depth than oral discussion. Because we would have thought about it before we typed. After some thinking, when you type it out, the content will be more substantial. But when we talk, it is sometimes quite impulsive.” The result reaffirmed that during class, learners are more likely to think “deeply” than oral discussion and off-class postings. 3.6 Ease of circulating and archiving files and documents In-class discussion provides a permanent record of learners’ thoughts and reflection. This advantage is very valuable compared with in-class oral discussions. During class, instructors used the online discussion platform to circulate slides and reading materials, and the learners also send documents to instructors and peers. On average, each learner circulated five documents, including power point slides, mind map, word document, PDF file, and video files. In the in-depth interviews, many learners mentioned the advantage of online discussion in circulating and archiving files and documents - “I think the biggest advantage of online discussion would be archiving. Like what I discussed in January will still be there. You can keep it in a way that you can go back to refer to that info that was said” (Lisa); “Whatever we have posted is also stored in the server itself so we can always refer to them” (Gloria). Dennis’s experience indicates that the recording feature of online discussion is very useful to the learners, especially for those who missed classes: “I missed the Classroom of the Future session because of the national service. But the good thing is that my friends posted their comments and reflections online. This was very useful. It’s a recorded black and white and I can see. Both the learner and absentee can see. ”

4. Disadvantages of In-Class Online Discussion 4.1 In-class online discussion lacks interaction The first disadvantage identified is that in-class online discussion lacks interactions, as compared to both oral discussion and off-class discussion. In in-class oral discussion, learners and instructors talk face-to face and interact actively. However, most of the online postings are independent postings without replying and commenting on others’ postings. Chi-square

W. Chen and C.K. Looi / Incorporating Online Discussion in Classroom Learning

155

test was employed to compare the level of interaction between in-class postings and off-class postings (Table 4). The result showed that in-class postings are less likely to comment or response to others’ posting than off-class postings (Chi-Square = 15.658, p < .01). Table 4. Chi-Square test of Level of Interaction and time of posting (N=223) Independent Statement In-class 109 84.5% Off-class 63 67.7% Note: **p<.01.

Level of Interaction Implicit commentary/ Explicit commentary/ response response 15 5 11.6% 3.9% 11 19 11.8% 20.4%

Chi-Square 15.658**

The reason why in-class online postings were less interactive than off-class postings is that learners may interact more orally when they were sitting in the classroom face-to face. The lack of interaction is one of the key disadvantages perceived by the learners. As told by Dennis, “When we are side-by-side but we cannot talk to one another and we have to talk through the computer, it will reduce the kind of social interaction. Therefore we must balance the amount of oral discussion and the amount of typing to discuss. … I am not comfortable with typing in class when we don’t even know each other. Once we know, then we will know how to respond, or to use specifically, the way to respond. That is definitely more useful.” 4.2 In-class online discussion needs more time and may be inefficient Although learners have more time to think during typing, the typing itself takes time. Too much online discussion in class may slow the progress of the class. In the class, listening is to some extent a compulsory practice for interacting with instructors and peers. It would be a problem to assure that every participant in online discussion reads all the messages, although many of them contain high-order cognitive thinking and in-depth information processing.

5. Conclusion The learners engaged in extensive writing in online discussion during the class when the learners shared their reflections, deeper thinking and experiences, even though the participation was not an assessment criterion of the course. In-class online discussion provided more perspectives as learners were able to think independently and were less reluctant to share their real feelings which might be different from the view held by others. Moreover, with in-class online discussion, the participation of discussion was equalized in two aspects – 1) everybody has an equal chance to participate and fully express his/her ideas, 2) learners judge their peers by the knowledge they articulated more than other factors, such us social status, external appearances, accent or charisma. As a result, the class became a more democratic learning environment and those “reticent” to speak in front of others were encouraged to express themselves, whereas those inactive in thinking were pushed to think and post more frequently. In-class online discussion was also found to contain more in-depth clarification, and inference skills. In addition, the in-class online discussions demonstrated

156

W. Chen and C.K. Looi / Incorporating Online Discussion in Classroom Learning

deeper levels of processing that involved relating new information to the participants’ experiences, critically evaluating ideas, and exploring strategies. Several disadvantages with in-class online discussion were also revealed by the study. First, it lacks interaction because most of the online postings were task-oriented and independent postings without replying and commenting on others’ postings. Second, too much online discussion in class may slow the progress of the class. Third, in-class online discussion does not assure every learner will read the online postings because reading online discussion was not a compulsory practice. While in-class online discussion offers the potential of encouraging participation and creating deep learning opportunities, making this happen requires good learning design and organization. Courses that include in-class online discussions as a supplement to oral discussion need to carefully integrate this activity into the overall course design, so students see it as integral to the class and not as a disassociated activity. To address the issue of lack interaction, in-class online discussion can be introduced after the learners get to know each other. The topics for online discussion during class should be carefully chosen. In this study, it was found that those novel topics (such as “Community of Practice”) and those practical topics closely relevant to their work (such as “Technology Planning”) attracted more highquality postings. In addition, as not all learners feel comfortable about in-class online discussion, and not all the postings were “heard” by all class members, the effectiveness of inclass online postings may vary from learner to learner. This design could promote selfdiscipline and requires students to take more responsibility for their own learning. References [1] Curtis, D. D., and Lawson, M. J. (2001) Exploring collaborative online learning. Journal of Asynchronous Learning Networks, 5, 1, 21-34. [2] Gay, G., Sturgill, A., Martin, W., and Huttenlocher, D. (1999) Document centered peer collaborations: An exploration of the educational uses of networked communication technologies. Journal of ComputerMediated Communication, 4, 3, unpaged. [3] Hara, N., Bonk, C., and Angeli, C. (1998) Content analysis of online discussion in an applied educational psychology course. Instructional Science, 28, 2, 115-152. [4] Harasim, L., Hiltz, S. R., Teles, L., and Turoff, M. (1995) Learning Networks: A Field Guide to Teaching and Learning Online. The MIT Press, Cambridge, MA. [5] Henri, F. (1992) Computer conferencing and content analysis. In A.R. Kaye (Ed.), Collaborative Learning through Computer Conferencing: The Najaden Papers, 115–136. Springer, NY. [6] Heller, H., and Kearsley, G. (1996) Using a computer BBS for graduate education: Issues and outcomes. In Z. Berge and M. Collins (Ed.), Computer-Mediated Communication and the Online Classroom. (Vol. III: Distance learning, pp. 129-137). Hamptom Press, NJ. [7] Hsi, S., and Hoadley, C (1997) Productive discussion in science: Gender equity through electronic discourse. Journal of Science Education and Technology, 6, 1, 23-36. [8] Moore, M. G. (2002) What does research say about the learners using computer-mediated communication in distance learning? American Journal of Distance Education, 16, 2, 65-81. [9] Pena-Pérez, J. (2000) Participation, interaction and meaning construction in a university-level course using a computer bulletin board as a supplement to regular class discussions. A case study. Unpublished doctoral Dissertation, Dept. of Education, Cornell University, Ithaca, NY. [10] Ruberg, L. F., Moore, D. M., and Taylor, C. D. (1996) Student participation, interaction, and regulation in a computer-mediated communication environment: A qualitative study. Journal of Educational Computing Research, 14, 3, 243–268. [11] Vygotsky, L. S. (1962) Thought and Language. The MIT Press, Cambridge, MA. [12] Jusri, T., and Lim, G. (2003) Significance of online teaching vs. face-to-face: Similarities and difference. Paper presented at E-LEARN 2003, the World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education. Phoenix, Arizona, USA, November 7-11.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

157

Using Agents for Enhancing Learning Effects in an Advanced Discussion Forum Yuejun ZHANG, Kinshuk, Øyvind SMESTAD, Jingyu YANG and Lynn JEFFERY Department of Information Systems, Massey University, New Zealand [email protected] Abstract: Software agents have long been used in educational environments to provide learning support. This paper describes an agent-based approach in a value-added online discussion forum system to facilitate enhanced interactions. The system deploys a multi-agent architecture in which four types of agents collaborate with one another to undertake tasks delegated by teachers and students. This multi-agent architecture has enabled several tertiary educational institutions in New Zealand to build a shared discussion platform that integrates their current learning management systems. Based on the monitoring of student activities, the system presents many advanced functionalities such as formation of formal answers to repeatedly raised questions (FAQs), delivery of useful messages to appropriate individuals, and organization of study groups with compatible students. Keywords: Software agents, multi-agent systems, discussion forum, interaction

1. Introduction Educational research has long identified interaction or knowledge sharing among students as a key success factor in learning [1, 2, 3]. Online Web based discussion forums provide a means to potentially allow students in different physical locations to interact with each other. Conventional online discussion forums, however, present several issues of concern that need to be addressed. The most significant issue in existing discussion forums is the lack of a mechanism to effectively monitor student activities. Usually, knowledge sharing and learning takes place adequately in an online discussion environment, but it might be appealing if the system could monitor students’ queries, identify their needs, and accordingly provide individualized help in an autonomous way. Sometimes students may even encounter difficulties, do things wrong, or fail to actively participate. In these situations, intervention from either a human tutor or a machine entity is necessary. Unfortunately, this issue has not yet been well tackled due to the lack of face-to-face opportunities which are otherwise easily found in a traditional classroom. In the case of online discussion forums, the tutors do not have direct control over the student activities. In fact, tutors may not even be available when a student needs help. 1.1 Research purpose In this paper, we describe an agent-based approach to this issue in a value-added online discussion forum system, called electronic Question & Answer Knowledge Environment (eQuake). The underlying idea is straightforward. On one hand, agents can be used to

158

Y. Zhang et al. / Using Agents for Enhancing Learning Effects in an Advanced Discussion Forum

interact with other entities on behalf of students; and on the other hand, agents can also be used to monitor student activities on behalf of other entities. Here, “other entities” indicate human entities including students and teachers as well as machine entities such as other agents. The agents work autonomously and silently in the background, collaborating with each other to complete various tasks. When a message is posted to the forum system, for example, an agent will analyze its content and accordingly take correct actions. For another example, when the tutor is not available online, a tutor agent may take over the role. eQuake is designed as a multi-agent architecture, in which four types of agents collaborate with one another to reach the system goals. This multi-agent architecture has enabled several tertiary educational institutions in New Zealand to build a shared discussion platform that integrates their currently used learning management systems (LMS). Based on the monitoring of student activities, the system presents many advanced functionalities such as formation of formal answers to repeatedly raised questions (FAQs), delivery of useful messages to appropriate individuals, and organization of study groups with compatible students. 1.2 Paper outline The outline of the rest of the paper is as follows. We begin with section 2 by giving a brief description of agent concept and related work in the educational area. In section 3, we check the rationale for the agent-based approach in eQuake, identifying the appropriateness of this approach for the problem domain. Then we present our multi-agent architecture in section 4, with a focus on the functionalities of each kind of agents and the collaborations among them. In section 5, the implementation aspects of eQuake are illustrated by focusing on the system architecture and the current deployment across multiple tertiary institutions in New Zealand. Finally, we conclude the paper in section 6, which also discusses some future research work. 2. Agents and related work 2.1 Agent concept and properties There is no agreement in the literature on a unified definition for the term “agent”. Just as Van Dyke Parunak [4] pointed out, for some, an agent is a software piece as long as it can travel over a network; for others, an agent represents a module that takes action on behalf of a human user. For still others, agenthood means a certain minimal level of intelligence, the use of a specified inter-agent language, or the ability to manipulate explicit models of beliefs, desires, and intentions. We present here the definition cited from the work of Kinshuk, Hong and Patel [5]: “An agent is a computational entity that acts on behalf of other entities in an autonomous fashion, performs its actions with some level of proactivity and/or reactiveness and exhibits some level of the key attributes of learning, cooperation and mobility.” Agents exhibit some properties upon which a common consensus has been reached. These properties include [6]: autonomy, social ability, responsiveness, and proactiveness. In addition, several other characteristics are also potentially desired for agents: adaptability, mobility, veracity, and rationality.

Y. Zhang et al. / Using Agents for Enhancing Learning Effects in an Advanced Discussion Forum

159

2.2 Educational agents and related work Software agents have long been applied in educational environments to provide learning support. Educational agents are a class of agents that assist a user in an education-related task. Their roles and advantages have been identified by many researchers such as Whatley [7], Jafari [8] and Johnson [9]. Some benefits which are offered by educational agents and closely relate to the requirements for our eQuake system can be summarized as follows: 1) Agents can perform tasks for students and instructors in a dynamic, personal, and smart teaching and learning environment. 2) Agents can be made to work actively and adapt to students. They exhibit the ability to recognize what the student needs to accomplish, and react to the students’ input. 3) Agents can continuously operate in the background on a student’s workstation and act autonomously to suggest ways in which the student might improve performance. 4) Agents can bridge the divide between time and place. Students may be dispersed and working at times to suit themselves, while the agents keep track of the students’ progress. 5) Agents can bring together students with similar interests or needs into a discussion area where they can receive help on particular problems. 6) Agents can monitor progress, give instruction when needed, help organize students’ work, and provide feedback for tutors. According to Chou, Chan, and Lin [10], educational agents can roughly be divided into two categories. One category presents to users a computer character with human characteristics that facilitate social learning. This category is usually referred to as lifelike or animated pedagogical agents. By using gestures, gaze, facial expressions, locomotion as well as verbal exchanges, animated pedagogical agents may greatly enhance the learning environment. Examples of this category include: Steve [11], Adele [12], Herman the Bug [13], Cosmo [14], WhizLow [15], and PPP Persona [16]. Another category includes pieces of software, which work invisibly within the system by autonomously dealing with delegated tasks and actively interacting with other entities. The agents in eQuake system fall into this category. Some related work in this category includes the agents in I-Help [17], in which the personal agents care for their learners by helping them to discover useful information and/or to find “ready, willing, and able” peer learners who can aid them in overcoming problems. Another example is the context-based information agents [18] that can observe conversations among a community of learners on the Web, interpret the learners’ inputs, and then assess the current context of the session. Further example includes the agents in an effective e-learning model [19], where the Open Forum Agent can parse and analyze the student query, and search the database for answers based on the parsed tokens. All of these agents can, to some degree, monitor student activities or provide adaptive help to students. Agents in eQuake go further in enhancing learning experience by facilitating intelligent learner support and cross-institution interactions. 3. Rationale for using agents in eQuake The rational for using agency technology in eQuake can be demonstrated from two aspects: agents’ properties and eQuake’s attributes. On one hand, agents have three main characteristics presented earlier in the definition of the term:

160

Y. Zhang et al. / Using Agents for Enhancing Learning Effects in an Advanced Discussion Forum

1) Autonomous – be independent and make their own decisions without the need for any external control 2) Pro-active – persistently pursue goals 3) Reactive – respond in a timely manner to changes in their environment These properties of agents match very well with the enhanced functionalities of eQuake system. eQuake is a web-based system in which students from multiple institutions engage in a student-centered community of learning to share and exchange study and subject-related problems/solutions, join study groups and receive expert advice. The system constantly monitors the student interaction and creates profiles of students based on their interaction with other students. It tags significant or repeated issues or queries raised by students for teachers who may formulate responses in the form of FAQs or respond individually to certain students. Once an FAQ item is added, the system, based on students' profiles, identifies those students who may benefit from that FAQ and advices them accordingly. Next time, when a student raises that particular issue, the system automatically intercepts the query and redirects the student to the FAQ. Students are then able to decide whether their query has been answered or if it should go to other students for further responses. Based on student profiles, the system is also able to propose study groups among compatible students, by taking into consideration their knowledge levels and learning styles. The key point to reaching these functionalities is that student activities and any other occurrences are sensed and timely reactions are undertaken. The “reactive” property of agents is a perfect fit for this requirement. Since eQuake is an online distributed Web-based system where both students and teachers interact with the system asynchronously, the system should be able to undertake tasks without instructions or supervisions from external entities. This situation requires system autonomies, and thus the “autonomous” property of agents has a significant role to play. It is also expected that the approach to tackling the stated problems be highly robust, and the system should continue to try to achieve the goals despite failed attempts. Regarding this aspect, the “pro-active” characteristic of agents is just what we want. On the other hand, Wooldridge [20] pointed out some of the system attributes that indicate the appropriateness of an agent-based solution. These attributes are: 1) The agent environment is open, or at least highly dynamic, uncertain, or complex. 2) Agents are a natural metaphor. 3) Data, control, or expertise is distributed in several locations. 4) Legacy systems are components of the whole system. eQuake possesses all these attributes, and therefore agenthood is the most suitable choice. Firstly, eQuake is a highly dynamic and complex system. As a discussion forum, eQuake allows students to post messages and interact with each other in an unrestricted manner. We cannot have a direct control on the student actions, and we never know what a student will do in the future. Secondly, agents are a natural metaphor in eQuake. It is reasonable to create a student proxy agent for each student and a tutor proxy agent for each teacher on behalf of them to undertake the interactions. Each agent is adaptable to its client, thus personalized help as well as adaptive learning is possible. Thirdly, data and control are distributed in several locations. As seen in the system architecture in section 5, eQuake is deployed across several institutions, each of which runs a local Web server and a database server of its own. Fourthly, legacy components, or in our case, heterogeneous LMSs, are used in eQuake, as seen also in section 5. To make these heterogeneous LMSs collaborate with one another and communicate with eQuake, one logical solution is to wrap them with an agent layer.

Y. Zhang et al. / Using Agents for Enhancing Learning Effects in an Advanced Discussion Forum

161

4. Multi-agent architecture in eQuake 4.1 Architecture eQuake is a multi-agent system, in which four types of agents, namely tutor proxy agent (TPA), student proxy agent (SPA), target selection agent (TSA), and query monitoring agent (QMA), collaborate with each other to achieve the system goals. The multi-agent architecture is shown in figure 1. Agents as Web Services: (Application layer)

Presentation layer: LMS User DB

Global DB

TPA Notification

LMS Plug-In

Data layer:

TSA Forum display

WS

Dispatcher

PA QMA

WS

Instance config. data

WS

DataService Datamodel (Hibernate)

SPA

Legend:

PA – Proxy Agent TPA – Tutor Proxy Agent SPA – Student Proxy Agent TSA – Target Selection Agent QMA – Query Monitoring Agent

Data Stores DB

WS – Web Service DB – Database LMS – Learning Management System

Figure 1. Multi-agent architecture in eQuake

Logically, the eQuake system is divided into three layers: presentation layer, application layer, and data layer. The presentation layer includes the existing LMS and the local database running in each institution, which is responsible for all interactions with users. This layer also includes a plug-in interface which facilitates the communication between the existing LMS and the agents in eQuake via the Web service (WS) protocol. The data layer, as it is named, manages data storage and data access for eQuake, enabling the integration of the distributed databases. The four types of agents are seated in the application layer, which accesses the data layer and provides services to the presentation layer. Each of the agents manages to complete its own tasks, and seeks for co-operations from other agents if needed. teacher 9 create FAQ

Q/A 5 retrieve

students

6 retrieve

TSA

TPA 11 notify 8 notify if repeated query

12 forward new FAQ

1 query

2 query

SPA

7 related Q/A & FAQ 13 new FAQ & rating

10 save FAQ

FAQ

QMA

3 query type 4 update

14 update

Student profile

Figure 2. Collaborations among agents when a query is posted

162

Y. Zhang et al. / Using Agents for Enhancing Learning Effects in an Advanced Discussion Forum

To illustrate the collaboration among agents, figure 2 shows a typical scenario when a student posts a query (e.g. a question) on the forum (the numbers in brackets in following discussion are those listed for activities in figure 2). First the student proxy agent (SPA) sends a request to the query monitoring agent (QMA) for analysis on the query content (2). The QMA then returns the analysis result to the SPA (3), which in return updates the student profile (4). It also retrieves existing question/answer pairs (5) as well as already available FAQs (6) which are both closely related to the current query. The SPA displays the retrieved contents to the student (7). If the QMA finds out that the query is a repeatedly raised question and the number of repetitions has reached a predefined threshold value, it then notifies the teacher proxy agent (TPA) of this situation (8). The latter consequently prompts the teacher to formalize an FAQ based on the query (9), and saves this FAQ to database (10). At the same time, the TPA notifies the target selection agent (TSA) of the new incoming FAQ (11). Upon receiving this notification, the TSA chooses those students, according to their profiles, who may be interested in this FAQ and then forwards this FAQ to the corresponding SPA (12). The SPA then delivers the FAQ to the student (13). 4.2 Functionalities of agents Although design goals of eQuake are achieved by the collaboration among all agents, each type of agent has its specific functionalities, which are briefly summarized in the following discussion. The student proxy agent is responsible for modeling student profiles, undertaking interactions with other students and the system on behalf of the student by enabling message posting, searching/forwarding related message or answers to frequently asked questions (FAQs) to the student and so on, and monitoring the student’s activities in eQuake by determining the nature of the student’s posted message, monitoring the student’s participation frequency, identifying if the student has a poor understanding of a topic, and so on. The tutor proxy agent is, on the other hand, in charge of modeling tutor profiles and helping the tutor interact with students and other agents. Examples of the interactions include enabling the creation of FAQ items, informing the target selection agent of a new FAQ, and forwarding answers to particular students. The target selection agent mainly takes two responsibilities. The first is to select students whose profiles match a new incoming FAQ, and the second is to select students with compatible profiles so that optimal study groups can be organized. Finally, the query monitoring agent is focused on a semantic analysis of the content of every incoming message posted by students or coming from the database. Based on this analysis, it can identify repeated questions and provide services to other agents. 5. Implementation 5.1 System architecture and deployment A local eQuake server, with the above-mentioned multi-agent architecture, is installed for each institution joining in the shared forum. A user interface or plug-in part is embedded in the institution’s existing LMS, so that the LMS can communicate with the local eQuake server. A local database is maintained for each local eQuake server to store posts by local users and track data for these users. eQuake servers (the actual underlying agents) located at all institutions communicate with each other using web services to complete the delegated

Y. Zhang et al. / Using Agents for Enhancing Learning Effects in an Advanced Discussion Forum

163

interaction tasks. A central server is also used to keep a list of available eQuake forums. This is especially useful for simply connecting new institutions to the network. Figure 3 shows the existing deployment of the system in three different institutions. Each institution runs an instance of eQuake on a local server that interacts with their existing LMS server. When getting data from outside the local server, requests are sent to all connected instances. The data structure is a homogeneous distributed database structure as the type of data stored at each physical site is identical. As stated above, a small portion of the data is kept in a central location, where data about the currently available eQuake forums is maintained. For the purpose of performance and reliability, the centrally kept data is also duplicated in each running instance of the system, as it rarely changes but is often queried.

Figure 3. Example deployment with local databases 5.2 Integration with heterogeneous LMSs As the user interface of our system is based on the Moodle (http://www.moodle.org/) forum module, it can easily be installed to a Moodle system as a new module/plug-in. Another system with which we have tested the integration of eQuake is the proprietary WebCT system (Campus edition 4.1) (http://www.webct.com/). We use a simple plug-in script to connect and launch our system inside the main frame of WebCT. The reason for the choice of Moodle and WebCT as the existing system for testing the integration with eQuake is that they represent a typical example of open source and closed source LMSs respectively, and because they both are being used by institutions participating in the project. 5.3 Evaluation A preliminary user evaluation has been carried out by system developers, project participants, teachers and students at collaborating institutions. The initial test results show a very promising prospect of the agency approach in eQuake. Through the testing process,

164

Y. Zhang et al. / Using Agents for Enhancing Learning Effects in an Advanced Discussion Forum

the development team iteratively received feedback on issues related to the functionality and the usability of the system and implemented improvements. Currently a formal user evaluation by students and teachers in several New Zealand institutions is being carried out. 6. Conclusion We have introduced a multi-agent approach in an advanced discussion forum system to enhance learning effects and to facilitate intensive interactions among students from multiple institutions. Through the agents’ autonomous work and collaboration, the system can model student profiles, monitor student activities, and provide students appropriate messages as well as individualized help. Preliminary tests show that the agency approach is promising and eQuake is a helpful tool for students in gaining more knowledge and understanding within their areas of study. One future improvement to this system is to include the monitoring of pedagogies. According to different learning styles, knowledge levels, individual preferences, and cognitive abilities of the learner, different pedagogical strategies are chosen manually by human teachers or automatically by the system. The system should be able to detect the selected pedagogies and behave properly to the students so that the pedagogies can be well executed. In addition, pedagogical strategies are not constant in a learning session. They usually change in line with learner needs. An obvious fact is that a learners’ knowledge level will be improved, for example, from novice to intermediate while the learning process is proceeding. In this case, normally the pedagogical strategies should be changed accordingly. Therefore, the system should monitor the changes of pedagogies, and adaptively modify its behavior to the learner. Our next step is to use agents to realize this goal in eQuake. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]

Brookfield, S. D. (1986). Understanding and facilitating adult learning. San Francisco: Jossey-Bass. Slavin, R. E. (1983). Cooperative learning. New York: Longman. Whitsed, N. (2004). Learning and teaching. Health Information & Libraries Journal, 21(1), 74-78. Van Dyke Parunak, H. (1998). Practical and industrial applications of agent-based systems. Retrieved November 4, 2005 from http://agents.umbc.edu/papers/apps98.pdf. Kinshuk, Hong, H., & Patel, A. (2002). Adaptivity through the Use of Mobile Agents in Web-based Student Modelling. International Journal of E-Learning, 1(3), 55-64. Wooldridge, M. and Jennings, N. R. (1995). Intelligent agents: theory and practice. The Knowledge Engineering Review, 10(2), 115-152. Whatley, J. (2004). Achieving virtual teamwork using software agents. Proceedings of the Fourth international Conference on Networked Learning 2004, 696-702. Jafari, A. (2002). Conceptualizing intelligent agents for teaching and learning, EDCAUSE Quarterly, 25(3), 28-34. Johnson, W. L. (1998). Pedagogical agents. Proceedings of the ICCE’98, vol. 1, 13-22. Chou, C. Y., Chan, T. W., Lin, C. J. (2003). Redefining the learning companion: the past, present, and future of educational agents. Computers & Education, 40(3), 255-269. Rickel, J. and Johnson, W. L. (1997). Steve: an animated pedagogical agent for procedural training in virtual environments. Proceedings of Animated Interface Agents: Making Them Intelligent, 71-76. Johnson, W. L., Shaw, E., Marshall, A., and LaBore, C. (2003). Evaluation of user interaction: the case of agent Adele. Proceedings of the 8th International Conference on Intelligent User Interfaces, 93-100. Lester, J. C., Stone, B. A., and Stelling, G. D. (1999). Lifelike pedagogical agents for mixed-initiative problem solving in constructivist learning environments. User Modeling and User-Adapted Interaction, 9, 1-44. Lester, J. C., Voerman, J. L., Towns, S. G., and Gallaway, C. B. (1999). Deictic believability: coordinating gesture, locomotion, and speech in lifelike pedagogical agents. Applied Artificial Intelligence, 13(4-5), 383-414. Lester, J. C., Zettlemoyer, L. S., Gregoire, J., and Bares, W. H. (1999). Explanatory lifelike avatars: performing user-designed tasks in 3d learning environments. Proceedings of the 3rd International Conference on Autonomous Agents. Andre, E., Rist, T., and Muller, J. (1999). Employ AI methods to control the behavior of animated interface agents. Applied Artificial Intelligence, 13(4-5), 415-448. Bull, S., Greer, J. and McCalla, G. (2003). The Caring Personal Agent. International Journal of Artificial Intelligence in Education, 13, 21-34. Razek, A. M., Frasson, C., and Kaltenbach, M. (2003). Context - based information agent for supporting intelligent distance learning environment. Proc. of the Twelfth International World Wide Web Conference. Agarwal, R., Deo, A. and Das, S. (2004). Intelligent agents in e-learning. ACM SIGSOFT software Engineering Notes, 29(2). Wooldridge, M. (2002). An Introduction to Multiagent Systems, chapter 16. West Sussex, England: John Wiley & Sons, Ltd.

Emotion & Personality

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

167

Research on Personality Mining System in E-Learning by Using Improved Association Rules Luo Qi1 2, Wu Yanwen1, Wan Liyong 1and Yu Ying1 Department of Information & Technology, Central China Normal University Wuhan, Hubei 430079, P.R.China 2 Department of Information Engineering ,Wuhan University of Science and Technology Zhongnan Branch Wuhan, Hubei 430223, P.R.China [email protected], [email protected] 1

Abstract: To meet the personalized needs of E- learning, an improved association mining rules was proposed in the paper. First, data cube from database was established. Then, frequent item-set that satisfies the minimum support on data cube was mined out. Furthermore, association rules of frequent item-set were generated. Finally, redundant association rules through the relative method in statistics were wiped off. The algorithm had two advantages, the first was that the execution time was short while searching for the frequent item-set; the second was that the precision of the rules was high. The algorithm was also used in personality mining system based on E-learning model (PMSEM).The results manifested that the algorithm was effective. Keywords: (-learning, Association rules, Personality Mining, Data cube

Introduction Nowadays, the importance of E-learning has transferred from how to solve the limit of space-time problem in traditional teaching to build up the personalized learning environment, and offer a kind of personalized knowledge service based on theories such as modern pedagogy, psychology, etc[1]. The learners are in different age level, sex, and social role, their culture ,education background, attention and interest are also exist a great difference. Giving corresponding learning content and tactics to realize teaching learners according to their needs is very difficult [2]. Its basic reason lies in being difficult by obtaining the relations between the learner's personality characteristics and learning behavior patterns accurately, automatically. In this way, it is necessary to mine out the association rules between personality characteristics and learning behavior patterns. The subsequent learners’ personality characteristics are deduced from their learning behaviors by using the rules above. Basing on it, personality learning model is set up and interesting groups are formed .Thus, personalized learning and cooperative learning are realized. At present, many scholars have carried on a great deal of researches on association algorithms, such as such as Apriori algorithm or improved Apriori algorithm FP-tree [3]. The execution time of their algorithms was long while searching for the frequent item-set and the rule’s interest degree was low. Even a large number of redundant rules are also included [4]. Besides, the complexity of space and time are high. According to this, an improved association rule mining algorithm is proposed in the paper. It has improved the traditional algorithm, and introduced data cube, relative method in statistics and adaptive adjusting mechanism in intelligence control [5]. The algorithm has two advantages, the first is that the execution time is short while searching for the frequent item-set; the second is that the precision of the rules is high. The algorithm is also used in personality mining system based on E-learning model (PMSEM).The results manifest that the algorithm is effective.

168

L. Qi et al. / Research on Personality Mining System in E-Learning

1. An Improved Association Rules The steps of an association rules mining based on data cube as follows: Step1: establishing data cube from database Step2: mining frequent item-set which satisfies the minimum support on data cube Step3: generating association rules of frequent item-set. Step4: redundant association rules are wiped off through the relative method. 1.1 Establishing data cube Supposed in personality model database, we observe the data by three dimensional angles. These three dimensions separately are learning behavior patterns dimension, learner's personality characteristics dimension, and time dimension. Then data cube is obtained through OLAP operation from data base [6]. Fig.1 is data cube by 3-D.

Fig. 1 Data cube by 3-D

1.2 Mining frequent item-set on data cube The paper has improved Apriori algorithm. The algorithm searching for the frequent item-set on data cube is called Personal_ Cube_Apriori algorithm. Lk Represents set of frequent k_ item-set, Ck Represents set of candidate k_ item-set The idea of adaptive adjusting of the Personal_ Cube_Apriori algorithm as follow: Firstly, the algorithm searches for frequent k_ item-set for each dimension. If some dimensions don’t have frequent k_ item-set, it shows that the dimension level is excessively low and we should drill through above and improve the dimension level. If in some dimensions, all frequent k_ item-set is frequent k_ item-set, it shows that the dimension level is excessively high and we should drill through under and lower dimension level. 1.3 Generating irredundant association rules After mining out frequent item-set, the process of generating association rules is composed of two steps˖ Step1: regarding to each frequent item-set l, all non- spatial subsets are generated. Step2: regarding to each non- spatial subset of frequent item-set l, if sup port _ count (l ) t min_ conf sup port _ count ( s )

(1)

Then the rule s=> (l-s) is generated. min_conf represent the minimum confidence thresholds, support count(l) represents the number of transaction containing item-set l, support count(s) represents the number of transaction containing item-set s. Lower interest degree rules are obtained in this way, so the redundant rules must be wiped off. Thus, the paper introduces the relative analysis method in statistics to wipe off the redundant rules. After mining out one association rule, we calculate the correlation among frequent item-set. Concept of correlation is formula 2

169

L. Qi et al. / Research on Personality Mining System in E-Learning

O(r ) E (r )

CORRr

(2)

CORRr More approaches to 1, the independence among set of r item-set is better. For example, the association rule ˴=>˵, X, Y are the item-set. If its correlation is worse, it must be the false strong rule. Relative analysis method is applied in k dimension data cube. If item or dimension Y is Y1 , Y2 , , Yr ̎values, X is X 1 , X 2 , X s s values, nij represents the number of X is Yi , and X is Xj. s

ni

¦n

ij

i 1, 2, , r

(3)

ij

j 1, 2, , s

(4)

ni n j

(5)

j 1

r

nj

¦n i 1

r

n

s

¦¦ n

ij

mij

n

i 1 j 1

F2

r

s

n¦¦ i 1 j 1

(nij mij ) 2

(6)

mij

F 2 Is correlation degree among items, if all items are independent, F 2 is 0. Supposed

the critical value is assigned. If F 2 is bigger than the critical value, X and Y are statistical correlation. Otherwise, they are not statistical correlation. Namely, they are independent. 2. PMSEM The technologies of personality, data mining are applied in the paper [7], while the model of personality mining system based on E- learning (PMSEM) is shown in Fig.2

Fig. 2 The model of Personality mining system based on E-learning

The main function of PMSEM is to find out the association rules between personality characteristics and learning behavior patterns. The subsequent learners’ personality characteristics can deduce from their learning behavior by using the rules above.

170

L. Qi et al. / Research on Personality Mining System in E-Learning

3. Application of improved association rules algorithm in E-learning Based on above research, we combine with the cooperation item of personalized knowledge service system in network education .The author constructs a personality mining system website. The system is also applied in network institute of central normal university. The results manifest the system support personalized E-learning better the experiment is that we carry on the investigation and construct the database for personality attributes of 360 learners, 300 learners are selected to regard as the data source and other 60 learners data are selected to regard as the examination data. Step1: Data pretreatment and clean. First, not complete attributes values in database are filled in though rough theory. Then, according to the distribution of each attribute extreme value, sparser values is removed by minimum support min_sup =30%. Date converses to Boolean attributes according to the actual value scope of each attribute .thus, 100 attributes are obtained. Step2: Generating data cube. Data cube is generated through OLAP operation. These are three dimensions such as learning behavior patterns dimension, learner's personality characteristics dimension, and time dimension. Step3: Generating association rules. The frequent item-set is searched out though Personality_ Cube_Apriori. Then, F 2 is calculated and some false strong rules are removed. As a result of the application environment limited, the partial learning behavior patterns dimension and learner's personality characteristics dimension are carried on analysis though improved association rule algorithm. Learning behavior patterns selects two aspects such as learning courseware and issue information on BBS. 9766 multi-dimensional association rules are obtained. 6794 association rules are obtained though F 2 ˘166.7. There are 10 former and back integrity rules of them. Step4: Inputting the following learners’ behavior matches above rules, so learning characteristics is obtained. On the basis, best learning content and learning strategy are provided to them. So, personalized learning and cooperative learning are realized Step5: 60 Learners’ data is regarded as the examination data. Compared with above data, the results manifest that the system is effective. 4. Conclusion In summary, an improved association algorithm is proposed in the paper. The results manifest that the algorithm is effective through above algorithm performance research. Meanwhile, the model of personality mining system is proposed. The results manifest that the system mines out the association rules between personality characteristics and learning behavior patterns. The subsequent learners’ personality characteristics can deduced accurately, automatically from their learning behavior by using the rules above. References [1] Wu Yanwen, Luo Qi, “Research on Personalized Knowledge Service System in Community E-Learning”. Edutainment 2006 Proceedings. Lecture Notes in Computer Science, Volume 3942, 2006.4, pp.115-152. [2] Liu Jun, Li Renhou and Zheng Qinhua, “Study on the Personality Mining Method for Learners in Network Learning”. Journal of Xian Jiaotong University, 2004, 38(6), pp.575-576. [3] Han J and Kamber M, “Data Mining Concept and Techniques”. Academic Prints, 2001, pp.100-103. [4] Lu Lina and Chen Yaping, “Research on algorithm Apriori of mining association rules”. MINI-MICRO SYSTEM, vol 21, 2001.9, pp.942-944. [5] Yuan Wei, "data mining center of Statistics department of china renmin univer-sity",Statistics and information forum, 2002( 1),pp.5-9. [6] Yan Shizhuan and Li Zhanhuai, "Commercial Decision System Based on Data Ware-house and OLAP", Microelectronics & Computer, 2006(2), pp.66-67. [7] Luo Qi and Xue Qiang, “Research on Application of Association Rule Mining Algorithm in Learning Community”, CAAI-11, Wuhan, 2005, pp.1458-1462.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

171

Analysis on Relationships of Emotional Transmissions between Participants and Their Emotional Aspects in Communication Using Bulletin Board System Shogo KATO a, Yuuki KATO b, Kanji AKAHORI c Faculty of Human Sciences, Waseda University, Japan b School of Social Welfare, Tokyo University of Social Welfare, Japan c The Center for R & D of Educational Technology, Tokyo Institute of Technology, Japan [email protected] a

Abstract: This paper focuses on communication using bulletin board system (BBS). A practical experiment was conducted to investigate the degree of influence emotional transmissions between senders (writers) of replies and receivers (readers) of replies have on the emotions which they experienced during communication using the BBS. The analysis focused on the readers of postings. Twelve participants of this experiment were divided into two groups based on the degrees of emotional transmissions: a High group and a Low group. The emotions experienced during communication using the BBS were compared between the High group and the Low group. The results of this experiment showed a tendency for negative emotions such as anger and anxiety to increase when emotional transmissions are low (i.e., the Low group). Keywords: emotional transmissions, computer-mediated communication, BBS

1. Introduction Until relatively recently, interaction has been conducted entirely FtFC, or at the least synchronously. It is, therefore, unsurprising that in such contexts, we are highly effective in judging people’s characteristics, such as familiarity, gender, emotion or temperament (e.g., Cheng, et al., 2001). Judgment of other’s psychological states is a significant aspect of human interpersonal communication. It refers to the interpersonal process by which people employ all available information and make general judgments. Recent FtFC researches (e.g., Krauss & Fussell, 1996) have consistently indicated that both nonverbal and verbal cues jointly affect the process of judgment (Patterson, 1994). Technology now mediates much communication. Phone, e-mail or video-conference: in each case, people must make do with limited cues to help them estimate other people’s emotional states, dispositions and personalities. This is especially so when Computer-mediated communication (CMC) users cannot see each other and the CMC environment is restricted in terms of nonverbal cues. This has resulted in studies conducted to investigate judgment in CMC contexts (e.g., Kato et al., 2001; Markey & Wells, 2002). In particular, Kato, et al. (2001) have focused on judgment of other’s emotional states in e-mail communications, and found gaps between the sender’s self-report of emotional states and the receiver’s judgment of the sender’s emotional states. Their findings have showed that the judgment of emotions in e-mail communication lacks accuracy, and there is a tendency to misjudge the partners’ negative emotions to be hostile emotions in e-mail communication. However, there is little research

172

S. Kato et al. / Analysis on Relationships of Emotional Transmissions

on relationships between emotions experienced by senders and receivers and their transmissions of emotions. 2. Objectives We performed a practical experiment in the form of a simulated argument among university students while communicating using the BBS, within a university classroom lesson. In this research, the students' emotional aspect was measured in the discussion and the following an analysis was performed: Analysis focusing on the readers of postings. They (readers) read the reply to their postings and interpreted the emotions of the writer of the reply. This action was analyzed. From the analysis, we then considered the relationships of emotional transmissions between participants and their emotional aspects in BBS communication. 3. Method The participants consisted of twelve students from an all woman university in Japan (4 forth graders and 8 third graders) who enrolled in the "Human Media" subject. This practical experiment was carried out within the subject "Human Media" which the participants enrolled for in December, 2005. The subject focused on the design of Web pages and concentrated on design methods for high usability. This subject was a half-yearly subject that consisted of lectures (a total of 12 times) and an exercise Students were organized into groups of 2 or 3 persons. In their groups, the students considered the aspect of usability in the second half of each lesson. Starting from the eighth class, every group worked on the task of designing Web pages with high usability. In the last class, discussions were conducted on the BBS. Each participant perused the Web page designs created by other groups and argued on the best design to be used on the BBS. The time allocated for this task was 1 hour, and a tree view type BBS was used. The participants discussed on the BBS using aliases and throughout the practice of BBS, they were asked to avoid talking FtF. The participants were asked to complete the following four kinds of questionnaires during the discussion using the BBS. These four questionnaires asked about twelve kinds of emotions based on Izard, et al. (1993) (Interest, Joy, Surprise, Sadness, Anger, Disgust, Contempt, Anxiety, Guilt, Shyness, Inward Hostility, and Willingness) (Kato, et al., 2001). Two of these questionnaires (Emotional Interpretations and Emotional States at Receiving) measure the emotional aspects on the side of the receivers, and the other two questionnaires (Emotional Expectations and Emotional States at Sending) measure the emotional aspects on the side of the senders. The questionnaires consisted of twelve items, one item for each of the twelve emotions. In this paper, the analysis focused on two kinds of questionnaires, Emotional Interpretation and Emotional States at sending. Measurement of Emotional Interpretations: During the discussion using BBS, whenever the participants received a reply from the other students, they had to answer a series of questions using a five-point scale (1=not at all true, 5=very true). This was used as the index of ‘emotional interpretations’ to examine how each participant interpreted their partners’ emotions using the received reply. Measurement of Emotional States at Sending: Whenever the participants sent a posting to the other students, they also had to answer a questionnaire using a five-point scale (1=not at all true, 5=very true) as the index of ‘emotional states’ at sending, to examine how each participant felt when composing and sending the posting. The participants were required to reply to the above-mentioned questionnaires in the following two situations. (1) Once a participant replied to postings displayed on the BBS, she answered the questionnaires on Emotional Expectations and Emotional States at Sending, respectively. (2) Once there was a new reply from the other participants to the participant's posting, and after reading the reply, she then answered the questionnaires on

S. Kato et al. / Analysis on Relationships of Emotional Transmissions

173

Emotional States at Receiving and Emotional Interpretation, respectively. In other words, the participants answered the questionnaires after they replied the postings of others and after receiving replies from others. 4. Results and Discussion 4.1 Procedure of analysis The method of analysis performed in this research is described as follows. This research focused on emotional interpretations as the means to grade the degree of emotional transmissions between participants. It examined the degree the writer of posting’s emotion could be interpreted when reading the posting on a BBS. Kato, et al. (2006) was used as the index to grade these two emotional transmissions. The correlation coefficient between the emotions of 12 ‘emotional states at sending’ (which the writer of the reply answered at the time of posting) and the emotions of 12 corresponding ‘emotional interpretations’ was calculated to analyze the Emotional Interpretations. Then, based on the value of the correlation coefficient, the relation was classified into heights and converted into grades of ‘emotional transmissions’. Consequently, the relationships between the grades of emotional transmissions and emotional aspects were analyzed in the following manner. With regards to the height of emotional transmissions within emotional interpretations, a comparison was made between the height of emotional transmissions, and the height of emotional interpretations, respectively. In addition, analysis of the questionnaires data resulted in the classification of the 12 feelings into positive emotions and negative emotions based on the factor analysis of Kato, et al. (2005). "Interest", "joy", "surprise", and "willingness" were classified into positive emotions, and the rest of the emotions were classified as negative emotions. A t-test was conducted to compare using the values which averaged the consultation values of the emotions which corresponds, respectively. The following explains the basic data such as the number of postings in the BBS. In total, the participants posted a number of 126 postings. 61 postings were replies to other postings. In addition, the combination of the emotional interpretations and emotional states at sending was 38. Therefore, the correlation coefficient of 38 was calculated on each relationship. In this research, the correlation coefficient of 0.80 or more is considered as high emotional transmissions and 0.65 or less is considered as low emotional transmissions. Consequently, in the emotional interpretations, the combination of 13 was in each height. 4.2 Analysis on receivers’ aspects The data collected was analyzed to answer this question - What kinds of emotions (under which a writer write postings) would enable the reader to more correctly interpret the writer’s emotion? The heights of the emotional transmissions based on emotional interpretations were compared using the data obtained from the questionnaire of emotional transmissions. Each of the positive emotions and negative emotions was compared using t-test. The result is shown in Table 1. With regards to positive emotions, a significant difference was not seen between heights. On the other hand, with regards to negative emotions, a significant difference (t(24)=2.16, p<0.05) was seen between heights. In low emotional transmissions, negative emotions were highly produced as shown by Table 1. Then, we investigated whether the reader was able to accurately interpret the writer’s emotions. The heights of the emotional transmissions based on emotional interpretations were compared using the data obtained from the questionnaire of emotional interpretations. Concerning each of the positive emotions and negative emotions, a comparison was made using t-test. The result is shown in Table 1. With regards to positive emotions, the

174

S. Kato et al. / Analysis on Relationships of Emotional Transmissions

significant difference (t(24)=2.11, p<0.05) was seen between heights. On the other hand, with regards to negative emotions, no significant difference was seen between heights. In high emotional transmissions, positive emotions were highly produced from Table 1. Table 1 Comparison of emotions in the High and Low groups t test, *p<0.05 Emotional states at sending Emotional interpretations Positive Negative Positive Negative High 2.98 1.02 * High 3.19 * 1.08 Low 2.71 1.16 Low 2.67 1.10 4.3 Discussion This paper considers the ‘emotional interpretation’ as ‘emotional transmissions’ on the BBS. The results showed that there was a tendency for inaccurate interpretation of emotions to be made when a writer posted under more negative emotions. In contrast, when a writer posted under more positive emotions, there was a tendency for more accurate interpretation of emotional to be made. The result of the experiment on e-mail communications (Kato, Kato & Akahori, 2006) showed similar tendency as that of the tendency of this research. The authors believe that a clearer result was not visible in this practice since there was almost no specific evocation of negative emotions. However, whilst referring and considering the knowledge of previous researches, it is thought that it is easier to inaccurately interpret and expect negative emotions than positive emotions. Therefore, in the case of communication using a BBS, it is thought that a participant's emotion is important. The implication from the result of this research is that misunderstanding between participants can be reduced when participants write postings and replies with more positive emotions. The following explains the limitations of this research and future work. It is necessary to analyze writer’s aspects, the relationship between emotional states at receiving and emotional expectations. In addition, as all the participants in the practice of this research were female, the authors believe that from now onwards, it is necessary to examine the results on male participants and to analyze gender differences. Moreover, the analysis performed in this paper did not examine the contents of postings. As the sentences within the postings are the medium of emotional transmissions, from now on, it is important to conduct a detailed analysis of the contents of the postings. In addition, this analysis should also include the examination on the flow of postings. Finally, we would like to express our sincere thanks to Ms. Safiza Markhayu Yusof for her assistance. References [1] Cheng, Y., O’Toole, A., & Abdi, H. (2001). Classifying adults’ and children’s faces by sex: Computational investigations of subcategorial feature encoding. Cognitive Science, 25, 819-838. [2] Izard, C.E., Libero, D.Z., Putnam, P., & Haynes, O.M. (1993). Stability of emotion experiences and their relations to traits of personality, Journal of Personality and Social Psychology, 64, 847-860. [3] Kato, Y., Kato, S., & Akahori, K. (2006). Effects of emotional cues transmitted in e-mail communication on the emotions experienced by senders and receivers. Computers in Human Behavior, (In Press) [4] Kato, Y., Sugimura, K., & Akahori, K. (2001). An affective aspect of computer-mediated Communication: analysis of communications by e-mail. Proceedings of ICCE/SchoolNet 2001, 636-642. [5] Krauss, R. M., & Fussell, S. R. (1996). Social Psychological models of interpersonal communication. In E. T. Higgins, & A. W. Kruglanski (Eds.), Social Psychology: Handbook of Basic Principles (pp. 655-701), NY: The Guilford Press. [6] Markey, P., & Wells, S. (2002). Interpersonal perception in internet chat rooms. Journal of Research in Personality, 36, 134-146. [7] Patterson, M. L. (1994). Strategic functions of nonverbal exchange. In J. A. Daly & J. M. Wiemann (Eds.), Strategic Interpersonal Communication (pp. 273-293), Hillsdale, NJ: Erlbaum.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

175

Learning Nonverbal Emotion Interaction In 3D Intelligent Virtual Environment For Children Zhen Liua Faculty of Information Science and Technology, Ningbo University, 315211, China [email protected]

a

Abstract: Nonverbal emotion interaction is very important for children, learning nonverbal emotion interaction by computer technology is an interesting subject. In a certain intelligent virtual environment (IVE), multi virtual characters interact with emotion and construct an virtual society. Modeling nonverbal social interaction is very important for constructing believable virtual characters. Nonverbal social interaction is a direct communication manner for virtual characters in an IVE. A cognitive architecture of virtual character is presented in this paper, the architecture outlines high-level knowledge of virtual characters in IVE, some new concepts on nonverbal social interaction is set up. A virtual character has the ability of social interaction with other virtual characters by cognitive architecture. Keywords: Nonverbal, emotion interaction, intelligent virtual environment

Introduction Children’s development is very important for modern society, there are nearly 400 million children in china, but each family has only one child. Chinese children get a lot of care from their parents, however, some children lack emotion expression for their parents, and much children lack the experience of emotion interaction with others. In recent years, children’s crime is on the rise, people have already realized that affective education has already become the indispensable contents for children. A child should has ability of emotion communication with other is an important aspect of affective education. Emotion communication is expressed by verbal and nonverbal manner. In fact, nonverbal communications are even more important than verbal information, particular in respect of emotion interaction. Nonverbal communication includes a variety of signals, body language, gestures, tough, physical distance, facial expression, and nonverbal vocalization [1]. Human beings need nonverbal emotion interaction from infancy to old age, and nonverbal emotion interaction is so vital to children that without it their emotional development will be stunted. A child should master some basic manner of nonverbal emotion interaction, and he should learn how to watch emotion meaning from other’s facial expression or body language. In general, a child will spend a lot of time on learning nonverbal emotion interaction in real life, the purpose of the paper will support a new method of learning nonverbal emotion interaction for children. In recent years, virtual reality and artificial life are blended each other, intelligent virtual environment (IVE), a new research field is born in the intersection between virtual reality and artificial life. In an IVE, there are a lot of virtual characters. In order to realize believable intelligent virtual characters, we can use cognitive model with emotion mechanism to control the artificial

176

Z. Liu / Learning Nonverbal Emotion Interaction in 3D Intelligent Virtual Environment

behavior of virtual characters. We can construct a vivid virtual world where some virtual characters interact each other with manner of nonverbal emotion interaction. Social interaction of virtual characters is an interesting research subject in intelligent virtual environment (IVE)[2][3][4], a believable virtual character has the ability of social interaction to other virtual characters with verbal and nonverbal manner. There are a lot of researches on nonverbal behavior expression of virtual characters, but little on how interaction triggers and ends when two virtual character en-counters with social status. For example, Thalman presented the concept of virtual society [5]. He proposed to use agent architecture to build such virtual society. Cassell et.al developed behavior expression toolkit for virtual characters with XML-based language [6], Pelachaud created a subtle method of facial expression for virtual characters [7], Chi et.all built a system called Emote to express natural movements of posture virtual character [8]. Unuma presented a method of creating emotion-based locomotion [9]. Rose used motion-blending technique to product emotional walking [10]. Based on these researches and author’s previous work [11]-[14], the paper present a new method to construct virtual characters that can interact each other with social status. A cognitive architecture of virtual character is presented in this paper, a virtual character is provided with a cognitive architecture to control nonverbal emotion interaction, and a demo system is realized on PC. 1. Cognitive Architecture of Virtual Characters Nonverbal social interaction is crucial for behaviors of virtual characters in a virtual society. The section is based on cognitive modeling [3]. A virtual character is regarded as an agent with a built-in cognitive architecture that can control behavior and action. On the basis of cognitive modeling, a virtual character can be equipped with high-level knowledge about virtual world, and he can collect information, express emotion, make navigation and move to goal. A cognitive architecture includes four parts as follows: (1) The first part can sense and perceive virtual environment’s information from memory. A virtual character should be equipped with visual, tactile and auditory sensors in order to execute the perception. Synthetic vision is an important method for visual perception [5], which can accurately simulate the vision from view of character, and we realized the method synthesis vision on PC. Synthetic vision will be costly. Furthermore, this method cannot get the detail semantic information of objects. Therefore, we present another efficient method for simulation of visual perception. Based on the Gibson’s theory of affordances[15], an affordance is invariance for environment. A character can perceive these affordances directly. In this paper, we can use the Gibson’s theory to guide navigation, affordances of objects hints navigation information. (2) The second part can make decision by information, it include plan module, behavior module, social norm, mental module. Plan module execute behavior plans by a set of productive rulers. Behavior module creates behavior codes by behavior plans codes. Inhibitory gain and fatigue are time sequence characteristic of behavior. The higher Inhibitory gain, the longer the duration of the behavior is and new behavior is excited only under new stimulus. Fatigue means that behavior with low degree of priority can obtain the chance to carry out, once a certain behavior is carried out, the behavior will stop at some time. We introduce the inhibitory gain coefficient (a real number greater than one) and fatigue coefficient (a real number smaller than one) to measure inhibitory gain and fatigue correspondingly. Social norm module includes status, interaction information and interaction rules, it controls the process of a nonverbal social interaction, it provides the social knowledge for virtual character. Mental module includes emotion, personality and motivation. Based on emotion OCC model [16], we can use a computational model of emotion [13]. Emotion module read external stimulus from memory (the perception module

Z. Liu / Learning Nonverbal Emotion Interaction in 3D Intelligent Virtual Environment

177

write stimulus information to memory module). Activation of an emotion is relative to external stimulus and inner mental variables. If an emotion is active, this module will create emotion expression, emotion expression code will be sent to behavior module. Emotion is core of the module, and personality is some stable psychological traits of a virtual character, and motivation variables include some physiology parameters of a virtual character. (3) The third part executes the behaviors through body posture and facial expression. We can use inverse kinematic arithmetic and motion blending to drive nonverbal emotion posture, and we can use geometry blending to create facial expression. (4) The four part includes database and knowledge. Database includes 3D geometry of virtual environment, original information, such as, the original location and parameters of virtual character, motion capture data, 3D model and location of objects, default motion plan scripts that record some goal location. Memory can serves as a center of information share among all other modules. Knowledge module includes guiding knowledge in environment for virtual characters. For example, the meanings of objects in environment are part of knowledge. 2. A Demo System of Virtual Character This paper uses inverse kinematics to simulate the walking animation of a virtual character, some high-level parameters (walking step, goal point, etc.) can control a walking process, and so a virtual character can walk to anywhere by outer stimulus. In order to enhance the reality of walking, some emotional motion capture clips can be blended with waking clips to create an emotional walking. We can use synthesis model of expression method to create infinite facial expressions.

happy Figure. 1

sad

happy and sad

virtual character has complex facial expression

Figure. 2 Nonverbal emotion interaction between two characters 3. Conclusion and future work Intelligent virtual environment (IVE) will play an important role in the affective education

178

Z. Liu / Learning Nonverbal Emotion Interaction in 3D Intelligent Virtual Environment

of children. In a certain IVE, multi virtual characters interact with nonverbal behaviors and construct a virtual society. This paper presents a cognitive architecture of social interaction for virtual characters. Social norm is an important component in cognitive architecture. A social norm includes status information, interaction signals and interaction rules. A demo system is realized on PC. Simulation of social interaction for virtual characters is a very difficult subject. This paper only gives a primary outline for interaction model in this paper. In fact, social interaction is related to many factors, such as different culture, emotion, personality, motivation etc. We can use the above demo system to teach children to learn nonverbal emotion interaction.

Acknowledgments The work described in this paper is co-supported by science and technology project of Zhejiang Province Science Department (grant no: 2006C33046), University Research Project of Zhejiang Province Education Department (grant no: 20051731), forepart professional research of ministry of science and technology of the People's Republic of China (grant no: 2005CCA04400), References [1] Zastrow. C, Kirst-Ashman. K.(1987) Understanding Human Behavior and the Social Environment. Nelson-Hall Publisher press, Chicago, 322-323. [2] Tu, X., Terzopoulos, D.(1994) Artificial fishes: Physics, locomotion, perception, behavior. Proceedings of SIGGRAPH’1994, ACM Press, 43-50. [3] Funge, J., Tu, X., Terzopoulos, D.(1999) Cognitive Modeling: Knowledge, Reasoning and Planning for Intelligent Characters. Proceedings of SIGGRAPH'99, ACM Press, 29-38 [4] Badler. N., Phillips. C., Webber. B.(1993) Simulating Humans: Computer Graphics Animation and Control. Oxford University Press, New York, 154-159. [5] Magnenat-Thalmann. N., Thalmann. D. (eds)(2004) Handbook of Virtual Humans. John Wiley & Sons. [6] Cassell, J., Vilhjalmsson, H.H., Bickmore, T.(2001) BEAT: the behavior expression animation toolkit. Proceedings of SIGGRAPH’2001, ACM Press, 477–486. [7] Pelachaud, C., Poggi, I.(2002) Subtleties of facial expressions in embodied characters. Journal of Visualization and Computer Animation. 13, 287–300. [8] Chi, D., Costa, M., Zhao, L., Badler, N.(2000) The emote model for emote and shape. Proceedings of SIGGRAPH’2000, ACM Press, 173–182. [9] Unuma. M., Anjyo. K., Takeuchi. R.(1995) Fourier Principles for Emotion-Based Human Figure Animation. Proceedings of SIGGRAPH'1995, ACM Press , 91-96. [10] Rose .C. F., Cohen. M., Bodenheimer. B.(1998) Verbs and Adverbs: Multidimensional Motion Interpolation. IEEE Computer Graphics&Application, 18, 5, 32-40. [11] Liu .Z.(2002) Emotional Behavior Animation of Virtual Human In Dynamic Virtual Environment. Proceedings of the 8th Joint International Computer Conference, Zhejiang University Press, 190-193. [12] Liu .Z, Pan. Z.G., Zhang. M.M.(2002) Behavior Animation of Virtual Human In Virtual Society. Proceedings of the twelfth International Conference on Artificial Reality and Telexistence(ICAT2002), 186-187. [13] Liu Z, Pan, Z.G.(2005) An Emotion Model of 3D Virtual Characters In Intelligent Virtual Environment. Proceedings of the first international conference on affective computing and intelligent interaction, ACII2005, LNCS 3784, Beijing, China, 629-636. [14] Liu.Z.(2006) a synthesis of emotion in emotion vector space, proceedings of 2006 international conference on artificial intelligence, Publishing House Bupt, 45-48. [15] Gibson, J.J.(1986) The ecological approach to visual perception. Lawrence Erlbaum Associates, Hillsdale, New Jersey. [16] Ortony, A., Clore, G.L., Collins, A.(1988) The cognitive structure of emotions. Cambridge University Press, New York.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

179

Development of Know-how Information Sharing System in Care Planning Processes Kaoru ETOa, Tatsunori MATSUIb, Yasuo KABASAWAa a Faculty of Engineering, Nippon Institute of Technology, Japan b Faculty of Human Sciences, Waseda Universty, Japan [email protected]

Abstract. The purpose of this study is to develop a computer support system for educating personnel who are involved in care management. We propose a system in which know-how information can be shared. We consider that visualizing and showing care plans drawn up by experts in various forms allows a beginner to see the differences between a beginner’s plan and expert plans. Sharing know-how information is possible by recording, accumulating, and giving titles to what has been noticed in comparing documents. This system can visualize the similarities among documents that interpreted the results of an assessment, and can flexibly change different viewpoints, additionally, can map a user's new document into a two-dimensional document space. We confirmed that these functions were appropriate.

Keywords: know-how information sharing, changing viewpoint, concept-base

Introduction In order to improve the quality of care, it is necessary that we establish a method of care manager training. There are four important processes in care management. This system supports the process in which the result of an assessment is interpreted. We decided to assist a beginner in noticing weaknesses in her care plan by visualizing and showing care plans drawn up by expert in various forms. Differences between a beginner's plan and an expert's appear most typically in their different viewpoints to interpret the results of an assessment. Making these differences noticeable is one method of extracting know-how information. In order to promote user awareness, we visualized similarities among a user's documents and an expert's documents, and achieved a function of flexibly changing viewpoints. We also confirmed that these results were appropriate.

180

K. Eto et al. / Development of Know-How Information Sharing System

1. The Method and Support Functions of Know-how Information Extraction We consider the support functions in which a viewpoint can be flexibly changed and the results can be visualized. We extract know-how information using these functions. This idea is shown in Fig.1. (1) A user refers to the statistic values of an original idea or a KOMI (Kanai Original Modern Instrument) chart, "graphical recording sheet" and then changes the viewpoints. (2) Based on these viewpoint changes, the user moves vertically and horizontally in the hierarchy of a concept-base, and calculates the degree of similarity each time. (3) The system then classifies the document from the calculation results. (4) Next, the system visualizes the results in a two-dimensional document space displayed on the computer. (5) By clicking on the document number according to its classification, the KOMI chart that originates the document is shown. (6) Finally, the user records, accumulates, and gives a title to what has been noticed.

Moving right and left 㧔view relationship㧕

M ovement on the hierarchy of the concept-base

Calculation of degree of similarity

M oving up(expansion) M oving down(focus on)

Change of viewpoint

User

Specific Knowledge recording

Sharing know-how Giving a title

Extracting know-how

Specific Knowledge

General Knowledge Separation of Specific Knowledge and General Knowledge

Awareness of difference

Change of grain size of cell on the chart

X

X

㧞 Y ԙ 㧝㧟 Ԙ Ԛ 㧡㧢

ԝ

Y 㧞 ԙ 㧝Ԙ 㧠 Ԙ ԛ 㧣㧟 Ԛ Ԟ Y 㧡 Ԝ 㧞㧟 ԙ

X

㧠 ԛ

Ԛ

㧢 ԝ

Accumulation of title

Fig.1.The model for extracting and sharing the know-how information

2. A Concept-base and Similarities between Documents We constructed a concept-base that has a tree structure with six levels. Viewpoints are changed by moving up and down the levels of this concept-base. The concept-base contains about 4300 terms that consist of keywords extracted from the documents that include the KOMI chart, the assessment items, and the textbook for the KOMI chart. The extracted keywords were encoded based on a Japanese language thesaurus. The code is used to identify the position on the hierarchy of the concept-base. The distance between keywords is determined by the nearness of the concept between keywords. The distance between keywords is actually defined as the sum total of the weight of the branch of the node which reaches other keywords from one keyword. The distance between documents is determined by the distance between the keyword sets [1].

181

K. Eto et al. / Development of Know-How Information Sharing System

3. Mapping the Document of a New Plan into Two-dimensions

Change viewpoint Display result

this function to highlight the effects that promoted an awareness of the differences between a user and an expert. We wish to determine the two-dimension coordinate for a new case.

Making new Read new care plan document

Expert's documents mapped into two-dimension space by Kruskal's [2] method previously. Beginner's new document map in same space. We expected

C are D esigner Extract docum ent M orphological analysis C alculate im portance of keyword C alculate sim ilarity E xtract three near cases C alculate coordinate D isplay on tow-dimension D isplay chart

C are D esigner ’s D atabase Radar C hart K O M I C hart Sum m ary C hart D ocum ents Concept-B ase M odel cases D atabase D ata of keyword set classified by field Coordinate D ata

The step is shown in Fig. 2. Fig.2 Procedure for mapping new case (1) Make new care plan using “Care-designer”, into Two-dimensions which we have developed the software for making a care plan [3]. (2) (3) (4) (5) (6) (7)

Extract document from the database of “Care-designer”. Analyze the document with software of morphological analysis [4]. Extract keywords from the result of analysis [5]. Calculate the distance between expert' documents and beginner's new document. Choose three cases that is the nearest to the new case. Determine the coordinate of new case by distances and coordinates of three cases.

4. Discussion The left of Fig.3 shows the results of having added the new case,”No.36”. Three cases that were the nearest to the new case from the similarity calculation results were No.33, 4, and 29. No.36 was not mapped on the position that was necessarily nearest to the three examples. However, the new case was mapped into an appropriate range. This result was based on the properties of the multidimensional scaling, by increasing the number of cases, the similarity calculation results and the results of mapping into two-dimensions became closer. Two cases, "No.1, No.2" were near to the new case. The clients in three cases have a common feature that suffered from dementia, cerebral palsy, etc. The results of analyzing the keywords are as follows: (1) The same keywords found were found on two occasions in case 1, and on five occasions in case 2; and (2) Keywords sharing a sibling relationship were found on four occasions in case 1 and on five occasions in case 2. In chart of Fig. 3, four items are the same result of assessment in the left chart, 49 items are the same in the center chart and 53 items are the same in the right chart. From these results, we concluded that two-dimensional mapping was appropriate.

182

K. Eto et al. / Development of Know-How Information Sharing System

Fig.3. Mapping into two-dimensions added the new case and displaying of the chart of cases near to a new case 5. Conclusion We have developed the system to make a beginner notice the difference between beginner's care plan and an expert's by visualizing a care plan with various forms that were drawn up by an expert. As a function, the similarities among documents that interpret the results of an assessment were calculated using a concept-base, and the results were then mapped into two-dimensions. In order to promote user awareness, we created a function that shows the position of the user's new plan in relation to an expert's plan group in a two-dimensional document space. We confirmed that the results of mapping these documents into two-dimensions were appropriate. We consider that these support functions make a beginner notice the difference between a beginner's viewpoint and an expert's viewpoint, and that know-how information can be extracted. As a future work, we will experiment to evaluate the effectiveness of the system in a hospital and nurses' college.

References [1] Eto K., Matsui T., Kabasawa Y. (2006) Development of Know-how Information Sharing System in Care Planning Processes – Mapping New Care Plan into Two-Dimensional Document Space, Proc. of KES’2006, Part2, pp.977-984. [2] Okada A.,Imaizumi T. (1994) Multidimensional scaling using Personal Computer Kyoritsu Shuppan, Tokyo (in Japanese). [3] Eto K., Matsuda I. (2002) Development of a Care Plan Tool Using Assessment Information Visualization, Trans. IEICE, Vol. J85-D-I, No. 7, pp. 701-710 (in Japanese). [4] http://mecab.sourceforge.jp/. [5] http://gensen.dl.itc.u-tokyo.ac.jp/.

CSCL

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

185

Student Learning and Team Formation in a Structured CSCL Environment Nobel Khandaker, Leen-Kiat Soh and Hong Jiang Computer Science and Engineering University of Nebraska-Lincoln 256 Avery Hall, Lincoln, NE 68588-0115 {knobel, lksoh, jiang}@cse.unl.edu Abstract. A computer-supported collaborative learning (CSCL) environment aims to facilitate student learning by letting them work in small teams with the support of computer technology. Two important factors that affect work in such scenario are: interaction among the students and compatibility or interactivity of the team members. I-MINDS is a tool designed to support structured Computer Supported Collaborative Learning. In I-MINDS we try to improve both the quality of student learning and the quality of student team work with the help of intelligent agents. I-MINDS facilitates student interaction through its forums and can build student teams. In our two-semester long experiment we studied the effect of the interaction environment on the learning and performance of students in face-to-face and structured CSCL scenarios. Moreover, we studied how a student’s self-reported efficacy, teamwork and interaction skills affect his or her performance in face-to-face and structured CSCL settings. Our results indicate that even though students prefer face-to-face interactions; structured CSCL environment may increase their individual performance. Furthermore, we find that factors such as the difficulty of the problem or task to be solved and team member compatibility influence the quality of teamwork in structured CSCL. Keywords: CSCL, Jigsaw, Collaboration, Team, Learning

Introduction Computer Supported Collaborative Learning (CSCL) has been studied as a tool to increase student learning [14, 4, 9]. In a CSCL environment, students work together in small teams to solve complex problems. In structured collaborative learning, students are guided within a set of prescribed activities so that each activity has a set goal and measurable outcomes. The long term goal of our research is to support structured collaborative learning using an intelligent multiagent system where students and instructors may or may not have face-to-face interactions. In a typical CSCL environment, students learn from each other by interacting with their team members and by teaching their team members [5]. However, not all student teams work well [4, 8]. In some cases the students work in teams rather than working as a team [4, 8, 11]. To work together as a team well, the students need to build a shared understanding of what they are working on as well as how they are working together [11]. Even though the team formation procedure has been investigated by researchers [6, 7, 13]— such as handling information flow during team formation and transitioning between phases of team formation, the relationship between a student’s performance and his or her teamwork activities within a CSCL environment has not been addressed. As a CSCL tool, I-MINDS [10, 14, 15, 16] provides an environment where student participants are able to interact with each other through text messaging. That is, students interact without face-to-face interactions in the same room. Furthermore, I-MINDS can

186

N. Khandaker et al. / Student Learning and Team Formation in a Structured CSCL Environment

automatically form student teams. The team formation algorithm in I-MINDS works by allowing students to form teams based on the performance and compatibility of the participating students. In this paper, we discuss the results of our two-semester long experiment of deploying I-MINDS in an introductory CS course. Our experiment was conducted by dividing the students in control and treatment sections. The students in the control section participated in structured collaborative learning using conventional face-to-face interaction. However, the students in the treatment section participated in structured computer-supported collaborative learning and interacted with each other through I-MINDS. During our experiment, we investigated the following issues: x How does the teamwork in conventional structured face-to-face collaborative learning compare to the teamwork in structured computer-supported collaborative learning? x How does the structured CSCL environment impact student learning or performance? x How do a student’s self-efficacy, perception of his or her peers, and perception of teamwork relate to his or her individual performance? Note that we have previously reported the structure of I-MINDS [17] and our implementation of Jigsaw collaborative learning model [1, 3]. Thus, in the following, we will only focus on the experiment and results, emphasizing on how students perceived teamwork and their peers, and the impact of I-MINDS on student learning and teamwork. 1. I-MINDS 1.1 I-MINDS Structure I-MINDS, which stands for Intelligent Multiagent Infrastructure for Distributed Systems in Education, consists of a set of intelligent agents. There are three types of agents: teacher agents, group agents (for supporting teams), and student agents. In I-MINDS, a teacher agent interacts mainly with the instructor. It helps the instructor deliver instructional content to the students and coordinates the team formation process in the virtual classroom. A teacher agent also helps the instructor answer student questions with the question ranking and question grouping modules. In addition, a teacher agent helps form student teams for structured collaborative learning, supporting the Jigsaw procedure. In an I-MINDS supported classroom, a student agent serves a unique student. It interacts with the student and exchanges information with the teacher agent and the group agents. It also maintains a dynamic profile of the student to whom it is assigned and a dynamic profile of the peers that the student has interacted with through I-MINDS. Finally, a group agent in I-MINDS is activated when there are structured collaborative learning activities. Structured collaborative learning involves specified activities that explicitly require students to cooperate. Currently, I-MINDS implements the Jigsaw model [1, 3]. The team activities monitored by the group agent include the number and type of messages sent among team members, self-reported teamwork capabilities, peer-based evaluations as a team member, and evaluation of each team. A detailed description of I-MINDS can be found in [14, 16]. 1.2 Team Formation in I-MINDS The teacher agent forms student teams using the VALCAM algorithm [15]. In VALCAM, the teacher agent acts as a coordinator for the coalition formation process and makes global decisions, such as, what should the least number of members in a coalition be, how long should the teams last, how the performance of each coalition should be evaluated, etc. The group agents then manage the teams. Each group agent monitors the performance and activities of the members of its assigned team. After the coalition has completed their tasks, the group agent also evaluates the performance of each student agent as a team member.

N. Khandaker et al. / Student Learning and Team Formation in a Structured CSCL Environment

187

The VALCAM algorithm is based on reinforcement learning of the student agents. In the beginning of every session the teacher agent chooses a few students and then initiates the team formation. During the team formation process, each student agent bids—an iterative Vickrey auction—to join its favorite team based on virtual currency earned from previous Jigsaw activities. Once the teams are formed, the group agents work with their team members to complete tasks assigned by the instructor. Finally, when the assigned task is completed, each student agent is rewarded with virtual currency based on its (student’s) performance as an individual and as a team member. Students who performed well will receive more virtual currency, allowing them to more successfully bid for their favorite teams in future sessions. Briefly, the VALCAM algorithm works as follows. Initially, the students or users are given some default amount of virtual currency to start with. In a typical coalition formation round, the auctioneer announces a task, and then the students post their self-efficacy in a blackboard. Then the auctioneer chooses the first members of the teams according to their current virtual currency balance. After the first members of the teams are chosen, the other members are assigned to each team by iterative auction. While bidding to join a team, a student agent considers how competent that team’s members think they are and what its past working experience with the members already in the team was. So, student agents who join a team first influence how the team takes shape. The number of members per team is the same or greater than the number of subtasks in the task given. Over time, as the students gain experience in their interactions with other students and participate in different tasks, students accumulate varying amounts of virtual currency: students who have been in a team that produces good results, or have performed well in the post-tests, or have been rated high by their peers will be rewarded more. As a result, the student agents of these students will be able to offer higher bids in order to join teams more likely as first members, and allow their students to form teams most compatible with them. So, the student agent in VALCAM incorporates the two important aspects of successful student collaboration: the competence of the members of the team it is trying to join and its interaction history with the members of the team it is trying to join. Details of this algorithm can be found in [14, 15]. 2. Experiment and Results 2.1 Experiment Setup To investigate the issues outlined in Introduction, a two-semester long study was carried out in the closed labs of CSCE 155 at the University of Nebraska-Lincoln, in Spring and Fall semesters of 2005. CSCE 155 is the first core course required of computer science and computer engineering majors. It has three 1-hour weekly lectures and one 2-hour weekly laboratory sessions. In each lab session, students were given specific lab activities to experiment with Java and practice hands-on to solve programming problems. For each semester, there were 2-3 lab sections where each section had about 15-25 students. The study utilized a control-treatment protocol. In the control section, students worked in Jigsaw collaborative learning teams without using I-MINDS. Students were allowed to move around in the room to join their Jigsaw teams (main group and focus groups [14]) to carry out face-to-face discussions. In the treatment section, students were told to stay at their computers and were only allowed to communicate via I-MINDS—with this setup, we essentially simulated a distance classroom environment. For each lab, the students were given a lab handout with a list of activities—thus, a lab is a task and its activities are the subtasks. The students of both the control and treatment sections were required to complete the tasks and subtasks in the four Jigsaw phases [14]. In each section, the instructor first announced the task and asked the students to fill

188

N. Khandaker et al. / Student Learning and Team Formation in a Structured CSCL Environment

out a Self Efficacy Questionnaire (SEQ) [14] to describe their competence in that area. Then the instructor announced the main groups. In the control section this was done by matching the strong student with the weak students (first goes with last and so on). In the treatment section, I-MINDS formed teams using the VALCAM algorithm. Once the main groups were formed, the teacher agent formed the focus groups by randomly selecting students from the main group. Then, every focus group was assigned one subtask randomly. After the subtask assignment, the focused exploration phase was started. Then the remaining Jigsaw Phases were carried out in order, during which the student agents and the group agents monitored and guided the activities of the students and the student teams, respectively. After all Jigsaw Phases were executed, all the students filled out the Peer Rating Questionnaire (PRQ) and Team-Based Efficacy Questionnaire (TEQ) [14] and took a 10-minute post-test. The post-test score was used as a measure of student individual performance in terms of understanding the lab topic. 2.2 Results First, we look at the average normalized post-test scores, as shown in Figure 1. Each normalized score is computed by dividing each student’s post-test score for a test day by the sum of the student’s post-test scores of all other lab days that did not involve the Jigsaw experiment. Therefore, this normalized score provides a measure to compare the performances of the control and treatment section students in a scale that does not depend on the individual student’s abilities.

Figure 1: Control vs. Treatment: Average normalized post-test scores for (a) Spring 2005, (b) Fall 2005

For both Fall 05 and Spring 05 experiments, one tailed t-test reveals that the average normalized post-test score of the treatment section is significantly higher than that of the control section ( D 5% , p value for Spring is 6 u10 4 and Fall is 5 u10 5 ). This indicates that I-MINDS-supported structured collaborative learning outperformed the conventional face-to-face one. This result also support those observed in [2, 12]. On average, students in the treatment section achieved better standard deviation—meaning that these students’ post-test scores were more tightly clustered than those of the control section (1.67 vs. 2.36 in Spring 2005, and 1.11 vs. 1.25 in Fall 2005). We also observe that students in the treatment sections seemed to improve over time, and their performance seemed to eventually overtake that of the control sections’—indicating that VALCAM, due to its learning mechanism, might have been effective in forming better and better teams over time. The Peer Rating Questionnaire (PRQ) surveys were conducted in both control and treatment sections after each lab session was completed. The PRQ is designed to quantify the compatibility of the team members after they have gone through the collaborative learning process. The average peer rating scores that each student gave to his or her team members can be used as a measurement of how well the team members were able to work with one another. Table 1 shows the results of the PRQ surveys. Students in the control

N. Khandaker et al. / Student Learning and Team Formation in a Structured CSCL Environment

189

section rated their peers better (higher means) and more consistently (lower standard deviation values) than those in the treatment section. This is possibly due to the convenience of face-to-face interaction since I-MINDS still lacks sufficient GUI features and multimedia capabilities to fully support real-time interactions. Spring 2005 Fall 2005 Control Section Treatment Section Control Section Treatment Section Mean Stdev. Mean Stdev. Mean Stdev. Mean Stdev. 1 42.10 2.73 32.45 5.78 35.39 2.30 33.71 4.69 2 36.62 7.05 37.72 4.60 34.87 5.32 35.80 12.21 3 39.91 4.80 34.63 8.08 36.03 3.19 36.37 5.18 4 N/A N/A N/A N/A 37.53 3.37 37.25 3.62 Mean 39.54 4.86 34.93 6.15 35.95 3.54 35.78 6.42 Table 1: Control vs. treatment sections: Results of the peer-rating questionnaires (PRQs) Session

On the other hand, there are indications that students in the treatment section for the Fall 2005 sections seemed to rate their peers better gradually as the semester progressed (from 33.71 to 35.80 to 36.37 and 37.25) and seemed to rate their peers more consistently as well. This might be due to the ability of the team formation algorithm in forming better teams over time, as indicated earlier. This evaluation in the form of PRQ then helped them choose better team members in the future sessions. The Team-Based Efficacy Questionnaire (TEQ) surveys were collected after each lab based on a set of questions designed to measure how a student viewed how well his or her team had performed, as shown in Table 2. Spring 2005 Fall 2005 Control Section Treatment Section Control Section Treatment Section Mean Stdev. Mean Stdev. Mean Stdev. Mean Stdev. 1 31.80 2.58 27.72 5.08 27.22 4.37 23.64 5.55 2 30.87 3.38 29.18 2.63 26.75 6.66 25.87 8.33 3 30.08 3.02 28.25 4.02 29.14 5.47 25.76 5.43 4 N/A N/A N/A N/A 29.12 4.52 26.78 8.15 Mean 30.92 2.99 28.38 3.91 28.05 5.25 25.51 6.86 Table 2: Control vs. Treatment sections: Results of the team-based efficacy questionnaires (TEQs) Session

It is observed that students in the control section approved of their team-based activities more than the students in the treatment section. There are two possible explanations. First, the ease of face-to-face interactions gave the impression that the team was doing better, which is consistent with our earlier observation with the peer rating results. Second, how the student agents form their teams did not necessarily meet the students’ preference. Note that a student did not have access to other survey results, including how his or her team members thought of him or her as a peer. However, the student agent did and perused this information in its bidding for the most useful or compatible team. Finally, the correlation between a student’s performance and the other parameters is investigated, as shown in Table 3. First, it is observed that the treatment section had higher correlation values in SEQ (0.41 vs. 0.28), PRQ (0.34 vs. 0.23), and TEQ (0.50 vs. 0.22) than did the control section. This indicates that the better students (with higher post-test scores) in I-MINDS teams rated their self-efficacies better, rated their peers better, and rated their team-based efficacies better than those in the traditional face-to-face teams. Looking more closely at how I-MINDS students interacted, we see that students who had better post-test scores were also the students who sent longer messages (with a correlation of 0.40). Thus, in this case, better students assumed a larger role in their respective teams during the treatment. Combining this observation with what has been reported earlier on the average

190

N. Khandaker et al. / Student Learning and Team Formation in a Structured CSCL Environment

normalized post-test scores, there are indications that better students helped other students do better in the treatment section and that resulted in better individual performances as evidenced in the post-test scores. Spring 2005 Fall 2005 Correlation with Post Test Correlation with Post Test Control Section Treatment Section Control Section Treatment Section SEQ 0.28 0.41 0.22 0.16 PRQ 0.23 0.34 -0.01 0.11 TEQ 0.22 0.50 0.09 0.12 Number of Messages Sent N/A 0.11 N/A 0.27 Avg. Length of Messages N/A 0.40 N/A 0.25 Table 3: Correlation between Post-test Score and Other Parameters Variable

However, the above observations were not repeated in the Fall 2005 study. Comparing the two sections, how students did during the collaborative learning activities did not correlate with how they performed individually in the post-tests. Compared to Spring 2005, better-performing students in Fall 2005 tended to send more messages (0.27 vs. 0.11), but shorter messages (0.25 vs. 0.40). Does that mean that the better-performing students in Fall 2005 were less patient with their peers? Further, students in the Spring 2005 treatment section reported a 0.41 correlation between their self-efficacies and their post-test scores, compared to only 0.16 in the Fall 2005 treatment section. That means that the students in the Fall 2005 treatment section were far less accurate in their knowledge of their own ability to solve the upcoming problem set, which is very important to form effective teams: students who think they are good at a particular topic when in fact they are not as good can misguide the team activities. This indicates that even though a student is doing very well individually, he or she may not be helpful to other team members. Therefore, when forming teams of learners, the individual competence is not the only factor, the compatibility and the history of the members working together should also be considered. Indeed, I-MINDS’ team formation algorithm—VALCAM—does consider the PRQ values while forming teams. However, since this algorithm uses reinforcement learning, it needs some training before it could form effective cooperating teams. The quality of interaction between the teammates depends on various things, their likings of each other, their expertise in the problem and the difficulty of the assigned problem. This last factor is vital because if the problem is too easy, interaction among team members becomes a liability instead of being an asset. From our close observation of the students, it was observed that more students in Fall 2005, on average, found the assigned problems to be easy, than those in Spring 2005—as they also achieved better course grades. Therefore, this could be a very possible reason for the lack of impact of structured collaborative learning (both control and treatment) in Fall 2005. This hints that a learner team in a CSCL environment could work better only when the problem is too difficult for one team member to solve by himself or herself. This could then also motivate students to exchange messages to help each other obtain a solution. As a whole, the findings of our experiment can be summarized in the following way: x From the user’s perspective, the conventional structured face-to-face collaborative learning is more preferable than the structured computer-supported collaborative learning. The higher TEQ and PRQ values of the control group point to that. However, the students’ perspective of teamwork is affected by various factors like ease of use of the communication tool, variety of communication methods, etc. I-MINDS only allowed students to communicate with each other using text messages. However, in face-to-face interaction, the students worked with each other in person. So, it is not fair to compare face-to-face interaction with simple text messaging. Considering this, we are now

N. Khandaker et al. / Student Learning and Team Formation in a Structured CSCL Environment

x

x

x

x

191

working on an advanced whiteboard in I-MINDS that would let the students work with their team members more closely and through more intuitive ways than text messages such as equations, flowcharts and handwriting/hand-drawing. Our results indicate that structured computer-supported collaborative learning helps students learn better than conventional structured face-to-face collaborative learning. The higher normalized post test score of the treatment section point to that. Since both the control and treatment groups followed the Jigsaw process, this observation may be attributed to the fact that the restricted mode of I-MINDS’ communication forced the students of the treatment section to be more explicit and articulate in expressing their ideas. So, they were able to gain a deeper understanding of the subject matter than the control section students. To gain further understanding, we are working on a message classification system that would allow us to understand how a student’s understanding or expertise in the given problem changes over time during a session. Our results indicate that a student’s self-efficacy assessment does not correlate with his or her performance in either setup. Since the students are solving problems in an area which is new to them, it is reasonable that they may not be able to judge their own performance accurately. However, our team formation algorithm uses the self-efficacy assessment of a student while forming teams as a measure of that student’s potential ability. So, we will look into lowering the weight or importance of this measure in the team formation algorithm (VALCAM), thus giving more importance to tractable measure (such as the quality of questions asked) that we use for computing a student’s competence. Further, we will also include past individual performance (e.g., post-test scores) of a student to help refine the self-efficacy measurement. According to our results, individual performance of a student is not correlated to his or her impression about the usefulness of his or her team. In some situations, the good students view the teamwork as not useful and the not-so-good students view the teamwork as useful and vice versa. The low correlation between a student’s performance and his or her TEQ values point to that. In general, if a strong student can solve the assigned problems without the help of the team members, he or she may perceive teamwork as unnecessary. As a result, cooperation or interaction between both parties suffers, resulting in low valuation of the teamwork (TEQ). So, students rate their teams lowly regardless of their own performance, generating low correlation values. To gain further insight into this correlation, our future experiments will involve challenging tasks or problems that can only be (better) solved through effective teamwork. Finally, our results indicate that, individual performance value of a student is not correlated with the peer review received by that student. A good student can receive a low review from his or her peers and vice versa. This happens because in teamwork, a student’s helpfulness is very important in determining how his or her peers perceive him or her as a team member, regardless of the student’s individual capabilities. The team members’ perception of each other contributes to their compatibility. The team formation algorithm (VALCAM) used PRQ values to measure the compatibility of the team members; it was not very successful because of two reasons. First, VALCAM uses reinforcement learning and thus requires more than three or four iterations to form compatible teams, whereas our experiments had only three or four iterations. Second, PRQ value alone may not be a sufficient measure of compatibility. For our future experiments, we will build a more precise and accurate compatibility measure that would model student dialogues, and semi-structured activities on digital whiteboards. This revised compatibility measure would help VALCAM build better teams.

192

N. Khandaker et al. / Student Learning and Team Formation in a Structured CSCL Environment

3. Conclusions and Future Work

In this paper, we have presented the results of our two-semester long experiment with I-MINDS, a structured CSCL delivery tool. Our results indicate that even though students prefer conventional structured face-to-face collaborative learning over computer-supported structured collaborative learning, working in the latter may improve their performance or learning. Furthermore, the results indicate that in both conventional face-to-face and computer-supported structured collaborative learning, a variety of factors affect the performance of a learner team. These factors include difficulty of the assigned problem and compatibility of team members. Therefore, these factors should be taken into account to build successful teams in structured collaborative learning environment. Acknowledgments

The authors would like to thank the National Center for Information Technology in Education and NSF (SBIR grant DMI-044129) for the research funding. The authors would also like to thank X. Liu, P. Vemuri, S. Namala, and X. Zhang for their programming work. References [1] Aronson, E., N. Blaney, J. Sikes, C. Stephan, and M. Snapp (1978). The Jigsaw Classroom, Beverly Hills, CA: Sage. [2] Beaudoin, M.F. (2002). Learning or lurking? Tracking the “invisible” online student. The Internet and Higher Education, 5(2), pp. 146-155. [3] Clarke, J. (1994). Pieces of the Puzzle: The Jigsaw Method. In S. Sharan (ed.) Handbook of Cooperative Learning Methods, Westport, CT: Greenwood Press. [4] Chalmers, C. & Nason, R. (2005). Group Metacognition in a Computer-Supported Collaborative Learning Environment. Proc. ICCE’05, Singapore, Singapore, pp. 35-41. [5] Chang, Y-W., Ching, E., Chen, C-T. & Chan, T-W. (2005). Computer-Mediated Face-to-Face Interaction Supporting for Peer Tutoring. Proc. ICCE’2005, Singapore, Singapore, pp. 51-58. [6] Daradoumis, T., Guitert, M., Giménez, F., Marquès, J.M. & Lloret, T. (2002). Supporting the composition of effective virtual groups for collaborative learning. Proc. ICCE’02, Palmerston, New Zealand, pp. 332-336. [7] Ikeda, M., Go, S., and Mizoguchi, R. (1997). Opportunistic group formation. In: Boulay, B. d. and Mizoguchi, R. (eds.) Proc AIED’97, IOS, Amsterdam, Netherlands. [8] Johnson, D. & Johnson, R. (1999). Making co-operative learning work. Theory into practice, 38(2), pp. 67-73. [9] Koschman, T., Hall, R., & Miyake, N. (Eds). (2002). CSCL 2: Carrying forward the conversation. Mahwah, NJ: Lawrence Erlbaum Associates. [10] Liu, X., Zhang, X., Soh, L.-K., Al-Jaroodi, J. & Jiang, H. (2003). A Distributed, Multiagent Infrastructure for Real-Time, Virtual Classrooms. Proc. ICCE’03, Hong Kong, China, pp. 640-647. [11] Mulder, I., Swaak, J. & Kessels, J. (2002). Assessing group learning and shared understanding in technology-mediated interaction. Education Technology & Society, 5(1), pp. 35-47. [12] Picciano, A. G. (2002). Beyond Student Perceptions: Issues of Interaction, Presence, and Performance in an Online Course. Journal of Asynchronous Learning Networks, 6(1), pp. 21-40. [13] Ruelland, D.; Brisebois, A. (2002). An electronic performance support system for the eLearner. Proc. ICCE’02. Palmerston, New Zealand, pp. 1072-1076. [14] Soh, L-K. (2004). On Cooperative Learning Teams for Multiagent Team Formation, in Technical Report WS-04-06 of the AAAI’s 2004 Workshop on Forming and Maintaining Coalitions and Teams in Adaptive Multiagent Systems, San Jose, CA, pp. 37-44. [15] Soh, L-K., Khandaker, N. & Jiang, H. (2006a). Multiagent Coalition Formation for Computer-supported Cooperative Learning. Proc. IAAI’2006, Boston, Massachusetts, pp. 1844-1851. [16] Soh, L-K., Khandaker, N., Liu, X. & Jiang, H. (2006b). Computer-Supported Cooperative Learning System with Multiagent Intelligence. Proc. AAMAS’2006, Hakodate, Japan, pp. 1556-1563. [17] Soh, L-K., Khandaker, N., Liu, X. & Jiang, H. (2005). Computer-Supported structured cooperative learning. Proc. ICCE’2005, Singapore, Singapore, pp. 428-435.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

193

An Integrated Framework for Fine-Grained Analysis and Design of Group Learning Activities Seiji Isotani, Riichiro Mizoguchi The Institute of Scientific and Industrial Research Osaka University, Japan [email protected] Abstract: To evaluate the teaching-learning process in collaborative learning sessions and its educational benefits for learners, we should analyze the interaction process during each session and select appropriate learning goals and tasks for each learner. However, the interaction process is very difficult to analyze, even for experts, and furthermore choosing appropriate goals/tasks can be even more challenging. The main objective of our work is to construct a conceptual structure based on ontology to help the interaction analysis and the learning design. We aim to make the tacit benefits for the learners explicit identifying the relationships among interactions and educational benefits. Through this conceptual structure we show how it is possible to analyze and design effective collaborative learning sessions proposing tasks and goals with justification by learning theories. Keywords: Collaborative learning, ontological engineering, instructional design, interaction analysis

Introduction Nowadays, collaborative learning (CL) has become a method increasingly popular used by teachers in classrooms and in e-learning environments. In spite of that, to design effective CL sessions or to analyze the interaction processes among learners, capturing what really happens in each session, has been a very complex job due to a lack of understandable models for representing what is going on [11]. Although there are many research efforts related to the evaluation/analysis of CL sessions, many just consider the quality of the group’s result as a "success" criterion [3]. Nevertheless, according to Dillenbourg [6], the key to understanding collaborative learning is to gain an understanding of the wealth of interactions among the individuals. Therefore, to provide an effective CL session, establishing parameters (goals and tasks) appropriate for each learner, we need comprehensible models to represent a CL session based on interactions among individuals. To deal with the problems presented above, our research requires techniques of ontological engineering to, based on learning theories, clarify the benefits of interactions among individuals during CL sessions. In this work we focus on unifying the models of interaction processes (sub-section 1.1) and learner’s growth (sub-section 1.2), presented respectively in [11] and [12], making the relationship among the desired interaction patterns with the learner’s knowledge acquisition process and the skills development process during a CL session. Through this unification, we intend to help design effective CL sessions by providing: (a) a simple and effective way to select tasks and goals for each learner and estimate their educational benefits; and (b) offering a guideline for blended learning, allowing the designer to combine different learning theories (such as Cognitive Apprenticeship, LPP, Peer Tutoring, etc.) to achieve some desired goal. This paper is organized as follows: First, we introduce two previous models to represent collaborative learning in terms of interaction patterns and learner’s development.

194

S. Isotani and R. Mizoguchi / An Integrated Framework for Fine-Grained Analysis

Next, we propose a new model which unifies the previous models and overcome some limitations of using them separately by offering new alternatives for designing, guiding and analyzing CL sessions. And finally, we present the conclusions of this work. 1. Models to Represent CL Session The goals of many researches in CSCL (Computer Supported Collaborative Learning) include to analyze the interaction processes, considering the interaction among individuals, and to identify their educational benefits [2, 3, 6]. However, it is not common to find models that allow the explicit representation of these processes and what is much more difficult is to find such models that represent the relationship among interaction processes and educational benefits based on learning theories. Such models enable the sharing of findings and the use of computers to support the analysis and design of effective CL sessions. The objective of the following sub-sections is to present the models developed by Inaba et al. which aid the explicit representation of a CL session in such a way that it can be understood, analyzed and shared by teachers, or even by computers. The first sub-section presents vocabularies and a model to represent the interaction processes among learners. The second presents a simplified model to represent the processes of knowledge acquisition and development of skills by the learner. These models will be used in the succeeding sections as foundations of a proposed framework for design and analysis of group activities. 1.1 A Model for Interaction Process To represent the interaction process, Inaba et al. [11], prepared two types of vocabularies: utterance-labels and utterance-types. To label each interaction, we needed a vocabulary at a concrete level (utterance-labels). On the other hand, to characterize a CL session easily, we needed a vocabulary at an abstract level (utterance-types). To satisfy this contradiction, Inaba et al. collected great amounts of data in several CL sessions, and together with other CSCL researchers, defined labels to represent the interactions among the users (utterance-labels). Beside that, through analyses of these labels were created groups of labels, called utterance-types, to represent the interaction process at an abstract level and to distinguish and to characterize each type of CL session. Through the definition of these vocabularies it is possible to define interaction patterns in seven types of interaction processes inspired by learning theories. Figure 1 shows an example of interaction pattern used in Cognitive Apprenticeship [4]. In this example, the interaction patterns are represented with labeled boxes (tasks/interactions described through the use of utterance-types) linked with possible transitions: necessary transitions (solid line), or desired transitions (dotted line). Utterance-type expected to “apprentice” role-player

Request to show the way to solve a problem Prompting

Utterance-type expected to “master” role-player

Showing problematic issues Prompting

Teaching the way to solve a problem

Start Start

Final Final Showing the way To solve a probem

Acceptance

Understanding

Necessary Transitions Desired Transitions

Showing the way to solve a problem

Figure 1: Example of interaction pattern: Cognitive Apprenticeship

When such models as presented in Figure 1 are available, we can explicitly represent typical interaction patterns, and thus, it is possible to compare any interaction process with interaction patterns inspired by learning theories [11]. With the construction of such a model for each desired learning theory we can determine whether the CL session was successful,

S. Isotani and R. Mizoguchi / An Integrated Framework for Fine-Grained Analysis

195

based on the learners' interactions, and estimate the educational benefits for each learner. 1.2 Learner’s Growth Model (LGM) The Learner's Growth Model (LGM) developed by Inaba et al. [12] represents, in a simplified way, the learner's knowledge acquisition process and skill development process, explaining the relationships between learning strategies and their respective educational benefits. For such representation we have to explain more about two processes: learning of knowledge and development of skill. The process of acquiring specific knowledge includes three qualitatively different kinds of learning: accretion, tuning and restructuring [14]. Accretion is to add and to interpret new information in terms of pre-existent knowledge. Tuning is to understand knowledge through its application in a specific situation. Restructuring is to consider the relationships of acquired knowledge and rebuild the existent knowledge structure. Considering the development of skills, there are also three phases of learning: the cognitive stage (rough and explanatory), the associative stage and the autonomous stage [1]. The cognitive stage involves an initial encoding of a target skill that allows the learner to present the desired behavior or, at least, some crude approximation. The associative stage is the improvement of the desired skill through practice. In this stage, mistakes presented initially are gradually detected and eliminated. The autonomous stage is one of gradual continued improvement in the performance of the skill. Inaba et al. [12], developed the LGM model by representing the states of knowledge acquisition and skill development in a graph. However, the original LGM model does not represent the state of restructuring knowledge. Thus, to allow the representation of all states, we worked on improve the LGM model and the result is showed in Figure 2. There are twenty states to represent the levels of the learner’s development at a certain moment of learning. Each state is represented by two triangles. The upper-right triangle represents the state of knowledge acquisition, while the lower-left triangle represents the state of skill development. The arrows show possible transitions between the states and s(x,y) is the simplified form of representing these states in our model: x represents the current state of skill development and y represents the current state of knowledge acquisition. For instance, s(0,0) represents the state where a learner does not have any knowledge or skills to use this knowledge; and s(0,1) represents the state of knowledge acquisition is accretion and the state of skill development is nothing. Using this model it is possible to represent educational benefits of several learning strategies based on learning theories as paths on a graph. Such representation will be explained in details on sub-section 2.2. S(0,0)

[Stages of Skill development]

S(2,0)

S(0,1)

nothing (0) rough cognitive stage (1)

S(1,0)

explanatory cognitive stage (2)

S(3,0) S(0,2)

S(1,1)

associative stage (3) S(3,1)

autonomous stage (4)

S(2,1)

S(4,0) [Stages of Knowledge acquisition]

S(0,3)

S(2,2) S(1,2) S(3,2)

nothing (0) S(4,1) accretion (1)

S(2,3)

tuning (2) restructuring (3) S(1,3) S(3,3)

S(4,3)

S(4,2)

Figure 2: Learner’s Growth Model (LGM)

196

S. Isotani and R. Mizoguchi / An Integrated Framework for Fine-Grained Analysis

2. Unifying the Models: Building and Merging Until now, with the developed models presented in section 1, it is possible to successfully identify which kind of collaboration occurs in a CL session, understanding the essence of the group's interactions (sub-section 1.2), and to estimate the expected educational benefits for each member (sub-section 1.3). Nevertheless, there are some limitations when these two models are not unified: 1. There is no relation among interactions and learner’s growth; 2. We can not explain a path in the LGM graph through a set of events; 3. Difficulty to blend learning theories based on the models. The meaning of blend theories is to blend different learning strategies producing a better learning process; 4. There is no way to intervene while a session is taking place. For example, if a learner, who has a misunderstanding, teaches another learner, he will transfer his problem to the other learner from the beginning until the end of the session. Our propose is to unify these two models extending the Collaborative Learning Ontology [9], which represents the CL process and works as a common vocabulary. We are aiming at supporting the design and analysis of CL processes by representing and storing models of CL in terms of ontologies. Unifying these two models helps to overcome the difficulties addressed above by clarifying the relationships among interaction patterns, learning strategies and learning goals. Furthermore, we believe unifying these models is the first step to explain what a learning theory is, making tacit characteristics explicit: for instance, clarifying expected benefits, use restrictions, guidelines for leading/performing activities, in addition to other important aspects of the teaching-learning process. 2.1 Building the Foundations Influential I_L event

I event Learning Theory

*

Instructional event

Learning Strategy Y<=I-goal

Means to achieve the goal

Instructor Agent

I-role Agent

Instructional action

Agent

Benefits for the Instructor

Action

You-role I-goal G

*

I-goal

Teaching-Learning Process Interaction Pattern Necessary Interaction Activity

* *

Influential I_L event

Desired Interaction Activity Influential I_L event

(a)

I-goal

L event Learning event

Learner Agent

Learning action Action

Benefits for the Learner I-goal

(b)

Figure 3: Example of interaction pattern: Cognitive Apprenticeship

To unify the models, first we represent the interaction patterns using a conceptual structure called Influential I_L event (Figure 3b) and after we propose a conceptual structure for representing an excerpt of the conceptual structure of Learning Theory (Figure 3a), which unifies the models. Using the Influential I_L event structure we divided the interaction process in two events: instructional event and learning event. Every instructional event has a reciprocity relationship with the learning events. In other words, during the teaching-learning process, when a person speaks, the other listens; when someone asks a question, the other answers; and so on. Each event has a corresponding action (or actions) and its possible educational benefits to the initiator. These actions and educational benefits are directly related to the context (learning theory) in which the events and the learning strategies are executed. The representation of the conceptual structure of Learning Theory in Figure 3a consists of two main parts: the Learning Strategy and the Teaching-Learning Process. The

S. Isotani and R. Mizoguchi / An Integrated Framework for Fine-Grained Analysis

197

Learning Strategy specifies how (Y<= I-goal) the learner (I-role) should interact with other person (You-role) to achieve his objectives (I-goal). For instance, in Cognitive Apprenticeship a learner interacts with other learners to guide him during the resolution of a problem. In this case the learning strategy (Y<= I-goal) used by this learner is "learn by guiding", his role (I-role) is known as "master role", the role of the learner who receives the guidance (You-role) is known as "apprentice role", and the goals of the learner who guide (I-goal) are to acquire cognitive skills (and meta-cognitive skills) at an autonomous level. Previous works of Inaba et al. [9, 10] show the strategies (Y<= I-goal), learner’s roles (I-role and You-role) and individual goals (I-goal) of several learning theories. The Teaching-Learning Process specifies the interaction pattern of a learning theory represented by the necessary and desired interaction activities (processes) among two people (for instance, master and apprentice). As we mentioned before, we can describe interactions using the I_L event for explicitly represent the interaction and its benefits from both points of view: for those who do the action and for those who receive the action. Thus, to specify the teaching-learning process we mapped the interaction pattern presented in section 1 to fit in our influential I_L event structure. At present, with this mapping, we identified more than 13 I_L events and its respective benefits used by seven different learning theories: Cognitive Apprenticeship [4], Anchored Instruction [5], Peer Tutoring [7], Cognitive Flexibility [16], LPP [13], Socio-Cultural Theory [17] and Distributed Cognition [15]. Table 1 shows some I_L events used by Cognitive Apprenticeship (CA) and Anchored Instruction (AI), and their expected benefits for instructor and learners. Table 1. Some Influential I_L events and its benefits in the context of two learning theories Influential I_L events Affirmative reaction

Clarify the problem Demonstration of how to solve a problem Instigating thinking

Monitoring

Event (Instructor/Learner) Acceptance/ Understanding Identifying learner’s problem/ Externalization of problem Demonstration/ Observing demonstration Argumentation/ Analyzing arguments

Checking/ Carrying out a task

Notifying how the learner is

Giving information/ Processing information

Requesting problem’s details

Asking about problematic understanding/ Pointing out problematic understanding

Setting up learning context

Set information context/ Contextualization of information

Showing a solution

Explanation/ Understanding explanation

Learning Theory

Expected benefits (I-goal) Instructor Learner

CA

s(3, 2) ĺ s(4, 2)

s(2, x) ĺ s(3, x), x=0,1,2

AI

s(2,y) ĺ s(3,y), y=1,2

s(x,1) ĺ s(x,2), x=1,2,3,4

CA

s(3, 2) ĺ s(4, 2)

s(0, x) ĺ s(1, x); s(1, x) ĺ s(2, x), x=0,1,2

CA

s(3, 2) ĺ s(4, 2)

s(0, x) ĺ s(1, x); s(1, x) ĺ s(2, x), x=0,1,2

CA

s(3, 2) ĺ s(4, 2)

s(1, x) ĺ s(2, x), x=0,1,2

CA

s(3, 2) ĺ s(4, 2)

s(1, x) ĺ s(2, x); s(2, x) ĺ s(3, x), x=0,1,2

AI

s(2, y) ĺ s(3, y), y=1,2; s(z, 1) ĺ s(z, 2), z=2,3

s(x,0) ĺ s(x,1); s(x,1) ĺ s(x,2), x=1,2,3,4

CA

s(3, 2) ĺ s(4, 2)

s(1, x) ĺ s(2, x), x=0,1,2

AI

s(2, y) ĺ s(3, y), y=1,2;

s(x,0) ĺ s(x,1), x=1,2,3,4

CA

s(3, 2) ĺ s(4, 2)

s(2, x) ĺ s(3, x), x=0,1,2

AI

s(2, y) ĺ s(3, y), y=1,2

No expected benefit

CA AI CA AI

s(3, 2) ĺ s(4, 2)

s(0, x) ĺ s(1, x), x=0,1,2

No expected benefit s(3, 2) ĺ s(4, 2) s(2, y) ĺ s(3, y), y=1,2;

s(x,0) ĺ s(x,1); s(x,1) ĺ s(x,2), x=1,2,3,4 s(2, x) ĺ s(3, x), x=0,1,2 s(x,1) ĺ s(x,2), x=1,2,3,4

In spite of the influential I_L event has one main objective it is worth to point out that for each learning theory the same I_L event may have different learning purposes, and

198

S. Isotani and R. Mizoguchi / An Integrated Framework for Fine-Grained Analysis

for this reason, it may have different actions and/or different expected benefits. It happens because each theory is looking for helping the learner concerning different states of knowledge and different states of skill development using different learning resources. For example, although the I_L event “Setting up the learning context” is used to contextualize the learner for a better understanding of the content, as we showed on Table 1, in the context of Anchored Instruction we expect learners to acquire some content specific knowledge, and in the context of Cognitive Apprenticeship we expect learners to develop some skills. 2.2 Merging the Models Observing Figure 3 that with the representation of interaction patterns through I_L events and using our conceptual structure of Learning Theory, we can identify the interactions and their benefits for Instructor and learner in the context of a learning theory, and thus, we realize the unification of the models presented in section 1. This unification can be graphically represented by a path on the LGM graph and associating each graph’s edges with the Influential I_L events which correspond to a specific change of the learner’s state in the graph. In Figures 4 and 5, we use the learning theory “Cognitive Apprenticeship” to demonstrate how we can unify the models, clarifying which activities of the chosen interaction pattern can help the learner's development during different phases of learning. Figure 4 shows the result of mapping the interaction pattern of Cognitive Apprenticeship (Figure 1) into Influential I_L events. The boxes are labeled with one number followed by one I_L event. As in Figure 1 we represent the transitions between boxes as necessary transitions (solid line), or desired transitions (dotted line). Each number in the boxes is used to represent the followed I_L event in our proposed model (Figure 5). 8: Showing a solution

7: Requesting problem's details

6: Instigating Thinking 5: Notifying how the learner is

1:Setting up learning context

2: Demonstration how to solve a problem

4: Monitoring

9: Affirmative reaction

3: Clarifying the problem

Figure 4: Interaction Pattern of Cognitive Apprenticeship represented by Influential I_L events

As one of the results of this work, we show in Figure 5 an example of the unification of the models graphically represented by improving our LGM Model (section 1.2) for Cognitive Apprenticeship augmented by the learning strategy “learning by apprenticeship”. This model is improved by labeling each arrow with specific Influential I_L events (interactions inspired by learning theories) that facilitate the transitions among states. We call this model of GMIP –Growth Model improved by Interaction Patterns. The bold arrows represent the transition from one state to the other which is facilitated through this learning strategy using the labeled interactions; the dashed arrows represent the facilitation of the transition. There are two kinds of interactions: the necessary interactions, represented by a black circle, and the desired interactions, represented by a white circle. The interactions are linked by ellipses. The dashed ellipse represents a directed link between two interactions (I_L events) in Figure 4 and the full ellipse represents a no-directed link between two interactions, it means that in Figure 4 there is a cycle between these two interactions. In the context of Cognitive Apprenticeship shown in Figure 5, the Influential I_L events “1:Setting up the learning context”, “2:Demonstration how to solve a problem” and “3:Clarify the problem” are events to facilitate learners who do not have any cognitive skill, s(0, x), to get some cognitive skills in rough cognitive stage, s(1, x). The same events “2” and “3” above together with “4:Monitoring”, “5:Notifying how the learner is”, and

S. Isotani and R. Mizoguchi / An Integrated Framework for Fine-Grained Analysis

199

“6:Instigating thinking” also facilitate learners with cognitive skills in rough cognitive stage, s(1, x), to achieve the explanatory cognitive stage, s(2, x), and so on for the other stages.

Figure 5: Example of GMIP for Cognitive Apprenticeship Model

The main contribution of our proposed model GMIP is to solve, at least partially, the problems presented in beginning of section 2. This model clarifies, more precisely, how interactions can affect learner’s development, facilitating the learning design based on events. Thus, it becomes a powerful tool helping designers to select events (interactions) and roles for each learner, based on interaction patterns and learning strategies appropriate for desired learning goals and sub-goals (and vice versa). Furthermore, it is possible to offer new alternatives for designing, guiding and analyzing CL sessions. For example: (a) for each sub-goal, it is possible for the teacher to intervene, for guiding learners or analyzing collaboration outcomes while a CL session is not finished, as opposed to adjustments after it has ended, as is usually the case. Observe that we are not trying to say that it is possible to intervene in real time. What we point out is the possibility of split the collaboration in several steps (sub-goals) allowing the teacher’s intervention after each step. Another interesting example is the possibility of blend learning strategies based on our proposed model. It can be done by blending two or more strategies to achieve one desired goal. Thus, using one learning strategy, after achieving a desired sub-goal, we can change for another learning strategy to obtain another desired sub-goal that the first one can not offer. That is, with our model we can realize a guideline for blended learning theories. With the above possible use of the GMIP model in our mind, to (a) provide input data to setup programs for designing CL sessions and to analyze group interactions; and (b) to develop programs for help collaborative learning; we have implemented the conceptual structure presented in section 2.1 using the ontology editor Hozo (available at http://www.ei.sanken.osaka-u.ac.jp/hozo) extending the CL Ontology [9]. 3. Conclusions The possibility of clarifying what a CL session is and to amplify its educational benefits, providing resources that facilitate its representation, design and analysis has been a great challenge. In this paper we used two models previously developed, the Interaction Pattern [11] and the Learner’s Growth Model [12], and worked on clarifying the relationships among interaction patterns, learning strategies and learning goals. As a result, we have

200

S. Isotani and R. Mizoguchi / An Integrated Framework for Fine-Grained Analysis

proposed an integrated model, called GMIP, which unifies the previous models through the development of a conceptual structure which extends the CL ontology and represents an excerpt of the learning theory concept. This model has been implemented using the Hozo ontology editor and can be used to develop programs for CL design and analysis. There are, at least two, main benefits provided by our GMIP model. First, it helps the analysis of group’s interactions contributing to a more precise analysis of a CL session, estimating educational benefits while a collaborative session is not finished. And second, it offers a guideline for blended learning based on learning theories which helps designers to identify more easily the role and kind of interactions (and actions) should be practiced by learners to achieve a desired goal or sub-goal in CL sessions. Our future researches include a study demonstrating some examples and possibilities to blend learning strategies semi-automatically based on GMIP model. This is another step forward in the improvement of ontology-aware authoring systems for collaborative learning that offer help in designing learning activities based on learning theories, while providing an easy way to analyze interactions among learners and to estimate educational benefits. Our ultimate goal is to completely develop such an ontology-aware authoring system. References [1] Anderson, J. R. (1982) “Acquisition of Cognitive Skill”, Psychological Review, 89(4), pp. 369-406. [2] Barros, B., & Verdejo, M.F. (2000) “Analyzing student interaction processes in order to improve collaboration. The DEGREE approach”, Int. Journal of Artificial Intelligence in Education, Vol. 11, pp. 221-241. [3] Collazos, Cesar A., Guerrero, Luis A., Pino, Jose A., Ochoa, Sergio F. (2004) “A method for evaluating computer-supported collaborative learning processes”, Int. Journal of Computer Applications in Technology, Vol. 19, No.3/4, pp. 151 – 161. [4] Collins, A. (1991) “Cognitive apprenticeship and instructional technology”. In: Idol, L., and Jones,B.F. (Eds.) Educational values and cognitive instruction., LEA.. [5] Cognition and Technology Group at Vanderbilt (1992) “Anchored instruction in science education” In: R. Duschl & R. Hamilton (Eds.), Philosophy of science, cognitive psychology, and educational theory and practice, Albany, NY: SUNY Press, pp. 244-273. [6] Dillenbourg, P. (1999) “What do you mena by Collaborative Learnng”, Collaborative Learning and Computational Approaches, Oxford: Elsevier Science, pp. 1-19. [7] Endlsey, W. R. (1980) “Peer tutorial instruction”, Educational Technology. [8] Salomon G. (1996) “Distributed Cognitions: Psychological and Educational Considerations”, Cambridge University Press. [9] Inaba, A., Supnithi, T., Ikeda, M., Mizoguchi, R., Toyoda, J. (2000) “How Can We Form Effective Collaborative Learning Groups?”, Proc. of the International Conference on Intelligent Tutoring Systems, Montreal, June, pp. 282-291. [10] Inaba A., Mizoguchi, R. (2004) “Learner’s Role and Predictable Educational Benefits in Collaborative Learning”, Proc. of International Conference on Intelligent Tutoring Systems, Alagoas, pp. 285-294. [11] Inaba, A., Ohkubo, R., Ikeda, M., & Mizoguchi, R. (2003a) “Models and Vocabulary to Represent Learner-to-Learner Interaction Process in Collaborative Learning”, Proc. of the International Conference on Computers in Education, Hong Kong, pp.1088-1096. [12] Inaba, A., Ikeda, M., & Mizoguchi, R. (2003b) “What Learning Patterns are Effective for a Learner’s Growth?”, Proc. of the International Conference on Artificial Intelligence in Education, Sydney, pp. 219-226. [13] Lave, J., Wenger, E. (1991) “Situated Learning: Legitimate peripheral participation”, Cambridge University Press. [14] Rumelhart, D.E., & Norman, D.A. (1978) “Accretion, Tuning, and Restructuring: Modes of Learning”, Semantic factors in cognition. LEA, pp. 37-53. [15] Salomon, G. (1993). “Distributed Cognitions”, Cambridge University Press. [16] Spiro, R.J., Coulson, R.L., Feltovich, P.J., & Anderson, D.K. (1988). Cognitive flexibility theory: Advanced knowledge acquisition in ill-structured domains. In The tenth annual conference of the cognitive science society. Hillsdale, NJ: Lawrence Erlbaum Associates, pp 375-383. [17] Vygotsky, L.S. (1978) Mind in Society: The development of the higher psychological processes. Cambridge, MA: Harvard University Press.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

201

The Development of a Grouping System in a Collaborative Learning Environment Pao-Ta Yu, Yen-Shou Lai, Chia-Ming Liu, Jenq-Muh Hsu Department of Computer Science and Information Engineering, National Chung Cheng University, Taiwan, ROC [email protected] Abstract: This study develops a mobile learning and grouping system based on thinking styles and Bloom’s taxonomy in a collaborative learning environment. The environment takes cooperative learning as the instructional strategy, and Student Teams-Achievement Divisions (STAD) as the cooperative learning method. The purpose is to propose a method of heterogeneous grouping that students not only use mobile learning but also use students’ thinking styles for grouping. Participants in the experiment include three classes of 96 sixth-grade students which are assigned to one control group and two experimental groups. Data collection includes mathematical anxiety questionnaires, learning sheets, and achievement test. The results show that the grouping method has better learning effects than mixed grouping and individual learning. Keywords: Cooperative learning, mobile learning, STAD, thinking styles

1. Introduction 1.1. Background In traditional classrooms, the teaching pattern is the hallmark of teacher-fronted classroom. Teachers usually use chalk, blackboard, and textbook as teaching tools. In recent years, mobile learning is developed. It takes place via the wireless platform such as a mobile phone, PDA, and laptop for teachers and students to build a learning environment. There is a rich and long history of practical use of cooperative learning. Many studies show that Student Teams-Achievement Divisions (STAD) cooperative learning group has different superior than control group in learning effects after teaching [1]. The experimental scopes include many course subjects, and there are positive effects about math learning effects. For this reason, the study tries to divide a class into heterogeneous groups based on two factors, including the question level of study sheet and the style of student. The questions of study sheet are classified with Bloom's Taxonomy. Students are classified with thinking styles. On the other hand, there are few studies for cooperative learning method regarding math anxiety. Therefore, the heterogeneous grouping and peer instructions have become a worth research topic that whether it can effectively decrease students’ mathematical anxiety in the mobile learning environment. 1.2. Literature review Cooperative learning is the instructional use of small groups within the classroom as well as across classrooms [2], so that students work together to maximize their own and each other's learning. It is one of the efficient ways to improve students' ability of thinking, solving

202

P.-T. Yu et al. / The Development of a Grouping System in a Collaborative Learning Environment

problems, combining and using knowledge technology, and students take this spirit of cooperation with them as they go out into the wide world [3]. STAD is one of Cooperative learning strategies. It can improve students' academic achievement, interpersonal relations among the colonies, and attitudes toward learning, schools, peers, and self. Bloom's taxonomy provides a helpful method of improving the exchange of test materials, ideals about testing. It is a classification system of educational objectives based on the level of student understanding necessary for achievement or mastery. In addition, it could be useful in motivating research on examining and on the relations between examining and education. Thinking styles are the theory of mental self-government. The theory of mental self-government addresses the forms of government. When people face many different situations, people choose styles of managing themselves with which they feel satisfied [4]. Table 1 shows the relationship table of Bloom’s taxonomy and thinking styles [5]. Table 1 The relationship table of Bloom’s taxonomy and thinking styles Bloom’s Taxonomy Synthesis Knowledge/ Comprehension Application/ Analysis/ Evaluation

Suitable Question Type Creativity Memory

Suitable Thinking Style Legislative Executive

Analysis/ Macroanalysis/ Microanalysis

Judicial

2. System Architecture The Mobile Classroom and grouping system adopt mobile device such as notebook or PDA. AODV is selected as the routing protocol that used for ad hoc mobile networks. The API of NetMeeting SDK is used in the Mobile Classroom. The Unified Modeling Language (UML) is used to communicate software design during software development activities. STAD Cooperative Learning Method is a combination of in-group cooperation, inter-group competition, and group learning. Students take a regular quiz and groups receive recognition for sum of the improvement scores of group participants. The study sheet is intended to help students prepare for learning material. Grouping method is based on two factors, including the question level of study sheet and the style of students. The questions of study sheet are classified with Bloom’s Taxonomy. Students are classified with thinking styles. After group learning, students take the quiz independently of their mates. Mobile Classroom is a software application that supports group working and group learning. It is used for student-to-teacher or student-to-peer communication through the wireless network. The functions of Mobile Classroom contain Application Sharing, Chat Room, Whiteboard, and File Transfer.

3. Research design and implementation 3.1 Participants and materials The participants were 96 sixth grade students (12 years old in average) selected from three classes in an elementary school. There were six course units in mathematics, arranged by the way of structural instruction method, and the experiment period was about two months. The

P.-T. Yu et al. / The Development of a Grouping System in a Collaborative Learning Environment

203

experimental groups were grouped with the automatic grouping method that proposed by this thesis (class A) and with randomly mixed grouping (class B) in the mobile learning environment. The control group used traditional instruction with individual learning (class C). Therefore, each class used various instruction methods to learn. The Mathematics Anxiety Rating Scale (MARS) was edited to measure the degree of students’ mathematical anxiety. There were 32 questionnaires, including worry, dislike, anxious test, and pressured sense. The questionnaires use five-point Likert scale, and a one-point is the most disagree factor, five-point is the most agree factor. The Cronbach’s coefficient Į was .90, so the reliability of this study was acceptable. ʳ

3.2 System implementation The experiment procedure had five steps: preparing, pretest, the training of cooperation skills and system operating, experiment, and posttest. In the experimental group A, the teacher prepares Thinking Styles Inventory and students take the survey. Next the teacher prepares and classifies the questions of study sheet. The questions of study sheet are classified with Bloom’s Taxonomy. The teacher can use automatic or manual grouping to create new cooperative learning groups. Fig. 1 shows the flow chart of the grouping procedure.

Fig. 1. The flow chart of the grouping procedure. In the experimental group B, the students were grouped randomly or grouped by themselves. For the two experimental groups, the teacher creates a mobile classroom and waits for students to join. When all students joined, the teacher provides the whole class with instruction on a particular subject. They can interact with each other by used Application Sharing, Chat Room, and Whiteboard. Students study the subject in their own groups in preparation for a quiz. They take a quiz independently of their group members. The cooperative learning group receives group recognition based on their scores and past average scores. In the control group C, the students accepted the traditional instruction. In order to compare the differences between control and experiment groups for the mobile classroom, one-way analysis of variance (one-way ANOVA) is applied. The significance level of the hypothesis test is 0.1.

204

P.-T. Yu et al. / The Development of a Grouping System in a Collaborative Learning Environment

4. Results The experiment results of mathematics achievement test are shown in Table 2. When we adopt cooperative learning, the mean of the experiment group A is 73.1 (SD= 9.6) and the group B is 70.8 (SD= 9.9). The control group C is 66.6 (SD=12.4). One-way ANOVA shows that there is difference between three groups (F=2.898, P<0.1). A posteriori Tukey HSD method is took for exploratory data analysis. It shows that there is no significant difference between experiment A and B (p=0.657), also between experiment B and C (p=0.293). But there is significant difference between experiment A and C (p=0.05). Table 2 The means (M) and standard deviations (SD) of Mathematics Achievement Test Sample sizes M SD A (Bloom- Thinking) 32 73.1 9.6 B (Random) 32 70.8 9.9 C (Individual) 32 66.6 12.4 Totals 96 70.2 10.9 The Mathematics Anxiety Rating Scale (MARS) shows that there is no difference between the three groups (F=0.46, P>0.1). It means that although mathematics anxiety scores of experiment groups are lower than control group, but it does not reach the statistical difference.

5. Conclusions The prime purpose of the system described in the study is to build mobile learning and students grouping environments into an existing cooperative learning. This system allows the instruction and learning activities don’t limited in the time and space. The teacher and students in this distributed environment can interact freely with each other. During the cooperative learning, the teacher can use the grouping method that Bloom's taxonomy will be a useful tool. It could be helpful in stimulating research on examining and on the relations between examining and education. For the future research, we can extend experiments on different learning areas, students, and teaching time, or extend teaching materials to Bloom’s Taxonomy of Affective Domain and Psychomotor Domain. Finally, it is useful to establish customized m-Learning and instructions management platform that participants can choose their favorite layout to view. Acknowledgments We would like to thank the National Council of Taiwan, R.O.C., for supporting this research under Contract Nos. NSC 4-2524-S-194-005- and NSC 5-2524-S-194-001-. References [1] Slavin, R. E. (1991). Synthesis of research on cooperative learning. Educational Leadership, 48, 71-82. [2] Ligorio, M. B., and Veermans, M. (2005). Perspectives and patterns in developing and implementing international web-based collaborative learning environments. Computers & Education, 45(3), 271-275. [3] George, M. J., and Michael, A. P. (2002). The teacher’s sourcebook for cooperative learning: practical techniques, basic principles, and frequently asked questions. Corwin Press, Inc. [4] Zhang, L. F. (2002). Measuring thinking styles in addition to measuring personality traits? Personality and Individual Differences, 33(3), 445-458. [5] Passig, P. (2003). A taxonomy of future higher thinking skills. Source Informatics in education, 2(1), 79-92.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

205

Students’ Understandings and Attitudes toward Group learning: An Empirical Study ab

Jianhua Zhao a, David McConnell b Department of Educational Research, Lancaster University, United Kingdom [email protected]; [email protected] Abstract: Group learning is a fundamental method to organise learners participating in a learning process, traditionally in the classroom-based environments, and recently in the web-based environments or the blended e-learning environments. The purpose of this study is to examine students’ understandings and attitudes toward group learning in the blended e-learning environments in Chinese Higher Education Institutions (CHEIs). 20 students were involved in a course - “Computer in Education (I)” – and group learning was used to organise the relevant learning and teaching process in a blended e-learning environment. The methodology of this study is action research and the data were collected through questionnaire survey, group interview, and personal blogs. Students’ understandings of group learning were examined and the results reveal big change from absence to familiarity between the beginning and the end of the course. This result also demonstrated that students’ attitudes to group learning were positive. Keywords: group learning, blended e-learning, knowledge building, group knowledge building, action research

1. Introduction Group learning is an effective method to organise students involved in a learning process, showing advantages of cooperative settings over competitive or individual settings, and is commonly used in the classroom-based learning environments (Webb, 1982; Slavin, 1980; Sharan, 1980), and in the web-based learning environments currently (McConnell, 2006; Masten & et al., 2002; Jonassen & Kwan, 2001). However, it is not a commonly used method in Chinese educational settings, because competitive and individual learning approaches are still dominated (Zhao, 2006; Xu, 1996; Zhao & Zhang, 1999). Even though an educational reform in China has taken place in the past decade, the conventional teaching ideology is still powerful and difficult to change within a short time. In this study, we examine students’ understandings and attitudes toward group learning in Chinese higher education in order to identify the reason why it is not commonly used in the Chinese context. We argue that there is no essential barrier for Chinese students engaging in group learning, but the reason is they do not use it properly because teachers and students are still dominated by the lecturing approach. Numerous researchers in the field have explored students’ perceptions (or understandings) and attitudes using different methods. Thompson and Ku (2005) conducted a research to examine Chinese graduate students’ experiences and attitudes toward online learning. The methodology they used was qualitative case study and the results demonstrate that it was an interesting experience for participants, but their attitudes toward this unfamiliar mode of learning were mixed. However, the seven participants in this study came from a research-intensive University in the western USA, so the results may not reflect on Chinese students studying in China.

206

J. Zhao and D. McConnell / Students’ Understandings and Attitudes Toward Group Learning

In this study, we developed a mixed approach to examine students’ understandings and attitudes toward group learning. 2. Methodology 2.1 Participants, the Course, and the Learning Environments Participants in this study were undergraduates at the School of Information Technology in Education (SITE), South China Normal University. SITE is the leading institution in Educational Communication and Technology field in China. 23 students took the course – Computer in Education (I), 3 students withdrew at the middle of the course. Two kinds of the classroom-based settings were used in this study. One setting is a conventional classroom with ICT facilities, such as digital project, wireless audio system, central control system, and digital camera, etc. Another was a network-based computer classroom where provided 32 computers for students. All computers were connected with internet. The online learning environment of this study was WebCL system developed by Networked Education Laboratory of Beijing Normal University. 2.2 Data collection A questionnaire survey was conducted three times in this course. At the beginning of the first class, the first questionnaire was distributed to students. The total number of questionnaires were 23 and 17 of them were collected. The rate of collection is 73.9%. All of the collected questionnaires were valid data for analysis. The second questionnaire was conducted at the middle of the course. The questions were designed mainly to examine students’ understanding of group learning. 19 of them (total number is 20) gave their anonymous response and all the collected questionnaires were valid. The valid rate of this survey is 95%. The third questionnaire was used at the end of course in order to examine students’ overall understanding and attitudes toward group learning. 16 students participated in this survey and 13 of them gave their feedback. The collected questionnaires were all valid and the valid rate is 81.25%. Three groups (group 2, 3, and 4) engaged in the first focus group interview at the middle of the course and two groups (group 1 and group 5) involved in the second one at the end of the course. Interview time for each group was around one hour. All the interview processes were audio-and-video recorded. All students attended the focus group interview actively. 3. Findings and Discussions 3.1 Students have little experience of group learning at the beginning of the course, and their attitudes toward group learning varied In order to examine students’ background about group learning, the question “have you use group learning before” was asked in the first survey. 14 students (63.6%) replied that they had this kind of experiences before and 8 students (36.4%) said they had no experience of it. For those students who had group learning experiences, what were their attitudes toward it? Their statements in the questionnaire introduced their own experiences of group learning. Some students had positive attitudes toward group learning. Kingston: I feel good when I engaged in the group learning process. However, I found it is not ideal approach for individual learning. Zhenzhen: Group learning can let us learn more and know more. Inter-group cooperation can be used to enforce the students’ friendship.

J. Zhao and D. McConnell / Students’ Understandings and Attitudes Toward Group Learning

207

These two students’ viewpoints represent most students who had replied positively. However, their understandings of group learning were still not properly. Actually, they had 15 years learning experiences from school to university, which implies that they form correct understanding of group learning. It is real because there are still not too much tutors like to use group learning to organise their teaching process in Chinese schools, colleges and universities. 3.2 Students’ understandings of group learning improved at the middle of the course, and students presented their positive attitudes to group learning For the question “do you think you completely understand and know how to engage in the group learning activities”, 26.3% students thought they already completely understood it, 52.6% students considered they understood it well, and 21.1% students acknowledged they had a generalised understanding about it. The results demonstrate that students have already understood group learning well. The data from focus group interviews reveal that they have already formed proper understandings of group learning, for example: Pearl: Um … I have no group learning experiences at all before I attended this course. I thought group learning provides a chance to students, which we can meet together. You know, we can learn how to cooperate in a group, um … because we had to finish our group task in time. Otherwise, we could be lost credits. Um … it is a kind of external pressure. I do believe everybody in the group have this pressure. However, you know, when we are familiar each other in a group, we can do our work well.

They had also introduced some problems they identified and experienced in their group learning process: Billy: The big problem for our group learning is time limitation. Um …, it is really boring for us to choose so many courses in a term. However, we had to do it. Otherwise, we would not graduate one year later. Um … I thought it is a big pressure for us, especially for this course. We have not too much time to use in this course

Students’ attitudes toward group learning were positive, such as: Tina: Actually, I like group learning very much, because I can get help from other students. Um … I thought this is a good way to foster cooperative and collective ability. Especially, um … lots of new ideas can be proposed by group. This is a good environment to in-depth understand and explore the knowledge what we learnt.

3.3 Students because familiar with group learning formed in-depth and appropriate understandings of group learning at the end of the course, and their attitudes toward group learning were positive 100% students considered that they had become familiar with group learning at the end of the course. Meanwhile, students presented their in-depth and appropriate understandings of group learning, for example: Tina: Members should positively attend group learning and communication. The time issue for group learning is quite important because the relevant activities should have the enough time to finish. We must have the background for the certain learning events. Otherwise, it is difficult to form the prerequisite of the group learning. Learning ability is important as well for group members. Everybody in the group should know how to communicate well.

The data from focus group interview demonstrate their understandings and attitudes toward group learning, such as: Luorongda: At the beginning of the course, I don’t like group learning at all. I had the experiences to attend group learning before. However, it is really boring! My group-mates have not this kind ability. Nobody likes to give their extra time to the group learning. So the effectiveness of group learning is quite bad. I think the group learning in this course changes

208

J. Zhao and D. McConnell / Students’ Understandings and Attitudes Toward Group Learning

my attitude to it. Currently, I like group learning very much, because everybody in the group is familiar to cooperate each other. Even though the time for us is also limited very much, we all feel we have accountability to do the group work well. This is the big change! So I also think individual accountability is quite important. Everybody in the group must feel they are not dispensable, because they are all important to the group work. Definitely, something is not ideal within the group learning process. Sometimes I have to suffer the time of group learning, because the efficiency and effectiveness are quite lower. However, I think I also can learn from this weakness of our group learning. A Chinese proverb said “the failure is the mother of success!”

4. Conclusions The findings of this study illustrate that these students have not understood the meaning of group learning well at the beginning of the course, but at the middle and the end of the course, their understandings of group learning were well-established. Students presented their diverse attitudes toward group learning at the beginning of the course, and their attitudes to positive at the end of the course. The results also demonstrate group learning is not a commonly used approach in the Chinese educational field, because lots of students had no group learning experiences when they involved in the course at the beginning. The finding they liked group learning at the end of the course reveals that it can be an effective method to organise the learning and teaching activities in Chinese universities in this subject areas. Acknowledgments We thank the students who were engaged in this course of study, and we cannot finish this paper without their contribution. References Boschee, F. (1991). Small-group learning in the information age. Clearing House, 65, 2. Johassen, D., & Kwan, H. II. (2001). Communication patterns in computer mediated versus face-to-face group problem sovling. Educational Technology, Research and Development, Vol. 49, No. 1. Masten, S. J., Chen, K. C., Graulau, J., Kari, S. L., & Lee, K. H. (2002). A web-based and group environment for introductory environmental engineering. Journal of Engineering Education, Vol. 91, No.1. McConnell, D. (2006). E-learning Groups and Communities. Maidenhead: Open University Press. Scharan, S. (1980). Cooperative learning in small groups: Recent methods and effects on achievement, attitudes and ethnics relations. Review of Educational Research, 50, 241~272. Slavin, R. E. (1980). Cooperative learning in teams: State of the art. Educational Psychologist, 15, 93~111. Thompson, L, & Ku, H. Y. (2005). Chinese graduate students’ experiences and attitudes toward online learning. Educational Media International, 42, 1, 33~47. Webb, N. M. (1982). Group composition, group interaction, and achievement in cooperative small groups. Journal of Educational Psychology, 74, 4, 474~484. Xu, Q. (1996). Characteristics and reform of educational administrative system in our country. Journal of Shanghai Higher Education Research, 6, 59~62. Zhang, X. & Zhang, J. (2004). Research on modern educational technology and instructional model. Journal of Wuhan Institute of Technology, No. 6. (in Chinese). Zhao, J., & McConnell, D. (2006). The Differences of Group learning in the Classroom-based and the Web-based Environments: Factor Analysis. In Sheena Banks, Vivien Hodgson, Chris Jones, Bob Kemp, David McConnell and Christine Smith (Eds.), the processing of the Fifth International Conference on Networked Learning. Lancaster: Lancaster University.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

209

The Effectiveness of Knowledge Building through Computer Supported Collaborative Learning among Elementary Students: A Case Study TSE Wing Cheung Alex, LEE Fong Lok, OU Yong Centre for the Advancement of Information Technology in Education, The Chinese University of Hong Kong, Hong Kong SAR, China [email protected] Abstract: This paper presents the third phase of a research project on Computer Supported Collaborative Learning (CSCL) involving students from 81 schools in Hong Kong and mainland China working on the topic of environmental protection. Instead of capturing an overall picture of all the participating schools, this phase of research focused on the effectiveness of knowledge building through CSCL by an in-depth investigation of the learning processes of students in two of the participating elementary schools. Scardamalia’s principles of knowledge building were adopted to evaluate the effectiveness of knowledge building and 1,298 messages of online discussions posted by 153 students over six months were systematically classified, analyzed, synthesized and evaluated. Though some overseas research reported that even young children can meaningfully engage in knowledge building and benefit from the process through CSCL, this study found that the knowledge building was not as effective as expected, still students were found to have the potential of knowledge building in certain dimensions. This study recommends that teachers can provide better support for their students to facilitate them to turn their own ideas into researchable questions for investigation, improve ideas by criticizing the contributions of others, contribute to a wide variety of ideas and build up their own ideas by critically referring and comparing authoritative sources. Keywords: knowledge building, Computer Supported Collaborative Learning, elementary students

Introduction There has been a booming development in Computer Supported Collaborative Learning (CSCL) over the past few years. The pool of professional and research literature increased rapidly. CSCL is one of the hot topics in various international conferences on Information and Communications Technologies in Education (ICTE). It witnessed a drastic development of the topic, revealing its significance in shaping teaching and learning of the world. As its name reflects, CSCL generally refers to a teaching and learning approach in which web-based or network-based platforms, like the Knowledge Forum, is adopted (Law & Wong, 2003). Over a period of time of implementing the CSCL platform, the participants can conduct online discussions collaboratively. CSCL can be particularly useful to facilitate collaborative knowledge building among participants. Such an emerging approach of CSCL can be versatile to suit the instructional needs of a wide variety of teaching and learning contexts, making CSCL more and more popular (MacKinnon, 2004; Chai, Tan & Hung, 2003; Hew & Cheung, 2003). The implementation of CSCL generally targets at nurturing higher-order thinking skills such as problem solving abilities and collaborative refinement of knowledge within a topic or authentic and complex contexts. Therefore, how to evaluate its effectiveness is really challenging to researchers. Meanwhile, CSCL is becoming more and more popular in

210

W.C.A. Tse et al. / The Effectiveness of Knowledge Building

various levels of schools. There are high demands for more in-depth researches to throw light on the issue so as to guide the implementation of CSCL on the right track. Though overseas research found that even young children could meaningfully engage in knowledge building and benefit from the process, there were many reports of implementation problems of CSCL in elementary schools (Law & Wong, 2003; Tse, Lee & Ou, 2006a; Tse, Lee & Ou, 2006b). It explains why this study chooses elementary students as the focus of study. The significance of this research is thereby revealed. 1. Three “I” Project This paper presents the third phase of the research project of CSCL among 81 schools in Hong Kong and mainland China working on the topic of environmental protection. The project is called the Global Learning Community among Primary Education through Interdisciplinary, Interschool and International Project Learning (Tse, Lee & Ou, 2006a; Tse, Lee & Ou, 2006b). It is called the 3I Project below. The students in Hong Kong conducted their project learning through discussion in web-based CSCL platform with the students in mainland China. Instead of capturing an overall picture of all the participating schools, this phase of research focused on the effectiveness of knowledge building through CSCL by an in-depth investigation of the learning processes of students in two of the participating elementary schools. 2. Two Schools of Study In this study, the students in a Hong Kong elementary school conducted their project learning with the students from an elementary school in a city of mainland China, through a web-based CSCL platform between November 2005 and April 2006. The 1,298 messages of online discussions were posted throughout the learning process. As found in the platform, the 153 students of the 2 schools were divided into 12 learning groups. For almost all of these students of Primary Five (Grade 5), it was their first time to learn in such a context and approach under CSCL. 3. Research Design Focusing on the effectiveness of online discussions of knowledge building, this case study of two schools mainly relied on online platform observations after the end of online discussions. It was both quantitative and qualitative. The quantitative approach mainly emphasized the descriptive analysis of the statistics from various online activities while the qualitative approach stressed the evaluation of the content of each message posted by the students. In this study, 1,298 messages were classified, analyzed, synthesized and evaluated. Scardamalia‘s principles of knowledge building were adopted to analyze the messages and the online interactions (Scardamalia, 2002). To maintain the credibility of the study, besides adopting data of online platform observations, focused-group student interviews and teacher interviews were also conducted among teachers and students of 2 schools. 4. Conceptual Framework: Knowledge building 4.1 Definitions of knowledge building What is knowledge building? Bereiter and Scardamalia created this phrase, referring it as a process of creating new cognitive artifacts resulting from collective discussion and synthesis of ideas. Such process of collective inquiry can lead to a “deeper understanding” through interactive questioning, dialogue and continuous improvement of ideas (Scardamalia, 2002). It is worth making the remark that knowledge building emphasizes a sense of “we” superceding the sense of “I.” In other words, participants should feel that the group is operating collectively and not just as an assemblage of individuals. Throughout the learning process of knowledge building, both understanding and collaboration are emphasized (Bereiter, Scardamalia, Cassells & Hewitt 1997; Scardamalia & Bereiter, 2003). 4.2 Twelve Principles of Knowledge Building One of the foci of concept of knowledge building is its 12 principles (Scardamalia, 2002). These 12 interrelated but distinguishable principles are briefly explained in Table 1. In this study, the relevant principles were adopted to evaluate the effectiveness of knowledge

W.C.A. Tse et al. / The Effectiveness of Knowledge Building

211

building through web-based CSCL. These principles have been adopted in various researches to evaluate different CSCL projects. They justified the workability and appropriateness of the principles to evaluate the effectiveness of knowledge building (Law & Wong, 2003; Scardamalia, 2002). No.

Principles

1

Real authentic problems

2

Improvable ideas

Students can continuously contribute to improve the collective ideas by criticizing the contributions of others.

3

Idea diversity

Students can contribute to a wide variety of ideas and provide extra information relevant to the problem.

4

Rise above

Students can summarize the ideas of others and work out directions for further study.

5

Epistemic agency

6

Community knowledge, collective responsibility

Students can compare and contrast other’s ideas. High level of this principle: students can deal with problems of goals, motivation, evaluation, and long-range planning. Students can share the responsibility for knowledge advancement by regularly making input responding to the collective goal.

7

Democratizing knowledge,

Students can share their ideas and value the contributions of others without domination by a minority.

8

Symmetrical knowledge advancement

The number of messages within and between groups were counted. In this study, since none of the messages were transferred among the members of various groups, the principle was not discussed in this study.

9

Pervasive knowledge building

10

Constructive use of authoritative sources

Knowledge resources are maximized when students share ideas and learn from each other to create a collective knowledge building environment both within and between communities. Instead of time being set aside after fundamental work is done, knowledge building becomes an integral part of practice among students. Students can build up their ideas by critically referring to and comparing authoritative sources.

11

Knowledge building discourse

Students can share, improve and even transfer their ideas by referring to a good practice of referencing.

12

Embedded and transformative assessment

Students can reflectively conduct self-evaluation and internal assessment as well as review their learning progress.

The activities of continuous idea refinement had been covered by other principles. Therefore, the major concern here just stressed on whether the messages were embedded with a good practice of referencing. The number of these messages was counted and analyzed in this study. For instance, after a period of discussion, a student posted a message and commented that the schoolmates wasted many time in discussing irrelevant topics and urged them to stick to the right track. The number of these messages was counted and analyzed in this study.

ideas,

Explanation of the principles / Criteria for classifying online messages Students can convert their own ideas or authentic problems into questions for investigation.

Examples of message in this study / Method of evaluation

For instance, a student was aware of the low awareness of environmental protection among his fellow schoolmates. He suggested a proposal of survey on the platform and it was supported and commented by other group members. The survey was conducted eventually. The number of these messages was counted and analyzed in this study. For example, a student asked the meaning of “4R”. Another student quickly responded that it referred to “Reduce, Reuse, Recycle and Replace”. Then, two more students supplemented the details of “4R”. As well, the percentages of posted messages read by students were counted. The number of these messages was counted and analyzed in this study. In this study, the investigators counted the number of message representing new idea within a group. These messages should be relevant to the foci of discussion. Otherwise, they were classified as irrelevant. The number of these messages was counted and analyzed in this study. For instance, after a period of discussion, a student posted a message and commented that the group members had overemphasized on single aspect and she suggested discussing another relevant but ignored issue. The number of these messages was counted and analyzed in this study. For example, a student posted a message and criticized a group member who usually asked his group members to meet his request for providing him certain kind of information but seldom posting constructive message. The number of these messages was counted and analyzed in this study. Instead of counting message one by one, this study worked out a, Message Posting Interval Index to evaluate whether the students shared the responsibility for knowledge advancement by posting messages regularly. As well, other messages showing students who valued the participation of the others were also counted and analyzed. Instead of counting message one by one, this study tried to count the divergences among the number of messages that individual students posted on the platform within the groups. The percentages of posted messages read by students were also counted.

This principle emphasizes that knowledge building is not only confined to specific situations or subjects but happens in and out of school. Since this study highly focused on online platform observations, the scope of the principle was much larger than that of this study. As a result, it was skipped in this study and it was a recommended topic for further study. To investigate the issue in depth, in classifying the posted messages, 4 sub-categories were constructed for this principle of knowledge building: “Statement without any backup”, “Statement without authoritative backup”, “Statement with authoritative backup”, “Just copied and pasted.” The number of these messages was counted and analyzed in this study.

Table 1: 12 principles of knowledge building (Scardamalia, 2002; Law & Wong, 2003) 5. Overall picture Table 2 shows the statistics from the online discussion on the CSCL platform among the students of the two schools. Over the six months of online discussions in the CSCL platform, the average number of messages posted by students per group was 108. The average number of messages posted by students per month was 216 pieces. However, not all

212

W.C.A. Tse et al. / The Effectiveness of Knowledge Building

the groups were so active. For instance, though there were 152 posted messages in group 8, the posted messages of group 4 were just 66. Overall, the above figures might only reflect a superficial phenomenon. To evaluate the effectiveness of knowledge building through CSCL, the investigators of this study adopted Scardamalia’s principles and the findings are discussed below one by one. The criteria, based on Scardamalia’s principles listed in Table 1, were adopted for classifying online messages. To increase the reliability of study, after the first round of classification of massages, the classification results were carefully audited. The figures were adjusted afterward. Learning group/ Items

Numbers of students in group

Numbers of messages posted by students

Average numbers of messages posted by students

1 2 3 4 5 6 7 8 9 10 11 12 Highest Lowest Average per group Average per month Total

12 13 14 13 14 12 12 13 13 12 11 14 14 11

111 95 92 66 99 105 145 152 79 136 90 128 152 66

9 7 7 5 7 9 12 12 6 11 8 9 12.1 5.1

12.8

108.2

8.5

--------------

216.3

17.1

153

1298

--------------

Numbers of Average Average Standard Average messages read numbers of percentage of derivation of the Message by students messages posted numbers of Posting read by messages read posted message Interval Index students by students per group (MPII) per group 393 33 29.5% 7.1 2.6 228 18 18.5% 9.6 2.4 175 13 13.6% 8.1 2.3 126 10 14.7% 5.5 2.2 258 18 18.6% 8.0 2.2 286 24 22.7% 6.6 2.8 415 35 23.9% 6.1 3.2 326 25 16.5% 8.4 2.8 249 19 24.2% 4.3 2.1 354 30 21.7% 4.6 3.5 219 20 22.1% 6.5 2.1 325 23 18.1% 6.7 2.5 415 34.6 29.5% 9.6 3.5 126 9.7 13.6% 4.3 2.1 279.5

22.2

20.3%

6.8

2.6

559

44.4

--------------

--------------

--------------

3354

-----------

--------------

--------------

--------------

Table 2: Overall statistics from the online discussion in the CSCL platform in the two schools (MPII is discussed in section 6.4) 6. In-depth Analysis 6.1 Real ideas, authentic problems The principle of Real ideas, authentic problems emphasizes that, in a quality process of knowledge building, students can convert their own ideas or problems into researchable questions for investigation (Scardamalia, 2002). As Table 3 shows, there were 41 messages which fell into this category. The average number of messages per group was only 3.4. It was found that minority students were able to input authentic problems into online discussions. These problems included inappropriate disposal of rubbish near their schools and homes, air pollution from vehicles, overuse of using plastic bags, etc. One of the groups even posted 10 messages in this category. No. of Principles of knowledge building / Items principles 1 2 3

10

Other

Real ideas, authentic problems Improvable ideas Idea diversity Statement without any backup Constructive Statement without authoritative use of backup authoritative Statement without authoritative sources backup Just copy and paste Self-introduction

Total numbers of posted message on platform 41 84 204 132

Average numbers of posted message 3.4ʳ 7ʳ 17ʳ 11ʳ

Maximum Minimum numbers of numbers of posted message posted message per group per group 10 0 8 0 18 0 26 0

116

9.6ʳ

23

60

5ʳ

10

0

296 198

42.6ʳ 16.5ʳ

32 16

0 1

0

Table 3: Statistics from the messages classified by some principles of knowledge building Though some students were able to turn their ideas found in their daily lives into the subject of online discussions, only three cases were found in which the students could turn

W.C.A. Tse et al. / The Effectiveness of Knowledge Building

213

these ideas into a researchable question and conducted investigation systematically. Figure 1 shows one of these rare examples in which a student of mainland China successfully conducted a survey in her school about students’ awareness of environmental protection.

Figure 1: A case of turning students’ authentic ideas into a researchable question for investigation. (The name of the student was intentionally deleted to protect her privacy) 6.2 Improvable ideas Under the principle of Improvable ideas, a quality process of knowledge building should involve a situation in which students can continuously contribute to improve their ideas by criticizing the contributions of others (Scardamalia, 2002). To achieve this, students should regularly read the messages contributed by others and comment on them. However, this study found that most of the students had not read most of the messages posted by other students. As Table 2 shows, the average percentage of posted messages read by students was only 20.3% (ranging from 13.6% to 29.5%). In other words, most of the students had not read about 80% of posted messages. On average, the students only read 22.2 messages out of 108.2 messages in their group. It seriously undermined the possibilities and foundation of improving ideas through online discussion. In fact, there were only 84 messages fell into the category of improving ideas. In some cases, it was found that some smart students did post some good topics or questions which could be further developed. However, it was found that most of these messages were not responded to by the others. In many cases, even though these good messages got responses from the others, some of these comments were either illogical or unreasonable. It reflected the fact that some students might not be able to identify the main and peripheral ideas among the primitive and loose ideas, not to mention achieve synthesis, comparison or evaluation of the posted ideas required by this principle. Overall, the effective cases of improving ideas were limited. It was difficult to reach a quality process of knowledge building. 6.3 Idea diversity The principle of Idea diversity stresses that, in a quality process of knowledge building, students can contribute to a wide variety of ideas and provide extra information relevant to the problem (Scardamalia, 2002). As shown in Table 3, 204 messages representing new ideas were found. Its average number of messages per group was 17. It seemed that the figure demonstrated a wide variety of ideas. However, the principle, in fact, not only stresses the diversity of ideas over the discussion, but also requires the extension of the depth of discussion over the mentioned topics. In this regard, it seemed that the performance of the students could be improved in deepening the ideas that they discussed. Firstly, as mentioned, they had not read most of the posted messages. How could they deepen the ideas of others? Secondly, most of the messages with good ideas were not responded to appropriately. These problems inevitably undermined the foundation of deepening the ideas that the students raised, adversely affecting the quality of knowledge building.

214

W.C.A. Tse et al. / The Effectiveness of Knowledge Building

The fourth principles of knowledge building should be Rise above (0.8) but it will not be discussed in detail in this study. Similar to other principles like Knowledge building discourse (1.2), Epistemic agency (0.6), Embedded and transformative assessment (1), Symmetric knowledge advancement (0), the average number of posted messages in this category per group was around or less than 1. The numbers in brackets show the average numbers of posted messages. It reflected that the online discussions of these two schools were far from meeting the requirements of these principles. In other words, reservations should be made about the effectiveness of knowledge building in this case. 6.4 Community knowledge, collective responsibility Under the principle of Community knowledge and collective responsibility, to reach a quality process of knowledge building, students should be able to share the responsibility for knowledge advancement by regularly making input by responding to the collective goal (Scardamalia, 2002). To evaluate whether the students achieved this principle, as mentioned before, this study worked out an MPII. It refers to the Message Posting Interval Index. The below case illustrates the calculation method of the MPII. The information of a student’s distribution of the number of messages posted on the CSCL platform is presented in Table 4. November 1 message

December 0 message

January 0 message

February 1 message

March 3 messages

April 0 message

MPII 3 message

Calculation method: When a student posts at least one message on the CSCL platform, he or she can get 1 point under the MPII. The maximum point under the MPII for each month is only 1 no matter how many messages a student contributes. Conversely, if a student does not post any message in a single month, he or she gets zero points for that month. After the end of online discussions, the points are accumulated as MPII. This student only posted messages in November, February and March. Therefore, the MPII of the student was only 3. In this study, the MPII was worked out to evaluate whether the students shared the responsibility for knowledge advancement by posting messages regularly. MPII can be counted per day, week or month. In this study, it was counted per month to project an overall picture.

Table 4: A student’s distribution of the number of messages posted on the CSCL platform For this case, the maximum MPII was 6 because it involved 6 months. Table 2 shows that the average MPII of 12 groups was only 2.6 (ranging from 2.1 to 3.5). It reflected that the “active period” of the students to post messages on the CSCL platform was only about two and a half months over the total learning process of 6 months. In other words, students did not regularly input messages on the platform and the performance of the students can be improved by more actively taking up their responsibility for knowledge advancement so as to improve the quality of knowledge building. Numbers of posted messages November

December

January

February

March

362

259

89

74

329

185

Average numbers of message per month per group

30.2

21.6

7.4

6.2

27.4

15.4

Average numbers of message per month per student

2.4ʳ

1.7ʳ

0.6ʳ

0.5ʳ

2.2ʳ

1.2ʳ

Maximum numbers of message per month per student

16

11

11

9

23

10

Minimum numbers of message per month per student

0

0

0

0

0

0

Items Total numbers of message per month

April

Table 5: Distribution of the number of posted messages on the CSCL platform over six months Though there were more messages posted on the platform in November and March, it was necessary to evaluate whether those regular inputs corresponded to the collective goal of the whole group as is requested by the principle of Community knowledge and collective responsibility. Otherwise, a quality process of knowledge building could not be guaranteed. Though the number of posted messages in November was high, as shown in Tables 3 and 5, 55% of messages (198 out of 362) were, in fact, self-introductions by the students. For instance, though a student in Group 7 had posted 10 messages in November, all of them, in fact, were self-introductions. Though such messages of self-introduction might help to build

W.C.A. Tse et al. / The Effectiveness of Knowledge Building

215

up a favourable atmosphere for online discussions, it was necessary to strike a balance between building up an appropriate discussion environment and conducting real discussions (Oshima & Oshima, 2002; Tse et al., 2006a). Otherwise, the interactions on the platform were only a chat room among the students. In fact, it was found that some students even made use of the platform to share their online games. Teachers should supervise such activities because it might adversely affect the atmosphere of online discussions. How about the situation of March? It was found that about 20% of messages posted in March could, in fact, be counted as irrelevant to the topic of discussion. Many students, in fact, mixed up the issues of environmental protection with public hygiene. The numerous messages about Avian Flu were typical examples. In some cases, even though some students could post messages relating to environmental protection, many messages might not be sharp enough to respond to the topic of their group. For instance, the pollution crisis of Harbin in the winter of 2005 became a hot topic among the messages posted by the students. It definitely was an issue of environmental protection, but it was irrelevant for some groups with topics about the habits of environmental protection of their daily lives. Overall, these problems made collective responsibility of knowledge advancement difficult to be reached effectively. The quality of knowledge building was negatively affected. 6.5 Democratizing knowledge The principle of Democratizing knowledge stresses that, to reach quality knowledge building, students should share their ideas and value the contributions of others without domination by a minority (Scardamalia, 2002). However, this study found that there were wide divergences among the number of messages that individual students posted on the platform within the groups. As Table 2 shows, the average standard derivation of the number of posted messages was as high as 6.8 (ranging from 4.3 to 9.6). In other words, the messages posted within the groups, in many cases, were dominated by a minority of group members, especially the students of mainland China. In fact, throughout the online discussion of 6 months, 82 students out of 153 students (54%) had posted less than 10 messages. There were 10 students who even sent none of the messages. It reflected that some students were not eager to share their ideas with others. The contributions of messages among the members were not even. On the other hand, did the students value the contributions of the others? As mentioned before, the low percentage of posted messages read by students reflected that many students generally did not really value each message posted on the platform. These findings demonstrated an unsatisfactory quality of knowledge building in this study. 6.6 Constructive use of authoritative sources Under the principle of Constructive use of authoritative sources, in the circumstances of quality knowledge building, students can build up their ideas by critically referring and comparing authoritative sources (Scardamalia, 2002). To investigate the issue in depth, in classifying the posted messages, 4 sub-categories were constructed for this single principle of knowledge building as shown in Table 3. They were namely, “Statement without any backup”, “Statement without authoritative backup”, “Statement with authoritative backup”, “Just copied and pasted.” This study found that, as shown in Table 3, 132 messages were statements without any backup. For instance, some students just commented “I agree” when they responded to the posted messages without specific reasons. 116 messages were backed up by certain reasons, but authoritative sources were not adopted. Among these 4 sub-categories, the highest number of posted message fell into the category of “Just copied and pasted”. There were 296 messages of these. In other words, though many students could adopt authoritative sources, they could not establish their ideas by critically referring and comparing these sources. It seemed that the students had little intention to examine these authoritative sources. Only 60 messages could achieve this target and most of them were posted by the students of mainland China. The students tried to add value, further interpreting or keep a critical stance towards the sources, but on average, there were only 5 messages per group for this category. Teachers of Hong Kong explained it might be due to the reason that the students of mainland China were generally more proficient in inputting Chinese character through keyboard. As a result, a large number of information in the

216

W.C.A. Tse et al. / The Effectiveness of Knowledge Building

platform was created by copying and pasting web resources that the students found. Most of the students did not digest the information well before posting it on the platform. During the interviews, the students and teachers of 2 sides also confessed this problem. Such an inappropriate style of discussion among the students, in fact, converted the discussion in the platform to just an integration of undigested information instead of the opportunities of knowledge building. It seemed that effort needed to be made to build up the analytical and critical thinking abilities of the students. 7. Discussion and Conclusion This study revealed the unsatisfactory quality of knowledge building through CSCL among the elementary students. At least 5 out of 12 principles of knowledge building could not be reached because the average number of posted messages of these principles was less than or round 1 per group. For those who argue that knowledge building is possible through CSCL among elementary students, they should be made aware of the problems that students encountered in this study (Law & Wong, 2003). Though CSCL is a popular approach, it can simply be a mechanical way of learning. How to make use of it well to facilitate quality learning for students is a challenge for teachers, researchers and educators. This study did uncover some problems of the elementary students. Though the underlying factors behind the problems were not deeply investigated in this study, the poor participation of the teachers should be one of the major factors. This study found that the teachers only posted 55 messages throughout the six months and some of these messages, in fact, were repeated among various groups. The roles of teachers are very important for effective implementation of CSCL. It had been discussed in the first and second phase of this research project (Tse et al., 2006a; Tse et al., 2006b). This study found that students did have potential for knowledge building in certain dimensions. Therefore, this study recommends that teachers can provide better support for their students to facilitate them to turn their own ideas into researchable questions for investigation, improve their ideas by criticizing the contributions of others, contribute to a wide variety of ideas and build up their ideas by critically referring and comparing authoritative sources. In this way, more effective and appropriate scaffolding can be provided to nurture competence of tomorrow’s citizens. References [1] Law, N. & Wong, E. (2003) Developmental trajectory in knowledge building. In B. Wasson, S. Ludvigsen & U. Hoppe (Eds.). Designing for Change in Networked Learning Environments: Proceedings of the International Conference on Computer Support for Collaborative Learning 2003 (p.57-66). Kluwer Academic Publishers, Dordrecht. [2] MacKinnon, G. R. (2004). Computer-mediated communication and science teacher training: Two constructivist examples. Journal of Technology and Teacher, 12 (1), 101-114. [3] Chai, C. S., Tan, S. C. & Hung, D. (2003). Fostering knowledge and building communities through Computer-supported collaborative learning. Paper presented at Herdsa Conference 2003. Christchurch, July 6-9, 2003. [4] Hew, K.F. & Cheung, W.S. (2003). Models to evaluate online learning communities of asynchronous discussion forums. Australian Journal of Educational Technology, 19(2), 241-259. [5] Tse, W.C., Lee, F. L. & Ou, Y. (2006a). Model of Evaluating Educational Multimedia Application for Teaching and Learning. Paper presented at Hong Kong International Information Technology in Education Conference 2006, Hong Kong, 7 February, 2006. [6] Tse, W.C., Lee, F. L. & Ou, Y. (2006b). The Ten Neglects of Teachers in Implementing Computer-supported Collaborative Learning. Paper presented at The 10th Global Chinese Conference on Computers in Education, Tsinghua University, Beijing, 2-5 June, 2006. [7] Scardamalia, M. (2002). Collective cognitive responsibility for the advancement of knowledge. In B. Smith (Ed.), Liberal education in a knowledge society (p. 67-98). Open Court, Chicago, MA. [8] Bereiter, C., Scardamalia, M., Cassells, C., & Hewitt, J. (1997). Postmodernism, knowledge building, and elementary science. Elementary School Journal, (97) 4, 329-340. [9] Scardamalia, M., & Bereiter, C. (2003). Knowledge Building. In J. W. Guthrie (Ed.), Encyclopedia of Education, Second Edition (p. 1370-1373). Macmillan Reference, New York. [10] Oshima, J. & Oshima, R. (2002). Coordination of asynchronous and synchronous communication: Differences in qualities of knowledge advancement discourse between experts and novices. In T. Koschmann (Ed.). CSCL 2: Carrying forward the conversation (p.55-84). L. Erlbaum Associates, Mahwah, NJ.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

217

Computer-Supported Content Analysis for Collaborative Knowledge Building in CSCL Jian Liao, Yanyan Li, Ying Zhou, Ronghuai Huang, Jingjing Wang Knowledge Science & Engineering Institute, Beijing Normal University, 100875, Beijing, China [email protected] Abstract ˖ Interaction analysis plays an important role in computer-supported

collaborative learning. This paper proposes an interaction analysis model CSCAC composed of three dimensions: group building level, member contribution, member support, which lays the foundation for interaction content analysis. Based on the

model and domain ontology, this paper explores the text mining and semantic analysis technologies to automatically analyze the interaction content for

collaborative knowledge building. An experiment is conducted to compare the auto-analysis result with that of manual analysis, which shows the the proposed approach is feasible and effective mostly to automatically analyze the interaction content in CSCL.

Keywords: CSCL, Collaborative Knowledge Building, Content analysis, Domain ontology, Concept Map, Keyword extraction , Hownet

Introduction With the development of e-learning, Computer-Supported Collaborative Learning˄CSCL˅ has drawn more and more attention of researchers. In the process of collaborative learning, the validity of evaluation to group collaboration is an important but difficult task. Henri˄1991˅is the first person who put forward an analysis model of understanding interaction contents.[1] Newman(1995) et al. design a content analysis model based on Henri, including the following ten dimensions: Relevance, Importance, NoveltyǃBringing outside knowledge/experience to bear on problemǃSticking to prejudice or assumptions, Ambiguities: clarified or confused, Linking ideas: Interpretation, Justification, Critical assessment, Practical utility (grounding), Width of understanding... And they use the X X

+

− X + X

−

to calculate the degree of cognitive process by each group member or formula the whole group. [2] Besides, Liu (2005), Zhu(1996), Pena-Shaff(2001)ˈN.M.Avouris and other scholars analyze and research on the interaction contents in collaborative learning from different dimensions. +

−

218

J. Liao et al. / Computer-Supported Content Analysis for Collaborative Knowledge Building

1 The Model of Computer-Supported Content Analysis for Collaborative Knowledge Building˄CSCAC˅ On the topic of how CSCL analyzes contents, the important representative scholar in CSCL Koschmann˄1996˅considers research on the meanings and the building processes of meanings in common activities as one of the main research contents of CSCL. Many researchers use “Collaborative Knowledge Building” to explain the building processes of meanings in common activities.[6] Therefore, the analysis of collaborative knowledge building in the collaborative groups is the important stand for setting up the whole content analysis model. The first problem for judging collaborative knowledge building is the knowledge representation. Many researchers regard Concept Map as an effective means of representing and measuring students’ knowledge structures. [6][8] So we can set up the domain ontology of collaborative learning contents by drawing concept maps related to the topic of group discussion. Keywords in interactive texts can be used to represent the individual knowledge of the group members. The relatively important keywords, which are regarded as the representation of group member’s individual knowledge, can be extracted by searching the interactive speeches of members. The group knowledge contain in each group members’, but not the simple sum of all individual knowledge. Some knowledge may be implied by all members, and some may only be in the possession of very few members. Therefore, the knowledge of collaborative groups needs to be analyzed in the holistic perspective. When comparing member or group knowledge with domain ontology, as domain ontology is described by concept and member or group knowledge is described by keyword and the relations between words and concepts are many to many relations. Therefore, we need to carry out semantic analysis of the keywords in order to obtain correct mapping between concepts and words. Member

Contribution Semantic mapping

Member

Semantic

Speech

Member Support

Domain

Ontology

Knowledge Comparison

Group

Knowledge

mapping

Figure 1. CSCAC Model.

Group Building Level

Based on the above knowledge representation and directed by the activity theory, we set up the following model of analyzing collaborative knowledge building, as shown in figure 1.In figure 1, the three circles around the small triangle in the middle represent the three core components of activity theory respectively, which are subject, object and group knowledge. The three sides of the small triangle represent the inter comparison methods

J. Liao et al. / Computer-Supported Content Analysis for Collaborative Knowledge Building

219

between different sorts of knowledge. The comparison between member knowledge and group knowledge or between member knowledge and other group members’ knowledge is carried out by analyzing the contents of their speeches, such as keyword matching method, etc. While the comparison between member knowledge and domain ontology or group knowledge and domain ontology is carried out by semantic mapping method. The three rectangles contained in the large triangle represent the three perspectives for analyzing collaborative knowledge building. The analysis of collaborative knowledge building is stated from three perspectives as follow. 2 Multi-dimensional Analysis 2.1 Level of group collaborative knowledge building (GBL) At present, many researches analyze the level of collaborative knowledge building from the perspectives of “contention degree” and “building degree”. Fisher(1993), Mercer(1995), Coelho˄1994˅and so on consider effective interaction as “exploratory talk”, whose characteristics is that the participants communicate in a critical but constructive way. [6] Therefore, we need to examine both the critical and constructive characteristics of the speech contents. We assume that the more decentralized the speech contents of the group members are, the more critical it will be; and also the closer the speech contents of all group members is compared to the domain ontology constructed by teachers and experts, the more constructive it will be. Definition 1˖Level of Group Collaborative Knowledge Building

GBL = R × U

Where R represents the Relevance of group speeches to the topic. U represents the inconsistence of the speeches among group members. M Definition 2˖Relevance R= L Relevance indicates the relevance of group speeches to the discussion topic, where M represents the number of concepts in the speeches of the whole group which cover the domain ontology. L represents the number of all concepts in the domain ontology. Definition 3˖Inconsistence

U = µ ∑ χ i2 = µ ∑∑ S

i =1

S

N

i =1 j =1

( Fij − F j' ) 2 F j'

Inconsistence means the difference of the speeches among group members, where N represents the sum of keywords in the whole discussion speeches. S represents the number

1 2 µ of the group members. represents an adjustment factor, whose value is S × N . χ i

represents the decentralization degree of the ith member’s speeches compared with the speeches of the whole group. The more the value is, the more inconsistent it will be. Fij represents the frequency of the jth keyword in the ith member’s speeches.

F j'

represents

220

J. Liao et al. / Computer-Supported Content Analysis for Collaborative Knowledge Building

the average number of the jth keyword mentioned by each member in the group speeches. 2.2 Member contributions to collaborative knowledge building We can estimate the contribution of group members to the task by comparing the knowledge of group members and the domain ontology structure presumed by teachers and experts and analyzing the similarity of the two sorts of knowledge. Furthermore, Newman(1995) and Liu(2005) both consider the novelty of members’ speeches and the extension of the topic to be discussed as the important criteria of evaluating the quality of interactive texts. MCi = αRi + βVi + γEi

Definition 4˖Member’s Contribution

Where Ri represents the Relevance of group member I’s speeches to the topic, whose

algorithm is shown as Definition 2. Ni represents the novelty of group member I’s speeches. Ei represents the extension of group member I’s discussion contents. α , β , γ

are adjustment factors, which represent the impact degrees of Relevance, novelty and extension respectively on member’s contribution. And their values are usually as α =0.5ˈ

β =0.3ˈ γ =0.2 respectively.

Definition 5˖Novelty

Vi =

Pi N

This indicates the novelty of member’s speeches, where

Fi

represents the number of

keywords mentioned for the first time by group member i. N represents the total number of keywords. For example, in a discussion, the total number of keywords is 370, among which member A first mentions 118, so we reach the novelty of member A’s speeches as 118/370=0.32. Definition 6˖Extension

Ei =

Ni N

This indicates the extension of member’s speeches, where Ni represents the number of keywords in each member’s speeches. N represents the total number of keywords in the whole discussion speeches. For instance, the number of keywords in member A’s speeches is 207 and the total number of keywords mentioned by all group members is 370, so the extension value is the ratio of 0.56. It can be proved that the values of

MC i , Ri , Vi , E i are all between [0ˈ1].

2.3 Group member’s support and assistance to the other group members The evaluation of the group member’s support to or the assistance from the other group members is done by the comparison of knowledge structure between individual

J. Liao et al. / Computer-Supported Content Analysis for Collaborative Knowledge Building

221

group members. We believe that the more similar of two members’ speeches, the mutual support will be greater. The algorithm of similarity is the same as the vector cosine method which is frequently used in calculating the document similarity in text classification and clustering. Definition 7˖Member’s Support

Sij =

Fi • F j

| Fi | × | F j |

∑F N

=

k =1

ik

× F jk

(∑ Fik2 )(∑ F jk2 ) N

N

k =1

k =1

Where Fi represents the vector of word frequency in user I’s speeches in the vector

space model (VSM). Fj represents the vector of word frequency of keywords in user j’s speeches. Fik represents the word frequency of the kth keyword in the speeches of the ith user. Fjk represents the word frequency of the kth keyword in the speeches of the jth user. N represents the total number of keywords. Therefore, we can obtain the matrix of the degree of member’s support S, which represents the degree of mutual support among group members. And it is also proved that the value of Sij is between [0,1]. 3 Crucial techniques in CSCAC Model 3.1 Building the domain ontology Domain ontology is to describe domain ontology, which is one of the bases for CSCAC model in analysis. It is realized by the concept software—EasyThinking Cognitive Assistant, which is developed by the Knowledge Science and Engineering Institute at Beijing Normal University. In this software, a concept is shown by the node in concept map and described in the word that can best represent the concept. The relation between concepts is shown by the arrow line connecting two nodes. The relations between concepts usually include hypernym, hyponym, attribute, part-of, agent, dative, time, site, etc. 3.2 Semantic mapping between keywords and concepts In order to compare the keywords in members’ speeches with the concepts in the concept map, we need to calculate the semantic similarity of words and those that represent the concepts to be compared with in the concept map. If the similarity is more than a threshold, which is usually taken as 0.8, we will consider the compared words are able to represent the corresponding concepts in the concept map. The semantic similarity can be calculated by How Net, which is knowledge database that mainly digs out the relations between concepts and the characteristics of each concept based on the sememes both in English and Chinese.[9] The semantic similarity between two words can be calculated by How Net and the algorithm can refer to article [10].

222

J. Liao et al. / Computer-Supported Content Analysis for Collaborative Knowledge Building

3.3 Keywords Extraction Keyword extraction is the most basic part of processing natural language.[7][11].As we need to use keywords to represent the knowledge of groups and members, we consider all nouns, verbs, adjectives except some stop words in the text as the keywords. By this definition, the precision of extraction can be considered the precision of the words splitting as. According to the report of National 973 Evaluative Team in machine translation project, the precision in Chinese character has been beyond 97% now, so the extraction is valid. The main factors that affect the experimental results are the correct percentage of word categorization and the degree of word base perfection. So as the experimental shows, we can believe that the keywords extracted by the keyword extraction process are able to represent the knowledge of groups and members. With this algorithm, we develop the module of keyword extraction. 4 Experimental Study In order to prove the validity of CSCAC model in collaborative learning, we design the following experiment. First of all, we develop the tool with C# to support the computation about the Collaborative Knowledge Building. Then we designs a collaborative task on the topic of “Collaborative Composition”, whose content is that each group discuss sufficiently in the chat room on the topic of “Who is the excellent teacher mostly needed in China” according to the assigned materials and compose a short essay of 300 words collaboratively within 1 hour. After this, the teacher designs the domain ontology of the topic of this task, as shown in the concept map of figure 2.

Figure 2. Domain Ontology of Collaborative Task.

Then the teacher logs in the platform to distribute the task to 5 groups. Each group have 4 students, which are selected randomly from a class of 75 students, and organizes the groups to carry out collaborative learning on the task. After the task accomplish, we have a survey in each team about the member contribution. In this survey, everyone will assess the rank of his(her) team member’s contribution according to their impact in this task. We take the means of the result of all members in a team as the member contribution value ranked by manual .In addition, we evaluate manually the interactive text of whole group according to their relevance and inconsistence .After this, we run our tool to statistic all value. The detail is shown in Table1. Thus ,we can compare the rank of Group Building Level and Member Contribution between the manual and auto evaluation result in SPSS. We take Kendall’s W Test to

223

J. Liao et al. / Computer-Supported Content Analysis for Collaborative Knowledge Building

compute the consistence of result between manual and auto evaluation to confirm the reliability of our model. The consistence in evaluation of Group is Kendall's W(a)=0.950 Asymp. Sig.=0.107,And the consistence in evaluation of Member˖Kendall's W(a)=0.840 Asymp. Sig.=0.032. The result proves that the evaluation result in Group Building Level and Member Contribution by CSCAC model is related to the manual evaluation highly. Team

Manua

ID

l Rank

1

5

CSCAC Rank

R

(GBL)

U

Member

Manual Rank

A

5(0.60)

0.76

2

B

0.80

3

C

1

2(0.74)

0.79

0.94

2

1(0.79)

0.78

3

3(0.73)

0.77

4

4(0.65)

0.79

0.28

0.36

3(0.47)

0.69 0.72

0.28 0.20

0.36 0.27

0.31

B

2

2(0.52)

0.77

0.23

0.32

1

1(0.57)

3

3(0.50)

0.77

0.71

0.36

0.24

0.42

0.34

D

4

4(0.47)

0.73

0.17

0.27

B

3

3(0.50)

0.75

0.22

0.29

1

1(0.61)

4

4(0.30)

0.72

0.53

0.45

0.05

0.56

0.08

D

2

2(0.53)

0.76

0.28

0.35

B

2

3(0.50)

0.74

0.22

0.30

3

C

0.83

0.70

0.24

2(0.51)

4

4(0.44)

0.74

0.71

0.25

0.14

0.32

0.20

D

1

1(0.59)

0.75

0.40

0.50

B

2

1(0.61)

0.76

0.42

0.52

A

5

1(0.50)

0.67

C

0.95

E

4(0.47)

A

4

V

2(0.50)

4

C

1.01

R

1

A

3

Rank

(MC)

D

A

2

CSCAC

4

C

4(0.42)

1

D

2(0.52)

3

3(0.47)

0.68

0.77

0.70

0.14

0.24

0.20

0.21

0.33

0.32

Table 1. Experimental Result.

7HDP 7HDP 7HDP 7HDP 7HDP

0HPEHU$ 0HPEHU% 0HPEHU& 0HPEHU'

*%/

5

8

(a) Group Building Level

0&

5

1

:

(b)Member Contribution(Team3)

(c)Member Support

Figure 3 .Visualization of Result in CSCAC

At last, we visualize the result above in bar chart and visualize the Member Support by network graph. These chart can be presented to the members timely. They are shown in figure3. In the figure 3(C), we choose a team’s result randomly as a sample. As the figure illustrates, the edge describes the support among the members. The higher the member

224

J. Liao et al. / Computer-Supported Content Analysis for Collaborative Knowledge Building

support mutually, the wider the edge will be. According to this graph, we can observe the support among the members clearly. 5 Conclusion This paper analyzes interactive text contents automatically by constructing an analysis framework of collaborative knowledge building. This method is able to give timely feedbacks on the collaborative situation of each group in collaborative learning to teachers and students, which will advance further collaborative learning effectively. And the experiment proves it to be practical. The research for the next step is to collect more data to prove effectiveness of each dimension, especially about member support, to promote the automatic analysis results from aspects of perfecting analysis model, elevating the precision of keyword extraction, taking more semantic method such as analyzing the relations between concepts in domain ontology and so on. References [1] Henri, P.(1991).Computer conferencing and content analysis. In Kaye,A.R(Ed.),Collaborative Learning through Computer Conferencing: The Najaden Papers.Berlin:Springer-Verlag,p117-136.

[2] Newman, D.,Webb,B.,& Cochrane,C.(1995).A content analysis method to measure critical thinking in

face-to-face and computer supported group learning. Interpersonal Computing and Technology: An Electronic Journal for the 21st Century,3(2),p56-77.

[3] Ronghuai Huang(2003).The Theories and Methods of Computer-Supported Cooperative Learning 2003

[4] Zhu,E.(1996).Meaning negotiation, Knowledge construction, and mentoring in a distance learning

course. Paper presented at the Proceedings of Selected Research and Development Presentations at the

1996 National Convention of the Association for Educational Communications and Technology, Indianapolis.

[5] Pena-Sha.,J.,Martin,W.,& Gay,G.(2001).An epistemological framework for analyzing student

interactions in computer-mediated communication environments. Journal of Interactive Learning Research,12,p41-68.6

[6] LIU Huang Lingzi; ZHU Lingli; CHEN Yiqin & HUANG Ronghuai,

π A Research about

Collaborative Knowledge Building through Interaction Analysisρ, Open Education Research Vol.11, No.2, P31-37, 2005.

[7] LI Su-Jian WANG Hou-Feng YU Shi-Wen XIN Cheng-Sheng

Research on Maximum Entropy

Model for Keyword Indexing Chinese Journal of Computers Vol. 27 No.9 Sept 2004

[8] Giouvanakis Thanasis; Fragidis Garyfallos; Paschaloudis Dimitrios; Tarabanis Konstantinos; Guoqing

Zhao; Embedding a Vocabulary-based Application for Concept Mapping into a Learning Management System Open Education Research Vol.11, No.5, P38-43, 2005.

[9] Dong, Zhendong (1988) Knowledge Description: What, How and Who? In: Proceedings of International Symposium on Electronic Dictionary, Tokyo, Japan.

[10] QunLiu SujianLi Word Similarity Computing Based on How-net Computational Linguistics and Chinese Language Processing

”( TaiBei, 2002 )”

[11] FENG Jin; LI Chunping Topic detection technology for Chinese text based on statistics and semantic information Journal of Tsinghua University(Science and Technology 2005, Vol. 45, No51 ,P1791

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

225

A Novel Web-based Collaborative Learning Supporting System with Navigation Function Chenyuan Tiana, Zuoliang Chenb, Shigayeshi Watanabec Faculty of Electro-Communications, University of Electro-Communications Chofu-shi, 182, Tokyo, Japan b [email protected] [email protected] Abstract: This paper describes a novel collaborative learning supporting system operating on the web and its experiment data analysis. We insist that in the Web-based Collaborative Learning Supporting (WCLS) system, the individual-paced learning process rather than the group-paced learning process should play the main role. The relationship of individual-paced learning and group-paced learning in WCLS system should be that the group-paced serves the individual-paced. To verify this idea we designed a WCLS, which has two navigation windows; one is Page Navigation (PN), and the other is Discussion Navigation (DN). Finally, we carried out a contrast experiment; two groups of students learned the same web-based textbook and took the same test, the result shows that the group using the navigation function outperforms the group who didn’t use the navigation function by about 6 percentage points. Keywords: Web-based learning, collaborative, individual-paced learning,group-paced learning, supporting system, navigation function

Introduction Computer and network technology now are playing a more and more important role in education. We have witnessed great successes as computer brings great convenience, high efficiency to the human learning activity. In the conventional WCLS research, the effort involved in the development of useful and powerful WCLS applications is only justified by whether the communication and collaboration or communication is well supported or not, whether they can be applied to a large number of learning situations or not and whether they can survive the evolution of functional requirements and technological changes or not. The importance of collaboration tends to be over-emphasized in most WCLS systems (such as Warendorf & Tan, 1997; Okazaki et al., 1996; Brusilovsky et al., 1996; Nakabayashi et al., 1997). We think that individual-paced learning process should be the center activity even in WCLS system, collaboration supporting should be no more than a complement to individual-paced learning, because we think that knowledge can only be acquired by the learner’s own mental efforts in individual learning process, any help from peers in collaboration process plays a less important role than the effort of oneself. Group-paced learning works efficiently only when it is integrated into individual-paced learning; group-paced learning alone can not support learning ideally well. In this paper, we refer web-based collaborative learning (WCL) as studying by discussion/communication over learners via text-based chat channel; we refer web-based individual learning (WIL) as studying the web-based textbook by self-conduct, without any interference/communication with peers. WCL and WIL are two different learning states in WCLS system, how to balance and integrate them in a WCLS system should be a big research problem. Based on the above opinions, in our system there are: 1. WIL process is the center process, the learner spends most of the learning time on this process; 2. WCL is only needed when one’s learning run into impasse in the WIL process; 3. necessary navigation

226

T. Chenyuan et al. / A Novel Web-Based Collaborative Learning Supporting System

information should be provided to learner so that one can switch freely between WIL process and WCL process. 1. System Overview Fig. 1 shows us the flow of learning process. The learner starts self-conduct learningfrom the first web page; after mastering the knowledge of that page, he/she will move on to the next page, repeats this action until the goalis reached. In the course of this individual learning, impasse always happens. Impasse here means the cases such as learner feels some content is too difficult to understand, or the learner is not very confident about what he/she has understood. Once the learner meets some impasse, it is the time for one to switch to WCL for help. Since all the learners use the same system to learn the same pages, it is possible that some other learners share the same impasse, some other learners have understood the content well; discussion witheach other should be an efficient way to look for answer. The learner can switch back to WIL from WCL at any time he/she wants.

The interface of the system is like the picture shown in Fig. 2. (a) is the textbook area; (b) is the navigation area, there are two navigation windows, one is Page Navigation, and the other is Discussion Navigation; (c) is the knowledge point list of the textbook; one page corresponds to one item in the knowledge point list; (d) is the text-based chat window; (e) is message dialogue. 2. Navigation Function Just web page textbook and communication channel obviously are not enough to serve our purpose. To make the WIL be the main learning process, three kinds of information should be presented to each learner: 1. the page-reading progress of every learner, which means, at the time being, to which page learners have read. 2. The discussion group should be categorized according to the topic of the group, which means discussion group is categorized corresponding to web pages. 3. The information of the current discussion groups, that means, at the time being, the information, such as who and who are discussing what. Therefore, we designed the navigation function: a. Page Navigation The page navigation is designed as shown in the left of Fig. 3. It is a graph window visualizing at some time point all the online learners’ reading progress. The vertical line represents the ID of learners, and the horizontal line represents the ID of textbook page. For example, in the left of Fig. 3, the learner 1_01 has finished reading page “߳”. b. Discussion Navigation By referring page navigation, learner can choose a person who is thought to be suitable to form a group. But if the learner chosen is in some other discussion at the time being, the quality of discussing with him/her will decrease in some sense. So, just knowing the page navigation is not enough, it is necessary to know all the ongoing states and actions of all the learners. As a measure, we proposed the discussion navigation, which is a graph window

T. Chenyuan et al. / A Novel Web-Based Collaborative Learning Supporting System

227

visualizing the information that who is in which group at the time being. The outlook is as shown in right of Fig. 3. The vertical line represents the ID of learners, and the horizontal line represents the ID of web page, which is the topic of the discussion group. As we can see from the right of Fig. 3, now learner [l_01] and learner [l_04] is in the discussion group of “ߦ”.

Fig. 3 page navigation and discussion navigation The two navigation windows have brought more advantage than we have expected. For example, it also makes the learner become aware of the existence of other learners’ progress to adjust the learning speed. It also contributes to the enthusiasm of participating in discussion. Moreover, by teaching each other, the learners are expected to be aware of various problems in their knowledge comprehension, it makes the learner learn the web pages more positively and actively. 3. Experiments and Evaluation A Japanese language web-based textbook about particle was used in this experiment. The web-based textbook has about 20 pages. The participators of this experiment are 10 short-term students, who were randomly chosen. They are the beginners of Japanese language; although the content of the web-based textbook is new to them, it is exactly what they were going to learn. We randomly divided the students into two groups, each has 5 students: x A group: learn the web-based textbook in 90 minutes, using the system as illustrated in above sections. x B group: learn the web-based textbook in 90 minutes, using the system, communication over learners is allowed but the navigation functions are also removed from the system. After the studying finished, two groups were asked to take a same test, there are two questions in the test, the first one is filling 37 blanks in the sentences using particles; the second one is writing a composition according to a set of pictures. Table1 and table2 show us the test result of question1 and question2 respectively: the number of correct answers Individual percentage= × 100% the total number of questions(37) ¦ Individual percentage × 100% Group percentage= the number of participaters Group Leaner ID

A

B

1_01 1_02 1_03 1_04 1_05 1_06 1_07 1_08 1_09 1_10

Number of Correct Answer 33 20 30 24 27 27 28 26 14 28

Individual Percentage

Group Percentage

89.19 54.05 81.08 64.86 72.79 72.79 75.68 70.27 37.84 75.68

Table 1. The test result of question 1

72.39

66.45

228

T. Chenyuan et al. / A Novel Web-Based Collaborative Learning Supporting System

As to the second question, the total number of incorrect answer means how many mistakes in each learner’s article, including any mistake; the number of incorrect answer in particle represents how many mistakes in particle usage. The Group percentage is calculated using the following formulation: ¦ the number of incorrect particale × 100% Group percentage= ¦ the total number of incorrect answer Group Leaner ID

A

B

1_01 1_02 1_03 1_04 1_05 1_06 1_07 1_08 1_09 1_10

Total Number of Incorrect Answer 3 5 3 8 1 6 3 11 8 7

Number of Incorrect Answer in Particle 2 1 1 3 1 1 1 4 2 4

Group Percentage

40.00

34.28

Table 2. The test result of question 2 Statistical analysis of the result of the first questionˈshowed us that the group-A outperformed the group-B at Group Percentage by about 6 percentage points. That was the result which we hoped for. We want to say that the proposed navigation functions made this difference. The result gives us much confidence and encouragement. The result of the second question makes us depressed a little, because Group Percentage of group-A is greater than that of group-B by about 5 percentage points. As a possible explanation, we think that when one write a composition in a test, one always use the phrases about which one is most confident. So, maybe although group-B has less knowledge of particles than group-A(this is reflected in the result of question 1), if they only used the particles which they are very confident, it will not be a surprise to see that the Group Percentage of group-A is greater than that of group-B.If this explanation is right, the synthetic result of question 1 and question 2 proved that our proposed system is valid. 4. Conclusions

We think it is necessary to change the traditional tendency that in WCLS systems the individual-paced learning was not attached greater importance than group-paced learning. In our system, individual-paced learning becomes the center process, and collaborative learning environment becomes the slave process. A navigation function is introduced to make our idea become reality. It is composed of two parts, one is Page Navigation, and the other is Discussion Navigation. Finally, we did an experiment to evaluate the system; the experiment results give us much confidence and encouragementabout the validity of our proposition. We hope our proposed system and experiment data analysis will contribute to the future development of WCLS, and we are confident about that. References [1] Elliot, C. (1997). Implementing Web-based intelligent tutors. Paper presented at workshop “Adaptive Systems and User Modelling on the Web Wide Web”, 6th International Conference on User Modelling, 2-5 June 1997, Chia Laguna, Sardinia. [2] Calvert, H. M. (2000). Document delivery options for distance education students and electronic reserve service at Ball State University Libraries. In P. S. Thomas (Ed.), The Ninth Off-Campus Library Services Conference Proceedings (pp. 73-82). Mount Pleasant, MI: Central Michigan University. [3] Zuoliang, C., Shigeyoshi, W. (2005). An Instance Structure Design of Virtual E-Learning Community. Proceedings of International Conference Cognition and Exploratory Learning in Digital Age (pp. 387-390), December 2005.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

229

Towards Auto-Coding of Collaborative Interaction Texts Based on Maximum Entropy Approach Jian Liao, Ronghuai Huang, Yanyan Li, Jingjing Wang, Jing Leng Knowledge Science & Engineering Institute, Beijing Normal University, 100875, Beijing, China [email protected] Abstract˖Content analysis is an important method in the research into collaborative learning. The usual method for deeper content analysis is to first categorize and code interactive texts and then analyze them. However, the content analysis method with categorizing and coding is time consuming in the first period of research for coding content analysis, which restricts greatly the amount and scale of the analyzed contents. Therefore, this paper proposes a method to automatically code the collaborative interaction text based on the maximum entropy approach. Taken a great number of interactive notes as input corpus, an experiment is conducted to test the efficacy of the proposed approach. The experiment results show that the coding effect of the proposed auto-coding approach is FORVH to that of the sentence-opener approach in which the coding category is manually chosen by students. Additionally, as corpus increase and features are improved, the coding effects will be further strengthened. Keywords˖Content analysis; maximum entropy; auto-coding; CSCL

1 Introduction In CSCL domainˈContent analysis based on coding is applied extensively. Henri ˄1991˅is the first person who puts forward an analysis model of understanding interactive contents.. Besides,Zhu(1996), Pena-Shaff(2001), N.M.Avouris and other scholars analyze and research on the interaction contents in collaborative learning from different dimensions. Content analysis becomes a hot topic and an emphasis in collaborative learning research. However, the content analysis method with categorizing and coding is manpower-and-time consuming in the first period of research for coding content analysis, which restricts greatly the amount and scale of the analyzed contents. Therefore, how to use various computer technologies to auto-coding the analyzed content is a subject of great research value.

230

J. Liao et al. / Towards Auto-Coding of Collaborative Interaction Texts

2 Auto-Coding based on Maximum Entropy Approach 2.1 Introduction of maximum entropy approach The concept of maximum entropy have computers become powerful enough to permit the widescale application of this concept to real world problems in statistical estimation and pattern recognition. Since the 1990’s, it has been widely used in word tagging, word sense disambiguation, machine translation of natural language processing. Specific algorithm can be referred to Reference [6]. 2.2 Framework of ACME Model Auto-coding based on Maximum Entropy (ACME˅model flow is shown in Figure 1. First of all, some records that have been coded be coding specialists are taken as training corpus Then the system will evaluate the features extracted by these specialists. Those features that are kept by the specialist will be held in the feature base. Training Corpus

Feature

Feature

Feature

(Coded Text)

Pre Selection

Evaluation

Selection

Feature Base

Feature Extraction Append Corpus

Create

Feature

Model

Extraction

ME Model

Auto Coding

Coding Evaluation Test Corpus

Manual

Test Corpus

(Coded Text)

Correction

(Uncoded Text)

Figure1 ACME Model

Next, the system extracts all features in each record of training corpus according to the features in the feature base. Then the features extracted from each record and the manual coding of this record are taken as the training text of maximum entropy. The correlative argument of the maximum entropy approach is calculated and these arguments are saved in the maximum entropy approach base. The process of auto-coding begins with extracting the existing features in the feature base from each record of test corpus. After getting the features of each record, maximum entropy approach will figure out the probability of each record belonging to each sort of coding and the largest probability of coding will be taken as the coding result. Then, coding specialists can correct the results of auto-coding and the system will evaluate the results of auto-coding according to the correction. The results of judgment will help the specialists further optimize coding system. Finally, the test corpuses that have been coded by the specialists lately are added to training corpus in order to extend it. Therefore, a kind of cyclical machine learning process is formed. The reliability of system recognition will be promoted with the continuous extension of corpus and continuous perfection of features.

J. Liao et al. / Towards Auto-Coding of Collaborative Interaction Texts

231

2.3 Definition of Features The simplest definition of feature is to take keywords as the feature. Additionalˈwe can consider better feature definition, such as fuzzy match frequently used in inquiry in the database. That is we can add asterisk wildcard like *, ˛and so on, which will extent the scale of matching at feature matching. 2.4 Selection of Feature z

The coding specialists pre-select the features in each record according to their experience; then the machine carries out feature evaluation automatically. The criteria can be described as Credibility. The credibility is defined as follow. Credibility = Correct/All Where “Correct” represents the number of records of the coding represented by the feature as the coding result and also including the feature in corpus, “All” represents the number of all records containing the feature in corpus.

2.5 The Creation of ME Model and Auto-coding Maximum entropy approach gets a serial of coding arguments according to the number of features appearing in each coded record of training corpus and saves it in a maximum entropy approach model. Thus, we can use the argument in this base directly for auto-coding. 2.6 Evaluation of Coding Results The coding result is judged mainly by two criteria, which are precision and recall rate. Definition1.

Precision=

The number of correct auto coding The total number of auto coding

Definition2. Recall rate=

The number of correct auto coding The total number of manual coding

3 Experimental Study

Correlative experiment is carried out for verifying the model. .First of all, we collect 9240 interactive records in collaborative learning. Then we adopt a set of coding system especially designed to recognize students’ interactive intention in the interactive process to code these records. There are 28 codes in this coding system . Next, we take these 9240 interactive records as corpus and select the features that are able to code corresponding codes. The features can be described in features with asterisk wildcards like þ*ÿand þ˛ÿ, such as Ā*Bye-bye*āˈĀI think*?*āetc. Then we numerate the credibility of each feature, After that, we select new records of

232

J. Liao et al. / Towards Auto-Coding of Collaborative Interaction Texts

speeches in collaborative learning as test corpus for testing. As a comparison, we also use the opener-sentence system which adopts the same code system and corpus. The test results are shown as Table1.

7HVW &RUSXV

)HDWXUHV

&RUUHFWLQ

6XPRI

$XWRFRGLQJ

$XWRFRGLQJ

6XPRI 0DQXDO 3UHFLVLRQ 5HFDOO &RGLQJ

2SHQHUVHQWHQFH

([SHULPHQW

([SHULPHQW

([SHULPHQW

([SHULPHQW

([SHULPHQW

Table1 Coding Result

In Experiment 1, we define all the records without coding by machine as a special coding. In Experiment 5, if the coding done by specialists appears in the first three places of the coding probability ranking of each record done by machine, we consider it to be successful coded.

4 Conclusion According to the experimental results, we can initially draw the following conclusions. First, though the performance of ACME still lower than the sentence-opener approach, it’s not obvious. Second, if quality and quantity of the corpus and features are improved, the result of auto-coding will be better. More researches will be carried out such as Taking regular express to describe features ,optimizing coding , refining the features, using a larger scale of corpus and so on.

References [1] Henri, P.(1991).Computer conferencing and content analysis .In Kaye,A.R(Ed.),Collaborative Learning through Computer Conferencing: The Najaden Papers.Berlin:Springer-Verlag,p117-136. [2] LIU Huang Lingzi; ZHU Lingli; CHEN Yiqin & HUANG Ronghuai Δ π A Research about Collaborative Knowledge Building through Interaction AnalysisρΔOpen Education Research Vol.11, No.2, P31-37, 2005. [3] Software for Content Analysis – A Review Will Lowe (2002)

http://people.iq.harvard.edu/~wlowe/Publications/rev.pdf [4] Wilson, T. (2001). Review of Atlas-ti. Information Research, 6(3). [5] Adam L.Berger

Stephen A.Della Pietra Vincent J.Della Pietra A Maximum Entropy Approach to

Natural Language Processing 1996 Association for Computational Linguistics [6] Stephen Della Pietra, Vincent Della Pietra, John Lafferty Inducing Features of Random Fields IEEE transaction pattern analysis and machine intelligence vol.19,NO.4 April 1997 [7] LI Su-Jian WANG Hou-Feng YU Shi-Wen XIN Cheng-Sheng

Research on Maximum Entropy

Model for Feature Indexing Chinese Journal of Computers Vol. 27 No.9 Sept. 2004

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

233

Web based Collaborative Environment for Engineering Graphics Education Shen Lianguan, Li MujunWang Xiaodong, Zhao Wei, Zheng J.J. Department of Precision Machinery and Precision Instrumentation, University of Science & Technology of China Hefei, Anhui 230026 China [email protected]

Abstract: A web based interactive environment specially for Engineering Graphics education is proposed, which provides a virtual collaborative education platform for geographically distributed teachers and students. The system of B/S (browser/server) mode, different from general netmeeting program, can make a vectorgraph of DWG format show on clients' browsers online synchronously. And then the graphics can be discussed by modifying the objects or putting designed marks on it, as if on a sheet of blueprint. The updated objects by a user can show on the others’ browsers immediately. The users can also talk and listen to each other to express their own opinions. In the system the users are organized in different groups according to respective interested topics. In a group there is a controller and the control power can be transferred among the members. The system serves as a bridge between students and teachers for their extracurricular activities. With the tool the student can give his/her haze on browser and ask for help or choose their desired teacher to give some advice, while the teacher can correct student’s schoolwork on their own browser. When the schoolwork is checked and corrected, no suspicion of its version is needed to have, the original drawing remains intact and the subsequent external marks are saved in different file respectively. Once the teacher wants to disabuse and give his instructions to one or some students he should only declare or inform them to come to the website at same time and no necessary to come to classroom. Now it has integrated together with a self developed Web based Engineering Graphics courseware and a praxis system into a complete web based engineering education system. Keywords: collaborative environment, Engineering Graphics, Web based CAD, virtual classroom Introduction In the movement of rapid development of manufacturing and information technology, the engineering education should be changed from the architectonic and systematic viewpoint. With the impetuous development of the multimedia and Internet technology in the 1990’s, the education reform has been brought its unprecedented opportunity. The curriculum of Engineering Graphics is one of the required and public elementary courses in the universities and colleges in China. The main goal of the course is to let students master the skills of drawing and reading engineering drawings. Its fundamentality and age-old characters have aroused our enthusiasm to give it new vigorous vitality. In consideration of the whole education chain we have proposed a series of measures. With the multimedia technology we have developed a package of the whole teaching material of Engineering Drawing courses and put it on the Web since 1998[1]. The courseware hammers at conversion between 3D model and 2D views and imitates the action of the teachers in class vividly, lively and accurately. So the students get the knowledge in a relaxed studying environment by audio-visual senses synchronously. As a result, a great ardour for studying curriculum has been excitated unexpectedly. The practice in the passed

234

L. Shen et al. / Web Based Collaborative Environment for Engineering Graphics Education

8 years has proved its favor for the raising of students’ quality. Fig. 1 is the image of one of the webpage with amicable man-machine interaction. As a matched component, we have constructed a Web based interactive praxis system. Its most notable features are Engineering drawing-based and web-based[2]. Fig.2 shows its Web Business logic Schema. Teaching practice has proved the mentioned measures are very effective for cutting the teaching time and enhancing teaching qualities. But some deficiencies have emerged out. Shortage of gesture language and direct communion between teachers and students face to face, especially, when the students have some suspicions or difficulties, has caused some puzzle dom. In the case, it is necessary to have an interactive communication platform to gather teacher and his (her) students and make teaching and learning synchronously and asynchronously possible. So we hope the extracurricular activities expanded collaboratively. And then an idea to construct a virtual classroom has bourgeoned. The kernel technology is to develop a web based CAD system (we name it Webcad).

Fig. 1. Architecture of the system Fig.2. Connection process

1. Fundamental aspects of the system 1.1. The architecture of the system To implement a reliable, fast-response and platform-independent cooperative system, Java Socket is used to construct the architecture for the distributed system. The system is mainly composed of three tiers: Browser/Server/Data, as shown in fig.1. On the client side, a web browser with an embedded applet provides the GUI (Graphical User Interface) for users. The Server side contains all the application logic and maintains the connections with clients, exchanges and manages the data from both database and the user workspace. In the 3-tier architecture mentioned above, the Java Database connectivity (JDBC) is used for the connection between the Data and the Server, while the Java Socket is used for the communication between the Client and the Server. The detailed connection process is described in the Fig. 2.When the user logins, a socket connection will be created. And if the connection is successful a new thread will be created in the Server, which will then take charge of all the data and transactions of the users. Then during the corporation work, when an action is performed by a user, this operation message will be sent to the Server, and then dispatched to all other users after processing. This enables the system to keep the data and operation synchronous among users.

L. Shen et al. / Web Based Collaborative Environment for Engineering Graphics Education

235

1. 2 The priority-based mechanism of collaborative control The collaborative discussing team is comprised of polyonymous participants. So it is necessary to establish an access control mechanism to resolve conflict occurred during the collaborative drawing. In the system, the server maintains a sequence with the objects created by the users. Besides, a simple but practical and effective access control model is adopted[3]. When an operation happens to an object, there are three possible cases, as described in the Fig. 3. A) When a user (user_1) manipulates his own objects, he has the priority (named Menu bar

Tool bar

Created by User_1

A)

User_1

Action Obj_A

Execution topic list

Created by User_1

B)

User_2

Action Obj_A

Request priority

granted Execution

user list

Not granted Us er_ A 3 ctio n

C) User_2 Action

Obj_A

Chat area

failed Compare The highest priority PriorityExecution

Notify all users

1 on User_ Acti

Fig .3 The priority-based mechanism

Text input field

Workspace for corporation design

Fig. 4 Overview of the interface

owner-priority) to perform the action. So the operation will be executed immediately. B) A user (user_2) has no priority to manipulate the objects created by others. When user_2 requests to operate the objects created by user_1, a message should be sent to the user who creates the object, and ask for the priority to perform the action. If the request is approved, user_2 will get the priority (named granted-priority). The operation will be executed and after that user_1 will lost the granted-priority. C) When concurrent manipulations from many users happen to one object, the server will compare the priorities of the users. Only the operation from the user who has the highest priority will be executed, and then a message will be sent to all users. The strategy to compare the priority in WebCAD is, 1) the owner-priority takes precedence of all granted-priority, and 2) the granted-priority is characterized by time order, and the previous granted-priority takes precedence of the later one. 2 Multi-functional system Fig. 4 shows the web page of the multifunctional webCAD[4]. No software needs to be installed in the client side. To enhance the facilities a number of tools are provided for the users. (A) Drawing tools General drawing functions necessary for creating draft are developed in the WebCAD, just like line, circle, arc, ellipse, NURBS, text annotation and dimension annotation, and so on. The results are stored in common format (DWG). It is very important to be compatible with the commercial CAD software (like AutoCAD). (B) Browse tools The functions provide a precondition for collaborative discussing. These functions are designed to load DWG format files from the database of the system or from any hyperlinked

236

L. Shen et al. / Web Based Collaborative Environment for Engineering Graphics Education

URL, and then to operate the drafts. The people gathered in a group can browse a same draft at the same time. Different groups can load different drafts. (C) Red-mark tools Red-mark tools are used to mark on drafts. The function is developed for teachers to correct the homework and for people to discuss and evaluate the pattern. A mechanism is developed to save the mark-up objects in SVG format[5] [6] and store the original draft in formats of DWG. In the way, the homework of the students will be kept in their original version. SVG (Scalable Vector Graphics) is a new XML-based (Extensible Markup Language) graphics standard from the W3C that will enable Web documents to be smaller, faster and more interactive. (D) Multimedia communicating means. Users can also speak or write to each other by using the tool provided by the WebCAD. This tool has been fulfilled based on the Java Multimedia Framework (JMF). Talker or writer is exclusive and the others are listeners or readers. To express their opinions vividly, the users can use the image tool to import images. Currently the support image file formats include BMP, jpg and Gif. 3. Conclusions

This paper introduces our research work on developing a web-based synchronous cooperative environment for Engineering Graphics Education. Based on Java technology, a 3-tier architecture is proposed. A number of tools (such as the browse tools, the red-mark tools, the communicate tools and so on) are provided to make it a multifunctional system. In the system users can view and modify CAD files online, and discuss their opinions conveniently and efficiently with one another. SVG, which is a XML language for rich graphical content, is employed to store the collaborative discuss results for its advantage of platform-independence, small size, and standardization. At the same time, the concurrency problems (for example, the access control and the version control) are also designed and solved to reduce the collision in the distributed synchronous cooperative design. The system has been applied to the Engineering drawing education in USTC, and proved to be an effective way. Future issues include the improvement of usability of the system, like the tolerance auto-generation module, the 3D module and so on. Acknowledgments

This project is financially supported by the Education Reform Fund from Anhui province and our university USTC. We also thank the people, Li, Shufang, Wang, Hongwei, Li Liyu and the like, who participate in the project in the passed 8 years. References [1]. Shen, LG et al.(2001)Development and Practice of web Based Course of Mechanicel Drawing. Proceedings of IceCE, Xi’an, 16-18. [2]. Shen, LG and Wang, RF (2004) Research and Reform of the Teaching of the Course in Mechanical Drafting. Jiaoyu yu xiandaihua/ Chinese Journal of Education and modernization, v73, 34-40. [3]. Shen, LG and Li, MJ, et al.(2006) Web based cooperative virtual product design environment shared by designers and customers. Lecture Notes in Computer Science, Springer-Verlag, Berlin Heidelberg, v3865, 384-393. [4]. Zhou, Zq and Shen, LG, et al.(2005) Research of customers oriented virtual design environment. Jixie Gongcheng Xuebao/Chinese Journal of Mechanical Engineering, v 41, n 6, June, 137-142. [5]. http://www.w3.org/XML/ [6]. http://www.w3.org/TR/SVG/intro.html

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

237

Time-based Self-learning Support Using Collaborative Learning Process Masahide Kakehi+, Tomoko Kojiri++ and Toyohide Watanabe+ Department of Systems and Social Informatics, Nagoya University, Japan ++ Information Technology Center, Nagoya University, Japan {kakehi, kojiri, watanabe}@watanabe.ss.is.nagoya-u.ac.jp

+

Abstract: During the self-learning, learners tend to review the interaction log of collaborative learning of the similar exercises which they have already solved. However, because some utterances are not important, learners take a lot of time and work to review all utterances. Additionally, learners cannot always take notice of effective utterances. In order to cope with such situation, our objective is to construct a system which extracts effective utterances from learners’ viewpoints and provides the collaborative learning log with effective utterances highlighted. The effective utterances for learners change according to interval time between collaborative learning and self-learning. In this paper, experimental results for investigating effective utterances based on the interval time from the collaborative learning are described. Keywords: Self-learning, collaborative learning log, effective utterances, learner’s viewpoint, interval time between collaborative learning and self-learning

1. Introduction Learners communicate with other learners through the network in the collaborative learning environment. During the learning, learners utter what they understood and what they thought. So, it is useful for learners to review the interaction log when they tackle similar type of exercise in self-learning. However, it is difficult for learners to find utterances that help them to solve the exercise from the interaction log because there contain a huge amount of utterances. Currently, we have constructed a system which detects effective utterances for each learner and provides the collaborative learning log with effective utterances highlighted [1,2]. This system focuses on exercises that contain several answering paths, such as programming. Effective utterances for learners are classified into two types: utterances that of they think effective during learning, and effective utterances which they cannot notice of during the learning. When learners try to derive the answer by the same answering path which they derived during the collaborative learning, they recall the former type of utterances. On the other hand, when they want to discover new ideas, the latter type of utterances is important. Therefore, in our approach, utterances are classified into four groups according to annotations that are added by learners during the collaborative learning: utterances which learners recognized as the same viewpoint, utterances which learners thought different viewpoint, utterances of the same viewpoint which learners did not notice, and utterances of the different viewpoint which learners did not notice. However, the types of utterances that learners require may change according to the interval time between collaborative learning and self-learning. In order to investigate characteristics of utterances needed for each moment, experiments that compare effective utterances according to learners’ characteristics and interval time from collaborative learning are arried out. By providing new selection method in considering the interval time and learners’ characteristics, our system can provide more effective utterances.

238

M. Kakehi et al. / Time-Based Self-Learning Support Using Collaborative Learning Process

2. System for providing effective utterances Our system detects effective utterances for each learner and provides the collaborative learning log with effective utterances highlighted [1,2]. Since effective utterances for learners are different, two kinds of annotations are prepared: the same viewpoint and different viewpoint. The annotation of the same viewpoint indicates that utterances are useful for a learner to derive an answer during the collaborative learning. The annotation of the different viewpoint means that utterances are not effective for deriving the answer by his answering path, but may be key utterances for deriving different answering paths. Difference between answering viewpoints of learners is determined by their annotations. When both learners attach annotations of the same viewpoint to the same utterance, their answering viewpoints are the same. If one learner adds an annotation of the same viewpoint and another attaches that of the different viewpoint, they are regarded to have the different viewpoints. However, if both learners add annotations of different viewpoints, whether their answering viewpoints are the same or not is not specified. Effective utterances that learners cannot notice during the learning can be established based on other learners’ answering viewpoints and their attached annotations. Table 1 indicates the meaning of utterances to a learner according to the viewpoint of learners who attach annotations. Utterances are classified into the same and different viewpoints on the basis of answering viewpoints of learners and annotations which learners adds annotation. Table 1: Viewpoint of candidate of effective utterances Annotation : Same Annotation : Different Learner’s viewpoint : Same Same viewpoint Different viewpoint Learner’s viewpoint : Different Different viewpoint ---------------------------Let us assume the situation shown in Figure 㧝. In this example, the learner A and the learner B have the same answering viewpoint, while the learner C is different. When A attaches the annotation of the same viewpoint to an utterance, the utterance is the same viewpoint for B, too. However, for C, the utterance provides an idea for the different method. Thus, in our system firstly, viewpoints of other learners are determined by comparing with their annotations. Then, based on the viewpoints of learners, annotated utterances are classified into the same or different viewpoints.

Figure 1: Viewpoint of utterances based on learners’ viewpoints 3. Experiment The objective of our experiment is to investigate the change of effective utterances of learners according to the interval time after the collaborative learning has been finished. Twenty examinees in our laboratory participated in this experiment. Firstly, they had formed four groups and had studied collaboratively. Then, they were divided into two groups. In the group 1, four examinees were asked to study the same exercise individually three months after the collaborative learning. In the group 2, three examinees did the same activity a few days after the collaborative learning. At the self-learning, the interaction log of the collaborative learning was provided. Examinees were asked to mark utterances that were used to solve the exercise in the self-learning.

M. Kakehi et al. / Time-Based Self-Learning Support Using Collaborative Learning Process

239

The ratio of marked utterances in the utterances that system extracted were 56.5% for group 1 and 83.3% for group 2. The experimental results indicate that utterances which our system extracted are effective for examinees of both groups. Especially, for examinees with short interval time, utterances which our system extracted were more effective. However, when the interval time is long, examinees may forget contents discussed in the collaborative learning process. For further analysis, in order to investigate the relations between learners’ characteristics and effective utterances, group 1 was focused and annotated utterances for individual examinees were analyzed. Table 2 shows ratios of their own utterances in those that they annotated. They show remarkable differences among students. Figures 2 and 3 represent the relations between examinees’ own utterances and the ratios of the same and different utterances that they had annotated. In Figure 2, the positive correlation can be seen, and a negative correlation can be seen in Figure 3. Table 2: Group 1 Ratio of annotation to own utterance Examinee A 83% Examinee B 25% Examinee C 0% Examinee D 75% Average 45.8% Figure 4 indicates the contributions of each examinee during the collaborative learning. Figure 4 represents the relations between ratio of examinee’s utterances in all annotated utterances and ratio of examinee’s annotations in all annotations. Examinee A added many annotations, while only a few utterances of him were annotated. Therefore, it is seemed that he did not to contribute to the learning. On the contrary, examinee D did not attach many annotations, while his utterances were added with a lot of annotations. Therefore, with the results of Figure 4, examinee D leaded the learning and other learners acquired knowledge from examinee D. According to the results of Table 2 and Figure 2, 3, and 4, utterances of the same viewpoint are effective for learners who did not attach many annotations and were added many annotations by other learners. On the other hand, it is effective for learners who attached many annotations and who were not added many annotations to present utterances of the different viewpoint.

Figure 2: Relation between learners’ Figure 3: Relation between learners’ utterances and effective utterances utterances and effective utterances of same viewpoint of different viewpoint

240

M. Kakehi et al. / Time-Based Self-Learning Support Using Collaborative Learning Process

Figure 4: Contributions of examinees for collaborative learning These facts mean that a learner who utters effective opinions in collaborative learning tends to refer utterances of the same viewpoint in self-learning. On the other hand, a learner who did not contribute to the collaborative learning is likely to select utterances of different viewpoints in self-learning. 4. Conclusion In this paper, the experiments were executed in order to investigate characteristics of effective utterances that learners require according to the interval time between collaborative learning and self-learning and learners’ contributions to the collaborative learning. Experimental results indicated that utterances extracted based on answering viewpoints are effective for learners who use interaction log of the collaborative learning after a long interval time. Moreover, it is proved that learners who lead the collaborative learning prefer to utterances of the same viewpoint and learners who did not utter the meaning opinions use the other learners' utterances and utterances of a different viewpoint. In our future work, we should consider the way to present effective utterances to learners according to the experimental results. Furthermore, we try to construct a system which supports learners who do not participate in collaborative learning to study the same exercises in self-learning according to the collaborative learning log of other learners who study collaboratively. Acknowledgments The authors would like to thank the 21st Century COE (Center of Excellence) Program for 2002, a project titled Intelligent Media (Speech and Images) Integration for Social Information Infrastructure, proposed by Nagoya University. References [1] M. Kakehi, T. Kojiri, and T. Watanabe: Annotation Interpretation of Collaborative Learning History for Self-Learning, Proc. of KES2006 [to be published in October 2006] [2] T. Kojiri, Y. Ogawa and T. Watanabe: Agent-oriented Support Environment in Web-based Collaborative Learning, Journal of Universal Computer Science, Vol. 7, Issue 3, (2001), 226-239.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

241

The Design of a Collaborative Learning Environment in a Mobile Technology Supported Classroom: Concept of Fraction Equivalence Siu Cheung KONG Department of Mathematics, Science, Social Sciences & Technology, The Hong Kong Institute of Education, Hong Kong [email protected] Abstract. Knowledge of fraction equivalence is fundamental for building knowledge of fractions. This study aims to design a collaborative learning environment for developing the concept of fraction equivalence in classroom settings. According to the cognitive conflict theory, the key for developing a concept is to arouse a strong interest or an appropriate level of anxiety in learners in recognizing the anomaly clearly and reappraising the cognitive conflict situation deeply. A mobile learning environment for collaborative engagement is created in this study to encourage the resolution of cognitive conflict. Two situations are designed to stimulate reflection of learners about the concept of fraction equivalence: one is triggered by the anomaly between the learning peers and the other is triggered by the anomaly between the learner and the computer system, which has the authority of correctness in the situation. Two pedagogical tools for encouraging reciprocal tutoring in the collaborative learning environment are elaborated. Keywords: cognitive conflict, collaborative learning, fraction equivalence, mobile learning

1. Introduction Collaborative learning is a process that encourages learners to participate in the coordinated and synchronous learning activities with a number of other learners [1]. It emphasizes the concept “every learner learns from everyone else” [2, p215]. This learning process provides learners with the opportunity to make contributions and appreciate the contributions of others. There are four characteristics of collaborative learning: sharing knowledge among peers, sharing learning authority, mediation from teachers, and heterogeneous grouping of learners [3]. In collaborative learning, learners take the role of knowledge provider in sharing their own knowledge and learning strategies with other group members. Learners can be a source of knowledge, which is traditionally regarded as the figure of learning authority in the learning process. The sharing of knowledge among teachers and learners thus leads to the sharing of learning authority among teachers and learners. Learners are empowered with learning control to a greater extent in collaborative learning. Teachers play the role of facilitator in collaborative learning to provide mediation for learning in groups, such as the adjustment of information flow and interaction among groups and group members. The heterogeneous grouping of learners is important in collaborative learning. This grouping strategy allows reciprocal tutoring and knowledge exchange among learners with diverse perspectives, experiences, and backgrounds.

242

S.C. Kong / The Design of a Collaborative Learning Environment

Collaborative learning is good for learners to develop knowledge and interpersonal communication skills. However, two obstacles decrease the effectiveness of the learning process: problems in class control during the active participation of learners and the unsatisfactory participation of particular quiet learners [4]. Mobile learning, an emergent learning approach, can address these two obstacles of collaborative learning. Mobile learning refers to the use of mobile technologies for learning and teaching. It is characterized as learning across space, time, topics, and technologies [5]. Mobile learning can assist collaborative activities across space and time, and can be used in traditional classrooms across topics and technologies. There are three attributes of existing mobile technologies that facilitate the design of collaborative learning activities in classroom settings. Firstly, the visualization capability of mobile devices enables learners to distribute cognitive loading to visualization tools. Learners are thus able to engage in activities for deep learning of subject matter that requires visualization support [6]. With the use of mobile devices, abstract concepts can be visualized and manipulated through visual representations. Learners may share and communicate their ideas and knowledge with visual support in collaborative learning activities. Secondly, most mobile devices are compatible with desktop computers. Software can be installed in mobile devices to perform required computation tasks. With the installation of appropriate software codes, such as coding for data-mining support in mobile devices, teachers are able to trace the progress and learning behavior of learners during learning tasks [6]. This also implies that with appropriate software design, teachers can make use of wireless-networked mobile devices to facilitate class control in collaborative learning activities, such as organizing and maintaining learners in heterogeneous groups and supervising the engagement of learners in collaborative tasks by generating questions with appropriate difficulties. Thirdly, the anonymity in the communication with the use of mobile devices helps to increase the participation of learners in collaborative learning activities by allowing them to express their ideas without revealing their identities to the class. The mobile device enables learners to privately work, reflect, and explore learning materials without others observing the development of their work. Learners can share the final product of their work with the class when they feel it has reached a satisfactory level. This dynamic combination of private work and public sharing may enhance learning in traditional classrooms [7]. Even quiet learners can share in the immense satisfaction of participating and contributing ideas without the fear of making incorrect responses.

2. Cognitive Conflict Understanding mathematical ideas often involves the restructuring of mathematical schema of learners. This restructuring process is intricately linked with the occurrence of cognitive conflict [8]. Cognitive conflict is a part of the psychological theories of cognitive change. It involves an inferred state of incompatibility between two inferred component states within the cognitive process [9]. In general, cognitive conflict is a perceptual state in which one notices the discrepancy between an anomalous situation and a preconception [10]. Since the 1980s, cognitive conflict has been regarded as a feasible teaching strategy because its nature of cognitive change induces introspection of learners towards the newly learnt conception that is incongruous with a preconception or existing misconception. According to the cognitive conflict process model, there are three stages in the engagement of cognitive conflict in learning [10]. The first stage is the preliminary stage,

S.C. Kong / The Design of a Collaborative Learning Environment

243

in which an anomalous situation that is different from belief of learners of pre-existing conceptions is introduced. The second stage is the conflict stage, in which learners recognize and reappraise the anomalous situation with the expression of interest or anxiety in resolving the cognitive conflict. The third stage is the resolution stage, in which learners try to resolve cognitive conflict in any way that they can. Cognitive conflict has three types of potential: constructive, destructive, and meaningless potential. When learners clearly recognize an anomaly and reappraise a cognitive conflict situation deeply with the expression of strong interest or appropriates anxiety, the cognitive conflict has constructive potential. When learners do not recognize an anomaly or simply ignore it with an expression of bad feeling, such as frustration or rejection, the cognitive conflict is regarded as destructive. When learners recognize an anomaly but accept it passively without interest and cognitive reappraisal, the cognitive conflict shows meaningless potential. Early studies have shown that the induction of constructive cognitive conflict promotes positive outcomes in classroom learning [11]. The creation of constructive cognitive conflict in learning largely depends on interdependency among learners. Hence, the creation of constructive cognitive conflict is closely related to the collaborative learning process that provides learners with ample opportunity to learn from peer-discussion. The concept of fraction equivalence is regarded as one of the most difficult topics in mathematical learning because of its abstract nature [12]. Knowledge of fraction equivalence is fundamental for building knowledge of fraction operations, such as adding or subtracting fractions with unlike denominators [13]. The construction of this mathematical knowledge is often accompanied by the occurrence of cognitive conflict [12]. As research findings have indicated that technology supported graphical modeling helps with the learning of fractions [12, 13, 14, 15], this research aims to design a collaborative learning environment for developing the concept of fraction equivalence in a mobile technology supported classroom. A mobile learning environment for collaborative engagement is created in this study to encourage the resolution of cognitive conflict and reciprocal tutoring in classroom settings.

3. Design of the Mobile Learning Environment for Collaborative Engagement 3.1 Mobile Technology Supported Classroom Collaboration is a coordinated, synchronous activity through a continued attempt to construct and maintain a shared conception of a problem [1]. In this study a series of synchronous interactions in a mobile technology supported classroom is designed to encourage learners to engage in learning tasks, and a mobile platform is established for immediate interaction between learners in collaborative pairs. Figure 1 depicts the mobile technology supported classroom for learning the concept of fraction equivalence in pairs. The learning activity of fraction equivalence takes place in a wireless-networked classroom. The mobile device that is used in this learning activity is a pocket PC because of its portability and relatively large screen. The teacher and learners are provided with a pocket PC. The pocket PC of teacher is pre-installed with the interface for managing the pair grouping and the organizing of learning activities. The pocket PC of learners is pre-installed with a graphical tool for learning fraction equivalence. The learners interact in pairs through a server that is connected to a SQL database. The server acts as a grouping coordinator of grouping requests from teachers and a communication coordinator of synchronous interactions between paired learners.

244

S.C. Kong / The Design of a Collaborative Learning Environment

Client Pocket PC

Server

Administrator Pocket PC

Student A Get

Communication Module

Post

Get

Grouping Module Post

Return Data Get Data Get Post

Student B

Teacher

Database

Record of Students

Figure 1: The mobile technology supported classroom for learning the concept of fraction equivalence

3.2 Pedagogical design for collaborative learning: Two situations for reflection Successful collaborative learning tasks should allow learners to achieve mutual engagement, negotiation, self-reflection, shared understanding, and mutual agreement [1]. The scale of grouping, the type of interaction, and the task of collaborative learning are critical factors for achieving these goals. To facilitate in-depth discussions among group members for developing the concept of fraction equivalence, learners are grouped into pairs with fellow classmates to conduct fraction comparison tasks. After logging onto the system, learners are placed in a context in which they decide the equivalence of two fractions with the support of graphical representations of the fractions. One learner is the designated question-setter, while the other is the question-replier. The learners alternate playing the two roles. There are three steps in this learning activity. Step 1 is the process of question-setting. In this step, the learners in the role of question-setter set and send out questions about the equivalence of two fraction expressions. The learners have to state whether the two fraction expressions that they set are equivalent to the aid of graphical representation of the two fraction expressions at the top of the interface. Once the learners are satisfied with the question set, they can click the “Confirm” button to send out the question to their partners through the server. Step 2 is the process of question-reply. In this step, the learners in the role of question-replier receive questions from their partners. When they receive the two fraction expressions, they have to decide on the equivalence of the two fraction expressions with the help of the visual representations of the fractions. After the learners have indicated their decisions, they click the “Confirm” button to send out their answers to the server. Step 3 is the process of judgment. In this step, the computer system plays the role of learning authority to assess the correctness of the questions that are set by the question-setters and the answers that are provided by the question-repliers. The computer system sends messages of “Correct” and “Incorrect” for the right and wrong questions or answers. There are two possible types of cognitive conflict that are engendered in the learning activity for the achievement of learning by reflection: one is triggered by the anomaly between the learning peers, and the other is triggered by the anomaly between the learner and the computer system. When one member of a pair of learners provides the correct question or answer, while another member gives the wrong question or answer, the computer system displays the message “Please Discuss” (see Figure 2a). This generates the first type of cognitive conflict ņ it invites learners to share understanding, and to engage in self-reflection and negotiation through collaborative interaction.

S.C. Kong / The Design of a Collaborative Learning Environment

Figure 2a: The computer system generates a message “Please Discuss” if group members have different views

245

Figure 2b: Each learner has to click the “Discussion Finished” button after the discussion

When both members agree to finish their discussion, they have to click the “Discussion Finished” button (see Figure 2b) to inform the computer system. The computer system then generates the message “Correct” or “Incorrect” for the question-setter and question-replier (see Figures 3a and 3b). These authority judgments create the second type of cognitive conflict when they differ from the judgments of the learners. This offers learners a second opportunity to engage in self-reflection and to share understanding through a post-task discussion.

Figure 3a: The computer system generates a message “Correct” for the right question set

Figure 3b: The computer system generates a message “Incorrect” for the wrong answer

4. Design for Resolution of Cognitive Conflicts 4.1 The status of groups of learners in learning the concept of fraction equivalence The aim of this study is to equip learners with basic knowledge of fraction equivalence through collaborative learning in a mobile technology supported classroom. It emphasizes the knowledge sharing of learners with graphical support in the learning process. The different learning progress of individual learners determines the learning progress of each

246

S.C. Kong / The Design of a Collaborative Learning Environment

group. Figure 4 depicts the status of groups of learners in the process of learning the concept of fraction equivalence. (Case 4) Student A -- 9 Student B -- 9 Learning Status 3 (Case 2)

(Case 3)

Student A -- 9 Student B -- 8

Student A -- 8 Student B -- 9

Learning Status 2

Learning Status 2 (Case 1) Student A -- 8 Student B -- 8 Learning Status 1

Figure 4: The status of groups of learners in the process of learning the concept of fraction equivalence

Case 1 is expected to occur commonly at the beginning of the learning process. In this case, both members of a group have a preconception or misconception about the equivalence of two fractions. The learners always set and reply to questions incorrectly. In this situation, learners always encounter both the first and second type of cognitive conflict. The groups of learners in this situation fall into learning status 1. Case 2 and Case 3 occur when one of the group members begins to grasp the concept of fraction equivalence better than his or her partner. The learner who has developed the concept of fraction equivalence begins to set correct questions and make responses to questions with correct answers, while his or her counterpart cannot always achieve this status. The learners in this situation have the first type of cognitive conflict and the learner who is developing the concept of fraction equivalence will continue to experience the second type of cognitive conflict, in which an anomalous situation exists between the learner and the judgment of the computer system. The groups of learners in these cases are in learning status 2. Case 4 occurs when both learners in a group have a good understanding of the concept of fraction equivalence. The group members always set and reply to questions correctly. In this case, cognitive conflict rarely occurs. The groups in this circumstance achieve learning status 3. This is the learning goal of all of the groups. Some groups may go through learning status 1 and 2 to reach learning status 3 and some of the groups may go directly from learning status 1 to 3. The groups in learning status 2 are heterogeneous groups in this study. 4.2 Design for encouraging reciprocal tutoring The ultimate goal of this design is to help all of the groups of learners to attain learning status 3 through the learning activities in the collaborative learning environment. In this regard, the groups in learning status 1 and 2 require the attention and mediation of teachers to promote productive knowledge sharing. In this collaborative learning environment, teachers play the role of mediator, rather than the authority on the judgment of correctness of equivalence of fractions. Hence, the role of the teacher is to encourage reciprocal tutoring, which can be achieved more productively in a heterogeneous group context and by promoting such tutoring activities in this group context. This may enhance the quality of arguments between group members and thus the induction of more constructive cognitive conflict. To realize this goal, two pedagogical tools are designed to encourage the reciprocal tutoring of learners. The first is the re-grouping of group members and the second is the changing of the modes of question-setting. Figure 5 shows the interface of

S.C. Kong / The Design of a Collaborative Learning Environment

247

the pocket PC of teacher for the re-grouping of learners and the changing of the mode of question-setting.

Figure 5: The interface of the pocket PC of teacher for the re-grouping of learners and the changing of the mode of question-setting

Teachers who observe groups that are working in learning status 1, in which both group members struggle with a concept over a period, can use the first pedagogical tool to swap a member from the group in learning status 1 with a member from a group in learning status 3. This helps to achieve more heterogeneous groups in the learning environment, which in turn helps to encourage prolific reciprocal tutoring. Teachers who detect groups that are working in learning status 2, in which one of the group members consistently designs incorrect questions, can use the second pedagogical tool to designate another learner as the sole question-setter by changing the mode of question-setting from “Turn-Taking” to “Designation”. This creates an environment that allows the learners with better understanding to tutor learners who are still developing the concept. Once it is speculated that all of the groups have attained learning status 3, in which the learners have a good understanding of the concept of fraction equivalence, teachers can change the mode of question-setting to “Random” for the entire class so that the role of question-setter is assigned randomly by the computer system. The “Random” question-setting mode provides the opportunity for learners to explore the concept further in a relaxed mode of inquiry. This helps to consolidate the learning outcomes of learners.

5. Conclusion This study aims to design a collaborative learning environment for developing the concept of fraction equivalence in mobile technology-supported classroom settings. In this study learners are asked to group into pairs and to join in a synchronous learning activity with the use of pocket PCs in a wireless-networked classroom environment. Two situations for the generation of cognitive conflict are designed to stimulate the reflection of learners to develop new concepts: one is triggered by the anomaly between the learning peers, while the other is triggered by the anomaly between the learner and the computer system that has the authority of correctness in the situation. Reciprocal tutoring is considered as the key strategy in this study that helps learners to resolve cognitive conflict. Thus, the role of the teacher in the classroom in this design is to act as a facilitator to mediate and to promote the sharing of knowledge and learning authority by helping learners to form heterogeneous

248

S.C. Kong / The Design of a Collaborative Learning Environment

groups and by adopting different modes of question-settings in these groups. The use of mobile technologies to encourage collaborative learning is a promising research direction that deserves research effort to study its effect on classroom learning environments. We have begun to study cases on learning the concept of fraction equivalence in this collaborative learning environment. Further work on large-scale studies in investigating whether learners recognize and reappraise the anomalies and how they attempt to resolve the cognitive conflict in the learning process will be attempted after the pilot case study. References [1]

[2]

[3]

[4] [5] [6]

[7] [8] [9]

[10]

[11] [12]

[14]

[14]

[15]

Roschelle, J., & Teasley, S.D. (1995). The construction of shared knowledge in collaborative problem solving. In C. O㧓Malley (Ed), Computer-Supported Collaborative Learning (pp. 145-168). Berlin: Springer-Verlag. Fischer, F., Bruhn, J., Gräsel, C., & Mandl, H. (2002). Fostering collaborative knowledge construction with visualization tools. Learning and Instruction, 12, 213-232. Dillenbourg, P. (1999). Introduction: what do you mean by “collaborative learning”? In P. Dillenbourg (Ed), Collaborative Learning: Cognitive and Computational Approaches (pp. 1-19). Amsterdam: Pergamon. Roschelle, J. (2003). Keynote paper: unlocking the learning value of wireless mobile devices. Journal of Computer Assisted Learning, 19, 260-272. Sharples, M., Taylor, J., & Vavoula, G. (2005). Towards a theory of mobile learning. Proceedings of mLearn 2005 Conference (9 pp.), Cape Town: mLearn 2005. Roschelle, J., & Pea, R. (2002). A walk on the WILD side: how wireless handhelds may change computer-supported collaborative learning. International Journal of Cognition and Technology, 1(1), 145-168. Vahey, P. and Crawford, V. (2003). Learning with handhelds: findings from classroom research. Menlo Park: SRI International. Tall, D. (1977). Conflicts and catastrophes in the learning of mathematics. Mathematical Education for Teaching, 2(4), 2-18. Cantor, G. N. (1983). Conflict, learning, and Piaget: comments on Zimmerman and Bloom㧓s “Toward an empirical test of the role of cognitive conflict in learning”. Developmental Review, 3, 39-53. Lee, G.H., Kwon, J.S., Park, S.S., Kim, J.W., Kwon, H.G., & Park, H.K. (2003). Development of an instrument for measuring cognitive conflict in secondary-level science classes. Journal of Research in Science Teaching, 40(6), 585-603. Limón, M. (2001). On the cognitive conflict as an instructional strategy for conceptual change: a critical appraisal. Learning and Instruction, 11, 357-380. Kong, S.C. and Kwok, L.F. (2002). Modeling a cognitive tool for teaching the addition/subtraction of common fractions. International Journal of Cognition and Technology, 1(2), 327-155. Kong, S. C. (2005). An experimental study on a cognitive tool for classroom use: a knowledge of fraction equivalence. In C.K.Looi, D. Jonassen, & M.Ikeda (Eds.), Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences (pp. 164-171). Amsterdam: IOS press. Kong, S.C. and Kwok, L.F. (2003). A graphical partitioning model for learning common fractions: designing affordances on a web-supported learning environment. Computers & Education, 40(2), 137-155. Kong, S.C., & Kwok, L.F. (2005). A cognitive tool for teaching the addition/subtraction of common fractions: a model of affordances. Computers & Education, 45(2), 245-265.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

249

Scientific Modeling of Technology-Mediated Collaborative Learning Processes YEUNG Yau-yuen Department of Mathematics, Science, Social Sciences and Technology Hong Kong Institute of Education, CHINA HONG KONG [email protected] Abstract: Based on the well-established Monte Carlo approach called Ising model in magnetism, a concrete scientific model has been reformulated to simulate the students’ processes of collaborative learning as induced by different kinds of group interactions between students who learn under the same intelligent tutoring system and technology-mediated learning platform/environment. Furthermore, a package of computer programs for this model has been specifically developed to provide numerical solutions or predictions for some hypothetical or real educational contexts which are of significant interest to the certain educators. Keywords: Collaborative learning, Technology-mediated learning, Computer simulation

Introduction Over the last six decades after the invention of the first computer, scientific computation or computer simulation has been employed intensively and successfully to solve a lot of scientific and engineering problems and simulate the corresponding processes in a wide variety of physical systems as ranging from nuclear and atomic energy levels, weather forecasting, oil exploration, aero plane design to evolution of the Universe (see, e.g. Refs. [1-4]). In recent years, many theoretical physicists are engaged to apply the traditional theories in thermal/statistical physics to model the price fluctuation in international financial markets which generate some essential information for assessing and managing the risk associated with a portfolio of investments and to predict some critical stock market phenomena such as market crashes [5-6]. The models and methods of successfully simulating the financial sector are mostly adopted from those concepts and skills (e.g. cellular automata, Ising model and group renormalisation etc. in Refs.[2-4, 7]) of statistical physics and solid state physics which are normally employed to understand the dynamical behaviors of various complex adaptive systems in which the macroscopic behaviors or properties of those systems are controlled by their interacting microscopic constituents. They have also been applied to some other branches of social sciences such as the dynamics of social impact (or the so-called voter problem) and other behavioral processes in sociology [8-10]. Lewenstein, Nowak and Latane [9] showed that the dynamics of social impact can be quantitatively described in terms of a class of probabilistic cellular automata whereas Kacperski and Holyst [10] further applied the similar approach to model the formation of opinion in a society as arising from a strong leader and from an external social impact acting uniformly on all individuals. Their model successfully simulated the processes in the formation of two basic stationary states of opinion, namely cluster of the strong leader’s adherents and unification of opinions. Despite of those successful applications of computer simulation in many areas of social sciences, educators and

250

Y.-y. Yeung / Scientific Modeling of Technology-Mediated Collaborative Learning Processes

educational psychologists are traditionally quite skeptical and uncomfortable with the use of mathematical and computational techniques to describe education because they have argued that social situations are too complex to be formulated in a few equations [8]. Consequently, there is very few research work done on the scientific modeling of the educational processes. During the last two decades, researchers have started to employ various theories and techniques in cognitive neuroscience and artificial neural networks to model the students’ learning in certain subject disciplines (see, e.g. [11-12]). Since student model [13] which can record and analyze data on students’ present state of knowledge and personal attributes forms one of the four key components of an intelligent tutoring system (ITS), Stathacopoulou and his co-workers [14] have further applied neural network-based fuzzy model to evaluate various characteristics of students in an ITS with very satisfactory results. On the other hand, Virvou et al [15] developed an evaluation agent that can simulate students’ behaviour in an ITS. Recently, Bordogna and Albano [16-18] have taken a multidisciplinary approach to model the human-based teaching and learning processes in a classroom using the common concepts and methods used in sociology, educational psychology, statistical physics and computational science. In essence, they found a high degree of similarity between the physical interactions underlying the formation of magnetization in magnetic substances and students’ learning in many classroom environments which involve both the student-student collaborative interactions and teachers’ influence. (see Figure 1).

S

S

H Figure 1: Alignment of spins (the direction represented by arrows) in a piece of iron under an applied magnetic field H. Complete magnetization occurs when all spins align along with H.

They suggested that the teacher in a classroom is equivalent to an external magnetic field H applied to magnetize a piece of iron and that the academic achievement of every student in a class is analogous to the alignment of the spins (or so-called small magnets) in individual atoms. For a strong magnetization to be formed in the iron, the applied magnetic field must be strong enough to align all the individual spins in the same direction. Individual spins, being small magnets in themselves, can influence the alignment of their neighboring spins, causing them to align in a certain direction for the minimization of the total interaction energy. On the other hand, the students’ academic achievement depends not only on the ability (e.g. level of subject knowledge and persuasiveness of the teaching methods) of the teacher, but also on the student-student interactions usually in form of collaborative or cooperative learning processes occurring in a classroom context. Of course, there are some negative or unfavorable effects such as the idle chattering or lack of attention, which tend to hinder the students’ learning (or misalign the spins). To make

Y.-y. Yeung / Scientific Modeling of Technology-Mediated Collaborative Learning Processes

251

quantitative prediction on the proper trends of the student’s learning path, Bordogna and Albano assigned parameter values to different kinds of teacher-student interactions, student-student interactions and student self interactions. They found that their predictions closely matched with those data gathered in the classroom by educational psychologists, hence verifying the basic validity of their model.

Reformulation of the Ising model According to the Ising model [7] which has been widely employed by physicists to simulate the magnetization processes, whenever a misaligned spin S is formed (e.g. due to thermal excitation or “noise” effect) in a magnet, it will quickly re-orientate itself to match its neighbours. In a similar sense, a struggling student will catch up more quickly if he/she interacts intensively with a group of able students who possess higher level of cognitive and metacognitive learning ability. After critically reviewing the basic assumptions used in Bordogna and Albano’s approach [1-3], a direct analogy comparison between the magnetization in a piece of iron and the students’ learning in a technology-mediated collaborative learning environment is given as follows: No. 1. 2. 3. 4. 5.

Magnetism Technology-mediated collaborative learning Electron spin Sj of the jth atom jth student’s knowledge Sj(t) at time t A magnet with N atoms A class of N students Magnetization as measure of the strength of a Average academic achievement <S> of the whole magnet class External applied magnetic field H Intelligent tutor’s influence or interaction JT with each student Spin-spin interaction (with parameters Jij) which Student-student interaction with parameters Jij is equivalent to an internal magnetic field called and Mij Weiss molecular field HM(i) =

6.

¦ J ij S j

j zi The overall magnetization is determined by the The overall learning achievement is determined average of individual spin’s total energy by 3 types of cognitive and metacognitive impact J ij S j Si CiTS ( t ),CiSS ( t ),CiSO ( t ) Ei = -H Si - HM(i) Si = HSi j zi arising from the intelligent tutor-student, student-student and student-other interactions. The absolute temperature T (=1/E in units of the The 3 types of “noise effect” or called “learning Boltzmann’s constant) is a measure of the environment temperature” for learning, which include all kinds of student-independent “noise effect” contextual factors, are denoted as 1/ETS, 1/ESS and 1/ESO. ESS(N) has a dependency on the class size N.

¦

7.

After making the analogy comparison, the second step is to postulate/reformulate the most appropriate formulae for the 3 types of cognitive and metacognitive impact. Again if we follow the Ising model [7] approach as adopted by Refs. [16-18], some plausible forms of the cognitive and metacognitive impact are given below: (I) cognitive and metacognitive impact for intelligent tutor-student interactions CiTS ( t )

S Si ( t ) J iT ( t )[ 1 Si ( t )ST ][ T ] 2

… (1)

in which Si(t) denotes the ith student’s prior knowledge (ranging from min.= -1 to max.=1) at the time t and ST represents the subject matter knowledge embedded in the intelligent tutoring system, normally having the fixed maximal value of 1 (unless its knowledge base contains erroneous or outdated pieces of knowledge). The coupling parameter JiT can be

252

Y.-y. Yeung / Scientific Modeling of Technology-Mediated Collaborative Learning Processes

written as a product of two terms: JiT(t) = Ji(t) ·JT in which the first term depends on the ith individual student’s cognitive and metacognitive abilities Ji(t) for learning with that particular ITS at time t. Value of the system-dependent factor JT is the same for all students and it depends on various educational aspects of the ITS such as (a) its instructional appropriateness and attractiveness of content presentation, (b) its effectiveness for probing the suitable questions and providing helpful hints, feedback and explanation for helping the students to overcome their learning difficulties, and (c) its ability for motivating, monitoring and assessing the students’ learning processes and for helping them develop appropriate learning strategies (i.e. metacognition). The present Eq.(1) is modified from Bordogna and Albano’s original approach [16-18] for human teachers by introducing the term in the second square bracket to account for situations of negative cognitive and metacognitive impact when the subject knowledge or instructional design of the intelligent tutoring system is very poor (e.g. ST < Si). Of course, some attributes of human teachers like teacher professionalism, teachers’ classroom management skills and teacher-student relationship could be omitted from the present model. (II) cognitive and metacognitive impact for student-student interactions CiSS ( t ) {

N

N

S j ( t ) Si ( t )

j 1, j zi

j 1, j z i

2

¦ J ij ( t )[ 1 Si ( t )S j ( t )] ¦ M ij ( t )[ 1 Si ( t )S j ( t )] }[

]

… (2)

which arises from two kinds of student-student interactions, namely (i) impact of jth student’s persuasiveness Jij(t) on the ith student which becomes favorable when Si < 0 but Sj >0; (ii) impact of jth student’s mutual support Mij(t) on the ith student which becomes favorable when Si and Sj are both positive or both negative, i.e. affinity factor when both students have similar knowledge level, learning style, metacognitive ability and common interests/hobbies etc. This equation is again modified from Bordogna and Albano’s approach [16-18] by replacing their sign function with the last term outside the curly bracket to properly account for the effect of a poor learning ability student on an able student when they interact or carry out some learning activities together because the former approach has omitted all cases with Sj < Si. The effect of different design principles for technology-mediated collaborative learning environments will be multiplicatively absorbed in the parameters Jij(t) and Mij(t). (III) cognitive and metacognitive impact for student-other interactions CiSO (t )

Ai [1 S i (t )]

… (3) th

in which Ai is a parameter representing the combined effect of the i student’s (a) self-learning ability and strategy (i.e. metacognitive ability in learning) and (b) interactions with or learning from other materials or persons (e.g. reference books, CD-ROM, TV programs, museum visits, friends, parents or school teachers etc.) rather than the intelligent tutoring system or students/classmates inside the same technology-mediated learning platform or environment. This equation is simplified from Bordogna and Albano’s corresponding equation with a special consideration to combine those two variables (a) and (b) into a single variable Ai because they are practically inseparable in this computer modeling of learning processes. To find the evolution of learning of an individual student, we shall apply the well-known Monte Carlo method (see Refs.[2, 7, 19]) for the simulation of those spin-like complex systems. At a given time interval 't, the student’s learning may lead to either (i) a knowledge gain of amount 'S which is a small incremental change of knowledge consistent with the artificial neural network model of learning [11-12], i.e.

Y.-y. Yeung / Scientific Modeling of Technology-Mediated Collaborative Learning Processes

Si(t + 't) = Si(t) + 'S

… (4)

Wi

… (5)

with a probability of Pi or (ii)

253

1 Wi

no knowledge gain, i.e. Si(t + 't) = Si(t)

… (4’)

with a probability of (1 – Pi) TS

SS

… (5’) SO

… (6) where W i e ETS Ci ( t ) E SS Ci ( t ) E SO Ci ( t ) is the generalized Metropolis rate (see [4, 7, 19] for details). It is remarked that the knowledge loss case, i.e. Si(t + 't) = Si(t) - 'S as proposed in Bordogna and Albano’s original approach [16-18] is replaced by the present formula (4’) because the latter seems to be more consistent with science educators’ view on learning [11-12].

Simulated Results and Discussion To obtain the numerical solution of Eqs. (4) and (4’) for plotting the average learning path of different kinds of students, we need to develop a package of computer programs for solving the set of equations (1-6) altogether. This would involve some very complex and intensive programming jobs which should best be first developed using the MATLAB software (see e.g. Ref.[3]) which provides a lot of built-in matrix functions to greatly reduce to amount of programming work. However, we have rewritten the computer programs using the C language so as to substantially reduce the computing time and remove its dependence on the proprietary MATLAB software for concurrently running it in different computer systems like Linux and Unix. In the Windows platforms, a user-friendly interface for inputting and adjusting parameters has been constructed using the Visual Basic software and so it enables those researchers who lack the versatile programming skills to easily apply this package for carrying the numerical simulations of various educational contexts that they want to investigate in detail. This computer simulation package has undergone many cycles of testing and debugging processes to ensure its validity and correctness. To illustrate the key features of simulated results that could be obtained from our package of computer simulation programs, we have applied it to study the following two sets of research questions which are simple enough for setting the relevant input parameters and are of broad interest to many educators and ITS developers: 1. Assuming the same distribution of student prior knowledge and learning abilities, how does the student-student interaction affect the students’ average learning paths under a favorable or an unfavorable learning environment? 2. Under a favorable learning environment with the same distribution of student learning abilities, how does the student-student interaction affect the students’ average learning paths if they have different level of prior knowledge? Since we do not have an ITS and e-learning platform at hand for extracting the real-life values of the input parameters required in the above equations, it is necessary to set some approximate assumptions and typical values of the parameters for generating the simulated results to address the above two questions. First, we assume the 3 kinds of noise temperature to be roughly the same, i.e. ETS = ESS = ESO = E (say) and Jij(t) and Mij(t) are both directly proportional to the product of Si(t) and Sj(t). Second, for simplicity, we consider and

254

Y.-y. Yeung / Scientific Modeling of Technology-Mediated Collaborative Learning Processes

set JT =1 (i.e. perfect ITS design) and the proportionality constant for Jij(t) and Mij(t) to be both 1. Third, in order to focus on the effects of the intelligent tutor-student and student-student interactions as induced by the e-learning platform/environment, we omit the student-other interactions by setting Eq.(3) to be zero. Fourth, to be compatible with Bordogna and Albano’s previous calculations, we take the class size N = 96 which has been divided into 32 groups (with different settings) with 3 students in each group. 1

Students' average knowledge level

0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

50

100

150

200

250 time t 300

Figure 2: Effect of collaborative learning on students’ average learning path under a favorable (upper pair of curves with 1/E = 0.25) and an unfavorable (lower pair of curves with 1/E = 2.0) learning environments. Collaborative learning is included for the thicker (red) curves whereas the thinner (blue) curves contain no student-student interaction.

To address the research question #1, we further assume an uniform distribution over a range from -0.2 to +0.2 for the students’ prior knowledge, i.e. -0.2 Si(t=0) 0.2. The noise temperature 1/E is set to be 0.25 and 2.0 to mimic the favorable and unfavorable learning environments, respectively. We have run the simulation programs for 300 classes each with 1500 Monte Carlo simulation time steps and the smallest unit of knowledge gain 'S is set to be 0.05. In order to show more details of the initial phase of knowledge development, the average results for the students’ learning path (or development of knowledge) are given for the first 300 time steps only in Figure 2 which clearly shows that students generally learn faster with higher ultimate knowledge level under a more favorable learning environment (with smaller “noise temperature 1/E”). Collaborative learning does help to enhance the students’ learning outcomes in both kinds of learning environments but its effect is more profound for the less favorable learning environment. Under a very favorable learning environment (i.e. curves for 1/E = 0.25) in which the ITS design is very effective in facilitating students’ learning, the student-student interactions have a negative effect on the average students’ learning at the early stage of their development of knowledge. The reason is likely due to the fact that the learning progress of the students with better prior knowledge were slowed down by interacting with the students with poor prior knowledge but the effect is only short term and observable mainly in the very favorable learning environment. For the research question #2, we further assume a uniform distribution for the students’ prior knowledge in which the three ranges (group A) -0.7 Si(t=0) -0.3, (group B) -0.2 Si(t=0) 0.2, and (group C) 0.3 Si(t=0) 0.7 correspond to 3 groups of students having lower, intermediate and higher level of prior knowledge, respectively. The noise

Y.-y. Yeung / Scientific Modeling of Technology-Mediated Collaborative Learning Processes

255

temperature 1/E is set to be 0.25 which mimics the favorable learning environments. As shown in Figure 3 with Monte Carlo time steps truncated to 100 only for showing more details of the initial phase of knowledge development, all three groups of students in average will sooner or later follow essentially the same learning path with the same average amount of ultimate knowledge acquired. However, the group C students (with better prior knowledge) is initially disadvantaged by the collaborative learning activities because they may be confused by interacting with other students who are having poorer prior knowledge or holding misconceptions in certain subject matter. Furthermore, they may have to sacrifice their self-learning time for the early phase of collaborative learning activities which are educationally less beneficial to them. However, their knowledge level will quickly start to increase when the other two groups of students (especially group B) have acquired good enough knowledge. Besides, those group C students maybe able to develop better metacognitive learning abilities and gain a deeper level of understanding of the subject matter because they need to get aware of the different/alternative conceptions held by other students and to apply higher order thinking skills to help other students rectify or remove their alternative conceptions/misconceptions. On the other hand, the group A students (having the poorest prior knowledge) gain the greatest advantage through collaborative learning as the initial rate of their knowledge development (i.e. slope of the curve) is the highest amongst the three groups of students. 1

Students' knowledge level

0.8

0.6

0.4

0.2

0

-0.2

-0.4

-0.6

0

10

20

30

40

50

60

70

80

90

100 time t

Figure 3: Average knowledge development paths for 3 groups of collaborative learning students having (A) lower (blue crosses), (B) intermediate (red circles) and (C) higher (green triangles) levels of prior knowledge (at time t = 0).

Conclusion Based on the Ising model of magnetism, a scientific model has been reformulated to simulate the collaborative learning processes provided by any intelligent tutoring systems and e-learning platforms or environments. A package of computer programs have been successfully developed to obtain some characteristic curves for describing the students’ average learning path under certain situations of significant interest in education. For more accurate application of this scientific model, it will involve the fitting of the model

256

Y.-y. Yeung / Scientific Modeling of Technology-Mediated Collaborative Learning Processes

parameters to the actual collaborative learning outcomes of the students as recorded in the ITS and/or e-learning platforms. The fitted values of those model parameters could be input into the computer package for predicting the changes in the students’ path of knowledge development as induced by the changes in the design of the technology-mediated learning environment. Hence, they can provide some useful hints or qualitative guidance to the ITS or e-learning developers for finding possible/innovative ways to improve their system design [13-15]. Besides, this will also form an acid test on the precision and validity of this new scientific model on technology-mediated collaborative learning processes and to a great extent on the study of other complex and dynamical systems in social sciences [20]. Acknowledgements This work evolved from an earlier project funded by the HKIEd Research Committee. References [1] Yeung, Y.Y. and Rudowicz, C. (1992). Ligand Field Analysis of the 3dN ions at Orthorhombic or Higher Symmetry Sites. Computers and Chemistry, 16, 207-216. [2] Wong, S.S.M. (1997). Computational Methods in Physics and Engineering. Singapore: World Scientific. [3] van Loan, C.F. (2000). Introduction to Scientific Computing: A Matrix-Vector Approach Using MATLAB. Upper Saddle River, N.J.: Prentice-Hall. [4] Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H. and Teller, E. (1953). Equation of state calculations by fast computing mechanics. Journal of Chemical Physics 21, 1087-1092. [5] Kertesz, J. and Kondor, I. (1998). Econophysics: An Emerging Science. Boston: Kluwer. [6] Chowdhury, D. and Stauffer, D. (1999). A generalized spin model of financial markets. European. Physical Journal B 8, 477-82. [7] Landau, D.P. and Binder, K. (2000). A Guide to Monte Carlo Simulations in Statistical Physics. Cambridge: Cambridge University Press. [8] Liebrand, W.B.G., Nowak, A., and Hegselmann, R. (Eds.)(1998). Computer Modeling of Social Processes. London: SAGE Publications. [9] Lewenstein, M., Nowak, A. and Latané B. (1992). Statistical mechanics of social Impact. Physical Review A 45, 763-76. [10] Kacperski, K. and Holyst, J. A. (1999). Opinion formation model with strong leader and external impact: a mean field approach. Physica (Amsterdam) 269A, 511-26. [11] Anderson, O.R. (1997). A neurocognitive perspective on current learning theory and science instructional strategies. Science Education, 81, 67-89. [12] Roth, Wolff-Michael (2000). Artificial Neural Networks for Modeling Knowing and Learning in Science. Journal of Research in Science Teaching, 37(1) , 63 – 80. [13] Zhou, Yujian, and Evens, M.W. (1999). A practical student model in an intelligent tutoring system. Proceedings of 11th IEEE International Conference on Tools with Artificial Intelligence, 13-18. [14] Stathacopoulou, R., Magoulas, G.D., and Grigoriadou, M. (1999). Neural network-based fuzzy modeling of the student in intelligent tutoring systems. Proceedings of International Joint Conference on Neural Networks, 5, 3517-3521. [15] Virvou, M., Manos, K., and Katsionis, G. (2003). An evaluation agent that simulates students' behaviour in intelligent tutoring systems. Proceedings of IEEE International Conference on Systems, Man and Cybernetics, 5, 4872-4877. [16] Bordogna C. M. and Albano E. V. (2001). Theoretical Description of Teaching-Learning Process: A Multidisciplinary Approach. Physical Review Letters, 87, 118701-4. [17] Bordogna C. M. and Albano E. V. (2001). Phase Transition in a Model for Social Learning via the Internet. International Journal of Modern Physics C, 12, 1241-1250. [18] Bordogna C. M. and Albano E. V. (2002). A cellular automata model for social-learning processes in a classroom context. European Physical Journal B, 25, 391-396. [19] Newman M. E. J. and Barkema G. T. (1999). Monte Carlo Methods in Statistical Physics. Oxford: Oxford University Press. [20] Kaput, J., Bar-Yam, Y., Jacobson, M., Jakobsson, E., Lemke, J. and Wilensky, U. (2000-2005). Two roles for complex systems in education: mainstream content and means for understanding the education system itself. Online: http://necsi.org/events/cxedk16/cxedk16_0.html

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

257

Exploring the Learning Effect of a Web-based Learning Community on EMBA Students a a

b

a

I-Fan Liu , Meng Chang Chen , Yeali Sun Department of Information Management, National Taiwan University, Taiwan b

Institute of Information Science, Academic Sinica, Taiwan [email protected]

Abstract: In this paper, we develop a Web-based learning community to assist learners’ on-line learning. The subjects of this research are 138 EMBA from a university in Taiwan who have taken a Management course. Most of them worked in Hi-tech companies. The Management course was taught by four co-teachers. The study was carried out from Sep. 2005 to Jan. 2006 – a period of 16 weeks. We explore the learning effect for EMBA students from Web-based learning community. Our important findings are: (1) EMBA students use the Web-based learning platform to obtain more knowledge about management and help to develop good social relationships with their peers. (2) By using the system, busy students can extend the study at home. In addition, the system improves interaction among students. (3) All most students use the discussion room a great deal, while others use it infrequently. (4) EMBA students are active in discussion with case study, which they provide are very interesting for professors and other students to know how other enterprises operate and what obstacles they encounter. (5) The level that different teachers integrate information technology into teaching differs greatly. Finally, we also propose some suggestions for two aspects of web-based learning community. Keywords: EMBA learning effect, Management, Web-based learning community

Background With the rapid development of information technology and the Internet has become a necessity for everyone. Thanks to easy access to information and the prevalence of broadband Internet, learning is no longer confined to classrooms. The government, with a view to promoting lifelong learning and organizing part-time university courses has allowed more colleges to set up graduate degree programs. According to the statistics from Taiwan’s Ministry of Education, there were 486 graduate degree programs, including EMBA, in 2005, with approximately 11,400 graduate students who held full-time jobs (double the number in 2002). In addition, colleges are now setting up more graduate degree extension programs, which present valuable lifelong learning opportunities for office workers. As we step into the new century and the era of the knowledge-based economy, the speed of change

258

I-F. Liu et al. / Exploring the Learning Effect of a Web-Based Learning Community

requires people to adapt rapidly to new technologies.

1. Literature Review The concept of a Web-based Learning Community developed from a “Virtual Community.” Boczkowsk [1] defines a Virtual Community as an online community whose members have the same interests and backgrounds. Here “community” stands for a group of people that forms an organization through interaction and communication [2]. It develops into a Virtual Community through use of the Internet [3]. From Qiu’s point of view[4], the Internet enables people from different backgrounds to conduct research and share their knowledge about specific domains, thereby shaping a unique “Web-based Learning Community” [5]. When designing a Web-based course, if we only put the learning materials on the Internet without an interactive mechanism, it cannot be called “Web-based Learning”. Stephen & Marshall [6] proposed that five levels of information technology integration should be taken into account designing a web-based learning course. The definitions of the five levels are as follows: (1) Level 1: Most classes consist of traditional classroom lectures, and brief syllabuses, timetables, or announcements are put on the website. (2) Level 2: Handouts like HTML documents and PowerPoint files are put on the website. (3) Level 3: some of the teaching is online. Teachers need special skills, such as homepage editing and multimedia skills. (4) Level 4: Web-based courses take over, and real classroom lectures become secondary. (5) Level 5: Online lectures replace all the traditional teaching activities.

2. Methodology In all, 138 EMBA students from a university in northern Taiwan participated in the study. The research was carried out from September 2005 to January 2006 – a period of 16 weeks. The Management course was taught by four teachers, denoted as A, B, C, and D. Based on the teaching requirements of the four professors, we developed a web-based learning platform to assist learners’ with online learning. The research questions addressed in this study are as follows: (1) Explore the interactive learning effect from case study of Management? (2) Explore teachers’ attitudes toward online applications and at which level do they use information technology as supported lecturing tools?

3. Data Analysis 3.1 Background of the Web-based Learning Community Members The subjects of this research are 138 EMBA students who have taken a management course. Their basic information is shown in Table 1.

I-F. Liu et al. / Exploring the Learning Effect of a Web-Based Learning Community

259

Table 1 Background of the Web-based Learning Community Members Category Gender

Age

Industry

Rank

Academic Background

Place of Living

Item Male Female 20-29 30-39 40-49 50-59 Hi-tech industry Traditional industries Finance industry Other Businesses R&D Institutes Others Senior supervisor Engineer Others Vocational school Undergraduate Master’s Degree Ph. D Northern Taiwan Central Taiwan Southern Taiwan

Member 101 37 11 106 18 3 103 5 4 8 12 6 79 34 25 12 113 11 2 132 3 3

Percentage (%) 73 27 8 77 13 2 74 4 3 6 9 4 57 27 16 9 82 8 1 96 2 2

The majority of the 138 learners were male - near triple the number of females. Most participants were in the 30 to 39 age bracket, and most worked as senior supervisors in hi-tech companies. Some have a Master’s degree or a Ph.D. 3.2 The Interactive Learning Effect from Case Studies in a Web-based Learning Community In traditional management classes, teachers and students survey case studies from newspapers, journals, and books. They rarely have the chance to explore a real enterprise’s operation, so they must imagine how the enterprise operates. When the EMBA students collect case studies from their companies, they get to know how a department is administered and are able to apply what they have learnt on the course to their jobs. Meanwhile, they can verify the viewpoints of the management, and improve the effectiveness of learning. 3.3 The Level of Information Technology Integrate into Teaching for Four Teachers In the co-teaching curriculum, Prof. A is responsible for the philosophy of management. He stresses intellectual property rights and considers that unpublished research should not be put on the website. Prof. B teaches financial management, and thinks that preview before class is very important. He uploads the digital handout files online three days before a class for the students to preview. Prof. C lectures on operation management, and hopes to video his class for students who are unable to attend class due to their work commitments. Prof. D lectures on knowledge management. He puts the material on the website before class every week long with his notes.

260

I-F. Liu et al. / Exploring the Learning Effect of a Web-Based Learning Community

Table 2 How the Four Co-teachers Use the Systemȷ s Platform Functions Online Announcements Download the Handouts Turn in Assignments Online View Other Students’ Assignments Multimedia Learning Base Schoolwork Discussion Area Message Board Schoolwork Chat Room Wireless Environment The Frequency The Level

Prof. A ԩ

Prof. B ԩ ԩ

Prof. C ԩ ԩ ԩ ԩ

Prof. D ԩ ԩ ԩ ԩ

ԩ

Rarely Used Level 1

Occasionally Level 2

Often Used Level 3~Level 4

ԩ ԩ ԩ ԩ Highly Used Level 4

4. Conclusion and Suggestion The important findings of this study are: (1) EMBA students use the Web-based learning platform to obtain more knowledge about management and helps develop good social relationships with their peers. (2) By using the system, busy students can extend the study at home. In addition, the system improves interaction among students. (3) All most students use the discussion room a great deal, while others use it infrequently. (4) EMBA students are active in discussion with case study, which they provide are very interesting for professors and other students to know how other enterprises operate and what obstacles they encounter. (5) The level that different teachers integrate information technology into teaching differs greatly. Finally, we propose some suggestions for two aspects of web-based learning community. (1) Web-based Learning Community Aspect z Offer the adaptive learning courses for EMBA students with different backgrounds and expertise. z Provide more information about case study of management online and enrich the website. (2) System Platform Aspect z The system interface should be more user-friendly and more simplified, providing information like FAQ, functions inquiry, and so on. z Strengthen the stability and security of the system. Reference [1] Boczkowsk, P. J. (1999) Mutual Shaping of Users and Technologies in a National Virtual Community, Journal of Communication, 49(2), 86-108. [2] Umiker-Sebeok, J. & Kim G. (1999) The social matrix of information need and behavior: Community, http://www.slis.indiana.edu/umikerse/L503/commun.htm [3] Wu, Q. Y. (1997) Real Community and Virtual Community - Integration, Confliction, and Erosion, http://www.ios.sinica.edu.tw/SEMINAR/infotec2/ [4] Qiu, G. F. (1996) Exploring of Emotional Learning Theory and Computer-supported Learning on Learning Community, Taipei: NTNU Bookstore press. [5] Heckscher, C. & Donnellon, A. M. (1994) The post-bureaucratic organization, London: Sage Publications (Eds.). [6] Stephen, W. H. & Marshall, G. J. (1999) The Five Levels of Web Use in Education: Factors to Consider in Planning Online Courses, Educational Technology, 28-32.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

261

Development of a Discussion Board System Designed for the Group Discussion that Includes Peer-Review Process Shigeru SASAKIa and Hiroyoshi WATANABEa School of Science and Engineering, Teikyo University, Japan {sasaki, hiro}@ics.teikyo-u.ac.jp

a

Abstract: In an advanced programming course, teachers can set tasks that students write unique program codes and reports. We included a peer-review process in the group discussion, in which students grade their friends programs and reports. Students can improve their programs and reports because they can compare and discuss their work with each other in the group discussion. However, because students write the graded points in their message in the peer-review process it has been taking a lot of work for teachers to total the points written in the messages. In this paper, we report a development of the discussion board system with some functions designed specially for the group discussion and its application in an advanced programming course. Keywords: Group discussion, discussion board system, peer-review, rubric

1. Introduction Authors have organized a group discussion including a peer-review process in an advanced programming course using a course management system WebCT [1]. Learners on an advanced programming course write unique program codes as they work on their own interests and preferences. Hence we expect that students will be able to increase their understanding of the subject by presenting programs and reports each other and discuss about them. We have carried out a practice of an exercise course with group discussion [1]. We also provided rubric for a peer-review process to make students evaluate the others’ work objectively [2]. From these practice we found that group discussion will work effectively when teachers can set tasks which make their students write unique outcomes. However, because the discussion function of WebCT doesn't have the grading function, it is necessary to write the evaluation results in the text. It makes students feel a little troublesome in the evaluation work. In addition, teachers could not implement group discussion easily because it takes considerable time to total the evaluation result. In this report, we have developed a discussion board system that have a necessary function for group discussion and have carried out practical classes using it.

2. Outline of the Course The advanced programming course is set for third year students in Information and Computer Science Department. The contents of this course are Windows programming, graphical filters, and 3D computer graphics. Students in this course are assumed to have ordinary programming skills in C++ and understood foundations of object-oriented

262

S. Sasaki and H. Watanabe / Development of a Discussion Board System

programming. We have organized this course based on a self-learning style in which learning materials are provided via WebCT and students can learn at their own pace. We have organized online group discussions using WebCT. Students were able to exchange worthwhile evaluations and comments [1].

3. Outline of the Group Discussion Board System 3.1 How to Execute Group Discussion The discussion groups are composed of 6 students. The members of the groups are changed for each problem, which gives students the opportunities to join various friends. Each student has to write a comment and an advice to all the other members of the group. We conduct a group discussion according to the following procedures. 1. After exercise class, students post their programs and reports to the discussion board until the first closing date. They can attach their programs and reports to the messages. 2. In the next step, students read the programs and reports of the other members and reply evaluations, questions, comments and advice to the message. 3. If they have replied to the all members in the group, they write answers to the messages they have received and develop their programs and reports further. 4. Students send in the final version of their programs and reports to teachers via WebCT until the final closing day. We recommended sending the final programs and reports to the discussion board as well. In the second step, students evaluate programs and reports of the other members into 3 grades. We made rubrics of each problem to clarify the viewpoint and the level of the evaluation so that students are able to make a concrete, objective evaluation. Makino reported the practice of collaborative learning using WebCT [3]. She pointed out that activities of students start from individual learning, shift to collaborative learning and then continue further individual learning. Through the activity including a group discussion, students experience the study phase in which they solve problems individually or with collaborative friends and the other study phase in which they study with the member of the same discussion group. Very active students might go back and forth between these phases. We consider that the collaborative friends and the discussion group chosen by the teachers are close to the Johnson's base group and formal group [4] respectively [1]. 3.2 Specifications of the Discussion Board System for Group Discussion To hold a group discussion presented above, we implemented the functions especially needed besides the function of a usual message board system might be following. Interface for peer-review and statistical function: In our group discussion model, students evaluate fellow students' programs and reports. Because a usual message board system doesn't have the function to grade them, students write the evaluation result in the text. If the points are written in the text, not only the work that students write points in the texts is troublesome, but also it takes teachers an enormous time and effort to total the points as they have to read the points from all messages. Teachers were not able to hold a group discussion easily for that reason. Therefore, we have designed the interface in our discussion board system in which students can choose the points from several options. The system also records students' points in the data base so that teachers are able to total these scores easily.

S. Sasaki and H. Watanabe / Development of a Discussion Board System

263

Grouped interface: Each student has her/his own message board, and s/he starts a new thread attaching her/his program and report. The member of the group is specified in the message board screen. Students can move to the other’s message board by clicking their name. It is useful as the members are changed for each problem. The message boards of the students who do not belong to the same group are displayed at the same time. Message deletion function: In this system, students who posted messages are able to delete the content of the message, though they were not able to delete their messages in the discussion board in WebCT. However, it remains in the record that messages have been posted. Cooperation with WebCT: We are using WebCT as a course management system. We made it possible for students to access the discussion board system from WebCT without passing the logon screen. It has been achieved by acquiring the user ID of the student who is logging in the WebCT from the cookie and by sending it to the message board system.

3.3 The Usage of the Discussion Board System Three kinds (an administrator, teachers, and students) are set as a user of the discussion board system. The administrator user registers and deletes "Teacher" user and "Student" user. The teacher user prepares the forum where the discussion about each problem is held, and selects the students who participates in the discussion from among the students whom the administrator registered. Then, the teacher sets the group for the discussion, and sets the points of view for evaluation. After the discussion ends, the teacher can obtain the total result of the mutual-evaluation. The student users first create new threads in which their programs and reports are attached. Next, they download the program and reports that the members in the same group posted, and then they execute the programs and read the reports. After that, they evaluate and comment on the programs and reports in the reply to the original message to which the program and the report were attached. In this reply, they can use grading facility of the discussion board system.

4. Practice of the Group Discussion that Uses the Discussion Board System We put the exercise class with a group discussion into practice in 2005. There were 49 students in the class. The advanced programming course consists of six 180 minute lessons. Students handed in their programs and reports after making group discussions in the first, third and fifth lesson. In the last lecture, we carried out the survey by asking to fill a questionnaire. First question asked if the discussion board system is user-friendly or not. 76 % students replied that it was "very user-friendly" or "convenient, if anything". It is suggested that students could use the discussion board system conveniently enough. The second question asked which discussion board they like - this system or WebCT discussion board. A lot of students (34%) answered that both are the same and the students who answered that this system was better and the students who answered that the discussion board in WebCT was better was similar ratio. This result also supports the thoughts that although these functions are basically useful, students might have felt inconvenient because they were not necessarily implemented in the best way. In addition, this system is very useful for the teachers. In this discussion board system, the evaluations from the members of the same group to each student's work is totaled and

264

S. Sasaki and H. Watanabe / Development of a Discussion Board System

displayed. Statistical functions can be implemented if necessary as all graded points are recorded in the data base. Because it has been taking a lot of work to total the points written in the messages, this system have made it possible to reduce teacher's time greatly.

5. Discussion We have been able to achieve the easy-to-use interface and the functions for a group discussion which includes peer-review process using this discussion board system. Especially, the grading function was useful for the students to input the points, furthermore was able to decrease the teacher's work to total these points. The discussion function of WebCT has some easy-to-use functions, which our discussion board system does not have. However, we will be able to implement some of them in our discussion board system in the near future. One of them might be the unread message notification function. We will be able to make the discussion board system easy to use by implementing some new functions. Second one will be the function to set threads public or private. Although students can read all messages in this discussion board system, sometimes we need a function which limits the member who can read someone's messages. The one of the main aim of this practice is to reduce teacher's work when holding group discussions. This discussion board system has brought enough effects because it has reduced teacher's work very much without making students' group discussion environments inconvenient.

6. Conclusion We have developed the discussion board system to hold group discussions which includes peer-review process and have took it into practice. For students, there were both merits and demerits in using the discussion board system compared with the discussion function of WebCT. On the other hand, teachers need to spend less time than before to total the points graded in peer-review process in the group discussion after introducing this system. We plan to improve the discussion board system by implementing some functions described in the discussion.

Acknowledgments We thank Takashi Suzuki, who wrote a JavaScript code for getting WebCT user ID from a cookie data.

References [1] Sasaki, S., and Watanabe, H. (2004) Application of group discussion in an advanced programming course using course management system. Proceedings of International Conference on Computers in Education 2004, 1139-1146. [2] Sasaki, S., and Watanabe, H. (2005) Peer-review in an advanced programming course using WebCT. Proceedings of the 3rd WebCT user conference in Japan, 89-93. [3] Makino, Y. (2003), WebCT and Design of Collaborative Learning, Proceedings of the 1st WebCT Japanese Study Group Meeting in Fukuoka, Japan, 53-58 [4] Johnson, D.W., Johnson, R.T., Smith, K.A. (1991). Active Learning: Cooperation in the College Classroom, Interaction Book Co.

Interface

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

267

Improving Creativity for Mathematical Problem Solving Using Web-based Multimedia Whiteboard System Hwang Wu-Yuin*, Chen Nian-Shing**, Dung Jian-Jie*, Yang Yi-Lun*** * National Central University, Taiwan,* * National Sun Yat-Sen University, Taiwan * * * National

Kaohsiung Normal University, Taiwan

Abstract: The aim of this study is to explore the relationship between representation and creativity, and their influence on mathematical problem solving. As mathematical problem solving needs complex cognitive abilities and representation translations, there is a question that whether using a multimedia whiteboard system can better facilitate students to learn this type of mathematical problem solving. The result has found that students can perform well in mathematical problem solving if they can better utilize the tools supported by a multimedia whiteboard system to do multiple representations for their solutions. Our creativity analysis also found that students who can do multiple representations well for their solutions and criticisms would have higher elaboration ability in the creativity test. The finding concludes that multiple representation skills are strongly related to student’s elaboration ability and are the key to mathematical problem solving. Key words: Representation, Creativity, Mathematical Problem Solving, Multimedia Whiteboard System

Introduction Some researches had pointed out that student’s good representation skills derived from problem solving is the key to acquire a successful solution (Gagne, 1985; Mayer, 1992). While solving math problems, Lesh (1987) proposed the three steps procedure for problem solving. The first step is translation of verbal or vocal to mathematical pattern, the second step is solution of mathematical pattern to arithmetic symbol. The final step is explanation of the solution by verbal writing or oral speaking. So, Lesh emphasized the importance of students’ transformation ability among multiple representations while doing application problem solving. According to the issues stated above, this study is to explore how primary school students use multiple representation skills including text, graph, symbol, rule, and formula in mathematical problem solving processes. We also want to examine the relationship between students’ multiple representations and creativities, and whether they are related to mathematical problem solving achievement.

268

W.-Y. Hwang et al. / Improving Creativity for Mathematical Problem Solving

1. Literature Review 1.1 Representation The definition of representation can be something stands for something else. For each one, there are external representations (real world) and internal representations (mind). Lesh etc. (1987αhad pointed out five outer representations used in mathematics education, including real world object representation, concrete representation, arithmetic symbol representation, spoken-language representation, and picture or graphic representation. Among them, the last three are more abstract and higher level representations for mathematical problem solving.(Johnson, 1998; Lesh, 1987) Some students favor visual or concrete representations, while others favor symbolic or abstract representations. Students with good solving abilities often are those who can skillfully manipulate their translation and representation of language representation (verb or sound), picture representation (picture, graphic) and formal representation (sentence, phrase, rule and formula). In the study (Hwang et. al., 2006), one multimedia whiteboard system was proposed to provide useful functionalities, by which the students can express their thought with different representations like text, image or voice. It was also found that some representations are strongly related to the achievement of math problem solving in that study. 1.2 Creativity Creativity means the cognitive skill of proposing a solution of a problem or making something useful or novel from ordinary. Guilford (1969) cited the paucity of creativity and imagination among all students, and he encouraged the study of creativity. Because creativity is the highest level of mental presentations for human beings, it has become the focus of psychologists and teachers. Williams (1971) has proposed the definition of creativity based on Guilford’s multiple intelligence theory (1969) and emphasized the importance of creativity on learning. According to Williams’s definition of creativity, it is composed of six cognitive factors, fluency, openness, flexibility, originality, elaboration and title as well as four affective factors, curiosity, imagination, challenge, and risk-taking. To promote students’ creativity through mathematics learning, the quality of the solution for a problem should not only be a formula or result, of which the answer only one. We should encourage students to express and explain their solutions in detail such that students can better create their own knowledge. Hence, the assessment method cannot just be either right or wrong to the result; more judgment should be put on the description of the solution (representation). 1.3 QCAI Evaluation Silver(1989) conducted a project called QUASAR (Quantitative UnderstandingΚ Amplifying Student Achievement and Reasoning). Its attempt is to promote different learning methods to cultivate students’ problem solving and critical thinking capabilities such that all levels of students can learn math well. Under this premise, teachers need to

W.-Y. Hwang et al. / Improving Creativity for Mathematical Problem Solving

269

adopt multiple materials and instructional strategies for encouraging students to do discussion, interpretation and innovation. In this way, students will be trained to thinking math instead of memorizing math. Several evaluation instruments were developed in QUASAR projects. The most famous and prevalent one is QUASAR Cognitive Assessment Instrument, which is also called QCAI with many open-ended questions. QCAI can evaluate students’ mathematical problem solving, reasoning, communication and representation etc. The specification of QCAI includes 4 components: mathematical contents, cognitive processes, mode of representations (text, picture, graph, table), and task context (Lane, et al, 1995; Lane,1999) In creative mathematics problem solving, the teachers and researchers want to know more than mathematical knowledge of the students, such as all the procedure and strategic knowledge they acquire. In this study, we focused on the types of representation skills (text, picture, graph, and table) in QCAI and employ the 5 levels scoring rubric to evaluate multiple representation skills of math problem solutions as shown in part A of Table 1. 2. Research Method 2.1 Setting & Procedure The subjects are 25 six-grade primary school students who were tested and selected as excellent students in mathematics in the school. They participated in mathematical problem solving learning activity using multimedia whiteboard system (Hwang etc., 2006). Its functionalities include drawing tools, voice recording tools and editing functions. Students can write down and modify their own mathematical calculation processes and related description about their solutions. The drawing tools include line, circle, rectangle and text; the editing functions include copy, paste, cut, move, undo and redo. There were 21 problems assigned to each student, including 16 numeric and 5 geometric problems. The period for conducting the experiment was last for one semester long (about 4 months). The students participated two math class sessions for a total of 80 minutes every week. 2.2 Evaluation Criteria During solving activity period, students can revise their solutions many times. Our research concerns not only about how many kinds of solutions (quantity grade) can students create but also how well their solutions are (quality grade). The students’ solutions are classified and evaluated into three types of representations: Text or Voice Representation, presented as ‘T’, Graph or Symbol Representation, presented as ‘G’, Rule or Formula Representation, presented as ‘R’. Each representation is marked with a quantity grade and a quality grade respectively. For quantity grade, the assessment is based on how many solutions are provided; two teachers reviewed students’ solutions and came out with a consensus grade for every individual student after discussion. The criteria is developed by the researchers, based on QCAI evaluation concept, performance is ranked into 5 categories (Level 1 ~ Level 5),

270

W.-Y. Hwang et al. / Improving Creativity for Mathematical Problem Solving

shown in part A of Table 1. The students were asked not only to solve problems but also to criticize two of other students’ solutions. Criticism needs higher level of cognitive ability. The criticism performance is not solely evaluated on its own, it is corresponding to the evaluated solution performance. And then the weighted criticism grade can be obtained by adding up the solution grade of the student being criticized according to part A of Table 1 and the original criticism grade obtained according to part B of Table 1 (Weighted Criticism Grade = Criticism Grade + Solution Grades of the Student being Criticized). Table 1 QCAI Evaluation Criteria

3. Result & Discussion 3.1 Solution Type Analysis In the analysis of the students’ representation performance, R (Rule or formula) is better than T (Text or Voice) and G (Graph or Symbol) on both quantity and quality scores; This result is matched with one teacher’s observation from the student’s solving process; she made the following comment: “During the solving activity, most students don’t have ideas about how to creatively solve the problem, but simply apply their remembered formula to get a answer.”

One student also said that it was not easy for him to use verbal, text, or graph to explain the meaning of his solutions. “Arithmetic series problem is very easy. But it is difficult for me to further elaborate in oral explanation and to write down texture description for my solutions.”

3.2 Representation Skill Analysis Students’ representation performance, both in quality and quantity were recorded and analyzed respectively. One-way ANOVA analysis and Scheffe’s test were used to compare the scores of representations T, G, and R in numeral or geometry as well as in numeral with geometry. We got the following four different types according to students’ representation performance. 1) Representation T is significantly lower than G and R, denoted as type

271

W.-Y. Hwang et al. / Improving Creativity for Mathematical Problem Solving

2) Representation G is significantly lower than T and R, denoted as type

.

3) Both representations T and G are significantly lower than R, denoted as type 4) There is no significant difference among three representations, denoted as type ”/”. The result shows that most students belong to type students are of type

or type

or type ”/”; while only a few

. All student types are shown in Table 2.

Student types are , , or were favored of using one or two representations in problem solving or criticism. We called these three type students as “Unbalanced Style” in our study. On the other hand, the students who used all three representations fairly are called “Balanced Style”. Table 2 Student’s Representation Types

3.3 Performance Analysis of Different Representation Styles Groups To properly investigate students’ favorite representation styles, both quality and quantity performance should be taken into account, no matter in numeric, geometric or problem solving performances. We classified the 13 students (ID 01, 02, 08, 09, 10, 11, 13, 14, 15, 16, 17, 23, 24) as the Representation Unbalanced Group and the other 12 students (ID 03Ε04Ε05Ε06Ε07Ε12Ε18Ε19Ε20Ε21Ε22Ε25) classified as the Representation Balanced Group. Independent sample T test has been done for the two groups to evaluate the creativity, problem solving, weighted criticism, and academic achievement. The results are shown in Table 3. Table 3 T test of Representation Styles between Balanced and Unbalanced Groups M CreativityElaboration Problem SolvingTotal-Quantity Total-Quality Number-Quantity Representation T Representation G Representation R Number-Quality Representation T

Balanced Group (N=12) SD

M

Unbalanced Group (N=13) SD

t value

27.25

7.94

19.46

8.31

2.39 *

74.75 286.25 24.13 7.63 8.03 8.42 88.54 27.46

5.99 46.21 2.21 1.23 .73 .82 15.37 6.57

62.69 225.00 21.89 8.00 5.31 8.62 73.89 24.89

9.02 43.31 3.72 1.24 2.05 .96 16.80 6.15

3.90 ** 3.42 * 1.81 -.76 4.58 *** -.55 2.27 * 1.01

272

W.-Y. Hwang et al. / Improving Creativity for Mathematical Problem Solving

Representation G Representation R Geometry-Quantity Representation T Representation G Representation R Geometry-Quality Representation T Representation G Representation R Weighted Criticism Academic AchievementPre-test Post-test (* p < .05, ** p < .01, *** p < .001)

28.54 32.25 13.25 4.33 4.13 4.71 54.58 17.13 16.00 21.54 125.86

5.31 4.34 1.90 1.07 1.09 .62 12.34 5.79 5.50 3.99 11.30

17.58 31.42 9.46 2.00 2.31 5.00 38.62 7.12 8.92 22.58 114.70

8.91 4.57 2.16 1.29 1.38 .00 10.09 4.86 5.61 2.90 8.76

3.77 *** .46 4.63 *** 4.89 *** 3.64 *** -1.63 3.55 ** 4.69 *** 3.18 ** -.75 2.39 *

95.31 95.63

3.35 3.08

95.58 94.39

2.39 3.36

-0.22 0.97

The Balanced Group students performed better than the Unbalanced Group in the Elaboration item of Williams’s creativity package (t=2.39, p < .05). For the problem solving solutions analysis, the Balanced Group students performed significantly better than the Unbalanced Group students in Total–Quantity (t=3.9, p < .01) and Total-Quality (t=3.42, p < .05) scores. The representation T and G are the critical factors that caused different performance. For the Number problem section, the Balanced Group have higher representation G scores than those of the Unbalanced Group both in quantity (t = 4.58, p < .001) and quality (t=3.77, p < .001). Moreover, for the Geometry problem section, the Balanced Group also have higher representation G scores than those of the Unbalanced Group both in quantity (t=3.64, p < .001) and quality (t =3.18, p <.01); furthermore, the scores of representation T are having the same situation between two groups both in quantity (t=4.89, p <.001) and quality (t= 4.69, p <.001). As for weighted criticism scores, the performance of the Balanced Group are better than the Unbalanced Group (t=2.39, p < .05). In summary, the manipulation and representation skills of T and G are the key for students in acquiring higher performance, no matter in Number or Geometry problem solving and peer assessment. As for representation R, there is no difference between the two groups. This situation can be explained by the teacher’s observation that most students in the Unbalanced Group often applied their pre-remembered formulas to solve problems without thoroughly comprehending the solving problems. Those students are used to memorizing mathematical formulas and employing the formulas directly to solve given mathematical problems; In this case, students are more tentative not to brainstorm and think deeply and broadly. Thus these students would have little chances to find out good solutions in creative problem solving. As for no significant difference in the academic achievement between the two groups, one possible reason is the 25 subjects participated in our research are all math-talented students; they always get very high marks. This can be supported by the average grade is close to 95, and this phenomenon is called the ceiling effect.

W.-Y. Hwang et al. / Improving Creativity for Mathematical Problem Solving

273

3.4 Correlation Analysis of Representation and Elaboration Since the two groups of students have significantly different performances on representation T and G in multimedia whiteboards as well as on elaboration item of creativity, the relationship between representation and creativity in Number, Geometry and Total scores should be further investigated. Thus, the Pearson correlation analysis was conducted. The results showed that elaboration in creativity is significantly correlated with representation T, and G in several parts of problem solving using multimedia whiteboards. In Number problem section, the quality scores of both T (P=.479*) and G (P=.406*) are correlated to elaboration. Besides, the total scores of representation T is also significantly correlated with elaboration in quantity (P=.437*) and quality (P=.472*); (* stands for p < .05). The meaning of elaboration is the ability to work out with extra material, illustration or clarifying in detail. Therefore, the role of elaboration in solving and criticizing activities is the ability that students can use to express and elaborate their own solutions or to criticize others’ solutions by using all kinds of possible means and perspectives. So, elaboration is the critical factor affecting student’s ability on using representation of T and G in creative problem solving. So it is really worthwhile to stimulate or cultivate student’s elaboration ability by using different representations to improve creative math problem solving 3.5 Problem Solving & Solution Criticizing Analysis Using multimedia whiteboard system, students can modify and improve their solutions as many times as they wish after they got some feedbacks from teachers or other students. As shown in the left-hand side of Figure 1, one student solved an arithmetic series problem twice using the multimedia whiteboard system, the teacher’s comments are shown to the right. Using the Multimedia Whiteboard can also help teachers to access what kind of mistakes students will make. The teachers have found several types of mistake which students often made in numeric and geometric problem solving. By analyzing the mistaken solutions, the teachers can exactly access where the students made mistake and what caused students’ misconception. umeric Problem 1: lease calculate the sum of the given arithmetic series problem.

Teacher’s comments on the two solutions for the same Student. You solved this problem for the first time and the answer is wrong. I think you just applied a straight forward thinking to solve this problem without any description and explanation. Please rethink and try again.

274

W.-Y. Hwang et al. / Improving Creativity for Mathematical Problem Solving

After the student participated in peer to peer interaction using the multimedia whiteboard system, he had learnt the know-how of this problem. He then solved the problem again on 2004/10/14, and the answer is correct with proper description and explanation for his solution. Note: There is a typo error in the equation (77-11) - 3 = 22, the – 3 should be ÷3.

Figure 1 Problem Solving & Refinement using Multimedia Whiteboard System 4. Conclusion & Suggestion This study explored how a Multimedia Whiteboard System facilitated students to learn mathematical problem solving. The students were classified into the representation Balanced group and the representation Unbalanced group according to their presented solutions. Both groups of students used representation R very well in problem solving and criticism. However, the key to achieve a higher performance in mathematical problem solving is the ability of coordinating representations of T and G; that is good students should have the ability to manipulate text, voice, graphic, or symbol all together in problem solving. We found that the Balanced Group students performed better than the Unbalanced Group students on representation T and G in problem solving, criticism activities and elaboration. References [1]Hwang W.Y., Chen N.S. and Hsu J. L.ΰ2006α “Development and Evaluation of Multimedia Whiteboard System for Improving Mathematical Problem Solving”, Computers & Education,43.. [2]Johnson, S. (1998). What’s in a representation, why do we care, and what does it mean? Examining evidence from psychology. Automation in Construction, 8. [3]Gagne, E.D. (1985). The Cognitive Psychology of School Learning. Boston: Little, Brown and company. [4]Lane, S. et al.(1995).Examination of the assumptions and properties of the graded : Item Response Model: An example of using a Mathematics performance assessment. Applied Measurement In Education, 8(4). [5]Lane ,S.(1999). Validity evidence for assessment. Reidy Interactive Lecture Series. October 14-15,Providence,RI. [6]Lesh, R., Post, T., & Behr, WI. (1987). Representations and translations among representations in mathematics learning and problem solving. In C. Janvier (Ed.), Problems of Representation in the Teaching and Learning of Mathematics.. Hillsdale, NJ: Erlbaum. [7]Mayer, R. E. (1992).Thinking , Problem Solving, Cognition. New York: W. H. Freeman and Company. [8]Parker, J. P. (1978). We all have Problems...who doesn't ? But can they all be solved. Gifted Child Today. March/April, 61-63. [9]Silver, E. A. (1989). "QUASAR." The Ford Foundation Letter 20: 1–3.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

275

Developing a VR-based Projectile System using Haptic Device for Learning Physics Atsushi Kanbea, Yukihiro Matsubaraa, Noriyuki Iwanea, Kimiko Hirayamaa a Faculty of Information Sciences, Hiroshima City University, Japan [email protected] Abstract: The learning support system with experience was developed in this study. Then, force-feedback device was introduced to the system to link bodily movement to the action in the virtual laboratory. We evaluated efficacy of using the force-feedback device and the way to hold the device, and the student could naturally feel the action in the virtual laboratory, and feeling of immersion, realistic feeling and operational feeling were increased. So, the student could effectively learn. Keywords: Virtual reality, Force-feedback device, Learning support system, Bodily movement

Introduction The method of discovery learning is characterized in that “students learn actively, and understand and acquire knowledge with experience.” There is a need for discovery learning with experience especially in school education. Virtual physics experimental environment with 3D microworld and learning support system simulating solid figures using virtual reality technologies have ever been suggested[1][2]. In these studies, learning systems with a high regard for experience that students can acquire new knowledge actively were developed. These systems are mainly operated by the mouse action and rely on information obtained from visual sensation. However, important elements of human’s ability that he/she visualizes objects, memorizes and constructs as cognizable models are triggered by the touch of them. The action of touch is an interactive real-time information exchange, so we expect that student’s understanding is enhanced through the act of touching objects directly[3]. In the previous study, Ogawa et al. proposed a system for learning electromagnetics using a haptic device[4]. In this study, we aim to develop the system that students can experience about upthrowing vertically using a force-feedback device. The system shows traces of a ball and position of it at a certain time using virtual reality technology. Additionally, students can control in a part of questions, and can feel reaction force which applies for a device more real. We develop a learning support system that is deepened students’ understanding through one’s experience and captures the students’ interest, and students can voluntarily and actively learn.

1. Discovery Learning Focused on Experience and Force-feedback Device Bruner, advocate of the discovery learning, cited four things about effectivity of discovery learning, “shift from the extrinsic reward to the intrinsic reward, enlargement of intellectual ability, helping for memory and learning of heuristic technique.” In fact, students increase

276

A. Kanbe et al. / Developing a VR-Based Projectile System Using Haptic Device

desire to learn, acquire knowledge quickly and memorize deeply, by discovery learning. Students also acquire skills that have wide application for other scenes[5]. The goals of discovery learning include “1) student acquires new knowledge actively, 2) student applies and validates already acquired knowledge and deepens understanding, 3) student learns with experience especially[1].” In particular, we focused on 3) in this study. Students not only are taught by teachers but also can be provided such reality as the thing in the real world in the study with an experience especially. In addition, student certainly thinks relative to his/her experience, so the study with experience provides a trigger of thinking to student. In the learning with experiences especially, the students go ahead with the study with experiencing themselves really, it must have direct-manipulability, action consistency and others. However, existing systems is mainly operated by the mouse action, so we can’t say that they has quality of direct-manipulability and action consistency. And so, we try to realize direct-manipulability and action consistency by the introduction of a force-feedback device. Force-feedback device provides feeling that the student senses when the student touches the virtual object and reaction force of the virtual object. The human computer interaction method is commonly only seeing and hearing, but the device can add “touch.” Our system uses a force-feedback device, PHANToM. This device has six degrees of freedom, and a user can manipulate freely. When the student moves virtual object, its manipulation is done in two dimensional space by a mouse, but using a force-feedback device, it is done in three dimensional space. The student can directly manipulate the virtual object by the PHANToM, so the student easily links motion in our system to body action. The student can also feels the weight of the virtual object, so the amount of information that is thought by the student increases and the quality of study improves. In the previous study, Inoue et al. developed the learning support system that students can experience physics experiments about the pulley[6]. The system used a force-feedback device to manipulate dragging down a thread hung over a fixed-pulley. While on the other hand, our system uses it to manipulate throwing up a ball. The student’s body motion during the experiment also changes by the different domain as shown in Figure 1. The student catches hold of a ball with a hand in our system. We expect that changing how to hold a force-feedback device and how to manipulate help the improvement of reality. In this study, we try to devise ways of manipulating a force-feedback device and ways of generating reaction force, and bring the virtual experiment close to actual action. The student can feel the experience in the virtual laboratory more naturally if it is closer to actual action, and it will lead to increasing “feel of immersion, feel of presence and feel of manipulation” that are characteristic of virtual reality.

2. Subject of Physics and Learning Support System 2.1 Target Domain This study deals with upward projectile in elementary dynamics as a target domain. Figure 2 shows basic formulas of upward projectile. We build the system that the student can observe the vertical motion of the object that the student upthrows straight, and can learn meaning of formulas shown as Figure 2 and potential energy.

2.2 Building of Virtual Laboratory Our system consists of the intelligent tutoring module, the CAI module, the input and output

A. Kanbe et al. / Developing a VR-Based Projectile System Using Haptic Device

277

Figure 1 Relating to body motion in learning

Figure 2 Upward projectile and basic formula interface, and the virtual laboratory as shown in Figure 3. In particular, we focus on interface and virtual laboratory in this study.

2.2.1 Learning environment building module This module builds experimental environment based on input signal of the mouse and keyboard, and information about initial speed calculated in control module of VR environment and motion of a body.

2.2.2 Control module of VR environment This module controls the virtual laboratory. It calculates initial speed to set to learning environment building module, the motion formula and information about reaction force to set to PHANToM controller. It has three components, those are described below. (1) the initial speed model It calculates initial speed of the virtual object. It receives position information and

278

A. Kanbe et al. / Developing a VR-Based Projectile System Using Haptic Device

Figure 3 System configuration calculates the initial speed. Initial speed when a ball upsprings is decided for one second before the student releases the stylus button of PHANToM. (2) motion model of the object This module calculates motion of the virtual object. It receives initial speed calculated by initial speed model and calculates motion of the object from the basic formula of falling motion of object. (3) model of resistance force of projection and impactive force of receiving This module calculates the reaction force applying for PHANToM based on the object’s motion model of the object. The models of generating reaction force include the model of resistance force of projection and the model of impactive force of receiving.

2.2.3 PHANToM controller This module controls a force-feedback device, PHANToM. It obtains position data of PHANToM when the student upthrows a ball and provides reaction force to PHANToM based on the data received from control module of VR environment. These enable the student to feel reaction force of the virtual object. Two components in this module are described below. (1) sensing element of the position data This element acquires position data of the PHANToM’s stylus. The position data of the stylus is provided to control module of VR environment. (2) element of control of reaction force This element controls reaction force applied to PHANToM. It receives the data of reaction force calculated by control module of VR environment and applies the reaction force to PHANToM.

2.3 Function of Systems and Learning Process Our system has realized the act of upthrowing and receiving a ball by building virtual laboratory and using force-feedback device, PHANToM. Pushing stylus button of PHANToM corresponds to grabbing a ball, and releasing the button corresponds to unleashing a ball. Receiving spontaneously occurs. The student can experience then

A. Kanbe et al. / Developing a VR-Based Projectile System Using Haptic Device

279

impactive force by lightly holding the stylus. A screenshot in Figure 4 shows functions of this system.

locus indication button radio buttons for mass selection

the button of the position indicating after specified seconds

the button of the highest point / fall

reset button

Figure 4 Example of the screenshot Here is an example of the learning scenario using “locus indication” in this system. Student can learn that downward force (gravity acceleration) applies to a ball going upward, Basic formula: v v0 gt ······(1) by the function of locus indication in this system. The learning process using a system is showed in Figure 5. The student decides the initial speed and pushes the locus indication button. And, the student looks the locus appeared on the screen and considers the force applied to a ball. The locus of a ball is indicated at the same interval of time, but the space between locus points gets closer at the highest point. So, the student can understand that downward force always applies to a ball, in other words, one aspect of formula (1). If initial speed is too slow, then there is little space of locus, and the student can’t understand that the space is small. In such case, the student repeats deciding initial speed and displaying the locus, then the student considers the force applied to a ball. If the student can understand that gravity acceleration applies to a ball, then the student finishes learning.

3. Evaluation and Discussion 3.1 Evaluation I 3.1.1 Objective and Procedure of the Experiment We investigated efficacy of introducing a force-feedback device for the learning with experience and feeling reaction force. Eight university students participated in experiments. All participants performed the tasks of upthrowing a ball using the system for the upthrowing task (Figure 6). In this system, height of two bars on the screen changed at random, and if the participant could upthrow a ball between bars, the task was accomplished. There were two systems used in experiments, one was the system which reaction force applied to a device, the other was the system which the force didn’t apply. Participants performed the task using each system once, and task success rate was investiga-

280

A. Kanbe et al. / Developing a VR-Based Projectile System Using Haptic Device Prerequisite

Start

If the force isn't applied, motion and condition of the object doesn't change.

Thought of student

Decision of initial speed

Manipulation

What can I understand using locus indication? Decide initial speed How does the locus form?

Locus indication

Push a locus indication button Verify the locus understand formula?

No

The locus of a ball gradually become narrower going to the top. The locus of a ball is indicated at the same interval of time. The downward force is applied for a ball in proportion to time.

Yes

End

Figure 5 Learning process of gravity acceleration for a ball ted. Half of the participants used the system with reaction force and the system without reaction force, in that order, and the others use in reverse order to lose advantage from sequence.

Figure 6 System for upthrowing task

3.1.2 Results and Discussion Figure 7 shows the results of a questionnaire answered after the experiment. The questionnaire consisted of four questions, “1) Are you interested in this system?”, “2) Do you feel upthrowing act naturally?”, “3) Is this system realistic?” and “4) Can you easily manipulate this system?” These questions were answered from 1 to 5. Larger number meant that participants’ answers fit the questions better. The system with reaction force gained higher marks in question 1) to 3) shown as Figure 7. Additionally, we asked to fill out an impression freely, and some comments about high enjoyment and high manipulability were gotten. Thus, students can be interested and quality of learning can be said to be enhanced by using the system with reaction force and providing tactile sense. Furthermore, the result that the system with reaction force had reality was obtained, so applying reaction force is thought that it led to increasing feel of immersion. Meanwhile marks of the system with reaction force were almost equal to one of the system without it in question 4) about manipulability, there is not all that much difference between the two. And now, we discuss task success rate. Figure 8 shows the difference of success rate between the system with reaction force and the system without it. We found that there was difference of success rate that depended on the absence or presence of reaction force. From here onwards, the system with reaction force can be said to tend to be able to manipulate easily. Hence it’s expected that the information which students can acquire increases by feeling reaction force, so

A. Kanbe et al. / Developing a VR-Based Projectile System Using Haptic Device

281

quality of learning also improves. 4

with reaction force

3

without reaction force

2 1 0 11)

2) 2

3) 3

44)

Question number

Figure 7 Results of questionnaire

success rate[%]

5

40 35 30 25 20 15 10 5 0 with reaction force

without reaction force

Figure 8 Difference of success rate between with and without reaction force

3.2 Evaluation II 3.2.1 Objective and Procedure of the Experiment We investigated task performance of upthrowing and ease of upthrowing each of how to hold PHANToM. Twelve university students participated in experiments. Participants were divided into two groups with six, P(Pencil)H(Hand) group and HP group. All participants performed two tasks the same as experiment I, and task success rate was investigated. The method of holding PHANToM was specified as experimental condition. PH group held PHANToM like a pencil (Figure 9, A) in first condition, and held PHANToM from below by participant’s palm (Figure 9, B) in second condition. HP group performed changing procedure first and second condition.

3.2.2 Results and Discussion The results are shown in Figure 10. This figure shows alteration of each participants’ scores. Solid lines mean the results of PH group and dotted lines mean the results of HP group. When we asked about which method can they upthrow easier, all members of PH group gave holding PHANToM from below by participant’s palm as the answer, 4 members of HP group gave the same as PH group member, and 2 members of HP group gave holding PHANToM like pencil as the answer. From here onwards, it’s determined that half of members of HP group feel that holding from below by their palm is easier to upthrow than holding like writing in pencil. In addition, success rate of the task also is higher when they hold from below by their palm. Therefore, it’s considered that to relate holding PHANToM to body motion causes the difference. Hence the system that can be accomplished higher success rate of task is easier to use and to experiment, so it can be expected that students can experiment effectively and learning efficiency increases.

Conclusion and Future Works Discovery learning helps spontaneous motivation and proactive thinking, and have efficacy for increasing willingness to learn and retention of knowledge. This study focused on learning with experience in discovery learning. Learning with experience needs direct-manipulability and action consistency, but traditional mouse action wasn’t obviously enough to learn. And so, force-feedback device was introduced and allowed to manipulate directly. Furthermore, the consistency between virtual experiment and body motion was considered, so enhancing feeling of immersion was tried. We aimed at the developing the

282

A. Kanbe et al. / Developing a VR-Based Projectile System Using Haptic Device

60

1 2 3 4 5 6 7 8 9 10 11 12

success rate[%]

50

A: hold PHANToM like pencils

B: hold PHANToM from below by participant’s palm

Figure 9 Ways to hold force-feedback device

40 30 20 10 0 first first

second second

Figure 10 Score of participants

learning support system about upward projectile using force-feedback device, and proposed a virtual laboratory. Efficacy of the developed system was verified by evaluation experiments. Through the experiments, we found that our system was able to motivate the students by discovery learning with experience especially. Students can intuitively and directly manipulate by introducing force-feedback device in the environment of discovery learning. Additionally, students can experience reaction force of the virtual object, so the action in virtual laboratory can be had reality, and it’s expected that quality of learning are improved. Moreover, students could feel the virtual experiment more naturally by relating the manipulation method of force-feedback device to body motion, and it linked to enhancing “feel of immersion, feel of presence and feel of manipulation” that are accentuated in virtual reality. Application range of force-feedback device was also expanded in aspects of learning domain and the device’s usage. Future works are conducting evaluation experiments in learning support, more naturally upthrowing, development of virtual laboratory of difference domain. Furthermore, we try to apply for challenged students. We will consider integration of education and rehabilitation referring to the previous study of rehabilitation using PHANToM[7]. This research has been made possible through Grant-in-aid for Scientific Research C (No.18500721) from the Ministry of Education, Culture, Sports, Science and technology of Japan.

References [1] Y. Matsubara., H. Tominaga, Z. Furukawa, T. Yamasaki and M. Nagamachi (2000) Development of Virtual Learning Environment for Discovery Learning in School Education. The IEICE Transactions on Information and Systems, 83-D-I, 10, 1109-1119 (in Japanese). [2] M. Nozue and K. Hayashibe (2004) Development of Support System for Learning Solid Figures in Mathematics Using Virtual Reality. Proceedings of the 9th Virtual Reality Society of Japan Annual Conference, 271-274 (in Japanese). [3] T.H. Massie and J.K. Salisbury (1994) The PHANToM Haptic Interface: A Device for Probing Virtual Objects. Proceedings of the ASME Winter Annual Meeting, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Vol.DSC-55-1, 295-300. [4] T. Ogawa, D. Mori, S. Suganuma, K. Murakami and T. Harada (2005) Experience-based Learning Support Environment using Haptic Device. Proceedings of the 43th Workshop of Advanced Learning Science and Technology, SIG-ALST-A403, 25-30 (in Japanese). [5] Bruner, J. S. (1961) The Act of Discovery. Harvard Educational Review, 31, 21-32. [6] M. Inoue, Y. Matsubara, N. Iwane, M. Nakamura and M. Ichitsubo (2005) VR-Based Dynamics Learning System Using Haptic Device and its Evaluation. Proceedings of the 5th IEEE International Conference on Advanced Learning Technologies, 917-921. [7] J. Broeren, M. Rydmark and K.S. Sunnerhagen (2004) Virtual Reality and Haptics as a Training Device for Movement Rehabilitation After Stroke: A single-Case Study. Archives of Physical Medicine and Rehabilitation, 85, 8, 1247-1250.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

283

Proposal for Digital Partners Project Chris Davies a, Jingjing Zhang b Department of Educational Studies, University of Oxford, UK b Department of Educational Studies, University of Oxford, UK [email protected] a

Abstract: In this paper, we describe the rationale behind an innovative interdisciplinary research project that brings together social science and computer science methods to develop and test a Digital Partner computer interface designed to support the learning and self-esteem of adults normally excluded from formal and lifelong learning opportunities. Keywords: Lifelong learning, informal learning, self esteem, online communities, computer partners.

Introduction As many as one third of the UK adult population do not participate in any formal education or training activities throughout their adult lives. Given the fact that formal education provision does not seem to be able to meet the challenges of the learning age for all learners, there is a need to explore how people from all backgrounds can be enabled to experience education as a lifelong enterprise through extended opportunities for informal learning. In the late 1990s a great deal of hope was pinned on the contribution that Information and Communication Technologies (ICTs) might make to the achievement of the ‘le@rning society’(Selwyn, Gorard and Furlong 2006). ICTs, it was argued, could help widen access and participation in education (DfES 2002), lead to an increase in the diversity of provision (Shuklina 2001) and lead to ‘better’ forms and outcomes of learning, including the learning of those with low educational skill. Yet despite these high hopes, the reality of increased participation has remained elusive. As Selwyn et al. conclude from their examination of the impact of ICTs on adult learning, digital technologies have not ‘broken with the old’, as many had hoped; participation remains profoundly shaped by what goes before it in terms of educational experiences. They therefore conclude that ‘much must change if the realities of adult learning in the digital age are ever to near its rhetorical promise’ (188). What must change, they suggest, is the development of more ‘bottom up’ and communitybased initiatives that respect and build on people’s existing skills and interests rather than attempting to direct them into formal learning and certification routes. It is only by developing initiatives of this sort that ‘hard to reach’ sections of the population can, through ICTs, potentially be drawn into learning activities with all the associated benefits of increased skills and greater self esteem. In our research, we aim to add substantially to current knowledge about how digital technologies can specifically support adults on the peripheries of the learning society, and who have not benefited from mainstream methods of education. In order to address this issue, we propose to develop an innovative concept of digital partners to support the informal learning of adults. We are carrying out a major interdisciplinary research project in order to design, implement and evaluate these digital partners, bringing the disciplinary perspectives of Social Science together with Computer Science.

284

C. Davies and J. Zhang / Proposal for Digital Partners Project

1. Research Methods The overarching question for the Digital Partners (DP) Project is: In what ways does the Digital Partner enable users both individually, and collaboratively in online communities, to make progress in learning and in their self-concepts as learners? Given the substantial laboratory-based development which the DP will undergo prior to implementation, and the experimental aspect of the field study, this research can be characterised as a design experiment. Utilising such an approach is particularly appropriate for an interdisciplinary research project aiming both to develop a new theoretically informed technological innovation and to evaluate when and how it works in real life contexts. Such work will not only lead to an end product – the Digital Partner – of value to particular individuals in specific contexts, but will also contribute to our theoretical understanding of informal learning. Indeed, the most fruitful design based work occurs in studies such as the one proposed here which encompasses a longitudinal element, where strong collaborative partnerships with the participants are prioritized. This project aims to contribute to the current debate about the validity of design experiments (Gorard et al. 2004) by testing this approach in a rigorous and innovative context of mixed method research. The design incorporates a large-scale national survey (pre- and post-test), a true experimental field study with random assignment to intervention and control groups, a strong qualitative component, and extensive use of online research methods. 2. Conception and Context of Digital Partners ( or Scenario) The distinctive focus of this research will be on the development, implementation and evaluation of an advanced technology platform designed specifically to support the informal learning of adults outside formal education settings. This will include both independent interactions with technology, and participation in a networked model of learning support where participants will collaborate in group interactions relevant to their own interests. The project will work with a wide range of adults of relatively low formal educational achievement and/or experience of adult learning, belonging to groups such as black and minority ethnic communities, people 50+, disaffected young people, isolated women, and people on low incomes. The technology platform will be a highly interactive and engaging personalised computer partner (the Digital Partner) designed specifically to help individuals to extend and capitalise upon skills and practices involved in their everyday uses of PCs and similar technologies (including mobile phones), encompassing information gathering, gaming, shopping and money management, job seeking, writing blogs & family histories, following sport, downloading music & movies, and social communications. Many possible learning gains are latent in these normal kinds of computer use, including the acquisition of literacy, numeracy, graphical skills, real world knowledge and technology skills, and productive participation in social networks. The potential for such naturally occurring skills development will be enhanced by the DP’s capacity to store and remind users of earlier achievements and of the processes by which these were achieved, thus encouraging users to strengthen and synthesise their computer-mediated skills. Over

C. Davies and J. Zhang / Proposal for Digital Partners Project

285

time, this process of DP-supported skill development will also offer potential for improvement in users’ self efficacy beliefs, both as social beings and as learners, and also in some cases improvements to personal circumstances by finding job opportunities or joining valuable social networks. One of the key challenges for this research will be the achievement of learning gains of a kind acceptable to people whose learner identities have been damaged by previous experiences of formal education. Once sustained independent use of the DP has been established, participants will be enabled to join at least one DP-facilitated online community to engage in synchronous and non-synchronous networked activity. Participants will be encouraged to take part in groups that will help them look for answers and solutions to problems that affect their everyday lives (networking for jobs, understanding more about their rights and responsibilities, finding out about support groups, finding out about learning opportunities). This focus builds on previous research into the characteristics, nature and significance of virtual communities. 3. The Idea of Digital Partners The DP interface will be a development of Wilks’ Companions project, which is creating a personalized, conversational, multimodal interface, one that “knows” its owner, and that is implemented on a range of platforms, indoor and nomadic, and based on integrated high-quality research in multimodal human-compute interfaces, intelligent agents and human language technology. The aim of this platform is for realistic machine conversation with a subject that will be helpful and emotionally supportive. The overall scientific and technological objective of Companions is the fusion of human-computer communication modalities at a higher level than anything achieved till now, so as to produce a humanmachine interface that stimulates a persistent human-like presence in a machine, one with emotional aspects that forms a long-term bond with its user. The Companion’s tasks so far have not included eLearning, and the specific research challenge for this project is to investigate whether such a conversational, permanent or long-term computer partner, can assist in a learning process with identifiable groups. In this project, this functionality will be demonstrated in three types of interface, one nomadic (i.e. mobile phones) one static (i.e. PCs), and one web-based. We intend that the DP be available both as a partner to speak to and one using only typed communication. The latter would lower the risk of comprehension errors substantially, and will be used in the first prototype. The development process of these functionalities can be represented as a spiral shape. The area of the spiral shape represents the range of DP’s functionality (i.e. the maximum numbers of functions DP can provide). The range grows bigger, as uses of DPs develop and are maintained over time. The nearer the function is to the centre of the spiral shape, the more common the function is. For example, the function “Knowledge Content” is a primary function, which will be implemented in the first prototype. Using this shape to represent the functionality of DPs shows that there is considerable scope for the development, as well as no discrimination against different functions. From the centre of the spiral shape (represented as the inception of a DP), any freeform path can be drawn to indicate the development of any DP’s functionality. The length of this path is not limited within the range of any spiral shape in Figure 2. DPs with different functions are chosen in terms of different pedagogic requirements within the educational context as well as personal interests for informal education.

286

C. Davies and J. Zhang / Proposal for Digital Partners Project

A key function of the Partner interface will be to build up a record of the activities carried out by users on their computers or equivalent devices, which will involve a process of tagging and segmenting types and content of tasks, thus creating a repository of computer uses for each user. This repository would also constitute an editable (by users themselves) interface or persona through which to interact within the wider online communities established by the project. This will have the potential both to impact positively upon selfconcept, and to open up additional avenues for support and learning in a more collaborative mode across the network community (Figure 1).

Figure 1 Construction of DP

Figure 2

Conclusion We are proposing what will be a challenging and important project, in the confidence that the research design and interdisciplinary team possess the strengths and flexibility necessary for a significant outcome.

References [1] Aldridge, F and Tuckett, A (2004) Business as usual…? : the NIACE survey on adult participation learning 2004 NIACE. [2] Cobb, P., Confrey, J., diSessa, A., Lehrer, R., and Schauble, L. (2003). “Design Experiments” in Educational Research, 32(1), 9-14. [3] DfES (2002) Get on with it. A post 16 e-learning Strategy Task Force Report. London: DfES [4] Gorard, S., Roberts, K. & Taylor, C. (2004) “What Kind of Creature is a Design Experiment?” British Educational Research Journal, 30(4), 577-590 [5] Marsh, H. W. & Hau, K. T. (2004). “Explaining paradoxical relations between academic self-concepts and achievements: Cross-cultural generalizability of the internal-external frame of reference predictions across 26 countries.” Journal of Educational Psychology, 96, 56-67 [6] Selwyn, N, Gorard, S and Furlong, J (2006) Adult Learning in the Digital Age: Information Technology and the Learning Society. London: Routledge [7] Shuklina, E. (2001) ‘The technologies of self education’ Russian Education and Society 43, 2: 57 [8] Wilks, Y. (2005) “Artificial Companions”. In International Scientific Reviews. (30 :2)

Assessment and Evaluation

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

289

Development of Portfolio Assessment Support System Yasuhiko Morimoto1), 2), Isao Kikukawa1), Maomi Ueno3), Setsuo Yokoyama4), and Youzou Miyadera4) 1)

Fuji Tokoha University 325 Obuchi, Fuji, Shizuoka, 417-0801 Japan Nagaoka University of Technology 1603-1 Kamitomioka, Nagaoka, Niigata, 940-2188 Japan 3) The University of Electro-Communications 1-5-1, Chofugaoka, Chofu, Tokyo, 182-8585 Japan 4) Tokyo Gakugei University 4-1-1 Nukuikita, Koganei, Tokyo, 184-8501 Japan 2)

[email protected], [email protected], [email protected], [email protected], [email protected]

Abstract: Portfolio assessment using electronic portfolios has attracted attention as a more authentic means of evaluating learning directly. However, teachers and learners need support designing and practicing portfolio assessment because no established support methods corresponding to lesson forms and assessment-activities exist. A portfolio assessment support system (PASS) to assist portfolio assessmentis developed. The system works based on the relations between lesson forms and portfolios that need to be collected (i.e., Portfolio assessment Design Semantics (PDS)) and the relations between practices and collected portfolios (i.e., Portfolio assessment Practice Semantics (PPS)). This paper described the development, practice and evaluation of PASS. Keywords: Portfolio assessment, Formal description, Support system, Assessment tool

1. Introduction In education, Portfolio assessment using electronic portfolios is spreading with computerized learning systems. However, portfolio assessment has not been carried out adequately in the educational field because of two problems, and teachers and learners both need help [1]. Problem(1): When teachers decide to use portfolios for lessons they intend to assess, they do not know what kinds of portfolios need to be collected. In other words, they need help in the design phase of portfolio assessment. Problem(2): Learners do not know which collected portfolios should be used for practice. In other words, learners also need help determining which portfolios they need to assess in the practice phase. Help with portfolio assessment is carried out with tools that using electronic portfolios [2 – 4]. However, these tools cannot assist teachers in selecting which portfolios to assess during the design phase. Moreover, teachers must match the type of collected portfolio and the method of practice to the tools being used because these tools individually determine the types of collected portfolios and the practice method. The relation between practices and collected portfolios is not clear. Therefore, these cannot solve problems (1) or (2). The purpose of our study is to support portfolio assessment at the design and practice phases, and solve problems (1) and (2) at the same time. In the first stage, we extracted and clarified the relations between lesson forms and portfolios that need to be collected (i.e., Portfolio assessment Design Semantics (PDS)) and the relations between practices and

290

Y. Morimoto et al. / Development of Portfolio Assessment Support System

collected portfolios (i.e., Portfolio assessment Practice Semantics (PPS)). We also developed a formal method of describing all relationships between PDS and PPS[5]. Moreover, in the next stage, we developed a support system based on the formal description method (PASS: Portfolio Assessment Support System). This paper describes the development, practice and evaluation of PASS 2. Portfolio Assessment Support System (PASS) PASS is a system with a framework that support portfolio assessment by working while dynamically changing interfaces based on formally described PDS and PPS.The formal method of describing PDS and PPS was developed because of the development of this system [5]. PDS expresses the relations between lesson forms and portfolios that need to be collected, and PPS expresses the relations between practices and collected portfolios. These relations are extracted practical records, such as lesson plans that have been used. As the relations between lesson forms and portfolios that need to be collected and the relations between practices and collected portfolios can be precisely and consistently described using the formal description, the system can seamlessly assist during the design and practice phases of portfolio assessment by working based on PDS and PPS. 3. System development 3.1 System configuration PASS has a client part and a server part, and consists of four subsystems: semantics management(SMS),design support, practice support, and DB management(DBMS). Furthermore, each consists of various modules. We used Perl, HTML, and JavaScript as development languages for the system, and also XMLDB. Semantics management subsystem (SMS) This subsystem consists of an SMS-service module and an SMS-control module. First, the SMS-service module parses PDS and PPS files, creates PDS and PPS tables, and manages the tables. Each table consists of three sub-tables: a symbol-table that stores symbol information, a structure-table that stores the definitions of the related structure between symbols, and a relations-table that stores the concrete relationships between symbols based on the definitions. The data structure of the symbol-table has a hush table, and the data structures of the structure-table and relations-table have tree structures. Using such data structures for the tables allows changeability of concrete relationships by changing the descriptions. Design support subsystem This subsystem assists teachers in designing portfolio assessment through cooperation with the design-interface and design-control modules. After receiving the teacher’s orders from the design-interface module, the design-control module requests a check of the semantics from the SMS and receives the results. The design-interface module generates dynamically adaptive interfaces according to the results and provides them to the teacher. Practice support subsystem This subsystem assists users with portfolio assessment through cooperation with the practice-interface and practice-control modules. After receiving the user’s orders from the practice-interface module, the practice-control module requests a check of the semantics from the SMS and receives the results. The practice-interface module generates dynamically adaptive interfaces according to the results, and provides them to the user. DB management subsystem (DBMS)

Y. Morimoto et al. / Development of Portfolio Assessment Support System

291

This subsystem consists of a DBMS-control module and a Portfolio-DB (XMLDB), and reads, writes, updates, and stores portfolios. 3.2 Flow of dynamic portfolio presentation based on PDS/PPS PASS adaptively helps users by dynamically generating interfaces based on the description (semantics) of PDS and PPS. 1) The learner selects an assessment activity to do on the interface presented by the practice-interface module. 2) The practice-control module queries the SMS-service module about semantics through the SMS-control module. The SMS-service module mechanically extracts a kind of necessary portfolio for the assessment activity selected in 1) from the PDS and PPS table. 3) The practice-control module reads the learner collection information on the kind of portfolio extracted in 2) from the database through the DBMS-control module. 4) The practice-interface module dynamically generates interfaces based on the results in 2) and 3), and presents the interfaces to the learner. 4. Lesson Practice 4.1 Outline This system was used in Web Programming I at Fuji Tokoha University (first semester in 2006, total learners: 23). The syllabus is shown in Table 1. Table 1 Schedule of Web Programming I Class number 1-3 4-9 10 11-14 15

Contents Foundations of Web (TCP/IP, HTTP, etc.) Creating Web pages with HTML Midterm presentation <portfolio conference> Development practice of Web sites Final presentation <portfolio conference>

4.2 Lesson method In the practice phase of lessons that introduce portfolio assessment, learners can reflect their learning deeper by deliberately repeating assessment activities while using more collected portfolios. Here, a chain of assessment activities is called a “deep assessment activity”. More enhanced portfolio assessment can be expected by repeating deep assessment activities.

Fig. 1 Portfolio assessment activity cycle To perform deep assessment activities, we proposed a portfolio assessment activity cycle (Fig. 1). Assessment activities were carried out in accordance with the cycle. The portfolio assessment activity subset in Fig. 1 is a subset of a chain of assessment activities

292

Y. Morimoto et al. / Development of Portfolio Assessment Support System

repeated in the second step of the portfolio assessment activity cycle. . Work example of PASS 5.1. Support during design phase A teacher designs portfolio assessment by selecting a lesson form (lesson style, person who sets tasks, person who creates rubrics, etc.) (Fig. 2). Based on PDS, PASS dynamically determines the collected portfolios according to teacher specification, generates adaptive interfaces, and presents them to the teacher. The teacher can also be presented with the lesson form and others based on PDS dynamically by previously specifying portfolios for learners. Who creates rubrics? What is a lesson style? Teache Collaboration by Teacher and Learne

Lesson styles

Learne

Fig. 2 Sample screens of the design support 5.2. Support during practice phase A screenshot of “Self-assessment” is shown in Fig. 3. On not only this screen but all screens, assessment activities are specified in the left frame, and the assessment activity specified is executed in the right frame. In the center frame, portfolios needed when the assessment activity is carried out are dynamically presented based on PPS and PDS. Therefore, learners can efficiently carry out assessment activities while referring to the portfolios in the center frame. The kind of assessment activities presented in the left frame dynamically change based on PPS. Assessmentform Menu

Collected portfolios that the learner needs forself-assessment

Fig. 3 Screenshot of “Self-assessment”

Y. Morimoto et al. / Development of Portfolio Assessment Support System

293

The learner selects a rubric on the right frame and performs self-assessment while referring to portfolios dynamically presented based on PPS and PDS in the center frame. When a learner wants to perform other assessment activities that use the portfolio under reference, the learner can find assessment activities that can be performed by right-clicking the referenced portfolio, which displays an activities list as a pop menu based on PPS and PDS. The learner therefore can perform other assessment activities by selecting them on the menu, and then the system dynamically changes to the screen of the specified assessment activity. 6. System evaluation 6.1 Purpose of evaluation We evaluated whether problems (1) and (2) were solved. Therefore, the purpose of the evaluation was as follows. (a) Evaluating the assistance with design of portfolio assessment according to each lesson form that teachers intend (Problem (1)). (b) Evaluating the assistance with practice of portfolio assessment adapted to learner assessment activities (Problem (2)). The evaluation did not evaluate the quality of the contents of collected portfolios or relationships the semantics expressed, but instead evaluated the effect of the supporting mechanism of PASS. The evaluation corresponding to purpose (a) is in section 6.2., and the evaluation corresponding to purpose (b) is in section 6.3. 6.2 Evaluation in design phase 6.2.1

Evaluation method

We compared the case using PASS with the case not using it. The testees were eight incumbent teachers (two junior high school teachers, three high school teachers, and three university teachers) who had experience using portfolio assessment in their classes. Procedures 1) Explain the outline of the lessons (two kinds). The evaluation was done for all testees in both cases using and not using PASS. In the case where PASS was not used, the testee used a general tool, such as word-processing software, if necessary. 2) Carry out Task-I. In Task-I, testees designed portfolio assessment of the lesson “Use of WWW”. Four people (Group A: T1~T4, on Fig.5) used PASS, while the remaining four people (Group B: T5~T8, on Fig.5) did not. They properly fixed the duration of lessons. 3) Carry out Task-II. In Task-II, testees designed portfolio assessment of the lesson “Use of multimedia”. Four people (group B) opposite to those in Task-I used PASS, while the remainder (group A) did not. 4) Question and interview the testees. Items of evaluation z Time required As a quantitative evaluation, times required for Task-I and Task-II were measured. z Questionnaires As a qualitative evaluation, the questionnaire in Fig. 4 was sent out to all testees.

294

Y. Morimoto et al. / Development of Portfolio Assessment Support System

(1) Were you able to design portfolio assessment easily? <ease> (2) Were you able to understand what kind of portfolios should be collected? <decision of portfolios> (3) Were you able to design satisfactory portfolio assessment? <satisfaction> (4) Was your design effective? <effectiveness> * Each item should be graded from one to five (five is positive).

Fig. 4 Questionnaire items regarding design phase 6.2.2

Results and consideration

A graph of times required for Task-I and Task-II is shown in Fig. 5.For those using PASS, the mean time was 9.8 minutes (SD=5.4), and for those not using PASS, the mean was 19.5 minutes (SD=7.2). The statistical difference was determined by a two-sided paired t test. The difference with P<0.01 was considered significant (t=-3.61, df= 7). Therefore, teachers tend to be able to design portfolio assessment by using PASS in a short time.

Fig. 5 Time required for Task-I and Task-II A summary of the results of the questionnaire is shown in Table 2. In results of the questionnaire, those using PASS gave high scores in all items, and each of the statistical differences was determined by a two-sided paired t test. Therefore, teachers who had experience taking portfolio assessments gave a high appraisal of the use of PASS. Table 2 Result of questionnaire about design phase Items

Using PASS – Mean (SD)

(1)<ease> (2)<decision of portfolios> (3)<satisfaction> (4)<effectiveness >

4.50 (0.50) 4.38 (0.70) 4.13 (0.60) 4.25 (0.66)

Not using PASS – Mean (SD)

t-value

2.38 (0.48) 6.06** 2.75 (0.83) 2.88* 3.00 (0.71) 2.55* 3.00 (0.71) 2.56* N = 8, *p < .05, **p < .01

By using PASS, teachers can design portfolio assessment according to lessons that they intend fast and easily. Moreover, teachers tend to be satisfied with the results, finding their assessment high effectiveness. Therefore, purpose (1) of the evaluation was achieved, and problem (1) was solved. 6.3 Evaluation in practice phase 6.3.1

Policy of evaluation in practice phase

PASS supports the practice phase by mechanically analyzing PPS and PDS as formally described and working based on this analysis. To achieve purpose (2), we therefore carried out evaluation experiments under the following three environments (ENV(1):Using PASS, ENV(2):Using pseudo-system, and ENV(3):Not using PASS and compared the results. In ENV(1), testees carried out assessment activities using PASS. In ENV(2), testees carried out assessment activities using a system with only a function to manage portfolios

Y. Morimoto et al. / Development of Portfolio Assessment Support System

295

(pseudo-system). The pseudo-system was made for this experiment by us and doesn’t have a function that works interpreting PPS and PDS. All collected portfolios are presented in a time series.In ENV(3), testees did not use any systems and carried out assessment activities while managing portfolios by hand. The evaluation experiment was conducted using folders (Windows XP standard) and digital files (doc files of MS-Word 2003) of each learner a teacher prepared beforehand. “Evaluation experiment I” was performed as a quantitative evaluation (6.3.2), and “Evaluation experiment II” was performed as a qualitative evaluation (6.3.3). 6.3.2 Evaluation experiment I A. Evaluation method This experiment compared cases where portfolio assessment activity subsets were performed under ENV(1)-(3). The testees were 12 learners (A-team: 6, B-team: 6). This experiment executed portfolio assessment activity subsets under three different environments by the same person, and we compared the following three results: “activity time”, “number of portfolios referred to”, and “achievement level of deep assessment activity”. PASS can be objectively judged by whether it can effectively help according to learner assessment activities by overlapping these results and comparing them. Evaluation items z activity time The total time spent on all the activities in the subset was measured. z number of portfolios referred to The total number of referred-to portfolios was measured. z Depth of assessment activity (by questionnaire) A questionnaire about the achievement level of deep assessment activities at each assessment activity of the subset was given (grading scale from one to five: five is high). Procedures Testees performed assessment activities of subsets under ENV(1) - (3) and were divided into teams A and B. The experiment was conducted according to the following procedure: A-teamENV(3)->ENV(1)->ENV(2) and B-team ENV(1)->ENV(3)->ENV(2). B. Results and consideration A graph of the experimental results of A-team is shown in a 3D-scatter diagram in Fig. 6. A similar graph for B-team is shown in Fig. 6.In ENV(1), many testees tended to think that they were able to execute deep evaluation activities in a comparatively short time by referring to a lot of portfolios.

Fig. 6 Results of experiment I: A-TeamFig. 7 Results of experiment I: B-Team

296

Y. Morimoto et al. / Development of Portfolio Assessment Support System

Therefore, deep assessment activities can be done while efficiently referring to portfolios by using PASS for a short time. 6.3.3

Evaluation experiment II

A. Evaluation method The questionnaire and the interview were done with testees who executed portfolio assessment activity subsets on ENV(1) - (3). The testees were 22learners. A qualitative evaluation concerning assessment activities was done by conducting this experiment. Items in the questionnaire are shown in Fig. 8. (1) (2) (3) (4)

Could you perform your assessment activities easily <ease> Could you understand what assessment activities should be done referring to which portfolios Could you perform satisfactory assessment activities <satisfaction> Could you work on assessment activities enthusiastically <enthusiasm> * Each item was graded from one to five (five is positive).

Fig. 8 Questionnaire items of experiment II B. Results and consideration In all question items, ENV(1) was highly praised. The statistical difference between ENV(1) and ENV(2) was determined by a two-sided paired t test. The difference with P < 0.01 was considered significant (t = 4.40, df = 20). Moreover, the statistical difference between ENV(1) and ENV(3) was determined by a two-sided paired t test. The difference with P < 0.01 was considered significant (t=3.89, df = 20). 6.4 Evaluation results and consideration of support of practice phase From evaluation experiments I and II, we found that learners tended to be able to perform deep assessment activities while efficiently referring easily to portfolios in a short time by using PASS. Therefore, purpose (2) of the evaluation was achieved, and problem (2) was solved. . Conclusion We described the development, practice, and evaluation of PASS. We found out that our formal description of PDS and PPS was very effective in the design and practice phases of portfolio assessment from evaluations. In the future, we want to develop a framework that formally describes the processes of assessment and implement it with a system that controls processes of assessment based on the formal descriptions. References [1] Morimoto, Y., Ueno, M., Takahashi, M., Yokoyama, S. & Miyadera, Y. (2005). Modeling Language for Supporting Portfolio Assessment, Proc. The 5th IEEE International Conference on Advanced Learning Technologies, Kaohsiung, Taiwan: IEEE Computer Society, 608- 612. [2] Chang, C. (2001). Construction and Evaluation of a Web-Based Learning Portfolio System: An Electric Assessment Tool, Innovations in Education and Teaching International, 38 (2), 144- 155. [3] Chen, G., Liu, C., Ou, K. & Lin, M. (2000). Web Learning Portfolios: A Tool For Supporting Performance Awareness, Innovations in Education and Teaching International, 38 (1), 19- 30. [4] Fukunaga, H., Nagase, H. & Shoji, K. (2001). Development of a Digital Portfolio System That Assists Reflection and Its Application to Learning, Japan Journal of Educational Technologies, 25 (Suppl.), 8388. [5] Morimoto, Y., Ueno, M., Kikukawa, I., Yokoyama, S. and Miyadera, Y. 2006), Formal Method of Description Supporting Portfolio Assessment, Journal of Educational Technology & Society, 9 (3), 88-99.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

297

Peer-Assessment in Web-based ePortfolios System: An Experimental Study Wang Youmei Dept. of Educational Technology, Wenzhou University, China [email protected] Abstract: Eportfolio is used as an important tool to facilitate learning reflection and

assessment in IT-infused instruction and online instruction. This study adopts both qualitative and quantitative approaches to explore the quality and effectiveness` of onymous or anonymous peer-assessment, self-assessment and reflection in WePS1.0

(Web-based ePortfolios System). Findings shows that anonymous assessment and the assessment on onymous peers have positive significance and better quality on learning,

and anonymous assessment was more critical but inferior quality. Peer-onymous assessment would promote learning and reflection in online instruction. Keywords: Eportfolios, Peer-Assessment, Reflection

Introduction There are many educators who advocate the use of eportfolios in education, both with students and teachers. And there are a variety of purposes for developing eportfolios: as an assessment tool, for marketing or employment, and to document the learning process and growth for learners of all ages, from pre-school through graduate school and into the professions (Barrett, 2005). With the development of online instruction, eportfolios usually was used as a new assessment mode focusing on learning process and learning reflection (Belanoff, et al., 1991), which depended on the implementation of peer-assessment and self-assessment. We know that online learners tend to play a hidden role, and identity of learner will affect the performance of learning (He, 2002), so it is an important issue to study the problems of onymous or anonymous learner in the process of assessment and learning. Current studies on this topic focus on the management and reflection of portfolios, most of which are qualitative studies. Based on the previous studies, this study carried out an experimental method to discuss the quality and effectiveness of peer-assessment with different anonymities in web-based eportfolios system.

298

Y. Wang / Peer-Assessment in Web-Based ePortfolios System: An Experimental Study

1. A Review of Relevant Literature 1.1 On eportfolios Eportfolio is a collection of authentic and diverse evidence, drawn from a larger archive representing what a person or organization has learned over time on which the person or organization has reflected, and designed for presentation to one or more audiences for a particular rhetorical purpose (NLII, 2003; Barrett, 2003). That is to say eportfolios is being an approach to assess learning performance of students in many educational institutions. Eportfolios assessment belongs to authentic assessment and formative assessment, aiming to break through the traditional assessment ways such as quantification assessment and paper test (Wang, 2003), and now it is a trend to build the web-based eportfolios assessment system to evaluate performance of online learning (Barrett, 2003). 1.2 On peer-assessment Peer-assessment has become more popular in the instructional assessment area. Topping (1998) put forward a definition regarding peer-assessment. According to his definition, Peer-assessment is that students attempt to play the role as teacher to give assessment on their classmates at the same grade. After studying 109 articles on peer-assessment in the ERIC database, Topping pointed out that some scholars had already used this kind of assessment technique in various subjects including science, information and social study, and that the peer-assessment has reached a certain level of reliability and effectiveness. Peer-assessment technique can be used to improve the high-order thinking and learning motivation of students, which is a good assessment model for adult learners. Jack McGourty had also found that there were consistency between self-assessment and peer-assessment; and some learning outcomes that cannot be tested in traditional assessment may be reflected in peer-assessment. Peer-assessment can not only engage students better in study, but also improve the quality of students learning and students’ ability of criticism and provide them with opportunities to take part in the assessment. And students could help each other and improved their learning performance (Falchikov, 1995). Topping (1998) analyzed 31 studies about applying peer-assessment into various projects in higher education during the year of 1980 to 1996, which was related to the reliability of peer-assessment. By calculating the correlation coefficient between peer-assessment and professionals assessment, Topping (1998) found 25 among 31 research articles showed that peer-assessment has high reliability. 1.3 computer-mediated peer-assessment Yong Zhao (1998) systematically summarized anonymous computer-mediated assessment. He took on a research to study aspects of assessment such as attention focus, critical and cognitive slack of students. His experimental courses were educational introduction to students and Spanish, which is an important course in two American universities. His experiment chose 22 and 23 students as samples respectively; he studied the influence of anonym on students’ cooperative learning and discussed the anonymous peer-assessment.

Y. Wang / Peer-Assessment in Web-Based ePortfolios System: An Experimental Study

299

It was found that anonym has double-side function, it can make the testees more critical but it also lowers the quality of assessment in the process of peer-assessment. However, the framework of this study came from western culture for its experimental background, and it still needs validating whether it can be analogized to eastern thoughts and learning environment, about which I talked with the famous expert on eportfolios——Dr. Helen Barrett. Based on her work experience in Singapore, She thought there is an obvious difference in self-reflection and peer-assessment between western and eastern students, which is the root of the framework for our study. 2. Method 2.1 Learning Environment The study was carried out by the WePS1.0 online, which was a web-based eportfolio system developed by us (Wang, 2004). In WePS1.0, We look upon eportfolios as a kind of aggregation in which learner can represent their learning objectives, activities, outcomes, performance, efforts, progress and reflection about learning process and results under an eportfolios learning environment. In WePS1.0, each student has a learning space online; actually it is a portal for students to learn. Students can collect the learning artifacts, assess himself and others write the reflection on learning and development, and there are powers for everyone to access others’ eportfolios with teachers’ accreditation. 2.2 Task and Design The time of study span was two years. And the participants were 40 students who major in educational technology coming from Wenhou university in China. They learned with WePS1.0 and took on eportfolios-based assessment each other for a long time. The experimental course was systematically instructional design and project management. In this study we chose one time randomly (this time is mid-term assessment) to analyse and test. Without being told the purpose of the experiment, 40 students were required to finish the eportfolios task including analyzing, designing and concluding constructivist cases after web-based learning the same as before. The scale for eportfolios was provided to them before learning. After finishing the task, they were to post their artifacts online with the requirements and carry through self-assessment and peer-assessment, which included qualitative and quantitative assessment, and then complete the questionnaires (38 of which were valid) provided by the teacher, the questions in which are all Likert’s scale. 2.3 Sample and Design The samples were organized in such way as is shown in the following table, based on the partition of anonymous and onymous peer-assessment:

300

Y. Wang / Peer-Assessment in Web-Based ePortfolios System: An Experimental Study

Table 1: sample and design Group

Random

Onymous requirement

Eportfolios task

Assessment

Anonymous

Choose 20

Register; hand in works

Complete tasks

Assess at least one

randomly in

real name. All the

consultation with

definition group

grouping students

terms of Onymous group

seats

The same as above

and assessment without information is anonymous.

combining

self-doing.

requirement

anonymous and

onymous peer

Register and hand in

Complete tasks

Assess at least one

with real name

consultation with

onymous peer

works and assessment

combining

self-doing.

anonymous and

2.4 Research Questions

Difference of focus in the process of assessment between two groups. Difference of attitude in the process of assessment between two groups. Difference of reflection on assessment in the process of assessment between two groups. Difference of effect and quality on assessment in the process of assessment in WePS1.0 between two groups

2.5 Data Collection and Processing The experimental data came from three aspects: the first was from the questionnaires on online learning, which was an attitudinal scale and converted into 5 scored statistically; the second from the score of eportfolios assessment (including self-assessment and peer-assessment) (max.100-score); the third from the quantitative assessment from teacher on assessors’ qualitative assessment, taking a 5-scale (5=Strongly agree and 1=Strongly disagree). Finally SPSS11.0 and EXCEL2003 were used to analyze the data. 3. Data analysis and Results The main purpose of this study was to discuss the attitude and effect of different onymous peer-assessment in eportfolios. For the samples were not many, it was reasonable to adopt the methods of t-test, and to explore the impact that different onymous peer-assessment has on learning. 3.1 Difference of focus in the process of assessment between two groups Referencing the scale proposed by Yong Zhao (1998), we set two questions to discuss the

Y. Wang / Peer-Assessment in Web-Based ePortfolios System: An Experimental Study

301

difference of focus in the process of assessment between two groups. The first question was: “I have thought of how the author will respond to my assessment when I assess others’ eportfolios”, the result indicated that comparatively, as the assessors, ominous group paid more concerns about the response of the assessees to their assessment during the process. As to the second question: “when assessing others, I pay more attention to the quality of the peers’ task and content rather than author themselves or other aspects”. Both two groups thought they paid more attention to the quality of the peers’ task and content rather than author themselves. But it was obvious that more testees of onymous group agree on this view than the other group When converting the qualitative table into 5-score scale and conducting single-sample t-test (two-tailed test, df=36, the same as follows), we found that the answers of two groups on the two questions had no “significance of difference” (p>.05), as shown in the following. Table 2 attention focus of assessment Question 1

groups Anonymous group Onymous group t-value

Question 2

M

SD

M

SD

3ˊ47

0ˊ77

4ˊ16

0.69

3ˊ42

1ˊ12

4ˊ16

0.17 (p>.05)

0.69

1.00 (p>.05)

3.2 Difference of attitude in the process of assessment between two groups In the web-based collaborative peer-assessment process, were there any differences in assessors’ attitude to assessment between two groups? We designed two questions to test and found that the onymous group members had more confidence in their assessment for their companions and their assessment were more effective than the anonymous group, and there were “significance of difference” (P<0.05) .However, both the two groups thought that they were diligent and responsible in this process, and there were no “significance of difference” (P>0.05). Table 3 self-cognition of the two groups on their assessment on their peers groups

Anonymous group Onymous group t-value

“my assessment can promote the peer’s learning”

M

“I fulfill the responsibility of peer-assessment seriously”

3ˊ26

SD

0ˊ73

M

SD

3ˊ84

0ˊ60

3ˊ95

0ˊ52

2ˊ66˄P<0.05˅

3ˊ89

0ˊ57

0.297˄P>0.05˅

302

Y. Wang / Peer-Assessment in Web-Based ePortfolios System: An Experimental Study

3.3 Difference of reflection on assessment in the process of assessment between two groups As learners, how did they think of the assessment given by their peers? We set three questions to discuss this. The results as shown in table 4: the onymous group thought their peers’ assessment on them was pertinent, and there were significance of difference. And they also thought the assessment can actually promote their study, Furthermore, it could promote their active self-reflection on his or her learning. The results on three questions all were significance of difference. Table 4 attitude and reflection of the two groups on peer-assessment groups

“peers` assessment is

“peers` assessment can

“peers`

M

SD

M

SD

M

SD

3ˊ79

0ˊ63

3ˊ74

0ˊ56

3ˊ79

0ˊ42

pertinent”

anonymous

3ˊ21

group

Onymous group

t-value

0ˊ86

promote learning”

3ˊ05

0ˊ78

assessment

can

promote active reflection on self performance”

3ˊ32

0ˊ75

2ˊ38(p=0.023∗)

3ˊ13˄p=0.004∗˅

2ˊ41˄p=0.021∗˅

∗P<.05

∗P<.01

∗P<.05

3.4 Difference of effect and quality on assessment in the process of assessment between in WePS1.0 two groups In order to study the quality of peer-assessment between different groups, we combined qualitative and quantitative methods to explore it. In table 5, we compared different assessing effects and quality, and criticism and quality of assessment in the two groups. Results showed that there was no significance of difference in the dimension of quantitative data. According to the view of Yong Zhao (1998), the lower the score was, the higher criticism of the assessment had, we could see that the anonym-to-anonym -assessment was the most critical, but there were no significance of difference. And in the qualitative dimension, the anonym-to-anonym-assessment was of the lowest quality and the onymous-to- onymous assessment was quite the contrary. Furthermore, there was no significance of difference in qualitative assessment for respective group. Table 5 intercross comparison of effect and quality of assessment between two groups in eportfolios with qualitative and quantitative methods Students groups

of

self-assessment

quantitative average(100=full score) onymous group

76ˊ8

onymous

assessment

81ˊ0

anonymous assessment

78ˊ6

single

sample

t-test

anonymous/onymous assessment

0.74(p=0.467)

in

Y. Wang / Peer-Assessment in Web-Based ePortfolios System: An Experimental Study

anonymous group

80.3

79.3

75.3

quality of qualitative assessment (5=the most suitable assessment) onymous group anonymous group

none

᮴ none

3.38

3.26

3.36

2.71

303

0.875(p=0.394)

0.027(p=0.98)

1.124(p=0.278)

4. Comparison with other researches As mentioned above, there are few studies in the field of peer-assessment with eportfolios. We mainly compare this one with the experimental results of Yong Zhao (1998), which were conducted in the CMC learning environment. The results are identical in the anonymous assessment that anonymous assessment is helpful to improve criticality but has lower quality. However, our study shows that about the focus during the process of assessment there are significant differences between onymous and anonymous assessment, comparatively, more members of onymous groups are more concerned of the author’s response on their assessment and more about the content of eportfolios rather than the author himself, but there were no significance of difference between two groups. In contrast, the conclusion of Yong Zhao is that in the condition of anonymous assessment, the assessors concern more of content. The difference may be caused by different attitudes of western and oriental students on collaborative learning and their comprehension of collaborative assessment, at the same time it may be related with teachers’ direction of the experiments. Certainly, it is a very important factor that the supporting environment of these two experiments is different. Yong Zhao adopted e-mail assessment in the CMC, while we adopted independent web-based eportfolios system supporting assessment and study. 5. Conclusion / Discussions With the development of e-learning and online instruction, the assessment methods and models will be more diversified, Eportfolio is one of them. In the learning and assessment system students’ registration with real names or not has direct impact on learning effect. Based on a two-year experimental research, the following conclusions are drawn: Onymous group members paid attention to the response of their learning peers when conducting assessment, and they argued that they paid more attention on portfolio content per se and there were no obvious differences between the two groups; The onymous group had more confidence in peer-assessing and they thought that their assessment was more effective than anonymous groups. Both onymous and anonymous groups thought they were responsible in their assessment on others; Onymous groups members recognized peer’s assessment on their learning more than the anonymous one;

304

Y. Wang / Peer-Assessment in Web-Based ePortfolios System: An Experimental Study

The quality of assessment of onymous-to- onymous assessment was good, the assessments between anonymous students were the most critical, but had the lowest quality. At last we must point out that, in this kind of studies, many factors such as samplesǃ courses and experiment platforms will affect the results. So we think this experimental data and conclusions are correct in a specifically context. And it provides a kind of thought which can be used as a reference for the implementation of e-learning and online instruction in the future. We hope that it can be of some help for the development of web-based instruction and its evaluation. Acknowledgments I thank the people, Dr. Helen Barrett, who talked with me about difference in self-reflectionǃpeer-assessment and implement of eportfolios between western and eastern students. References

[1] Barrett, H. (2003) Presentation at First International Conference on the e-Portfolio, Poiters, France,

October 9, 2003. http://electronicportfolios.org/portfolios/eifel.pdf,2005-12-10

[2] Barrett, H. (2005) “Electronic Portfolios as Digital Stories of Deep Learning: Emerging Digital Tools to Support Reflection in Learner-Centered Portfolios.”

http://electronicportfolios.org/digistory/epstory.html,2006-5-4

[3] Belanoff, Pat & Dickson, Marcia (eds.). (1991) Portfolios: Process and Product. Poutsmouth: Heinemann

[4] Falchikov, N (1995). Peer feedback marking: Developing peer assessment. Innovations in Education &

Training International.1995, 32, 175-187

[5] He Kekang (2002), Instructional system design, Beijing Normal University Press, Beijing, China

[6] Keith Topping (1998), Peer assessment between students in colleges and universities. Educational Research, 1998, 68(3), 249-276.

[7] Wang Youmei(2003) eportfolios-based assessment strategies in IT-enhanced teaching, E-Education Research, 2003,128˄12˅,61-66

[8] Wang Youmei (2004), Web-based eportfolios system in e-learning environment, Open Education Research, 2004, 51(5), 56-58

[9] Yong Zhao (1998).The Effects of Anonymity on Computer-mediated Peer Review. International Journal of Educational Telecommunications. 1998, 4(4), 311-345.

[10] Zhu zhiting (2001) IT in education and educational reform, http://202.121.80.14/zhzht/ .2006-05-01

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

305

Construction and Performance Evaluation of High Quality Curriculum Integrated with Information Technology Xie YouRu, Yin Rui Educational Technology Institute, South China Normal University, China [email protected] Abstract: Based on the project ‘Construction of High Quality Curriculum Integrated with Information Technology’ undertaken by the author, this paper makes a detailed discussion on the original ideas, goals and contents, strategies and measures of the project as well. Meanwhile, this paper describes comprehensive performance evaluation in terms of the degree to which the project influenced on best-curriculum construction, teachers’ professional development and students’ learning abilities. The purpose of the paper is to provide beneficial references and experiences to facilitate the further curriculum construction and reform in universities. Keywords: Information technology, high quality curriculum, curriculum construction, performance evaluation

Introduction 2004ˈ‘Qiu Shi Project’, one large-scale project focused on educational informationlization, was carried out in South China Normal University. ‘Construction of High Quality Curriculum Integrated with Information Technology’, one of the most important sub-projects, was purposed to make information technology widely and effectively used via the development and utilization of network resources in 100 curricula. 1. Backgrounds 1.1 Satisfying the Need of Instruction in the Information Era In the information era, traditional instructional structure is in the face of great challenges. Positive efforts should be made to conduct applied researches on information technology, explore new instructional models and pedagogical strategies, so as to foster groups of innovative talents. According to the proposal put forward in ‘Suggestion on the further reinforcement of Undergraduate Education in Universities’ (note: it was an official file released by the Ministry of Education P.R.C.), multimedia-assisted instruction should be widely popularized in universities throughout the country. 1.2 Satisfying the Need of Best-Curriculum Construction Best-curriculum construction is a key to promote teaching quality and reform in universities. Best-curriculum is the curriculum characteristic with first-class teams of

306

Y. Xie and R. Yin / Construction and Performance Evaluation of High Quality Curriculum

teachers, instructional contents, models, materials and management. All levels of educational administration and universities should develop the innovative educational concept, increase the application of information technology in the curriculum, and establish three-level system of best-curriculum construction. (Note: three-level system of best-curriculum construction means that best-curriculum should be constructed in school-level, province-level and nation-level respectively.) 2. Goals and Contents 2.1 Goals of the Project There are three following goals to be attained: • Improving teaching quality and furthering teaching reform. • Promoting teachers’ professional development. • Fostering groups of innovative talents with good information literacy. 2.2 Contents of the Project To achieve the above goals, we should accomplish the five following contents: • Lay down a scheme of curriculum construction and teaching reform. The scheme should be stressed that more than 30% curriculum contents must be involved. • Integrate and develop network resources, which is the prerequisite for curriculum construction and teaching reform. • Conduct experiments on teaching reform. The experiments should be carried out under the direction of the theory of instructional design. Additionally, the experiments should lay emphasis on application of research methods and take the favourable advantages of information technology into consideration. • Produce videotapes which reflect curriculum construction and teaching reform. • Summarize the achievements of curriculum construction and teaching reform, including instructional models, pedagogical strategies, and performance etc. 3. Exploration on Instructional Models Instructional model is defined as the stable relationship of all activity elements and systematic structure of activity processes in certain instructional environment under the direction of teaching and learning theories. Based on the implementation of the project, we proposed five typical instructional models as followings: • Lecturing Instructional Model. Teacher gives a lecture and presents the corresponding web-based courseware, then asks some questions. Meanwhile, students search network resources and think around the questions. • Theme-based Inquiry Model. Teacher poses some themes and provides ample network resources. Meanwhile, students lay down plan, search network resources relative to the themes, write research reports, and make presentation. • Collaborative Learning Model. Students are organized by means of groups. They exchange network resources, discuss problems and accomplish tasks collectively. • Case-based Instructional Model. Case is the key element. Case presenting, discussing, analyzing and solving are four main fundamental processes. • Skill Training Instructional Model. Teacher demonstrates the essential parts of

Y. Xie and R. Yin / Construction and Performance Evaluation of High Quality Curriculum

307

skill trained so as to set an example to students and then assigns skill training tasks. Students drill and practise to master the skill perfectly. 4. Performance Evaluation 4.1 Promoting Best-Curriculum Construction Statistics reveal that the project is beneficial to establish three-level system of best-curriculum construction during 2005~2006, which is shown in Table 1. Table 1 Percentage of High Quality Curriculum Account for Best-Curriculum In South China Normal University during 2005~2006 Year

2005 2006

First-Class Curriculum province-level 10 school-level 25 school-level 15

High Quality Curriculum 8 12 10

Percentage 80% 48% 66.7%

4.2 Facilitating Teachers’ Professional Development After the project, we established an evaluation criterion system to measure how and whether the project is effective to facilitate teachers’ professional development. 30 teachers were randomly chosen as respondents. The results are shown in Table 2. Table 2 Statistics on Teachers’ Professional Development Influenced by the Project First-Level Criterion

Skills of Utilizing Information Technology to Teach Abilities of Utilizing Information Technology to Reform

Criterion

Second-Level Criterion

Mastering Theory of Educational Technology Using Information Technology Tools Utilizing Network Resources Research Method

Second-Level Criterion total score

Second-Level Criterion score ratio

12.87

85.80%

10.15

67.67%

Research Achievement Total

22.30

25.02 13.05

89.20%

First-Level Criterion total score

First-Level Criterion score ratio

60.19

85.99%

23.20

77.33%

83.39

83.39%

83.40% 87.0%

According to the above data in Table 2, we can easily draw the following conclusions: • The total score ratio is 83.39%, which indicates that the project can promote teachers’ professional development effectively. • It reveals that the project makes more helpful influence on skills of utilizing information technology to teach than abilities of utilizing information technology to reform, as the former score ratio is 85.99% and the latter is 77.33%. • As for the first first-level criterion, its three corresponding second-level criterion score ratios are 89.20%, 85.80% and 83.40% respectively. We can infer that the project is significantly beneficial to help teachers to master theory of educational technology, enhance skills of using information technology tools and resources, 4.3 Improving Students’ Learning Abilities To analyze the degree to which the project made effect on improving students’ learning abilities, we also established an evaluation criterion system. 191 students were randomly chosen as respondents. The results are shown in Table 3.

308

Y. Xie and R. Yin / Construction and Performance Evaluation of High Quality Curriculum

Table 3 Statistics on Students’ Learning Abilities Influenced by the Project First-Level Criterion Information Literacy

Learning Abilities of Utilizing Information Technology

Criterion

Second-Level Criterion

Searching Network Resources Distinguishing and Selecting Network Resources Organizing and Managing Network Resources Using Information Technology Tools Utilizing Network Resources to Learn Individually Utilizing Communication Tools to Learn Collaboratively Utilizing Inquiry Tools to Solve Problems Utilizing Evaluation Tools to Make Self-Assessment and Reflection Total

Second-Level Criterion total score 8.80

Second-Level Criterion score ratio 88.0%

7.55

75.5%

11.45

76.33%

9.94

66.26%

9.83

65.53%

9.83

65.53%

8.03

7.17

80.3%

First-Level Criterion total score

First-Level Criterion score ratio

31.55

78.88%

41.05

68.42%

72.6

72.6%

71.7%

According to the above data in Table 3, we can easily draw the following conclusions: • The total score ratio is 72.6%, which indicates that the project is helpful to improve students’ learning abilities to some extent. • It indicates that the project has slightly impact on students’ learning abilities of utilizing information technology. By contrast, the project positively affects on students’ information literacy, as the first-level score ratio is 68.42% and 78.88%. • As for the second first-level criterion, its three latter second-level criterion score ratios are less than 70%. It suggests that the project is not obviously helpful to improve students’ learning abilities of utilizing information technology, especially utilizing information technology to learn collaboratively, solve problems, and make self-assessment and reflection. 5. Conclusion Through the practice, we make it clear that the project ‘Construction of High Quality Curriculum Integrated with Information Technology’ have fairly positive influences on best-curriculum construction, teachers’ professional development and students’ learning abilities. However, there are still some problems need to be resolved, for example, how to effectively change students’ learning methods to improve their learning abilities by utilizing information technology. Such problems and the like will be explored and studied further. References [1] Notice on First-Class Curriculum Construction in Higher Education. (an official file promulgated by the Ministry of Education P.R.C.) http://218.192.175.180/gaojiao/index.asp [2] Xie YouRu. (2005) Experimental Research on Curriculum Reform Based on Network Resources in Guang Dong Universities. e-Education Research, 5. [3] Zhang YanHong. (2006) Performance evaluation of ‘Qiu Shi Project’ in South China Normal University. (Bachelor’s paper)

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

309

Effectiveness of WebQuest in the Teaching of STS in Secondary Biology Ka-leung Tse, Sai-wing Pun CAITE, The Chinese University of Hong Kong [email protected] Abstract: This paper reports an exploratory case-study (one class of secondary 5 students) investigating the effectiveness of WebQuest in enhancing the teaching of STS in biology through collaborative project work. The results generally support that WebQuest is effective in the teaching of STS in biology by promoting positive attitude towards collaboration, arousing interest in learning and enhancing sophisticated thinking skills. Keywords: collaborative work, WebQuest, STS, project learning,

Introduction Science provides the knowledge for technology to generate solutions to solve problems in the real-world (Bybee, 2000, cited in Wonacott, 2001). In contemporary science education, Science-Technology-Society (STS) has become an emergent discipline which aims to study the history, nature and social impact of science and technology. National Science Teachers Association (NSTA, 1990) claims that “the experience of science education through STS strategies will create a scientifically literate citizenry for the 21st century”. In Hong Kong, the Secondary 4-5 Biology Curriculum Guide (Curriculum Development Council, 2005) suggests that higher ordered thinking skills such as problem-solving and informed decision-making are best learnt from the study of STS content through project learning or problem-based learning. However, teachers in Hong Kong are not ready in the adoption of STS contents in teaching. As pointed out by Aikenhead (1994), the major reason given by those teachers who defend traditional science instruction is that the time spent on STS contents may mean a reduction in the time for teaching science concepts and a subsequent reduction in general achievement and preparation for future study. Owing to time constraints, it is the usual practice of many schools in Hong Kong to select just one or two items (or even none) from the list of STS topics for project work in the two-year S4-5 biology curriculum, with each project lasting from one month to one school year. Unfortunately, the outcomes are mostly “copy-and-paste” products. In fact, the goals set out in the Curriculum Guide can hardly be achieved. Therefore, effective teaching strategies are needed to help improve the quality of student project work while students can still learn relevant biological concepts without sacrificing any achievement in the subject contents. The STS and WebQuest A WebQuest is an inquiry-oriented online activity in which some or all of the information that students in groups interact with comes from resources on the Web. These resources are provided by the teacher and students spend their time using information, not searching for it. The goal of WebQuest is to help students think and reason at higher levels of analysis, synthesis, and evaluation through a scaffolding process. Furthermore, the learning materials retrieved from the Web should be related to our society (Dodge, 2001). Accordingly, its use in STS learning and teaching is worth exploring. “The idea behind the STS program is to provide a real-world connection for the student between the classroom and society” (Yager, 1990, cited in King, 2001, p. 1). As supported by many science education researchers (Heath, 1992, cited in McCann, 2000), the best way to teach STS is through collaborative projects in which students are in charge of their own leaning, make informed decisions and

310

K.-l. Tse and S.-w. Pun / Effectiveness of WebQuest in the Teaching of STS in Secondary Biology

solve problems by themselves. All the advantageous features of WebQuest coincide with the instructional needs of STS learning and teaching – student-centred, authentic, motivating, collaborative, demand of critical thinking and cognitive abilities. Thus, the feasibility of WebQuest in the learning and teaching of STS is apparent. As an attempt to assess the effectiveness of WebQuest in enhancing STS biology learning and teaching, the authors conducted a case study on the effectiveness of a WebQuest on the STS topic, “Human Infertility”, in a secondary 5 science class in a Hong Kong school. The major aim of the study is to investigate how the WebQuest is effective in the teaching and learning of the biology STS topic in respect to students’ collaboration, affective and cognitive qualities in the science class under test. Methodology The study was carried out in a boys’ school with one secondary 5 science class (41 students). Basing on the templates downloadable from the WebQuest Page, a WebQuest was constructed on the STS topic “Human Infertility” which is a social issue related to the topic “Human Reproduction”. The 41 students in the class were divided randomly into seven groups of six or five to complete the tasks in the WebQuest and produce a report in one week’s time. In their projects, students were required to report on the common causes of infertility and the corresponding treatments, and to advise a couple having a specified infertility problem the possible ways to have children. Each group of students was required to hand in a group report in the form of a PowerPoint for assessment. A session of the WebQuest workshop was arranged on the first day for the students to browse and study the on-line resources and make preliminary discussions on the issue. Another session was arranged on the last day for them to evaluate their processed work, make modifications and finalise their reports. Both sessions took place in the computer lab of the school. Field observations were carried out during the two sessions of workshop to collect information on their behaviours in the WebQuest activities. Students were requested to complete a questionnaire after the WebQuest activities to reflect on their own learning outcomes. One member from each group was then invited to attend a focus group interview in which additional information as well as confirmation of the data collected from the questionnaire were sought. Students’ completed projects were sent to three experienced biology teachers who were required to rate them basing on a list of cognitive criteria, and related information was obtained through an evaluation questionnaire. Findings and Results The student questionnaire required students to rate their own learning outcomes in three categories: collaboration (CL), affective outcomes (AF) and cognitive outcomes (CG). The data obtained were analysed with SPSS and the descriptive statistics of the results are shown in Figure 1. A mean score of more than 3 indicates a positive feedback or a gain in that quality. According to the statistical results, mean scores of over 3 are obtained for items in all categories. The items were also analysed in groups. In order to obtain an overall picture and to find out the extent to which the items were correlated to the underlying factor(s) in each category, factor analysis was performed. Items that did not measure learning outcomes, such as the level of leadership, were not included in the groupings. A mean score was then calculated for each extracted factor basing on the loadings of the corresponding items for comparison across the categories. The results are shown in the Figure 2 below: According to the results of analysis, it is found that students tended to think that they gained most in the sophisticated cognitive abilities through the WebQuest activity – the abilities that were intended to develop in the WebQuest activity. Another advantageous feature of

K.-l. Tse and S.-w. Pun / Effectiveness of WebQuest in the Teaching of STS in Secondary Biology

311

WebQuest to promote collaboration is also revealed from the results. On the other hand, the achievements in general feeling and attitudes towards such learning experience and the general cognitive skills developed are less prominent.

ʳ

Figure 1

Graphical representation of the result of the student survey

ˇˁ˃˃

ˠ˸˴́ʳ̆˶̂̅˸

ˆˁ˃˃

˅ˁ˃˃

˄ˁ˃˃

˃ˁ˃˃ ˖˟

˔˙

˖˚ʳ˄

˖˚ʳ˅

˙˴˶̇̂̅̆

Figure 2

Graphical representation of the mean scores for categorical factors

312

K.-l. Tse and S.-w. Pun / Effectiveness of WebQuest in the Teaching of STS in Secondary Biology

The results of student survey showed that WebQuest could enable collaboration, promote positive attitudes and behaviours towards project learning and develop cognitive abilities, especially those sophisticated decision making and problem solving skills. In the focus group interview, it was discovered that the most serious obstacles to the WebQuest project learning were the workloads from concurrent homework and tests from other subjects and the lack of class time for web-based activities. “But, despite the fact that we had been working from the beginning, we still felt that time was insufficient,¨ said a student “Because there were many other things, a lot of homework to do.” The positive effects on collaboration and other aspects of the affective domain were supported by the information obtained from field observations while the gains in cognitive skills were supported by teachers’ assessments on students’ projects. The teacher evaluators generally considered the task non-demanding when it was carried out with the help of WebQuest. They pointed out that the clear instructions and rich guidelines in the WebQuest had helped students learn by their own ways as well as the science concepts. One teacher favoured the scaffolding feature of WebQuest. She thought that the guidelines and Web links provided had facilitated students to locate information and organise their work. Conclusion and Recommendation WebQuest is effective in achieving the intended outcomes for the teaching of STS in biology in the context of this case study. First, it helps promote collaboration and other positive attitudes and cognitive behaviours. Second, it helps develop higher order thinking skills, especially the decision making and problem solving skills which are essential for life-long learning. Finally, the rich scaffolding features in the WebQuest help students in organising their work as well as improving relevance of their project contents. It is suggested that the applicability of WebQuest to other learning scenarios of STS such as debate, role-play, etc. and even cross-science curriculum may be worth further explorations. However, before the WebQuest strategy can be well incorporated into the school curriculum, we need to develop among our students a positive attitude towards project learning and make them aware of the importance of the skills learnt. References Aikenhead, G. S. (1994). Consequences to learning science through STS: A research perspective. Chapter in J. Solomon & G. Aikenhead, STS education: International perspectives on reform. New York: Teachers College Press. Dodge, B. (2001). FOCUS: Five rules for writing a great WebQuest. Learning & Leading with Technology, 28(8). Curriculum Development Council. ( 2005). Biology Curriculum Guide (Secondary 4-5). Hong Kong Special Administrative Region, China: Printing Department. King, K. P. (2001). Examination of the Science-Technology-Society approach to the curriculum. Retrieved November 12, 2005, from http://www.cedu.niu.edu/scied/courses/common_files/sts_overview.pdf McCann, W. S. (2000). Teaching about societal issues in science classrooms. ERIC Digest. Retrieved November 16, 2005, from http://www.ericdigests.org/2000-1/societal.html Newman, I. & Benz, C. R. (1998). Qualitative-quantitative research methodology: Exploring the interactive continuum. Carbondale, Ill.: Southern Illinois University Press. NSTA (1990). Science/Technology/Society: A new effort for providing appropriate science for all. Retrieved November 16, 2005, from http://www.nsta.org/positionstatement&psid=34 Wonacott, M. E. (2001). Technological Literacy. ERIC Digest. Retrieved November 16, 2005, from http://www.ericdigests.org/2002-3/literacy.htm

Mobile and Web-Based Learning

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

315

Effects of Using Digital Contents Designed for PDA as a Teaching Aid in an Observational Learning of Planktons for Fieldworks on a Ship Hitoshi MIYATA Mitsuo Ishigami Faculty of Education, Shiga University, Japan [email protected] [email protected] Abstract. We developed “The World of Planktons in Lake Biwa,” which is digital contents for use as a teaching aid in the observational learning of planktons that is conducted aboard a ship as part of science lessons. This content consists of a search page with 350 still images to be used to search and identify over 200 types of planktons living in Lake Biwa; pages providing 45 video clips of actions that are difficult to observe directly, such as the hatching of plankton eggs and cell division; pages explaining cell division; and a “digital sketch” page on which students can upload and publish hand-drawn pictures. The developed content was used in actual lessons for fifth-grade primary school students observing planktons aboard a ship. We validated the effectiveness of the digital content from the perspectives of (1) “interest, motivation, and attitude”; (2) “thinking and judging”; (3) “technique and expression”; and (4) “knowledge and understanding.” Keywords: Digital Contents for PDA, Web-based Database System, Improvement of classroom instructions through the use of ICT, Video-on-demand contents by wireless LAN, Supports for observational study in science education

Introduction During the “Symposium on Informatization of Education in Primary and Secondary Schools,” hosted by the Ministry of Education, Culture, Sports, Science and Technology, Shimizu (2004) stated that in order to promote the informatization of education, it is helpful to demonstrate that students’ scholastic performance will improve through the use of information technology (IT) [1]. Numerous investigations (Yamamoto et al. 2003) have pursued the potential ways in which classroom instructions can be enhanced through the use of multimedia teaching aids and digital contents, including video clips that have been designed for learning purposes [2]. Consequently, we decided to work with the Education Committee of Shiga Prefecture (Japan) to develop digital content to assist in the observational learning of planktons that is conducted aboard a ship in Lake Biwa. Mobile phones and Pocket PCs have become very popular in the last few years due to the increase of availability of different wireless services [3][4]. These small, but powerful Mobile devices among other possibilities enable GPRS(General Packet Radio Services) Internet and Pocket PCs with enabled wireless Internet communication [5][6]. In that way, they influence the need for developing Internet based software adapted for Mobile and Pocket PC devices [7]. It should be mentioned the fact that the cost of the recent mobile devices and the cost of GPRS Internet is still a little bit high, so the usage of mobile phones like information medium is rational consideration. The Mobile and Pocket PC Web-access to the e-Book has standard functionality like searching and viewing information about Planktons etc., viewing images, audio, video files and electronic materials. With this solution, the access to data is faster than before. Students can access from any place and any time if they have GPRS service on their mobile phones

316

H. Miyata and M. Ishigami / Effects of Using Digital Contents Designed for PDA

and wireless access in the environment where are they located for the Pocket PCs even on the deck of the boat on the wide lake. This Mobile and Pocket PC access to the e-Library offers new possibilities for teachers and students in a so called wireless campus of advanced education. 1. Objective of Research This research has two objectives. The first is to develop “The World of Planktons in Lake Biwa,” which is digital content for use as a teaching aid in the observational learning of planktons in Lake Biwa. The second is to validate the effectiveness of the developed content by applying it to an actual plankton observation class conducted aboard a ship; this class comprised of fifth-grade primary school students. More specifically, the effectiveness of the content was considered from the following four perspectives: 1)

Whether the students were able to develop a more intimate relationship with nature and further enhance their positive attitudes toward motivated investigations by promoting the use of IT in observational science lessons (“interest, motivation, and attitude”); 2) Whether there was a broadening of learning activities through the search for information not only in library books but also in multimedia encyclopedias or the Internet after the actual observation of planktons (“thinking and judging”); 3) Whether the students have learnt to become more creative in their use of tools and machines after the experience of using a microscope and personal digital assistant (PDA) in the observational learning (“technique and expression”); 4) Whether it was possible to help students identify planktons and enable them to understand phenomena that are difficult to observe directly by making them view video clips demonstrating the hatching of eggs and cell division of planktons as part of the science lessons (“knowledge and understanding”). 2. Research Method 2.1. Investigation of requirements prior to content development We boarded the learning ship in February 2004 and surveyed the current curriculum for the observational learning of planktons and investigated the requirements for improving the course instruction by interviewing six primary school teachers and two field advisors. 2.2. Methodology for validating the effectiveness of the content As a method for validating the effectiveness of the content, we conducted a questionnaire survey on the students based on three of the abovementioned four perspectives: (1) interest, motivation, and attitude, (2) thinking and judging, and (3) expression and technique. We also set an objective paper-based test to measure (4)—the students’ knowledge and understanding. In addition, we surveyed the students’ opinions on general aspects of the lessons that utilized digital content from such standpoints as “feeling of being part of the activities,” “satisfaction,” and “self confidence.” Further, we distributed a checklist to the teachers that was to be used to observe the students and collected the recorded observations. We also conducted interviews with the teachers who instructed the activities; these interviews pertained to any changes that they might have noticed in the students’ behaviors. In conjunction with the above analyses, we also examined the learning portfolio created by the students, including sketches drawn from their observations.

H. Miyata and M. Ishigami / Effects of Using Digital Contents Designed for PDA

317

3. Content and functionalities of the digital content titled “The World of Planktons in Lake Biwa” Based on the results of prior investigations, the number of planktons included in the content for this research was increased. There were also requests to create video clips of the actions for which direct observation would be problematic. The following items and f unctionalities were included in the content: (1) color photographs of more than 200 types of planktons living in Lake Biwa, as well as a search function allowing the students to search for the names of the planktons not only using keywords but also based on the shape and visceral image perceived by the students; (2) interest-provoking video clips of phenomena that are difficult to observe, such as the division of plankton cells and hatching of plankton larvae; and (3) a portfolio functionality in which students can store hand-drawn observational sketches and notes recorded on their PDAs on a network server; this can be used for learning through the sharing and exchange of ideas. Figure 1 shows some photographs and video clips as displayed on a PDA. 4. Architecture and Implementation of the system The architecture of the web-based e-Book system is shown on Figure 2. As can be seen from the Figure 2, the architecture is classical client-server architecture, providing different communication ways between the client and the server. The server includes the database, PHP module and web-server. The web-server communicates with the database in both directions over the PHP module. The client sends a request to the server, so server accepts that request and sends back the requested data to the client. The received data by the server are processed to new-web page. The client-server communication for Pocket PC can be via Wireless LAN, based on the 802.11b standard. The mobile client communicates with server via GPRS Internet. Figure 2. Architecture of the web-based e-Book system

318

H. Miyata and M. Ishigami / Effects of Using Digital Contents Designed for PDA

Implementation of the system For full functionality, this system requires server and client. The server side contains database with some stored procedures and functions developed in Oracle 9.2i, and a web server Apache which is an integral part of Oracle 9.2i. The web server contains the source code of the web based application developed with PHP 5 together with HTML using the Macromedia Dreamweaver MX 2004. One of the Clients on Figure 3 is Pocket PC which works on Windows CE platform and uses Internet Explorer. The communication between client and server is typically via wireless Internet. The Pocket PC has integrated wireless card and communicates with web based application via access point. The setting of the Pocket PC for this communication must be appropriate, and the user must have enabled entrance to the access point. The Second Client in this case is Mobile phone which works on Symbian OS platform and uses Opera Web browser. The communication between client and server is typically via GPRS Internet. The setting of the Mobile device for this communication must be appropriate and the user must have enabled a GPRS service by the mobile operator. For correct display of the Cyrillic characters on the Opera browser, on the Preferences the encoding must be set to "Cyrillic (Windows-1251)".

5. Use of the digital contents in actual lessons for fieldworks A preliminary investigation was conducted with 80 students on May 24, 2004 in order to obtain feedback on the usability and level of difficulty of the content. The actual “production” lessons were conducted on July 8 and 15 with fifth-grade students. Students from Schools A and I (Group A) and Schools H and S (Group B) participated in the lessons. Each group spent two days aboard the learning ship “Uminoko,” which was provided by the Education Committee of Shiga Prefecture. A total of 320 students (165 boys and 155 girls) from primary schools in Shiga Prefecture participated: 157 students in Group A and 163 in Group B. For the observational learning, 20 groups were formed, each group consisting of 40 students. Students learned in pairs, and each pair shared one microscope and one PDA. In terms of the duration of the learning activities, 90 minutes were allocated for collecting and observing samples of planktons. The flow of learning activities for Groups A and B are shown in Figure 4. The digital content was used during the observation in the southern lake. There was no significant difference in the scores of the basic science aptitude test between

H. Miyata and M. Ishigami / Effects of Using Digital Contents Designed for PDA

319

the groups and in the measure of affinity to science. Therefore, the groups were considered as being homogeneous. The observation of planktons using a microscope was performed in pairs, with the students taking turns at using the microscope and attempting to identify the names of the planktons. The teachers did not reveal the correct names of the planktons but gave advice on how to look up the names. When the learning activities were concluded, the students completed an attitude survey, a questionnaire for evaluating the learning activities, and an objective test on the knowledge gained from the learning experience.

6. Results and Discussions 6.1. Students’ evaluation of the digital contents Table 1 shows the result of the Table 1. Students’ evaluation of the contents investigation on the students’ (number of respondents) (N = 320) opinions of the developed digital Good Bad No answer content titled “The World of Overall usability 316 2 2 Planktons in Lake Biwa” with respect Clarity of photographs and 315 3 2 to the clarity of the images, usability, Appropriate display size 316 2 2 317 1 2 and method of plankton identification. Identification using shapes 2 Most of the students replied positively Ease of operations for sketching 305 13 300 18 2 to the questions pertaining to the Ease of operations for posting comments usability of the content and the clarity of the pictures and video clips. Most students responded that it was easy to use the shape-based search functionality to identify plankton names. However, there were also students who responded that it was difficult for them to operate the feature to make sketches of planktons and the functionality to post observational comments on the server. Thus, there is a need to further improve the usability of these functionalities. Figure 5 shows a scene from the plankton observational learning at the cabin in the ship.

320

H. Miyata and M. Ishigami / Effects of Using Digital Contents Designed for PDA

6.2. Validating the effectiveness of the content 6.2.1. Concerning knowledge and understanding Table 2 shows the results of the objective test conducted to measure the students’ knowledge and understanding after the learning experience. The test problems consisted of (1) identifying the name of the planktons by looking at their photographs (8 questions); (2) selecting the correct diagrams depicting the hatching of plankton eggs (4 questions); (3) selecting Figure 5. A scene from the plankton observational learning either “True” or “False” for diagrams depicting plankton cell divisions and other plankton behavior accompanied by an explanation (4 questions); and (4) free-text responses on any observations made during the study of planktons. We performed a 2 × 2 analysis Table 2. Results from objective tests for of variance (ANOVA) with two measuring knowledge and understanding intra-group and two inter-group (number of points) relations. The results revealed a statistically significant interaction Group A Group B Inter A-B using a significance level of 1%. Day 1 46.1 69.9 *p < .05 Multiple comparisons of the mean Day 2 75.8 60.2 Ns scores were conducted using the least Intra-group **p < .01 Ns significant difference (LSD) method. A statistically significant difference in the test scores for Group A was observed between Day 1, when printed sample sheets were used, and Day 2, when the digital content was viewed on a PDA. In addition, there was a significant difference in the scores after Day 1 between Groups A and B, which used the printed sample sheets and the digital content viewed on a PDA, respectively. However, there was no significant difference between Day 1 and Day 2 within Group B, in which PDA content was used on the first day and printed sample sheets on the second day. The above results can be interpreted as follows. We analyzed the reason for the Group A’s low scores after Day 1 by focusing on the individual test questions. Further, as shown in Figure 6, the average scores for the questions related to identification of the planktons, hatching, and cell division were all lower for Day 1. Moreover, 32% of the students left at least one question on the identification of plankton names unanswered. On Day 1, color-printed sample sheets were used for looking up the names and types of planktons; however, the sheet contained only 12 representative plankton types, and the students were asked to write “other planktons” if they were unable to identify the name of the plankton. In addition, the sample sheets had the planktons photographed only from camera angles that made it easy to distinguish the characteristics of the planktons, similar to the photographs used in textbooks. The actual planktons that the students observed through the microscope were moving, thus making the angle of observation different from the camera angle of textbook photographs. We observed actual situations in which identification proved to be difficult. In contrast, on Day 2, the percentage of students who did not answer the questions pertaining to identification reduced to 8%; we believe that this led to an increase in the average score. Using the content viewed on the PDA, the students could easily identify the planktons based on their shapes, such as “circular,” “chained,” or “stick shaped,” and they

H. Miyata and M. Ishigami / Effects of Using Digital Contents Designed for PDA

321

were able to observe and compare planktons from various angles by viewing the video clips. We inferred that this facilitated the students’ knowledge acquisition and understanding. This inference was also confirmed in free-text statements obtained from the students after the lessons, such as “it was easy to search by the shapes” and “I saw that raft-shaped planktons look like a needle when viewed obliquely from below.” As seen in Figure 6, although there was no significant difference between the average scores for the questions related to hatching and cell division, there was an increase in the scores between Day 1 and Day 2. By analyzing the students’ free-text responses, we were able to confirm statements such as “the explanation was easy to understand as we could see from the video clips how plankton larvae are hatched from the eggs” and “I understood the meaning of ‘copying oneself’ by looking at how planktons divided by viewing the video clips.” The video content was noticed to supplement those aspects of learning that are difficult to grasp merely by looking at the printed images of the planktons on the sample sheets. Figure 6. Analysis of test questions

6.2.2. Thinking and judging The results of the questionnaire asking the students to evaluate the learning content using a four-point scale are shown in Figure 7 and Figure 8 for Groups A and B, respectively. The result of the ANOVA with a 5% significance level revealed a significant interaction. Therefore, we performed a multiple comparison of the mean values using the LSD method. It was clarified that in Group A, the evaluation of the learning activities when the content was viewed on the PDA was statistically significantly higher compared to that when printed sample sheets were used for the “thinking and judging” questions (e.g., “Were you able to think about the hatching and cell division processes of planktons by yourself?” and “Were you able to form your own opinion based on your understanding of the observations?”). The free-text responses by the students included “It 2TKPVU &C[ 㧼㧰㧭 &C[ became clear to me that the hatching of tadpoles and hatching of planktons 㪫㪼㪺㪿㫅㫀㫈㫌㪼㪆㪜㫏㫇㫉㪼㫊㫊㫀㫆㫅 have certain similarities” and “I 㪋 understood that cells can divide into 㪊㪉 two or into 16. I understood that you 㪈 have to start by dividing into two before you can divide into 16.” 㪠㫅㫋㪼㫉㪼㫊㫋㪆㪤㫆㫋㫀㫍㪸㫋㫀㫆㫅㪫㪿㫀㫅㫂㫀㫅㪾㪆㪡㫌㪻㪾㪼㫄㪼㫅㫋 Answers like these confirmed the students’ deep and broad understanding. Figure 7. Group A’s profile of Interest, technique, and thinking

322

H. Miyata and M. Ishigami / Effects of Using Digital Contents Designed for PDA

6.2.3. Interest and motivation, and technique and expression

2TKPVU &C[

㧼㧰㧭 &C[

There were no significant 㪫㪼㪺㪿㫅㫀㫈㫌㪼㪆㪜㫏㫇㫉㪼㫊㫊㫀㫆㫅 differences in the measure of interest 㪋 and motivation and technique and 㪊 expression: both these measures were 㪉 high. This result can be attributed to 㪈 the fact that the microscope was used 㪠㫅㫋㪼㫉㪼㫊㫋㪆㪤㫆㫋㫀㫍㪸㫋㫀㫆㫅㪫㪿㫀㫅㫂㫀㫅㪾㪆㪡㫌㪻㪾㪼㫄㪼㫅㫋 in both activities and the incorporation of a hands-on observational learning style helped to promote the students’ interest and motivate them. In addition, the measure for technique and expression Figure 8. Group B’s profile of Interest, technique, and thinking in the case in which the PDA was used was only slightly higher compared to the case in which printed sample sheets were used. The students’ free-text responses revealed that they had more difficulty focusing the microscope and handling the slide glass. Further, the ambiguity of the questions pertaining to “the use of the PDA and the microscope” may have affected the results. Based on the above results and discussions, we were able to validate the effectiveness of the digital content that we developed in the areas of knowledge and understanding, thinking and judging, and interest and motivation. We plan to extend the scope of this research by conducting a detailed investigation through which we will be able to distinguish between the operation of the microscope and the hand-written sketching feature on the PDA with respect to technique and expression, thereby improving the content. References [1] Shimizu Y., “Improving the education effectiveness through the use of ICT”, Proceedings of the 2nd Conference on Informatization of Primary and Secondary School Education, 2004, pp. 1–13. [2] Yamamoto T., Ikeda Y., and Shimizu Y., “The effects of using videos for instruction of physical education ‘jumping box’”, Japan Society for Educational Technology Journal of Papers, 27(S), 2003, pp. 153–156. [3] J. S. Pierce, H. Mahaney, Opportunistic Annexing for Handheld Devices: Opportunities and Challenges (Georgia Institute of Technology, 2004). [4] R. Laberge, S.Vujosevic, Building PDA Databases for Wireless and Mobile Development, Wiley Publishing, Inc., Indianapolis, 2003. [5] K. Anderson, GPRS - A key step for the mobile Internet (WirelessDevNet.com, August 27, 2002). http://www.3gamericas.org/English/News_rom/published_articles/gprs_key_step.cfm [6] S. Buckingham: What is General Packet Radio Service(GPRS)? (Mobile Lifestreams Ltd, 2000). http://www.gsmworld.com/technology/gprs/intro.shtml [7] Kinshuk & T.Goh, Mobile Adaptation with Multiple Representation Approach as Educational Pedagogy (In W. Uhr , W. Esswein & E. Schoop (Eds.), Wirtschaftsinforma-tik, Dresden Germany, 2003). A part of this research was conducted as part of the research titled “Investigative Research for Improving Course Instruction through the use of ICT” (representative: Shimizu Y.), commissioned by the Ministry of Education, Culture, Sports, Science and Technology in Japan for 2005.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

323

A Study of Message Reading Efficiency of Color Screen Mobile Phones Xuemin Zhanga, Bo Wanga, Lilin Raoa, Bin Yanga, Yongna Lia, Xueming Lua Beijing key lab of Applied Psychology, National key lab of Cognitive Neuroscience and learning, School of Psychology, Beijing Normal University, China [email protected]

a

Abstract: This study explored the message reading efficiency of color screen mobile phones from the aspects of screen background and Chinese text message presenting style. Our results showed that although both background and TIPS affect message reading efficiency respectively, these two factors intervene in each other. A suggested optimal combination to expedite message reading is to present text message in a scrolling-by-page style matched with a blank white background. This study may provide value to the design of mobile phone message reading, mobile phone net play and other small screen HCI, such as palmtop computer, electronic dictionary and etc. Keywords: Color screen mobile phones, message reading, reading efficiency, background, text information presenting style (TIPS), Chinese

1. The research question As the development of the HCI designing, the limitation of the small screen of mobile phones is becoming increasingly serious. The small screen results in small output graphs, fonts and few words, such limitations bound to lead to the context incoherence and reading comprehension obstacle which increases the users’ workloads on cognitive processing [1]. The aim of this study is to look for an optimal message presentation to expedite message reading, especially designed for markets with Chinese speaking consumers. Dillon et al found that, even though only a few lines of words could be displayed on small screens, there was no significant obstacle in terms of reading and understanding information [2]; Jari Laarni contrasted three kinds of small screen palmtop mobile devices, PDA, communicator and mobile phones, and found that the reading efficiency was better when the information was scrolled gradually than by a whole page once [3]; Max Melchior’s research showed that line-hinted presenting style could make the usage more convenient [4]; Wang Anxiang et al, studied the best combination of the speed and interval between two moves and its effect on the optical fatigue using information presented in scrolling style on big screen, which conclusion of great value to the tech-parameter of scrolling presenting style [5]; Zhang Xuemin, He Li et al, by their synthetical study to the visual design of black-and-white mobile phone, found that the operating efficiency was best when green font color matched with bright blue and small font matched with small icons [6]. However, the previous studies are limited with: 1) No Chinese text has been used to present information. Chinese characters are rather related to images than phonic that features in many alphabet languages. It’s remained unanswered that how the way Chinese text presented affect the speed that process such information; 2)results are based on other mobile devices instead of modern color screen mobile phones feature colorful screen which can present true color pictures and may exert many effects on users.

324

X. Zhang et al. / A Study of Message Reading Efficiency of Color Screen Mobile Phones

In our study, we chose color screen mobile phones that were commonly used by public nowadays, displaying Chinese characters as text messages, to explore the effects of TIPS and background.

2. Experimental Method 61 subjects were used, aged from 18 to 25 with normal color perception; Standard computer and mobile phone simulator (Figure on the right); 48 messages from different fields were used, all matched and standardized; Two kinds of backgrounds were applied: picture background (seaside picture) and white background without any picture. The research applied 2ͪ3 within-subject experiment design. The two factors were screen background (picture background or white background without any picture) and TIPS (scrolling by line, one-line-at-a-time (өstyle); scrolling by page, one-page-at-a-time (Ӫstyle), and manual scrolling up and down, constant scrolling (ӫstyle)), which were both within-subject factors. The operation was done with mouse. The message scrolled one line or page when “up” or “down” was clicked once in the former two styles; inӫstyle, when “up” or “down” was clicked, the message kept scrolling until the mouse was unheld. The messages were displayed randomly. Before the experiment, the subjects were required to do exercises, and when finishing reading, they were required to answer three questions and the reading time for each message referred to the interval from the appearance of the message until they pressed the space key to finish reading.

3. Results Data with the accuracy under 80% and reaction time less than 1s or more than 60s were abandoned. 38 subjects were left with mean accuracy 89.02%. The average reading time in seconds for one message (ms) with different message presenting styles conjunct with different screen background could be seen in Table 1. Table1 Average reading time in seconds per message with different TIPS conjunct with different screen background (ms) ¢ £ ¤ ¢ £ ¤ Non-picture Non-picture Non-picture Picture Picture Picture Average 18740.49 15608.43 17475.26 16410.91 19324.25 16479.84 SD 8101.92 4342.20 5895.74 4714.98 5603.48 5704.89 In different TIPS, analysis of variance in different backgrounds showed that, in blank background, different TIPS had significant main effect on reading efficiency, F(2 ˈ 38)=5.691 ˈ P<0.005. According to results of further multiple comparison, the reading efficiency in £style was significantly better than that in¢and¤styles between which there was no significant difference. RT one line- RT one page=3132ms(P<0.05)˗RT one line- RT scrolling=1265ms(P>0.05)˗RT one page- RT scrolling=-1866ms (P<0.05). Treatment

X. Zhang et al. / A Study of Message Reading Efficiency of Color Screen Mobile Phones

325

In picture background, TIPS still had significant main effect, F(2ˈ38)=10.310ˈP<0.001. However, opposite to what we had found in non-picture background, the reading efficiency in£style was significantly lower than that in ¢and¤styles, which was in the accordance with Jari Laarni’s conclusion, although there was no significant difference between¢and ¤styles. RT one page- RT one line=2913ms (P<0.05)˗RT one line- RT scrolling=-69ms(P>0.05)˗RT one page- RT scrolling=2845ms (P<0.05), in picture background. In different TIPS and backgrounds, the interaction between backgrounds and TIPS was significant, F(5ˈ76)=16.315ˈP<0.001. Table 2 T test to the matching sample of the two backgrounds in different reading styles ¢ £ ¤ T 2.140 -4.946 1.650 P 0.039* 0.000*** 0.107 T test was done to investigate background‘s effect on reading efficiency in the same TIPS. The statistical results are shown in Table 2. In¢style, the reading speed in picture background was significantly faster than that in non-picture background; in£style, the reading speed in picture background was significantly lower than that in non-picture background; And in¤style, there was no significant difference. To conclude this study, reading efficiency is significantly affected by the combination of background and TIPS. Based on this study, an optimal combination to expedite reading speed is to present text information in one-page-at-a time style on a blank background.

4. Discussion When Chinese text was presented in¢style, the reading efficiency with picture background was significantly improved than that with blank background. This result supported that, in Į style, users processed the background picture and text messages separately and took the background picture as spatial clue, which saved cognitive energy and promoted their reading speed. Secondly, when Chinese text was presented in¤style, the difference of reading efficiency was not significant between picture background and blank background. In¤style, no matter whether there was background picture, subjects had to control the movement of the message by repeatedly and continuously pressing the key where the subjects were likely to get lost and had to search the connection point renewably. It seemed that the interference of background in message reading was overwhelmed by the distraction of tracking texts in such TIPS. This hypnosis is also supported by the result that there was no significant difference in reading efficiency between¢and¤styles matched with whichever backgrounds. Lastly, when Chinese character presented in Ӫ style, the reading efficiency with picture background was significantly lower than that with non-picture. In£style, the subjects didn’t have to switch lines frequently so that stay reading for a longer before scrolling down, the text became relatively invariable. ; Under such circumstance, the re-appearance of background picture every time when switched pages played a major roll of distraction rather than line switching in other two TIPS. It suggested that the subjects might take a new processing about the background picture instead of processing the picture and messages separately, which would also interfere with the subjects’ working memory and resulted in low reading efficiency. As for the effect of TIPS on the reading efficiency in blank background, the above experiment result also showed that the reading efficiencies of¢and¤styles were basically consistent in blank background while both significantly lower than that in£style. A reasonable explanation lied in the similar operation, frequent line breaks and lack of spatial

326

X. Zhang et al. / A Study of Message Reading Efficiency of Color Screen Mobile Phones

clue of the first two styles and that the subjects had to again concentrate on the proper connection part of the message [4]; and some correlative research on eyes’ movement also showed that [7, 8] the spatial clue of the message contributed to a good reading efficiency. However, because of the movement of not only the subjects’ eyes but also the messages, the perceptive clue of the context was destroyed, which deprived of the subjects’ forecast to the rear part of the message, took more time for them to find the connection point and caused the getting lost phenomenon [4]; and thus there was a reduction in the reading efficiency which in these two styles was not significantly different but lower than that in£style. In£ style, since there was no background picture, the subjects concentrated on all the messages in the full screen and pressed less and wouldn’t scroll until they had understood the message; Therefore the getting lost phenomenon decreased, which led to a faster reading. In picture background, opposite to the result in blank background, the reading efficiency of both Į and Ȗ styles was significantly better than that in£style. It was thus clear that background picture had interaction on the effect of the three reading styles. The main reason for this was probably that, in¢style, the effect of background picture on the reading efficiency was not significant and the subjects mainly focused their attention on frequent line breaks and the constant rolling message and the picture, and, as invariable spatial clue, the picture in a sense overcame the incoherence caused by the message movement[13,14], which increased the reading speed. However, in£style, the message was presented on a comparatively invariable background where the message did not move, which required the subjects to pay more attention to distinguish the message from the picture; and when eyes moved to a new line, it was not only the characters but also the pattern under them that were new stimulus to the subjects, which made it difficult for subjects to distinguish the characters from the pattern and, at the same time, directly influenced the working memory; therefore, the background picture, to some extent, interfered with the reading efficiency and resulted in its reduction. Moreover, the picture in the research was a seaside landscape and in its lower part was sand pattern. In¢style, the light-colored sand was the equal of white background, which, instead of interfering with the message reading, made the new line more striking and improved the reading efficiency.

References [1] Brewster,S.A.,Murray,R.(2000). Presenting dynamic information on mobile computers. Personal Technologies.2000. 4 (4) :209-212 [2] Dillon,A.,Richardson,J.& McKnight,C. (1990). The Effect of Display Size and Text Splitting on Reading Lengthy Text from the Screen. Behaviour and Information Technology. 9(3):215-227 [3] Jari Laarni. (2002).Searching for Optimal Methods of Presenting Dynamic Text on Different Types of Screens. 2002.10 [4] Max Melchior. Perceptually guided scrolling for reading constant text on small screen devices. In Dunlop and Brewster (Eds:), Proceedings of Mobile HCI 2001: Third International Workshop on Human-Computer Interaction with Mobile Devices. Available at: http://www.cs.strath.ac.uk/ ~mdd/ mobilehci01/procs/(accessed on 2002-6-29) [5] Wang Anxiang, etc. (2002). The effect of pre-guiding dynamic information presentation design on users’ visual efficiency and optical fatigue. Journal of the Chinese Institute of Industrial Engineers, 2002.19(2): 69-78 [6] Zhang Xuemin, He Li, etc. (2004). A Study of Display Efficiency of Mobile User Interface Design. Applied Psychology. 2004,10ΰ1α:33-38 [7] Annie Piolat. Effect of screen presentation on text reading and revising. Human-Computer Studies, 1997, (47), 565-589. [8] Baccino,T.& Pynte,J. Spatial coding and discourse models during text reading. Language and Cognitive Processes, 1994, 9, 143-155.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

327

The Design of a Web-based Learning Platform: A Case Study in Taiwan a

b

I-Fan Liu , Meng Chang Chen , Yeali Sun a

a

Department of Information Management, National Taiwan University, Taiwan b

Institute of Information Science, Academic Sinica, Taiwan [email protected]

Abstract: In this study, we develop a Web-based learning platform to assist students taking the “Supply Chain Management” course on the Internet. The design is based on Cooperative Learning Theory and the system function models developed by the authors. We also investigate the function utilities required by students who use the platform. The results show that, in descending order, the utility rates of the three function categories are: (1) Functions of Cooperative Learning (e.g., online tutor, assignment discussion room, real-time discussion room, and online assignment hand-in) equal 57%. (2) Functions of Personal Learning (e.g., learning resource sharing, personal learning notes, video instructions, and viewing of peer-learners’ assignments) equal 38%. (3) Functions of Web-based Course Evaluation (e.g., online exams and peer review), equal 5%. Based on these results, we propose some concrete suggestions for five aspects of web-based learning, namely, the Learners’ Aspect, the Web-based Course Aspect, the Evaluation Aspect, the System Aspect, and the Educators’ Aspect. Keywords: Cooperative learning theory, Supply chain, Web-based learning platform

Introduction In many business and engineering schools in universities, Supply Chain Management (SCM) is one of the important courses for students who major in Industrial Engineering or Information Management. The course attempts to understand the impact of the business environment on the supply chain, and related supply chain management techniques, such as the use of optimization methods and planning of mathematical models.

1. Literature Review Cooperative Learning is defined as participation in a certain activity by two or more people, where each member communicates and coordinates in order to achieve his/her goals through interaction and cooperation [1]. Li [2] posits that learners exchange ideas and learn from each other, which creates useful knowledge during the learning process. Research by Johnson & Johnson [3] indicates that cooperative learning helps raise

328

I-F. Liu et al. / The Design of a Web-Based Learning Platform

learners’ performance, which not only makes learners more willing to spend further time studying the content of a course, but the motivation to learn and the effect of learning are also enhanced at the same time. The research of Zhang [4] indicates that learners can express their ideas and identify their mistakes through authentication during cooperative learning, so that they can construct new viewpoints. The major objectives of Cooperative Learning are to increase interactivity through cooperative behavior and emphasize positive independence [5]. In this study, we design a web-based learning platform to assist students in learning “Supply Chain Management” on the Internet. The design is based on Cooperative Learning Theory, and focuses on the system function models developed by researchers. Our research question is: How can we determine the system functions required by students who use the web-based learning platform? This research integrates learning theory with practical system development to provide a different perspective on the design of web-based learning platform.

2. System Design 2.1 Model Design of SCM The supply chain management system is comprised of a 3-tier architecture, as shown in Figure 1. The client-tier contains four models, which are responsible for the display of user interfaces, handling maps, and communicating with the middleware. There are eight models in the middleware. They are responsible for a large number of mathematical calculations, the results of which are sent to the client-tier or to the database for storage. The back-end database server is responsible for the protection and storage of information, and also provides the middleware with an application program server for retrieval and update.

Figure 1 Framework of SCM with Different Function Models

Figure 2 Framework of the Web-based Learning Platform

I-F. Liu et al. / The Design of a Web-Based Learning Platform

329

2.2 Design of the Web-based Learning Platform This platform’s design is also a 3-tier architecture, divided into the front-end (client-tier), middleware (application server), and back-end (database server), as shown in Figure 2. When this platform is being developed, the number of learners who will be online at the same time should be considered in order to minimize the load balance of the system, and improve its usability and reliability. 2.3 Functionalities of the Web-based Learning Platform The web-based learning platform is divided into three main function categories: (1) Functions of Personal Learning (a) Learning resource sharing (b) Personal learning notes (c) Video instruction (d) Viewing peer-learners’ assignments (2) Functions of Cooperative Learning (a) Online tutor (b) Assignment discussion room (c) Real-time discussion room (d) Online assignment hand-in (3) Functions of Web-based Course Evaluation (a) Online exams (b) Peer review 3. Research Methodology In this research, we used related learning theory to design and build a web-based learning platform, which is implemented in a network environment and applied practically in a course of study. Thirty graduate students from a university in Taiwan participated in the study, which was based on a course called “Supply Chain Management”. Two-thirds of the time was devoted to lectures, i.e., face-to-face teaching activities in the classroom, and the students spent the remaining time studying on the Internet. The research was carried out from September 2005 to January 2006 – a period of 16 weeks. 4. Data collection The following data was collected for this research: (1) basic personal files of the 30 students who logged into the system; (2) learning portfolio files; and (3) Web-based questionnaires. The demographics of the 30 students who took the course were as follows. (1) 63% were male and 37% were female. (2) The average age of the students was 25 years. (3) With regard to the method of Internet connection, 4 students used a modem; 10 used the campus network; and 16 used broadband. (4) In terms of place of residence, 13 students (43%) stayed in dormitories; 11 (37%) lived near the campus; and 6 (20%) lived within 5 km of the campus. (5) In terms of Internet usage, except for two students who used the Internet once every two days on average, all participants used the Internet daily. 5. Results According to the learners’ log-in file, the open-ended questions, and the results of the interviews, the students thought that some functions would be very helpful in web-based

330

I-F. Liu et al. / The Design of a Web-Based Learning Platform

learning. In this research, we specify three main function categories. The function categories in descending order based on their utility rates are: 1) Functions of Cooperative Learning (e.g., online tutor, assignment discussion room, real-time discussion room, and online assignment hand-in) equal 57%. 2) Functions of Personal Learning (e.g., learning resource sharing, personal learning notes, video instructions, and viewing of peer-learners’ assignments) equal 38%. 3) Functions of Web-based Course Evaluation (e.g., online exams, peer review) equal 5%. Thus, besides Web-based learning, students spend a substantial amount of time interacting and doing cooperative work with other students or the tutor via Internet. 6. Suggestions The major contribution of this study is that it integrates learning theory with a Web-based learning platform. Based on the results, we propose some concrete suggestions for five aspects of web-based learning. (1) Learners’ Aspect z Strengthen their computer skills, including their basic ability to use software or change the settings of a Web-based environment and their typing speed. (2) Web-based Course Aspect z Instructors should provide adaptable courses and different learning perspectives for students from different departments. The evaluation methods should also be tailored to students from different departments so that the students can be evaluated fairly. (3) Evaluation Aspect z Increase more indicators for evaluation among peer students. For example, students’ participation, sense of responsibility, degree of task completion, and contributions to the group. (4) System Aspect z Introduce a credit mechanism: Use a credit mechanism to offer extra credits, for example, students who help one another could get extra credits. (5) Educators’ Aspect z Educators should spend more time observing the students’ learning conditions online and help them when required; the observations should also be recorded in the teaching dairy. References [1] Slavin, R. E. (1990) Cooperative Learning: Theory, Research, and Practice, Englewood. [2] Li, Q. X., Ma, R. C. & Li, Z. N. (2002) Development of Web-Based Learning and Web-Based Schools and the Clash to Teachers’ Professional Performance, Information Education Magazine, 79, 2-12. [3] Johnson, R. T., & Johnson, D. W. (1994) Learning together in the social studies classroom, In R. J. Stahl, (Ed.) Cooperative learning in social studies: A handbook for teachers, Menlo Park CA: Addison-Wesley Publishing Company, 51-77. [4] Zhang, S. Z. (1999) Essentials of Practicing Teaching Materials Instructions – Methods and Research, Taipei, Wu-Nan Culture Enterprise Press. [5] Hooper, S. (2001) Cooperative learning and computer-base instruction, Educational Technology Research & Organization Manager’s Report, 19, 12-23.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

331

A Study of Implementing Web-Based Learning Systems to Enhance Learning for the Supply Chain Management (SCM) Course in Higher Education I-Fan Liua, Shelley S.-C. Youngb Graduate Institute of Information Systems and Applications National Tsing Hua University, Taiwan [email protected]

a&b

Abstract: This study explores how to adequately utilize the Information technology to enhance learning for a course entitled “Supply Chain Management” in a higher education setting in Taiwan. The preliminary data indicate that most of the students who have taken the design and utility of this web-base learning system were positive to the SCM. Finally, this paper discusses the related issues and also makes suggestions for future study. Keywords: Web-Based Learning System, Supply Chain Teaching, Cooperative Learning, Constructive Learning

Introduction Due to the development of the Internet, the method of learning is also undergoing some changes - the learning location is not only the physical classrooms, learning time is not restricted; it is because these phenomena are all transferred to the Internet, which do not only allow learning on the Internet with people from different backgrounds, but also achieve the effect of interactive learning through communication and discussion. The application of web-based learning in university studies has become a trend; however, the most common way at present is only to provide study materials online, which lacks the support of learning theory, and the interaction in learning between fellow students, so that the effect of learning cannot be improved effectively. Therefore, how to combine the related learning theories, and to integrate with digital study materials effectively, through simultaneous and non-simultaneous learning mechanisms, to develop a platform for interactive web-based learning system, is discussed to provide references for educators for future education.

1. Background of this study--Current Situation of Teaching Supply Chain Management Courses In many business schools and engineering schools of universities and research institutes, Supply Chain Management (SCM) is a very important course for students in Industrial Engineering or Information Management. This course aims at investigating the impact of business environment on the characteristics of supply chain, and the related supply chain management techniques, such as the use of optimization methods and planning of mathematical models. Because of severe competition in the market and shortening of

332

I-F. Liu and S.S.-C. Young / A Study of Implementing Web-Based Learning Systems

products’ life cycles, organizations have to focus on effective management of their supply chains. The objective of this course is to allow students to understand the concepts, strategies and management techniques of supply chain, so that they can be familiarized with organizations’ upper, middle and lower levels of chain behaviors and operating processes before they step into the job market.

2. Research questions This research designs a web-based learning system to assist in the teaching of “Supply Chain Management”. The viewpoint of design is based on the related web-based learning theories, and focuses on the system functions modules developed by the related web-based learning theories. Because of the needs of this web-based course, this research is going to investigate the following topics: x Investigate the effect of learning of students who use this supply chain management system. x Investigate the effect of learning of students who use web-based learning system platforms. x Investigate the effect of interactive learning through this system, and evaluate whether the system is integrated with learning theories. 3. Literature review The concept of “web-based learning” originates from long-distance learning. Early teaching technologies were facilities such as televisions, video recorders and computers, which were the main sources of teaching supplements. Since the Internet has become popular, web-based learning is considered directly as network training or online training [1]. Because the characteristics of the Internet are not restricted by time and space, the transfer of knowledge has become more rapid and effective. The phrases generated that are related to “web-based learning” include web-based education, online learning, and even considered as synonyms [2][3]. This research is carried out through web-based learning system, and focuses on online learning, which generally implies the learning methods on the Internet. 4. Research Methodology This research is based on the knowledge structure of related literature studies for web-based learning theories, to design and build system functionalities according to analysis results, which include a supply chain management system and a web-based learning system platform, and will be implemented in a network environment and be practically applied to the course. In terms of data collection, questionnaires and interviews will be performed at the end of semester to qualitatively investigate the effect of learning of learners with this system, and also evaluate whether the system functionalities meet their needs. Furthermore, quantitative statistical data will be used as the basis of information analysis.

4.1 Participants and Teaching Condition The participants of this research are 30 research students from a research institute of a national university at the North of the country. The course is entitled “Supply Chain Management”, in which 2/3 of the class time is face-to-face lecture, where lecturer and

I-F. Liu and S.S.-C. Young / A Study of Implementing Web-Based Learning Systems

333

students carry out related teaching activities in the classroom and emphasize on “teaching oriented”; the other 1/3 of the time is conducted on the Internet, where teaching assistant and students carry out related learning and instructional activities on the network and emphasize on “learning oriented”. 4.2 Research Time and Place The system of this research was developed from March to August 2005; and this research was carried out from September 2005 to January 2006 for 16 weeks. The places where this study was conducted were: (1) physical classrooms, (2) online environment. The researchers participated in both the physical course and web-based course for 16 weeks, recorded and observed the whole process of the web-based learning and learning conditions of the participants. 5. Data analysis 5.1 Background information of the participants The target of this research is 30 students who have taken this course; their basic information is as follows). (1) Within the 30 students, 63% are male and 37% are female. (2) The average age of the students is 25 year old. (3) The institutes of Information Management and Industrial Management have 10 students each; the institutes of Technology Management and Transportation Management have 5 students each. (4) In terms of the method of connection, 4 students use modem to connect onto the Internet; 10 students are connect via campus network; and 16 students use broadband. (5) In terms of place of residence, 13 students or 43% of the total number stay in dormitories; 11 students or 37% of students stay near campus; and 6 or 20% of students stay outside the county. (6) In terms of the frequency of Internet usage, besides two students who on average use the Internet once every two days, almost 93% of students have the habit of using the Internet everyday.

5.2 Analysis of Learning Effect of Students with the SCM System This research focuses on the questionnaires of learning effect of students who are using this SCM system, and combines the information for interviews for integrated analysis. Question 1: “Through the operation of SCM system, it helps me to understand the actual operations of supply chain in organizations.” The majority, which is 14 students, agrees and 9 students strongly agree. It shows that almost 80% of the students think that through the operation of the SCM system, it helps them to understand the actual operations of supply chain in organizations. The students who do not agree think that real supply chain systems should be more complicated; large-scale organizations may have more than millions of materials and the parameter settings within a supply chain are also very large (DEM06). Question 2: “The interface design of the SCM system is user-friendly; it helps me to quickly achieve the effect of learning.” 17 students, which are the majority, strongly agree and 10 students agree; nearly 90% of the students think that the interface of the SCM system is user-friendly, and agree that it helps to achieve results quickly. Some students think that when they encounter difficulty during their learning with the system, they can always use the function of online help (IIM06), or ask for help from other fellow students online (DEM02, IIM12), so that the most satisfactory solutions are obtained in the shortest time.

334

I-F. Liu and S.S.-C. Young / A Study of Implementing Web-Based Learning Systems

On the other hand, there is one student who disagrees thinks that the connection condition of the system is poor (DTM02); after an interview it is found that that student uses 56K modem to connect to the Internet, which results in problems such as bad connecting quality and low transfer rate. The researcher suggests that the use of broadband connection would solve the problem.

6. Conclusion Based on the research questions that this research intends to investigate, and according to the data collected from observations, questionnaires, learning process from system’s log-in information and interviews, and analysis, the major findings are as follows: x Through the use of this web-based learning system, it is indicated that students made obvious progress in learning. Through the use of this SCM system, students are able to understand the actual operations of supply chain in organizations; through the individual learning functions, interactive and cooperative learning functions, and web-based evaluation functions, the interests in learning of students have increased, which result in an obvious improvement in learning effect. With this 1/3 web-based course, it definitely compromises with the insufficiency of traditional classroom learning. x This web-based course fulfills the highest level of information integration in teaching of Stephen and Marshall.This Supply Chain Management course uses 1/3 of its time for web-based learning, all teaching and learning activities are performed online, so it meets the highest level – level 5, of information integration in teaching of Stephen and Marshall. x Among fellow students or students and assistant, there shows a high rate of interactions. x After verification, the design of this web-base learning system meets the related learning theories. According to the results, the design of this web-based learning system has created a virtual learning community. Students are able to operate on the supply chain system, use the web-based learning system platform and learn through interaction with fellow students; the results prove that these meet the web-based learning theory of learning theory, constructive learning theory, and the essence of cooperative learning theory.

Acknowledgments This project is partially funded by the National Science Council of Taiwan under the research project code NSC93-2520-S-007-001. References [1] Zou, J. P. (2000), e-Learning is the Only Way to Win in Knowledge Organizations, Newsletter for Information Professionals, 59, pp. 1-4. [2] Huang, B. L. (2001). From the Development of Online Learning to Organization’s Online Training, e-Organization Manager’s Report, 19, pp. 12-23. [3] Jay, A., A (1998). Trainer’s Guide to Web-Based Instruction, American Society for Training & Development (ASTD).

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

335

What Is Expected of a Facilitator in a Virtual Learning Environment? Ni Chang, Ed.D Indiana University South Bend, USA [email protected] Abstract: Online teaching and learning significantly differs from traditional teaching and learning [1] because of the change of context. How to facilitate students via web-based information delivery systems is fundamentally imperative. However, there is a dearth of literature regarding facilitation across online interaction [2]. Therefore, the heart of this paper will focus on responsibilities and accountabilities expected of an e-facilitator. Keywords: facilitator, online instructor, online communication

1. Introduction Cambridge Advanced Learner's Dictionary [3] defines “facilitate” as “to make possible or easier.” In education, it means that an instructor exerts every effort to promote student learning for the results of their motivated and sustained inquiry. With the development and advancement of course delivery systems, there is a sharp difference between traditional teaching and virtual instructional environments [1, 4]. The former is dependent of location and time, moderately requiring learners’ self-regulation, while the latter takes place anytime and anywhere, highly requiring learners’ self-regulation [5, 6]. The former is implemented in reading and writing, while the latter is exercised by speaking and listening. The former makes an instructor and learners easily visible to each other, while the latter leaves the instructor and learners not only a feeling of isolation, but also invisible to one another. All the changes from in-class instruction to online teaching explicitly result in a huge curb in the process of learning. To ease such a process, it is expected of an online instructor to adjust his or her mind set to facilitate online learners. However, there is a dearth of literature addressing online facilitation [2]. Therefore, this paper is timely as it places an emphasis on responsibilities and accountabilities of an e-facilitator.

2. Instructional Planning 2.1 Designing a Learning Environment Some faculty members have to create new learning environments from scratch while others need to make adjustment of extant online courses by revising learning content. In any case, a virtual learning environment should be designed by keeping in mind students’ needs and interests to maintain optimal enthusiasm over the course of study [8]. This type of learning environment also purposefully provides students with a big picture: “what’s in it for me” [7] and is intended to minimize a misconception that online discussion is merely a chore [8]. In designing an online course, an instructor also needs to enhance “people’s visibility” by

336

N. Chang / What Is Expected of a Facilitator in a Virtual Learning Environment?

forming online collaborative groups with two to three people in each group to make learners feel that they are interacting with real people rather than a machine. 2.2 Learning Structure At the beginning of a semester, it is essential for an instructor to offer a clear explanation of a course design [9]. A good understanding of where they click to receive lectures and where assignments are to be submitted helps minimize unnecessary frustration of students [7]. Students also need to know expected rules, procedures, and course objectives so that all involved parties are able to monitor progress and to develop in-depth understandings of course content [4, 8, 9]. Biggs stated, “Teaching is effective when it supports those activities appropriate to understanding the curriculum objectives” [6]. 3. Discourse Representation and Analysis 3.1 Monitoring While monitoring online discussion for possible sarcastic and inappropriate language [12, 13], an online instructor is also expected to undertake responsibility for working up students’ enthusiasm for learning [12]. Examples of responsibilities include flexible changes of topics that are not originally planned [14] and mediating conflicts that have emerged so as to provide group members with an opportunity to acquire problem solving skills for the future unexpected [14]. Although, from Lim and Cheah’s perspective, developing human relationship and helping students collaborate in a mutual manner are managerial responsibilities, they are, in effect, the characteristics of online facilitation. 3.2 Moderating to Motivate and Engage in Student Learning In communication, as an instructor uses appropriate tone of voice, the online learners’ comfortable level in this virtual learning environment escalates. Hall argued that humor and special languages used in communication enhance humanizing online discussions. Students are likely to deconstruct a posted message that is of their interest and reconstruct it that represents their mental models [8]. Positive feeling and emotion toward online learning advance cohesive group collaboration and “a sense of belongingness” [16, p. 41]. During online course delivery, the responsibility of an instructor to moderate online participation is intended to show students that he or she is there for them through feedback. Feedback or a note from an instructor to students is “faculty visibility” that may potentially strengthen the student desire to learn and that make online communication interactive. This ideology is in line with Pelz’s statement, “Interactivity is heart and soul of online asynchronous discussion.” Hall confirmed that desirable learning outcomes relied on an instructor’s mediation besides a well-structured course.

3.3 Feedback and Self-Regulation Berge noted that feedback encouraged students to reflect upon online discussions [4], stimulating students’ thought process and directing them to meet objectives [16]. Without feedback, as found by Lim & Cheah, students felt lost. Shank agreed that online learners

N. Chang / What Is Expected of a Facilitator in a Virtual Learning Environment?

337

expected timely and quality feedback by an instructor on a daily basis. These researchers concluded that constructive feedback reinforced learned concepts. Therefore, it is primarily important that feedback must be content-focused rather than participation-oriented. Appropriate feedback indeed is fundamental and crucial to students. One student made a comment: “After we’ve posted our discussions, we want someone to tell us whether the suggestions are workable or whether the comments are valid” [16, p. 42]. Hall concurred that feedback on performance enhanced students’ conceptual understanding and was a consistent signal to students that an instructor cared about their learning. 3.4 Formative and Summative Assessment While providing constructive feedback to students, the instructor also has a chance to know the level of student learning. In other words, the feedback works as evaluation of their learning experiences [12]. That is to say, grading is the responsibility of an instructor, but also is a moment of teaching germane to individual student’s level of understanding. Grading and teaching are interrelated. Hall stressed that through formative and summative assessment, conceptual understanding of students increased. Comments made by an instructor or other peers on student work enable an individual student to gain deep insight into content under discussion. 4. Conclusion This paper makes a case of what it means by facilitation in a virtual learning environment. Garrison and Anderson stated, “Teaching presence is the facilitation and direction of cognitive and social process for the realization of personally meaningful and educationally worthwhile learning outcomes [in 5] (p. 12). Facilitation in the online environment is the heart of personal, meaningful, and education learning. Facilitation begins even before a course is to be design. Quality facilitation, during the course delivery, leads to maintain an instructor’s visibility, to offer quality resources, and to remove barriers [17]. Facilitation is against the practice of “sage on the stage;” but advocates “guide on the side.” It does not support dominant lecturing of an instructor, but highlights useful responses and comments to students’ work by an instructor for the purpose of teaching them individually (content-focused rather than participation-oriented). Through the use of the Internet or web-based learning tools, students’ reflexivity (to question obtained information rather than to simply translate it into one’s words without thinking and reflection—“a knock-on effect”) is underpinned by monitoring and formative assessment of an instructor [9, p. 157]. An online instructor’s consistent and visible support buttresses students’ emotional involvement, promoting “active engagement in undertaking achievable tasks” [9, p. 149]. Although some strategies and practices applied to facilitation in an e-classroom seem to be similar to those used in a traditional setting [18], they are significantly different in nature and characteristics due to the paradigm shift from listening and speaking in a conventional classroom setting to reading and writing in a virtual learning environment and from visibility to invisibility. Facilitation, in fact, embraces pedagogical, managerial, social, and technical areas. Regardless of whether it is before, during, or after a course delivery, these aspects are in existence; facilitation takes place in learning, teaching, and assessment. Facilitation is of great importance in a web-based educational realm, which is worth much more attention and which requires many continual efforts to know and discover better.

338

N. Chang / What Is Expected of a Facilitator in a Virtual Learning Environment?

References [1] Coppola, W., Hilz, R., & Rotter, N. (2001, January). Becoming a virtual professor: Pedagogical roles and ALN. Paper presented at the 34the Hawaii International Conference on System Sciences, Maui, Hawaii. [2] Bonk, C. J., & Dennen, V. P. (2003). Frameworks for research, design, benchmarks, training, and pedagogy in web-based distance education. In M. G. Moore, & W. G. Anderson, (Eds.), Handbook of distance education. (pp. 331-348). Mahwah, JN: L. Erlbaum Associates. [3] Cambridge Advanced Learner's Dictionary (2003). London: Cambridge University Press. Feenberg, A. (2000 June). Online pedagogy with discussion management software. Paper presented at the AAUP Annual Meeting, Washington, D.C. [4] Wilson, et al. (2003). Instructors’ adaptation to online graduate education in health promotion: A qualitative study. Journal of Distance Education, 18(2), 1-15. [5] Pelz, B. (2004). (My) three principles of effective online pedagogy. JALN, 8(3), 33-46. [6] Sloan Consortium (2006). Growing by degrees: Online education in the United States, 2005. Retrieved January 22, 2006, from http://www.sloanc.org/publications/survey/survey05.asp [7] Feenberg, A. (2000 January). Online pedagogy with discussion management software. Paper presented at AAUP Annual Meeting, Washington, D.C. [8] Lim, C. P., & Cheah, P. T. (2003). The role of the tutor in asynchronous discussion boards: A case study of a pre-service teacher course. Education Media International. Retrieved November 20, 2005, from www.tandf.co.uk/journals/routledge/09523987.html [9] Hall, R. (2002). Aligning learning, teaching and assessment using the web: An evaluation of pedagogic approaches. British Journal of educational Technology, 33(2), 149-158. [10] Klemm, W. R. (1998). Eight ways to get students more engaged in online conference. T.H.E. Journal. Retrieved January 12, 2006, from http://www.thejournal.com/magazine/vault/A1997.cfm [11] Biggs, J. (1999). Teaching for quality learning at university. Buckingham: Open University Press/Society for Research in Higher Education. [12] Berge, Z. L. (1995). Facilitating computer conferencing: Recommendations from the field. Educational Technology, 35(1), 22-30. [13] Rossman, M. H. (1999). Successful online teaching using an asynchronous learner discussion forum. Journal of Asynchronous Learning Networks, 3(2), 91-97. [14] Strickland, C. (1998). A personal experience with electronic community. CMC Magazine. Retrieved, January 23, 2006, from http://www.december.com/cmc/mag/1998/jun/strick.html [15] Ko, S. S., & Rossen, S. (2001). Classroom management. In S. S. Ko, & S. Rossen (Eds.), Teaching online: A practical guide (pp. 211-253). Boston, MA: Houghton Mifflin. [16] Laurillard, D. (2002). Rethinking university teaching (2nd ed.). New York: RoutledgeFalmer. [17] Bischoff, A. (2000). The elements of effective online teaching: Overcoming the barriers to success. In K. W. White, & B. H. Weight (Eds.), The online teaching guide: A handbook of attitude strategies, and techniques for the virtual classroom (pp. 5772). Boston, MA: Allyn and Bacon. [18] Addesso, P. (2000). Online facilitation: Individual and group possibilities. In K. W. White, & B. H. Weight (Eds.), The online teaching guide: A handbook of attitude strategies, and techniques for the virtual classroom (pp. 112-123). Boston, MA: Allyn and Bacon.

Social Networking & Blog

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

341

Proactivity, Autonomy & Social Networking: Transitional Environments for the Japanese Educational Context a

Deborah C. Turka , John W. Brinea Center for Language Research, University of Aizu, Japan [email protected] ; [email protected]

Abstract: In this paper, we describe the theoretical framework for a transitional educational environment for the Japanese context. Building on a previously developed LMS structure for a Technical Reading course, this new course aims to focus on fostering conditions for greater self-reflection, peer feedback, intertextuality, accountability and student control. It uses a combination of LMS (Moodle) features and the blogging capability of a social networking environment (Elgg). This paper will report on its preliminary findings on the student use of the Elgg blogging feature and the Moodle workshop module. Keywords: Autonomy, social networking, LMS, blogging, peer review, project-based learning, cultural learning practices, writing

Introduction This paper will describe the development of a transitional educational environment to promote greater proactivity among our English language students at the University of Aizu. Taking account of the Japanese cultural learning practices, this environment builds on a previously highly scaffolded LMS course for Technical Reading 2 implemented during the second semester of 2005. The subsequent Technical Writing course (in 2006) was designed to focus on fostering conditions for greater self-reflection, peer feedback, intertextuality, accountability and student control. The use of peer review techniques was a new skill for the students to learn and use, as many of them relied upon teacher feedback in their previous learning histories. The Technical Writing courses used a combination of LMS (Moodle) features and the blogging capability of a social networking environment, known as Elgg. Moodle has a structured, modular appearance that makes it easy for students and teachers to follow the learning path. Elgg, on the other hand, has different properties that require students and teachers to seek out the information they desire, or the blog to which they would like to subscribe. There are advantages and drawbacks to both of these systems. This paper will describe the theoretical framework that underpinned the course design and will report preliminary findings on the student use of the Elgg blogging feature and the Moodle workshop module. 1. Background to the research 1.1 Project-based learning Projects or macrotasks have recently attracted greater pedagogic focus for language learning and teaching [1] as they seem to be able to provide teachers and learners with the kinds of multi-faceted opportunities being sought; engagement opportunities that have

342

D.C. Turk and J.W. Brine / Proactivity, Autonomy & Social Networking

relevant and/or real life stakes. The implementation of projects and multi-faceted engagement opportunities may be one way in which a learner’s imagination can be fostered, and through this, they may also enhance their ability to take risks within the community (e.g. approach other members rather than work independently, hold conversations with other members of the community about a particular problem or issue, or share ideas). Risk-taking, inquiry and reflection could lead to an increase of more informed or engaged interaction and thus greater opportunities for learning. These skills can also be seen as ones that may enable learners to become more sensitive to the contexts with which they are interacting [2]. Examples of these kinds of large scale projects or tasks successfully implemented for language learning include the Thai News Network [3], a variety of Italian language radio broadcasting projects carried out by Scotellaro commencing in 1997 [4] and the restaurant-café projects outlined in Spruck Wrigley (1998) [5]. As all of these projects interact directly with the target language community (i.e. they are grounded in real life situations), the participants have personal stakes in the success of the project and thus in their own contribution to the community and the task itself. These large-scale projects have the potential for bridging the gap which is perceived to exist between formal and informal learning environments. Within the fields of research in informal education and lifelong learning, the validity of informal learning in environments other than institutions, designed for the express purpose of educating students, is considered to be a topic of both significance and contention: significance, in that the amount of informal learning taking place is considered to be substantial when participating in the workplace, local communities and leisure activities [6]; and contention, in that informal learning is not sufficiently validated [7], valued or taken into account by formal learning contexts [8]. As argued by Fischer (1999) the transition from school to the workplace is “insufficiently supported” and this insufficiency has led to a gap between the skills and knowledge learned in the university context and the skills required to adequately prepare them for the needs of the workplace [9]. Thus the focus of our courses is not to instruct students solely on vocabulary items and technical concepts, but to focus on developing a variety of learning skills that students will need to perform in the workplace context. We view the ability to work in diverse groupings, to take on rotating or changeable roles, and to work within environments of differing levels of formality and guided work activities to be of importance for our students’ transition to the workplace. 1.2 Formal and informal learning environments The physical environment itself evidently forms a vital role in the learning process. It can be designed to complement, or reinforce pedagogic objectives. Formal learning environments, in which the pedagogic agenda is pre-determined, have the advantage of providing students with support structures to guide their learning. However, under these circumstances, the students are not empowered with the ability to control their own learning, nor the evaluative process [10]. The task of designing informal environments to accommodate diverse goaldirected [11] [12] or self-directed learning behaviours therefore presents a challenge. The advantage of creating these conditions for learning is that it can provide learners with a greater range of referents, spaces in which to explore their own creativity and establish a social presence, or identity. It can provide them with the skills to inform themselves and to interact under diverse circumstances. While the environment itself denotes a form of structure, its internal flexibility can be modified to a certain extent. For example, a community of smaller projects could be

D.C. Turk and J.W. Brine / Proactivity, Autonomy & Social Networking

343

devised and implemented within the one integrated learning environment. The use of the term ‘community’ here refers not to merely a group of non-related projects, but rather groups of individuals working on group projects that are all inherently related to, and which rely upon each other. Such a community of projects would be driven my mutual care, sharing and equal responsibility [13] for the success and progress of the community as a whole, and not merely the success of the group project. The aim of this type of environment would be to allow learners to select the kinds of projects they wish to pursue. Providing more flexibility in the overall task design would allow learners to choose projects and tasks that appeal to their individual interests. It would also expose students to a range of interactions, diverse social structures and could increase the value of the opinions and performance of others. Public or social spaces would need to be created in order to foster the conditions under which the exploration and application of various forms of communication and the product of one’s imagination could be evaluated. These social spaces should require no particular entry or exit point, and promote visual, oral and discursive presence. This decentralised model of an online environment would allow participants to make their own connections to people and information. Moreover, the participants and their audience would dynamically and dialogically shape the content and the connections between different aspects of the community. The students could select the access restrictions for each submission to their account leading, ultimately, to an informal learning environment which is structured and controlled by the students. Decentralised, informal learning conditions, however, have the potential to conflict with the cultural learning practices that are present within the Japanese educational context. Japanese students' classroom behaviours reflect their cultural beliefs and their social expectations. From kindergarten through high school, teachers assign Japanese children into small mixed-ability groups called han. The children not only work in their han during teacher assigned times, but they also work as a group on many shared common tasks. These groups play together, work together, eat together, and learn together. The leaders of the han are referred to as hancho and are responsible for acting as a teacher's apprentice and reporting the han's status to the class. Other forms of student leaders are the toban, or the class leaders responsible for mediating problems among the students, and the “teaching children” called oshiego [14]. One result of using student groups and leaders is that children are taught to regulate their own social harmony on a small scale, as they must maintain peace among their peers in the han. This is the model of classroom behaviour and expectations that students bring to the university classroom [15]. These behaviours have a tendency to promote group-oriented cooperative learning conditions (in which the teacher denotes the authority) and thus lend themselves well to structured learning environments such as a learning management system (LMS). An LMS on its own might not necessarily create the conditions for students to gain the skills and confidence to become self-directed, or proactive participants in the learning process [16]. Hence, a scaffolded development system is required in order to manage the transition from a structured group-oriented system to one of self-direction in both a social and individual sense. This socially networked environment would aim to create conditions under which self-reflection, peer feedback, intertextuality, accountability and student control could be fostered. The subsequent sections of this paper outline an approach that is currently being taken at the University of Aizu. This work-in-progress is researching possible ways of scaffolding students from structured group-oriented learning environments towards greater autonomy and the ability to proactively participate in socially networked communities. Within our university context, learner autonomy is increasingly required as students

344

D.C. Turk and J.W. Brine / Proactivity, Autonomy & Social Networking

progress through their degree. The university specialises in bilingual computer science education and all students have a thesis-writing requirement for graduation. This research thesis is written in English. 2. Methodology: From Technical Reading to Technical Writing This on-going action research project aims to investigate the patterns and processes of use that emerge from the implementation of pre-determined management procedures for integrating online and face-to-face teaching and learning modes for both individual and group work. These management procedures are driven by the socio-constructivist principles and aim to foster greater proactivity and learner autonomy amongst the students in our courses. [The structured environment of the LMS provides a learning foundation for students, but our objective is to design a context which fosters the autonomy necessary for more advanced language learning.] Apart from developing their ability to read and write English documentation for computer science and engineering, we also aim to engage students in multi-faceted activities that require dialogue, negotiation, the use of cooperative and collaborative skills and the development of frameworks for inquiry into computer-related subject matter. In this particular paper, we will briefly outline the previous work completed in the Technical Reading course and address the transition from compulsory group work to self-selected individual-group work opportunities offered in the Technical writing course. The potential intertextual nature of blogging and the student-to-student peer review aspects of the Technical Writing course specifically will be discussed to demonstrate how Moodle and Elgg facilitated these aspects. Finally we will overview some of the preliminary findings of the research. In Brine and Turk [17], and other papers, we have described an environment designed for using LMS to teach six second-year Technical Reading classes at the University of Aizu (approximately 200 students). We identified the characteristics of Japanese group work in the public education system to inform the LMS design process and the classroom management system. Next, we selected a group work system used to teach university engineering classes in the USA [18], but which emphasises continuous cooperative and collaborative group work. In addition, we implemented these structures in Moodle, a popular open source LMS. This implementation of Moodle was done to support the enhanced group work structures both in and outside class. The group work was divided into both cooperative and collaborative roles, and these roles were rotated amongst the group members. It is also important to note that these groupings were persistent throughout the semester to support both the cultural learning practices of the students as well as the group work organisation system. After a semester of learning within this highly structured environment, many of the students progressed on to a different course, Technical Writing. This course is offered at the third-year level and forms part of their preparation to write their research thesis at the end of their fourth year of study. In this course, we wanted to utilize the now familiar Moodle system, reduce the structured nature of the Technical Reading course, and start to introduce the goals of student self-reflection, peer feedback, intertextuality, accountability and student control within the teaching and learning framework. Approximately 88 students are involved in this iteration of the Technical Writing course. 2.1 Integrating Environments The Technical Writing environment is based a structured learning management system (LMS) called Moodle. Moodle, while being perhaps one of the most flexible LMSes

D.C. Turk and J.W. Brine / Proactivity, Autonomy & Social Networking

345

available, currently exhibits the primary drawback of all LMSes in that it has been designed to manage the learning process. It is precisely the belief that learning can be modularised or managed that is, as Siemens [19] would argue, contrary to the fuzzy, chaotic nature of the learning process. In contrast to Moodle, Elgg is an online social networking environment (www.elgg.org) that consists of weblog and e-portfolio communities, and which has full RSS (Real Simple Syndication) capability. This socially networked environment moves away from the structured models of online learning communities and management systems in favour of rhizomatic [20], distributed models of student interaction. In the Elgg community, students post blogs describing their project ideas/content, submit drafts for group discussion, revision, and peer review, and present a series of completed products or e-portfolios [21]. Students can assemble objects or artefacts to represent themselves and their ideas, they can link their blogs to “friends”, invite others to join their community (from anywhere in the world), share resources and search the community for blogs that contain interesting draft texts and other relevant discussion topics. Through this socially networked community, students can shape their own online presence and take control over the learning process. Taking into account their previous learning practices, the modified Technical Writing environment loosens the group roles and structures to allow students more freedom of choice. This time, the groupings can be self-determined (they can work independently, in pairs or groups of their choice), and the students carry out a project which has specified product outcomes (i.e. proposals and other documents). Generic topic questions are distributed to the students, however, ultimately their own interests will determine the project topic. Some examples of student projects are: the development of a Rescue Support Network System (RSNS) for disaster management in Japan, an internet satellite communications systems for rural areas, a powdered green tea maker and further improvements for the well-known iPod. A direct link was created between the Elgg blogging community and the Technical writing course in Moodle. Moodle was designed to be the formal structure – one which included the evaluation system, a formal photo of each student, resources for class workshops, peer review workshops and the general course outline. Each student had an account in Moodle. This account also gave them access to the Elgg community. This was considered the informal structure as is constantly under development by the students and the social networking capability restricts the teacher’s ability to track, evaluate, influence or control the form which it takes. While students can select their own images (a photo or another representation), their identity is always known to logged in users of the community (as their login name appears underneath their image). Anyone logged into this community can view the publicly accessible work of other community members and can add comments to the blog entries. Postings made to the Elgg community could be submitted as a link for the purpose of peer review or teacher assessment. Assessment pieces consist of a set of technical documents that are likely to be produced in the initial stages of a generic project. These documents include Q&A documents, project briefs, letters of transmittal, procedures, and proposals. These documents are considered an integral part of each student’s project topic, and the content for these documents is drawn from their research. 3. Preliminary Findings from Elgg and the Moodle Workshop In this paper, we will briefly focus on two aspects of the Technical Writing course. These are the use of Elgg as a tool for encouraging self-reflection and intertextuality, and the use

346

D.C. Turk and J.W. Brine / Proactivity, Autonomy & Social Networking

of the Moodle workshop for peer reviewing drafts of the prescribed assessment documents. 3.1 Using Elgg As previously stated, Elgg was considered to be the development space for students in the Technical Writing course. It was used for storing files and web links, completing in-class practice activities, and developing draft documents. The use of Elgg is also producing some other interesting behavioural and pedagogical results. Regardless of their selected grouping for the project work (some were working independently), students quickly linked their blogs to friends, teachers and co-project workers. They also linked to other interesting blogs and the blogs of students completing related project topics. By adding these community members as their ‘friends’, it enabled them to gain easy access to the entries posted by these individuals. Another emergent result is an increase of textual comparisons. Having access to other student work, students are starting to pay more attention to the similarities and differences between their work and others. As the document development continues, students are seeking, and wanting to verify, good models of student work. Students share recommendations about the blog entries of student work and quickly begin discussing its merits in an attempt to improve their own. They will also query the teacher as to his/her opinion of a student’s blog entry to further validate their ‘model’ selection. This comparative, reflective behaviour was something we had previously found difficult to develop within the forums and glossaries in Moodle. The peer review workshops being held periodically in Moodle, might also have reinforced this comparative, reflective mode of behaviour. 3.2 Peer Review Workshops in Moodle For each of the assessable project documents, the students were required to submit their rough draft to peer review. Then, after reviewing another student’s work, they received comments on their own work from an anonymous student reviewer. Then they could use these comments to revise their work prior to submitting it for teacher assessment. The students received some basic training on writing constructive feedback at the beginning of the course, and they were provided with a review rubric to help them evaluate the document. Students could choose whether they wished to add written comments for each criterion and/or give a general comment on the overall document. It is important to note that while the reviewers were anonymously assigned to review documents, the reviewee was not necessarily anonymous. For example, if a reviewee attached his/her blog link, and s/he used their real name as a login name for Elgg, then the reviewer was able to identify the author of the document. Initially, the students reacted with hesitancy (and perhaps even fear) at the prospect of having review another student’s work. Peer review formed a small component of their overall grade and thus made them responsible for adding to or subtracting from a student’s overall grade. However, once students were assured that reviewers’ names were anonymous, the peer review component appears to have been accepted by the vast majority of the students. However, there were varying degrees of participation in the peer review component. From a total sample of 88 students, 76% of students on average submitted work online and an average of 84% of students, who had submitted their documents, carried out reviews on their peers’ work. It should be noted that a small number of students either handed in their work late directly to the teacher. As a result, these students so not appear in the figures used for ‘submitted work’.

D.C. Turk and J.W. Brine / Proactivity, Autonomy & Social Networking

347

On occasion, students who tried to circumvent the peer review system (by posting an empty file for example) had to be penalised by their reviewers. Some students posted a document, but then chose not (or forgot) to complete their required review of another student’s work. This can be seen in the comparison chart of submitted and peer reviewed work (see Figure 1 below).

Approximately half of the students completed the review rubric selected the appropriate criteria (in their view) but abstained from adding comments or suggestions. The remaining group completed the review rubric and also provided the reviewee with positive comments and constructive comments on how they believe the reviewee could improve their document. 4. Limitations and future research As it currently stands, the system is not yet sufficient for our needs. The workshop module of Moodle is undergoing development to resolve a number of pedagogic and technical issues related to its use. Minor bugs can cause errors that require manual editing of the database. The workshop also does not currently provide contingencies for situations where students post documents for peer review and then abstain from their role of reviewer. In such cases, the reviewees may be penalised by the LMS workshop grading system. Better ways of dealing with these issues need to be found, and we hope that they will be addressed/rectified in upcoming versions of Moodle. We would also like to see the workshop module developed further to include additional features. After students have submitted their documents for review, it would be greatly beneficial from a ‘learning management’ perspective, for students to receive a reminder email when the review process has begun. This two-stage process might be the cause of non-reviewed submissions to the workshop module. From a pedagogic standpoint, we are aware that additional scaffolding appears to be required in our teaching context to ‘ease’ the transition from a highly structured learning model of classroom organisation and learning management, to one that is primarily student driven and teacher supported. The way in which the social networking environment was implemented for this particular group of learners (and at this point in time), does not appear to have been suitably structured to provide all students with the adequate scaffolding to make the transition from a formal learning environment to a semiinformal learning environment. Other interim tasks and activities may need to be designed into the next iteration of the course in order to better support the learning, and transition, process. Evidently, we are continuing to develop these courses and will report on further findings in the near future. We look forward to presenting the student feedback on these, and other components of the Technical Writing course. We feel that these practical implementations will contribute to the body of research into the facilitation of learner autonomy within the fields of language learning, technical communications training and education in general.

348

D.C. Turk and J.W. Brine / Proactivity, Autonomy & Social Networking

References [1] Gaillet, L.L. (1994) An historical perspective on collaborative learning. Journal of Advanced Composition, 14,1, 93-110. [2] Turk, D. (2005) Improving the Process? A study of learner autonomy, interaction and technologyenhanced language-learning environments. Unpublished doctoral thesis, University of Canberra, Australia. [3] Burapatana, M., and Lian, A.B. (2002) Thai News Network: Critical thinking in Thai reading programs. Interdisciplinary Research Seminars, University of Canberra, Australia. URL: http://www.anialian.com/TNN_project.html [4] Lian, A.B., Dolan, D., Scotellaro, G., and Lian, A.P. (2004) Narizoma: Critical Pedagogy in Practice. In Son, J. (ed.). Computer-Assisted Language Learning: Concepts, Contexts and Practices.(APACALL series) New York: iUniverse Inc. [5] Spruck Wrigley, H. (1998) Knowledge in Action: The Promise of Project-Based Learning. Focus on Basics. 2, Issue D (December). http://gseweb.harvard.edu/~ncsall/fob/1998/wrigley.htm [6] Livingstone, D.W. (2000) Exploring the Icebergs of Adult Learning: Findings of the First Canadian Survey of Informal Learning Practices. The Canadian Journal for the Study of Adult Education. Special Milennium Issue. [7] Colardyn, D., and Bjornavold, J. (2004) Validation of Formal, Non-Formal and Informal Learning: policy and practices in EU Member States. European Journal of Education, 39, 1, 71-72. [8] Gorard, S., Fevre, R., and Rees, G. (1999) The Apparent Decline of Informal Learning. Oxford Review of Education, 25, 4, December, 437-455. [9] Fischer, G. (1999) Lifelong Learning: Changing Mindsets. Proceedings of the 7th International Conference of Computers in Education, November 4-7, Chiba, Japan, 21-30. [10] Littlewood, W. (1999). Defining and Developing Autonomy in East Asian Contexts. Applied Linguistics, 20, 1, 71-94. [11] Lian, A.P., and Mestre, M.C. (1985) Goal-Directed Communicative Interaction and Macrosimulation. Revue de Phonetique Applique. 73-75. [12] Tennant, M. (1999) Is learning transferable? In D. Boud and J. Garrick (eds.) Understanding Learning at Work, London: Routledge. [13] Bauman, Z. (2001) Community: Seeking Safety in an Insecure World. Cambridge: Polity Press. [14] White, M. (1987) The Japanese educational challenge: A commitment to children. The Free Press: New York. [15] Brine, J.W., and Turk, D.C. (2006) Group Work and Role Rotation using a Learning Management System in a Japanese Computer Science University. Conference Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies, 547-551. [16] Littlewood, W. (1999). Defining and Developing Autonomy in East Asian Contexts. Applied Linguistics, 20, 1, 71-94. [17] Brine, J.W., and Turk, D.C. (2006) Group Work and Role Rotation using a Learning Management System in a Japanese Computer Science University. Conference Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies, 547-551. [18] Oakley, B., Felder, R. M., Brent, R., and Elhajj. I. (2004) Turning student groups into effective teams. Journal of Student Centered Learning, 2, 1, 9–34. [19] Siemens, G. (2004) Learning Management Systems: The wrong place to start learning. http://www.elearnspace.org/Articles/lms.htm [20] Deleuze, G., and Guattari, F. (1987) A Thousand Plateaus: capitalism & schizophrenia. Great Britain: Bookcraft. [21] The E-Learning Framework (2004) What is an ePortfolio? http://cetis.ac.uk:8080/frameworks/learning_domain_services/eportfolio/petal/whatiseportfolio/view

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

349

Effects of Peers Interactivity and Self-regulated Learning Strategies on Learning Art Appreciation through Weblog CHEUNG Sau Hung, Percy Lai Yin KWOK Chinese University of Hong Kong, Hong Kong, China [email protected] Abstract: Twenty-five visual arts learners experienced eight weeks of studying art appreciation in sculptures to try out new senior (Grade 10-11) art curriculum with weblog in the contemporary art classroom. Ten weblogs were created to facilitate collaborative learners in the virtual learning community to co-construct scare art knowledge, enrich aesthetic abilities, and develop generic skills and positive attitude for becoming competent learners in the long run. The study aimed to gain understanding of learners’ perception with weblog, to scrutinize potential functions of weblog to integrate web interactivity, social cognitive learning theories and uses of self-regulated learning strategies to achieve intended learning goals of art appreciation. On the whole, learners could direct diversified learner-to-learner and peer-to-content web interactivity that empowered themselves confidence and motivation to co-construct an archive of sculpture research in weblogs. With positive effect of peer collaboration, all learners could assume holistic responsibility. Keywords: Peer interactivity, self-regulated learning strategies, art appreciation, weblogs

Introduction Hong Kong is preparing to enhance quality education of young people to meet challenges of this ever-changing technological world (CDC, 2001). Education Commission (EC) (2000) has recommended the adoption of a 3-year senior secondary academic system to facilitate the implementation of a more flexible, coherent and diversified senior secondary curriculum. Curriculum Development Council (CDC, 2000) and the Hong Kong Examinations and Assessment Authority (HKEAA) jointly issued the Proposed New Senior Secondary Curriculum and Assessment Framework of Visual Arts. The framework would bring about great change in art learning (both in learning and assessment mode from sheer practical performance to knowledge-based written examination in critical art) and exert considerable pressure upon art teachers. Facing this trend, the current study has addressed the importance of aesthetic education in the contemporary art class and the way how some art learners acquired art knowledge and developed critical thinking in art appreciation which policy makers and curriculum developers might overlook. 1. Literature Review 1.1 Critical thinking in visual arts appreciation Art education has moved from an expressionist viewpoint toward a more conceptual approach. Feldman (1994) developed a widely used sequential approach for art educators to conduct to art criticism, based on description, analysis, interpretation, and judgment. Such formal criticism method, demands a strong sequential relationship as a learner is guided from concrete details to abstract concepts, from knowledge and comprehension to analysis and evaluation. In the case of critiquing one’s own creation and mental imagination, the learner exercises an opportunity to reach the pinnacle in the application of high-order thinking skills. Learners will develop meta-thinking skills and access cognitive domains that will enrich learning and functioning inside and outside the classroom.

350

S.H. Cheung and P.L.Y. Kwok / Effects of Peers Interactivity

1.2 Self-regulated learning and metacognition According to Zimmerman & Risemberg (1997), self-regulation refers to the self-directive process through which learners transform their mental abilities into task-related academic skills. A self-regulated learning (SRL) theory not only includes discovery learning, self-education through learning task or computer-assisted instruction, but also embraces social forms of learning such as modeling, guidance and feedback from peers, coaches, and teachers. Metacognition is the main aspect of self-regulation. Planning, monitoring and regulating are three active processes that make up metacognitive, self-regulatory activities (Pintrich et al., 1991). 1.3 Self-regulated learning strategies in new technologies environment Scardamalia & Bereiter (1994) contend that it should be students (not the computer) who engage in problem-solving, planning and goal setting. An essential design feature of web-based environment is to provide shared mental tools to the development of constructive systems supporting metacognition and problem-solving skills. Moreover, research bodies have been carried out in recent years in a socio-constructivist perspective (e.g. Paris, Bymes & Paris, 2001; Volet & Jarvela, 2001), focusing on how context characteristics and demands of the situations affect learning in general, and in particular how self-motivation and self-regulated learning are further enhanced. In response to new changes in schools with information and communication technologies, new web-based learning environments should be studied to see how they influence the process of self-regulation (Montalvo & Torres, 2004). 1.4 Interactivity in web-based learning Moore (1992) provides useful guidelines to discuss the attainment of interactivity into three types of interactivity: learner-content, learner-instructor and learner-learner. The level of interactivity (communication, participation and feedback) or those interactions among students, and between students and their teachers has major impacts on the quality of computer-mediated education programs (Muirhead, 2001). 1.5 Weblog and learning community Educational use of weblogs is increasingly being brought into existence every day (Hirsch, 2005). Baim (2004) argues that weblog help create learning community by indicating that learners develop a better grounding in fundamental course concepts and use their online discussions to refine ideas and even raise thought-provoking questions during face-to-face discussion. Weblog as a new computer-mediated communication tool has its potential to create learning community. However, relatively little research has been conducted about learners perception of using weblog for self-learning or about the impact of social interaction in building weblog environment. The current study endeavors to scrutinize how weblog shape up learner perceptions of building learning communities with self-learning strategies, and how its interactivity fosters online journaling of art appreciation. 2. Research Design and Directions 2.1 Target Groups 25 Grade 10-11 subjects were found from a secondary school in Northern part of New Territories. In 2005-06, they were allocated into Science and Commerce streams to pursuit their respective electives, apart from English, Chinese and Mathematics. In consideration of the characteristics of subjects, the art course program was designed for 25 learners with references to Association in Counseling and Child Guidance (ACCG, 2006) to launch learning programs for academically low achievers (ALA). They voluntarily joined the study and their academic performance would be assessed and recorded in the open examination.

S.H. Cheung and P.L.Y. Kwok / Effects of Peers Interactivity

351

2.2 Research Questions z What were learner’s perceptions of using weblog to build up learning community to achieve stated learning goals in art appreciation? z In what ways did learners develop interactivity in the weblog environment to enhance confidence of learning in art appreciation? z How did visual arts learners use self-regulated learning strategies to co-construct learning community to promote achievement of stated learning goals in art appreciation class? 3. Research Methodology In this study, qualitative research methods such as observation of group discussion, field notes, and structured interviews were carried out during the 8-week implementation process of using weblog in art appreciation. Four stages, namely, preliminary, interim evaluation, consolidation and final evaluation, were undertaken. 4. Results 4.1 Low perception towards the functionality of weblog in learning community As a result of varied perceptivity of learners’ contribution, the weblog could not fully function as a facilitator to foster students’ positive attitude of knowledge building. They were hesitant to become active and constructive learner. Student subjects were not likely to perceive the experience of discovery learning with appropriate ability of synthesis and co-construction of required research materials of independent learning. In their weblogs, all groups had created hyperlinks to make convenient knowledge sharing and interaction, according to their own interests. However, half of learners fairly agreed on the motivational effect of the weblog’s interactive functionality in discovery learning. The result of the responding weblog records was relatively unsatisfactory, reflecting little success of encouraging interactivity among learners and the public. Indeed it took time for the learning culture to make progress so as to realize the proposition of knowledge-building community which could provide possibilities for knowledge advancement. 4.2 Weblog environment promotes authentic and interactive learning experiences Students commented favorably on their collaboration with other students. The collaboration among peers creating confidence was prominent in their weblog interactivity and became essence of success of their academic result. Such results helped answer the second research question that diversified online peer-to-content and peer-to-peer interactivity gave learners confidence and motivation to make comprehension, understanding, integration, interpretation, composition of articles, in responding to their studying artifacts. The interactivity of intra-group and inter-groups validated learners and enhanced learners’ confidence to work with learning contents and had faith to work ahead. However, learners were not satisfied with the weblog as an asynchronous tool to delay online conversation to make decision. Instead they were able to direct peer interactivity in uploading the art critics in the weblog (as an instant messenger tool) to assist online learning and the formation of learning community. 4.3 Exercising group metacognition and self-regulated learning strategies By effect of group metacognition, learners worked hard together to do their best in their blogs, presentation and reports even if they did not prefer to do it. Strong coherence of peer relationship had always showed positive effects during weblog activities, web conferencing, lectures and report presentation. Formation of grouping in this project was temporary. It was difficult for peer groups to develop long term goal with group metacognitive strategies. They preferred to set short-term plan to meet the basic requirement of the project and were

352

S.H. Cheung and P.L.Y. Kwok / Effects of Peers Interactivity

eager to expect the coming of next project with more drawing and studio assignments. During the interim evaluation stage, each group made self-evaluation and then got comments from other peer groups. It was also surprising to see that the atmosphere of peer group evaluation were in harmony and enthusiastic and each group was attentive to accept peer groups’ comments. They could help learners set up short term goals and urged themselves and others to learn better. Notably, cooperative work projects in the web or in the class time did provide a social environment for students to develop deeper relationships with others and then an online learning community operated successfully during the course. Ultimately, all learners could assume positive attitude and holistic responsibility. 5. Discussion Much basic research has been done on project work, but applied research on interactivity and self-regulated learning strategies has been rare in primary and secondary schools. Even though the pattern of interactivity in weblog is still imperfect, its impact has been remarkable upon the contemporary art classroom. However, the current study ignored other variables and factors affecting student performance, such as prior knowledge and interactivity in the web and parental guidance. Nevertheless, the study has provided a valuable framework for future weblog interactivity investigations. Areas for further research concerning use of self-regulating learning strategies during intra-group and inter-group activities, exploration of alternative teacher evaluation online systems, content analysis of textual material produced during on-line discussions for critical thinking and comparative cross-school studies need to be conducted. References [1] Association in Counseling and Child Guidance (ACCG) (2006) Low achiever. Association in Counseling and Child Guidance. Retrieved on December 26, 2006, from: http://www.accg.net/low_achiever.htm [2] Baim, S. (2004) Blogs help create learning community. Online Classroom, Aug Issue, p.5. Retrieved on December 31, 2006, from: http://web26.epnet.com.easyaccess1.lib.cuhk.edu.hk/ [3] Curriculum Development Council (CDC) (2000) Key learning area. Arts education. HKSAR: Hong Kong Curriculum Development Council. [4] Curriculum Development Council (CDC) (2001) Learning to learn, life-long learning and whole-person development. HKSAR: Hong Kong Curriculum Development Council. [5] Education Commission (EC) (2000) Education reform. Learning for life. Learning through life: Reform proposals for the education system in Hong Kong. HKSAR: Education Commission. [6] Feldman, E. B. (1994) Practical Art criticism. Englewood Cliffs, NJ: Prentice-Hall. [7] Hirsch, J. (2005) Learning collaboratively with technology: students' social interactions demand new applications of digital learning tools. School Administrator, August 2005 Issue. Retrieved on December 26, 2006, from: http://www.findarticles.com/p/articles/mi_m0JSD/is_7_62/ai_n15622438 [8] Montalvo, F.T., & Torres, M.C.G. (2004) Self-regulated learning: Current and future directions. Electronic Journal of Research in Educational Pshychology, 2(1), 1-34. [9] Moore, M.G. (1992). Three types of interaction. The American Journal of Distance Education, 3 (2), 1-6. [10] Muirhead, B. (2001) Enhancing social interaction in computer-mediated distance education. Ed Journal and Ed at a Distance Magazine,.15 (4). Retrieved on Feb 16, 2006, from: http://www.usdla.org/html/journal/APR01_Issue/index.html [11] Paris, S.G., Bymes, J.P., & Paris, A.H. (2001) Constructing theories, identities, and actions of self-regulated learners. In Zimmerman B.J., & Schunk, D.H. (Eds.) (2001). Self-regulated learning and academic achievement: Theoretical perspectives (pp.253-287). Hillsdale, NJ: Erlbaum. [12] Pintrich, P., Smith, D., Garcia, T., & McKeachie, W. (1991). A manual for the use of the motivated strategies for learning questionnaire: Technical report. USA: The Regents of The University of Michigan. [13] Scardamalia, M., & Bereiter, C. (1994) Computer support for knowledge-building communities. The Journal of the Learning Sciences 3(3), 265-283. [14] Volet, S., & Jarvela, S. (Eds) (2001) Motivation in learning contexts: Theoretical advances and methodological implications. Amsterdam: Elsevier. [15] Zimmerman, B.J., & Risemberg, R. (1997) Self-regulatory dimensions of academic learning and motivation. In Chen, C. S. (2002) Self-regulated learning strategies and achievement in an introduction to information System Course. Information Technology, Learning and Performance Journal, 20 (1), 11.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

353

An Application of Social Network Analysis in Evaluation of CSCL Yonggu Wanga,b, Kedong Lib Education Science & Technology College, Zhejiang University of Technology, China b Education Information Technology School, South China Normal University, China [email protected]

a

Abstract: It is a new direction in the research of CSCL to evaluate the process of knowledge building by social network analysis. This paper introduces the history, definition and roles of Social network analysis, and explains the methods to evaluate the process of knowledge building in CSCL by measures of social network analysis. Keywords: Social network analysis, CSCL, knowledge building, evaluation

Introduction The practice of CSCL has spread out continuously in recent years. Many researchers try to explore the elements influencing knowledge building in the CSCL environment, such as abilities of self control, individual responsibilities, consciousness of sharing knowledge, and skills of collaborative communication, etc(Yonggu Wang & Xiaojuan Li, 2004). In our opinion, this kind of evaluation is important, but the methodology can only evaluate the individual attributes in CSCL, without considering the social attributes in CSCL. However, the social network analysis can visualize patterns of social interaction and structures by the graphic tools, and provide many kinds of network measures to evaluate the knowledge building of CSCL in the social perspectives. Overall, social network analysis provides a new paradigm and method for evaluating CSCL.

1. What is a Social Network Analysis (SNA)? SNA has a history beginning in the early 1930s across many disciplines, including anthropology, psychology, sociology, mathematics, and economics. SNA became much more popular among researchers in the early 1970s when advances in computer technology made it possible to study large groups. Social network analysis as a methodology did not appear in the American Evaluation Association conference program until 1998. Liu Jun (2005) pointed out that Social network analysis could be used to construct the model of social relationship, to discover and describe the social relationship among actors in social networks, and to study the influence on the function of group as well as individual attributes in the group. In the perspectives of collaborative learning, the relationships refer to information communication, knowledge sharing, and trust among students. According to Pattison (1994), the social context of CSCL implies individual social position or social role in a social network, the social position and role are associated with certain regular patterns of social interaction. Pattison claimed that individual position in a social network plays a part in determining the specific information to which the individual is

354

Y. Wang and K. Li / An Application of Social Network Analysis in Evaluation of CSCL

exposed. Furthermore, individuals in different social positions receive different information so that they may come to construct knowledge differently. Carley (1986) emphasized that there is an interaction between social structures and cognitive structures, and social interaction is a driving force behind knowledge acquisition. Thus, in the research of CSCL evaluation, social network analyses can be used to reveal the elements which determine students’ social positions and interactions in the CSCL environment. 2. Methods of Evaluation In the research of CSCL evaluation, SNA analyzes relationships of interaction in the social network by the algebra matrix and graph theory tools, describes patterns of interaction and characteristics of networks with network measures. In this paper, we will introduce five network variables in SNA and their applications in the evaluation of CSCL. 2.1 Indegree Indegree is the number of connections each node has from other nodesˈand is a measure of the extent to which one is chosen as partners by others in the network (Brad Richardson & Nancy Graf, 2004). In a CSCL environment, Indegree is used to provide information about the number of those who read or build on the discourse topic of a certain member, such as in BBS, virtual learning communities and so son. Most of the applications of indegree in CSCL evaluation are listed below: x A high indegree for a specific student indicates that his ideas may be influential in the discourse, and the student has more prestige in the network. x Average Indegree all the members can describes the characteristics of interactive collaboration in the whole CSCL environment. A high average indegree in the network indicates that there is a high degree of building on each other’s discourse topics. x Indegree divergence indicates that there is a large variance in the contribution to the collective intellectual artifact of network, and there is social inequity in the social network. 2.2 OutDegree Outdegree is also a measure of calculating the centrality of a social network. Outdegree is the number of connections each node has to other nodes. In a CSCL environment, Outdegree is a measure which represents the total number of notes by other students that a specific student read or builds on. In a similar way with indegree, we can use three indicators listed below in the evaluation of CSCL. x The size of Outdegree of a student also describes the level of interaction with others. Outdegree is also considered a measure of influence: those with more connections to other nodes have relatively more influence on the activities of the network. But for the reversed direction of information with indegree, outdegree indicates whether the student is an active and enthused learner in a CSCL environment. x The average of outdegree over the network is equal to the average indegree. It describes the whole characteristics of social interaction in a CSCL environment. x A large variance in the outdegree over the network may reflect different levels of commitment to collaborative effort, but it may also reflect individual differences in the qualities of the rest of the network (Li Sha & Jan van Aalst, 2003).

Y. Wang and K. Li / An Application of Social Network Analysis in Evaluation of CSCL

355

2.3 Betweenness Interaction between two non-adjoining actors is dependent on other actors in the network, especially the actors located on paths between the two actors. These actors who have a certain kind of power controlling and restricting the interaction of the two non-adjoining actors are called brokers. The betweenness is a measure to evaluate the extent of brokers’ importance located paths between two actors. For example, in figure 1, if the actor B want to communicate with the actor C, it must transit the actor G. So the actor G have the power of broker or gatekeeper.

Figure 1 Betweenness In a CSCL environment, the betweenness of a student is determined by the extent to which the student is a “broker”. If information flows among students are often indirect or via a given student, the student has a high betweenness. If a network has a high average betweenness, it indicates that there are at least a few information brokers. Such brokers hold powerful social positions in the network, this is undesirable. The network would support a more democratic form of knowledge building when there are many direct links between students, rather than individual links (Li Sha & Jan van Aalst, 2003). 2.4 Density To get an indication of the overall linkage of members in the network we need conduct density calculations. This gives an indication of the level of engagement in the network. Density calculations indicate how active the members are involved in the discourse (Maarten de Laat,2002). So, in a CSCL environment, density can show how dense is the participation within it.

Figure 2 A MDS Map Multidimensional Scaling calculations (MDS) can visualize the patterns of interaction among the students. A MDS in Figure 2 is generated using the tool of NetDraw. The number of discourse topics the students of this network have post and built on indicates how close members are situated on the MDS map. The stress value indicates the quality of the MDS map, we can draw conclusions from the stress value.

356

Y. Wang and K. Li / An Application of Social Network Analysis in Evaluation of CSCL

2.5 Cohesion Cohesion of a social network is presented by cliques generated by the network. In a social network, a clique is a sub-set of a network in which the actors are more closely and intensely tied to one another than they are to other members of the network (Robert A., Hanneman & Mark Riddle, 2005). Cliques can be identified by the cohesion index(C-Idx). The cohesion index is a measure of the degree to which there are strong links within the clique rather than outside of it. There are several methods to make cohesion analysis to describe the quality of knowledge building in a CSCL environment, such as cohesion index, the number cliques and the number of members in one clique. x A low C-Idx indicates there are substantive relationships of interaction among students of different cliques, or a student take on many roles in different cliques. This will widen the scope of communication of information in the whole network. x The number of cliques of a network can somewhat indicates the relationships of interaction among students. If the number of cliques is larger, the interaction among students is dense. This is propitious to knowledge building. x The number of members is large in each clique, this means that the scope of communication among students is wide, which improves the quality of knowledge building.

3. Conclusions Relationships of interaction in a CSCL environment determine social positions of the member, and social positions influence the information which a student is exposed. So, relationships of interaction among students can influence the quality of knowledge building. This paper introduces the meaning and application in knowledge building of variables of social network analysis, such as indegree, outdegree, betweeness, density, and cohesion. However, analyzing the knowledge building with individual knowledge and skill attributes is a blind pot of social network analysis, the research about how relationships of interaction in social networks influence the individual attributes will be an innovative direction in CSCL research.

References [1] Yonggu Wang, Xiaojuan Li. (2004) Web based Adaptive Collaborative Learning Environment Designing, ProceedingV of The 3rd International Conference on Web-based Learning, Beijing, Aug. 8-11,2004. [] Robert A., Hanneman, Mark Riddle. (2005) ,ntroduction to social network methods, http://faculty.ucr.edu/~hanneman/ [] Brad Richardson, Nancy Graf (2004) Measuring Strengths in Community Collaboratives. The Prevention Report, http://www.uiowa.edu/~nrcfcp/publications/documents/20041.pdf. [] Maarten de Laat (2002) Network and content analysis in an online community discourse, The Proceedings of Networked Learning 2002 Conference [] Li Sha, Jan van Aalst (2003) An Application of Social Network Analysis to Knowledge Building, proceeding of annual meeting of the American Educational Research Association, Chicago, April 21-25, 2003.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

357

Understanding Asynchronous Teaching And Learning Dialogues - An integrative approach E. Vassa, F. Concannona, M. LeVoib, K. Littletonc, D. Miellb a Faculty of Education, University of Otago, New Zealand b Psychology Department, the Open University, UK c Faculty of Education and Language Studies, the Open University, UK [email protected] Abstract: This paper discusses methodological considerations in research investigating processes of asynchronous online learning. It challenges existing conceptualisations of online discourse which foreground the cognitive dimension of computer-mediated interactions. Informed by two studies about online university courses at the Open University, UK and the University of Otago, NZ, the paper emphasises the need to re-examine the relationship between the cognitive, the social and the affective aspects of shared work. Based on socio-cultural theorising, it calls for a new framework for the analysis of online discourse which recognizes these three aspects as closely interlinked and central in the process of shared knowledge building. Keywords: socio-cultural theory, social and cultural dimensions of learning, analysis of group interactions

Introduction Despite the increasing number of courses taught through teaching and learning dialogues in an online medium, there is little certain knowledge of the factors that influence the educational quality and outcomes of these dialogues. The relatively new field of Computer Supported Collaborative Learning (CSCL) research has focused upon notions of shared knowledge construction. Some would argue that CSCL contexts are a new paradigm in instructional technology, providing a platform for envisaging learning as essentially social and cultural rather than individual and cognitive [1]. Nevertheless, a number of CSCL studies concentrate on processes associated with knowledge building and see the social context as playing a minor supporting part ([2, 3]). These studies code utterances with a focus on knowledge building functions, foregrounding the cognitive dimension of interactions and thus abstracting these away from the social context. For instance, in a recent study messages were coded according to their role in the cyclical process of knowledge construction as triggers, exploration, integration and resolution, separating these cognitive categories from the fifth additional category of social or other [2]. Consequently, in many CSCL systems, for example CSILE [4] or DUNES [5] the shared knowledge construction metaphor is already written into what is called the ontology of the system so that participants can only communicate using forms that pre-code their utterances into aspects of knowledge building such as claims, rebuttals, queries and so on. Such codes and systems embed a pre-understanding of knowledge-building and how it relates to what some writers refer to as the social dimension. However, current socio-cultural work on online teaching and learning is questioning some of the existing assumptions about knowledge building and the social dimension in

358

E. Vass et al. / Understanding Asynchronous Teaching and Learning Dialogues

the CSCL domain [6, 7]. Socio-cultural research brings social, affective and cognitive aspects of learning and development together, emphasising the need to explore all three dimensions when studying shared learning situations such as face-to-face or online collaboration. This approach also underlines the complex and highly interwoven nature of these dimensions [8]. For instance, a study on a virtual summer-school for psychology undergraduates in the UK found that complex and subtle relational work was an essential part of establishing and maintaining the kinds of interactions between participants in which productive teaching and learning exchanges are possible [7]. The findings of this study provided strong indication of the need to re-examine the relationship between the cognitive, the social and the affective aspects of shared work. An important strand of analysis in the study [7] focused on the tutor’s role in supporting joint meaning making and fostering a collaborative community of enquiry. One of the key issues to emerge from this work concerned the inadequacy of the traditional distinction between the ‘cognitive’, ‘social’ and ‘affective’. Based on socio-cultural theory of learning [9], this paper seeks to explore these issues further.

Analytic framework The proposed analysis builds on existing models developed for the study of face-to-face educational dialogues and applies these to the new context of asynchronous online discussion. Although there are many differences at a discursive level between face-to-face and online discourse, a recent study has shown the value in linking the two disparate learning contexts with regards to methodology [6]. Our research utilizes methods developed by Mercer and Wegerif [10] to explore the construction of knowledge in face to face classroom interaction. The analytic work includes the message-level documentation of the cyclical process of online practical inquiry [11]. Each message is assigned to a phase within the cycle (e.g. trigger, exploration, integration or solution). In addition, cognitive, social and affective presence will be examined and the links between these three aspects of online presence explored within each phase. The definition of cognitive presence follows Garrison and Anderson’s description [11]. It can be broadly defined as the sharing of information, ideas and reflections associated with one of the four functions within the practical inquiry cycle. Social presence is defined as discourse aimed at organizing and managing the group and maintaining effective group processes (for example, by inviting, encouraging and recognizing other participants’ input). Affective presence is defined as the expression of feelings, use of humour, and self-disclosure. To aid the analytic process through visual mapping, a conference activity graph [12] of each cycle is drawn, where each message is numbered and the interaction between these is indicated by the use of arrows. Our aim is to demonstrate that each phase builds on cognitive as well as social and affective presence, and unveil the interconnected nature of these dimensions of online presence. By showing that cognitive, affective and social aspects of the dialogue are all integral to the shared knowledge building process, we aim to reveal the limited descriptive as well as explanatory power of existing analytic approaches. Our ultimate aim is to develop an integrative framework which captures shared online knowledge building in its full complexity. The methodological explorations presented in the paper may ground future work studying how collaborating partners negotiate shared understanding and support each other in the process of learning in virtual spaces. The analytic work is informed by ongoing research using two data sets: the data available from a UK-based study [7], and the data obtained from our current study at the

E. Vass et al. / Understanding Asynchronous Teaching and Learning Dialogues

359

University of Otago, New Zealand. The UK-based study built on data from a virtual summer school of psychology students at the Open University. The communication environment uses First Class conferencing software. In this study we focused on two groups (six students in each) (N=12). The New Zealand-based study follows a core paper in the Postgraduate Diploma in Teaching presented online by the Faculty of Education, University of Otago. The course is a full year (26-week) paper, with 13 students enrolled in 2006. The communication environment uses Web Crossing software.

Analytical challenges: rationale for the integrative framework Collaborative discourse serves both social and intellectual purposes and both of these aspects are crucial in joint knowledge building [8]. Successful collaboration requires the participants’ shared understanding, which involves the development of both personal and task-related intersubjectivity. In line with this argument, recent conceptualisations show social presence as a facilitating condition for the creation of a challenging but respectful learning environment [11]. This is in contrast with previous work positing an inverse correlation between the effectiveness of computer-mediated collaboration (CMC) and socio-emotional communication [13]. Yet, even studies which recognize the centrality of the social or affective aspects of discourse envisage them more as a pre-requisite or antecedent to joint critical thinking, which do not play a continuous, integral part in the joint knowledge building process. Thus, our first analytic challenge lies in demonstrating the all-pervasiveness of the social aspects of online discourse, which is not restricted to the initial phases in the collaborative experience. For example, the joint exploration of an idea or a problem (cognitive presence) may lead to the discussion of role division and group management. Equally, an exchange on different participants’ availability at a particular point in time may lead to modifications in the ways in which the ‘cognitive cycle’ is shaped during that time (for instance, who is involved in the exploration or integration of ideas, how exactly and to what extent). If this is so, discourse aimed at organizing and managing the group or simply supporting each other can play an integral part in shared knowledge building. Another problematic issue is that, in the majority of studies, social presence is often seen as encompassing both affective postings (the expression of emotions or self-disclosure) and cohesive communication (establishing team spirit, trust and group cohesion) [11]. This blurs the boundary between postings with social and affective content, and masks the complexity of online interactive behaviour. A central function of discourse with affective content may be to drive the knowledge building process. The expression of emotions may serve as the ‘trigger’ in the knowledge building cycle, and self-disclosure may support processes of exploration and integration. Thus, affective dimensions can be linked to motivational aspects and shape the content of collaborative work, whereas social dimensions may be associated with management and organisation, shaping the processes of shared knowledge building. So our second analytic challenge is to develop a framework with which we can distinguish these two dimensions of online presence, and thus capture online discourse in its full complexity. Elsewhere, it was argued that analytic models used to study face-to-face or online discourse need to appreciate the layeredness of language, and recognize the complex functional make-up of discourse [8]. Thus, we see the third analytic challenge in depicting the layeredness of asynchronous online discourse and the intertwined nature of cognitive, social and affective functions of online interactions.

360

E. Vass et al. / Understanding Asynchronous Teaching and Learning Dialogues

Summary This paper suggests that cognitive processes involved in shared knowledge building are inextricably interwoven and go hand-in-hand with the development of a social, collaborative community of enquiry. It is also posited that the affective and cognitive dimensions of online presence are closely linked, and messages with affective content can be integral to the cyclical process of practical inquiry. We argue that each type of contribution may foster and develop a supportive community as well as build knowledge. These arguments indicate that further research needs to be undertaken which takes these analytic considerations into account. In our analysis of the two data sets outlined above, our aim is challenge the existing boundaries between social, affective and cognitive functions and show the complexity of online collaborative work. We aim to develop an integrative model for the analysis of online discourse and undertake the examination of the interconnection of cognitive, affective and social aspects of online discourse. References [1] Koschmann, T. (1996). Paradigm shifts and instructional technology, in CSCL: Theory and practice, T. Koschmann, Editor. LEA: Mahwah, NJ. [2] Meyer, K.A. (2003). Face-to-face versus threaded discussions: The role of time and higher-order thinking. Journal of Asynchronous Learning Networks. 7(3): 55-65. [3] Aviv, R., Erlich, Z., Ravid, G. & Geva, A. (2003) Network Analysis of Knowledge Construction in Asynchronous Learning Networks. Journal of Asynchronous Learning Networks. 7(3): 1-23. [4] Scardamilia, M. and Bereiter, C. (1991). Higher Levels of Agency for Children in Knowledge Building: A Challenge for the Design of New Knowledge Media. The Journal of the Learning Sciences. 1(1): 37-68. [5] DUNES - Dialogic and argumentative Negotiation Educational Software. http://www.tessera.gr/dunes/. [6] Littleton, K. & Whitelock, D. (2004) Guiding the Creation of Knowledge and Understanding in a Virtual Learning Environment. CyberPsychology & Behaviour. Vol. 7, No.2: 173-181. [7] Littleton, K., Miell, D., LeVoi, M., Vass, E., Whitelock, D. & Wegerif, R. (2005). Relational work and productive interactions in an asynchronous conference environment. Paper Presented in EARLI. 2005. Nicosia, Cyprus. [8] Vass, E., (2004) Understanding collaborative creativity: An observational study of the effects of the social and the educational context on the processes of young children's joint creative writing. PhD Thesis. April 2004, The Open University. [9] Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA: Harvard University Press. [10] Wegerif, R. & Mercer, N. (1997). Using computer-based text analysis to integrate quantitative and qualitative methods in the investigation of collaborative learning. Language and Education. 11: 271-286. [11] Garrison, D.R. & Anderson, T. (2003). E-Learning in the 21st Century. A Framework for Research and Practice. Routledge Falmer: London. [12] Hara, N., Bonk, C.J. & Angeli, C (2000). Content analysis of online discussion in an applied educational psychology course. Instructional Science. 28: 115-152. [13] Walther, J.B. & Tidwell, L.C. (1995). Nonverbal cues in computer-mediated communication and the effects of chronemics on relational communication. Journal of Organizational Computing. 5(4): 355-378.

Cultural Issues

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

363

Participatory Agent-based Gaming Methodology in Cross-cultural Education: Exploring Efficient and Sustainable Civil Society and Community a

Reiko Hishiyamaa c, Toru Ishidab c Faculty for the Study of Contemporary Society, Kyoto Women’s University, Japan b Department of Social Informatics, Kyoto University, Japan c National Institute of Information and Communication Technologies, Japan [email protected], [email protected] Abstract: In this paper, we describe new methodologies for reinforcing the social consensus building of the multicultural coexistence assistance using participatory simulation in civil societies. In this methodology, both human and intelligent software agents participate in a participatory gaming simulation. They have mutual interactions on the computer network, and accumulate decision-making steps. This simulation, which has mutual interaction between humans and agents, can achieve the gathering and analyzing of beneficial information to solve problems in multicultural civil societies. This type of simulation will help to provide knowledge for the formulation of social consensus, and to create new opportunities for discussion with people with different cultures and social backgrounds. Keywords: Agent-based participatory gaming simulation, Multicultural assistance model, Experimental education, Experimental analysis, e-Learning on the Internet

1. Introduction With the advance of globalization, citizens live together in multicultural backgrounds. In this situation, it is essential to promote the formation of social consensus and improvement of citizen literacy. Meanwhile, simulation or gaming simulation is starting to garner attention in order to cope with societal challenges. We can see many trials, for example, of town management systems[1], ecological management[2-4], virtual market business management[5], etc., and these are used not only for societal problem solving, but also for citizen education and the theoretical elucidation of social issues. The aim of this paper is to introduce and study the integration of two methods: humanand agent-based participatory gaming simulation and citizen panel discussion on perspectives from civil society. This hybrid system will facilitate citizens' learning and communication among citizens with different cultures and social backgrounds. In this paper, we describe a multicultural assistance model and design it as a human- and agent-based participatory gaming simulation. In this simulation, citizens have an opportunity to experience decision-making processes in response to changes in the social environment in the simulation. In the participatory simulation, it is not easy to invite lots of citizens who have different policy or reflect the diverse views of society. In order to realize local communities with diversity and vitality in the simulation, we design intelligent software agents and invite them to participate in the simulation. We analyze the educational effect of

364

R. Hishiyama and T. Ishida / Participatory Agent-Based Gaming Methodology

the incorporation of intelligent agents into participatory simulation.The rest of the paper is organized as follows. Section 2 defines the social model of multicultural coexistence assistance and describes the design of the simulation including the intelligent software agent model. Section 3 describes our experiment setup. Section 4 presents the experimental results, and Section 5 concludes the paper and lists future work.

2. Modeling of multicultural assistance 2.1 Overview Following an increase in the number of foreign residents, daily life support services for foreign residents are expected to grow, involving the provision of daily information required by foreign residents, such as the location of hospitals with foreign language-speaking doctors/nurses or medical translators, consultation concerning daily problems, and handling of alien registration procedures. Foreign-language FM radio broadcasting is also expected to expand as are NPOs 1 related to daily life support for foreign residents. The foreign residents’ survey or panel discussion has been conducted on issues such as the measurement of performance and satisfaction, and the improvement of services. The citizen panels[6] have been used with two main objectives in mind: one is to consult concerning future support policies; the other is the creation of conversation spaces. This type of structured method is useful for reviewing, assessing, and synthesizing citizens’ knowledge in local communities. Participatory gaming simulation is the third policy exercise technique to provide an interface between scientists, academics, and policy makers. This could serve as a communication interface in order to create excellent dialogue between cultures. As mentioned above, the main characteristic of participatory gaming simulation is to integrate policy exercise techniques and citizen panels into social simulation. The chief aim is to encourage dialogue about support services for foreign residents using the participatory gaming simulation as an e-learning pedagogical model on a regional scale.

2.2 The modeling framework This section presents the computational model of the details of multicultural assistance. Our modeling framework consists of a government and 10 regional assistance offices as regional support centers, Cisub (i=1, 2, …, 10), for foreign residents. Each regional support center is responsible for setting multicultural assistance policies and conducting services for foreign residents. The government is responsible for overseeing all support offices, prioritizing budget allocation to each support center according to the assessment of their performance, which is the foreign residents’ satisfaction index. The inverse proportion relationship to the foreign residents’ satisfaction index, įi (t-1), in term (t-1) is adopted as prioritizing budget allocation in term t. The proportion of the overall budget, F(t), is allocated to each center as a support grant, Fi(t), and they conduct the multicultural assistance program. Each regional support center can decide the content of multicultural support and its level, as shown in Table 1. The definitions of these services are referenced from the real unprofitable activity case of the Center for Multicultural Information and Assistance[7], an NPO in Kyoto, Japan. 1

Nonprofit organizations.

365

R. Hishiyama and T. Ishida / Participatory Agent-Based Gaming Methodology

Contents

Table 1. Setting the support contents and their levels Level 1 Level 2 Level 3

Cost

0

200

Level 4

400

600

Counseling service

No

Once a week

4 times a week

7 timesa week

Medical interpreter dispatch service

No

3 hospitals

10 hospitals

20 hospitals

Cultural exchange event

No

Once a year

6 times a year

12 times a year

There are 3 types: counseling services, medical interpreter dispatch services, and cultural exchange events. Each type has 3 levels. The higher the support level, the more people find the service to be satisfactory. In each term, they can select any support services simultaneously. They have to pay the cost shown in Table 1 if they perform these services. The total service cost, ai(t)=( ailif(t), aimed(t), aicul(t)), hinges on the performance of support as a combination of the selected support type and level. If Level 3 or 4 is selected, the government tacks on the intensive support power value, İi(t)=( İilif(t), İimed(t), İicul(t)), to welcome and encourage ongoing service. Thus, the total of the foreign residents’ satisfaction index, įi(t), is calculated as ai(t)+ İi(t). The line plot in Figure 2 shows the three defined types of intensive support power, which are plotted as round-by-round data. These data are based on administrative support policies adopted by the real city government. In the simulation, each center has to roll over the profits [or losses] to the next fiscal term (t+1).

2.3 Introduction of agent participants Participatory gaming simulation requires many human participants who have various policies. In order to conduct the simulation, participants must gather in the same place at the same time. Therefore, it is not easy to conduct large-scale simulation. We propose the introduction of intelligent software agent as participants instead of human participants in order to conduct the simulation without any geographic or time constraints. Even more than overcoming these constraints, agents can play a wide range of roles in the simulation. We can design agents who have various policies. If agents share the simulation space with humans and they can exert influence on human beings, they can lead to the fulfillment of role of humans. In this research, the three intelligent software agent models representing the basic concept of the policy are as follows: Type-1: Agents have a policy that faithfully reflects the survey results of consultation needs for foreign residents by the NPO, the Center for Multicultural Information and Assistance. Type-2: Agents have a policy that reflects the research results of consultation needs, which has a less significant effect than Type-1 agents’ policy. Type-3: Agents make decisions at each support level that exceed the average of all other participant support levels.

366

R. Hishiyama and T. Ishida / Participatory Agent-Based Gaming Methodology

3. Implementation of the simulation We implement a human- and agent-based participatory gaming simulation environment based on the model described in Section 2, where a human participates in the simulation game in the role of a director of a support center. The simulation consists of ten decision-making rounds, each corresponding to one virtual fiscal year. The simulation engine resides on the Web server. A simulation designer prepares the gaming scenario model written as a computable expression. The simulator calculates the costs and benefits of the policy using this model and sends operational performance data, for example, the service contents (including another participant’s data), balance of payments position, etc.Simulation of the round-processing type provides the chance to conduct self-evaluation of decision making and behavioral modifications. The architecture of the simulation system is shown in Figure 1.

Figure 1. The architecture of the simulation system

4. Experiments and results 4.1 Experimental settings As a first step of this research, the 14 participants are divided into two groups: A and B. One group has 7 human participants and 3 agents (1 Type-1 agent, 1 Type-2 agent, and 1 Type-3 agent) that coexist in the same simulation space, the other group has 7 human participants and 3 agents (1 Type-1 agent, 2 Type-2 agents) that coexist in the same simulation. A facilitator leads each group in the simulation simultaneously; thus, there are no human overlaps between Group A and Group B. Each participant is assigned a special project where he or she manages his or her own support center. We conduct an anonymous survey after performing the simulation trials. The purpose of the survey is to study the effects of intelligent software agents on the view of human participants through participatory gaming simulation. We ask the participants to tell us, “Who was computer agent?” “How concerned you were about the decision making of others?” The experimental procedure is as follows. After handing out materials as a scenario that defined the participant’s role, we give a short lecture to confirm important points. The participants play the simulation via the assigned roles and answer the questionnaire. At the

367

R. Hishiyama and T. Ishida / Participatory Agent-Based Gaming Methodology

end of the experiment, the participants are invited to a debriefing. They can discuss the results with each other and make a connection between experiences via the participatory gaming simulation and experiences in real-life situations.

4.2 Experimental results The participatory gaming simulation was played in ten rounds. The participants made decisions about the level of each service function according to the monitored live data. Figure 2 shows acquired satisfaction in Group A at the end of gaming simulation (after Round 10), Figure 3 shows acquired satisfaction in Group B at the end of gaming simulation (after Round 10). Figure 4 shows acquired profit or loss in Group A at the end of gaming simulation (after Round 10), and Figure 5 shows acquired profit or loss in Group B at the end of gaming simulation (after Round 10). Figure 6 shows temporal changes in cumulative profit or loss. Centers 4, 5 and 6 in Group A and Centers 1, 8 and 9 in Group B are conducted by intelligent software agents. Focused on agents’ behaviors in Group A, Center 4 is in top position in the acquired profit although he/she has no advantage in satisfaction. Center 5 showed the best-balanced performance in profit and satisfaction. Center 6 is in top position in satisfaction although he/she has no advantage in acquired profit. Focused on agents’ behavior in Group B, Center 1 showed the best-balanced performance in profit and satisfaction. Centers 8 and 9 are in top position in satisfaction although they have no advantage in acquired profit. 㪞㫉㫆㫌㫇㪘㩷㪑㩷㩷㩷㪚㫌㫄㫌㫃㪸㫋㫀㫍㪼㩷㫃㫀㪽㪼㩷㫊㪸㫋㫀㫊㪽㪸㪺㫋㫀㫆㫅㩷㫆㪽㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㪽㫆㫉㪼㫀㪾㫅㩷㫉㪼㫊㫀㪻㪼㫅㫋㫊䇭㫀㫅㩷㫉㫆㫌㫅㪻㩷㪈㪇

㪞㫉㫆㫌㫇㩷㪙㩷㪑㩷㩷㪚㫌㫄㫌㫃㪸㫋㫀㫍㪼㩷㫃㫀㪽㪼㩷㫊㪸㫋㫀㫊㪽㪸㪺㫋㫀㫆㫅㩷㫆㪽㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㩷㪽㫆㫉㪼㫀㪾㫅㩷㫉㪼㫊㫀㪻㪼㫅㫋㫊䇭㫀㫅㩷㫉㫆㫌㫅㪻㩷㪈㪇

㪋㪌㪇

㪋㪌㪇

㪋㪇㪇

㪋㪇㪇

㪊㪌㪇

㪊㪌㪇㪊㪇㪇

㪉㪌㪇

㪠㪥㪛㪜㪯

㪠㪥㪛㪜㪯

㪊㪇㪇㪉㪇㪇㪈㪌㪇

㪉㪌㪇㪉㪇㪇㪈㪌㪇

㪈㪇㪇

㪈㪇㪇

㪌㪇

㪌㪇

㪇㪈

㪉

㪊

㪋

㪌

㪍

㪎

㪏

㪐

㪇

㪈㪇

㪈

㪚㪼㫅㫋㪼㫉㩷㪥㫆㪅

㪉

㪊

㪋

㪌

㪍

㪚㪼㫅㫋㪼㫉㩷㪥㫆㪅 Figure 2 (left). Group A: cumulative life satisfaction Figure 3 (right). Group B: cumulative life satisfaction

㪋㪇㪇㪇

㪊㪇㪇㪇

㪊㪇㪇㪇

㪉㪇㪇㪇

㪉㪇㪇㪇

㪄㪈㪇㪇㪇

㪉

㪊

㪋

㪌

㪍

㪄㪉㪇㪇㪇

㪎

㪏

㪐

㪈㪇

㪘㫄㫆㫌㫅㫋

㪘㫄㫆㫌㫅㫋

㪈㪇㪇㪇㪈

㪈㪇

㪇㪄㪈㪇㪇㪇

㪄㪊㪇㪇㪇㪄㪋㪇㪇㪇

㪄㪊㪇㪇㪇

㪈

㪉

㪊

㪋

㪌

㪍

㪄㪋㪇㪇㪇

㪚㪼㫅㫋㪼㫉㩷㪥㫆㪅

㪐

㪈㪇㪇㪇

㪄㪉㪇㪇㪇

㪄㪌㪇㪇㪇

㪏

㪞㫉㫆㫌㫇㩷㪙㩷㪑㩷㪚㫌㫄㫌㫃㪸㫋㫀㫍㪼㩷㫇㫉㫆㪽㫀㫋㩷㪸㫅㪻㩷㫃㫆㫊㫊㩷㫀㫅㩷㫉㫆㫌㫅㪻㩷㪈㪇

㪞㫉㫆㫌㫇㩷㪘㩷㪑㩷㪚㫌㫄㫌㫃㪸㫋㫀㫍㪼㩷㫇㫉㫆㪽㫀㫋㩷㪸㫅㪻㩷㫃㫆㫊㫊㩷㫀㫅㩷㫉㫆㫌㫅㪻㩷㪈㪇㪋㪇㪇㪇

㪇

㪎

㪚㪼㫅㫋㪼㫉㩷㪥㫆㪅

Figure 4 (left). Group A: cumulative profit and loss Figure 5 (right). Group B: cumulative profit and loss

㪎

㪏

㪐

㪈㪇

368

R. Hishiyama and T. Ishida / Participatory Agent-Based Gaming Methodology

㪞㫉㫆㫌㫇㩷㪘㩷㪑㩷㪩㫆㫌㫅㪻㩷㫋㫉㪸㫅㫊㫀㫋㫀㫆㫅㩷㫆㪽㩷㪺㫌㫄㫌㫃㪸㫋㫀㫍㪼㩷㫇㫉㫆㪽㫀㫋㩷㪸㫅㪻㩷㫃㫆㫊㫊

㪋㪇㪇㪇㪊㪇㪇㪇㪉㪇㪇㪇

㪘㫄㫆㫌㫅㫋

㪈㪇㪇㪇㪇㪄㪈㪇㪇㪇

㪩㪈

㪩㪉

㪩㪊

㪩㪋

㪩㪌

㪩㪍

㪩㪎

㪩㪏

㪩㪐

㪩㪈㪇

㪄㪉㪇㪇㪇㪄㪊㪇㪇㪇

㪟㪈㪟㪉㪟㪊㪘㪋㪘㪌㪘㪍㪟㪎㪟㪏㪟㪐㪟㪈㪇

㪄㪋㪇㪇㪇㪄㪌㪇㪇㪇㪩㫆㫌㫅㪻㩷㩿㩷㪝㫀㫊㪺㪸㫃㩷㪰㪼㪸㫉㩷㪀

Figure 6(a). Group A: round transition of cumulative profit and loss 㪞㫉㫆㫌㫇㩷㪙㩷㪑㩷㪩㫆㫌㫅㪻㩷㫋㫉㪸㫅㫊㫀㫋㫀㫆㫅㩷㫆㪽㩷㪺㫌㫄㫌㫃㪸㫋㫀㫍㪼㩷㫇㫉㫆㪽㫀㫋㩷㪸㫅㪻㩷㫃㫆㫊㫊㪋㪇㪇㪇㪊㪇㪇㪇㪉㪇㪇㪇

㪘㫄㫆㫌㫅㫋

㪈㪇㪇㪇㪇㪩㪈

㪩㪉

㪩㪊

㪩㪋

㪩㪌

㪩㪍

㪩㪎

㪩㪏

㪄㪈㪇㪇㪇㪄㪉㪇㪇㪇

㪩㪐

㪩㪈㪇㪩㪈㪈

㪘㪈㪟㪉㪟㪊㪟㪋㪟㪌㪟㪌㪟㪎㪘㪏㪘㪐㪟㪈㪇

㪄㪊㪇㪇㪇㪄㪋㪇㪇㪇㪩㫆㫌㫅㪻㩷㩿㩷㪝㫀㫊㫀㪺㪸㫃㩷㪰㪼㪸㫉㩷㪀

Figure 6(b). Group B: round transition of cumulative profit and loss

Decision making in each round of the experiment is shown in Figures 7(a), 7(b) and 7(c). If the participants show a deficit, they first try to cut financial expenditure on cultural exchange events. Secondly, they try to cut medical interpreter dispatch services and counseling services to avoid a deficit. This seemed to be the same trend in experimental results of Group B. They tried to spend their budget on medical interpreter dispatch services and counseling services when most centers has a surplus after Round 5 in Group A. Thus, these two services are high on their list of priorities in multicultural support policy. After the simulation, we asked participants which centers were owned and conducted by intelligent software agents during the simulation. In Group A, of a population of 7, 4 answered that Center 6 was conducted by a software agent. However, of a population of 7, 4 answered that Center 3 was also conducted by a software agent. Of a population of 7, 3 answered that Center 8 was conducted by a software agent. At least more than one participant estimated that agents conduct all of the centers (from Center 1 to Center 10). Thus, we assumed that it was difficult for participants to recognize whether decisions had been made by humans or by agents. On the other hand, in Group B, most participants answered that Centers 8 and 9 were conducted by agents because of the constant high-level

369

R. Hishiyama and T. Ishida / Participatory Agent-Based Gaming Methodology

support regardless of deficit. We prepared a questionnaire to evaluate how much are you affected by other people's opinions about your decision making and how were you affected by other people's ideas during this simulation. Of a population of 14 (participants of Groups A and B), 12 answered that they referred greatly to other centers’ decision making. According to the answers, most of the data that they referred to was the other centers’ satisfaction and selected services. 㪞㫉㫆㫌㫇㩷㪘㩷㪑㩷㩷㫊㪼㫃㪼㪺㫋㪼㪻㩷㫊㪼㫉㫍㫀㪺㪼㩷㫃㪼㫍㪼㫃㩷㫆㪽㫄㪼㪻㫀㪺㪸㫃㩷㫀㫅㫋㪼㫉㫇㫉㪼㫋㪼㫉㩷㪻㫀㫊㫇㪸㫋㪺㪿㩷㫊㪼㫉㫍㫀㪺㪼

㪎

㪎

㪍

㪍

㪫㪿㪼㩷㫅㫌㫄㪹㪼㫉㩷㫆㪽㩷㪺㪼㫅㫋㪼㫉㫊

㪫㪿㪼㩷㫅㫌㫄㪹㪼㫉㩷㫆㪽㩷㪺㪼㫅㫋㪼㫉㫊

㪞㫉㫆㫌㫇㩷㪘㩷㪑㩷㩷㫊㪼㫃㪼㪺㫋㪼㪻㩷㫊㪼㫉㫍㫀㪺㪼㩷㫃㪼㫍㪼㫃㩷㫆㪽㪺㫆㫌㫅㫊㪼㫃㫃㫀㫅㪾㩷㫊㪼㫉㫍㫀㪺㪼㩷㪽㫆㫉㩷㪝㫆㫉㪼㫀㪾㫅㪼㫉㫊

㪌

㪌㪎㩷㫋㫀㫄㪼㫊㩷㪸㩷㫎㪼㪼㫂㪋㩷㫋㫀㫄㪼㫊㩷㪸㩷㫎㪼㪼㫂㫆㫅㪺㪼㩷㪸㩷㫎㪼㪼㫂㫅㫆

㪋㪊

㪉㪇㩷㪿㫆㫊㫇㫀㫋㪸㫃㫊㪈㪇㩷㪿㫆㫊㫇㫀㫋㪸㫃㫊㪊㩷㪿㫆㫊㫇㫀㫋㪸㫃㫊㫅㫆

㪋㪊㪉

㪉

㪈

㪈

㪇

㪇㪝㪰㪈

㪝㪰㪉

㪝㪰㪊

㪝㪰㪋

㪝㪰㪌

㪝㪰㪍

㪝㪰㪎

㪩㫆㫌㫅㪻㩷㩿㩷㪝㫀㫊㪺㪸㫃㩷㫐㪼㪸㫉㩷㪀

㪝㪰㪏

㪝㪰㪐

㪝㪰㪈

㪝㪰㪈㪇

㪝㪰㪉

㪝㪰㪊

㪝㪰㪋

㪝㪰㪌

㪝㪰㪍

㪝㪰㪎

㪝㪰㪏

㪩㫆㫌㫅㪻㩷㩿㩷㪝㫀㫊㪺㪸㫃㩷㪰㪼㪸㫉㩷㪀 Figure 7(a) (left). Group A: counseling services Figure 7(b)(right). Group A: medical interpreter dispatch services

㪝㪰㪐

㪝㪰㪈㪇

㪞㫉㫆㫌㫇㩷㪘㩷㪑㩷㩷㩷㫊㪼㫃㪼㪺㫋㪼㪻㩷㫊㪼㫉㫍㫀㪺㪼㩷㫃㪼㫍㪼㫃㩷㫆㪽㪺㫌㫃㫋㫌㫉㪸㫃㩷㪼㫏㪺㪿㪸㫅㪾㪼㩷㪼㫍㪼㫅㫋㪎㪈㪉㩷㫋㫀㫄㪼㫊㩷㪸㩷㫐㪼㪸㫉㪍㩷㫋㫀㫄㪼㫊㩷㪸㩷㫐㪼㪸㫉㫅㫆㪺㪼㩷㪸㩷㫐㪼㪸㫉㫅㫆

㪫㪿㪼㩷㫅㫌㫄㪹㪼㫉㩷㫆㪽㩷㪺㪼㫅㫋㪼㫉㫊

㪍㪌㪋㪊㪉㪈㪇㪝㪰㪈

㪝㪰㪉

㪝㪰㪊

㪝㪰㪋

㪝㪰㪌

㪝㪰㪍

㪝㪰㪎

㪩㫆㫌㫅㪻㩷㩿㩷㪝㫀㫊㪺㪸㫃㩷㫐㪼㪸㫉㩷㪀

㪝㪰㪏

㪝㪰㪐

㪝㪰㪈㪇

Figure 7(c). Group A: cultural exchange events Figure 7. Group A: selected service levels

4.3 Discussion Participatory gaming simulation provides findings on the support policy. It also establishes the policy that participants are expected to seek necessary support actions. Thus, it is hoped that counseling services and medical interpreter dispatch services should be conducted in priority to multicultural event support. It also establishes the policy that places special emphasis on counseling services and medical interpreter dispatch services in approximately equal measure. We are interested in supportive relationships between humans and agents. If the agents’

370

R. Hishiyama and T. Ishida / Participatory Agent-Based Gaming Methodology

behaviors stay within the scope of human assumption and respond to diverse situations, they will be able to be integrated into participatory simulation and interact with humans. It is possible that the agents will be able to participate in simulation without being identified as agents and play the same role as humans. Furthermore, according to the survey, human participants refer to other centers’ performance data. Therefore, we can assume that participatory gaming simulation, as simulation where humans and agents play, bring about greater synergy among them. It is difficult to conduct large-scale participatory simulation experiments because we require many citizen participants. If we invite software agents instead of human citizen participants, we can provide the potential for large-scale simulation.

5. Conclusions The experiments showed that participatory gaming simulation provides opportunities for participation in the decision-making processes of policies and administrative measures. Human- and agent-based participatory simulation will play an important role as a common communication interface between humans and agents. Citizenship education using this type of simulation drives down the cost of the participation of human citizens. Therefore, humanand agent-based simulation is beneficial for civic education. As a future work, we are planning to conduct large-scale human- and agent-based participatory gaming simulation, where foreign citizens are invited from remote locations via the Internet. Using the model acquired in this experiment, we plan to conduct the next simulation and gain an understanding of the differences or similarities among citizens with various cultural backgrounds.

References [1] Kaneda, T., Yokoi, I and Takahashi, S., “Town Management Gaming Simulation by using Pedestrian Shop-Around Behavior Model,” International Simulation and Gamins Association, CD-ROM, 2001. [2] Underwood, S. and Duke, R. SEIDL, the Ecosystem Philosophy Game, Role Manual, 1995. [3] Dahinden and Urs et al. Using computer models in participatory integrated assessment – Experiences gathered in the ULYSSES project and recommendations for further steps, ULYSSES working paper, 1999. [4] Torii, D., Ishida, T, Bonneaud, S and Drogoul, A. Layering Social Interaction Scenarios on Environmental Simulation. Workshop on Multiagent and Multiagent-based Simulation (MAMABS), International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS-04), pp. 61-70, 2004. [5] Terano, T. et al. U-Mart: An Artificial Market Testbed for Economics and Multiagent Systems, 2nd International Workshop on Agent-based Approaches in Economics and Social Complex Systems, pp.55-62, 2002. [6] Renn et al. Public Participation in Decision Making – A Three-Step Procedure, Policy Sciences, 26, pp.189-214, 1993. [7] Center for Multicultural Information and Assistance, in Kyoto, Japan.(NPO Tabunka Kyosei Center in Kyoto) http://www.tabunka.jp/english/english.html

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

371

Internet for Senior Citizens in China: Survey and Proposal Wei ZHOU* Takami YASUDA** Shigeki YOKOI** Graduate School of Information Science, Nagoya University Furo-cho Chikusa-ku, Nagoya 464-8601 Japan *[email protected] **{yasuda ,yokoi}@is.nagoya-u.ac.jp Abstract: China has become the second largest Internet-using country in the world, but the percentage of use by those aged above 50 is extremely low, at less than 3.5%, and there are few studies aimed at senior citizens. In this paper we first investigate senior citizens’ Internet anxiety and analyze the sub-types of Internet anxiety and other attributes. Next, we further explore seniors’ willingness to use the Internet as well as difficulties and obstacles they often face. Then, to ease their anxieties and satisfy their needs, we propose a system called the Senior Internet Support & Learning Environment and demonstrate it. Finally we demonstrate the proposal’s usability. The implications of the study are valuable: it is based on the conditions of China and highlights the differences faced in China in comparison to overseas studies. Moreover, the proposed environment can provide Internet developers with a better understanding of the situation of local senior citizens with respect to the Internet, as well as a framework for further implementing the development phase for narrowing the digital divide in China. Keywords: Internet anxiety, digital divide, Internet needs, senior citizens, China

1. Introduction The Internet has been developing rapidly in China. Although the Internet penetration rate is high among the younger generation, the majority of the elderly do not seem to use it. A recent survey [1] by CNNIC revealed that over 71.0% of those aged under 30 years use the Internet, whereas the percentage of use by those aged over 50 years old is extremely low, at less than 3.5%. For comparison, in Japan the figure is 15% (aged 65 and over) (2004), and 27.7% in America (aged 65 and over) (2005). The digital divide, as in many other developing countries, has been a major concern in China, and is most noticeable along educational and age dimensions [5]. In China there have been many studies conducted on Internet support methods, but most focus on support for children and working adults, not for the elderly. Consequently, there are hardly any reports and research papers on senior Internet users, especially on Internet support methods for them. Therefore, the research questions we should consider are: (1) What anxieties do senior citizens have when using or preparing to use the Internet? (2) What are their needs toward the Internet? (3) What implementation strategies could be suggested based on the findings in (1) and (2)? The Internet has rapidly and dramatically changed the way people live and learn. In order to encourage and help more senior citizens to enjoy their lives in the Internet age, we conducted a survey for senior citizens of their Internet anxieties, information needs, and learning and support needs (in section 3). Based on the survey results (in section 4), we propose an Internet support and learning environment for seniors (in section 5), and finally demonstrate it (in section 6). The implications of the research are valuable. It is an

372

W. Zhou et al. / Internet for Senior Citizens in China: Survey and Proposal

experimental study based specifically on the conditions of senior citizens in China and highlights the differences from research undertaken overseas. Furthermore, it provides Internet applications developers, planners, and designers with a better understanding of the Internet learning and support needs of local senior citizens, as well as a reference for further implementing the development phase for narrowing the digital divide in China. 2. Related Study 2.1 Internet anxiety One factor that inhibits Internet usage is “Internet anxiety,” a concept that Presno [8] proposed in 1998. Not surprisingly, people are expected to avoid behaviors that arouse nervousness. Before the Internet had widely been connected to standalone computers, a lot of careful research had been done on computer anxiety. We think that there are factors unique to Internet anxiety that is not included in computer anxiety as other studies [8-9] mentioned, so in our study we mainly consider Internet anxiety. Although studies on Internet anxiety are abundant, the majority of them use college students as samples, or those based on the general public or employees in companies. However, there are few studies aimed at senior citizens, especially in China. The purpose of this paper is to investigate Internet anxiety as experienced by senior citizens and discuss the methods and solutions to ease it. 2.2 Internet support methods Since current websites are deemed twice as difficult for seniors to use than they are for younger users [11], it is necessary to put more emphasis on making the Web more usable for older people. Based on usability [6] and accessibility [7] guidelines, many studies made the Websites easier to use. The Web browser tool is another important factor that can be exaggerated into serious usability problems. Therefore, many new browser tools [4] are being developed and designed for senior citizens. Besides the technology support mentioned above, social support methods had been planned and implemented in several countries in recent years. Actual social support environment, such as “E-namokun” project [2] we have done since 2004 in Japan, and virtual society Websites, such as SeniorNet in US, have been established to provide a community for senior citizens to communicate, learn, and find information. As a developing country, there is a wide gap in social foundation between China and other countries. Therefore, the support methods for senior citizens in China should consider not only the support from the social foundation but also the effect on senior citizens’ self-learning and support from the circle of their friends and family. In summary, unlike such countries which have recognized Internet anxiety and developed many Websites specifically for senior citizens, and brought a social foundation to maturity, in China, such work can hardly be found, nor has much research effort been expended in this field. Therefore, it is significant to conduct this research and find a suitable solution for senior citizens in China.

W. Zhou et al. / Internet for Senior Citizens in China: Survey and Proposal

373

3. Methodology The study uses 50 years of age or older as survey age because in most areas of China the actual retirement age year is 50 (for females) and 55 (for males). Most of the participants are living in Shijiazhuang city, the capital of Hebei province, whose Internet usage percentages of local population, domain names and Websites are somewhat average level in China [1]. The participants were reached through two channels: 1.the first channel consisted of senior citizens from Hebei Senior University, an informal university organization open only to senior citizens; 2. the second channel was parents of the authors’ acquaintances, for comparison, none of whom have participated in any social learning organization. The questionnaire comprises six sections and 24 questions including: 1 demographic, 2 physical health condition, 3 Internet usage and experience, 4 information needs, 5 Internet anxiety, and 6 support and learning Needs. The questions and response choices were considered with various other studies [1, 2, 3, and 9]. To ensure the questionnaire speaks the user’s language, a pilot survey (five seniors) and interview (two seniors) were conducted before the actual questionnaire was administered. Data collection was conducted from January 15, 2006 to March 30, 2006. 52 questionnaires were collected by Hebei Senior University and 51 questionnaires by supporters (authors and acquaintances). All of the questionnaires were carefully checked and their validity was confirmed. Of the 103 questionnaires returned, 22 were from non-Internet users, so Sections III, IV, and IV were not answered by them. The other 81 were Internet users and completed the forms fully. The results comprised a mix of both genders, with 59 (57.3%) of males and 44 (42.7%) of females. These subjects were aged from 50 to 55 with 29 (28.2%) of the total, 23 (22.3%) were 56 to 60, 25 (24.3%) 61-65 years old, 18 (17.5%) 66-70 years old, and 8 (7.8%) were over 71 years old. Above half the respondents thought of themselves as having “satisfactory” health. There are several limitations to this survey: A convenient sampling approach was used instead of random sampling, so it was hard to determine the probability that any particular population element has been included in the sample; the proportion of the sample from Hebei Senior University was 50.5%, maybe result in a positive influence on the learning willingness aspect. Nevertheless, the data collected in this study provided some useful insights into the support and learning methods for senior citizens, and could serve as inputs for a more comprehensive survey in the future. 4. Survey Results 4.1 Internet Anxiety Regarding Internet anxiety, we used eight question items that 78 users answered. All question items were scored on a five-point Likert-type scale from “totally agree” scored as 1 to “totally disagree” scored as 5. To investigate the scale’s factor structure, we conducted a series of exploratory factor analyses using Promax rotation on those eight items (KMO=0.698, p<0.001). As a result, the following three factors were elicited (Table 1): “Internet ability anxiety” (questions 1,3,6,8) was interpreted as the factor related to a lack of Internet knowledge and ability to search for or select appropriate information; “Internet reliability anxiety” (questions 2,4,5), was interpreted as the factor relating to strain and anxiety in exchanging information on the Web, such as viruses, security and privacy; and “Internet acceptability anxiety” (question 7) was interpreted as the factor dealing with avoiding using the Internet. From the table we can say that senior citizens have more ability anxiety and reliability anxiety. However, we found they have less acceptability anxiety, which means most respondents consider it was as common an occurrence to obtain information from the Internet as from other Medias such as newspaper.

374

W. Zhou et al. / Internet for Senior Citizens in China: Survey and Proposal

Table1. Result of factor analyses for Internet anxiety Question items Mean SD 1. I become uncomfortable when hearing technical terms about 1.84 0.97 Internet. 2. I become uneasy that I cannot find trusted information. 2.32 0.97 3. The Internet has too much information and I become irritated. 2.03 0.99 4. I become uneasy that personal information may leak onto the 2.10 1.17 Internet. 5. I become uneasy that the computer will become infected when 1.68 0.97 using the Internet. 6. If something happens, I will get into trouble because I won't 1.86 1.02 know how to resolve the problem. 7. I want to obtain information from paper sources rather than 3.76 0.88 from the Internet. 8. Since the Internet is progressing rapidly, I worry that I will be 1.90 0.97 left behind. Mean SD

F1 .672

F2 .197

F3 .065

-.146 .764 -.027

.831 -.028 .885

.277 .201 -.165

.242

.703

-.077

.935

-.122

.101

.192

.021

.931

.625

.013

-.429

1.91 0.75

2.03 0.85

3.76 0.88

Extraction Method: Principal Component Analysis. Rotation Method: Promax with Kaiser Normalization. The lower the mean scores, the higher the anxiety.

A Pearson’s correlation test showed that there was a positive relationship between the “ability anxiety” and “reliability anxiety” (r = 0.49, p<0.05). It is easy to understand that seniors who had little prior Internet knowledge or ability will easily experience reliability anxiety. However, the correlation of “ability anxiety” to “acceptability anxiety,” and “reliability anxiety” to “acceptability anxiety” was extremely low. It may be explained that in spite of low Internet ability and unreliability of the Internet, seniors have accepted the Internet as a familiar part of modern society, and would like to face the challenges the Internet presents. Moreover, we conducted one-way ANOVA to analyze the Internet anxiety factor among users for different attributes. We found that there were no statistical differences between males’ anxiety factors and females’ factors, and among different age groups. But in terms of the “health”, people with good “eyesight” (F=6.15, p<0.01), good “precision of movement” (F=6.03, p<0.01), and good “memory and understanding” (F=4.32, p<0.05) felt less ability anxiety than people with poor health. The results also indicated that those actively participating in courses at social facilities (F=10.25, p<0.01) expressed less Internet ability anxiety than those who did not. This result suggests that social and public communities have a positive effect on easing senior citizens’ ability anxiety. In the Internet usage frequency and usage experience (long user or short user), the results showed that the more time (F=3.39, p<0.05) spent on the Internet, the more pleasure users had from it. There was also a reduction in Internet ability anxiety (F=2.81, p<0.05) deriving from more use. Moreover, users who had more than one year of Internet experience (F=6.72, p<0.001) had more desire to use Internet than those who had little Internet experience. In summary, senior citizens have ability anxiety and reliability anxiety, especially those suffering poor health, those who do not attend social facilities, and Internet novices. However, despite these points, they did consider that they would like to obtain information from the Internet just the same as from other media sources, most of them want to face the Internet’s challenges.

W. Zhou et al. / Internet for Senior Citizens in China: Survey and Proposal

375

4.2 Internet Needs 4.2.1 Information Needs To identify the Web-based information needs of senior citizens, we examined three related factors based on the questionnaire responses. These are (1) Topics of interest; (2) Information-finding methods; and (3) Internet services. All these were multiple-choice questions, with 88 users answered them. Regarding the topics of interest of senior citizens, the most popular topic was news (71.59%), just like for other age groups from the CNNIC survey results, but the topics of shopping, finance, and fashion were lower than those of other generations. The most interesting find is that learning/reading (27.27%) is higher than the average level (CNNIC 9.1%), which means that senior citizens want to learn more things in their leisure time. As for the factor of Internet services, overall the respondents’ main uses of the Internet were to seek information and surf the Net (84.09%). They also used email (29.55%) and chat (19.32%). This demonstrated that senior citizens also want to communicate with others online to enrich their lives during retirement. As for the factor of information-finding methods, unlike other generations, who use Internet keywords and directory search functions in the majority, senior citizens would rather get recommendations of useful Websites or information from friends, family, and acquaintances (54.55%). These results suggest that there was some association between the respondents’ Internet competency and the information-finding methods. 4.2.2 Learning Needs The results also revealed that 84.1% of the senior citizens surveyed want to learn more Internet knowledge. When asked about which factors were obstacles, 97.09% answered that there were many factors that disturb learning. The main reason was health (69.90%) (The ability of memory and understanding typically deteriorate). From the second reason: “ There are no suitable learning materials,” (39.81%) we convinced that developing learning materials considering senior citizens needs and physical attribute is necessary thing to promote learning. Beside it, the lack of social learning courses (30.10%), social facilities (30.10%), and guider/instructor (33.01%) were very serious reasons. 4.2.3 Support Needs (1) Web Interface Design As for Web interface design and usability, users had some complaints such as the flickering words and pictures that easily cause weariness (30.23%), the font size is too small to see (26.16%), there is too much content on one page (23.26%), the Web links are not easy to distinguish (11.63%), and so on. Therefore, it is necessary to put more emphasis on usability and accessibility design to change the current layout of Web pages and make it easier for older people to understand. (2) Internet Browser Tool Of the 78 people who answered the questions on this topic, 85.89% used IE (Microsoft Internet Explorer) as the Web browser. They met several difficulties such as: 1. there are too many unused functions (59.26%); 2. the pointer is too small to find easily (24.07%); and 3. the buttons are too small to operate (15.74%). These replies indicate that conventionally designed browsers are likely to create additional problems for seniors. It is necessary to note that aging is associated with a narrowing of the visual field and slower movement, so the interfaces designed for seniors should take into account these attributes. (3) Social Support Method

376

W. Zhou et al. / Internet for Senior Citizens in China: Survey and Proposal

To identify which support methods are suitable for senior citizens in China, two questions were asked, with 94 users answered. Regardless of whether their lifestyle involves living with their family or not, most seniors still want support from their family and friends (94.68%). One reason is that since service costs are higher (40.43%), seniors would rather obtain support from family and friends rather than from computer companies. On the other hand, they worried somewhat that this may impose on others (38.30%). Meanwhile, they also worried that they could not find a suitable guide to solve their Internet-related problems (58.51%), thus they would be in a bind when new difficulties arise (43.62%). Some of these senior citizens, consequently, would like to choose to learn and find solutions by themselves (28.72%). Therefore, we claim that, in contrast to other countries, the support method for senior citizens in China should consider the effect of senior citizens’ self-learning and support from a circle of their friends and family. 5. Discussion and Proposal The findings of the survey revealed that: (1) Senior citizens experience ability anxiety and reliability anxiety, especially those with poor health, those who do not visit social facilities, and Internet novices. However, most of them still want to learn more Internet knowledge to face the challenges it presents; (2) Senior citizens have special needs in terms of information, learning, and support. Also, they tend to have encountered significant obstacles, in particular the difficulty of finding useful sites, trouble with using browser tools, unfriendly Web interface design, and the limitations of existing support methods. The most interesting finding is that senior citizens have a positive attitude toward learning about and using the Internet even though they have experienced Internet anxiety and obstacles. Therefore, more people of their age should be encouraged to use the Internet, but to do so it is necessary to provide a support and learning environment that can ease their anxiety and satisfy their needs. Based on the survey we propose a model called the “Senior Internet Support & Learning Environment,” schematically illustrated in Figure 1. The environment comprises four parts, which are explained as follows:

Figure1. Senior Internet Support & Learning Environment

(1)Web navigator site for seniors For senior citizens, the Internet information needs are different from those of other generations. From the survey results on information needs, we can see seniors have some special features in terms of information-find methods, and this can explain to a certain extent that the current directory Websites are not popular nor entirely suitable for seniors. We suggest building a Web navigator site especially for seniors. The directory structures are based the survey results of Internet information needs. Meanwhile, the navigator should also have a keyword search function because this is the search method used most by of senior citizens. Furthermore, because most seniors would like to receive others’ advice on Internet

W. Zhou et al. / Internet for Senior Citizens in China: Survey and Proposal

377

information searching, the Web navigator should also provide a free space where users are able to upload their favorite site links and obtain or give advice about these links. (2) IT learning site for seniors We aim to provide senior citizens with an easy way to understand and access learning materials such as texts and learning CDs, etc. Also, to improve access, an online learning site is necessary. To minimize the level of ability and reliability anxiety, the learning content should focus on Internet foundation knowledge, popular Internet terms, search ability, security, and anti-virus knowledge. It is worth considering the understandability of content in order to let seniors, whose memory and comprehension often fades with time, easily grasp the knowledge. (3)IT community site for seniors and supporters The survey result showed that public service facilities have positive effect on Internet usage. In recent years, such public learning centers for seniors are developing in China, so real learning and support communities should be considered. Moreover, in addition to offline social communities, a Web-based senior community site should be built that can ease the seniors’ worries when seeking support. Through providing a virtual and easily accessible space, in the IT community site seniors can ask any question they think of while using the Internet and supporters can discuss, answer, and provide suitable solutions to share their knowledge and wisdom. (4)Browser tool for seniors The browser tool is other factor that affects using the Internet. Current browser designs are particularly detrimental to the ease with which older adults “consume” information from the Web. A browser tool for seniors should consider the attribute of “eyesight” and “precision of movement,” to make it easier for seniors to browse the Internet. The four components are not isolated; they are closely related to one another and form a totally integrated environment. First, the design of each must follow the same design guidelines: usability and accessibility for seniors to make it easier for them to use and to reduce their anxiety. Design guidelines include visibility improvement, operation improvement, and consideration of cognitive factors. Next, the information must be linked among the three Websites. That is to say the information generated by one site should be reused for other site. For example, if a word or phase appears in a community site, it should be given a link to the IT learning site to provide a suitable and complete explanation of it to ensure that seniors understand thus actively participate in community sites. Finally, the content of the three Websites should follow two select criteria: (1) content should ease seniors’ Internet anxiety; and (2) content should satisfy what seniors want to learn, and provide support for it. Based on these, the environment can be viewed as a useful and necessary environment for senior citizens in China. 6. Evaluation To demonstrate the proposal’s usability, we built a prototype of the proposed system and did an experiment on June 14, 2006 in the computer room of Hebei Senior University in China. All of the senior participants were Internet beginners with only basic computer knowledge. 24 participants used the proposed system and 19 answered questionnaires (Table 2). The higher score means the higher satisfaction. The results showed that the proposed system effectively eased Internet anxieties and satisfied their needs and even improved learning and problem-solving abilities. Compared with IE, most participants agreed that the new browser design was user-friendly and using it they could easily browse the Internet.

378

W. Zhou et al. / Internet for Senior Citizens in China: Survey and Proposal

Browser tool

Web navigator site

IT Learning Site

IT community site

Table2. Five-stage evaluation for proposed system Are the tools easy-to-use compared with other tools? Is it easy to browse homepages? Is the interface design suitable for seniors? Is it easy to use? Is the directory suitable for seniors? Is the interface design suitable for seniors? Is learning by Internet an effective method? Is the learning content easy to understand? Is the interface design suitable for seniors? Did it help solve your problems? Are the contents and solutions easy to understand? Is the interface design suitable for seniors?

4.21 4.47 4.47 4.42 4.36 4.42 4.57 4.10 4.36 4.26 4.31 4.47

7. Conclusions In this paper we attempted to determine the Internet anxiety and Internet needs of senior citizens in China through a survey. We investigated senior citizens’ experience with using Internet, identified three types of Internet anxiety: ability, reliability, and acceptability, and identified attributes that can most often result in anxiety. The study further explored the willingness of senior citizens to learn and request support, and what were the most frequently occurring difficulties. Based on the survey, we proposed the Senior Internet Support & Learning Environment, a system that incorporates a Web navigator site, an IT learning site, an IT community site, and a Web browser tool. Finally, we demonstrate it by experiment. Certainly, further studies could be done with a broader sampling frame and larger sampling size to achieve greater reliability and generalization. Future studies could also enrich the functions and structures of the environment, achieve it, and finally implement it to help more senior citizens learn about and use Internet. Acknowledgments We would like to thank all people connected with the study, especially the Hebei Senior University. This research was supported in part by the 21st Century COE program “Intellectual integration (IMI) of voice images for the social information base” of MEXT and the MEXT Research Subsidy, and by Grant-in-Aid for Scientific Research Foundation. References [1] CNNIC. (2006) The 17th Statistical Survey Report on Internet Development in China. [2] Zhou, W., Yasuda, T., and Yokoi, S. (2006) E-NamoSupport: A Web-based Helpdesk Support Environment for Senior Citizens. WEBIST2006, Setubal, Portugal, pp. 29-36. [3] Chong, S. P. and Theng, Y. L. (2004) A Study of Web-Based Information Needs of Senior Citizens in Singapore. Lecture Notes in Computer Science, Vol. 3196, pp. 16-33. [4] Dickinson, A. Gregor, P., McIver, L., Hill, R., and Milne, S. (2005) The Non-Browser: helping older novice computer users to access the Web. Accessible Design in the Digital World Conference 2005, Dundee, Scotland, pp. 23 - 25. [5] Zhu, J. J. H. and Wang, E. (2005) Operational definition and preliminary test of the Digital Divide Index. Communications of the ACM, Vol. 48, Issue 4, pp. 49-53. [6] ISO/DIS 9241-11 (1995) Draft International Standard. Ergonomic requirements for office work with visual display terminals (VDTs). Part 11: Guidance on usability. [7] W3C (World Wide Web Consortium). (1999) Web Content Accessibility Guidelines 1.0. [8] Presno, C. (1998) Taking the Byte out of Internet Anxiety. J. Educational Computing Research, 18(2) pp. 147-161. [9] Umeda, K., Kurashiro, M., Ejima, T., and Nozaki, H. (2005) The Development of the IASv1 Internet Anxiety Scale, International Conference on Computers in Education, Singapore. [10] Chou, C. (2003) Incidences and Correlates of Internet Anxiety among High School Teachers in Taiwan. Computers in Human Behavior, Vol. 19, pp. 731-749. [11] Coyne P. K. and Nielsen, J. (2003) Web Usability for Senior Citizens: Design Guidelines Based on Usability Studies with People Age 65 and Older. Nielsen Normal Group Report.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

379

Why Do Students Engage in e-Learning: A Chinese Perspective Zhenhong Zhang, Ronghuai Huang Knowledge Science and Engineering Institute, School of Educational Technology, Beijing Normal University, Beijing 100875, P. R. China [email protected] Abstract: This paper presents a study on how Chinese students understand e-learning. Twenty university students who took part in a course “Introduction to e-learning” were interviewed and data were analyzed using the method of phenomenography. The findings can be presented as four qualitatively different conceptions of e-learning: e-learning as (1) supplement to face-to-face learning or the metaphor of after-meal dissert; (2) informal resource-based learning or the metaphor of a mobile library; (3) activity-based learning or the metaphor of a ladder to knowledge; (4) community-based learning or the metaphor of a food sharing banquet. The aim of the study is to shed light on Chinese students’ conceptions of e-learning, which are of significance to e-tutors in offering e-learning services. Keywords: e-learning, conception, phenomenography

Introduction E-learning is an umbrella term that is used to describe learning involving the application of new multimedia technologies and the Internet. In China e-learning has enjoyed fast development and become indispensable in the educational system. China began practicing e-learning in 1998 and there are 68 institutes of higher education offering e-learning services at present. Statistics indicate that, by the end of 2004, over 10% of college students have their higher education through e-learning [1]. Literature review shows that much research is about what the teacher could do or has done online, while the student experience of those activities goes largely undocumented [2]. Yet research into students’ conceptions of e-learning is the starting point for a system approach to e-learning development. This study aims to explore the variation of ways e-learners understand e-learning and make recommendations to e-tutors who plan to offer e-learning services. 1. Method 1.1 Phenomenography A phenomenographic approach was used for this study. Phenomenography is a field of descriptive research concerned with the different ways of experiencing and understanding people have of various phenomena, developed in Sweden in the 1970s in the field of education [3, 4]. It has been repeatedly found that phenomena, aspects of reality, are experienced and understood in a relatively limited number of qualitatively different ways [3]. In phenomenography, in-depth interview transcripts were handled as a whole to find out the whole variation in the conceptions articulated. The outcome of a phenomenographic analysis is a system of categories of description which covers the total variation in the conceptions expressed in the research population [5].

380

Z. Zhang and R. Huang / Why Do Students Engage in e-Learning: A Chinese Perspective

1.2 Procedures and data analysis The study was carried out during March-April 2006 in Beijing Normal University, P. R. China. Twenty students enrolled in a one-semester course “Introduction to e-learning”, including 18 graduates and 2 undergraduates. The course had seven modules and each module lasted about 2 weeks. The students were asked to read certain materials, communicate using an asynchronous web-based tool, and complete a set of assignments. The lead author served as the teaching assistant and tutor throughout the course. The 20 students were interviewed in the middle of the semester by the author using phenomenographic interview technique that allowed openness and variation in responses. The interviews were recorded, transcribed and analyzed. As to ethical considerations, participants were provided verbal explanation of the purpose of the study prior to the interview. Assurance of maintenance of confidentiality and anonymity was given. Phenomenographic techniques [6] were employed to elicit a description of categories of the 20 students’ conceptions of e-learning. Each transcript was read and re-read many times for the researcher to become familiar with the data [3, 4]. Key elements that characterize the students’ experiences of e-learning were recognized, including learning resources, learning activities, e-tutor’s support, collaboration learning, emotional presence, and difficulties in learning. Variation of conceptions of e-learning was derived from the analysis of the relations between these key elements. Rigor in the analysis was established by following the tenets of phenomenographic approach [6]. In phenomenographic research, replication of categories of description as findings is not considered necessary, but that another researcher can recognize the categories of description once they have been identified. 2. Findings The findings can be presented as four qualitatively different aspects on the students’ conceptions of e-learning. The analysis of each understanding is described using the key elements as a conceptual framework collectively forming four categories of description. The categories are illustrated with characteristic quotations translated from Chinese and provided with metaphors (figure 1, 2, 3, and 4) for a communicative purpose.

Figure 1

Figure 2

Figure 3

Figure 4

2.1 Supplement to face-to-face learning or the metaphor of after-meal dissert (Figure 1) E-learning is perceived as defective in nature and cannot match face-to-face teaching and learning in that there is a lack of communication and guidance. Thus e-learning is merely an alternative for those who cannot obtain face-to-face learning, but for those who can and are willing to commute, e-learning is a supplement. One student stated in the interview, “I feel isolated and a lack of sense of belonging in pure e-learning style. And it is so hard for me to always find time for the totally self-controlled e-learning because there is not someone to supervise me as a teacher in the classroom.” Students can have additional learning resources and complete other course components in e-learning, yet teacher’s lectures in the classroom are the best and indispensable form of learning. There is a degree of resistance to e-learning among some

Z. Zhang and R. Huang / Why Do Students Engage in e-Learning: A Chinese Perspective

381

students, in particular inexperienced ones. For example, one student said, “I feel comfortable and close to the teacher in face-to-face learning style. I am often affected by the teacher’s passion, which is lacking in e-learning. After a lecture I feel certain that I’ve learned something, but that’s not the case with e-learning” Another student said, “I like the feeling of sitting in a classroom and learning under the guidance of the teacher together with peers. In e-learning what I see and work with is a “cold” computer. I expect to know the people who prepare the learning materials or to find someone to talk with and work with.” In the metaphor of after-meal dissert, face-to-face learning is compared to the indispensable meal and e-learning to dissert—a supplement after the meal. 2.2 Informal resource-based learning or the metaphor of a mobile library (Figure 2) E-learning stems from the demand for the ability to adjust quickly to and assimilate an ever increasing amount of information. The learning environment for today’s learners is no longer set within the walls of a school, but rather is everywhere, especially the Web and email [7]. The greatest charm of e-learning is that it is self-paced, self-regulated and self-controlled. It provides access to various forms of resources and materials which may otherwise be not available. Cognitive or emotional support from teachers and peers may not be necessary as there is no pre-defined learning goal. For example, one student said, “I could find what I need on the Internet and learn on my own, so there is really no need to go into a classroom to listen to the teacher’s lectures. I like e-learning because I have the right to decide what to learn, when to learn, where to learn, and how to learn.” Another student stated, “E-learning suits me well as I don’t need to commute 3 hours a day from home to the university. I like the feeling of managing my own time and I can learn on my own the kind of stuff that is dumped on me in the classroom.” In this sense, e-learning is like reading books in a mobile library: the library can be accessed at any time and place, and people can pick up whatever they feel like reading. 2.3 Activity-based learning or the metaphor of a ladder to knowledge (Figure 3) E-learning is not only the presentation of resources, but also participation in activities that support knowledge construction. Most e-learning activities involve a series of tasks which lead the students to achieving learning goals. In the traditional class teachers serve as scaffold and guide whereas in e-learning activities take the teachers’ place. One student said in the interview, “Reading online materials on my own is so boring that I may easily be lost in the mass of information. E-learning activities are just like teachers who tell me what to do next and can lead me to an in-depth understanding of the materials. They also remind me of the learning goals and assist me to achieve them.” Effective and interesting learning activities guided by e-tutors could support interactivity, which is proved to be an essential part of e-learning. Content is supposed to be delivered online with high degree of interaction with tutor, dialogue with other students through individual or collaborative activities. A student said, “I experienced and enjoyed the learning process in which I took part in activities on my own or with group members. A course with only materials but without activities cannot facilitate learning.” In the metaphor of a ladder to knowledge activities are compared to a ladder on which students rely in the learning process to build knowledge and negotiate meaning. 2.4 Community-based learning or the metaphor of a food sharing banquet (Figure 4) E-learning resources, activities, e-tutor’s support, collaboration, emotional presence are all included in the concept of community. Community is a general sense of connection,

382

Z. Zhang and R. Huang / Why Do Students Engage in e-Learning: A Chinese Perspective

belonging, and comfort that develops over time among members of a group who share purpose or commitment to a common goal [8]. Consisting of such elements as collaboration, common goals, friendship, familiarity, it is a team experience in which people are influencing others and at the same time influenced by others. A student said, “I think exchange of ideas with peers and e-tutors is more important than learning knowledge. I value the opportunity to share information with them and bright ideas just come out from our communications. I believe most people in the community have the same opinion.” Community enhances a sense of belonging and reduces the learners’ feeling of isolation. Learning together in a community, students have the opportunity to extend and deepen their learning experience, test out new ideas by sharing them with a supportive group, and receive critical and constructive feedback [9]. A student said, “I like meeting people in e-learning who have similar experiences, feelings, and interests. Working with them makes me feel less isolated and I like sharing and discussing asynchronously with peers. In this kind of e-learning, each learner’s ideas are valuable, appreciated and contribute to the whole class, which greatly increases my motivation in learning.” In the metaphor of a banquet, e-learning is compared to the banquet while every member’s background, previous knowledge and experiences, etc. to the food people take. 3. Conclusions Understanding of e-learning is so diverse that various needs should be catered for. Our study shows that there are four categories of conceptions of e-learning among Chinese students: supplement to face-to-face learning; informal resource-based learning; activity-based learning; community-based learning. Findings imply that resources are key to e-learners who consider e-learning informal or supplemental. To others, tutor’s design and guidance and peers’ support are what make e-learning rewarding. This study has only been conducted in China and within one course and findings may not be representative of students in other countries or other courses. Yet we still hope the findings could contribute to an intercultural understanding of e-learning and cross-cultural communication of e-learning pedagogies. Acknowledgments We thank the Ministry of Education in China and Higher Education Funding Council for England for supporting the eChina~UK eLearning Programme, of which the course is a part. References [1.]Zhang, Y. (2004) Unavoidable Challenge: Investigation of and Thoughts on Distance Higher Education in China, in Education Newspaper of China. Beijing. [2.]Alexander, S. (2001) E-learning developments and experiences. Education & Training, 43(4): 240-248. [3.]Marton, F. (1981) Studying conceptions of reality--a metatheretical note. Scandinavian Journal of Educational Research, 25(3): 159-169. [4.]Marton, F. (1986) Phenomenography--a research approach to investigating different understandings of reality. Journal of Thought, 21(3): 28-49. [5.]Marton, F. (1994) Phenomenography. 2nd ed. The International Encyclopedia of Education, ed. T. Husen and T.N. Postlethwaite. Vol. 8, Oxford: Pergamon. 4424-4429. [6.]Bowden, J. (2000) The nature of phenomenographic research. Phenomenography, ed. J. Bowden and E. WalshMelbourne, Victoria: RMIT University Press. [7.]Jun, J. (2005) Understanding E-dropout. International Journal on E-Learning, 4(2): 229-240. [8.]Conrad, D. (2005) Building and Maintaining Community in Cohort-Based Online Learning. Journal of Distance Education, 20(1): 1-20. [9.]Palloff, R.M. and K. Pratt. (2005) Online Learning Communities Revisited. in 21st Annual Conference on Distance Teaching and Learning.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

383

Enabling a Multilateral Distance Class between China, Korea and Japan: Effective Utilization of Networking Technologies Yuri NISHIHORIa, Keizo NAGAOKAb, Nozomu NISHINAGAc, Kenji TANAKAc, Yuichi YAMAMOTOa, Haruhiko SATOa, Masahiro HARADAd, Ruimin SHENe, Jinjin FENG e, Myunghee Ju KANGf a Hokkaido University, Japan b Waseda University, Japan d Tokyo Electron Limited, Japan c National Institute of Information & Communications Technology, Japan e Shanghai Jiao Tong University, China f Ewha Womens University, Korea [email protected] Abstract: Our project investigates how a multilateral distance class can be facilitated between China, Korea and Japan through the use of interactive communication tools, including a high-quality video conferencing system, a text-based discussion system, and a network-based display-sharing system. The main objectives of our project are to implement high-quality, high-fidelity environments over long-distance broadband networks, and to provide students of EFL (English as a Foreign Language) in Asia with better chances for international understanding using a common language. With regard to the platforms for collaborative learning, Chat’n’Debate and Multi-Culture Box were created and used over the Japan Gigabit Network as well as the Internet. Results are reported based on the Principal Component Analysis of a questionnaire given to the participants in these three countries. Keywords: CSCL, video conferencing system, Ruff-HDV, Chat’n’Debate, Culture Box

Introduction The role of networking technologies has become increasingly more important with the employment of broadband networks which can facilitate international understanding anywhere around the globe. The development of IP networks or the Internet made it possible for us to send all phases of voice, images, and text data, simultaneously and inexpensively, facilitating language classes in a global setting. There has been much data accumulated between educational institutions in two countries, and a number of studies have been conducted in this area [1]. For multilateral institutions, however, it is still a challenge to develop educational software on the Internet. Our project promoted synchronous and symmetric communication within high-quality video conferencing in multilateral language classrooms where non-verbal communication plays an important role for mutual understanding. 1. Multilateral Class Overview A multilateral learning experiment was conducted in October, 2005. Classrooms in Hokkaido University, Japan, Jiao Tong University, Shanghai, and Ewha Womens University, Seoul, were connected through the Internet. Students learning English in these three countries enjoyed this collaborative session in which interactive communication was achieved by means of a text-based chat called “Chat’n’Debate”[2], a full-specification

384

Y. Nishihori et al. / Enabling a Multilateral Distance Class Between China, Korea and Japan

High-definition Digital Video (HDV) conferencing system called “Ruff-HDV”, and a web-based voting system called Multi-Culture Box which was developed from Culture Box [3]. Hokkaido

Shanghai

Seoul Shanghai

Figure 1 Multilateral Class on the Screen

Japan

Korea

Figure 2 Multi-Culture Box in a Multilateral Project

Along with the main face-to-face activity, three other activities were organized for this class: (1) chat, (2) an on-the-spot questionnaire with voting and comments, and (3) an object lesson. The text data in the chat discussions displayed on the large screen in front of the class played a meaningful role, serving as a medium to channel the collected opinions and feelings of the three classes. The participants were not just exchanging their opinions, but were connected interactively and this was instrumental in fostering a sense of belonging among them. With Chat’n’Debate, the opinions of all participants could be understood at a glance on the screen.

2. Network Configuration and a Problem Stated The three universities were connected using (1) an Internet connection between Korea and China, and between Japan and China, and (2) the high-speed network for academic research between JGN2 in Japan and KOREN in Korea (Fig.3).

Figure 3 Configuration Details : WAN (Wide Area Network)

Y. Nishihori et al. / Enabling a Multilateral Distance Class Between China, Korea and Japan

385

With regard to securing a high quality of transmission, however, a serious problem arose in the above-mentioned wide area network. In principle, an MCU (Multi-point Control Unit) should have been placed in Japan, but this has caused a bottleneck between China and Japan (Fig. 4). In order to circumvent this, an MCU was placed within Korea and used to send and receive images easily between China and Japan (Fig.5). Since the network between China and Korea was stable, images from China were easily and stably transmitted to Japan, and vice versa, from Japan to China by way of Korea. By taking this evasive action, the bottleneck problem was solved, thereby enabling stable and multilateral communication among these three points.

Figure 4 Bottleneck Problem

Figure 5 Circumventing an MCU Problem

3. Data Collection and Evaluation We measured students’ perceived usefulness of this type of multilateral distance class from five perspectives; whether this class was (1) enjoyable, (2) informative, (3) better than ordinary language classes, (4) preferable in terms of its activities and (5) successful in meeting students’ expectations. After this experimental class, an anonymous questionnaire in English was distributed to the three sets of students. They awarded a numerical score from five (strongly agree) to one (strongly disagree) on the five-point Likert scale to the following questions, each of which represents the above five perspectives respectively.The results indicate that there was a high degree of unanimity among participants: they were very positive in actively involving themselves in the real-time exchange of opinions.

Figure 6 Questionnaire Results

386

Y. Nishihori et al. / Enabling a Multilateral Distance Class Between China, Korea and Japan

For the detailed analysis to be able to capture the tendencies in the students’ reactions, specificly for this multilateral class, PCA (principal component analysis) [4] was employed in our study in order to obtain this multi-dimensional data. In order to grasp the correlations within a multilateral context, a number of correlated variables are transformed into a small number of uncorrelated variables (Table 1 & 2). Fig. 7 shows the data obtained by applying the PCA analysis using a plot of the new data points. -1

0

1

q2

q4 q3

0

0.0

q1

-1 -2

-0.6

-0.4

q5

Table 2 Principal Component Score PC1 PC2 PC3 JPN -2.5685065 0.5846437 -1.100805e-16 CHN 0.3218889 -1.4626122 1.549405e-14 KOR 2.2466176 0.8779685 -4.971686e-15

1

KOR Korea JPN Japan

-0.2

PC2

0.2

0.4

-2

Table 1 Principal Component Loading PC1 PC2 PC3 q1 0.9963985 0.08479405 7.517198e-16 q2 -0.8375678 0.54633344 4.832634e-16 q3 0.9086710 0.41751282 -2.690394e-15 q4 0.8398175 0.54286884 1.992162e-15 q5 0.8311057 -0.55611452 5.142347e-16

China CHN -0.6

-0.4

-0.2

0.0

0.2

0.4

PC1

Figure 7 Dimensional Plot of PC1 and PC2

According to the above plot and labels of each PC, we can conclude that Japanese students found the multilateral class informative, but not so enjoyable; Chinese students found it not so enjoyable, but meeting their expectations; Korean students found it enjoyable on the whole. In this way, we can grasp an overall picture of reactions in each country, which occur simultaneously in a multilateral class. 4. Conclusion and Future Considerations We described the experimental class and discussed how the use of a high-quality video-conferencing system, a chat system, and a web-based voting system promoted the establishment of effective multilateral environments. Our study has also shown the many promising possibilities of future work in this area covering many places on the globe. We plan to further develop a system that supports multilateral language learning in global network environments. Acknowledgements This research is supported by the Grants-in-Aid for Scientific Research from JSPS (Japan Society for the Promotion of Science): Scientific Research B-17300275 References [1] Warschauer, M. & Kern, R. (2000) Networked Language Teaching: Concepts and Practice, Cambridge Univ. Press. [2] Nishihori, Y., Okabe, S. & Kurosaki, D. (1999), “EFL Writing in Computer Supported Collaborative Language Learning: Chat'n'Debate”, Advanced Research in Computers & Communications in Education, 342-349, IOS Press [3] Nishinaga, N., Nishihori, Y., Nagaoka, K., Harris, D., Okabe, S., Yamamoto, Y. & Tanaka, K. (2004), “Enabling a Cross-Cultural Collaborative Community - Networking Technologies to Form Meaningful Environments for Higher Education”, Proceedings of ITHET2004, IEEE Catalog Number: 04EX898C (ISBN: 0-7803-8597-7). [4] Jolliffe, I.T. (2002) Principal Component Analysis, Springer-Verlag

Content and Knowledge Management

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

389

Digital Video Database: Supporting Student Teachers’ Learning about Teaching during Teaching Practice Winnie SO, Vincent Hung, Walker Yip Senior Lecturer, The Hong Kong Institute of Education, Hong Kong Assistant Education Technical Officer, The Hong Kong Institute of Education, Hong Kong Technician, The Hong Kong Institute of Education, Hong Kong [email protected] Abstract: This article introduces the setting-up of an online digital video database by the teacher education institute. An investigation into the application of the online digital videos database to support student teachers learning to teach during their teaching practice period is discussed, which includes the details of trimming the videos of teaching practice into learning objects for peer evaluation in order to construct good teaching & learning practices for sharing. Analysis of peer evaluation and feedbacks in the discussion forum among three student teachers involved in this study was conducted to gain a better understanding of the learning objects chosen as good practices of teaching and the nature of suggestions given. The focus group discussion afterwards provided evidences of student teachers’ learning in the process of sharing learning objects on the online digital video database. Keywords: online digital video database, learning objects, reusable resources, learning about teaching

Introduction As technology development has made tremendous progress, educators endeavoured to have better integration of technology to teaching and learning and to improve quality of learning through the effective use of technology. In teacher education, there are suggestions of integrating technology into teaching and learning by establishing online learning resources database with an aim to change the traditional way of making teaching materials individually to a new trend of developing online shared learning resources. The most challenging part is not on the setting up the online learning resources database, but how resources can be trimmed into meaningful learning objects for easy searching in the online learning resources database, and then the trimmed learning objects would be re-uploaded to the server for sharing to create reusable resources.

Video database Videos have been becoming more popular as resources in teaching and learning in the past few years, especially in teacher education aiming to demonstrate good practices of teaching or for analysis of teaching behaviours in authentic situations. Production of videos is time consuming, involving a lot of human resources and materials cost. It is not

390

W. So et al. / Digital Video Database: Supporting Student Teachers’ Learning About Teaching

worthwhile that the videos are only used by a specific teacher in specific classes. Though there are significant changes in the format of videos in the past ten years from analog VHS to digital media, limitations in sharing of videos still exist. In other words, the advances in video technology with the emergence of digital form of videos, do not necessarily help with the management, classification, retrieval and the reuse of videos. With the rapid development of information technology, shareable and reusable of the resources and teaching materials become one of the hot topics in global higher education sectors. Though the principles of sharable and reusable digital materials have been developed into the concept of “learning objects”, it is still far from concrete conceptualization about how to share and how to reuse. The digital video database is using Text-based Retrieval technique (Marques and Furht, 2002). This technique makes use of free text to describe (annotate) the content and situation of the videos. Users can input the keyword and employs Text-based Retrieval technique to carry out the retrieval. The development of the on-line digital video database is more or less applying the concept of learning objects and the purpose is the sharing and reusing of the digital videos. Learning objects should be small and unified in digital format so that they can be shared among different digital platform by different users. For the video database, MPEG video and streaming video in Windows Media Video (wmv) are two basic unified digital video formats that can be shared by different users with Internet access. Streaming video provides a standard video format for viewing with Windows Media Player which is bundled with every Windows XP operating system. For the reuse of video for a specific teaching purpose, MPEG video provides another unified format for users to download the video to their own computer for further editing and reuse.

Learning object in the video database Wiley (2000) defined learning object as “any digital resource that can be reused to support learning.” Learning object produced by publishers, teachers, support staff and even students, are stored in digital database, where they can be easily accessed, recombined and reused within online courses (Duncan, 2003). Haughey & Muirhead (2005) conceived learning objects as an effective and efficient means for providing virtual content that could be shared with others, that is, learning objects make learning resources to be shareable and reusable. Standardisation of learning objects is required to utilize widely (Littlejohn, 2003). Digital video database can store a various type of video formats such as AVI, MPEG and streaming video file WMV, MOV, etc. It ensures interoperability of resources among different electronic environment and platform as Windows and Macintosh. With an agreed classification system, it is easier to find what you are looking for in a database (Littlejohn, 2003). In view of a learning object economy, resources are often conceptualised as blocks of content that interlinked so as to produce a course. Analogous with Lego bricks, these blocks can be recombined with other blocks and reused in a different course (Littlejohn, 2003).

Design of study An on-line digital video database is designed and developed for storage of videos in the teacher education institute. Evolution of the digital videos database to an interactive resource database is also introduced. Besides, an investigation of how this interactive resource database supports student teachers in constructing knowledge of good practices

W. So et al. / Digital Video Database: Supporting Student Teachers’ Learning About Teaching

391

of teaching is conducted. Development of the digital video database The major features of the database include “Video Vault”, “Video Analysis”, and Transaction Bin”. The “Video Vault” allows uploading of teaching video clips in MPEG, ASX, WMV or MOV format by user without copyright dispute, and it is open to public for viewing and/or downloading. The streaming rate of WMV video is set in between 300kbps to 700kbps. The “Video Analysis” assists users to make a series of annotation into a video. The user applies this function to create many video bookmarks in one video, and to input some typical attributes, like short description and title of the segment for classification easily. The Annotation, Video Bookmark and Classification improve the efficiency of Video Analysis in the video database. It is easy to find the proper video clips for analysis and research through the keyword search in different categories, or content description. “Transaction Bin” is designed for assisting the post-production of video teaching resources. Semi-finished or finished post-production video clips can be put into the registered “Transaction Bin” for viewing and amendment. All videos in the transaction bin can be accessed from one hyperlink. With this function, the virtual videos are distributed to a large number of users without any duplication of the video clips and it releases lot of system resources for video storage. Furthermore, user can view or download the video clips with one click only. To utilize the built-in video communication panel, the users can post the comment of a segment for the system administrator to follow up or update. A proper use of this function improves the transaction rate, saves time and reduces CD ROM consumption. Evolving into an interactive resource database A Bulletin Board is set up in the digital video database. Learning community members can create “Mark-in” / “Mark-out” to produce video segments which have discussion value with the function of “Video Marker”. Comments or suggestions for this segment are posted for discussion. In order to achieve knowledge building, other community members can also view and reply to the message of a segment through the “Video Bookmark” (Haga, H. 2004). Figure 1 is an interface of “Video Bookmark” with two areas. The Video Panel and Bookmark Area is for creating “Mark–in” and “Mark-out” to indicate the “Start Time” and “End Time” of the segment, and the video is controlled by using the basic pause, stop, and fast forward buttons. Besides, the Comment Edit Area is used for display of the time code, theme and comment of the segment. Figure 2 is the interface of Bulletin Board with three areas. The Bulletin Board header displays the video title and provider. The file converter is set up to increase portability, compatibility and shareability of the Bulletin Board. Content of Bulletin Board includes the hyperlink of the segment that can be converted in PDF format for user to save it for later use. The Theme Area is for the display of the title, reply button and hyperlink of the segment from the suggestion provider. User can click this hyperlink to view the related segment. The Discussion Forum is for the display of content and response to the shared video uploaded by other community members.

392

W. So et al. / Digital Video Database: Supporting Student Teachers’ Learning About Teaching

Comment Edit Area

Video Panel and Bookmark Area

Figure 1 Interface of video bookmark Bulletin Board header Theme Area

Discussion Forum

Figure 2 Interface of bulletin board

Using the digital video server for sharing of practices in teaching In order to test if the interactive functions of the video database could help student teachers construct knowledge of teaching with sharing and suggestions from a learning community, the digital video database was used by a small learning community of three student teachers during their teaching practices period in December, 2005. The community members were requested to select a few lessons for video shooting, and then upload the videos to the database to form a learning community using the “Transaction Bin” of the database. Each member then used the “Mark-in” / “Mark-out” function to divide their own 30-minute lesson video into several short clips similar to the concept of learning objects, marked with a short description, for sharing with the community members. Afterwards, the community members viewed the video clips through the hyperlink in the bulletin board and shared their comments and opinions. Members were required to indicate an overall remark like “Good” or “Suggestions” when they commented on each learning object of video clip. Analysis of the nature and categories of the comments and feedbacks were carried out to gain more understanding of the learning of good practices of teaching by student teachers. A focus group discussion was conducted with the three teacher participants of the learning community to discuss their construction of knowledge on good practices of teaching, and the constraints in using the digital video server, and the barriers of video shooting of lessons were identified.

Result Nature and categories of comments and responses There were altogether five lesson videos uploaded with the learning community by the three members, resulted in 39 comments on their practices of teaching (Table 1). The

W. So et al. / Digital Video Database: Supporting Student Teachers’ Learning About Teaching

393

pattern of interactions included 27 comments by community members to the video providers; other comments had received feedbacks either from the video providers or other community members. The pattern of interactions from the learning community was considered interactive in nature. Table 1: Interactive nature of comments and suggestions within the learning community Further feedbacks Video Comments from Feedbacks from video from other member provider member provider / other member Video 1 8 3 0 Video 2 4 1 0 Video 3 4 2 0 Video 4 4 2 1 Video 5 7 3 0 Total 27 11 1 The comments were on both the pupils and teachers sides. Comments on the pupils’ side were mainly about pupils’ participation while comments on the teachers’ side could be divided into five categories: classroom management, teaching design, teaching technique, teacher effectiveness, and use of resources. Indications of “Good”, “Suggestions” and “Good with suggestions” were marked with each of the comments. From the analysis (Table 2), there are evidences that the digital video database could help student teachers construct understanding of good practices of teaching. Table2: Analysis of comments with the videos Comments

Good Good with suggestions Suggestions Total

Teaching techniques

Use of resources

Teacher effectiveness

Teaching design

12.77% 10.64%

12.77%

12.77% 4.26%

12.77% 2.13%

23.41%

10.64% 23.41%

17.03%

14.9%

Classroom management

Pupils’ participation

Total

38.31% 29.8% 12.77% 12.77%

6.38% 6.38%

29.79%

Student Teachers’ learning about good practices of teaching All participating student teachers realised that the use of the digital video server in sharing practices of teaching among community members helped them a lot in developing teaching techniques. The suggestions and feedbacks made by the community members benefited both the student teachers who posted the videos and the viewers. On one end, by viewing video-taped lessons shared by the community members, student teachers got authentic examples of unfavourable teaching behaviour that caution them from recommitting the same error in their future teaching in classrooms. On the other hand, student teachers who posted the video could also review their own teaching in receiving comments and suggestions from other viewers, video providers could give further feedbacks to viewers by providing background information and rationale of their teaching for further discussion. All the interactions and discussions provided practical learning experiences for student teachers who were novices in the teaching profession. All student teachers appreciated these experiences for their development of teaching competence in the aspects of “improving teaching technique”; “enriching teaching resources”; “sharing of joys and difficulties in teaching”. Student teachers could learn from each other the effective classroom management strategies and to be aware of the ineffective classroom

394

W. So et al. / Digital Video Database: Supporting Student Teachers’ Learning About Teaching

management behaviour. The sharing of teaching resources for their reference would be very useful for future planning of teaching. Furthermore, student teachers reported that the sharing of joys and difficulties in teaching was supportive, providing information and explanation and feedback to questions could help to relieve frustrations from unsatisfactory performance in classrooms during teaching practice period. The constraints of viewing the video segments In using the video database for sharing, it is better to divide the 30-minute video into shorter segments of learning objects. However, the student teachers reflected that it was difficult to comment the video clips by viewing only a short video segment. It was because they could not have enough information to give comments, for example, they might not know the lesson plan of the teacher, the background, emotion or behaviour of the pupils in commenting on a short video clip of lesson segment. In general, the smaller or more granular a resource, the greater the possibility of it is being reused in another educational context (Littlejohn, 2003). However, larger resources usually have greater educational value: it may be less time-consuming to reuse a larger resource, such as video of a whole lesson, rather than to construct many small and basic components from a large resource. Therefore, in terms of resource size, there is often a tension between increasing educational value and maximising reusability (Littlejohn, 2003). The barriers to video shooting of lessons Before concluding the on-line digital video database is successful, the possibility of lesson video shooting in authentic classrooms needed to be considered. Though the original aim was to investigate whether the digital video database could support resources sharing and knowledge building of student teachers through interaction, unexpected barriers to lesson video shooting identified during the forum discussion. There were barriers rested with the school level, teacher level and pupil level. It was not easy for schools to support video shooting of lessons because school principals worried about offending the personal privacy of the pupils if the videos were used improperly by the public. Teachers also concerned about the additional workload. Pupils were not used to be video shot and abnormal behaviour under the camera were observed and captured.

Conclusion The study with the three student teachers found that the design concept of “Transaction Bin” was not limited to apply in one-way channel from administrator to user (Top-down). They were expanded to peer-to-peer interactive platform. Furthermore, the on line video server had been transformed to a learning community where students can share and make suggestions between peers. Since the function of “Transaction Bin” was plugged in the “Community Management”, the user could invite other users to become members of the community for sharing the video to each others in the “Video Vault”. It seems prefect to have a huge central unit storing all online teaching resources and a mature searching system so that the users can find the resources easily. Owing to the non-standardised teaching resources format, categorization and searching system, users from other groups are unable to share or get access to the online resources. And users have to undergo some processes before uploading the resources to this central unit for effective and efficient management. However, in the present situation, it is common that all users

W. So et al. / Digital Video Database: Supporting Student Teachers’ Learning About Teaching

395

make and save their resources in their own formats and would not be easily shared with the others. The digital video server described in this article required users to upload and save the resources by themselves to a system with standard formats which enable other users to download the resources. To conclude, though the establishment of the digital video database was originally to provide an online database for users to upload and download video resources for teaching, the experiences gained in the present study in teacher education on good practices of teaching suggested the interactions and sharing among student teachers with the online video database enhanced knowledge building.

References [1] Duncan, C., & Ekmekcioglu, C. (2003). Digital libraries and repositories. In A. Littlejohn, (Ed) Reusing online resources: a sustainable approach to e-learning (pp.135-145). Creative Print and Design (Wales), Ebbw Vale. [2] Haga, H. (2004). Concept of Video Bookmark (videomark) and its Application to the Collaborative Indexing of Lecture Video in Video-based Distance Education. International Journal on E-Learning 3(3), 32-37. [3] Littlejohn, A. (2003). Issues in reusing online resources. In A. Littlejohn. (Ed) Reusing online resources: a sustainable approach to e-learning (pp.1-8). Creative Print and Design (Wales), Ebbw Vale. [4] Haughey, M., & Muirhead, B. (2005). The pedagogical and multimedia designs of learning objects for schools. Australasian Journal of Educational Technology, 21(4), 470-490. [5] Machionini, G., & Geisler, G. (2002). The open video library. Retrieved July 15, 2003, from http://www.dlib.org/dlib/december02/marchionini/12marchionini.html [6] Marques, O. and Furht, B. (2002). Content-based visual information retrieval. In Timothy Shih, K. (Ed), Distributed multimedia databases: Techniques & applications, Hershey, PA: Idea Group Publishing, 37-55. [7] Wiley, D. A. (2000) Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. In D.A. Wiley (Ed) The Instructional Use of Learning Objects. (http://resuability.org/read/chapters/wiley.doc)

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

397

Building and Evaluation of a Semantic Web System that Provides Teachers with Lesson Plans Toshinobu KASAIa, Haruhisa Yamaguchia, Kazuo NAGANOb, Riichiro MIZOGUCHIc a Faculty of Education, Okayama University, Japan b Faculty of Liberal Arts, University of the Sacred Heart, Japan c Institute of Scientific and Industrial Research, Osaka University, Japan [email protected] Abstract: In Japan, the “Period of Integrated Study” program began in elementary and secondary education in 2002. The "Period of Integrated Study" program emphasizes cultivating the ability to solve various problems in society. Most goals of this program involve meta-ability, which cannot be fully learned by traditional Japanese instructional methods. For this reason, it is necessary and important to provide instructors with a powerful help system that can locate and provide access to a variety of useful information resources. To this end, we have proposed a framework that reconstructs the resources according to various viewpoints based on Semantic Web technology. Further, we proposed a Goal Transition Model to show a skeleton of the transition of instructional goals based on ontologies and built a system that provides teachers with lesson plans effectively by applying this Model. In this paper, we describe support functions of this system and report results of an experiment carried out for evaluation of the functions. Keywords: Semantic Web, Ontology, Lesson Plan, Practical Skill, Learning Object

Introduction In Japanese elementary and secondary education, the acquisition academic knowledge had been regarded as important rather than the enhancement of practical skills. In April 2002, however, the Ministry of Education started the "Period of Integrated Study" program in the elementary and secondary education system. The objective of this program is to cultivate learners' ways of learning and thinking and an attitude of trying to creatively solve problems by themselves. However, because Japanese teachers have little experiences with instruction in practical skills, they lack the specific skills for instructional design. Many organizations provide web pages with useful resources for teachers such as digital content, lesson plans, and Q & A [1][2][3]. However, it is very difficult to collect the necessary resources for teachers because there are so many of these relevant web pages, and their formats and viewpoints are not unified, even when the resources have the same purpose. One of the causes of these problems is that various concepts related to practical skills have not yet been clearly defined. Because most of the guidelines and commentaries about the “Period of Integrated Study” present the concepts in a disorganized fashion, we believe that these concepts cannot be conveyed to teachers effectively. To solve this problem, it is necessary to clarify and articulate the fundamental concepts underlying the practical skills needed, and we believe that ontological engineering can assist in meeting this goal. The ontology we have developed provides a common vocabulary/concepts and fundamental conceptual structure for instruction in practical skills, and its existence can promote the reuse and sharing of these concepts among teachers. However, because the ontology is quite

398

T. Kasai et al. / Building and Evaluation of a Semantic Web System

abstract, we think that it would not be effective to instruct teachers in it. Therefore, in this study, we use the ontology as a basis and introduce educational goals for practical skills to define other useful information. If useful web resources for the “Period of Integrated Study” are tagged on the basis of ontology, they can be accessed according to the various viewpoints they might have. This framework can be realized based on Semantic Web technology, which is an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in cooperation [4]. We also proposed a Goal Transition Model that shows a skeleton of the transition of the instructional goals based on ontologies [5]. This model depends on the situation in which the learning objective is to cultivate the ability to solve various problems. If the skeleton of each provided lesson plan is expressed based on this model, teachers can judge whether the plan would be appropriate for their instructional objectives without reading it in detail. Furthermore, by using this model, support functions which search resources more effectively and provide teachers with resources more suitable under the assumption that learning happens during the problem-solving process can be realized. In this paper, we have implemented the support functions. And, we carried out an experiment for evaluation of the effectiveness of the functions with students who belong to Faculty of Education in our University. The remainder of this paper is structured as follows:in section 1, we describe the outline of the framework and the Goal Transition Model that we already proposed in the past papers [5][7] for realization of the support functions that we implemented. In section 2, we explain the functions in detail. In section 3, we report the results of experiment for evaluation of the functions. Finally, in section 4, we present the summary and future work.

1. Outline of Our Approach 1.1 Outline of the Framework that Supports Teachers based on the Semantic Web Technology The framework we employ is an example of the Semantic Web application system that is open to the decentralized world. An outline of this framework is shown in Figure 1. This framework includes two sets of metadata, one of which is based onthe Goal List of IT (Information Technology) education [6], which was taken from research that has been conducted by one of the authors for the last several years. The Goal List is a classification of the goals of IT educationin the “Period of Integrated Study” in terms that are familiar to teachers.The purpose of the Goal List is to provide teachers with viewpoints from which to evaluate the learner's activity during instruction in IT education. For the purpose of this classification, examples of more concrete learning activities that are easy for teachers to understand are provided with a level that shows when learners should attain this goal. For example, "Level 1: A student can express his/her feeling" is provided for "a: Expression of information" as an example of concrete learning activities.Because this Goal List was not generated based on the ontology theory, its quality is not as high as that of an ontology [7]. However, this Goal List has already been so widely used with the same purpose as an ontology that many information resources that support teachers for IT education in Japan are annotated using this list. The other set of metadata is based on our ontologies. We authored the metadata of various resources about IT education in RDF using the ontology of the goal of IT education and the ontology of the fundamental academic ability as the tag. These ontologies were built on the editor “Hozo” [8], which is an environment for building ontologies. The ontology of the goal of IT education consists solely of the concepts of the goal of IT education.

T. Kasai et al. / Building and Evaluation of a Semantic Web System

399

Figure 1 The outline of our approach, which is compliant with the openness of the Semantic Web

Stratification based on the is-a relation is the essential property of these concepts, and ensures that no confusion of various concepts occurs; such confusion can obstruct teachers' understanding of the concepts of IT education. In other paper, we described this ontology in more detail and showed its advantages by comparing it with the other classification from the viewpoint of the ontology theory[7].In this study, we realize semantic integration between the metadata based on separate ontologies by clearly describing relations between our ontologies and the Goal List. For realization of the alignment of the Goal List and our ontology, we describe the goal concepts of the ontology, which are contained in each example of the learning activity of the Goal List. To do this, another ontology which defines more general ability is necessary, because the Goal List contains more general ability than the goal of IT education defined in our ontology. Thus, we extracted and classified the goal of the "Period of Integrated Study" as the ontology of fundamental academic ability. The "Period of Integrated Study" was created to cultivate ways of learning and thinking, and an attitude of trying to solve or pursue problems independently and creatively. Thanks to this framework, for example, the system can reconstruct lesson plans that were tagged based on the Goal List from the viewpoint of our ontologies, and provide teachers with these plans. In addition, the system can integrate lesson plans based on the Goal List with digital contents based on our ontologies, which can be used in each step in the lesson plans. With this framework, it will become possible for teachers to make better use of the many resources available to them, for a wider range of purposes.And, this framework provides teachers with benefits of both the Goal List and our ontologies. The benefit of the Goal List is that its expression is easy for teachers to understand, so teachers themselves can describe metadata from various resources. In fact, intensive activities had already been established to collect and store educational resources with metadata using “Goal List”. This leads us to expect that the scalability of this framework will continue to improve. On the other hand, the benefits of our ontologies are that defined concepts show essential properties without confusion with other concepts and have high generalizability, which means that our ontologies can be applied in various situations and that specialized support in each situation becomes possible. An example is the support using the Goal Transition Model, which we proposed in other paper [5]. Next, we describe this model briefly.

400

T. Kasai et al. / Building and Evaluation of a Semantic Web System

1.2 The Goal Transition Model This model depends on the situation where the purpose of learning activities is problem-solving (parts of the problem-solving process), since the “Period of Integrated Study” program emphasizes cultivating the ability to solve various problems in society. We defined a general process for problem-solving by referring to the Geography Standards Project [9] and extracted the practical skills from our ontologies that are necessary to carry out the essential processing in each step of this process. In this study, we call these skills “core skills” in the problem-solving process. The problem-solving process is shown on the left side of Figure 2, and the core skills are shown on the right side of Figure 2.

Figure 2 The problem-solving process and the core skills in this process

Most lesson plans of the "Period of Integrated Study" program provided via the Internet aim to cultivate practical skills to be used in the problem-solving process. If all of the core skills of the problem-solving process are extracted in order from each lesson plan, it is possible to express a skeleton of the instruction from the perspective of the problem-solving process.The core skills express the true nature of this process in the context of problem-solving learning. In this study, we call this skeleton the "Goal Transition Model." All concepts that can be used in this model are defined in our two ontologies. An example of a Goal Transition Model extracted from an actual lesson plan is shown on the right at the top of Figure 4.

2. Implemented Two Functions to Provide Teachers with Lesson Plans based on the Goal Transition Model The functions use the simple lesson plans on the Web (called Digital Recipes) provided by Okayama Prefectural Information Education Center [2] and the Meeting of Tuesday [1]. These Digital Recipes are accessible to the public as resources based on the concepts proposed by the Goal List. An example of a Digital Recipe is shown on the left at the top of Figure 4. In this Digital Recipe, for each step in the flow of the instructions, the evaluations are expressed in language that is easy for teachers to understand based on the Goal List. These Digital Recipes were provided for teachers in two static ways by Okayama Prefectural Information Education Center and the Meeting of Tuesday, as shown in Figure 3. One is how to enumerate all Digital Recipes without strategy, which is shown on the left side of Figure 3. The other is how to construct the lists of a Digital Recipe from the

T. Kasai et al. / Building and Evaluation of a Semantic Web System

401

A Title of a Digital Recipe

A Title of Digital Recipe An example of learning activity in the Goal List

Figure 3 The screen shots of Web pages that provide teachers with Digital Recipes

viewpoint of the Goal List, which is shown on the left side of Figure 3. However, because Digital Recipes were not described as metadata, they are not provided dynamically in other ways according to the requirements of teachers. We authored the metadata of these Digital Recipes in RDF: Resource Description Framework [10]from the viewpoint of the Goal Listto use them in our proposed framework. The system analyzes the metadata and extracts concepts of the Goal List tagged in this resource, and then the system extracts the concepts of the ontology of the goal of IT education and the ontology of the fundamental academic ability related to those concepts of the Goal List from the description of relationships between our two ontologies and the Goal List. Next, the system connects and outputs the core skills in the order of the problem-solving process. Furthermore, the system outputs each other concept at the right side of the core skill contained in the same learning activity that contains the concept. Here, when the different concepts that are in the same step of the problem-solving process are repeated, the system outputs these concepts in parallel from the previous core skill. This is because concepts in the same step cannot be arranged. One function builds the Goal Transition Model of a Digital Recipe automatically and provides teachers with the model, as shown at the top of Figure 4. For this function, teachers can obtain the skeleton of this lesson from the viewpoint of educational goals without going through the lesson plan in detail. This skeleton provides teachers with a description of the true nature of the lesson, which can be difficult to uncover among superficial information such as learning activities, information systems, digital contents and the like. Therefore, we think that this function is useful for teachers who are not accustomed to cultivating practical skills in their students. The other function searches necessary Digital Recipes from the viewpoint of the problem-solving process according to the requirement of teachers. By clicking on the place that shows each step in the problem-solving process, teachers can get lists of Digital Recipes containing the learning activities required, as shown at the bottom of Figure 4. Furthermore, teachers can click on the place between the steps (for example, place "A" in Figure 4) to get lists of Digital Recipes that contain the learning activities of both steps. In Japan, although IT education and the "Period of Integrated Study" program attach importance to cultivating the ability to solve problems, a function that can search the necessary lesson plans that are open to the public from the viewpoint of a step in the problem-solving process is nearly nonexistent. In this study, this function is realized by using the framework of the Semantic Web based on ontologies and the Goal Transition Model.

402

T. Kasai et al. / Building and Evaluation of a Semantic Web System

Figure 4 Two functions based on the Goal Transition Model

3. Experiment We conducted an experiment to investigate the effectiveness of the functions that we built based on the Goal Transition Model. Eighteen students of Faculty of Education, Okayama University were the test subjects for this experiment. They are studying at the University to become teachers. Seven of the students had taught at an elementary school or a junior high school. After we lectured to all of the students on the purpose of the "Period of Integrated Study," we asked them to design a lesson plan to cultivate learners' ability to solve various problems while referring to Digital Recipes. Half of them, group A, referenced Digital Recipes in the usual way shown in Figure 3, while the others, group B, referenced them using our proposed functions shown in Figure 4. For the experiment, we prepared 28 Digital Recipes and their metadata. We recorded the number of Digital Recipes that were read while each student designed a lesson plan, and the number of Digital Recipes that were actually referenced in the student’s design. Table 1 shows the data of all students in groups A and B. Here, we did not count the number of Goal Transition Models that each student viewed instead of a Digital Recipe. The students in group B who used our implemented functions based on the Goal Transition Model referred to more Digital Recipes to design Lesson Plans than the number of Digital Recipes referred to by the students in group A (proved by Student’s t-test). Furthermore, the percentage of the number of Digital Recipes that students in group B referenced compared to the number that they just read was also higher than in group A. This result showed that students could get necessary information more efficiently than in the usual method by employing the functions that we built based on the Goal Transition Model.

403

T. Kasai et al. / Building and Evaluation of a Semantic Web System

Table 1 The average of the data of all students in group A and B and the result of an analysis by Student’s t-test

Group A Student

Group B

Average

Referenced Read Referenced/ Referenced Read Referenced/ Student Recipes Recipes Read Recipes Recipes Read

㪊㪉㪉㪊㪊㪉㪈㪊㪉

a b c d e f g h i

㪈㪉㪈㪌㪈㪇㪈㪋㪈㪎㪈㪊㪏㪈㪉㪈㪊

㪇㪅㪉㪌㪇㪅㪈㪊㪊㪊㪊㪇㪅㪉㪇㪅㪉㪈㪋㪉㪐㪇㪅㪈㪎㪍㪋㪎㪇㪅㪈㪌㪊㪏㪌㪇㪅㪈㪉㪌㪇㪅㪉㪌㪇㪅㪈㪌㪊㪏㪌

j k l m n o p q r

㪋㪊㪌㪊㪊㪋㪊㪉㪋

㪈㪈㪈㪇㪈㪉㪈㪌㪈㪇㪈㪈㪈㪇㪐㪈㪉

㪇㪅㪊㪍㪊㪍㪋㪇㪅㪊㪇㪅㪋㪈㪍㪍㪎㪇㪅㪉㪇㪅㪊㪇㪅㪊㪍㪊㪍㪋㪇㪅㪊㪇㪅㪉㪉㪉㪉㪉㪇㪅㪊㪊㪊㪊㪊

Referenced Read Referenced/ Recipes Recipes Read

Group A Group B Value t 㪁㫇㩷㪓㩷㪅㪇㪈

㪉㪅㪊㪊㪈㪉㪅㪍㪎㪊㪅㪋㪋㪈㪈㪅㪈㪈㪄㪉㪅㪐㪌㪁㪈㪅㪋㪎

㪇㪅㪈㪎㪇㪅㪊㪈㪄㪋㪅㪈㪉㪁

The result of an analysis by Student's unpaired t-test

The effectiveness of the Goal Transition Model and our proposed functions based on this model was evaluated with a questionnaire. Students in group B that used our implemented functions gave a score between one and five to each of five questions, with one being the lowest and five being the highest. The results of the questionnaire are shown in Table 2. Questions 1 and 4 show that the expression of the Goal Transition Model was suitable and effective for students. From the results of questions 2 and 3, students were quite satisfied with our implemented functions based on the Goal Transition Model as a way of providing the Digital Recipes. And, the question 5 shows that the Goal Transition Model supports teachers to design a lesson plan, but it does not go far enough. Table 2 The results of questionnaires to evaluate our system No.

㪨㪈㪨㪉㪨㪊㪨㪋㪨㪌

Question

㪛㫆㩷㫐㫆㫌㩷㫋㪿㫀㫅㫂㩷㫋㪿㪼㩷㪞㫆㪸㫃㩷㪫㫉㪸㫅㫊㫀㫋㫀㫆㫅㩷㪤㫆㪻㪼㫃㩷㪺㫆㫌㫃㪻㩷㪼㫏㫇㫉㪼㫊㫊㩷㪸㩷㫊㫂㪼㫃㪼㫋㫆㫅㫆㪽㩷㪼㪸㪺㪿㩷㪛㫀㪾㫀㫋㪸㫃㩷㪩㪼㪺㫀㫇㪼㪖㪛㫆㩷㫐㫆㫌㩷㫋㪿㫀㫅㫂㩷㫋㪿㪼㩷㪞㫆㪸㫃㩷㪫㫉㪸㫅㫊㫀㫋㫀㫆㫅㩷㪤㫆㪻㪼㫃㩷㫎㪸㫊㩷㫌㫊㪼㪽㫌㫃㩷㫋㫆㩷㫃㫆㫆㫂㩷㪽㫆㫉㫅㪼㪺㪼㫊㫊㪸㫉㫐㩷㪛㫀㪾㫀㫋㪸㫃㩷㪩㪼㪺㫀㫇㪼㫊㪖㪛㫆㩷㫐㫆㫌㩷㫋㪿㫀㫅㫂㩷㫋㪿㪼㩷㫊㪼㪸㫉㪺㪿㩷㪽㫌㫅㪺㫋㫀㫆㫅㩷㪽㫉㫆㫄㩷㫋㪿㪼㩷㫍㫀㪼㫎㫇㫆㫀㫅㫋㩷㫆㪽㩷㫋㪿㪼㫇㫉㫆㪹㫃㪼㫄㪄㫊㫆㫃㫍㫀㫅㪾㩷㫇㫉㫆㪺㪼㫊㫊㩷㫎㪸㫊㩷㫌㫊㪼㪽㫌㫃㩷㫋㫆㩷㫃㫆㫆㫂㩷㪽㫆㫉㩷㫅㪼㪺㪼㫊㫊㪸㫉㫐㩷㪛㫀㪾㫀㫋㪸㫃㪛㫆㩷㫐㫆㫌㩷㫋㪿㫀㫅㫂㩷㫋㪿㪼㩷㪞㫆㪸㫃㩷㪫㫉㪸㫅㫊㫀㫋㫀㫆㫅㩷㪤㫆㪻㪼㫃㩷㪿㪼㫃㫇㩷㫐㫆㫌㩷㫋㫆㩷㫌㫅㪻㪼㫉㫊㫋㪸㫅㪻㩷㫋㪿㪼㪼㫊㫊㪼㫅㪺㪼㩷㫆㪽㩷㪼㪸㪺㪿㩷㫃㪼㫊㫊㫆㫅㩷㫇㫃㪸㫅㪖㪛㫆㩷㫐㫆㫌㩷㫋㪿㫀㫅㫂㩷㫉㪼㪽㪼㫉㫉㫀㫅㪾㩷㫋㫆㩷㫅㫆㫋㩷㫆㫅㫃㫐㩷㫋㪿㪼㩷㪛㫀㪾㫀㫋㪸㫃㩷㪩㪼㪺㫀㫇㪼㫊㩷㪹㫌㫋㩷㪸㫃㫊㫆㩷㫋㪿㪼㪞㫆㪸㫃㩷㪫㫉㪸㫅㫊㫀㫋㫀㫆㫅㩷㪤㫆㪻㪼㫃㩷㫊㫌㫇㫇㫆㫉㫋㩷㫐㫆㫌㩷㫀㫅㩷㪻㪼㫊㫀㪾㫅㫀㫅㪾㩷㪸㩷㫃㪼㫊㫊㫆㫅㩷㫇㫃㪸㫅㪖

Average

㪋㪅㪎㪏㪋㪅㪌㪍㪋㪅㪊㪊㪋㪅㪋㪋㪊㪅㪎㪏

4. Summary We have proposed a framework which is compliant with the openness of the Semantic Web to provide teachers with useful resources more effectively in the past papers [5][7]. In this framework, we built two ontologies; the ontology of the goal of IT education and the ontology of the fundamental academic ability, and we realized the semantic integration to make use of the results of another research [6] by aligning these ontologies based on Semantic Web technology. Further, we proposed the Goal Transition Model to especially support teachers in the situation in which the learning objective is to cultivate the ability to solve various problems.

404

T. Kasai et al. / Building and Evaluation of a Semantic Web System

In this paper, we described the support functions that we implemented to provide teachers with simple lesson plans on the Web using the Goal Transition Model based on the framework that we have proposed. And, we reported the results of an experiment that was carried out with students who belong to Faculty of Education, for evaluating the functions. The result of the experiment showed the effectiveness of our support functions based on the Goal Transition Model.

Acknowledgments This work is supported in part by Grant-in-Aid for Young Scientists (B) No. 18700151 from the Ministry of Education, Culture, Sports, Science and Technology, Japan.

References [1] The Meeting of Tuesday (2002), The curriculum lists of information education in the "Period of Integrated Study", HomePage of the Meeting of Tuesday, Available at http://www.kayoo.org/sozai/. [2] Okayama Prefectural Information Center (2002), Okayama Prefectural Information Education Center, Digital Contents Recipes and Worksheets, Available at http://www2.jyose.pref.okayama.jp/cec/webresipi/index.htm. [3] NICER (2003). National Information Center for Educational Resources, Available at http://www.nicer.go.jp [4] Berners-Lee T., Hendler J., Lassila O. (2001), The Semantic Web, Scientific American. [5] Kasai, T., Yamaguchi, H., Nagano, K., Mizoguchi, R. (2005), Goal Transition Model and Its Application for Supporting Teachers based on Ontologies, Proc. of the Artificial Intelligence in Education (AIED2005), pp.330-338. [6] The Meeting of Tuesday (2001), The Meeting of Tuesday, The Goal List of Information Education, Mail-Magazine of the Meeting of Tuesday, Available at http://kayoo.org/home /project/list.html. [7] Kasai, T., Yamaguchi, H., Nagano, K., Mizoguchi, R. (2006), Building an ontology of IT education goals, International Journal of Continuing Engineering Education and Life-Long Learning, Vol.16, Nos. 1/2, pp.1-17. [8] Kozaki, K., Kitamura, Y., Ikeda M., and Mizoguchi, R. (2000). Development of an Environment for Building Ontologies which is based on a Fundamental Consideration of "Relationship" and "Role", The Sixth Pacific Knowledge Acquisition Workshop (PKAW2000), 205-221. [9] Geography Education Standards Project (1994), Geography for Life: National Geography Standards, National Geographic Research and Exploration. [10] Lassila O., Swick R. (1999), Resource Description Framework (RDF) Model and Syntax Specification, W3C Recommendation.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

405

An Improved Learning Content Management System Framework a

Liyong Wana, Chengling Zhaoa, Ming Zhaob, Qi Luoa, Libing Jianga Department of Information Technology, Central China Normal University, Wuhan, China b Institute of Compute Science, Yangtze University, Jingzhou, Hubei, China [email protected] Abstract: The core of an e-Learning system typically consists of a learning content management system (LCMS). It is aimed at managing learning content which is typically stored in a database. An LCMS provides many essential benefits to e-Learning, but the current system architecture has some disadvantages. The authors analyzed these disadvantages and proposed an improved LCMS (ILCMS) framework. The new framework evolves from local to global. It starts with a local system for local users, and then connects with other local systems to build the broader network. Keywords: learning content management system, local learning content repository, access control, digital rights management, system registry

Introduction E-learning has attracted a lot of attention in recent years. In a typical learning environment, there are several groups of people involved: authors and learners, which are the main players, and administrators and trainers. The core of an e-Learning system typically consists of a learning content management system (LCMS). An LCMS is aimed at managing learning content which is typically stored in a database. An LCMS can also ease content reusability, support content development, manage learners, track their progress and ease collaboration. International Data Corporation (IDC) defines a LCMS as a system that is used to create, store, assemble, and deliver personalized e-Learning content in the form of learning objects. Different LCMSs share the following elements: x A learning object repository. x An authoring tool for creating learning objects. x A dynamic publishing tool for delivering personalized learning content. x An administration tool for providing management function [1]. An LCMS provides many essential benefits to e-Learning, but the current system architecture has three disadvantages: x In the world of the LCMS, everything is contained in one very large learning object repository. When the demands increase, the repository will grow in size. This means that educational institutions must make a major investment in software and expertise in order to access learning contents [2]. x The LCMS are based on a local content database. It is developed for users inside a large organization. If different educational institutions want to share and exchange their learning content, the issues will occurred. x With the size of the learning content become larger and lager, the workload of adding and updating learning contents will become heavier and heavier. The speed of search and management will also slow down.

406

L. Wan et al. / An Improved Learning Content Management System Framework

The authors proposed an improved LCMS (ILCMS) framework. The new framework evolves from local to global. It starts with a local system for local users, and then connects with other local systems to build the broader network [3].

1. The improved LCMS framework The improved LCMS (ILCMS) framework aims to support connection of existing content repositories where these are combined into a single source for content creation and management. The overall model is illustrated in Figure 1. x Local learning content repositories. Learning content remains in local content repositories that are managed and operate under their local rules. x System repositories. The system repositories maintain two components: the repository registry and the repository metadata catalog. The repository registry contains descriptions of all local learning content repositories. The repository metadata catalog mainly maintains a collection of all learning content metadata harvested from every local learning content repository. Content is located by searching against the metadata catalog. The catalog may also maintain additional indexing information, usage data, and context [3]. x Common service module. The common service module provides the core technical and administrative services. These services include identification, authorization, access control, version control, digital rights management (DRM), and security. x Application module. The application module consists of end-user interfaces and the common user tools, such as search, registration, personalization, delivery, assessment, etc. these tools are used to catalog, find, manage and publish learning content and content objects [3]. Application module

Local content Local content Repository Repository

Common service module

System repositories

Figure 1: The improved LCMS framework 1.1 Local content repositories Local learning object repository is a central database where content is stored and managed. The power of the repository is that it allows the objects to be reused and enable developers to easily create customized learning content for targeted users. The participants in the framework can operate separately under their own rules. The local content repository architecture is illustrated in Figure 2.

L. Wan et al. / An Improved Learning Content Management System Framework

407

Media creation Tool

Authoring tool

Learning Object Repository

Metadata Repository

Knowledge capture Tool

Figure 2: The local content repository architecture 1.2 System repositories A key concept of the improved LCMS framework is the harvest of metadata from the individual source repositories into the single federation metadata registry. All of the local repositories are registered in the repository registry. As said above, each of these local content repositories will need to operate under a different set of rules and policies, be implemented on a different technology base or platform, and use a different set of interoperability standards. For example, one local repository may rely on LOM metadata, and another may use Dublin Core to describe all content objects. Thus, we need to define a model to permit the development of participant under a collection of different technology, policy and management schemes. The model can translate a set of metadata standards, package standards, and repository standards into a common standard. 1.3 The common service module x

x

x x

Access control. Access control defines the types of operations that can be performed on learning content objects by individuals and groups of individuals [4]. The two most common terms associated with access control are authentication and authorization. LCMSs should govern access to users and groups based on their privilege to view, copy, link, export, edit, delete, and change access list. Version control. Version control maintains the version record of an object as it moves through its lifecycle. Version control has applications for object sequencing. It can define which version is appropriate for an instructional sequence [4]. It also allow for some other applications. For example, a user can be maintained during the content development cycle, making it possible to identify the source of an edit and perhaps “roll-back” to previous versions if the edit is not acceptable. Lifecycle Management. The ability to identify out-of-date content is a key component of lifecycle management. An LCMS should be capable of identifying objects that have an “expiration date” so that invalid content is not propagated [4]. Digital rights management (DRM). The learning content repositories pose several challenges for owners of content who want to manage access rights and protect their content from unauthorized users. Digital Rights management was used for all members of the Federation. When designing a DRM system, there are two main questions to answer: how are rights expressed and how are rights enforced? There is a desire to use rights documentation metadata as part of the searching criteria

408

L. Wan et al. / An Improved Learning Content Management System Framework

learning objects. In addition, there is a widely-held need to provide automated tools to document rights for new, composite or derivative works by drawing upon rights documentation associated with existing works. 1.4 The application module x x x

x

x

The dynamic delivery interface. To serve up a learning object or contents based on learner profiles, pre-test, and or user queries. Also allow authors to localize the look and feel of courses. Cross-media publishing. Learning content can be used for various publication types (online learning course, paper-based lecture notes, CD-ROM, presentation, etc.) Assessment tool. LCMS should support assessment creation and management. To measure learners’ performance against specific learning goals [4]. Assessment includes multiple-choice questions, multiple right answer, short answer and true/false. Evaluation criteria include support for item randomization, number of attempts, hints, assessment response feedback, weighting, and reporting. Management. Management should include three aspects. User management- to manage users, groups and roles account registration and information. Collaboration management-to support multiple forms of interaction such as e-mail, threaded discussion, whiteboard, and chat. Event Management–to capture all interaction between users and system for the logging and tracking purposes. Search service. The true power of metadata depends on the search capabilities. A good LCMS must provide the ability to construct search and retrieval capabilities based on learning content metadata. There are two approaches for searching. One is the “abstract definition” of the searches. Indexes required supporting these searches and the “logic” one applies to the LOM to construct the indexes. The other Implementation is bindings e.g. Z39.50 [5].

2. Conclusion and future work In this paper, we mainly described an improved learning content management system (ILCMS) framework. The framework consists of three modules: Local learning content repositories, system repositories, common service module and application module. This framework will change the current architecture of current LCMSs in a certain extent. But the feasibility of the ILCMS still needs us to test in the future.

References [1] Liyong Wan, Chengling Zhao ,Qingtang Liu and Junyi Sun. Work in Progress-An Evaluation Model for Learning Content Management Systems: from a Perspective of Knowledge Management. Proceedings of 35th ASEE/IEEE Frontiers in Education Conference .Indianapolis, IN. [2] Stephen Downes. An Introduction to RSS for Educational Designers. Retrieved 8 August, 2005 from www.downes.ca/files/RSS_Educ.htm. [] Daniel R. Rehak, Philip Dodds, Laurence Lannom. A Model and Infrastructure for Federated Learning Content Repositories. Proceedings of WWW 2005, May 10--14, 2005, Chiba, Japan. [] Jeff Katzman and Jeff Caton. Evaluating Learning Content Management Systems (LCMS).Copyright peer3 Corporation. [] Anderson Neil McLean and Clifford Lynch. Interoperability between Information and Learning Environments– Bridging the Gaps. Retrieved 8 August, 2005 from www.imsglobal.org/DLims_white_paper_publicdraft_1.pdf .

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

409

Resource Based Solution to Teachers’ Knowledge Management Hongtao Sun a, Lu Wang b, Hongwei Dai b Educaitonal Technology Center, Central University for Nationalities , China b Dept. Educational Technology, Capital Normal University, China [email protected]

a

Abstract: With the rapid development of ICT (Information and Communication Technology) based education, more and more multimedia resources are available to schools. However the quality of these resources and their utilization are far from satisfactory. To fully utilize these multimedia resources in classroom instruction involves a combination of teachers’ practical knowledge and presentation knowledge. Instruction design can be the bridge between multimedia teaching/learning resources and teachers’ knowledge. In this paper, a solution of representing teachers’ knowledge is suggested. With the help of a set of rules on how teachers use the multimedia teaching resources in a visualized way, the practical knowledge of teachers is to be represented and thus, reused. In this process, the understanding of multimedia teaching/leaning resources in instruction becomes explicit and can be shared among teachers. Keywords: multimedia resources, knowledge management, instruction design, teachers’ knowledge

Introduction Nowadays, more and more investment has been spent on the multimedia resources in schools. But how the utilization of these resources is? During research on national key project, Teachers’ ICT based Professional Development Basement, we train several hundreds of teachers in Beijing to improve their capacity of using ICT to facilitate their teaching. A paradox about the resources was found that although lots of money has been spent on multimedia resources, teachers find it difficult to use to assistant their instruction efficiently. Some investigations to the resources application of schools teachers also prove that it is a real problem to teachers [1][2]. Different teachers hold different opinions about the relation between resources and instruction. Accordingly, they use different pedagogical strategies and organize different instructional activities in their classrooms. [3] The various strategies and activities show the difference between each teacher’s knowledge about resources and instruction. And strategies and activities constitute the main body of instruction designs. If we bridge teachers’ practical knowledge and the available multimedia teaching resources with the process of instructional design, not only instructional features of these resources can be discovered and the utilization of these resources will take on a new look, but also the personal, tacit knowledge of teachers can be described in a way that shows an explicit relationship between the teaching method and the related multimedia teaching resources. With the research on multimedia teaching resources, instruction design and teachers’ knowledge, multimedia resources repository, lessons preparing tools and knowledge management system are likely integrated together.

410

S. Hongtao et al. / Resource Based Solution to Teachers’ Knowledge Management

1. Literature Review With the development of ICT based education, more and more resources can be easily accessed. However, it is difficult for teachers to find the proper ones to aid their instruction. Generally speaking, the resource repository, which usually includes multimedia resources like video, sound, picture etc., contains better organized resources and focuses on school instruction. But even resource repository has several disadvantages. Lack of standard in data description, focusing on quantity rather than quality and neglecting the instructional features of resources is the most common problems of repository [4]. The investigation to teachers in Guangdong, Beijing and other three provinces showed that teachers were not satisfied with the quality of resources [1][2]. Researchers hold different ideas to defined teachers’ knowledge. Elbaz (1991) used “personal knowledge” indicating that teachers’ knowledge was unique. [5] Schwab (1971) believed that it was ‘the wisdom of practice’. Eraut (1994) suggested that it was tacit. From all the definitions above, some characteristics of teachers’ knowledge can be summarized as personal, practical and tacit. Accordingly, it is difficult to express and share. The success of knowledge management in enterprises gives some illumination that knowledge management should focus on the core of work. [6] The exploitation of teachers’ practical knowledge should pay great attention to their daily work. Focus on classroom instruction will help to manage and share their practical knowledge. Words and characters have their limitations to describe knowledge. Eppler & Burkard (2004) stated that visualization of knowledge can facilitate the spread and innovation of knowledge [7]. Concept Map, Mind Map and Knowledge Map are all successful solutions to visualize knowledge [8].But is visualization enough? The visualization can describe the knowledge in a clearer way. However, the difficulty of knowledge sharing still remains without a set of corresponding rules which make knowledge able to be represented and reused. If knowledge has the visual form, and is assembled by certain rules, it will be easily searched and reassembled. The spread and share of knowledge will take on a new look. Both resource and knowledge need to be described in a standardized way. Research on learning object (LO) provides a solution. Although the definition to LO is different [9] [10] . All the definitions describe the most important feature of LO, reusable. To enable learning object reuse, standard should be defined to describe LO. The standard of Learning Object Metadata provides a mechanism for describing content [9]. But standard alone is not sufficient to allow a learning object to be reused. In order to reuse learning objects or resources, principle and strategy should be defined to figure out the relationships between them. Scorm2004 used principles of sequence and navigation to describe the relationships. The idea of transmitting a serial of contents to a sequence of activities is helpful to the design of our rules mentioned above. 2. Research Problems As discussed above, the efficiency of resources utilization in education are far from satisfactory. Even resources repository, which is most likely helpful to teachers, cannot provide useful information about instructional usage of resources. Resource repository pays great attention to the amount of resources, but overlooks the relation between instruction and resources, which is more important to teachers. In the meanwhile, resources are difficult to search and manage due to lack of standardized description. Teachers are still puzzled about how to get multimedia resources effectively and use them to facilitate their classroom instruction. A possible solution to this puzzle is to let

S. Hongtao et al. / Resource Based Solution to Teachers’ Knowledge Management

411

teachers share their experience of using multimedia resources to teaching. However, behind the experience of resource utilization is the difference of teachers’ knowledge which is personal, practical and tacit. How can they share knowledge like that? Knowledge management shed a light on that difficulty. With focusing on one of the core parts of teachers’ daily work, instructional design, it is likely to bridge multimedia teaching resource utilization and teachers’ knowledge. If we find a proper way to describe the relationship between multimedia resource, instruction and teachers’ knowledge, not only the puzzle of instruction use of resources can be solved, but also the knowledge of teachers can be represented and shared. Also, with the illustration of LOM, standardized description will be helpful to search and reuse resources. Ideas from knowledge representation suggest that with the help of rule expressions and graphical representation like concept maps, knowledge can be represented more clearly and delivered more easily. Thus, knowledge of people from the same field can be aggregated to generate a more comprehensive picture of knowledge. 3. Proposed Solution The goal of this study is to represent teachers’ knowledge in relation to the utilization of multimedia teaching resources, which starts from the use of resources, covers the whole process of Instructional Design and presents the teachers knowledge in a visualized way. Different ways of resources using result from the different understanding of curriculum knowledge (what to teach) and method of instruction (how to teach), which contain practical knowledge of teachers. Thus, instructional design is significant to teachers’ knowledge. From the interviews to teachers, observations to teachers’ daily work and literature review, it was found that although the practice of instructional design varied from teacher to teacher, they accept the steps showed in Fig. 1 as a useful process. Figure 1 Steps of Instruction Design

If, in the process above, teachers record the way of resources use in knowledge units and instruction activity, they also describe instructional features of multimedia resources in the meanwhile. Standardized metadata description like LOM can help to describe data in details and describe relationships among data. A set of description rules should be provided for teachers to describe resource, knowledge unit, and instruction activity. LOM use detailed rules to describe data. But it is so tedious for teachers. If activity, knowledge unit, and multimedia resources are described separately, teachers can link different kinds of data together. So attributes described in upper level data will be inherited by lower level data. After all the steps, teachers will get a picture of resources, knowledge units and activities. All the relations are stored in database. From resources to instructional design, teachers use their practical knowledge to finish a process of design work. The relationship between multimedia resources and classroom instruction is presented. After the classroom instruction, they may make some changes to their design. With all the information stored, teachers can share not only the instruction design, but also the resources and activities used in their instruction. After instruction design and classroom instruction, teachers will get a clearer view of the relations among the knowledge units. It is time to provide chances for teachers to describe the relations. But it is not ensure that they can describe it easily. Relations among knowledge units will trouble them a lot. To solve this problem, visualization, like concept map, could be helpful. In order to help teacher to fulfill the tasks, templates are provided, in

412

S. Hongtao et al. / Resource Based Solution to Teachers’ Knowledge Management

which some simple relations presented in visual way [11]. The relations present in visual form should have corresponding rules to describe them. If knowledge structure templates in visual form are provided and a set of rules are defined accordingly in advance, teachers can describe the relationships of knowledge units in a visualized way with the help of structure template. After the description, they will get a knowledge map. With the rules defined, the visualize relations can be translated into codes. The codes that describe relations of knowledge units enable teachers to unite two or more maps by defining relations of the core knowledge units in each map. Finally, teachers get a map of their own knowledge. A map not only contains relations and knowledge units, but also the resources used and activities designed. The practical knowledge of what to teach and how to teacher is described, in relation to the multimedia teaching resources. With the help of codes, the knowledge units and relations can be searched and reused. The knowledge map can be redrawn by different people, in different time and places. In this way, the visualized practical knowledge can be shared and spread. Borrowing the idea of WIKI, several teachers can construct knowledge map collaboratively in the same curriculum, even in related curriculums. In conclusion, with the proposed solution, the resources issue is likely to be solved. The resources used and the way teachers used them, are recorded. Teachers not only can share the resources, but also can share the ideas they use resources to classroom instruction. Standardized description, like metadata, will enables search, management and share of resources. On the other hand, teachers’ practical knowledge can be described in the form of both visual and code. The representation, delivery and spread of teachers knowledge will take on a new look. In further research, the rules to describe relation among knowledge units will be finished and a software toolkit will be developed. Acknowledgments Prof. Lam-for Kwok, City University of Hong Kong, gave lot of instructions to this paper. I show my heartful thanks here! References [1]Zhan Bin(2005), The Investigation to the Resource Application of K12 Schools in Guangdong Province. China Audiovisual Education. 2005, 03, 71-75. (Chinese) [2]Xu Enqin & Liu Mei-feng(2005). The investigation to the Demand of K12 School Teachers. China Audiovisual Education. 2005,03,74-77. (Chinese) [3]Brown, J., et al. (1991). Learning to Learn. Policies and Guidelines for the Implementation of Resource-based Learning in Newfoundland and Labrador Schools. [4]Ma Xiu-feng & Qi xiao-tao(2004). Research on Web Based Instructional Repository. China Distance Education .2004, 02, 55-58. (Chinese) [5]Connelly, F.M. , Clandinin, D. J & He, M F (1997). Teachers’ Personal Practical Knowledge on the Professional Knowledge Landscape. Teaching and Teacher Education, Vol. 13, No.7 665-674. [6]Mertins, K., Heisig, P., Vorbeck, J. (2004). Knowledge Management-Concepts and Best Practices 2nd Edition. Tsinghua Press. (Chinese) [7]Eppler,M.J. & Burkard,R.A.(2004). Knowledge Visualization Towards a New Discipline and its Fields of Application. ICAWorking Paper 2004, University of Lugano, Lugano. [8]Zhao Guoqing, Huang Ronghuai & Lu Zhijian (2005). Theory and Methodology of Knowledge Visualization..Vol.11, No.1, 23-27. (Chinese) [9]LTCS IEEE, IEEE 1484 Η 12, The Learning Object Metadata standard, http://ieeeltsc.org/wg12LOM/lomDescription [10]Wiley (2002). Connecting Learning Objects to Instructional Design Theory: A definition, a metaphor, and a taxonomy. http://www.reusability.org/read/. [11]Thinking maps. (2004). Thingking Maps. http://thinkingmaps.com.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

413

Collaborative Building of Japanese Kanji Pronunciation Database for Learning Japanese by Chinese Fei Yuana, Jing Yuanb, Rong Wanga, Hiroyuki Mitsuharaa Kazuhide Kanenishic, Yoneo Yanoa a Faculty of Engineering, Tokushima University, Japan b School of Economics and Management, Xi’an Shiyou University, China c Center for Advanced Information Technology, Tokushima University, Japan [email protected] Abstract: For Chinese learners who learn Japanese, it is difficult to enhance speaking ability through reading Japanese teaching materials and articles. Because many Kanji have the same or approx meaning in Japanese and Chinese, therefore during reading the Japanese educational materials Chinese learners are not necessary to master the pronunciation of Kanji in Japanese to understand. Currently few systems deal with learning the pronunciation of Japanese Kanji in reading and furthermore enhancing speaking ability. In this paper, we present a self-directed reading environment for Chinese learners to learn the pronunciation of Japanese Kanji in daily reading and describe developing a collaborative-building database to realize it. Keywords: CALL system, Collaborative building, database, Japanese Kanji pronunciation, self-directed learning

Introduction Learners of Japanese language is increasing. At the same time, Information technology is developing with high speed and e-Learning is becoming to a widespread use [1] [2]. Many CALL(Computer Assisted Language Learning) [3] [4] systems are based on Internet, through which learners can obtain more amount of information more quickly whenever and everywhere. For Chinese learners who learn Japanese, many Internet CALL systems have been developed. Most of these CALL systems focus on supporting learners to share and read teaching materials or articles to increase the amount of information and expand learner’s scope. This method is effective to master the grammar and structure of Japanese, and because many Kanji have the same or approx meaning in Japanese and Chinese, Chinese learners can enhance the reading comprehension and writing ability of Japanese sooner than learners who can’t learn about Kanji in his mother language. But because the pronunciation of Kanji is different in Chinese and Japanese, learners can’t learn the pronunciation of Kanji during daily reading through genaral CALL system. In this paper, we present a self-directed reading environment, which aims to enhance learners’ speaking ability through daily reading, named KLSE (Kanji Learning Support Environment). In this paper, we present our system design and implementation. The rest of this paper is arranged as follows. Section 1 proposes the overall architecture. Section 2 presents the

414

F. Yuan et al. / Collaborative Building of Japanese Kanji Pronunciation Database

database structure, which is the foundation of the system. Section 3 presents summary and future work directions.

1. Outline of KLSE In KLSE three types of users, learner, author and teacher are identified. Teachers are Japanese or Japanese teacher, who will check the correctness of Japanese Kanji information and the appropriateness of reading resources for learners. All readers registered in KLSE will be identified as learners. All of learners and teachers will be authors, and they will build and expand the Japanese Kanji database collaboratively.

Figure1: System architecture

Fig. 1 illustrates the system architecture. KLSE performs the four elements as follows: (1) KLSE provides updatable reading resources in Japanese. Up-to-date Japanese reading resources are the base of KLSE. A database is built to store them. In KLSE all users can insert new reading resource into system. KLSE provides function to transform electronic reading resources into text-only format. Teachers will check the appropriateness of the reading resources. (2) KLSE supports to building an expandable, modifiable Kanji dictionary database To provide Japanese Kanji information during reading process, we built a Japanese Kanji dictionary database (DDB) including Kanji pronunciation, example words and example sentences. DDB is identified to be built by all of users in KLSE. To make the Japanese Kanji pronunciation support function effective in the first stage of system use, teachers should build a small size DDB previously and along with the increase of users and activization of reading/learning actions, DDB will have more and more amount of information and provide more efficient support to learners. The correctness of DDB data will be checked by all users, but only teachers can modify the existed Kanji information data. KLSE provides function that user notice teacher immediately if he finds some Kanji with incorrect information. (3) KLSE provides real-time Kanji pronunciation consulting support DDB provided the base of Japanese Kanji pronunciation support, KLSE provides two kinds of functions to realize the real-time support. (1) For single Kanji, if learner moves mouse on the Kanji, KLSE will search the DDB and show the result by JavaScript module. (2) For the term including Kanji, KLSE will show the consulting result after learner drags it. For the Kanji and terms which pronunciation information haven’t been stored in DDB,

415

F. Yuan et al. / Collaborative Building of Japanese Kanji Pronunciation Database

KLSE can’t show the appropriate consulting result immediately, but KLSE will store them into DDB and set the status as “waiting for answer”, then display them to all users. While some author input the Kanji pronunciation information of a Kanji or term, KLSE will delete the “waiting for answer” status and notice the learner who is waiting for the answer. (4) KLSE provides self-directed functions to support review and self-evaluation All the personal information should be recorded and managed to provide self-directed learning support. KLSE build a personal database (PDB) for every learner. PDB stores the learner’s learning history including the information of learned Kanji and term, and keep the data exchange with DDB. For example, while the “waiting for answer” Kanji or term displayed in DDB is answered, DDB will notice learner and transfer the Kanji pronunciation to learner’s PDB. While learner consulted the “waiting for answer” Kanji or term, the Kanji pronunciation data also will be transferred to DDB to share amongst all users. Self-directed learning support is based on DDB and PDB. In the following Section 4, we will describe the database structure and the data exchange of DDB and PDB.

2. Database Building and Data Flow As described in Section 1, three kinds of databases exist in KLSE. Reading resource database stores the text data of Japanese readings. Dictionary database (DDB) and personal database (PDB) store the Japanese Kanji pronunciation data and the other relative Kanji information such as example words, example sentences and so on. Tab. 1 shows the example of structure of DDB and PDB. Kanji

pronunciation*

notice*

DDB

ⶹ

Ƕ

0

PDB

ⶹ

Ƕ

0

proficiency

2

article ID

waiting for

Who is

answer*

waiting

0

0023

0002

0

example*

ⶹ㛑 ⶹ㛑

Marked with * means that field must be same value in DDB and PDB

Table 1: Data configuration in DDB and PDB

As Tab. 1 shows, DDB and PDB have the same fields such as Kanji pronunciation and example words or sentences. The field “status 1” illustrates the whether the Kanji (term) is noticed to teacher for check. “0” means “not noticed”, “1” means “noticed”. “Proficiency” is identified only in PDB, “0” means the learner’s proficiency is low, “1” is normal, “2” is high. “Article ID” is also identified only in PDB, it points to the article where learner learned the Kanji (term) last time. In the following three situations, data flow of DDB and PDB will happen. (1) Mark the “waiting for answer” status While learner can’t consult a Kanji or term from the system, the Kanji (term) will be insert into either PDB or DDB without pronunciation and the value of “waiting for answer” will be written as “1”. It means, the “waiting for answer” Kanji (term) always exist in pairs in PDB and DDB. In DDB, the “waiting for answer” Kanji (term) will be displayed to waiting for authors’ answer. In PDB, it will be stored into learning history, where learner can write the answer himself while he knows the Pronunciation of the Kanji (term). (2) Release the “Waiting for answer” status Releasing the “waiting for answer” status happens in two patterns. One is releasing from DDB. While author wrote the pronunciation of “waiting for answer” Kanji or term in DDB interface, the pronunciation data will be written into “pronunciation” field and the value of field “waiting for answer” will be modified to “0”. It

416

F. Yuan et al. / Collaborative Building of Japanese Kanji Pronunciation Database

means only when the value of field “pronunciation” is blank, the value of field “waiting for answer” is “1”. Because the field “who is waiting” in DDB recorded whose PDB has the same “waiting for answer” Kanji (term), KLSE will immediately write the pronunciation data into that PDB and modify the value of field “waiting for answer” in PDB to “0”. Furthermore, KLSE will notice learner that his “waiting for answer” Kanji (term) has been answered while learner login system. The other pattern is releasing the “waiting for answer” status from PDB. While learner get to know the pronunciation of Kanji (term) and write it into PDB through his own learning history, the “waiting for answer” status in PDB will be released. Like the previous pattern releasing from DDB, the “waiting for answer” status in DDB will be released consistent to in PDB. (3) Notice teacher to check correctness The reliability of Kanji pronunciation information should be checked. But it is difficult that only several teachers check the large amount of data. In KLSE we present a method that all users do the correctness check cooperatively. To ensure only the incorrect data be revised KLSE allows only teachers to modified the data in DDB, but if learners find the incorrect information in DDB or PDB, they can modify the value of field “notice” to “1”. When login the system, all the Kanji (term) information will be listed for the teacher to check and revise in DDB. On the contrary, the Japanese kanji information in PDB will be checked to keep consistent to DDB. Therefore, if teachers revised the data in DDB, the relative data in PDB will also be revised by system, and KLSE will notice learner to check.

3. Summary and Future Work Directions In this paper, we focus on enhancement of Japanese speaking ability of Chinese learners through learning Japanese Kanji pronunciation in reading environment. We presented a self-directed reading environment named KLSE (Kanji Learning Support Environment) and described the system design and database configuration. Furthermore, as the applications of system, we described how KLSE supports the learner’s self-directed learning. One of our principal future works is to finish the system development. After that we are planning to open the system to the web and evaluate the system.

Acknowledgments This research was supported in part by Grand-in-Aid for Young Scientists (B) No.16700561 and Scientific Research (B)(2) No.16300271 from the Japan Society for the Promotion of Science. Reference [1] Elizabeth, S. and Jordan, P. (2000), A Framework for Enabling an Internet Learning Communityˈ Educational Technology & Society 3(3), 2000, pp.393-408. [2] Yuan, F., Mitsuhara, H., Kanenishi, K. and Yano, Y. (2005): A Web-based Collaborative Authoring System for Web Educational Material based on Evolutionary Information Sharing Approach, Journal of Information and Systems in Education, Vol.4, No.1, 2005, pp.24-36. [3] Egbert, J. and Hanson-Smith (Eds.) E. (1999). CALL Environments: Research, Practice, and Critical Issues. USA. [4] Johann G. and Judith, K. (2002): A review of intelligent CALL systems. Computer Assisted Language Learning 15: 329-342.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

417

Using IT to Power and Support Problem-Based Engaged Learning a

Wee-Meng Hoea, Irene Tana Jurong Secondary School, Singapore [email protected]

Abstract: In this paper, we describe the use of IT to support and enhance student-centred learning in a Singapore secondary school. Using problem-based learning as the key pedagogical model, the school management derived policies and strategies for technology implementation. In addition, the school attempts to achieve intercultural understanding through the extension of learning beyond the classroom. Keywords: Constructionism; data-loggers; IT in education; learning management system; problem-based learning (PBL); robotics; talent development; tablet PCs

Introduction Jurong Secondary School (JSS) is a multi-racial government school in the western part of Singapore. Founded in 1963, JSS is an established school (see the school website at http://www.jurongsec.moe.edu.sg) with a supportive school advisory committee and alumni association, which include members who are successful politicians, professionals and businessmen. Apart from a strong grounding in content knowledge and discipline-specific skills, JSS wants her students to be flexible, creative and resilient in order to better prepare them for the future. The learning environment also aims to enable students to acquire the correct values and life skills for success in an ever-changing world economy. One of the strategic goals of the school is to enable students to achieve intercultural understanding and to have a global perspective of issues.

1. Using Problems to Effect Engaged Learning In 2004, the school embarked on creating an authentic, learner-centered environment. All teachers of the school were trained in problem-based learning (PBL) as a pedagogical approach. The PBL approach hinges on using real world scenarios or simulated phenomena of day-to-day happenings as the starting point of learning. It was found that the curiosity of the students kept them motivated and engaged when problem scenarios were presented through texts, graphics and different media. Learning was enhanced as students identified new areas of knowledge they needed to acquire, and analyze and integrate information to solve the problems [1]. PBL is an active-learning and learner-centered approach in which ill-structured problems are used as the starting point and anchor for the inquiry and learning process. As students go through the PBL process, they learn to confront a problem systematically and conduct self-directed research, constructing their own knowledge in a meaningful way. Metacognition is also enhanced as students' thinking is also made visible through regular

418

W.-M. Hoe and I. Tan / Using IT to Power and Support Problem-Based Engaged Learning

reflections. In PBL lessons conducted in JSS, students work in groups of four to five to solve problem scenarios crafted by teachers. They go through the process of defining the problems, analyzing them, generating ideas or hypotheses, identifying the learning issues, independent and collaborative problem solving and integrating of new knowledge. Finally they come up with solutions, present them and then evaluate them with other students and the teachers. The school has adopted PBL in the development of the curriculum for a talent management program, and has found that the authenticity and challenge posed by ill-structured problems maintained student motivation and stimulated creative and divergent thinking [2].

2. Use of Technology to Power Learning: Developing an IT Learnscape 2.1 Robotics Invention System The school believes in empowering students with the tools for learning. One such important tool is technology. Instead of buying and installing technology simply to keep in trend, what is unique about technology implementation in JSS is that it is strongly driven by pedagogy. For example, the use of LEGO® MindstormsTM, an IT robotics invention system, is adopted to create an authentic learning environment where students learn in a constructionist manner. In the problem posed to the students, the impetus to study mathematics and physics was derived from the desire to build a model that would perform specific tasks. At the same time, practical activities and the use of multi-media appealed to the kinesthetic and visual intelligences of the students to fuel their learning [3]. 2.2 Developing an IT Learnscape The choice of student-centered pedagogies to bring about engaged learning provides teachers with the impetus to use IT in their teaching. Various developments are currently taking place to transform the entire school into a "learnscape". Building on the use of PBL in teaching, the school is constructing two PBL studios for stimulating cooperative learning and interactivity. Appropriate technology that could be incorporated in these learning studios include: Smart boards (Interactive white boards), document cameras, projectors, wireless internet connection, classroom audio system, and multi-media audio-visual cart (for ease of movement). 2.3 Information On-Demand - Wireless Connection and Online Learning Management System To support self-directed learning and research, which is a key feature of PBL, information has to be available on-demand. The school has installed a wireless infrastructure so that pupils could be connected to the internet as and when required. Students are able to research for information any time during lesson. With an online learning management system (LMS), teachers post online lesson scenarios and assignments, which students are able to access at any stage of the problem. Submission of some student assignments is also done online; these include the digital records of group discussions and brainstorming. In addition, with the LMS, students may continue their learning from home even when they cannot attend school. Teachers further make use of the LMS to gather feedback about their teaching through online surveys; this enables teachers to adapt their instructional methods according to the needs and learning progress of their students.

W.-M. Hoe and I. Tan / Using IT to Power and Support Problem-Based Engaged Learning

419

2.4 Digital Record of Learning Process - Use of Tablet PCs The students' process of knowledge construction is as important as the content knowledge finally gained. The use of tablet PCs in the process of solving the problem tasks presented to the students is highly effective in documenting the learning progress of the students. In the PBL lessons, students actively employ the inking function of the tablet PCs to brainstorm and revisit their problem as they move on to different stages of problem solving. All the rich information of brainstorming and planning is digitally captured to enable both students and teachers to monitor the learning progress. In addition, students make use of the inking function for the sketching of pictures, diagrams and maps which is particularly relevant in many subjects, for example, as students study the inter-relationships between organisms in ecology. Networking and collaboration amongst students are also enhanced as discussions can be parked online. This helps to facilitate deep understanding as learners can share different perspectives on the same issue. The tablet PCs improve knowledge management as documentation of learning is easily captured and students maintain an e-portfolio of not just their learning outcomes but also the learning process. Finally, the digital records captured in the students' tablet PCs are used as part of formative assessment. 2.5 Real-Time Data Collection and Analysis - Use of Data-loggers and Live Web Cam Data-loggers are also used to facilitate data collection and analysis, such as in examining water samples in biochemistry, and to provide real time display of data through interfaces with the tablet PCs, which helps students in the extrapolation and processing of information. Information obtained in the field can be fed to other students via wireless connection so that students in the classroom may benefit from real time acquisition of data. The medium-sized tablet PCs are relatively light-weight to allow students to carry them around with ease. The school is exploring the use of live web cam so that pictures and video clips could be fed from the field back to the classroom, redefining the boundaries of the classroom.

420

W.-M. Hoe and I. Tan / Using IT to Power and Support Problem-Based Engaged Learning

3. Enhancing Intercultural Understanding and Global Perspective By virtue of Singapore being a multi-racial society, the PBL cycle encourages intercultural understanding as students work in groups and are presented with multiple perspectives. This enriches their learning, which is deepened as knowledge is constructed after evaluating and assimilating information from different points of view. The tablet PCs and online learning management system facilitate the collaboration and discussion process. Ideas and views are not limited within the classroom. JSS is opening her doors to the community, who are our partners in education, as members of the community use the facilities of the school and contribute to the richness of students' learning. The school has also begun initial tie-ups with foreign schools, and would be arranging for overseas immersion programs in China and possibly Indonesia and Malaysia. As learning takes on an international stage, intercultural understanding will truly be enhanced through IT-enabled learning.

Conclusion JSS is one of the first six secondary schools selected by the Singapore Ministry of Education (MOE) to develop a school-based science curriculum using PBL as the pedagogical approach, as well as a pilot school funded by MOE to re-design the physical landscape for more flexible usage (under the Flexible School Infrastructure Scheme). The school is also awarded funding by MOE to embark on its IT learning program (under the Lead@IT Scheme). Work is still in progress and we are confident that using IT to power and support problem-based engaged learning would greatly enhance students' learning in an era of globalization.

Acknowledgments We thank the Singapore Ministry of Education for supporting the school projects, as well as all staff and partners of Jurong Secondary School for making engaged, authentic and meaningful learning possible for our students.

References [1] Tan, O.S. (2003). Problem-based learning innovation: Using problems to power learning in the 21st century. Singapore: Thomson Learning. [2] Tham, Y.C., Hoe, W.M. & Tan, O.S. (2005). Singapore's leading STAR. Snapshots (Secondary Edition): The Specialist Schools and Academies Trust Journal of Innovation in Education, 3(3), 13-17. [3] Hoe, W.M. & Lau, W.C.A. (2006). Innovative problem-based learning and scientific inquiry. Paper presented at ERAS Conference 2006 held in Singapore.

IT at School & Teacher Training

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

423

A Study of Innovative Uses of ICT in Primary Education a&b

Shelley S.-C. Younga, Hsin- Ho, Kub Graduate Institute of Information Systems and Applications, National Tsing Hua University, Taiwan [email protected]

Abstract: The purpose of this study was to explore the innovative uses of ICT in the Taiwan Schools Cyberfair as a means to enhance student learning in an extra curricular setting. Four winning groups in the Taiwan Schools Cyberfair, containing forty-eight subjects, were purposefully selected for study. Significantly, three innovative collaborative learning models for applying ICTs in Project-based learning (PBL) mode have been identified. In addition, some specific issues relating to PBL are discussed in the conclusion for reference. Keywords: Technology-Facilitated Learning, Project-Based Learning (PBL), Online Learning, Collaborative Learning.

Introduction Applying the Internet to facilitate teaching and learning has now become a welcome instructional practice in many schools. Flexibility and variety can be added to formal or informal learning settings[1]. The flexible, highly-interactive multimedia learning tools enable synchronous, asynchronous, individual or group learning in a virtual classroom on the Internet [2]. Effective incorporation of ICTs in enhancing learning and improving teaching to better primary education has become a pressing issue drawing researcher concern nowadays [3]. Alternative learning through the Internet has become a trend in education. 1. Background of This Study--Taiwan School Cyberfair The “Taiwan School Cyberfair” originated from the “International School Cyberfair,” and was formally authorized in 2000. Since 2000, the contest has been held once a year to promote information education, reduce the digital divide, encourage innovative teaching practices, facilitate cultural exchange and broaden students’ international awareness. The official website provides students with system-support helpful to their projects. Students desiring to join the contest are required to sign up online, so that they become formal competitors after receiving validation from the organizers. The website contains contact information, databases storing all previous works, and a quest system which allows groups to submit work and learn from each other. In addition, group workshops, a chat room and a discussion board enable students to interact and exchange information with each other. In this interactive online environment, students can learn to actively pursue and build knowledge.

424

S.S.-C. Young and H.-H. Ku / A Study of Innovative Uses of ICT in Primary Education

2. Literature Review Active diffusion of information technology through investments in national information infrastructures and education has become a world trend [3]. Taiwan is no exception to this global phenomenon. For example, effective use of information technology in improving learning and teaching to reform education and to increase manpower has become a pressing issue [4][5]. In Taiwan the traditional goal of education and the prevailing social value on obtaining a high qualification continue to dominate the core practices of schooling [4][5][6][7]. The use of ICT in teaching practices in schools is relatively still marginal. Most of time the positive effects of applying IT yield from the specially-funded experiments in certain given periods of time. Those teachers who were early adaptors to technology were more likely to adapt to the change by taking on innovative projects such as providing additional, alternative options for students [5]. For instance, teachers encouraged their students to use their time outside of regular classes to participate in some Web-based related activities, such as a School Fair contest. The alternative instructional practices explore the potential of the ICT with possible pedagogical and learning strategies such as PBL and cooperative learning, but are generally part of school’s extra-curriculum. Project-based Learning (PBL) is a class-oriented learning approach. Unlike traditional learning which is short-term and mostly constrained in the classroom settings, subject independent and teacher-focused, PBL involves long-term, theme-based learning and student-centered activities that focus on daily life problems. When conducting PBL, teachers encourage students to choose topics of their own interest, and set specific questions to look for answers in a well-planned proposal. Students can learn and find solutions during the research process. Teachers actively supervise the students in the role of project facilitator instead of controlling them [8]. Teachers instruct the students not only as a source of knowledge, but also as co-learners in their activities. In the PBL environment, the teachers are no longer the center of learning. Rather, students design their own activities and answer driving questions. Through the process of question-raising, cooperation, data-collection, communication and result-demonstration, a highly engaging atmosphere and rich learning environment that focuses on students is thus created [9]. Moreover, a teacher in the PBL environment is not only a leader in the classroom but also doing the most complete and important job. S/He is not only a course designer or an assistant in learning activities but also a judge to evaluate learning effects [10]. In PBL, by asking questions, cooperating with others, analyzing data and communicating with each other, students can create a student-centered learning environment or learning community [11]. Project-based collaborative learning uses ICT to arouse students’ motivation to learn, impelling them to actively participate in online discussions and deep research [12]. They collaboratively investigate questions of daily life and work with peers or teachers for the solutions [12]. Although the contest events draw much participation and attention nationwide, not much related research work has yet been conducted on this basis. Therefore, through in-depth study of the four representative elementary schools that have performed outstandingly in the contest, we hope to shed light on the innovative uses of ICT with PBL and try to identify the collaborative models employed in the given extra-curricular educational settings supported by the ICT. 3. Research Questions In order to explore the potential and innovative uses of ICT in primary education for alternative learning opportunities, specific questions explored in the study include the following:

S.S.-C. Young and H.-H. Ku / A Study of Innovative Uses of ICT in Primary Education

425

Since participating in the School Cyberfair contest requires group commitment and long-time (at least 6 months) engagement, what are the possible student motivations in being actively involved in the Web-based project contest? What might be the cooperative mode between students and teachers in producing web projects based on the PBL activities with the support of information technology? What are the students’ learning results in this kind of extra-curricular learning setting?

4. Methodology The methodology employed in this study is primarily a qualitative approach. In addition, quantitative data including student demographic information is also included. The researchers conducted individual and group interviews to the students and the teachers at school sites. Online questionnaires requested personal and background details of the teachers and students (demographical data, such as sex, age, computer skills, and Internet use) were administered to the participants at the beginning of the study. In addition, observations were conducted intensively on the Internet, focusing on subjects’ online PBL learning activities, including the collection of their electronic data from journals, chats, discussions, and email exchanges on the Internet. The research was also concerned with how the students collectively designed and presented their Web materials and how they interacted with the distributed students. Triangulation was used to improve the probability that findings and interpretations found would be reliable. 4.1 Data Collection The data-collecting methods and tools for this study included a questionnaire survey, telephone interviews, and online data-collection, activity observations, and quantitative data analysis of the web projects produced by each group. 4.2 Duration of the Study The study lasted for about a year and a half, from September 2003 to February 2005, including three major phases: phase 1. Preparation and orientation: September 2003 to January 2004: participants’ school learning activities in preparation for the Cyberfair contest; 2. Participation and collaboration: February 2004 to July 2004: Participants’ collective effort in the Web project, online journal keeping; and 3. Follow-up data collection: August 2004 to February 2005: follow up interviews with the students and teachers at school sites. 5. Results and Discussion For a better understanding of the participants and the 4 school cases involved in this study, we first focus on the participants’ profiles and then address the research questions, including looking at students’ motivations in participating in the PBL Web project contest; three Innovative PBL collaborative models enabled by ICT; learning effects of PBL activities; and analysis of applications of ICT in project-based learning.

426

S.S.-C. Young and H.-H. Ku / A Study of Innovative Uses of ICT in Primary Education

5.1 Participants and 4 School Cases The participants for this study were selected according to their ethnic origins, regions, contest themes and willingness of participation. Eventually, the four outstanding groups, including forty-eight subjects (36 students and 12 teachers), were chosen from four schools from different parts of Taiwan, representing eastern, southern, rural and urban areas. They participated in the theme of “Local tourist resources for elementary school level.” Their schools and locations are listed in table 1. For their privacy and confidentiality the information has been coded anonymously. Table 1. Basic information for the participants School Code Location Students Teachers Total 1 School B Northern mountain area 10 3 13 2 School H Northern city area 8 3 11 3 School U Central rural area 10 3 13 4 School K Off-shore remote island 8 3 11 Total 36 12 48 Each of the groups had 8 to10 students with 3 teachers as advisors (Table 1). The teachers in this study consisted of 9 males and 3 females. Their age range was 25 to 46 with a mean of 29.58 years old. They created a PBL learning environment for students to make their own Web project. Among the four schools, two of them (School B and School H) were mini schools where the student numbers ranged from 65 to 134 students. School B is in Fu-xing village in a remote mountainous area in northern Taiwan. Most of its students are aboriginals. School H is located at Dan-shui town, Taipei County. School U is in Yong-jing village, Chaung-hua County, in the central part of Taiwan. School K is in Lie-yu village, Jin-men County, on an off-shore remote island. 5.2 Student motivations in Participating in the PBL Web Project Contest According to the data collected from the student questionnaires, interviews and observations, their motivations to be engaged long term in the PBL Web project contest held by the Cyberfair can be categorised into 5 types: 5.2.1 Personal interest According to our analysis of the questionnaires, almost all of the students found that the PBL scenario combining teaching and outdoor activities in the real world very “interesting” and “fun”. (B04, B05, B06, B07, H01, H04, H06, H08, U04, U05, U09, U10, K02, & K05)” The students (U01 & H03) were also curious about the way of doing project research which was not common in regular lecture-based teaching settings. Their learning interests were intrigued and they were excited to take part in related learning activities outside of school during their leisure time. Data from the questionnaires revealed that the students gave overwhelmingly positive feedback to PBL combining the networked system offered by the contest organizer, and regarded that they had gained a lot through the PBL activities. The following comment reflects this sentiment: “I found that this activity is great, even if I didn’t win the prize…Learning how to make use of the Internet correctly and knowing more about computer technology made our life rich and interesting! (H05)”

S.S.-C. Young and H.-H. Ku / A Study of Innovative Uses of ICT in Primary Education

427

5.2.2 Gaining situated knowledge and skills Students got involved in authentic affairs in the real-world and outdoor activities through PBL activities. In conducting a PBL Web project, they had to collectively make a plan in advance, allocate individual work, find supporting resources, go out to collect documents, take pictures and interview people using digital cameras and recorders, etc. They applied their newly acquired IT skills and knowledge in real-life situations. The students felt that they learned more knowledge and skills from the extracurricular activities (B09, B10, H01, H04, H05, H06, U02, U03, U04, U05, U07, K01& K06) [13]. 5.2.3 Winning awards The main motivation for most of the students to participate in this project making activity was to win a prize. Some students said that if they won, they felt very proud because they could “receive the awards and medals in public (U05 & H05).” The goal of achieving a high placing intrigues the students’ inner sense of honor, impelling students to actively participate in the contest (U04, U05, H05, K05 & K07). 5.2.4 Encouragement from teachers or schools There are two kinds of encouragement: a. teachers’ encouragement to students; and b. the encouragement from school administrative officials. Teachers’ encouragement. Teachers in the 4 schools actively asked their students to take part in the contest and pushed them to make the projects with their additional support during out-of-class time. Encouragement from the school. For example, the principal of School B has encouraged students to participate in the contest since 2000. At Schools H, U, and K, the school officials offered a lot of assistance in obtaining software and hardware as well as providing other resources for the students. 5.3 Three PBL Collaborative Models Enabled by ICT: Scgp; DcgSp & Dcgp During the PBL process, it came to our attention that the students in the 4 schools had the flexibility to break the class or grade constraints by expanding their cooperation beyond their own classes. Utilization of the information networking technology meant that students could form their project group transcending the limitations of classes, grades, space and time – something which was otherwise not possible before the advent of ICT. Excitedly, they communicated, shared information and sent digital files on the Internet. These kinds of cross-class or cross-grade cooperative groups gave students more stimulus and experiences, allowing students to create different learning environments in the given situations. Table 2. Three PBL Collaborative Models Enabled by ICT Models Ways of collaboration Representing schools 1 Scgp Collaboration with classmates School U & K (Same class, grade & project) 2 DcgSp Collaboration with different grades School B (Different class, grade, Same project) 3 Dcgp Collaboration with different grades & School H projects (Different class, grade & project)

428

S.S.-C. Young and H.-H. Ku / A Study of Innovative Uses of ICT in Primary Education

By analyzing the make-up of each group, the observed data and student ways of collaboration, we could induce three collaborative models: (1) Scgp Model (Same class, grade & project)˗(2) DcgSp Model (Different class, grade, Same project) and, (3) Dcgp Model (Different class, grade & project (see Table 2). 5.3.1 Scgp Model (Same class, grade & project) The groups in school U and K belonged to the same class. Three teachers helped organize group members to produce the project together after school. The members communicated and shared individual results through e-mail and networked file-transferring software. For those teachers, the Internet became an important tool to monitor student progress and give students’ advice on time management. This model is common nowadays in school settings, which makes it easier for teachers to teach and for students to interact with each other. 5.3.2 DcgSp Model (Different class, grade, Same project) The working model of school B showed that the leading advisor was a teacher of fifth grade. After discussing with students, the advisor led the group to initiate their project and conduct interviews. A sixth grade teacher was involved in this project, too. Three sixth grade students were invited to work with them. Therefore, the project was made by a group of fifth-grade and six-grade students who made concessions in their leisure time to work on this project. In this case, the students were tied closely in this project, having incredibly great interaction. 5.3.3 Dcgp Model (Different class, grade & project) The collaborative Dcgp model of school H was a cross-grade integration. Because of the great promotion of the competition by the school, there were up to four groups participating in the Web project contest. The group in our study was composed of fifth-grade students, while the other three groups were formed by sixth graders. Though the project title of each group was different, the content was all centered on the natural and cultural neighborhood of the school, which resulted in repeating the same interviewed objects. Thus the data collected by different groups could be integrated and shared. Juniors and senior students could exchange resources, share information and learn techniques by e-mail and FTPs. The model made the process more competitive, and enabled the students to be actively involved in their project (HT1). The researchers noticed that the project participants of schools B and H went through all kinds of cross-grade learning activities. Both teachers and students had to deal with the difficulties of time, space, student proficiency and age difference. However, the PBL learning environment that combined the applications of networked technology made it possible. Furthermore, in addition to the teachers, the senior students also helped the younger ones with their learning. The students could upgrade their ability by stimulating or challenging each other. The new cooperative models shed light on innovative approaches of learning and teaching with the integration of ICT in traditional educational settings.

S.S.-C. Young and H.-H. Ku / A Study of Innovative Uses of ICT in Primary Education

429

5.4 Learning Effects of PBL Activities Based on the questionnaires, students’ online journals and interviews with the teachers, the learning effects of PBL Web project producing activities could be concluded as follows: Gaining more communicative skills through group collaboration and physical and online interactions: students assumed different jobs according to individual abilities and IT skills. Outside of their schools, they learned how to interact well with the interviewees in order to collect data required for the project. They looked for help from the community and gained more communicative skills. Ultimately, they acquired respect from each other through the information exchange with the other groups online. Motivation as a powerful learning booster: The interview data and our observations indicated that the Web-project contest in real-world situations, extra-curricular PBL learning and coordinated interaction with the community promoted strong learning motivation from the students. Because the theme of the Web project matched the students’ interests, learning eventually became more interesting and effective. For instance, the students knew that the purpose of producing the Web project could help promote tourism of their hometown, so their honor and emotions were aroused. Therefore, students worked hard during the process and won the award that was a high recognition of their endeavors. 6. Conclusion According to our research questions, we make the following conclusions based on our analysis of data collection and study information. Students are seriously engaged in the PBL activities, because their learning motivations were highly activated by the real-world situations. The design of these PBL activities was to combine daily life situations with learning motivations to construct students’ project-based knowledge in order to answer the questions raised in each of their projects. Thus the students could follow their directions and accomplish their established project goals. Three collaborative learning models would inspire flexible learning activities across grades in current educational system. Three collaborative models have been identified in this study. In addition to the Scgp model that has been practiced in current teaching within one class, the other two (DcgSp model and Dcgp model) created new teaching practices and learning opportunities in primary education that fully utilized the Internet and ICT to carry out PBL activities across classes and grades which could not have otherwise been accomplished without the appropriate integration of the information technology. The identified models could serve as a good reference for educators for their teaching practices in order to get students involved in active usage of IT for authentic situations and fulfill student’s competences and IT skills in mixed groups. Nowadays, with the pervasiveness of networking computers in schools and the abundant digitized resources on the Internet, students should be made aware of their role in how to become active learners, and be given more opportunities such as practicing projector theme-based learning that focuses on daily life problems. Furthermore, they can satisfy their learning interests by themselves through actively acquiring and synthesizing information on the resourceful Internet, when provided with well-designed courseware to support different types of PBL models in settings both inside and outside of the classroom.

430

S.S.-C. Young and H.-H. Ku / A Study of Innovative Uses of ICT in Primary Education

Acknowledgments This project is funded by the National Science Council of Taiwan under the research project code NSC93-2520-S-007-001. References [1] Young, S. S. C. (2000). A long-term study of a series of web course trainings. Proceedings of the Global Chinese Conference on Computer in Education (GCCCE 2000), 916-923, Singapore, 05.29-31, 2000. [2] Young, S. S. C. (2000). Internet-supported Instruction in the New Millennium: The Implementation of the A.L.I.C.E. System. Instructional Technology & Media, 50, 2-11. [3] Halverson, R., & Collins, A. (2006, in press) Information technologies and the future of schooling in the United States, Journal of Research and Practice in Technology Enhanced Learning (RPTEL), APSCE. [4] Young, S. S. C. (2001). Diffusion of Information Technology in Education: Issues, Concerns and Implication for Educators Informed by Research, SITE 2001, Orlando, Florida, USA, 03. 5-10, 2001. (Invited talk) [5] Young, S. S. C. (2006, in press). Diffusion of information technology for education in Taiwan: Reflections and concerns in response to Halverson and Collins' paper, Journal of Research and Practice in Technology Enhanced Learning (RPTEL), APSCE [6] Young, S. S. C. (1999a). A Study of Project-based Training Program for Teachers. Proceedings of the Global Chinese Conference on Computer in Education (GCCCE ‘99), 501-507, Macau, 06.07-09, 1999. [7] Young, S. S.C. (1999b) A study of a Web-related Course Training and Its Diffusion. Chinese Journal of Science Education),V7(4),299-314, Taipei, Taiwan (in Chinese). [8] Thomas, J. W. (2000). A review of research of project-based learning. [On-line], Available: http://www.k12reform.org/foundation/pbl/research/ (2003/6/3) [9] Marx, R. W., Blumenfeld, P. C., Krajcik, J.S., Blunk, M., Crawford, B., Kelly, B., & Meyer,K. M.(1997). Enacting Project-based science, The Elementary School Journal, Vol. 97, 341-358. [10] Delisle, R. (1997). How to use problem-based learning in the classroom. Association for Supervion and Curriculum Development. [11] Blumenfeld, P. C., Soloway, E., Marx, R. W., Krajcik, J. S., Guzdial,M. & Palincsar, A. (1991). Motivating project-based learning: sustaining the doing, supporting the learning. Educational Psychologist, 26(3&4), 369-398. [12] Polman, J., & Fishman, B. (1995). Electronic communication tools in the classroom: Student and environmental characteristic as predictors of adoption. Paper presented at the Annual Meeting of the American Educational Research Association. San Francisco, CA. [13] Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

431

Impacts of Grade 7-9 Students’ Computer Usage after School on Academic Achievement: A School Case Study Yiu Fai WONG, Percy Lai Yin KWOK Chinese University of Hong Kong, Hong Kong, China [email protected] Abstract: Explorable areas concerning actual impacts of computer usage on various types of related activities after school on student learning, and relationship between time spent on those activities and student personality styles (in terms of social and intellectual selves and learning attitudes) have still been under-researched in local and international perspectives. This study aims to fill up these research gaps by studying 528 Secondary 1-3 (Grade 7-9) students, aged from 12 to 16, in a secondary school in Hong Kong in terms of a questionnaire to record their activities on using computers after school and to identify their personality styles in Hong Kong. By taking mid-term school examination results for granted, the study not only reveals correlation between examination marks and time using computers, but also depicts variation of their participating activities on using computers. Keywords: Computer usage, digital divide, youth studies

Introduction Education and Manpower Bureau of the Government of Hong Kong Special Administrative Region has introduced Information Technology (IT) in education since 1998 and set up a fiveyear strategy for the revolutionary development of IT in education in primary and secondary schools in Hong Kong (EMB, 1998). Notably, the penetration rate of personal computers and the Internet was very high. 91.3% of students had computer at home, and 92.6% of them had gained proper access to the Internet. Within these 5 years, over 20,000 digital curriculum resources and materials have developed. Most of them are available for sharing online (EMB, 2004). Generally speaking, there were annual increases in the number of households with PC at home (6.95%) and the number of households connected to Internet (10.05%) from 2003 to 2004. Notably, there were increases in the time spent in using PC on various online activities. However, there were decreases in both percentages of persons and the average time that spent on using PC on digital entertainment, according to a household survey (Census & Statistics Department, 2004). 1. Literature Review Several new forms of learning have been developed with the help of IT, such as WebCT, Web Quest, and other online distance learning systems in Hong Kong such as Read Daily developed by the Chinese University of Hong Kong and SMILE (System for Multimedia Integrated Learning) developed by Hong Kong Polytechnic University. As a result, the time and space of learning have been extended from schools to student home. Besides constructivism theories, IT has been found to play an essential role in lots of different learning theories that can be used to help guide a teaching / learning process (Tom, 1997). When students use computers at home,

432

Y.F. Wong and P.L.Y. Kwok / Impacts of Grade 7-9 Students’ Computer Usage After School

most likely they are working on their own without the supervision of a teacher and a wellplanned learning strategy. Even though they are not undergoing a process of teaching, cognitive growth may still take place as information seeking (one in a total of four modes of organizational scanning, argued by Chun et al., 1998) is also a cognitive process (Kalervo & Wilson, 2003). Several local research bodies (Cheuk, et al., 2001; CITE, 2001) simultaneously report that as computers and the Internet become more popular in Hong Kong, students spend more time on using computers in their leisure time. However, this does not necessarily entail that the students get better academic benefits from using computers. Coleman (2004) also shows that those youngsters become online addicts spending at least 5.5 hours daily online on average and spend most of their leisure time to play online games. They are poor in personal relationship and they lack sense of achievement and a value of life. About 110 thousand (aged from 15 to 19) youngsters are "hidden" from the society. They only stay at home to play online games for several months, without any intention to contact anybody, to go to school or get a proper job, according to a recent study on youth in Hong Kong (Cheuk, et al., 2001). The fact is that most students prefer to use computers in entertainment rather than use them as a learning tool. Previous pieces of research literature attempt to study the effects of using computer in a specific teaching area or teaching method, without considering the habits of how students used computer. Other past studies merely focus on how students use computers and show that they mainly use it on electronic games and ICQ. All such research bodies show that using computers via those means does not help students learn or sometimes even cause bad learning effects. Yet the current study aims to investigate various effects of various learning activities on using computers, and their other effects on particular Key Learning Areas (KLAs) of new secondary school curricula in Hong Kong, instead of considering overall results. Lower form students, aged from 12 to 16, are subject to a critical personality development stage (Sigel & Irving, 1968). They may confuse self-values and even self-identity when they interact with others (Erikson, 1971). At this stage, they are most affected by their emotions, feelings and peers. Their personality also affects their point of views on life, academic studies and the choice of living style, which in turn results in different choices of activities when they use computers. This current study endeavors to find out how styles of using computers affect students’ social self-concept, intellectual self-concept and learning attitudes. Digital divide refers to the gaps between individuals at different social-economic levels when they are provided with an access to IT and their uses of the Internet (HKSAR, 2005). Previous studies on digital divide reminded us that an access to ICT not only meant possession of the necessary computers, software and connections, but also basic skills of using them and actual usage of these resources (Alfred, 1998; Don, 1997; van Dijk, 1997). Past research literature also found that television-viewing could widen the educational gap between the more able and the less able students (Winn, 2002) and those who surfed on the web probably read more information (Joseph & Andrea, 2003). As most students currently have computers at home and IT in Education has implemented in Hong Kong for several years, digital divide apparently seems not to be a great problem for Hong Kong students (Education and Manpower Bureau, 2004). Nevertheless, openness, conscientiousness, extraversion, agreeableness and neuroticism of a student determine what he/she likes and what he/she avoids (Shelley, 2005). If the personality of a student does affect the way he/she use a computer, and that may result in the intention and opportunities of accessing information in leisure activities (Li et al., 1999). This may cause individual disparities in mental development. A recent cross-national study, involving one hundred thousand 15-year-old students from 21 countries in the 2000 Programme for International Student Assessment (Clare, 2005), has revealed that the more

Y.F. Wong and P.L.Y. Kwok / Impacts of Grade 7-9 Students’ Computer Usage After School

433

pupils used computers, the worse they performed in reading, writing and arithmetic. Despite no concrete evidence for improving academic results using computers and the fact that students mainly use computers for entertainment in Hong Kong (Cheuk et al., 2001), the effects of using computers should not be overlooked. It is because information technology has been proved to be an effective learning and teaching media (Exploring IT in Education, 2002). Those students with greater access to computers are potentially getting a greater chance of gaining access to relevant information, and thereby gaining more benefits in enriching their knowledge, compared to those who do not use computers. 2. Research directions The target group of the study was junior Secondary 1-3 (Grade 7-9) students, aged from 12 to 16 in a school. They were growing up in line with the Five-year Strategy of IT in Education in Hong Kong from 1998 to 2003. They were the first generations in Hong Kong to implement I.T in their learning and have computers being widely used at home. Moreover, the junior secondary curricula were quite similar, removing side effects on different streaming and different subject electives. Since the pilot test did not give a significant result on the various effects of using computers, all students in Grade 7 to Grade 9 were taken as the sample. There were totally 528 students. Most studying subjects were living in nearby public estates whilst others in private houses and rural villages. They often stayed there after school and during public holidays. Most of students had installed computers at home. However, most of their parents held negative opinions against their use of computers, for fear of their children spending too much time on playing computer games or using ICQ. Some parents even did not allow their children to use computers after school. In the survey, 512 students had computers at home and 11 did not have. 486 (92%) students had broadband installed at home and 42 (8%) did not have. 407 (70%) parents held positive attitudes towards their children using computers, 105 (20%) parents held negative ones and the remaining (10%) held no opinions. Locations to use computers were found in the range: school (28%); students’ own home (91%), friends’ home (21%), cyber café (13%), public library (8%) and shopping mail (1%). Main research questions focused on whether there was: 1. correlation between total time on using computers and academic results 2. correlation between personality and time spent on various computer activities 3. correlation between personality and academic results 4. a complete regression model connecting all correlations According to EMB, the existing subjects are grouped under eight key learning areas (KLAs), namely, English Language Education, Chinese Language Education, Mathematics Education, Science Education, Personal, Social and Humanities Education, Arts Education. and Physical Education. The studies of Society and Environment score was obtained from Geography and Chinese History. The Technology score was derived by combining those from Graphical Communication, Business Fundamental, Computer Literacy, Design & Technology and Home Economics. In Arts Education, the Arts scores in the study combined the scores of Art and Design, and Music Education. In the study, activities on using computers (based on pilot tests and previous research literature like Tsang et al., 2002) included playing computer games, enjoying audio visual entertainment, doing homework, communication, web browsing, searching information, entering a forum, self-learning and doing creations. The notion of students' social self concept is obtained from ‘Assessment Program for Affective and Social Outcomes (APASO)’. EMB has delivered APASO toolkit to schools since 2003 to provide a self-assessment platform to

434

Y.F. Wong and P.L.Y. Kwok / Impacts of Grade 7-9 Students’ Computer Usage After School

primary and secondary schools. Schools have been highly encouraged to use the tool to measure the performance of the students in various social aspects. APASO makes use of Chinese Adolescent Self-Esteem Scale (CASES) to evaluate students' self concept. Self concept is subdivided into 6 categories: overall self concept, social self concept, intellectual self concept, appearance self concept, ethic self concept and family self concept. There were ten survey questions on measurable social self concept and ten more on intellectual self in the research (c.f. table 1). Table 1. Ten parameters for measuring social self and intellectual self Social Self I can get alone well with others easily I lack the popular traits most people have I have lots of friends. I can’t express myself well I am skillful at getting along with people. I am not skillful at communicating with people My friends like to talk with me. Many people like to have transactions with me. I can mix well with others. I am welcome by most of my classmates.

Intellectual Self Quite often I suspect my intellectual ability I am not confident in my academic performance. In fact I am not as smart as others I am intelligent in my schoolwork. I have good memory. I work well with my plan. I am happy with my school performance. I work hard to get good school grades. I can't improve my studies I have the will of progressing.

The part concerning attitudes to learning was adopted form the Learning Process Questionnaire (Biggs & Moore, 1993) which has been widely used as a tool to measure the inspiration and attitude towards learning in many countries and a variety of cultural backgrounds. The survey questions on deep learning strategy were selected in this study as follows:

While I am studying, I try to think of how useful the material that I am learning would be in real life. In reading new material, I am reminded of things I already know, and see them in a new light. I like to do enough work on a topic to form my own point of view before I am satisfied. I try to relate what I learn in one subject to what I have learned in other subjects. I find most new topics interesting and spend extra time trying to find out more about them. I spend much of my free time finding out more about interesting topics which have been discussed in different classes.

3. Research Methodology An on-line questionnaire was set to collect the data during computer lesson. It consisted of 90 questions, with a 4-point or 6-point Likert scale. Survey data was then collected by an online worksheet, on which students only needed to click the answers with a mouse. The data on the questionnaire was sent to a central database file (with no transcription errors) before a sophisticated quantitative data analysis was conducted via SPSS (version 10). 4. Results 4.1 Relationships between total time using computers and academic results in the 8 KLAs The mean average score was 55.77. The mean times on various activities such as computer games, AV, homework, communication, web-browsing, information searching, forum, selflearning and creation were 3.09, 2.90, 1.80, 3.03, 2.65, 1.95, 2.12, 1.36 and 1.46 respectively (counted as "hours in 7 days"). Pearson Correlations between average score and those activities were -0.156 (p=0.000 sig.), -0.120 (p=0.000 sig.), 0.125 (p=0.002 sig.), -0.118 (p=0.004 sig.), 0.20 (p=0.330 not sig.), 0.013 (p=0.389 not sig.), -0.076 (p=0.044 sig.), 0.039 (p=0.192 not sig.)

Y.F. Wong and P.L.Y. Kwok / Impacts of Grade 7-9 Students’ Computer Usage After School

435

and 0.043 (p=0.165 not sig.) respectively. And the overall effect of these activities on average score was significant: R=0.252 (F=3.766, p=0.000 sig.). Form the above results, H0 (no correlation) is rejected (with p<-.05). There were correlations between the average score and the activities on using computers. The standardized regression model can be further given by: Average score = - 0.135 x (Time on computer games) – 0.119 x (Time on entertainment) + 0.137 x (Time on homework) – 0.064 x (Time on communication) – 0.051 x (Time on forum) + ei

Spending times on computer games, entertainment, communication and forum were found to have significant negative effects on the average score. On the other hand, using computer to do homework would promote academic results. 4.2 Relationships between personality and time spent on various computer activities Personality measured in terms of social self concept, intellectual self concept and attitude to learning had significant relationship with times spent on various activities of using computer. Different personality of students would choose different activities when using a computer. The regression coefficients of the three personality aspects correlated to total time on using computers can be summarized in table 2. Table 2. Correlation between personality and time spent on using computers Item Social Self Intellectual Self Attitude to Learning

Pearson Correlation 0.080 -0.122

sig

R

t

sig

0.042* 0.004*

0.115 (b21)) -0.182 (b22)

2.332 -3.227

0.020* 0.001*

-0.012

0.396

0.048 (b23)

0.852

0.395

According to table 2, time on using computers could be expressed as: Total time on using computers= 0.115 x Social self – 0.182 x Intellectual Self + 0.048 x Attitude to learn + e2

Students with a higher score in social self tended to use more time on computers, whereas students with a higher score in intellectual self tend to use less time on computers. However, correlation of the attitude to learning to total time on using computers was not significant enough. 4.3 Relationships between personality and academic results Intellectual self concept and attitude to learning were found to be significantly correlated to academic results. Coefficient (R) between intellectual self concept and academic results was 0.437 (p=0.000 sig) whilst coefficient (R) between attitude to learning to academic results was 0.033 (p=0.515 not sig) in table 3. Table 3. Correlation between personality and academic results Factor Social self Concept Intellectual self Concept Attitude to learning

Pearson Correlation -0.021

Sig

R

t

Sig.

0.320

-0.161

-3.643

0.000

0.407

0.000

0.437

8.893

0.000

0.211

0.000

0.033

0.652

0.515

436

Y.F. Wong and P.L.Y. Kwok / Impacts of Grade 7-9 Students’ Computer Usage After School

Both intellectual self concept and attitude to learning had positive relationships with academic results. A higher status in the intellectual self concept would have better academic results. Having deep learning strategy would also get better academic results. Academic results could be further expressed in terms of a regression model: Academic results = -0.161 x Social self + 0.437 x Intellectual Self + 0.033 x Attitude to learn + e 4.4 Regression model By referring back to the path equations being defined earlier, the two equations can be presented as follows (c.f. figure 1): Academic results = - 0.161 x Social self + 0.437 x Intellectual self + 0.033 x Attitude to learn + 0.082 x Total time on using computers + e1 Total time on using computers = 0.115 x Social self – 0.182 x Intellectual self + 0.048 x Attitude to learn + e2

Figure 1. The finalized path analysis Average results = - 0.135 x (Time on computer games) – 0.119 x (Time on entertainment) + 0.137 x (Time on homework) – 0.064 x (Time on communication) – 0.051 x (Time on forum) + ei

Time on using computers could be further specified with time spent on the three activities with the most significant impacts on the examination results. And the path model was revised in figure 2:

Y.F. Wong and P.L.Y. Kwok / Impacts of Grade 7-9 Students’ Computer Usage After School

437

Figure 2. Path Analysis of three activities exerting significant impacts on academic results 5. Discussion and Implications 5.1 Impacts of total time on using computers on academic results It was found that there was a negative relationship between total time on using computers and academic results, indirectly implying that a student spending more time on computers might have lower examination scores. Yet such result could not necessarily entail that students used too much time on computer games, ICQ and audio-visual entertainment without academic return. The current study did not investigate qualitatively educational functions of various types of computer activities that student subjects participated in and there was no follow-up case study on how students spending time on computer activities would lead to low academic achievements. The crux of matter lies in how to help students to use computers as an information-seeking or knowledge-building tool for learning, instead of a video game machine or an online video phone for mere recreation. Under the theme of IT literacy, detailed guidance for students, teachers and parents should be established for meaningful and purposive learning. 5.2 Correlation between activities on using computer and academic results It was found that some of the activities using computers had benefits in some KLAs. Web browsing, audio-visual entertainment, online communication and searching information exerted positive impacts on learning Chinese and English. Even though the student subjects did not have an initiative in reading, they were required to read a lot when they used computers. They encountered much information when they browsed through the net, or when they searched for audio-visual materials. When they used online communication tools such as ICQ and online chat, they read fast and write fast, and might use Chinese or English, or bilingually for communication, and this was a cyber-world practice of using language in real situation. Therefore, we should not simply overstate relationships between time on using computers and the activities involved. Particular kind of activities might bring benefits to certain learning areas, whilst they might widen information gap among students of various attributes. 5.3 Personality and the choice of activities when using computers It was found that students of different personality chose different kind of activities when using computers. Variety of choices might imply that different activities would affect students. If

438

Y.F. Wong and P.L.Y. Kwok / Impacts of Grade 7-9 Students’ Computer Usage After School

students of certain characteristic select certain types of activities, it would be more likely for them to be benefited or harmed by those activities. We should notice that the choice of activities could have impacts on students' personality. 5.4 Digital divide among students Today, there are still a small number of students that do not have computers at home. Perhaps such students may use computers in schools or contact cyber-world through Internet café. It is worth investigating how computer or Internet usage exerts significant impacts on cognitive and psychological growths of schooling children. The information gap between them is not only limited to the information they get, but also extends to what kind of person they turn to be. References Alfred K. (1998). The electronic information gap. Retrieved on March 7, 2005, from http://www.ifla.org/VII/dg/srdg5.htm [2] Biggs, J. & Moore, P. J. (1993). The process of learning (3rd ed.). New York: Prentice-Hall. [3] Census & Statistics Department (2004). Thematic household survey report – Report no. 20. Retrieved on December 1, 2004, from http://www.info.gov.hk/censtatd/eng/hkstat/social_topics_index.html [4] Centre for Information Technology in Education (CITE) (2001). A study on the Influence of IT on Youth: Executive Summary. Hong Kong: Commission on Youth. Retrieved on June 30, 2005, from http://www.info.gov.hk/coy/eng/report/it.htm [5] Cheuk, H.Y. et al. (2001). Net Teenagers: Survey on secondary school students’ web-browsing experiences. Hong Kong: The Christian Family Service Centre Retrieved on March 3, 2005, from http://www.breakthrough.org.hk/ir/youthdatabank/mc/mc_02.htm [Chinese]. [6] Chun, W.C., Brian, D. & Don, T. (1998). A behavioral model of information seeking on the web: preliminary results of a study of how managers and IT specialists use the web. ASIS Annual Meeting Paper. Retrieved on November 14, 2004, from http://www.ischool.utexas.edu/~donturn/papers/asis98/asis98.html [7] Clare, John. (2005). Pupils make more progress in 3Rs without aid of computers. The Telegraph (U.K.), March 21, 2005. [8] Coleman, S. S. (2004). Internet addiction. Retrieved on March 25, 2005, from http://www.dioceseofmonterey.org/observer/sep04 [9] Don T. (1997). Growing up digital: The rise of the net generation. New York: McGraw-Hill. [10] Education and Manpower Bureau (EMB) (1998). Information Technology for Learning in a New Era FiveYear Strategy 1998/89 to 2002/03. Hong Kong: Education and Manpower Bureau. [11] Education and Manpower Bureau (EMB) (2004). Empowering Learning and Teaching with Information Technology. Hong Kong: Education and Manpower Bureau. [12] Erikson, E. H. (1971). Identity, youth, and crisis. USA: W.W. Norton & Company, Inc. [13] Exploring IT in Education (2002). The Journal of Quality School Education, 2. [14] The Government of Hong Kong Special Administrative Region (HKSAR) (2005). Building a Digitally Inclusive Society. Retrieved on Sept. 1, 2005, from http://www.info.gov.hk/digital21/eng/programme/digitaldivide.html [1]

ˮ˄ˈ˰ʳʳvan Dijk J. A.G.M. (1997). Widening information gaps and policies of prevention – 1: Advice to the Information Society Forum of the European Commission. Retrieved on March 1, 2005, from http://www.thechronicle.demon.co.uk/archive/infogap.htm [16] Joseph T. & Andrea L.K. (2003). The Wired Homestead. Cambridge: Massachusetts Institute of Technology. [17] Kalervo, J. & Wilson, T.D. (2003). On conceptual models for information seeking and retrieval research. Information Research, 9 (1). Retrieved on November 11, 2004, from http://www.infomationr.net/ir/91/paper163.html [18] Li, W. M., Chen, M. & Li, C.M. (1999). The personality attributes and leisure activities of the Internet users: A Taiwanese case study. Retrieved on January 1, 2005, from http://www.sba.muohio.edu/abas/1999/chenmiao.pdf. [19] Shelley W. (2005). The Big 5 Personality Traits. Retrieved on November 02, 2004, from: http://psychology.about.com/cs/personth/a/big5.htm. [20] Sigel & Irving E. (1968). Logical thinking in children: Research based on Piaget's theory. New York: Holt, Rinehart and Winston. [21] Tom R. (1997). Evaluating what really matters in computer-based education. Retrieved on October 1, 2004, from http://www.educationau.edu.au/archives/CP/REFS/reeves.htm [22] Winn, M. (2002). The plug-in drug: Television, computers, and family life, 25th Anniversary ed. New York: Penguin.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

439

Using Interactive Whiteboards (IWB) to Enhance Learning and Teaching in Hong Kong Schools Fong-lok LEE, Sai-wing PUN, Sandy Siu-cheung LI*, Siu-cheung KONG**, Wai-hung IP The Chinese University of Hong Kong, Hong Kong *Hong Kong Baptist University, Hong Kong **The Hong Kong Institute of Education, Hong Kong [email protected] Abstract: Interactive whiteboard (IWB) has been widely used as a learning tool in the western classrooms since 1991. Many researches reported that the technology benefits students by increasing their engagement in classroom activities, arousing learners’ motivation and helping with their knowledge retention [1]. Hong Kong, as a Chinese society has its unique learning culture and environment [2], what benefits western students may not apply there. It is therefore worthwhile to explore the usefulness of IWB in Hong Kong schools. This paper reports on the research of what benefits can bring to Hong Kong students through the teaching with IWB. Result of this study shows that in general teachers and students in Hong Kong welcome the use of IWB in their classrooms. Keywords: Interactive whiteboards (IWB), learning and teaching, Hong Kong

1. Background Interactive whiteboard (IWB) has been widely used as a whole class teaching tool for active learning in many western countries [3]. Researches in the U.S., the UK, Australia and Canada demonstrated that the use of IWB has positive influence in classroom atmospheres [1]. Teachers can share and re-use materials, save and print any notes made on the board [4, 7], which then helps saving time for interactions and group learning activities [3, 5, 6]. Thus, there are more opportunities for students to participate in class activities and collaborate [8]. In addition, IWB was found to be attractive to students. In 2002, Levy found that IWB could help teachers deliver more enjoyable and interesting lessons with more effective explanations, which might enhance students’ learning motivation and outcomes [8]. Moreover, the use of modeling, demonstrating, and annotating in teaching with IWB helped teachers to explain concepts or procedures, and to show problem solving work easily [8]. It was also found that students could better understand complex concepts as a result of clear, efficient, and dynamic presentation [6]. Although IWB brings lots of benefits to classroom teaching in many western countries, the teaching effect with IWB in Hong Kong is not yet known. Hong Kong, as a Chinese society and with massive class size in normal classrooms, has a different context from that of the western countries [2]. Because of such differences, a study of the educational value of IWB in the context of Hong Kong schools is therefore important before the device is used in a large scale. This paper reports the findings of such a study done in 2005.

440

F.-l. Lee et al. / Using Interactive Whiteboards (IWB) to Enhance Learning and Teaching

2. Research Design With the aims to study whether the use of IWB can promote classroom’s interactions, students’ learning motivation, and improve teachers’ presentations, both quality and quantity researches were performed. The following paragraphs briefly outline the methods used. 2.1 Sample Four secondary, 4 primary and 2 special schools in Hong Kong were selected based on a criterion that students of average abilities were participated in this study. 2.2 Instruments To collect the perceptions of teachers and students on the use of IWB, two sets of questionnaires were designed. In addition, participating teachers were invited to complete a reflection report, which was aimed to collect their reflections after using IWB in their teachings. The school administers were also required to complete progress reports on their progress in implementing IWB in their schools. 2.3 Procedure The study was performed in two phases, with the first phase in the spring term and the second phase in the autumn term of two different academic years in 2005. In each of the two phases, teachers and randomly selected students from 28 selected classes were interviewed after class observation. Questionnaires were distributed to collect teachers’ and students’ perceptions on IWB. Furthermore, teachers’ reflection reports and progress reports were collected. Table 1 summarizes the schedule and the data obtained. Table 1. Research schedule and data of the IWB study Initial visit (10)

Phase 1 (April – July) class observations (28), Perception surveys teachers’ interview (28), for teacher (87) and students’ interview (163), students (1023), teachers’ weekly self progress reports reflection reports (10)

Phase 2 (October - December) class observations (28), Perception surveys teachers’ interview (28), for teacher (93) and students’ interview (162), students (1152), teachers’ weekly self progress reports reflection reports (10)

3. Findings and Analysis Due to page limit, the following sections only report parts of the findings obtained from the questionnaires and supplemented by the interview data. For easier presentation, items in the questionnaires were categorized into 3 areas, namely interactivity, motivation and teachers’ presentation, according to the research aims of this study. 3.1 Interactivity IWB can be used to enhance classroom interactivity effectively. The mean score of survey questions related to interactivity are shown in Table 2. It is found that both the teachers and students are positive towards the interactivity promoted by the IWB. In the interview, many students claimed that teachers gave them more chance to answer questions and participate. Also, some teachers said that students with learning difficulties are more

F.-l. Lee et al. / Using Interactive Whiteboards (IWB) to Enhance Learning and Teaching

441

willing to participate in class activities. This in turn made students more active as they needed to interact with teachers. Table 2. The result of perception survey related to interactivity Interactivity

Student

Teacher

More eye contact between teachers and students Teachers can respond to students questions or offer his/her views Students are more ready to raise my hands and answer the teachers’ questions Students have more opportunities to discuss with the teacher

Mean 3.26 3.88 3.51 3.65

Mean 3.43 3.43 3.48 3.72

3.2 Motivation Being a novelty and technologically advanced teaching tool, the IWB catches students’ attention. Table 3 shows the mean scores of the survey questions related to motivation. It is found that the teachers and the students agree that the use of IWBs motivate students’ learning. Many students expressed that IWB looked cool and attractive. Some teachers and students thought that the clarity of display, the interesting effects and the new modes of operations of the IWB maintained students’ concentration in the lesson. Also, various teaching materials can be displayed on the same board, which holds students’ focuses throughout the lesson. Moreover, some teachers stated that the sense of achievement was increased when students worked on the IWB. All these factors motivate students to learn. Table 3. The result of perception survey related to motivation Motivation Students find lessons more enjoyable Students are more attentive in class Students have more opportunities to learn through playing games during the lesson

Student

Teacher

Mean 3.88 3.76 3.58

Mean 3.85 3.70 3.42

3.3 Teachers’ Presentation IWB is a convenient teaching tool. From the survey result in table 4, teachers’ presentation is improved with the help of IWBs. IWB enables the teachers to explain abstract concepts more clearly by highlighting, overwriting displayed information and using multimedia. A lot of the teachers reported that these features improved students’ understanding and memory of the taught contents. With TV programmes, videos and teaching contents shown on the same board, students do not need to turn their heads alternately to the TV and the blackboard, whereas teachers do not need to control various objects as before. Thus, the lesson can move on smoothly. Table 4. The result of perception survey related to Teachers’ Presentation Teachers’ Presentation Teaching materials are more clearly displayed Students find teachers’ instructions easier to understand It is easier to memorise the contents taught on the IWB Teachers would display more varieties of teaching materials (e.g. videos, pictures and sounds)

Student

Teacher

Mean 3.82 3.82 3.70 4.03

Mean 3.77 3.73 3.74 3.49

442

F.-l. Lee et al. / Using Interactive Whiteboards (IWB) to Enhance Learning and Teaching

4. Discussions and Conclusions Findings from the current study show that Hong Kong students and teachers generally welcomed the use of IWBs in the lessons. It was reported that IWB can bring new insights in teaching and that more graphics, examples, games, and so on can be used to enhance students’ understanding and to arouse their study interests. Teachers also reported that a greater variety of materials such as videos, sounds, pictures and information from the Internet can be used in the lesson to assist their teaching. In conclusion, the present study shows that the use of IWB in Hong Kong has similarities in its effect on increasing interactivity, arousing learning motivation and improving teachers’ presentation when compared to studies in the western world. Although the result is encouraging, further studies are recommended to avoid the possible contamination of the novice effect.

Acknowledgments This study was supported by the Hong Kong Education and Manpower Bureau in conducting a research project called “The Development of Using Interactive Whiteboards in Enhancing Learning and Teaching in Schools” in 2005.

References [1] Smart Technologies Inc. (2004). Interactive Whiteboards and Learning: A Review of Classroom Case Studies and Research Literature. Retrieved May 6, 2005, from http://education.smarttech.com/NR/rdonlyres/30258C60-24D0-43D5-A1D2-BDE1A93B6F93/0/Interact iveWhiteboardsAndLearning.pdf. [2] Huang, S. (2004) Immigrant Chinese Students’ Perceptions of their English Communicative Competence Relative to their Classroom English Training and their Field. Contact, 30, 3, pp.38. Canada: The Association of Teachers of English as a Second Language of Ontario. Retrieved June 22, 2006, from http://www.teslontario.org/new/publ/contact/ContactJuly04.pdf. [3] British Educational Communications and Technology Agency (BECTA) (2004). Getting the most from your interactive whiteboard – A guide for primary schools. Retrieved June 18, 2005, from http://www.becta.org.uk/. [4] Glover, D. & Miller, D. (2001). Running with technology: the pedagogic impact of the large-scale introduction of interactive whiteboards in one secondary school. Journal of Information Technology for Teacher Education, 10, 3, pp.257-276. [5] Gerard, F. et al. (1999). Using SMART Board in foreign language classrooms. Paper presented at SITE 99: Society for Information Technology and Teacher Education International Conference, San Antonio, Texas, 28 February - 4 March 1999. [6] Smith, H. (2001). SmartBoard evaluation: final report. Retrieved March 31, 2006 from Kent National Grid for Learning Website, http://www.kented.org.uk/ngfl/ict/IWB/whiteboards/report.html. [7] Walker, D. (2002), White enlightening. Times Educational Supplement, 13 September 2002, p.19. [8] Levy, P. (2002). Interactive whiteboards in learning and teaching in two Sheffield schools: a developmental study. Sheffield: Department of Information Studies, University of Sheffield. Retrieved April 10, 2005, from http://dis.shef.ac.uk/eirg/projects/wboards.htm.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

443

Conditions Facilitating The Implementation of Information Communication Technology Integration in Malaysian Smart School Wan Zah Wan Ali, Hajar Mohd Nor, Azimi Hamzah, Nor Hayati Alwi Faculty of Educational Studies Universiti Putra Malaysia, Malaysia [email protected] Abstract: In this paper, we describe the conditions that facilitated the implementation of Information Communication Technology (ICT) integration in the Malaysian Smart School locally known as Sekolah Bestari. The description was based on a study that was carried out using qualitative methodology on 21 informants, mainly teachers to discover why the teachers in this school have minimal use of ICT in their teaching. The study also attempts to identify problems that emerge during the process of integration. Three technology-rich Malaysian secondary schools, the Smart Schools, of different technology levels were involved in this study. Based on the data analysis, two sets of conditions were revealed. They were the essential conditions and the supporting conditions. The essential conditions see to implementation of ICT integration while, supporting conditions see to the continuation of implementation. The findings also revealed that teachers in this study employed four levels of approaches in integrating ICT in the schools. It seemed that the levels of approach were influenced by the presence and absence of those conditions. Time, course content and technical malfunction were found to be the main problems that the teachers faced during this process. Keywords: ICT integration in classroom, innovation in education, condition of change in education

1. Introduction Under the Smart School project, about 8,000 school will be equipped with computer facilities by the end of the year 2005. By the year 2010, it is projected that about 10,000 primary and secondary schools will have computer facilities. However, despite these computer facilities made available, there is no guarantees that teachers will use the technology extensively in their teaching. Smart School reports (MoE, 2000; MoE, 2001) and research findings (Sathiamoorthy, 2001; Lee, 2000) indicated that there was minimal use of ICT in schools. Why do teachers in Smart School have minimal use of ICT in the classroom? What are the conditions that facilitate these teachers to integrate ICT in the Smart School? It is with question in mind that this study is carried out. 2. Theoretical Framework Theoretically, there are many models on change in education. Meaning of Educational Change by Fullan (2001), Conditions of Change framework by Ely (1999) and Diffusion of Innovation Model by Rogers (1995) are some of the models that have been identified to give better understanding on the implementation of ICT integration experienced by the teachers teaching in secondary schools. These models also act as a guide to answer the research objectives which are:

444

W.Z. Wan Ali et al. / Conditions Facilitating the Implementation

1. 2. 3.

to identify the existence of conditions that facilitate the implementation of ICT integration in Smart School, to explore the existence of conditions that facilitate the implementation of ICT integration in Smart School, and to discover problems that emerges during the process of the integration of ICT in the Smart School.

3. Methodology A qualitative methodology was employed to explore the informants at its natural setting. Three technology-rich Malaysian secondary schools, the Sekolah Bestari, of different technology levels were carried out to identify the conditions. A total of twenty-one informants, who comprised of twelve teachers, three principals, three heads of curriculum department and three ICT coordinators were interviewed. The teachers were also observed during the classroom observation to look at their use of ICT at work. The data generated by interviews, classroom observations and document reviews were transcribed, coded and categorized relating to common conditions arising from the data. 4. Findings The study has identified the conditions that facilitated the implementation of ICT integration in the Malaysian secondary school curriculum. Two sets of conditions were revealed as in Table 1. They were the essential conditions and supporting conditions. The study found that these conditions are interrelated to one another. 4.1 The essential conditions The essential conditions identified were availability of ICT resources and acquisition of ICT knowledge. These conditions were able to see the implementation of ICT integration in the curriculum. If one of these conditions was not present then implementation of ICT integration would not take place. Lack of knowledge had been the caused for the teachers to be less confident to integrate ICT in the curriculum. 4.2 The Supporting Conditions The next set of conditions was the supporting conditions. The supporting conditions comprised of the accessibility of ICT resources, existence of support, desire to change, school practices, influence of external forces and teacher’s commitment to the innovation determined continuous implementation of ICT integration in the schools. The findings demonstrated a relationship between the presence of these conditions and the continuation of implementation of ICT integration. It was found that the presence of these conditions in schools enabled them to continue with the implementation of ICT integration. However, the lack or absence of these conditions resulted in the slow down or discontinuation of the integration of ICT in the curriculum.

445

W.Z. Wan Ali et al. / Conditions Facilitating the Implementation

Table 1: Conditions Facilitating the Implementation of ICT Integration Essential Conditions Accessibility of ICT resources

Schools School A

9

9

-

-

-

-

-

-

School B

9

9

9

9

-

-

9

9

School C

9

9

9

9

9

9

9

9

School practices

Ample support

Desire to change

Acquisition of ICT knowledge

Teacher’s commitment to the innovation

Influence of external forces

Supporting Conditions

Availability of ICT resources

Conditions

Outcome

Discontinuation of implementation Slow implementation Successful implementation

4.3 Levels of Approaches in Integrating ICT in the Curriculum The findings also revealed that teachers in the study employed four levels of approaches in integrating ICT in the curriculum. As it shows in Table 2, these teachers integrated ICT as verbal resources at level one, as printed resources at level two, as hands-on experience at level three and a combination of all the approaches at level four. It seemed that the levels of approach were influenced by the presence and absence of the conditions that facilitated the implementation of ICT integration in curriculum. Insufficient hardware had caused teachers not to integrate ICT aggressively in their teaching. Table 2: Level of ICT Integration Approaches in the Curriculum Level Level 1

Approaches ICT as verbal resources

Level 2

ICT as printed resources

Level 3

ICT as hands-on experience

Level 4

A combination of all the levels. ICT as hands-on, printed resources and verbal resources.

Situation Teacher teaches with the aid of ICT as verbal resource. Teaches gives the website addresses or name of courseware that would help students to enhance their understanding of the topics. Teacher teaches with the aid of ICT as printed resources. Distributed printed downloaded information as teaching aids. Teacher teaches with the aid of computer, courseware, software or Internet only. Teacher teaches with the aid of computer, courseware, software or Internet in delivering the lesson. She or he also gives out handouts with information printed from the Internet or courseware

4.4 Problems Teachers Faced during the Process of Integrating ICT in the Schools. The study also discovered problems that the teachers faced during the process of integrating ICT in the schools. The issues that emerged in implementation of ICT integration in the Malaysian technology-rich school were time factor, irrelevancy of course content and technical malfunction.

446

W.Z. Wan Ali et al. / Conditions Facilitating the Implementation

5. Discussion and Implication All the conditions found in this study were also found to be true to Ely’s eight conditions (1998), Fullan’s (2001) four factors affecting implementation and Rogers’ (1995) Diffusion Model as discussed in the literature. In the literature, Ely stated that there were eight conditions that should be present in implementing technology. However, only six of the conditions were found to be true in this study. Reward as one of the conditions mentioned by Ely was not present in this study but still, the innovation could still be implemented. Time was also one of Ely’s conditions. Time factor was not one of conditions that facilitate the implementation of ICT integration in the Malaysian technology-rich secondary school. Time in this study was one of the problems that emerged during the implementation. With or without time factor, the teachers in this study would integrate ICT in the curriculum. Rogers innovation attributes help to explain teachers’ different rate of adoption. In this study, two of the innovation attributes contributed to the cause of teachers not integrating ICT in the curriculum. Factors affecting implementation proposed by Fullan had also contributed to the implementation of ICT integration in these schools. The characteristics of innovations, the stakeholder involved and the external factors were equally important in these three schools. The teachers who were the main stakeholders of the innovation with the support from the principal and others determined the implementation of the innovation. From the data put forward from interviews, it has clearly shown that the teachers lacked knowledge on integrating ICT in classroom. Therefore, schools and Ministry of Education should improve their course outline in teaching teachers “when” and “how” to integrate ICT. Implementation of any innovation needs to be introduced one at a time. For example, in getting the teachers to integrate ICT in their lessons, they must be confident and comfortable with the technology first. The SSMS should come later. The teacher should also be given a choice of using the courseware bought by the schools. 6. Conclusion The ICT has become part of the society for communication between people, searching for entertainment and education, virtual meeting place, shopping and many more. Thus education plays a very important role to provide the platform and strong foundation to people. Conditions for successful implementation of ICT integration in schools are essential requirement and must be met to achieve the Malaysian vision 2020 of becoming develop country. References [1] Ely, Donald P. (1999) Conditions that facilitate the implementation of education technology innovations. Educational Technology, 38(6), 23-26. [2] Fullan, Michael (2001) The new meaning of educational change. (Ed. Ketiga). New York: Teachers College Press. [3] Ministry of Education Malaysia. (2000). Report: Collaborative monitoring of the implementation of smart school pilot project year 2000. Bahagian Sekolah, Kementerian Pendidikan Malaysia. [4] Ministry of Education Malaysia.. (2001). Education development 2001-2010. Kuala Lumpur: Ministry of Education. [5] Rogers, E.M. (1995). Diffusion of innovasion (4th Ed). New York: The Free Press. [6] Sathiamoorthy Kannan (2001). Intergration of computer into teaching-learning: A study of concerns and the managing ability among smart school teachers. Prosiding International Conference on Technology and Vocational-Technical Education: Globalization and Future Trend, 12-13 November, vol. 1 & II, Universiti Kebangsaan Malaysia, Malaysia.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

447

A Study of the Present Status of IT Teachers Training In Microsoft’s ‘Partners in Learning’ Project — A Content Analysis Approach Huang GuiJinga, Xu Yongb, Zhang JinBaob, Wang XiaoYuanc School of Educational Technologies,Beijing Normal University,China [email protected] Abstract: In this study based in a content analysis approach, the researchers utilized an open questionnaire to survey a sample of 35 IT teachers who have participated in IT teachers training in Microsoft’s ‘Partners in Learning’ Project. In the analysis, the researchers coded the collected data and categorized the data into category analysis tables from 3 aspects. Finally, according to the results of analysis, the researchers presented an evaluation of actual implementation status of MSPiL in the root level and thus revealed the effects MSPiL have in teachers training. Keywords: Content Analysis, Microsoft’s ‘Partners in Learning’ Project, Training for IT teachers.

Introduction To further support the informatization of China’s K-12 education, Microsoft (China) Co. Ltd and Chinese Ministry of Education signed ‘a framework in support ICT improvement in China’s K-12 education’. The MSPiL project group recognized that IT teachers are a key factor in promoting the informatization of fundamental educations. Therefore, this project identifies trainees as the IT teachers who are responsible for teaching IT courses, maintaining IT equipment, and supporting other teachers in IT applications. From 2004 on, MSPiL project has started IT teachers training in provinces such as Yunnan, Hubei, Gansu, Sichuan, Shan’xi and Guangxi. It has also maintained that teachers’ training is the core of the project. After more than 3 years of implementation of MSPiL, its actual effects in teachers training need an in-depth exploration. This paper carried out content analyses on the questionnaires returned by 35 IT teachers who have been surveyed, and revealed the real status of implementation on the root level of MSPiL

1. Issues at hand The researchers randomly sampled 35 teachers who have been enrolled in intermediate training in MSPiL project and surveyed them with an open questionnaire which is centered around three aspects: other teachers’ and school leaders’ recognition of computer and network technology after training, progress made in training and effects in application after

448

G. Huang et al. / A Study of the Present Status of IT Teachers Training

training. The questionnaires were 100% retrieved and 95.2% are valid. The questions include: z

Can you tell us briefly the changes that have taken place in your school in teachers’ and students’ use of computers? And the efforts school leaders made in promoting the use of computers.

z

Please tell us briefly what you have gained after participating in MSPiL’s training.

z

Can you tell us briefly which knowledge and concepts from MSPiL’s training program you have applied in practice?

In the questionnaires, 35 subjects gave their answers to these questions from different angles and on different levels. Due to the openness of the questions, their feedbacks are diversified, multidimensional and multileveled. This caused considerable difficulty in extracting effective information from their descriptions. Content analysis, as a research method that can objectively, systematically and quantitatively describe the overt content in communications, can help us efficiently extract the information and uncover the real meaning behind the text. 2. Applying content analysis on the questionnaires The essence of content analysis is to analyze the amount of information contained in the literature and its variations. Its purpose is to present a repeatable and effective evaluation of the content based on hard data. (Wu, 1991) 2.1 Determining categories and unit of analysis, evaluating and recording Data analysis is a process to categorize, filter, construct and reconstruct data (Yuan, 2002). In coding and analyzing the data, the researchers cannot design and implement the categories in one go. It is rather an evolving process consists of repeated coding, analyzing, filtering and construction. The following categories were finally decided after multiple trials and summaries. Some old categories were revised or rejected, and some subordinates and superordinates were divided from original categories.

2.1.1 Coding The ‘grounded theory’ Glaser and Strauss brought out in 1967 is an effective method to construct theories from bottom up. It induces concepts and propositions directly from the crude data and then raises it to theory (Chen, 2004). The most important step in grounded theory is to code the data in the order of three stages: open coding, selective coding and axial coding. In this content analysis research, the researches adopted the first two coding methods: open coding and selective coding. First in the open coding, the researchers picked out all the ‘local concepts’ from the subjects’ feedbacks and set up the initial categories (Li, 2003). The researchers then randomly select 5 questionnaires from the sample and verify the categories. The initial categories were modified according the result of verification. Several cycles of the procedure were conducted until the categories can pass every sample test.

G. Huang et al. / A Study of the Present Status of IT Teachers Training

449

(1) Open coding After the analysis and summary of the data obtained from questionnaires, the following ‘local concepts’ were discovered. (2) Selective coding The researchers analyzed carefully the connections between these ‘local concepts’ and discovered 5 major categories. The connections between the categories and related local concepts are summarized in the following table. Major Categories The level of use of computers and networks Behaviors & attitudes

Local Concepts building computer classrooms and reading roomsˈbuying computer classroom equipmentˈconstruction of websitesˈsearching information on the internet broadened horizon, learning initiative, interest in learning, more confidence, clear concepts, urging one’s self, strengthening relations

Professional knowledge & skills

Communication, improving learning efficiency, knowledge, capability, clear understanding professional level, improved skills ,

Measures used in implementation

Office applications, levels of management, application in teaching, specificity of trainings, being practical, application in practice, broad contents.

2.1.2 Determining categories, recording and evaluating According to some peripheral data obtained in the survey and the focus of this research, the researchers evaluated and categorized the data into three categories: (1) after the training, which changes the students, teachers and leaders have shown in their educational concepts about computer and network technologies; (2 ) what experiences and progresses have IT teachers gained after the training; (3) What have IT teachers done to apply the theories they learned to their practices after the training. The content analysis went through multiple cycles of sampling and coding.

3. Analysis of the effects of IT teachers training According to the frequency and percentage data in the three tables, several types of effects that MSPiL project has made were revealed. (1) After participating in MSPiL teachers training, IT teachers have made improvements on computer and network skills. Among the teachers, 19.05% have heightened interests in and enthusiasm for the application of information technology. After training, 23.81% of teachers considerably enhanced their professional level and IT application capabilities; Up to 28.58%of teachers had a clearer understanding and recognition of computer and information technology. (2) School leaders’ efforts in promoting informatization of education. Leaders’ support and encouragement are vital to the enhancement of IT capabilities and application/implementation of IT, almost the most important factor.

450

G. Huang et al. / A Study of the Present Status of IT Teachers Training

For software and hardware constructions, electronic preparation rooms are most common. Next come multimedia classrooms, QQ groups, MSN. There is also a high proportion of forum use. After the training, a lot of schools began to encourage the use of computer rooms and many teachers bought computers. These should be attributed to idea changes.leaders are very supportive of enhancement of IT capabilities, especially the software and hardware construction – 47.62%. In the application of IT, 19.05% of leaders leads in using the equipment, 14.29% encourage teachers to use computers. 19.05% of schools established electronic preparation rooms. (3) Teachers’ progress made after MSPiL training It is clear that 19.05% of teachers think they have considerably enhanced they problem solving abilities; 15.29% of teachers think their knowledge structures are more systematic; 14.29% of teachers think it has strengthened their communications with students, colleagues and leaders; 10.52% of teachers think they are more confident now. (4) Teachers’ application of the theories they learned to practices after training According to the data, 33.33% of teachers think that proper use of the hardware is the most practical; 28.58% think hardware maintenance is very useful in practice; 14.29% think they have a better knowledge of the equipment; 19.05% think that network configuration, office skills and network management are most often used. 4. Conclusion During the three years of implementation, MSPiL project’s teachers training program has had considerable effects on improving IT teachers’ professional capabilities. Teachers who have participated generally agree that they can now correctly recognize and properly apply computer and information technologies, in contrast with their past unclear understandings. What’s more, the trainings have made their knowledge more systematic and endowed them with a holistic view of professional knowledge, which they have applied in work. When asked about they progress in IT, they most often talked about problem solving abilities and personal interactions. They also said that trainings have made them broader in their visions and more confident. As to the teaching methods, many teachers expressed their attention to the students’ abilities to study independently and the creation of more opportunities for students to take part in. Meantime, more teachers are beginning to pay attention to the teaching and learning of network ethics and norms in network applications.

Reference [] Bryce Allen, Content Analysis in Library and Information Science, Foreign Information Science, 1993(1). [2] Wu Shizhong (1991), Outline of Content Analysis, Information and documentation work. [3] Li Kedong (2003), Research Methodology in Educational Technology, Beijing: Beijing Normal University Press. [4] Yuan Zhenguo (2002), Methodology in Education Researches, Beijing: Higher Education Press. [5] Chen Xiangming (2002), Quantitative Research Methodology and Social Science Researches, Beijing: Education Science Press.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

451

inPD: An Emerging Theoretical Framework for Educational Professional Development in the Information Age Simon Hughes Canterbury Christ Church University, UK [email protected] Abstract: This paper seeks to enunciate a theoretical framework for i-enabled Continuing Professional Development (CPD). It deliberately avoids the metaphorical application of the prefix ‘e’, to symbolise a more holistic approach to professional development than is currently afforded by assumptions in e-learning. Drawing on the Apple MacintoshTM metaphor of ‘i’ approaches to life (iTunes, iMovie, iPod etc), it plays with the more personal contiguities of ‘I’-ness. It extrapolates from the post-modern concept of the ‘self’, which can be seen to be central to all personal development activities and then explores what in might mean where some of the factors identified here, interactive, intelligent, international and integrated are taken to be critical in effective professional development. Keywords: interactive, intelligent, international, integrated

Introduction CPD activity was once described as enabling people to become professionals. This paper postulates a new theoretical framework for evaluating professional development activity and the ways in which this is facilitated and shaped by technology. The shorthand term for this new framework is inPD. This descriptor is explored and extrapolated through an auto/biographical reflection by the author who continues to search for possible answers to the question: ‘How did I get from being a Religious Education (RE) teacher to my present role?’ [1]. It was as part of that self-critical reflective process that the concept of inPD began to emerge. The key research tool underpinning this paper is thus ‘auto/biography’ combined with references to an eclectic body of literature derived from reading across a wide range of interests. This is seen as valid since the discourse for e-learning, inPD or even ICT in education has not been defined fully, nor its parameters set. Linden West [2], whose work in auto/biography is helpful in determining a professional learning journey, constantly exhorts the need to ‘play’ in a post-modern sense with an idea. That is what the last section of the paper attempts. No truth claims are made on the basis of the research since it is small-scale and non-generalisable. What is hoped, however, is that readers will find within the words something that may be of use to them: ‘its validity primarily lies in the meaningfulness of the analysis to other sense-making practitioners wrestling with similar questions, as well as the extent to which the interpretations illuminate the struggles of learners elsewhere, in analogous situations.’ [3]

452

S. Hughes / in PD: An Emerging Theoretical Framework

1. The Concept of inPD inPD has emerged from the synthesis of a number of disparate sources but appears timely given the move in the United Kingdom (UK) to associate teachers’ further professional development with Masters Level study. Whether the emphasis is on Continuing Professional Development (CPD) or Postgraduate Professional Development (PPD), the key is the PD of a practitioner in education. What follows is an explanation of other confluent sources for that construct which are laid out for critical evaluation. 1.1 The influence of the Apple MacintoshTM iLife Suite of Programs It is contended that one of the most easily recognizable artefacts of the new century is the iPod. Behind the customizable hardware is innovative software that delivers music direct from the internet to the device. Critical to the success of the marketing of this product has been the adoption of the lower case ‘i’ which hints at personalization, at self but also at interactivity. The Apple brand, with its own instantly recognizable logo, has been galvanized by its release of the iPod and its carefully crafted software partners – all with the prefix ‘i’. iTunes has radicalized the music industry in ways unimaginable 10 years ago. There are some key moments in very recent history which point to this technology overturning long-established recording industry custom and practice. Gnarls Barkley topped the charts in April 2006 with a track ‘Crazy’ that was only available as a download (See http://news.bbc.co.uk/1/hi/entertainment/4870150.stm ). The Arctic Monkeys also reached the top of the album charts with ‘Whatever People Say I Am, That’s What I’m Not’, having established a fan base entirely on the Internet (See http://news.bbc.co.uk/1/hi/entertainment/4369856.stm) and Apple rewarded Alex Ostrovsky of West Bloomfield, Michigan with a collection of gifts for downloading the billionth track from the iTunes website on 23rd February 2006. Getting a recording contract used to be possible only with expensive industry-aware management. Releasing a record now is just as hard but having your music heard, or even purchased, is as simple as … <save as>… <share>… The point is that the business of the music business is music. Listening to music is an entirely subjective activity, so selecting material becomes a matter of personalized choice which reflects ‘self’. Buying exactly what you want, rather than a whole album is, to a generation used to ‘zapping’ [4] from one sensory experience to another, much more appealing and economic. Technology, in this instance, has thus enabled a revolution in what is listened to commercially. Similarly creating a personalized album of images gathered by mobile phone, PDA, digital camera or scanned from old photographic paper prints is also possible using iPhoto. Of course there are PC versions too. Moreover the availability of cheap Read-Only Memory (ROM) makes it is also possible to build a collection of video text organized and edited using iMovie. This enables a person to represent themselves using multimedia but it also allows them the ability to access and engage with high quality audio-visual materials delivered direct to their desktop, PDA or DVD player. A message for education appears at this point: Learners can access resources, materials and professional conversations anywhere that is convenient to them. 1.2 Post-modern Approaches to the Self Mark Taylor [5], adopting a postmodern approach to his studies, uses the concept of ‘erring’ as a way of playing with ideas. He says, ‘Erring extends to the reader an invitation to participate in a “thought experiment”’ [6]. As long ago as 1984 he described the role of the self in making sense of experience and literature and makes this important observation: ‘This infinite interrelationship of interpretations cannot be captured in a closed book; it must

S. Hughes / in PD: An Emerging Theoretical Framework

453

be written in an open text. Texts point beyond themselves to other texts. In view of this intertextuality, it becomes apparent that writing is a ceaseless process in which writer is already reader and reader necessarily becomes writer.’[7] This is a remarkable prefigurement of the ‘intertextuality’ of the Internet - incidentally today it would be described as non-linear hypertextuality. Fundamentally, moreover, it points to the dynamic relationship between a self and that which it is reading/learning. The reflective cycle, and its little sister action research, depend on the learner interpreting the messages of their learning and applying them in the context in which they are working. Thus the self takes control of the truth of a situated reality. Charles Taylor charted the evolution of the concept of the self through the history of western philosophy arriving at an analysis of the then contemporary scene. He says, ‘Modernism succeeds… in the search for sources which can restore depth, richness and meaning to life’ [8]. Presumably Taylor would celebrate then any modern technology which restores depth, richness and meaning to the learning process. He would certainly acknowledge the need for tools which can liberate the self from ignorance, intellectual impoverishment or alienation. Andy Law describes this as ‘liberation technology’ [9]. 1.3 The iPod in Learning In the same way that music is the business of the record industry so learning is the business of the education industry. What matters in education is learning; not the manner by which it occurs. This means that if iPods can facilitate learning they should be embraced as a matter of urgency. It is an accident of history that the act of publishing an audio file over the internet in MP3 format has acquired the title ‘podcasting’. The resonance with the iPod is unmistakable but there is only brand coincidence. Noteworthy, however, is the emergence now of what has been described as ’iPoducation’ where materials, resources and learning tools are downloadable to the device for use in personalized learning. For example at http://portables.about.com/b/a/257148.htm, academic reference is made to the development of the iPod dictionary. Whilst that ‘tool’ is targeted on pupil learners, work going on at Stanford University is potentially significant for anyone involved in the facilitation of professional learning. The ‘iTunes in Association with Stanford University’ project (See http://itunes.stanford.edu) aims to deliver lectures, seminars and other learning resources provided by Stanford ‘Faculty’ in MP3 format through the iTunes interface. ‘Podcasting’ has ‘gone up’ to University. 2. inPD At this point it is possible to lay out the case for inPD. Its articulation depends on the acceptance of three factors: the iconic nature of the motif ‘i’ as an agreed symbol of contemporary life which signals something to do with technology; the theoretical understanding of the relationship of the ‘self’ to the learning process suggested above; the assertion that technology liberates people to become professional learners. The story of the author’s professional development runs alongside the acquisition of a range of IT, ICT and e-learning skills. From squatting around a tape-driven games terminal outputting through an old TV as a student, to the development of advanced knowledge management systems under contract, there has been a 25 year gap but in that time contemporary technologies have been intriguing partners. There has been an interactive relationship between them. Learning in and through technologies is consolidated with

454

S. Hughes / in PD: An Emerging Theoretical Framework

reflections on reading, critical incidents and other professional experiences. MacGilchrist, Myers and Reed [10] famously combined these factors into the notion of intelligent schools. Stacey’s [11] notion of ‘organizational intelligence’ has been a constant reminder of the need to work with data to develop as big a picture as possible of the factors affecting performance in a learning context. It has been useful also as a key to evaluating certain events, behaviours and actions in working context. Since 2002, the author has been able to travel and test emerging ideas and evolving practices. Sharing such professional development activities with partners in international contexts has enabled personal enrichment as well as benchmarking current projects against the standards pertaining in other countries. Undertaking CPD work in Malaysia was a formative experience personally and professionally but it was also instructive about the need for robust technical infrastructure if professional development is to be facilitated at a distance. The dynamic interaction of knowledge, problem-solving skills, reflexivity and computer-aided working means that the self embedded in this piece of writing is integrated fully in a personal and professional development continuum. This is indeed a holistic approach to professional learning where ‘i’ multiplied by, at least, the four factors identified will lead to development. 3. Conclusion In short inPD is about a personalized professional learning journey facilitated through the medium of technology. Needed now, to test this hypothesis, are further extrapolation of the examples above, collection of more auto/biographical accounts of similar professional development and research projects based around the theoretical framework. Central to Figure 1: Graphical Representation of inPD this emerging way of working may have to be learning pathways, identified by the learner themselves and negotiated with institutions willing to risk validating study designed by the student themselves

References [1] The author is currently Director of Learning and Teaching with ICT in the Faculty of Education at Canterbury Christ Church University, a Director of the Training and Development Agency (TDA) funded Teacher Training Resource Bank www.ttrb.ac.uk and the Project Director of RE-Net www.re-net.ac.uk [2] See West L. (2004 edition) Beyond Fragments: Adults, Motivation and Higher Education Taylor and Francis, London [3] See West L. (2004) Op. cit. p.13 [4] The concept of ‘homo zappiens’ is documented in the literature See Rae (2004) Where, When and How do University Students acquire their ICT Skills? http://www.ics.ltsn.ac.uk/pub/italics/Vol4-1/rae.pdf. [5] See Taylor M. (1984) Erring A Postmodern A/Theology University of Chicago PressChicago [6] Op. cit. p. 17 [7] ibid. p. 16 [8] See Taylor C. (1989) Sources of the Self University of Cambridge, Cambridge p. 495 [9] See Law A. (2001) Open Minds: 21st Century Business Lessons and Innovations from St. Luke’s Thomson, London p.160ff. [10] See MacGilchrist B. Myers K., Reed J. (2004 edition) The Intelligent School Sage, London [11] See Stacey R. (2002 edition) Strategic Management and Organisational Dynamics: The Challenge of Complexity FT Prentice Hall, London

Reflection and Self-Directed Learning

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

457

Automated Mentoring for Reflection in an Eportfolio Tzemin Chunga, Mun Kew Leongb, Joel P.L. Looc a National Institute of Education, Singapore b Institute of Infocomm Research, Singapore c CommonTown Pte Ltd., Singapore [email protected] Abstract: In this paper, we describe a system that provides automated mentoring through the use of reflective thinking in eportfolios. Reflective thinking is a cognitive tool to cultivate deep learning. It is most effective when done within a community of peers where social interaction is emphasized. Moreover, it is also important that reflection is guided by a mentor so that meaningful exchange can take place. We currently propose a system that dynamically generates reflective prompts from past and current eportfolios and adapt such prompts to students. The automatic mentoring process goes through four steps, namely, priming, guidance, momentum, and termination. The aim of the system is to help students acquire new knowledge and skills, achieve deep learning and move towards higher-order learning. Keywords: eportfolio, portfolio, community of practice, automated mentoring, reflection, reflective thinking

Introduction Reflection is a cognitive tool integrated in portfolios to support deep learning. Reflection encourages the active construction of knowledge through comparing new activities with existing mental schemas of a student. This process encourages the students to refine their knowledge instead of memorizing knowledge. In traditional portfolios, reflection is likened to a self-dialogue which takes place at the end of a project or artefact. These standalone portfolios lack an essential element in achieving active learning – exploratory dialogue that results from engaging a group of students who share the same interests and work together to solve problems. These dialogues are facilitated by a mentor. Lipman describes this process as “a Vygotskian-like teacher-guided community of inquiry that places an emphasis on social interaction and cooperative learning” (Cook, 2002, Section 2). According to Lipman (2003), this is “reflective model of education” practice where the teacher facilitates activities in which students take an active and reflective role towards their own learning. He refers to this kind of community as a thinking community where deep learning takes place. In a thinking community, reflection is a key factor as the “improvement of thinking involves reflection (p. 26). The opportunity for an online CoP to develop is provided by linking eportfolios from the same class of students. By doing so, the “stagnant” or “standalone” limitation of traditional portfolios, which allow reflection but do not support social interaction and cooperative inquiry with the guidance of a mentor, can be overcome. In this paper, we introduce eportfolios within communities of practice (CoPs) which allow automated mentoring for reflection.

458

T. Chung et al. / Automated Mentoring for Reflection in an Eportfolio

1. Reflection Reflection is a process of making sense of ideas and concepts in relation to our existing knowledge structure and our feeling (Smith, 2001). When a student talks to a peer, a teacher, writes a report, or creates a portfolio, he “allows himself to experience surprise, puzzlement, or confusion in a situation which he finds uncertain or unique” (Schon, 1983, p. 68). Smith further explains that reflection is an evaluation process to help verify if current practice is effective and if not, how to adapt and modify it. Reflection is a pedagogically sound cognitive tool where it leads to insight in informing us of our design and planning (Russell, 2002). Kimball (2005) points out succinctly the important role played by reflection in portfolios: it “undergrids the entire pedagogy of portfolios” (p. 451). However, if reflection does not encourage a dialogue that includes reasoning and judgment about knowledge, it is unlikely to lead to in-depth learning. Meaningful dialogues can only take place when they are guided by a mentor within a community of practice (CoP) (Lipman, 1991). In such educational or Vygotskian-like inquiries, social interaction or cooperative learning is key. Dialogues that encourage exploration are more likely to lead to deep learning. As a community explores, it evolves. Ideas and activities that are meaningful to the community become its culture and will be available for integrating into its member’s perspective (Cook, 2002). The traditional stages of reflection are self-awareness, description, critical analysis, synthesis, and evaluation. This is the cognitive model of learning by reflection. We are interested, however, in the active process of reflection, i.e., in how a mentor can intervene to assist in healthy and effective reflection. We propose that reflection can be broken down into 4 steps: Priming, Guiding, Momentum, and Termination 1.1 Step 1 Priming Priming is the process which gets a user to start reflection in any form. 1.2 Step 2 Guidance Making reflective statements can become meaningless if it is not based on reflective thinking. Reflective thinking is a complex and difficult process that requires careful mentoring. Students are unlikely to reflect successfully without considerable guidance (Kimball, 2005). Guidance occurs during the process of reflection when the mentor intervenes to help guide the user. Furthermore, guidance need not come from the mentor. It can be from a peer who has performed the same task, or from an automatic model (Lipman, 2003). The reflection process should be recursive (Kimball, 2005). A reflection triggers prompting of the next till a certain condition, such as completion of an artefact or readiness to move on, is met. In order to achieve knowledge integration, reflection prompts should take the form of self-monitoring prompts such as reflecting on the areas that are confusing, or difficult, areas that needed to be improved on, and plan for future learning activities. Activities prompts that remind learners to perform certain activities during the inquiry process such as to make sure certain articles are read, are less effective (Davis, 2000). Both types of prompts have their roles to play and they complement each other. First, to check for completeness, self-monitoring and activities prompts are needed. Second, to prompt for future growth, self-monitoring prompts that focus on planning are essential. Reflection, therefore, can take two forms: First, completeness, you want to make sure an artefact is well covered, or, it has more or less met its goal; second, growth, which refers to advancing from the current artefact to a higher-order artefact.

T. Chung et al. / Automated Mentoring for Reflection in an Eportfolio

459

1.3 Step 3 Momentum Once a user starts reflection, how do you get him to continue the effort to ensure he gets good benefit out of it? A mentor needs to ensure that the user spend sufficient time and effort to achieve self-improvement. Students should have the flexibility to include their reflection at any developmental stage of their portfolio and be able to link from an artefact or the portfolio proper. To keep the momentum going, the mentor can leverage on the community dialog. Methods such as getting the students to put up meaningful questions for the group to look for answers, give recognition to those whose questions are discussed, ask follow up questions, include everyone in the activities unless there is adequate justification, forge friendship and bonds, encourage consideration of alternatives based on evidence, and build on each other’s ideas (Lipman, 2003). These activities can help the student to achieve two types of improvement: first, extrinsically, the student performs well in class; second, intrinsically, the student finds inner satisfaction in the newly acquired skill or knowledge. With continual feedback, encouragement, and challenges pitched at the level of the student (Bailin, Case, Coombs, & Daniels, 1999) during these activities, the learners are likely to participate and reflect enthusiastically. In the study of a Mexican learning community, the researchers found that when children learn in a community which promotes modeling and guidance, dialogues, exploratory activities, and metacognitive reflection, the children will participate eagerly (Rojas-Drummond, Fernandez, Gomez, Marquez, Martinez, & Velez, n.d.). 1.4 Step 4 Termination It is when they have done enough and should move on to higher-order learning. Lipman (2003) describes this stage as achieving “reflective equilibrium”. It is when all the follow up questions, or reflective prompts, for that level have been exhausted; when one can apply the acquired knowledge to exercise sound judgment and has the “capacity to listen to or be open to reason” (p. 97). 2. Eportfolio and Community of Practice (CoP) We suggest eportfolio allied with CoP can be the platform to provide automated mentoring for reflection. An eportfolio differs from a static port in that it is never isolated, i.e., one of a kind. It is always one portfolio among many others in the same community (classroom, school, etc). This community, in the right circumstances, forms a special kind of CoP, in particular, a classroom CoP where all students work on the same assignments within a curriculum allows itself to be mined for communal completeness. What this means is that for any artefact common to all the students, it is likely that what one student may miss or forget, another student would notice or remember. Given a sufficiently large community, the community will be well described. In a larger community, such as the school, which exits over a long time, will not only have artefacts from any given class, but also more advanced artefacts from classes which have come before. For example, current grade 3 semester one artefacts will co-exist with artefacts from last year’s and artefacts from higher grades. The techniques of clustering, association, and sequencing in data mining (Palace, 1996) allow us to dynamically link up similar eportfolios. This will provide the condition for automatic mentoring of reflection. The system can automatically generate clusters of eportfolios, or, ‘Similar folios’ in our terminology. Association allows weight to be attached

460

T. Chung et al. / Automated Mentoring for Reflection in an Eportfolio

to prompts and sequencing arranges the order of prompts. This enables reflective prompts to be dynamically generated and customized for each portfolio. Reflection thinking within a community of practice (CoP) need not be confined within a text box, done just before or after uploading an artefact. It need not be guided by a list of static questions. It also need not be following the traditional portfolio development process in rigid sequence of “collect, select, reflect”. Reflection can take place anytime in the process of creating an artefact. Mentoring can be dynamically created and it can be customized for every learner so that the learner can continually benefit from the collective experience of the CoP it belongs to. Reflection prompts that are dynamically generated from reflective statements in the community forms our basis for automatic mentoring of reflection. Reflections are question and answer pairs. The questions can be asked by the teacher or the students themselves and they are often generic enough to be applied to different portfolios so long as the nature of the student’s works are similar. The answers are written by students in reply to the questions. A technique for automated mentoring is not unlike that used by popular Internet search engines – based on similarity and popularity. A large enough database of questions will enable the system to prompt the student with questions based on the type of artefacts and the nature of the portfolio.

Similarity: Similarity index of the artefact type (picture, document, video or audio) and associated text (title, caption, annotation and keywords). Popularity: The number of times the question is used as a reflection question.

Clustering and indexing techniques also enable effective search for similar past or current projects. The search result collection is a loose form of CoP from which students can start peer-to-peer learning and generating ideas for further work and reflections. There may be the issue of privacy of the eportfolios both between classmates and of students who have left the grade. We have always maintained that eportfolios should have at least two levels of privacy. The first is for the user themselves who can set who have access to their eportfolios. The second should be at the school level to protect the identity of the students or selectively lock personal information of the students.

3. Automated Mentoring in Eportfolios Eportfolios in the community of practice (CoP) can provide automated mentoring for each step of the reflection process. Step 1 Priming. It refers to looking at what others are doing and decide whether you want to do the same thing? Priming should start from the moment an eportfolio is created and given a title. A title such as ‘Making a short film’ will automatically connect the eportfolio to a community of students involved in film-making. More importantly, it makes this collection of eportfolios accessible to prime a newbie film-maker in the process of planning and actually shooting a film. This automatic associative power can even link up eportfolios at an artefact level. For example, an film-making eportfolio which has an artefact on screenplay

T. Chung et al. / Automated Mentoring for Reflection in an Eportfolio

461

can be associated with an eportfolio on drama which also happens to have an artefact on screenplay. Step 2 Guidance. It is provided by prompts that are from automatically generated assessment rubrics. There are two types of guidance, completeness and growth. Completeness guidance provides direction for meeting the requirements based on past eportfolios of the same academic level. Growth guidance refers to prompts that guide students to go beyond their current level of learning. Prompts from higher-order portfolios are mined and suggested to the student. When a new artefact is detected, the system generates a list of reflective prompts using data-mining techniques. The student can then accept and respond to these prompts. Currently the student need not accept all the reflective prompts as successive refinements for the system are being carried out. Steps 3 Momentum. The student is prompted to update and reflect on his eportfolio regularly through email. Prompts are taken from automatically generated assessment rubrics. The system will adapt prompts to the student’s ability. When students have successfully responded to the prompts, they will know they have acquired some new skills or knowledge. This can be both intrinsically and extrinsically rewarding. Step 4 Termination. An eportfolio is successfully terminated when the student has responded to the highest level of prompts based on the dynamically generated assessment rubrics. When all the reflective prompts have been responded satisfactorily and the teacher has no new prompts to add, the teacher can termination the eportfolio and urge the student to move on the next level of learning. While we have focused on the steps of the process of reflection above, it is also possible to look at the traditional stages of reflection in the same manner. This requires more sophisticated content analysis techniques and is beyond the scope of this paper. 4. System Description The current eportfolio system enables students to freely document and reflect on their work. It aims to capture the various aspects of a student’s learning experience including curriculum-based learning, project-based learning, personal hobbies, extra-curriculum activities and leadership development. It also provides ample opportunities for students and teachers to interact and has various automated mentoring features to aid in the reflective process.

462

T. Chung et al. / Automated Mentoring for Reflection in an Eportfolio

Our eportfolio system captures the various aspect of the student’s learning experience. A student’s work, known as ‘folios’, are collected in various folders.

There are also spaces for resume, goals and displaying of awards.

Figure 1. Eportfolio

The ‘Similar folios’ option matches the current folio with past and current folios and connect the student to a CoP of people who work on similar rojects.

Figure 2. A Project Folio

Reflections are enabled for all artefacts and the main folio itself.

Similar folios are useful resources for the student to learn from what others have done and to generate new ideas.

Figure 3. List of Similar Folios

T. Chung et al. / Automated Mentoring for Reflection in an Eportfolio

463

Each artefact has its own reflection space, activated by clicking on the ‘r’ icon. To encourage peer-to-peer learning, there is also a place for others to comment on the artefact.

Figure 4. Reflection Space and Comment Space

The ‘suggest’ option brings up a list of questions automatically collected by the system based on relevancy (the most popular questions from similar projects).

Figure 5. The “Suggest” Option Reflection questions are suggested based on the folio type, text similarity and popularity of the questions (how many times they were used in the past).

Figure 6. Automated Reflection Prompts The ‘Other ePortfolios’ option displays the eportfolios of the student in the same class. This feature enables students to browse, comment and learn from one another easily.

Figure 7. Class of Eportfolios

464

T. Chung et al. / Automated Mentoring for Reflection in an Eportfolio

5. Conclusion While inquiry is effective in a social context, reflective thinking is a key component to support deep learning. When combining these two powerful notions, we have a thinking community. Eportfolio provides the necessary condition for cultivating a thinking community. Reflective thinking is a natural occurrence in eportfolios and they can be easily link up for a community to emerge. Data mining technologies can be employed for the clustering of eportfolios. This allows learners with similar interests to find each other and search can be performed to find relevant artefacts. With a large enough database, reflective prompts can be dynamically generated from past reflective statements with high level of accuracy. Reflective prompts are likened to a mentor scaffolding the apprentice. Students can then refine their knowledge by modeling after the mentor and plan for future learning activities. With this, deep learning is likely to take place where the students can recognize the relationships among the subject matters being investigated and strike a balance between being open to other’s opinions and at the same time, take pride in the originality of their work (Lipman, 2003). Acknowledgments We thank Prof. Ulrich Bernath for his mentoring, support, and friendship. References [1] Bailin, S. Case, R. Coombs, J., & Daniels, L. (1999). Common misconceptions of critical thinking [Electronic version]. Journal of Curriculum Studies, 31(3), 269 – 283. [2] Cook, J. (2002). The Role of Dialogue in Computer-Based Learning and Observing Learning: An Evolutionary Approach to Theory. Journal of Interactive Media in Education, 2002 (5). Retrieved May 20, 2006 from http://www-jime.open.ac.uk/2002/5/cook-02-5-t.html [3] Davis, E.A.(2000). Scaffolding students' knowledge integration: prompts for reflection in KIE. International Journal of Science Education, 22(8), 819 – 837. Retrieved September 2, 2006, from Ingentaconnect Database. [4] Kimball, M. (2005). Database e-portfolio systems: A critical appraisal [Electronic version]. Computers and Composition, 22, 434-458 [5] Lipman, M. (2003). Thinking in education (2 nd ed.) [Electronic version]. Cambridge, UK: Cambridge University Press. [6] Palace, B. (1996). Data Mining. Technology Note prepared for Management 274A Anderson Graduate School of Management at UCLA. Retrieved May 20, 2006 from http://www.anderson.ucla.edu/faculty/jason.frand/teacher/technologies/palace/index.htm [7] Rojas-Drummond, S., Fernandez, M., Gomez, L., Marquez, A., Martinez, M., & Velez, M. (n.d.). Voices from a Mexican learning community: Experiences of implementing a fifth dimension program in primary schools. Retrieved September 2, 2006, from http://72.14.235.104/search?q=cache:KhgbgfilbukJ:ucerc.edu/Downloads/VOICES_UCLINKS.pdf+Roj as-Drummond,+Fernandez,+Gomez,+Marquez,+Martinez,+%26+Velez,&hl=en&ct=clnk&cd=1 [8] Russell, M. (2002, Spring). The role of reflection in adult learning. Pathways: From Maine Road to Quality. Retrieved May 20, 2006, from http://muskie.usm.maine.edu/maineroads/pdfs/pathways602.pdf [9] Schon, (1983). The reflective practitioner. New York: Basic Books [10] Smith, K. (2001). donald schon (schön): learning, reflection and change. infed. Retrieved May 20, 2006, from http://www.infed.org/thinkers/et-schon.htm

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

465

Guided Map for Scaffolding Navigation Planning as Meta-Cognitive Activity in Hyperspace Akihiro Kashihara, Mitsuyoshi Nakaya, and Koichi Ota Dept. of Information and Communication Engineering, The University of Electro-Communications, Japan [email protected]

Abstract: Planning navigation process in hyperspace is an important activity for learning hypermedia/hypertext-based contents such as existing Web contents. However, it is quite hard for learners to plan their navigation process in a self-directed way. The main issue addressed in this paper is how to scaffold the self-directed navigation planning process. Our approach to this issue is to provide learners with a guided map, which not only represents hyperspace but also highlights representative pages and links to be learned in the hyperspace. We propose a navigation history mining method that can extract these representative pages and links from navigation histories that could be gathered from peers. This paper also discusses the potential for adaptive scaffolding, which can adapt the guided map to learners' capabilities of navigation planning. Keywords: Guided map, navigation history mining, scaffolding, navigation planning

Introduction Hypermedia/hypertext-based learning contents such as existing Web contents generally provide learners with hyperspace, which consists of pages and their links. In the hyperspace, the learners can navigate the pages in a self-directed way. The self-directed navigation involves making a sequence of the pages, which is called navigation path [5]. It also involves constructing knowledge, in which the learners would make semantic relationships among the contents learned at the navigated pages [9],[11]. Such navigation with knowledge construction is called navigational learning [8]. In navigational learning, it is very important to make a navigation path since the knowledge construction process is influenced by the navigation path. In order for learners to succeed in their navigational learning, they need to plan a navigation path to be followed for achieving a learning goal before navigating the hyperspace [10]. However, it is not easy for them to carry out the navigation planning. In order to resolve this problem, we have developed a navigation planning assistant (PA for short) [7]. PA intends to scaffold the navigation planning process not only with hyperspace map but also with page and path previewers. These previewers can present the overview information of pages and navigation paths according to learners’ operations on the hyperspace map. On the other hand, navigation planning can be viewed as a meta-cognitive activity for controlling navigation and knowledge construction process in navigational learning [9], [10]. Meta-cognition is generally quite hard for learners [6]. Learners are often stuck on navigation planning even when the scaffolds such as hyperspace map, page and path previewer are provided. It is particularly hard to plan suitable navigation paths in the following cases:

466

A. Kashihara et al. / Guided Map for Scaffolding Navigation Planning

In case of the first use of hypermedia/hypertext-based contents, In case the hyperspace is huge, and In case learners are unfamiliar with learning hypermedia/hypertext-based contents. This paper addresses the issue of how to assist the learners who have difficulties in self-directed navigational planning for learning with existing Web contents. Our approach to this issue is to propose a guided map as scaffold for navigation planning, which represents hyperspace provided by the Web contents, and which highlights representative pages and links to be followed in the hyperspace for achieving their learning goal. The key point towards generating the guided map is how to find out the representative pages and links. This paper accordingly proposes a navigation history mining method, which can extract such representative pages and links from navigation histories that could be gathered from peers who learned the same contents with the same learning goal. However, it is difficult to identify these representatives from such histories as are generated by Web browser since the navigation histories do not clearly show which pages have been really learned and do not imply how the pages learned have been integrated for knowledge construction. Generating a guided map accordingly requires navigation histories that could represent the knowledge construction processes as properly as possible. We have already developed an Interactive History system (IH for short), which allows learners to annotate their navigation history with knowledge construction process [8]. The proposed history mining method uses navigation histories generated from peers who use IH to identify representative pages and links. This paper also discusses how to adapt the guided map to learners’ capabilities of navigation planning, which makes adaptive scaffolding for navigation planning possible.

1. Navigation Planning Let us first consider navigation planning in self-directed navigational learning. In order to enhance the efficiency of self-directed navigational learning in hyperspace, it is necessary for learners to plan a navigation path, which includes pages the learners need to visit for achieving their learning goal, before navigating the hyperspace [5], [11]. Such navigation planning can be viewed as a meta-cognitive activity for controlling the navigational learning processes. The goal of navigation planning is to plan a navigation path to be followed for achieving a learning goal. The planned path is expected to work as advanced organizer [1] for self-directed navigational learning process. However, navigation planning is not so easy for learners since they need to concurrently make diverse cognitive efforts not only at planning a navigation path but also at navigating the pages to understand the page contents [9]. In Web browser-based learning environments, the learners also tend to concentrate on understanding the contents of Web pages [8]. They accordingly have difficulty in continuing and maintaining the navigation planning process. The Web browser, in addition, is not suitable for viewing through a navigation path from the current page browsed. Current navigation aids, which support navigation planning, can be divided into two types, which provide global and local views of hyperspace. As representative aids for global view, there are hyperspace and concept maps [3],[4]. Although these maps can provide learners with a space, apart from hyperspace, for considering navigation paths, the information provided by the maps may be insufficient for planning a navigation path [7]. As representative navigation aids for local view, there is adaptive hypermedia [2], whose main purpose is to help learners select the page next to the current page. The

A. Kashihara et al. / Guided Map for Scaffolding Navigation Planning

467

Hyperspace Map

Path Previewer Link List

Figure 1. User Interface of PA. adaptive aids call learners’ attention to comprehending the contents of pages, but decrease their awareness of navigation planning, which is required in the self-directed learning. In addition, the adaptive aids are not always applicable to existing Web contents since they generally have no clear description of semantic relationships among the Web pages, which is indispensable for executing the adaptation. In order to address the issue of how to support navigation planning in self-directed navigational learning with existing Web contents, we have developed PA, which provides learners with hyperspace map, page previewer, and path previewer. These previewers can generate the previews of pages and navigation paths, which can be viewed as the middle of the global and local views of hyperspace. Figure 1 shows the user interface of PA. The hyperspace map represents hyperspace of Web contents selected by learners as network of nodes corresponding to the Web pages. It is automatically generated when the learners select the Web contents. Nodes in the map are tagged with page titles indicated by title tags in the HTML document files. Double-clicking any node in the map, the learners can have an overview of the Web page corresponding to the clicked node, which is generated by the page previewer. The page previewer extracts words, sentences, or images indicated by representative HTML tags such as headings, font size, etc. to display them as page preview. The page previewer helps them to decide from which page they start planning a navigation path. When learners decide the starting point of the navigation path, they can start the path previewer. The path previewer makes a sequence of previewed pages from the current page. The path preview window has a link list, which includes anchors of the links the current page contains. Selecting any one from the list, they can have an overview of the

468

A. Kashihara et al. / Guided Map for Scaffolding Navigation Planning

page, to which the selected link points, next to the preview of the current page. They can then put the page previewed into the sequence, making a navigation path. The learners are next expected to follow the navigation path plan to navigate the Web pages with Web browser. However, they can explore pages with the Web browser, which are not included in the plan. When they also want to change or cancel the navigation path plan during navigation, they can return to the navigation path planning. We have had a case study with PA, of which purpose was to ascertain if it facilitates navigation process in hyperspace compared to using only Web browser (See [10] in detail). The results suggest that PA produced more efficient navigation for integrating the contents of some Web pages in a more complicated hyperspace. We have also discovered learners who had difficulties in planning a suitable navigation path even with PA. This suggests that PA does not always provide proper scaffolding for such learners. Let us next propose a guided map as more suitable scaffolding method.

2. Framework for Scaffolding Navigation Planning The guided map represents the hyperspace map including highlighted pages and links, which the learners could navigate for achieving their learning goal. It is generated by navigation history mining, which can extract the representative pages and links from navigation histories that could be gathered from peers who learned the same Web contents with the same learning goal. Our framework uses IH to identify the representatives. In IH, the knowledge construction process is modeled as follows. Learners generally start navigating the pages for achieving a learning goal. The movement between the various pages is often driven by a local goal called navigation goal to search for the page that fulfills it. Such navigation goal is also regarded as a sub goal of the learning goal. The navigational learning process includes producing and achieving a number of navigation goals. We currently classify navigation goals into six: Supplement, Elaborate, Compare, Justify, Rethink, and Apply. We refer to the process of fulfilling a navigation goal as primary navigation process [8]. This is represented as a link from the starting page where the navigation goal arises to the terminal page where it is fulfilled. The knowledge construction process can be modeled as a number of primary navigation processes [10]. In each primary navigation process, learners would integrate the contents learned at the starting and terminal pages. For instance, a learner may search for the meaning of an unknown term to supplement what he/she has learned at the current page or look for elaboration of the description given at the current page. Carrying out several primary navigation processes, learners would construct knowledge from the contents they have integrated in each primary navigation process. IH allows learners to annotate a navigation history, which includes the pages sequenced in order of time they have visited, with their primary navigation processes. Figure 2 shows an example of annotated navigation history. IH monitors learners’ navigation in the Web browser to generate the navigation history in the Annotated Navigation History window. Each node corresponds to the page visited. The learners can make annotations of the primary navigation processes, which they have carried out, by means of the Navigation Goal Input window. (See [9] in more detail.) In order to generate a guided map, our framework prepares a repository that accumulates navigation histories peers generated with IH, and that classifies them according to Web contents they learned and to learning goals they had. It generates a set of navigation histories called focused set from the repository, which have been generated from the same Web contents as learners use and the same learning goal as they have. The focused set is inputted into navigation history mining.

A. Kashihara et al. / Guided Map for Scaffolding Navigation Planning

469

Figure 2. User Interface of IH. Each primary navigation process in the focused set is regarded as association rule Ps->Pt that represents an association between two learning events in the starting and terminal pages. It means that learning event in the starting page Ps is concurrent with learning event in the terminal page Pt. In order to extract representative primary navigation processes from the focused set, we introduce the minimum support and minimum confidence as thresholds for deciding the representatives. The representative primary navigation processes are highlighted on the guided map as shown in Figure 3. Consulting the guided map with the page/path previewers, learners can plan a navigation path for achieving their learning goal. Although they do not always follow the representative pages and links, they are expected to use it as scaffold or advanced organizer for navigation planning. In our framework, in addition, the guided map can be adapted to learners’ capabilities of navigation planning. This map adaptation could be executed by controlling the minimum support and confidence thresholds (Sth and Cth).

3. Guided Map 3.1 Navigation History Mining Prior to navigation history mining, the focused set is generated from the history repository. Each annotated navigation history generated from each peer in the focused set is called transaction. The number of the peers becomes the number of transactions.

470

A. Kashihara et al. / Guided Map for Scaffolding Navigation Planning

Figure 3. An Example of Guided Map (Sth=25%, Cth=25%). Each primary navigation process (Pi->Pj) included in the transactions is extracted from the focused set. The support and confidence values are then calculated as follows: Support (Pi->Pj) = the number of transactions including (Pi->Pj) / the number of transactions, and Confidence (Pi->Pj) = the number of transactions including (Pi->Pj) / the number of transactions including primary navigation processes whose starting page is Pi. The higher support value means that more peers carry out the primary navigation process. The higher confidence value means that there is a higher probability that the primary navigation process is carried out from the page Pi. The navigation history mining method outputs the primary navigation processes whose support and confidence values are higher than the pre-defined support and confidence thresholds.

3.2 Guided Map Generation The guided map highlights primary navigation processes that more peers carried out in learning the same contents with the same learning goal. It is generated from the output of the navigation history mining. Figure 3 shows an example of the guided map. This map is generated from a focused set that includes navigation histories generated by 16 graduate and undergraduate students who learned the Web contents about stock investment with the goal of learning the basics about the stock investment. The total number of the pages included in the contents was 85,

A. Kashihara et al. / Guided Map for Scaffolding Navigation Planning

471

and the average number of links per page was 5.84. It had a quite complex hyperspace. All students were unfamiliar with the domain knowledge. The average number of primary navigation processes the students produced with IH was 16.2. The guided map shown in Figure 3 is generated under the condition that both of the support and confidence thresholds are 25 %. The number of the highlighted primary navigation processes is 18, all of which are carried out by more than a quarter of the peers, and which are carried out from the starting pages with a probability of more than 25 %. The guided map intends to suggest which pages and links learners could follow. In most cases, as shown in Figure 3, highlighted primary navigation processes are connected each other, and compose paths. However, following the paths does not always lead learners to plan navigation paths since each primary navigation process are independently mined and the primary navigation processes with higher support and confidence values than their thresholds are highlighted as representatives on the map. The guided map accordingly gives learners a scaffold for making a navigation path plan, but they need to think how to construct their knowledge from the highlighted pages and links. On the other hand, the guided map may be insufficient for learners who are unfamiliar with learning in hyperspace. It is instructive to suggest to such learners the knowledge construction process by means of the guided map. Towards this issue, we have developed a method of generating a guided tour. This method first uses the output of the navigation history mining to identify a representative peer whose history includes as many representative primary navigation processes as possible. It then highlights the primary navigation processes, which are included in his/her annotated navigation history, on the guided map. Since all the highlighted processes are carried out by the representative peer, their connections can be viewed as representative navigation and knowledge construction processes in the hyperspace. Consequently, the connections highlighted can work as a guided tour for learners, which present the overview information of navigation and knowledge construction processes in the hyperspace.

3.3 Adaptive Scaffolding Let us here discuss the potential for adaptive scaffolding with the guided map. In case learners have high capability of navigation planning, they are encouraged to plan a navigation path only with the hyperspace map so that their self-directedness in navigation planning can be facilitated as much as possible. In case of learners who have low capability of navigation planning, they are encouraged to use the guided tour so that they can get an overview of the knowledge construction process from the guided map. In case learners have middle capability of navigation planning, they are encouraged to use the guided map to plan their knowledge construction processes by themselves. In this case, the choices of navigation paths to be planned from the guided map are important. If there are a lot of highlighted primary navigation processes in the guided map, choices of navigation paths to be planned from the map increase. The large choices may give rise to difficulty for learners who have lower (middle-low) capability of navigation planning. One solution to this problem is to set the support and confidence thresholds to higher percentages in order to select primary navigation processes with higher support and confidence. If there are few highlighted primary navigation processes in the guided map, on the other hand, the choices decrease. This may prevent learners who have higher (middle-high) capability from making the large choices. One solution to this problem is to set the support and confidence thresholds to lower percentages to increase primary navigation processes to be highlighted. The above-mentioned adaptive scaffolding with the guided map requires a learner modeling method that can identify learners’ capability of navigation planning from the use

472

A. Kashihara et al. / Guided Map for Scaffolding Navigation Planning

of the guided map, page, and path previewers. We are currently addressing this learner modeling issue of how to classify learners’ capability into 4 levels such as high, middle-high, middle-low, and low. The guided map can be properly adapted by controlling the support and confidence thresholds according to the identified level.

4. Conclusion This paper has proposed a guided map for scaffolding self-directed navigation planning as meta-cognitive activity in hyperspace, and a navigation history mining method for generating the guided map. The important point of this method is to extract representative pages and links in the hyperspace from navigation histories generated by peers who learned the same contents with the same learning goal and to highlight them on the map. IH makes the navigation history mining possible. The guided map is expected to work as scaffold or advanced organizer for navigation planning process. We have also discussed the potential for adaptive scaffolding with the guided map. According to learners’ capability of navigation planning, the guided map can act the role of guided tour. The number of representative pages and links to be highlighted can be also changed by controlling the support and confidence thresholds for deciding the representatives. We believe the guided map is informative for learners who have difficulty in navigation planning. We now plan to ascertain whether the guided map can facilitate learners’ navigation planning process. In addition, we would like to address the learner modeling method that makes the adaptive scaffolding feasible.

Acknowledgments The work is supported in part by Grant-in-Aid for Scientific Research (C) (No. 18500703) from the Ministry of Education, Science, and Culture of Japan.

References [1] Ausubel, D., and Fitzgerald, D. (1961) The role of discriminability in meaningful verbal learning and retention. Journal of Educational Psychology, 52, 266-274. [2] Brusilovsky, P. (1996) Methods and Techniques of Adaptive Hypermedia. Journal of User Modeling and User-Adapted Interaction, 6, 87-129. [3] Domel, P. (1994) WebMap - A Graphical Hypertext Navigation Tool. Proc. of Second International WWW Conference. [4] Gaines, B.R., and Shaw M.L. G. (1995) WebMap: Concept Mapping on the Web. Proc. of Second International WWW Conference. [5] Hammond, N. (1993) Learning with Hypertext: Problems, Principles and Prospects. In McKnight, C., Dillon, A., and Richardson, J. (eds): HYPERTEXT A Psychological Perspective, 51-69. [6] Jonassen, D.H. (2000) Computers as Mindtools for Schools (2nd ed.). Merrill Prentice Hall. [7] Kashihara, A., Hasegawa, S., and Toyoda, J. (2002) How to Facilitate Navigation Planning in Self-directed Learning on the Web. Proc. of the AH2002 Workshop on Adaptive Systems for Web-based Education, 117-124. [8] Kashihara, A., and Hasegawa, S., (2003) LearningBench: A Self-Directed Learning Environment on the Web. Proc. of ED-MEDIA2003, 1032-1039. [9] Kashihara, A., and Hasegawa, S. (2004) Meta-Learning on the Web. Proc. of ICCE2004, 1963-1972. [10] Kashihara, A., and Hasegawa, S. (2005) A Model of Meta-Learning for Web-based Navigational Learning. International Journal of Advanced Technology for Learning, 2, 4, ACTA Press, 198-206. [11] Thuering, M., Hannemann, J., and Haake, J. M., (1995) Hypermedia and cognition: Designing for comprehension. Communication of the ACM, 38, 8, 57-66.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

473

Self-directed Learning in Technology Supported Project Work Allan H.K. Yuen & Liping Deng Faculty of Education, The University of Hong Kong, Hong Kong [email protected] Abstract: Based on the analysis of 83 case studies developed in the Module 2 of the Second International Information Technology in Education Study(SITES), project work was found to be the most popular format of pedagogical innovation. A careful examination of the project descriptions revealed that the 34 cases of project work could be further distinguished into five approaches: research project, thematic project, study trip, discussion project, and aggregated-task project. With one case chosen from each category as the focus, this paper attempts to examine technology supported project work from the perspective of self-directed learning. We seek to reveal how project work is related to self-directed learning and how technology helps the teaching and learning process. Keywords: Project-based learning, self-directed learning, technology support

Introduction The notion of project work in particular technology supported project work has become increasingly important in schools around the world. Project work was reported to have positive effects on students’ motivation, performance, collaboration and self-regulation [1]. In most project work, investigation is driven or anchored by authentic questions and results in certain kind of artifacts. These driving questions make project work contextualized in real-life topics and the resulting artifacts become the products for review, evaluation and reflection [2]. Due to the authenticity and complexity of the task, project work is usually group-based, multidisciplinary and time-consuming [1]. This paper seeks to examine technology supported project work as pedagogical innovation through the lens of self direction. Five cases of project work were chosen for in-depth analysis from the perspective of learner control over various dimensions of project work. The research questions this paper focuses on are: 1) how is project work related to self-directed learning? 2) How does technology contribute to the teaching and learning process in project work? 1. Project Work, Technology and Self Direction In our study, self-direction is viewed as learner control in formal instructional setting. In traditional classroom context, teacher usually takes control over learning objectives, content, strategies and methods for evaluation. Innovative project work can provide opportunities for students to take major responsibility for and ownership of both learning process and product [3]. In collaborative project work, students construct knowledge socially in small groups, in which the locus of authority shifts from teacher to student groups [4]. Hence there are three major stakeholders in project work: student, teacher and group[5]. As a result, in our

474

A.H.K. Yuen and L. Deng / Self-Directed Learning in Technology Supported Project Work

examination of relationship between self-directed learning and project work, we will scrutinize over the power dynamics of these three parties. To have a clear picture of how self direction is executed, it is necessary to take closer look at dimensions of learner control in project work. Through adapting Candy’s [6] model of self-directed learning, Blumberg [7] summarized learners’ self-directed skills in PBL as the ability to 1) define what to learn; 2) plan; 3) seek, use, and evaluate the effectiveness of resources; and 4) evaluation. However, an important element of project work – artifacts [8] - is absent from this framework. As we mentioned earlier, artifacts are usually the driving force of project work. By modified Blumberg’s framework, we have this five-dimension framework as the basis for our analysis: (1) task analysis and goal-setting; (2) planning; (3) information seeking and evaluation; (4) artifacts construction; and (5) evaluation. 2. Method The Second International Information Technology in Education Study (SITES Module 2) collected 174 case studies from 28 participating countries/regions. National research teams for the study wrote a report for each of the case for international comparison. These case reports provide rich data for comparison of ICT-supported innovation in schools around the world. In the process of analyzing, the Hong Kong research team considered 83 reports to have sufficient details. Through qualitative analysis of the 83 cases, the innovative pedagogical practices were categorized into six types: project work, scientific investigation, media production, virtual schools or online courses, task-based learning, and expository teaching. Among six categories, the most popular format was project work (34 cases) [9]. These 34 cases of project work were further divided into five approaches, that is, research project, thematic project, study trip, online discussion and aggregated-task project. On the basis of the categorization, we randomly chose one case from each group for in-depth analysis from the perspective of students’ self direction and use of technology. 3. Comparative Study of Five Selected Cases Table 1 presents the major findings from our qualitative analysis of five cases in accordance with the five-dimension framework elaborated earlier. Since ICT tools were the main focus of the investigation, we also looked into whether students, groups and teachers used technology in the process of project. When control is detected by any party, a “/” is entered at the appropriate place. If technology is used in the process, the “/” will be changed to “X”. Table 1: Comparative analysis of five cases AU001 CL010 CN011 DK007 ZA001 S T G S T G S T G S T G S T G Task analysis & Goal-setting Planning Information Seeking & Evaluation Artifacts Construction Evaluation

/ X X X X

/ X / / X X X X X X X X X X X /

Notes: S = Individual Student; T = Teacher; G = Student Group

/ / X X X /

/ / X X

/ / X X X X X X / / / /

/ X X / /

/ / X X /

The evidence for self-direction is compelling. Three out of five cases show the signs of students’ control in all the five dimensions. Information seeking and artifacts construction are the two dimensions with strongest learner control. Four out of five cases witness individual-based learner direction in setting learning goal, information research and product development. The group-based activity of locating resources is shared by all the cases,

A.H.K. Yuen and L. Deng / Self-Directed Learning in Technology Supported Project Work

475

while four cases report group efforts in building artifacts. Four out of five cases witness learner or group control in task analysis and evaluation. The dimension with comparatively weaker learner control is planning with three cases involved. Except for CL010, all cases involve students in the process of defining or refining research questions or topics to work on. Except for CL010 and CN011, students worked with teachers in planning out the learning activities, while only ZA001 had the group-based planning activity. In the process of learning, students were involved in information search, analysis, synthesis and evaluation of the resources. Except for AU001, all the cases show explicit evidence of collaborative work in artifact construction. Additionally, authentic assessment or performance assessment is used in all these cases. Students participated in either self-evaluation or peer evaluation except for CN011 in which learners were evaluated based on their knowledge, communication and presentation skills, learning attitude and group work. Self-direction on the part of students does not lessen the importance of the teachers. Innovative teachers play important role in monitoring process and assisting students with cognitive, technical and collaborative issues. They provide resources, suggestions and feedback as appropriate. In all of the five cases, teachers are not regarded as the sole source of information. Students got outside of classroom either physically or via ICT to communicate with people in real-life context or to consult outside experts. 4. Roles of Technology in Project Work With the increasing capacity and availability of ICT tools, project work and technology has been closely connected. Moss[10] examined how technology was used in the student-centered, individual-based project work. Technology was found to serve two major purposes: one as research tool, the other as presentation vehicle. To probe deeper into the role of ICT in project-based learning, we used the framework advanced by Jonassen and associates[11] regarding the roles of technologies in learning-related activities. They categorized the role of technical tools as the following five types: 1) as tools to support knowledge construction; 2) as information vehicle for exploring knowledge; 3) as context to support learning by doing; 4) as social medium to support learning by conversing; 5) as intellectual partner to support learning by reflecting. The following discussion will utilize this framework to explicate how technology facilitates and enriches learning experience in the five cases. Table 2: Roles of Technology Tools for knowledge construction information vehicle context social medium intellectual partner

AU001 x x x x

CL010 x x x

CN011 x x

DK007 x x

ZA001 x x

x

x

The most prominent role of technology in these five projects work is as information vehicle. Students in all five cases use a wide range of ICT tools to locate, store, edit and manipulate information. Except for CL010, ICT also facilitates knowledge construction in all the projects. In the case of AU001, DK007 and ZA001, ICT tools serve as social medium to enable and facilitate communication and collaboration. Only two cases - AU001 and CL010 - witness technology used as intellectual partner in assisting student’s reflection and self-evaluation, while only one case - CL010 - uses ICT tools as context for learning. None of the five cases witnesses all five roles of ICT in project process. In all these five cases,

476

A.H.K. Yuen and L. Deng / Self-Directed Learning in Technology Supported Project Work

students’ ICT skills are part of curriculum, but they are not learned as separate subjects, but intertwined into the projects. This is align with the argument made by Moursund [12] that in constructivist-based curriculum, IT is “integrated into content area as well as being a content area in its own right” (p. 17). 5. Conclusion In this study, we chose five cases of ICT-assisted project work for in-depth analysis through the lens of learner control. At the same time, we scrutinized over how technology helped the process of teaching, learning and group interaction. The result shows abundant evidence of self-direction on the part of students in setting learning objectives, planning, information seeking, artifacts construction and evaluation. Students take major responsibility and initiative in investigation and production. Innovative teachers play important role in monitoring project progress, assisting and scaffolding students’ investigation and providing feedback. Technology is found to serve mainly the purpose of information seeking tool, artifact construction vehicle and social medium. Future study can involve more cases to provide quantitative data on the pattern of ICT use in different types of project work. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]

Means, B. and K. Olson, Technology's role in education reform: Findings from a national study of innovating schools. 1995, U.S. Department of Education: Washington, DC. Guzdial, M., Technological support for project-based learning, in ASCD Yearbook 1998: Learning with Technology, C. Dede, Editor. 1998, Association for Supervision and Curiculum Development: Alexanderia, VA. p. 47-71. Blumenfeld, P.C., et al., Motivating project-based learning: Sustaining the doing, supporting the learning. Educational Psychologist, 1991. 26(3-4): p. 369-398. Bruffee, K.A., Collaborative learning: Higher education, interdependence, and the authority of knowledge. 2 ed. 1999, Baltimore: Johns Hopkins University Press. Zimmerman, B.J., A commentary on self-directed learning, in Problem-based learning : A research perspective on learning interactions, D.H. Evensen and C.E. Hmelo, Editors. 2000, Lawrence Erlbaum Associates: Mahwah. p. 299-314. Candy, P.C., Self-direction for lifelong learning: A comprehensive guide to theory and practice. 1991, San Francisco: Jossey-Bass. Blumberg, P., Evaluating the evidence that problem-based learners are self-directed learners: A review of the literature, in Problem-based learning: A research perspective on learning interactions, D.H. Evensen and C.E. Hmelo, Editors. 2000, Lawrence Erlbaum Associates: Mahwah. p. 199-226. Krajcik, J.S. and P.C. Blumenfeld, A collaborative model for helping middle grade science teachers learn project-based instruction. Elementary School Journal, 1994. 94(5): p. 483-497. Law, N., et al., A comparative study of “innovative pedagogical practices using technology”: A secondary analysis by the Hong Kong study centre. 2003, Centre for Information Technology in Education, University of Hong Kong: Hong Kong. Moss, D.M., Bringing together technology and students: Examining the use of technology in a project-based class. Journal of Educational Computing Research, 2000. 22(2): p. 155-69. Jonassen, D.H., et al., Learning to solve problems with technology: A constructivist perspective. 2nd ed. 2003, Upper Saddle River: Merrill Prentice Hall. Moursund, D., Project-based learning: Using information technology. Second ed. 2003, Eugene: ISTE

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

477

Multi-step Annotation to Promote Reflective Learning with a Mobile Phone Gotoda Na, Matsuura Kb, Kanenishi Kb, Niki Ka, Yano Ya a Faculty of Engineering, Tokushima University, Japan b Center for Advanced Information Technology, Tokushima University, Japan [email protected] Abstract: The current Japanese society makes people bring mobile phone all the time and some university officially admits students to use it in the lecture. This paper proposes multi-step annotation environment using mobile phone as a trigger to start reflective learning. In the lecturing time, only thing to do is marking the important slide of the power-point by way of e-mail on the mobile phone. After the lecture, a student, who is the author of the e-mail, has to fulfill the annotating content marked by the e-mail. This paper reports the design and development of such a system. Keywords: e-Learning, cellular-phone, annotation, reflection

Introduction Current e-learning practices indicate the necessity and efficiency of on-line courses

;CPQGVCN. Some advanced projects in Japanese society have tried to adopt individual mobile-phone for practical purposes; i.e. getting participants list (Kobayashi, et al., 2005). With such a background, the e-learning project at Tokushima University has decided to integrate the mobile-phone based functions into the traditional e-Learning system (Thornton, et al., 2004). However, a course teacher often regards the mobile phone as disturbing deviceagainst the lecture. The features of such a device are that every mobile phone has indeed smaller number of buttons than that of normal computers, and its display size is also smaller. Hence, with respect to these characteristic viewpoints for installing mobile phone into the campus-based lecture, it is used as just a trigger for subsequent reflective learning step. Although the normal computer is used mainly for learning, the mobile-phone based learning activity promotes learners to remind the important or interesting points in the real-time lecture. Three-step reflection is performed through this proposal. 1. Blended learning scenario This section illustrates the learning scenario from teacher’s and learners’ viewpoints (See Fig.1). All the annotations by learners on some contents are linked to each slide materials. 1.1 Teacher’s viewpoint At the teacher side, the supporting tool collects mouse event in the power-point slides at the lecture. This tool also collects all the characters in every slide. In addition, all screenshots with time-stump are captured at the time the slide contents changes in the

478

N. Gotoda et al. / Multi-Step Annotation to Promote Reflective Learning with a Mobile Phone

slideshow mode. All these information are uploaded to the contents management server (=CMS) to be delivered properly. In order for a learner to take notes onto the contents, we introduced screen image, which is not based on the power-point object.

Figure 15[UVGO%QPHKIWTCVKQP

1.2 Learner’s viewpoint When a learner finds something important slides for her/him, s/he wants to add tiny sign to remind them easily for the reviewing process. Mobile phone function becomes alternative method to input this sign into the system. The detail contents linked by this sign would be rewritten after the lecture. Then, detail investigation and extended study in free time by the owner are carried out. This multi-step annotation is called reflective approach by our e-learning project. 2. Multi-step annotation 2.1 Reflective learning with multi-step annotation Reflection (Kashihara, et al., 2004) in this paper is always regarded as the in-personal learning activity whereas it sometimes means interpersonal learning activity. Through the overtimes learning activities in modifying something electronically, a learner remembers the contents stronger than one-time learning. Reflective learning of our environment occurs with the multi-step annotation. The concrete methodology is shown as follows; (1st step) Using mobile phone in the lecture time, just a trigger command (See table 1) is sent to the server via the e-mail on a mobile-phone. No other inputs are necessary. To implement easiest way, only one or two times button-push are required. (2nd step) At the immediate aftermath of finishing a lecture, a learner has to make the content of that trigger command be clear by inputting the reason why these commands were interesting. A small reflection occurs to the contents owner at this time but s/he has no explicit knowledge about them. (3rd step) Until the next time lecture, a content owner freely studies and the outcome from the study influences to the content slot of the system by inputting solving way of her/his question or comment to make something clear.

N. Gotoda et al. / Multi-Step Annotation to Promote Reflective Learning with a Mobile Phone

479

2.2 Command code details Table 1 Key assignment on mobile phone

Publication limits

Personal Classroom University WorldWide

Low 1 4 7 *

Importance Middle 2 5 8 0

High 3 6 9 #

When a learner sends an e-mail to remember where the important part is, the operation must be in an easiest way not to disturb her/his concentration. To implement the workflow based on this principal, following interaction between a learner and the system was designed. (1) Lecture code is randomly generated by the system. A teacher can request to generate it and get the unique ID of the lecture at this process. (2) A student is told the login ID and password to enter. Once a student logs in the system s/he can register her/his e-mail address of the mobile phone to the system. This process is usually done at the first lecture time. (3) At beginning of the lecture, a lecturer has to tell the ID for all the students. Then a student has to send an e-mail (more than once) to get authentication code for each e-mail address. The system can check the address with the registered one and reply to the address in case the matching process is succeeded. (4) Whenever a student wants, s/he can just send a trigger command in the subject of an e-mail. Concrete commands are shown in table 1. Phone keys are put in two dimensions (Table 1). Hence, two types of attributes could be defined. In more detail, horizontal line has three degree and vertical one has four. Former axis deals with the importance, whose degree covers from low to high, of an annotation. Another axis is used for publishing limits of her/his annotation contents. Only things to do for a student is to select one key into the subject and send the e-mail. 3. Development 3.1 Technical details The mobile phone is used for only the first step (See Fig.1). Another two steps are allowed to use either desktop PC or still mobile phone. In regard to special functions on this general architecture, agent technology is used to promote reflective learning. For example, a system sends a message to the annotation owner to input the content to the command in case a student seems to do nothing about this item after making the trigger. Another device was made to synchronize the e-mail-based annotation items and the slide contents. The system automatically synchronize them judging from the arriving time of the e-mail. According to this function, the owner and students available to see find well associated contents (See right part of Fig.2).

480

N. Gotoda et al. / Multi-Step Annotation to Promote Reflective Learning with a Mobile Phone

3.2 User Interface A student uses this environment after the lecture to modify annotation contents or attributes etc (Figure 2). Left side frame presents the annotation item list. Upper-right is used for the power-point slide image, whose annotations are displayed together as the tab titles. Lower-right presents each annotation contents with two types of attributes, limitation range and the degree of importance.

Contents of a slide

List items of annotations

Contents of annotations

Figure 2Snapshot for the 2nd and third step

4. Conclusion This paper describes multi-step annotation environment for promoting blended-learning. Our approach focuses on the trigger function, which is implemented on the personal mobile phone, to start annotation. Acknowledgments This research got financial supports of Grant-in-Aid for Scientific Research of the Ministry of Education, Science, Sports and Culture of Japan, No. (B)(2) 16300271 and Wakate 17700609, Tokutei 17011053. References [1] Yano, Y., Matsuura, K., and Ogata, H.: Design and Implementation of an Asynchronous Virtual Classroom - Retrospective and Prospective View -, International Journal of Information and Systems in Education, Vol.2., No.1, pp.14-22, 2003. [2] Kobayashi T., Kim J. and Machida N. (2005) A Student ID System Using Cell Phone and Its Ecaluation, Proceedings of Intl. Workshop on Wireless and Mobile Technologies in Education 2005, pp.45-47, Japan. [3] Thornton, P., Houser, C. (2004) Using Mobile Phones in Education, Proceedings of International Workshop on Wireless and Mobile Technologies in Education 2004, pp.3-10, Taiwan. [4] Kashihara A. and Hasegawa S. (2004) Meta-Learning on the Web, Proc. of International Conference on Computers in Education (ICCE2004), pp.1963-1972, Melbourne, Australia.

Game and Edutainment

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

483

Property Exchange Method for Automatic Generation of Computer-Based Learning Games Takanobu UMETSUab, Tsukasa HIRASHIMAb, Akira TAKEUCHIa Computer Sicence and Systems Engineering, Kyushu Institute of Technology, Japan b Graduate School of Engineering, Hiroshima University, Japan [email protected]

a

Abstract: Learning games are very useful to realize highly motivated learning. Developing effective and high-motivating learning games, embedding learning materials into a game is a promising approach. As a concrete method to embed learning materials into a game, we propose Property Exchange Methodin this paper. The method can deal with card games where players manipulate cards following their rules. The rules usually only deal with limited attributes of the cards. Therefore, if the learning materials possess the same attributes, players can manipulate the materials in the same way following the same rules. Since players can play the game with the same rule dealing with the same attributes, it is expected that the game keep the characteristics of the original game. In this paper, we also introduce a system that automatically generates computer-based learning games based on the method. An experiment use of the generator is also reported. Keywords: Design Methodology, Intelligent Learning Environment, Game-Based Learning, Edutainment

Introduction Motivation is one of the most important factors in learning. Many researchers of learning environments, therefore, pay special attention to learning games as a remarkable approach to realize highly motivated learning. However, making a learning game is not easy task because the developers satisfy both motivation and learning effect. Although there are several investigations for design methods of learning games, most of them are only a kind of guidelines that the developers have to keep [1-6]. The developers, therefore, are required to have enough knowledge and experiences regarding learning games. Based on these considerations, we are investigating a concrete and simple method to design a learning game systematically. In our research, we don't try to design a learning game from scratch. Instead, we have adopted an approach to transform an existent game into a learning game while keeping the characteristics of the original game. Developing effective and high-motivating learning games, embedding learning materials into a game is a promising approach. As a concrete method to embed learning materials into a game, we propose Property Exchange Methodin this paper. The method can deal with card games where players manipulate cards following their rules. The rules usually only deal with limited attributes of the cards. Therefore, if the learning materials possess the same attributes, players can manipulate the materials in the same way following the same rules. Since players can play the game with the same rule dealing with the same attributes, it is expected that the game keeps the characteristics of the original game. In following chapters, we explain Property Exchange Method in more details. We also introduce a learning game generator developed based on the method. The generator generates a computer-based learning game from a computer-based game and a set of

484

T. Umetsu et al. / Property Exchange Method for Automatic Generation

learning materials. We have already generated several computer-based learning games for Trick-Taking and confirmed that they can be played as games, are useful for learning the mateirlas and keep the characteristics of Trick-Taking.

1. Design Method for Learning Games Many arguments about the process of developing a learning game have been discussed in order to alleviate the difficulty in the process. For examples, Malone suggested that a learning game should be designed that to be skillful in the game is valuable for the target learning [1]. Prensky also proposed that it is important to carefully decide the combination of a game style (which is adventure, action, sports, role-play, strategy or simulation) and a learning style (which is drill, questioning or observing) [2]. Klawe analyzed the role of navigation and interaction in a learning game [3]. Maragos insisted the role of multiple stages in a learning game [4].Halff dealt with adventure games, in which the player assumes the role of a character in a fantasy world. He proposed that a learning game should restrict behaviors of a player in order to make the player learn in suitable order [5]. In ICCE2004, we proposed “Partial Exchange Method” to transform an existent game into a learning game by exchanging a part of the existent game for learning materials when the part is similar to the learning materials [6]. Although these design methods help developers design learning games as guidelines, most of the process of developing learning games left to a developer’s discretion. For example, in Partial Exchange Method, it is difficult to find an adequate set of learning materials and a part of existent game because there is no clear definition about the part or the similarity. Therefore, the developer is required to have enough knowledge and experiences regarding learning games. Based on these considerations, we propose “Property Exchange Method” as a concrete and systematic method. The method is a specialized method of Partial Exchange Method. Partial Exchange Method deals with all kinds of games, but Property Exchange Method deals with only card games. However, the limitation makes it possible to find an adequate set of learning materials and a part of existent card game systematically. Based on the method, we have developed a system that automatically generates computer-based learning games. We also report the results of experimental use of it.

2. Property Exchange Method 2.1 Concept Although a lot card games uses the same set of cards usually called “playing cards”, the plays of the games are different. Therefore, the play of a game is mainly decided by rules of the game. Then, a game is sometimes played in the same way with different kinds of sets of cards. For example, UNO can be played with either the playing cards or special set of cards for UNO. In other words, it is necessary only a part of properties of the card to play a card game. Based on these considerations, if we can prepare new set of cards that have enough properties that are required in the original game, we can generate a new game that is composed of the rules of the original game and the new set of cards. Then, it is expected that playing the new game is similar with the original one. This is the basic concept of the Property Exchange Method. Figure 1 shows the outline of the method. In this section, the method is explained in more details.

T. Umetsu et al. / Property Exchange Method for Automatic Generation

485

rule manipulate data type

data type

Cards that have the same or substitute data type card property property data type data type value value

learning target concept property property data type data type value value

exchange set of cards

set of learning target data

Figure 1. Model of Property Exchange Method First, we discuss what a card is. A card has a several properties. For example, a card called “seven of hearts”in the playing cards has “Ɔ” as the mark (it is also known as suit) and “seven” as the number. “Seven of hearts” is a card name. “Mark” and “number” are property names. “Ɔ” and “seven” are values of the properties. In the card games, the properties of a card are usually used with the properties of the other cards. Then, the way to use the properties are categorized into three, (1) distinguishing (the same or not), (2) ordering (less or grater, before or after), and (3) calculation. For example of Fan-tan, which is a major card game, the rules make players “put playing cards sequentially by number in each mark”. The players order the values of numbers and distinguish the values of marks. Therefore, the rules require cards to have characteristics that is “the values of the properties of the cards can be distinguished”, “the values of the properties of the cards can be ordered” or “the values of properties of the cards can be calculated”. Whether the manipulations of values can be carried out or not depends on the relations between values of the properties that have the same name, so that the characteristics are the relations. We call the relation “data type”. In other words, the rules require specific data type to the properties. Therefore, if a property has the same data type with another property, it is possible to exchange them without changing related rules. So, if we can prepare new set of cards that have the same data types with them of the original cards, we can make a new game by exchanging them without changing the rules. It is expected that the play of the new game is similar with the original one because the new set of cards that can be manipulate by the rules in the same way as the original cards. When the new card is composed of learning materials, the new game is a learning game of the learning. In the following subsections, the data type and the learning materials are explained in more details.

2.2 Data Type The manipulations of values can be carried out by using relations between values of the same property. We call the characteristics of the relations “data type” and categorize it into three. We call the attributes that can be distinguished “nominal scale”, the attributes that can be ordered “ordinal scale” and the attributes that can be calculated “interval scale”.

486

T. Umetsu et al. / Property Exchange Method for Automatic Generation

In nominal scale, values are only labels or names. If two values have the same name associated with them, they belong to the same category, and that is the only significance that they have. The only comparisons that can be made between variable values are equality and inequality. There are no "less than" or "greater than" relations among them, nor operations such as addition or subtraction. For examples, a mark of playing cards, color of card, name of elements in chemistry and name of countries are properties of nominal scale. In ordinal scale, values have order relations. Values represent the rank order (1st, 2nd, 3rd etc) of the properties measured. The natural numbers are ordinal. Comparisons of greater and less can be made, in addition to equality and inequality. However, operations such as conventional addition and subtraction cannot perform on this type. For examples, alphabet, ranking of production and experimental procedure are properties of ordinal scale. Interval scale has all the features of ordinal scale. In addition, equal differences between values represent equivalent intervals, so that differences between arbitrary pairs of values can be meaningfully compared. Operations such as four arithmetic operations can perform on values of this type. For examples, number of playing cards, electron numbers of elements in chemistry and years of historical events are properties of interval scale. The rules make players distinguish, order or calculate a value of a property. In other words, the rules require the property that has particular data type. If the rules distinguish values, the rules require the property has nominal scale, ordinal scale or interval scale. If the rules order values, the rules require the property has ordinal scale or interval scale. If the rules calculate values, the rules require the property has interval scale. For examples, the rules of Fan-tan, which is a playing cards game, is “put a card sequentially by number in each mark”. In this case a set of cards that has a property of nominal scale and a property of ordinal scale is minimum required.

2.3 Learning Style In learning, a learning material often has a name and several properties as well as a card of a card game. A property also has a name. A property has different value for each learning material. We usually have to memorize the values of properties of learning materials and their relations. For example, learners have to memorize values of properties of elements in order to check characteristics of the elements in chemistry. The element has properties, which are “period”, “group”, “electron number”, “atomic symbol” and so on. The characteristics of element depend on values of the properties, so that we memorize the values. Furthermore, the relations of elements should be memorized. For example, both lithium and sodium belong to the same group, so that they have similar characteristics. Memorizing the relation is also useful for checking characteristics of elements. We call such learning material “learning target concept”. One of activities for memorizing values of properties of learning target concept is checking the set of values, properties and a name of learning target concept many times. The activity makes learners associate values with the name of the learning target concept. For examples, we usually speak up the set many times, remind the set a number of times, and so on, for memorizing. As already mentioned, developers can generate a learning game that is similar with the original game by exchanging the set of cards of the game for the set of cards that are composed of learning target concepts when the learning target concepts have the data types that are required by the rules. In the learning game made by the exchange, players manipulate the values of the learning target concepts many times. The manipulating accompanies the process of checking the set of values, properties and a name of learning target concept, so that playing the game is useful for memorizing the learning target concept. In addition, the players distinguish, order and calculate the values of the learning

487

T. Umetsu et al. / Property Exchange Method for Automatic Generation

target concept, so that the players check relations of the values of the learning target concept. It contributes to memorize the relations. For example, the rules require players to “put an element sequentially by group in each period”. The players must check a set of values of groups and periods of element in order to manipulate them. The players also check order relations of the groups of the elements and equivalence relations of the periods of the elements in order to put the elements sequentially by group in each period.

3. Example of Property Exchange Method In this chapter, we explain an example of developing a learning game based on Property Exchange Method. First, a developer prepares information of a set of cards and learning target concepts. Information of a set of cards consists of “game name”, “card set name”, “card name”, “property name”, “minimum required data type for the property" and “value”. If the rules only distinguish the value, minimum required data type is nominal scale. If the rules order but don’t calculate, minimum required data type is ordinal scale. If the rules calculate, minimum required data type is interval scale. Information of a set of learning target concepts consists of “concepts set name”, “concept name”, “property name”, “data type of the property” and “value”. Table 1 and 2 shows examples of them. The rules of Casino calculate number of playing cards, so that require only a number property that have interval scale. Mark of playing cards is not described because the rules don’t manipulate it.

Game Card set name name Casino Playing cards

Concept set name Historical events

Table 1. Information of a set of cards Card name Property Minimum name required data type Six of hearts Number Interval scale Seven of heats Number Interval scale

Value 6 7

Table 2. Information of a set of concepts Concept Property Data type Value name name Egyptian Year Interval scale 1952 Revolution Place Nominal scale Arab Republic of Egypt Suez Year Interval scale 1956 Crisis Place Nominal scale Arab Republic of Egypt

Second, the developer is required to find exchangeable pairs of card set and concepts set. When the data types of the concept set are as same as or substitute data type for required data types for the property of the card set, they are exchangeable. Interval scale is substitute data type for ordinal scale or nominal scale. Ordinal scale is substitute data type for nominal scale. For example, the rules of Casino require interval scale in Table 1, so that playing cards of Casino and historical events that have interval scale are exchangeable pair. Third, the developer exchanges the pair by exchanging the card set name for concept set name, the card name for concept name, property name of the card for property name of the concept that has required data type, value of the card for value of the concept. The name of the concept, property name of the concept and the value of the concept are written on each card. For example of the card is “Egyptian Revolution. Year=1952” in the case of Table 1 and 2. Forth, the developer generates a learning game by exchanging the new card set for the original card set. In the case of Table 1 and 2, players manipulate year of historical

488

T. Umetsu et al. / Property Exchange Method for Automatic Generation

events by calculating in the new game many times, so that the players can memorize the year of historical events and their interval.

4. Learning Game Generator In this chapter, we introduce a learning game generator based on the method. The experimental use of it is also reported.

4.1 System Specification

Learning game designer

Card game designer

Computer-based game (1) card (1) property

Set of concepts (1) concept (1) property (1) data type

(1) data type

(1) value

(1) value

(3) rule

(2) rule explanation for game

system Pair-finder

Card generator

(a) Set of cards for learning game

(c) Computer-based learning game

Explanation generator

(b) Rule explanation for learning game

Figure 2. System chart Figure 2 shows system chart. There are 3 kinds of outputs. When developer inputs (1) information of a set of cards and learning target concepts, the system outputs (a) information of new set of cards for the learning game. An example of (1) information is Table 1 and 2. An example of (a) information is “Egyptian Revolution. Year=1952”. When the developer inputs (1) information and (2) rule explanation of the card game, the system outputs (b) rule explanation of the learning game. The (2) rule explanation has to be text independent particular card by using the words “card set”, ”property”, “value” and so on. For example, when (2) rule explanation is “put seven of hearts, eight of hearts and nine of hearts”, the system cannot output. When (2) rule explanation is “put cards sequentially by value of property_1 in each value of property_2”, the system can output “put cards sequentially by value of year of historical event in each value of place of historical event”. If the developer gets the (a) information and (b) onto paper, players can play the learning game with printed cards following the rule explanation of the learning game. When developer inputs (1) information and (3) computer-based card game, the system outputs (c) computer-based learning game. The system makes (a) information of new set of cards by (1) information of a set of cards and exchange (a) the information of new set of cards for the data of cards of (3) computer-based learning game in order to generate (c) computer-based learning game.

T. Umetsu et al. / Property Exchange Method for Automatic Generation

489

4.2 Evaluations We conducted two experimental evaluations in order to access the system. One is an experiment for confirming the system can output various learning games. The other is for conforming the system can output learning games that have the same characteristics with the original games. In the first experiment, we tried to input (1) information of 5 games (Memory, Speed, Fan-tan, Casino and Trick-Taking) and 4 learning target concepts (elements, historical events, procedure of using microscope, English words) and (2) rule explanations of 5 games. Each set of cards of the 5 games was different data types, and the 4 learning target concepts were also different data types. When we input them into the generator, the generator output (a) sets of card and (b) rule explanations for 50 learning games. We confirmed that we could play the 50 learning games by getting (a) sets of card and (b) rule explanations onto papers. The results suggest that the system can output various and many learning games. In the second experiment, we have developed a computer-based Trick-Taking game, which is playing cards game on Windows. When we input the computer-based game and 3 learning target materials (elements, historical events, procedure of using microscope), the system output 13 computer-based learning games. We asked six subjects to play the 13 games and answer questionnaires. The six subjects often play Trick-Taking and two of six subjects are ranked player on Internet-Trick-Taking. The subjects suggested that they played each 13 games twice in order to understand the derivation of Trick-Taking. One play took about 5 minutes so that they play the 13 games for 130 minutes. The questionnaires are as follows. z Q1 was “How much do you know the learning target concepts?” 5. completely-known 4. much 3. so-so 2. a little 1. completely-unknown z Q2 was “Do you accept the activity of playing as a game?” 5. completely 4. almost 3. so-so 2. incompletely 1. different z Q3 was “Do you think that the learning game is useful for memorizing?” 5. very good 4. good 3. so-so 2. bad 1. awful z Q4 was “Do you think the learning game keep the characteristics of the original game?” 5. completely 4. almost 3. so-so 2. incompletely 1. different z Q5 was “Do you line the learning game?” 5. very much 4. much 3. so-so 2 a little 1. dislike The results are shown Table 3. Thirteen alphabets from A to M mean the 13 learning games. Numbers in the Table mean averages of the answers. The distributions of the answers are normal distributions. The results of Q2 and Q3 suggested that the generated games were accepted as useful learning games. The results of Q4 and Q5 suggested that most learning games kept the characteristics of the original games, but Game-E, -F -G, -H and -I didn’t keep the characteristics. We asked the subjects Q1 in order to investigate repercussions of knowledge about the learning target concepts on the results of Q2-5. The results of Q1 had no influence on other questionnaires, so that difficulty of memorizing has little concern with the learning target concepts.

Q1 Q2 Q3 Q4 Q5

A 4.7 5 4.5 4.2 4.7

B 4.7 5 4.5 4.2 4.7

C 4.5 5 4.5 4.2 4.7

Table 3. Results of Q1-Q5 D E F G H I J K L M 4.5 4.5 4.5 4.7 4.7 4.5 2.5 2.5 2.5 4.8 5 5 5 5 5 5 5 5 5 5 4.3 4.3 4.3 4.5 4.5 4.3 4.8 4.8 4.8 4.5 4.3 2 2 3.5 3.5 1.8 4.8 4.8 4.8 4.3 4.3 2.8 2.8 4.2 4.2 2 4.7 4.7 4.7 4.3

490

T. Umetsu et al. / Property Exchange Method for Automatic Generation

Table 4. Results of Q4, 5 after the adjustment E F G H I Q4 4.7 4.7 4.7 4.7 4.7 Q5 4.7 4.7 4.7 4.7 4.7 Closely examining the Q4 and Q5 results, we think that the rules and data types are important factors to provide characteristics of the game, but there are other important factors. We think the number of values and the distribution of values are changed in E, F, G, H and I, so that they didn’t keep the characteristics. Therefore, we adjusted the number and distribution of values in E, F, G, H and I as the original game in order to confirm the thinking. After the adjustment, we asked six subjects to play them and answer questionnaires again. The results are shown Table 4. The results suggested that if the developers can allow for the conditions of the number and distribution of values, it is expected to remove the problem. Although the subjects answered these learning games were useful for learning, we didn’t investigate the learning effectiveness of these learning games. However, the system could output the learning game that showed an effect on learning [6]. Under the circumstances, we think that Property Exchange Method can support designing learning games and the system on the method can generate computer-based learning game automatically. Although the system occasionally generates a learning game that doesn’t keep the characteristics of the original game, we can design another learning game that keep the characteristics from the learning game by adjusting the values of properties.

5. Conclusions In this paper, we have proposed Property Exchange method, which is a method to transform an existent card game into a learning game by exchanging learning target concepts for cards of the game while keeping characteristics of the game. The method only deals with card games. Instead the method is so concrete method that the application based on the method can generate computer-based learning games automatically from computer-based game, learning materials and their data types. We confirmed that the application could output computer-based learning games that keep characteristics of the original game. In future work, we should investigate factors that produce characteristics of a game in detail, because the application rarely output computer-based learning game that didn’t keep characteristics of the original game in the experimental use of the application.

References [1] T.W. Malone (1981) Toward a Theory of Intrinsically motivating instruction. Cognitive Science, Vol.5, pp.130-145. [2] M. Prensky (2001) Digital Game-Based Learning. McGraw-Hill. [3] M.M. Klawe (1998) When Does The Use Of Computer Games And Other Interactive Multimedia Software Help Students Learn Mathematics? http://www.cs.ubc.ca/nest/egems/reports/NCTM.doc [4] K. Maragos, M. Grigoriadou (2005) Towards the design of Intelligent Educational Gaming Systems. Proc. of AIED05 WORKSHOP5, pp.35-38. [5] H.M. HALFF (2005) Adventure Games for Science Education: Generative Methods in Exploratory Environments. Proc. of AIED05 WORKSHOP5, pp.12-20. [6] T. Umetsu, T. Hirashima, A. Takeuchi (2004) Partial Exchange Method for Designing Learning Games and Its Application. Proc. of ICCE2004, pp. 257-264.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

491

Development and Validation of an Animation-Based Test in the Area of Earth Sciences Huang-Ching WU a, Chun-Yen CHANG b GDepartment of Earth Sciences National Taiwan Normal University Taipei, Taiwan [email protected] [email protected] Abstract: This study tries to develop an animation-based test (ABT) in the area of Earth science. The advantages of ABT includes: (1) To present more authentic situation in an animated testing environment; (2) To assess the learning outcomes with appropriate validity and reliability; (3) To be a more attractive way of testing. The content of the test focuses on four domains in Earth science including astronomy, meteorology, oceanography and geology. “Attitude toward Animated Assessment Scale”ʳ (AAAS) was adapted in this study in order to explore examinees’ perceptions and attitudes toward ABT. This study found the ABT to be generally validate and reliable and test-takers had better performance in the animation-based testing environment. Keywords: animation, computerized assessment

Introduction With the developments of economic and easy-to-use software, visuo-spatial adjuncts such as animation can be created easily to support learners in constructing mental representation. Recent studies focus on using animation to enhance learning, and the research outcomes are not thoroughly equivalent, which means that animation usefulness is dependent to various environments. However, the studies which mentioned about the effects of combining test with animation are quite a few. On the other hand, from the prospective of assessment delivering, students tend to fail to get right answers in testing process because the situation attached to it is hard to be understood. Since the achievements evaluation covers achievement and value judgments, up-mentioned problem should be avoided in order to enhance the objectivity of the test. Thus, this system aims to reduce the difficulties of reading items by using animation as an adjunct.

1. Literature Review 1.1 Instructional roles of animation In the field of science education, animation is a kind of dynamic representations and there are lots of researches focusing on the impact of animation in science learning. However, purposes of animation-related studies change slightly in recent years. Previously, researchers noticed the uniqueness of animation and devoted to analyze its superiority to

492

H.-C. Wu and C.-Y. Chang / Development and Validation of an Animation-Based Test

conventional representation in educational use. As people became increasingly familiarized with animation, and meanwhile the inconsistency results to animation’s superiority were continually being observed, researchers started to figure out how should we use the animation to its optimum, thereby strategy-oriented animation and theoretical-designed animation were emerged. In the future, it merits more studies to address to what extent and to whom does the animation influence the most. Before that, we will outline several instructional roles for animation. Based on lots of previous studies addressing the issue of dynamic display, Park and Hopkins summarized some roles of animation and conditions for animation use, which provide a clear sight for instructors to develop dynamic device and to decide how to use it properly [1]. z z z z z

As an attention guide. As an aid for illustration. As a representation of domain knowledge. As a device model for forming a mental image. As a visual analogy or reasoning anchor for understanding abstract and symbolic concepts or processes.

1.2 Individual differences This study tries to clarify the effects of animation in students’ testing performance. It is reasonable that different people will have different gaining from the same adjunct. The effect of visuo-spatial adjuncts aids depend on prior knowledge, cognitive abilities, and learning skills [2]. It is pointed out that pictorial illustrations can have a decorative and motivational function in materials for first graders who learn to read [3]. Since individuals seem to be experts in cognitive economy, they are therefore skillful in finding shortcuts in problem solving. Even though animation is powerful in providing additional working memory for students, it is suggested that one should not provide alternative routes for understanding when the learner should be trained in understanding a specific kind of representation [2].

2. System Description 2.1 Test development In total 20 simulation multiple-choice items had been developed: 5 items for geologic concepts, 5 items for oceanographic concepts, 5 items for meteorological concepts and 5 items for astronomical concepts. Source of the concepts selected from the textbook, called “The Fundamental Earth Science” adopted in 10th grade native senior high schools currently. To certify the correctness and validity, investigator solicited the opinion of competent judges from the expert. One science education expert and 5 earth science experts judged the quality of items.

2.2 Internal consistency of ABT The Kuder-Richardson Formula 20 (KR-20) was used to determine the reliability coefficient of ABT. The reliability coefficient was about 0.66.

H.-C. Wu and C.-Y. Chang / Development and Validation of an Animation-Based Test

493

2.3 Item analysis According to empirical testing results, difficulties of ABT and GBT were fairly the same, while the difficulty of ABT was calculated to be 0.5 and GBT was 0.49. Since 0.5 is quite an acceptable number to explain how difficult a test is ought to be, these two engaged tests were fundamentally adequate for test-taker since they did not find them too hard or too easy. In addition to difficulty, discrimination is also important for a test. Whether test designer or instructor look forward to distinguish students’ achievement by the control of item quality. A question with good discrimination stands for the meaning that all the high-achieved students will answer it correctly, while all the other low achievers will answer it incorrectly. Reasonably, this is an idealized situation since there are inevitable factors which may influence this coefficient. In our study, discrimination coefficient of ABT was .36, and GBT was .35. Ebel and Frisbie provide standard for evaluating the discrimination, and they mentioned that a high quality test should has at least discrimination of .30 in average, while .40 or higher stands for a even better item quality.

3. Method 3.1 Participants A total of 327 10th grades students participated in this study. These participants were chose randomly from 640 10th grades students in one of Taipei’s normal high school. Ranking of this high school is in the middle place among over 20 other senior high schools.

3.2 Design A comparison-group experimental design was employed. 327 students were randomly assigned to group 1 and group 2. ABT and GBT were both manipulated by each group. Group 1 started with SBT and took GBT afterwards. Group 2 reversely started with GBT and took ABT.

4. Result and discussion 4.1 Validation of animation assessment Schuwirth [4] states that performance on assessment should increase with increased expertise, which should be an indication of stability and validity of the test. In this study, participants were entered the testing room with different prior knowledge, while several of them has finished standard earth science curriculum and the others were still on the half way learning it. Therefore, ABT ought to be able to detect this known difference among participants. Empirically, students in the “learned” group scored 55.41 in average, and the other students in the “learning” group scored 47.02, with separate standard deviation of 15.54 and 14.92. Indeed, learned groupers scored significantly higher than learning groupers ( t=4.714, p<0.05). On the other hand, there was a positive relationship between ABT and other earth science assessments, scores ranged from .3 to .6. In conclusion, the validation of ABT was acceptable, however, there are still spaces for improvements.

494

H.-C. Wu and C.-Y. Chang / Development and Validation of an Animation-Based Test

4.2 Difference between two testing scores We assume the usefulness of animation when it is combined with test, and the result of this experiment was positive from figure 1. According to the result, there did exist difference in ABT and GBT groups. ABT group scored 57.17 while GBT group scored 52.88 (t=2, p<0.05). The difference is nearly significant with the effect size of .3.

4.3 Brief Conclusion Obviously, students were more or less affected by animation in testing process. This could be explained by the potential scaffold provided by animation. The advanced analysis is about the interaction between individual difference and animation test performance. This will be addressed in the future studies since we have found out that the animation did work in assessment, but we would devote more efforts continually to figure out who benefit from it the most. ˄˃˃

ˆ˅˃

ˋ˃

˦˶̂̅˸

ˉ˃

ˇ˃

˅˃

˔˕˧ʳ̆˶̂̅˸

˚˕˧ʳ̆˶̂̅˸

˃

˦˕˧

˚˕˧

˔˿˿ʳ˚̅̂̈̃

˙˼˺̈̅˸ʳ˄

References [1] Park O and Hopkins, R. (1993) Instructional conditions for using dynamic visual displays: a review. Instructional Science, 21, 427-449 [2] Schnotz, W. (2002) Towards an Integrated View of Learning From Text and Displays. Educational Psychology Review, 14 [3] Carney, R. N. and Levin, J. R. (2002) Pictorial Illustrations Still Improve Students' Learning from Text. Educational Psychology Review, 14 [4] Bakx, A.W.E.A. and Sijtsma, K. (2002) Development and evaluation of a students-centered multimedia self-assessment instrument for social-communicative competence. Instructional Science, 30, 335-359

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

495

Weaving Pedagogy into Gaming: Learning Design Principles for Developers Yam San CHEE, Yi LIU, Khai Seng HONG National Institute of Education, Nanyang Technological University, Singapore [email protected] Abstract: The current level of interest in gaming for learning is not matched by availability of detailed guiding principles and case studies of how learning designers should go about designing computer games that are pedagogically effective. In this paper, we share our learning design experience in the creation of Ideal Force, a multiplayer 3D computer game for helping students to overcome common misunderstandings related to projectile motion. Based on lessons learned through usability testing and iterative design, we articulate three principles of learning design for educational game developers. Keywords: pedagogy, game based learning, design principles, usability testing

1. Introduction The current generation of 3D computer games and videogames has drawn increasing attention from schoolteachers and researchers. Educators have begun to recognize the power of computer games to motivate students and to sustain their active engagement. The recent work of game theorists such as Gee (see, for example, [1] [2]) has helped to illuminate our understanding of the power of videogames for student learning and literacy. While there is much research literature that investigates the use of computer games for learning and education (for example, [3] [4]), there is very little literature that deals with games that researchers themselves have developed. This state of affairs is understandable given the time, effort, cost, and complexity of developing 3D games. Notwithstanding, as more researchers apply themselves to developing their own games for use in the classroom, there will be an increasingly urgent need for design principles that can guide the research community in producing effective learning designs for games. In this paper, we share our design experience in the creation of Ideal Force, a multiplayer 3D computer game for helping students to overcome misunderstandings related to projectile motion. Based on lessons learned through testing and iterative design, we articulate three principles of learning design for educational game developers. 2. Background to the Game The game Ideal Force was designed for 13-16 year old students. Our game design philosophy is rooted in active learning and learning by doing principles. Arising from prior work on the physics of projectile motion [5], we developed a multiplayer computer game that seeks to help students overcome three common misunderstandings related to projectile motion. Here, we share the game design evolution for each misconception. 2.1 Misconception 1: Relation between Angle of Elevation and Horizontal Distance A common curriculum goal is to have students understand the relationship between the

496

Y.S. Chee et al. / Weaving Pedagogy into Gaming

angle at which a projectile is fired and the horizontal distance that it travels. In particular, students should understand that the maximum horizontal distance is attained when the angle of elevation at which a projectile is fired is set to 45 degrees. In Ideal Force, we initially presented students with the goal of hitting an enemy base which was positioned such that it could be hit by a tank missile only if (i) the tank was at the shoreline, and (ii) the tank fired the missile at an angle of 45 degrees. The original design intention, shown in Figure 1, was that students should be able to learn through experimentation that any angle of elevation other than 45 degrees would cause the tank missile to fall short of the enemy base. During the testing, we found that, with the time pressure of game play, students typically acted hurriedly and did not experiment systematically to learn the target concept. Consequently, we redesigned the game to create an intervening graph representation that allowed students to observe the relationship between the angle of elevation of the missile and the horizontal distance it would travel. The graph updated as a user changed the angle of elevation using the control panel shown on the bottom right of Figure 2. This redesign foregrounded and aided students in reasoning about the relationship between angle of elevation and horizontal distance. The graph served as a scaffold that helped to direct students’ attention to pertinent variables, thereby avoiding their trying to hit the enemy base in a random, undirected manner. 2.2 Misconception 2: Constant Vertical Speed of Descending Projectiles The second common misconception relates to the fact that the horizontal and the vertical components of a projectile’s motion are always independent. However, students commonly believe that an object free-falling vertically from an aircraft will strike the ground before another released at the same time with some forward velocity. To counter the above misconceptions, we designed the game play to require students to successfully hit two shield-generating domes simultaneously using two separate bombs while flying a helicopter. The firing control panel in Figure 3 consisted of three buttons. Two of the buttons controlled the firing of individual bombs, in succession, while the third button allowed the two bombs to be fired simultaneously. We intended that students would experiment with the different button combinations and (eventually) learn that only the dual-bomb combo button would allow them to achieve their goal. In practice, they were not able to discern a clear causal relationship between the single bomb and dual bomb buttons that they activated. Hence, a revision of design was needed (see Figure 4).

Figure 1. Initial design

Figure 2. Final Design

Figure 3. Initial design.

Figure 4. Final Design

In the final design, the control panel for firing bombs at the domes was augmented with a graph representation that allowed students to set the speed and time delay for firing the bombs once the “fire” button was pressed. The student’s task was further simplified by allowing them to drag slider controls for the speed (ie. horizontal velocity) of the bombs when fired. As the slider was dragged to the right, the encircled target area at the end of the parabolic trajectory line moved forward, thereby allowing the students to be sure that

Y.S. Chee et al. / Weaving Pedagogy into Gaming

497

the targets would be hit. What was left for the students to focus on was to choose the time delay separating the firing of the two bombs after the “fire” button was pressed. With this final design, students would succeed in their goal only if they set the time delay for both bombs to some equal value. The provision of the intermediate representation provided students with a basis for reasoning about the pertinent variable at hand (time delay) while shielding them from the complexity of having to hit the targets successfully. 2.3 Misconception 3: Deceleration and Acceleration of a Projectile over its Trajectory Students often have the misconception that, over the path of its parabolic trajectory, a projectile first increases speed before it decreases speed and finally stops. This kind of thinking implies adherence to impetus theory [6]. We designed the game play such that tanks belonging to one team need to intercept incoming missiles from enemy tanks using one of two types of laser beam. Firing and striking an incoming missile using a red laser beam works only if the interception occurs on the part of the trajectory of the missile where it is decelerating. Conversely, a green laser beam only works if the interception occurs during a missile accelerating situation. This design was intended to get students to think explicitly about how a missile changes speed over the course of its trajectory. In our first design, shown in Figure 5, students had to identify the predicted path on the X–Z plane and the Y–Z plane by using the mouse to click within the planes. Students then had to choose the red or green laser beam to fire. However, this did not work out well. As the learning focus was not on predicting the missile motion in 3D space, the interception control pane was simplified by “flattening” the 3D representation into a 2D graph in the final design (see Figure 6). Identifying the accelerating and decelerating portions of the missile’s path now became the student’s primary task. Moreover, a visual indication of the size of the vertical and horizontal vectors was also added. These vectors, updated in real time, were introduced to help students engage in reasoning about the projectile’s change in vertical speed over the course of its trajectory.

Figure 5. Initial missile interception design.

Figure 6. Final missile interception design.

3. Three Principles of Learning Design for Curriculum Games Based on the design experiences, we set out three learning design principles that other researchers involved in the design of curriculum games may find useful. (A fourth principle in the original paper has been removed due to space limitations.) 3.1 P1: Provide Mapping from Virtual World Level to Intermediate Representation Level Intermediate level representations can greatly assist students bridging the space between direct experience and abstract formalisms in which the terms and language of a scientific

498

Y.S. Chee et al. / Weaving Pedagogy into Gaming

domain are usually expressed. Providing suitable representations that map what students experience directly in a game world to an intermediate representation level (e.g. a graph depicting the trajectory of a missile) can greatly facilitate the assimilation and internalization of target domain understanding. Figure 2 illustrates a bridging representation that maps from the experience of game play to thinking in terms of the target scientific domain. The representation is further enhanced by foregrounding the key pertinent variables—angle of elevation and horizontal distance traveled—that change as the student adjusts the angle of the tank’s firing turret. 3.2 P2: Provide Intermediate Representations as Control Interfaces to Induce Domaindependent Expert Thinking We learned that in addition to using intermediate representations, we could also design those representations as control interfaces that “encourage” students to think in domainrelated terms before they act on the game interface (and not only after they had acted). Figure 6 illustrates using an intermediate representation, a graph, as a control interface for action based on domain thinking. Using the representation in this way, student thinking is scaffolded in domain-like terms early, and students are not left to flounder helplessly later. 3.3 P3: Simplify Game Play and Interaction to Make Users Focus on Learning Objectives During usability tests, we found that students spent much time on portions of the game that were not closely allied to learning objectives. For example, in the helicopter bombing activity, students concentrated on hitting the dome targets on the ground rather than focusing on hitting two domes simultaneously as was required. In addition, the initial design for intercepting a missile using a laser beam (shown in Figure 5) was complex and not well aligned to achieving the learning objective. Subsequent redesign of these two elements helped to simplify game play. Student effort was more productive, and game interaction was simplified. Thus, the game interface became more usable. 4. Conclusion In this paper, we shared our design experience in developing a 3D computer game intended to help students overcome misconceptions related to projectile motion. We described how our learning design evolved for the three misconceptions that the game addresses and articulated three learning design principles intended to help game designers achieve effective game designs aligned to learning objectives. References [1] Gee, J. P. (2003). What Video Games Have to Teach Us About Learning and Literacy. NY: Palgrave Macmillan. [2] Gee, J. P. (2005). Why Video Games are Good for Your Soul: Pleasure and Learning. Melbourne, Australia: theLearner.com. [3] Amory, A., Naicker, K., Vincent, J., & Adams, C. (1999). The use of computer games as an educational tool: Identification of appropriate game types and game elements. British Journal of Educational Technology, 30 (4), 311–322. [4] Kafai, Y. B. (2001). The educational potential of electronic games: From games-to-teach to games-tolearn. (http://culturalpolicy.uchicago.edu/conf2001/papers/kafai.html; retrieved 27 October 2005.) [5] Chee, Y. S., & Liu, Y. (2004). Grounding concept in percept: Learning physics experientially in multiuser virtual worlds. In Kinshuk, et al (Eds.), Proceedings of the 4th IEEE International Conference on Advanced Learning Technologies (pp. 340–344). Los Alamitos, CA: IEEE Computer Society. [6] McCloskey, M. (1983). Intuitive physics. Scientific American, 248 (4), 114–122.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

499

VR Edutainment Material Interlude for Dynamics Experiment and the Development Platform Prelude Yuma Hanafusaa, Takashi Inouea, Hiroyuki Tominagaa Toshihiro Hayashib, Toshinori Yamasakia a Faculty of Engineering, Kagawa University, Japan b Information Technology Center, Kagawa University, Japan [email protected] Abstract: VR teaching materials are useful for physics education. We propose Interlude framework for self-learning of various dynamics experiments. It is operated by HI input devices for 3D movement. It offers 3DCG demonstrations of dynamics phenomenon and trial exercises with several parameters in microworld. The purpose is to promote interest about dynamics for beginners, and to nourish their physical sense. We introduce non-reality visual expression as edutainment methodology to emphasize the important entities of dynamics scenes intuitively. Interlude materials are based on common platform Prelude, which contains the runtime interface and component library according to our aspect model. Keywords: VR edutainment, development platform, dynamics experiment learning

Introduction Recently, teachers attempt to use VR teaching materials for physics in high school. The most popular of them is for elementary dynamics experiments [1][2][3]. It is based on a microworld, which implements dynamics laws at minimum required for educational purpose. It offers 3DCG demonstration of dynamics phenomenon. It includes an observer as user with interactive operation to feel active impression of presence. A user may use HI (human interface) input devices for 3D movement in VR experience. It promotes interest and concern about dynamics for beginners by walk-through operation and observation from multiple viewpoints. It increases the opportunity of try-and-error experiments repeatedly in various different conditions. It gives them enjoyable learning and intuitive understanding for correlation of cause and result about dynamics phenomenon. We proposed Interlude framework for self-learning of elementary dynamics experiments [4]. It contains various VR teaching materials with 3DCG common components. It adopts flexible presentation and educational game exercises as edutainment methodology. It is based on common platform Prelude to construct them efficiently. We introduce both Interlude and Prelude. 1. VR edutainment materials Interlude and the development platform Prelude Interlude material treats learning fields about Newton's elementary dynamics with mass point; velocity and movement, force and equation of motion, acceleration and gravity, momentum and impulse, and energy and work. It describes typical scenes of popular phenomenon in fundamental experiments to understand dynamics laws qualitatively. For example, it sets angle of a slope and its coefficient of friction, and puts a small ball on the slope as start position. It offers two kinds of input devices (Figure 1). A user pushes and

500

Y. Hanafusa et al. / VR Edutainment Material Interlude for Dynamics Experiment

twists entities intuitively in VR laboratory with an entity control device like 3D mouse. At the same time, he goes in walk-through and observes from multiple viewpoints with a viewpoint control device like head tracker. It also uses output devices, like HMD (head mount display) and 3D display for stereovision. To construct various Interlude materials with integrated style efficiently, we implement common development platform Prelude on Java language. It provides runtime interface and component library. We adopt it3d interface based on JNI (Java Native Library) to control various HI input device [5]. It treats 3D mouse Magellan and haptic pen PHANToM. We extend it for more kinds of devices uniformly. On the other hand, we need VR component library which can describe microworld and educational information. We construct graphical front-end and physical back-end of Interlude materials independently by separating microworld aspect and presentation aspect. We prepare each component library as extended Java3D. We adopt scene graph of Java3D for presentation structure. We also use collision detection in Java3D for physics engine. And we add some components and functions about various flexible outlooks for some educational purpose. You can change different presentation style for the same experiment with reusability of components.

Figure 1: Prelude platform and Interlude material

2. The runtime interface of Prelude Prelude platform provides the runtime interface. It consists of device control module, internal execution module and image display module. The device control module treats two kinds of HI input devices as entity control and viewpoint control. The entity control device manipulates subjects for observation as 3D pointer in VR laboratory. The viewpoint control device moves 3D viewpoint and looking direction by head tracking sensor. It makes easy to observe dynamics phenomenon from multiple viewpoints. In the case of throwing a small ball with the gravity, a user observes uniform motion from horizontal direction, and uniform accelerated motion from vertical direction. The module offers several operation device classes to control both devices simultaneously. It prepares data pack structure for various types according to each device uniformly. The data pack contains state data and signal data. The state data are continuous 3D position at adequate sampling interval and device condition like input mode. The signal data are instant action event and system exception. The data pack reduces some redundant state data, while acquiring all signal data exactly. The data analyzer in the internal execution module reads data pack from the data buffer. It calculates 3D position and movement of all entities in a microworld. It decides object behavior and causes some dynamics phenomenon like collision as physics engine. The VR player interprets Interlude scenario and illustrates 3DCG demonstration in VR laboratory. The image display module outputs to display control device with stereovision.

Y. Hanafusa et al. / VR Edutainment Material Interlude for Dynamics Experiment

501

3. The component library for Interlude We propose construction model for Interlude edutainment materials to construct them efficiently. We consider the difference between physical model and viewing model. A mass point as dynamics term is zero size object not to be seen in actual, while you may illustrate a small ball as the outlook. We offer the component library as extended Java 3D according to microworld and presentation aspect in the model. These components have common basic functions. Interlude developer may use extended objects to define subclass by inheritance for the purpose. The microworld aspect as physical model contains entity and phenomena object which describes physical condition and conceptual dynamics scene. The presentation aspect as viewing model represents the outlook and user-interface of the microworld. Both aspects are separated and utilized independently. They are just combined as common BranchGroup in scene graph of Java 3D. The feature enables to reuse a set of objects in each aspect like design pattern. You may exchange the presentation against the same microworld. The microworld aspect is consists of World, Entity, Phenomenon and Force class. World class is a container for the others. Entity class contains some description elements in typical dynamics scenes. Unit class is an observed target like mass point and small ball. It has several physical attributes; mass, energy, velocity and force. Mechanism class is experiment equipment like spring, which gives action to Unit class. Background class is a still and fixed object. Phenomenon class is a local physics engine according to a simple dynamics law. It gives physical interaction to concerned Entity object with Force object. We treat only elementary dynamics with mass point. Phenomenon object has simple calculation about gravity in free fall, friction on slope, resistance from reaction and collision. The presentation aspect consists of four class packages. For edutainment materials of VR laboratory, we think that both reality and non-reality presentation is important. The reality one is friendly experiential manipulation and practical expression of physical behavior. The non-reality one is visual impressive expression with deformation. The former is represented by VS (visual shape) and EA (event acceptor) object. The latter is represented by ES (expression support) object. AF (annotation folder) objects shows educational information embedded in VR laboratory seamlessly. VS object is a standard shape as default corresponding to each physical entity in the microworld aspect. The primitive one is a versatile basic shape of Java3D added TransformGroup in scene graph. The compound one consists of some primitives for specific experiment equipment. ES object does not have physical entity as ghost in presentation aspect. It works for observation support and emphasizing dynamics phenomenon to remark important physical attributes and changing for friendly interest and intuitive understanding. For observation support, trace line represent movement of observed target in VR space. Vector shape expresses force object for decomposition and resultant force using the parallelogram [6]. Symbol shapes and deformation texture show human impression to emphasize dynamics phenomenon, which may be used in cartoon and animation. For example, a slope with friction is illustrated with rough texture surface. An instant of collision is illustrated with star symbols, of which number shows degree of impulse. AF object is GUI component for 3D frame. EA object is a tag area at specific part of an entity to accept user event from manipulation by HI devices. If it is clicked by a pointer, a linked frame will open. If another entity moves through it as internal event, a new experimental scene starts as the next step. 4. Some examples of Interlude edutainment materials First, we explain the construction method of microworld. We mention a simple collision experiment with two mass points. You put two instances of mass point as physical entity in a world object. The mass point has 1.0 radius size as minimum value in Java3D for collision

502

Y. Hanafusa et al. / VR Edutainment Material Interlude for Dynamics Experiment

detection. You connect two mass points to a collision phenomenon object. The collision object calculates momentum by the conservation law and the restitution coefficient. For another example, we mention a descent experiment on a slope of mass point with friction. In this case, you bind the mass point and the slope to a descent phenomenon object. The descent object calculates resistance and friction force with the coefficient and the slope angle. It determines fixed acceleration of the mass point. It causes changing of velocity and position as uniform acceleration movement along the slope. Second, we explain the combination of the presentation aspect and the microworld aspect. You create VS objects with adequate parameters, such as shape, size and location, according to physical entity objects. You construct TransformGroup as a root node, which has two BranchGroup as sub-tree from both aspects (Figure 3). The presentation aspect overlaps the microworld aspect in VR space of Java3D. You see a small ball with 10 radius size as the outlook, while you really observe collision of a mass point. Moreover, you may adopt non-reality presentation. You put a star symbol near collision point, of which color and texture shows degree of impulse.

Figure 2: Examples of presentation objects

Figure 3: Implementation by Java3D

5. Conclusion We introduced VR edutainment material Interlude about various elementary dynamics experiment for self-learning of beginners. We designed common development platform Prelude for Interlude. The runtime interface controls HI input devices for 3D movement and describes 3DCG demonstration by Java3D. We considered the construction model with the microworld and presentation aspect. We implemented a prototype of the runtime interface. We prepared basic components. We showed examples with non-reality presentation. In future works, we propose XML description for dynamics scenario and implement all modules and components to realize various Interlude materials for practical use. Acknowledgments This research has been made possible through Grant-in Aid for Scientific Research (B)(17300269) from Japan Society for the Promotion of Science. References [1] Interactive Physics, http://interactivephysics.design-simulation.com/IP/index.php [2] M. Inoue, Y. Matsubara, et.al, (2005), VR-based dynamics learning system using haptic device and its evaluation, ICALT 2005 (Kaohsiung), 917-921. [3] N. Avradinis, S. Voshinakis and T. Panayiotopoulos, (2000), Using Virtual Reality Techniques for the Simulation of Physics, 4th Systemics, Cybernetics and Informatics International Conference. [4] Y. Hanafusa, T. Inoue, et. al, (2006), Reality and non-reality of Edutainment materials Interlude with VR simulation for dynamics experiment, ED-MEDIA 2006 (Orlando), 2765-2772. [5] it3d (Interactive Toolkit library for 3D applications), http://ship.nime.ac.jp/~it3d/ [6] A. Kageyama, Y. Tamura and T. Sato, (2000), Visualization of vector field by virtual reality, Theoretical Physics Supplement, no.138, 665-973.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

503

Using the “Record-Replay” Function for Elaboration of Knowledge in Educational Games Junjie Shang, Morris S.Y. Jong, Fong-Lok Lee, Jimmy H.M. Lee, Marti K.H. Wong, Eric T.H. Luk, Kevin K.F. Cheung Centre for the Advancement of Information Technology in Education The Chinese University of Hong Kong [email protected] Abstract: Some scholars argue that simply using an educational game does not ensure that learners can generate the kinds of understandings that educators might desire. In order to maintain students’ interest in playing the game and, at the same time, helps them to learn, some scholars argue that reflection and debriefing during playing was the most effective approach. In this paper, we discuss the “Record-Replay” function, which is one of the technical and educational innovations in VISOLE that enables teachers and students to replay recorded gameplays, in order to help students to re-examine their decisions made while playing the game. Teachers can also guide student in making what-if analyses, making it a good tool for reflection and debriefing. Besides, “Record-Replay” allows teachers to assess students, and researchers to study and investigate the effectiveness of the game. Keywords: Record-Replay, educational games, debriefing, assessment, VISOLE

Introduction In recent years, with the rapid development of computer games, more and more educators believe that games can be applied into education (e.g., [1]). In fact, many research findings indicated that games could arouse the students’ intrinsic motivation [2] , and can improve the students’ basic skills such as eye-hand coordination [3], problem solving skills [4], and collaborative skills [5]. In addition, the games can facilitate affective learning, active learning, situated learning, and collaborative learning (e.g. [1] [6]). However, some studies also showed although teachers believe that games are able to motivate students to learn, they are not sure “how much” the students could learn from it. Moreover, in some commercial simulation games, the players are only motivated to “Win the Game” which may not yield any knowledge gain [7]. Therefore, the challenge for educational games is to keep students’ interest in playing the game and at the same time to help them to learn. Some scholars (e.g., [8] ) suggest that reflection and debriefing during or after playing sessions is the most effective approach. Adopting this perspective, we designed a number of methods to help the teachers and students to conduct the reflection and debriefing in the project of VISOLE (Virtual Interactive Student-Oriented Learning Environment) [9]. Among them, the “Record-Replay” function is a technical and educational innovation. With it, teachers can easily check students’ actions in the game and guide students in the debriefing and reflection process. In addition, it is also a good tool for teachers to assess the students’ performance and for researchers to investigate the effectiveness of the game. 1. VISOLE 1.1 The Concept VISOLE is a learning style that uses a virtual game environment to facilitate learning. The web-based game environment is a simulation of the real world where students participate as “citizens” and take part in shaping the development of the virtual world. It provides a platform for participants to apply the theoretical knowledge to solve problems in a near-real

504

J. Shang et al. / Using the “Record-Replay” Function for Elaboration of Knowledge

environment, as well as to develop high-level skills for communication and problem solving in addition to subject knowledge [9]. VISOLE is usually divided into three stages : Stage 1: Multi-disciplinary Scaffolding In the first part, teachers act as facilitators to guide students to construct the knowledge based on the reading materials provided and other authentic information on the Web. Stage 2: Online Game-based Learning In the second part, a virtual Game-based environment with near-real simulation is provided to students. Students are free to explore in the environment, to initiate changes, to solve problems arouse by players in the same environment, or any other activities that might happen in real environment. Stage 3: Just-in-Time and Summative Debriefing and Reflection In the last part, teachers help students to reflect and debrief all the learning process and explain the representative scenarios and assess the performances of groups and students. 1.2 An Example—FARMTASIA FARMTASIA (http://www.farmtasia.com) is the first example that has been developed under the VISOLE framework, which aims to facilitate the students to learn the subject domains of geography, biology, natural environment and hazard, government, economics, production system and technology, and to improve the students’ self-learning, collaborative, problem-solving, and information technology skills. In the game, students are divided into groups of four, in which each player plays the role of a farm manager interacting and competing against other group members for resources and revenues in nature and the global economic market [10]. The final reputation of the players in the virtual world, governed by good public policies, is also a vital and critical judging criterion, which is determined upon their practices on sustainable development and environmental protection. The whole play usually needs about 8 days, one hour every day. At the end of the whole play, the system will assess the students’ grade by the money, reputation and other activities in the farm. 2. The “Record-Replay” Function For the teacher to easily review the students’ activities in the farm and to more conveniently extract scenarios for conducting case studies in the class debriefing meetings, an innovative “Record-Replay” function are implemented into the game system. The function works like a video tape recorder, which can record and replay all of the students’ actions and decisions step by step. Fig. 1 shows a 3-month history (game time) of a player. Teachers can also choose to list the history of the students’ work in the last one week, one month, three months, or one year. They can also choose to replay selected game scenarios (see Fig. 2). Fast forwarding and rewinding, and display speed adjustment are also provided at the clicks of a button.

Fig. 1. Screen shots of the activities

Fig. 2. Screen shots of the Replay

J. Shang et al. / Using the “Record-Replay” Function for Elaboration of Knowledge

505

3. Effectiveness 3.1 For Reflection and Debriefing As discussed above, reflection and debriefing form an important part of the VISOLE game-based learning process (e.g., [8]). For example, students can analyze and evaluate their activities in the game and try to apply it into the real world whenever they are playing or have finished the game. By continuously evaluating and revising their activities in the game, students can learn the knowledge in more depth within short time. Furthermore, through their multiple thinking and rethinking in dealing with the knowledge in the game, they can be more familiar with it and thus apply it to the real word more easily [11]. In the second stage of the VISOLE approach, students are required to submit a daily reflective journal and an overall debriefing report on their own whole experience at the end of the entire gaming period. When students write their own reflective journals and reports, the “Record-Replay” function is an important tool enabling them to replay the game day by day to review what happened in the past and to make better planning for the future. During the learning process, teachers will arrange some face-to-face class meetings, namely, debriefing sessions. With the help of the “Record-Replay” function, teachers can observe and understand students’ activities and progresses, and they can extract interesting or critical scenarios from the virtual world as case studies for class discussions [10]. The “Record-Replay” function is welcomed by both teachers and students. In a pilot study conducted in Hong Kong on April, 2006, when the teacher replayed and evaluated students’ scenarios in a debriefing session, all students were very excited and devoted. Most of them commented that the “Record-Replay” function was effective for deepening their understanding of the subject matters. 3.2 For Assessment Assessment is sometimes a vital part in the learning process. Under constructivism and the reform of the new curriculum, formative assessment began attracting the scholars’ attention, which aims to emphasize the process of learning (e.g., [12]). The aim of formative assessment is not only to identify and evaluate students’ cognitive level, but also to facilitate the students to learn, and to improve the effectiveness of the teaching and learning. In VISOLE, the “Record-Replay” function provides an excellent tool for teachers to conduct formative evaluation, because it can automatically record all of the students’ actions and scenarios in the game, by which the teacher can assess the students’ performance in the game effectively and efficiently. 3.3 For Research The most commonly employed data collection methods in educational studies are scales, surveys, and interviews. However, these methods tend to have the following limitations: x Most data is self-reported by students. Thus the reliability of the data is subjective. x Researchers may not get rich data given the length constraint of scales and questionnaires and time constraint of interviews. x Students’ normal school life may be interrupted as they have to spend time filling in questionnaires or participating in interviews In contrast, the “Record-Replay” function has the following advantages: x It can provide rich, impersonal, and authentic data. x The data is recorded automatically by the system without disturbing students. x The data generated by “Record-Replay” function is in electronic format, which can be easily handled and processed. We can adopt both quantitative method and qualitative methods to analyze the data provided by the “Record-Replay” function. For example, by computing the sum of the

506

J. Shang et al. / Using the “Record-Replay” Function for Elaboration of Knowledge

frequencies of the actions that the student did in a bout, we can find whether the student has assiduously played in it; by comparing the former bout with the latter bout, we can find whether the student has learned the knowledge and improved problem solving skills. 4. Conclusion and the future work As discussed above, the “Record-Replay” function is not only a useful instrument for debriefing, but also a convenient and powerful tool for evaluation and research. According to the preliminary results of the pilot study conducted in April, 2006, the “Record-Replay” function is ranked well and should be a feature to have in educational games and software. In the future, we expect to improve in the following aspects. First, the system should automatically create the movie file, such as windows media player format (*.wma) or real player format (*.rm). In that way teachers and students can easily watch and share their experiences with each other. Second, the system can perform student assessment automatically and intelligently using Artificial Intelligence techniques and educational assessment theories. For example, the system will display some messages to provide comments and criticisms to a student when the student is watching his/her scenarios. Third, in debriefing, the system can present a daily report and present an overall report at the end of the game. In the report, the system should tell the students their success, failure, and mistakes, so that the students can do better in the next playing session. Thus students can really improve their high order skills. References [1] Prensky, M. (2000). Digital Game-Based Learning. New York: McGraw Hill. [2] Jong, M., Shang, J.J., Lee, F.L., Lee, J.M.H., Law, H.Y. (2006). Learning online: A comparative study of a situated game-based appproach and a traditional web-based approach. Proceedings of Edutainment 2006: International Conference of E-Learning and Games. Hangzhou, 16-18 April, China. [3] Greenfield, P.M. (1984). Mind and Media: The Effects of Television, Computers and Video Games. Cambridge, Mass.: Harvard University Press. [4] Whitebread, D (1997). Developing children’s problem-solving: the educational uses of adventure games, in: McFarlane, A (ed) Information Technology and Authentic Learning. London: Routledge, 13-37 [5] Bruckman, A. & Bonte, A. (1997). MOOSE goes to school: A comparison of three classrooms using a CSCL environment. Proceedings of the Computer Supported Collaborative Learning Conference, Toronto, CA. [6] M.E. Bredemeier and C.E. Greenblatt. The educational effectiveness of games: A synthesis of findings. Simulation & Gaming, 12(3), 307-332, 1981 [7] Lundy, J. (1984).The Effects of competition in business games. In Ments, M.V. & Hearnden, K. Effective use of games & simulation—(Perspectives on gaming and simulation 10). England,Direct Design(Bournemouth) Ltd.,Printers, 27-34. [8] Crooball, D. (1992). Debriefing. Simulation & Gaming, 23(2), 141-142 [9] Lee, J. H. M. & Lee, F. L. (2001). Virtual Interactive Student-Oriented Learning Environment (VISOLE)---Extending the Frontier of WEB-Based Learning. The Scholarship of teaching learning organized by University Grant Council, Hong Kong. [10] Luk, E.T.H., Wong, M.K.H., Cheung, K.K.F Lee, F.L., & Lee,J.M.H. (2006). Design and Implementation of Farmtasia: a GameDesigned for the VISOLE Teaching Stryle. Proceedings of Edutainment 2006: International Conference of E-Learning and Games. Hangzhou, 16-18 April, China. [11] Shang, J.J., Lee, F.L., Lee, J.H.M. (2005). Design Strategies and Principles in VISOLE. The Proceedings of International Conference of Computers in Education, 28 Nov-2 Dec, Singapore. [12] Black, P. (1993). Formative and Summative Assessment by Teachers. Studies in Science Education, 29,49-97

The work described in this paper was substantially supported by a grant (CUHK4200/02H) from the Research Grants Council of Hong Kong SAR.

Participation/Attitude Toward Learning

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

509

Investigating Learner Autonomy toward e-learning Shu-Sheng Liawa, Hsiu-Mei Huangb General Education Center, China Medical University, Taiwan b Department of Management Science, National Taichung Institute of Technology, Taiwan a [email protected] b [email protected] a

Abstract: Although e-learning is a capable tool for world wide students, learner autonomy is a critical issue for enhancing e-learning usage. The purpose of this study is to investigate individual learner autonomy toward e-learning. Based on Taiwan and UK University’s students, this research proposes a web questionnaire survey to explore learners’ attitudes toward e-learning. After statistical analyses, both Taiwan’s and UK’s students’ learner autonomy can be predicted by perceived usefulness of sharing, quality of learning functions, and quality of interaction. Moreover, from Taiwan’s and UK’s students, learner autonomy has high correlation with perceived enjoyment of multi-format contents. Additionally, the results of this study find that Taiwan’s students have more positive attitudes toward e-learning than UK’s students have. Keywords: Learner autonomy, sharing, interaction, system quality

1. Introduction E-learning environments are characterized by place and time independence, integrated presentation and communication facilities, and opportunities for re-use of instructional materials. The role and important of technology in the development of e-learning environments is often overstated by technology designers. Instead, designers may seek to understand the basic components of what constitutes an e-learning system and learners’ attitudes toward the system. Indeed, e-learning framework is crucial in guiding the decisions relating to the choice and development of each component in relation to the objectives of learners’ needs or outlined in the organizational e-learning strategy [1]. Regarding to e-learning strategy for fit learners’ needs, McGraw [2] provided statements: First, a common language and vision to describe e-learning for the organization or learners needs. Second, governing principles and organization-wide support policies. Third, creation of content that make learning compelling, engaging, and relevant to target learners’ needs. Fourth, support for individual learner profiles, activities and tasks. And fifth, a standard-driven technical architecture that can link to existing systems and be accessed efficiently. The purpose of this study is to examine learner autonomy toward e-learning. Indeed, understanding learner autonomy toward e-learning is necessary to ensure that e-learning stands the best possible chance to succeed. Learner autonomy difference may happen on different countries. Therefore, based on Taiwan’s and UK’s students, the aims of this study is to investigate Taiwan’s and UK’s students’ learner autonomy toward e-learning.

510

S.-S. Liaw and H.-M. Huang / Investigating Learner Autonomy Toward e-Learning

2. Literature review Recent research studies on learner autonomy in interactive or communicative learning environments have begun to acknowledge the social and situated nature of learner autonomy and its collaborative interplay with personal agency [3], [4]. According to these studies, learner autonomy can only be understood as a social process. Lim & Chai [5] state that the focus is on how students learn autonomously as a consequence of diverse and changing intrapersonal, interpersonal and contextual conditions. It means that learner autonomy does not represent a learner’s obliviousness to social influences, but rather his or her intentional use of these forces. Therefore, it cannot be assumed that students will automatically take up the learner autonomy provided by interaction and communication in the e-learning environment. Essentially, there are two different threats that will negatively affect students’ learner autonomy.

First, students may lack the learning strategies to learn with the e-learning systems. Studies conducted by Hurd, Beaven, and Ortega [6], Oliver and Hannafin [7], and Schwienhorst [8] found that most students did not know how to monitor and regulate or control and sustain the learning process when using communicative tools or distance learning programs for learning. Even for more experienced learners using communicative learning packages, the teacher had to keep on reminding the mature students to take down notes and yet refrain from taking down everything from the communicative package [9]. Many learners also failed to apply appropriate and efficient strategies to learning problems encountered. These involved sorting out how to respond to the communicative tool when it asked questions and how to act upon its feedback, how to keep track of and reflect on the concepts covered, how to establish links and relationships between and among topics, when to explore and probe more in-depth into a topic, and when and how to make notes [5]. And second, single-format learning instruction may let students lost their interests and attentions. In the study of a multi-form story [10], named plotline in multiple versions, the program brings a new dimension to the element of choice in autonomous learning. Learners control the course of the narrative. Murray [10] writes that: in a learner-centered educational application, a well-designed computer-based fiction should have choices that are meaningful and not arbitrary and that learners can invest themselves in making. This technology, therefore, adds a new dimension to be considered in the study of learner autonomy, the learners' sense of agency in relation to the narrative structure. Moreover, the usual multimedia and interactive tools enable the learner to proceed according to his or her own pace and proficiency. These include functions such as stop, repeat segment, forward, access to glosses, captions, and other help, including an alternative slower sound track. Links between text and video make it possible to have a transcription for each utterance, to click on words in the transcription to have a glossary explanation appear, and to link the glossary to the video in order to see and hear multiple examples of the same word or phrase used in context [10]. 3. Research hypotheses A pool of e-learning tools, including WebCT, Blackboard, and any other e-learning platforms, mediate the interactions between the subject and object. The student belongs to a community consisting of his/her classmates and teachers that shares and negotiates, mediated by shared rules and procedures, and division of labor. The rules include individual self-regulations, such as learner autonomy. The role that each individual of the community has to play the division of labor. The individual student is expected to be an economist at

S.-S. Liaw and H.-M. Huang / Investigating Learner Autonomy Toward e-Learning

511

work, gathering, representing, interpreting, and analyzing data. The teacher takes on more of a mediator role than he/she will take in a non e-learning environment. He/she shares with students the well-defined tasks of questioning, clarifying, summarizing, and predicting to help them understand the relationships among the variables under study [5]. Therefore, from those previous studies, this research proposes the following hypotheses:

HA: Taiwan’s and UK’s learners have highly positive attitudes toward using e-learning for learning activities. HB: Taiwan’s and UK’s learners do not have significant attitudes’ difference toward e-learning.

HC: In e-learning, learner autonomy of Taiwan’s and UK’s learners will be positive affected by multimedia learning contents and learners-teachers cooperation. 4. Research design 4.1 Participants In this study, a questionnaire survey was conducted on Taiwan’s and UK’s university students. All participants were asked to answer a web questionnaire survey that included demographic information and two different components (computer and Internet experience, attitudes toward e-learning). All subjects were asked to respond to in one week and their responses were guaranteed confidentiality. A total of 232 responses were collected of which 102 UK’s students and 130 Taiwan’s students. 4.2 Instruments The questionnaire web site is shown as Figure 1. The data for this study was gathered by means of a questionnaire. The questionnaire included attitudes toward the e-learning system. Participants were asked to indicate their attitudes toward the system. These 15 questions were all 7-point likert scales (from 1 which means "strongly disagree" to 7 which means "strongly agree"). These 15 questions are divided into 5 different factors: learner autonomy, perceived enjoyment of multi-format contents, quality of learning functions, perceived usefulness of sharing, and quality of interaction.

512

S.-S. Liaw and H.-M. Huang / Investigating Learner Autonomy Toward e-Learning

Figure 1: Main screen of the web questionnaire survey 5. Results Descriptive statistics (means (M) and standard deviations (SD)) of computer and Internet experience were shown in Table 1. Descriptive statistics (means (M) and standard deviations (SD)) of e-learning attitudes were shown in Table 2. The alpha reliability of attitudes toward e-learning were to be highly accepted (α=0.94 of UK students and α=0.95 of Taiwan students). The high alphas reliabilities give a support for questionnaire content reliability. Table 1: Descriptive statistics of computer, Internet and e-learning experience (from 1 which means "no experience" to 7 which means "well experienced") Variables M of UK S.D. of UK S.D. of M of learners learners Taiwan Taiwan learners learners Experience using computers. 5.92 1.21 6.00 0.99 Experience using the Internet Experience using web browsers. Experience using E-mail. Experience using e-learning.

6.02 5.98 5.98 4.80

1.12 1.26 1.14 1.55

6.08 5.78 5.93 4.42

0.96 1.07 1.05 1.51

Table 2: The item-total correlations of attitudes (from 1 which means “strongly disagree” to 7 which means “strongly agree”) Items UK Taiwan students students M S.D. M S.D. Learner autonomy: 4.44 1.28 4.89 1.11 I can learn actively in e-learning. 4.52 1.27 4.80 1.05 I can read e-learning contents actively. 4.31 1.30 4.88 1.06 I have opportunities to learn autonomously in e-learning. 4.48 1.28 4.99 1.22 Perceived enjoyment of multi-format contents: 4.42 1.25 5.01 1.10 I like multimedia e-learning contents. 4.34 1.34 4.95 1.04 I like video e-learning contents. 4.63 1.13 4.99 1.18

513

S.-S. Liaw and H.-M. Huang / Investigating Learner Autonomy Toward e-Learning

I like animated e-learning contents. Quality of learning functions: I am satisfied with the operating methodology of e-learning. I am satisfied with quality of e-learning. I am satisfied with the functions of e-learning. Perceived usefulness of sharing: I believe teacher’s helps and suggestions are important in e-learning. I would like to share e-learning experience with other classmates. I believe e-learning systems should include some communication functions. Quality of interaction: I am satisfied with the response time of e-learning. I am satisfied with speed of the Internet. I am satisfied with the download time of e-learning programs.

4.29 4.13 4.02

1.29 1.30 1.27

5.09 1.09 4.81 1.28 4.73 1.31

4.03 4.34 4.43 4.37

1.50 1.14 1.52 1.60

4.78 4.91 5.16 5.20

4.16

1.56

5.00 1.26

4.75

1.40

5.27 1.23

4.05 3.84 4.15 4.16

0.99 0.94 1.12 0.91

4.74 4.74 4.85 4.63

1.29 1.24 1.23 1.19

1.31 1.35 1.25 1.32

The Pearson correlation coefficients among the variables are presented in Table 3 (UK’s students) and Table 4 (Taiwan’s students). The bi-variate relationships indicated that many of the variables significantly correlated with each other. Table 3: Correlation analysis of UK’s students’ attitudes Variables 2 3 4 1. Learner autonomy 0.69** 0.66** 0.78** 2. Perceived enjoyment of 0.69** 0.60** multi-format contents 3. Quality of learning functions 0.38** 4. Perceived usefulness of sharing 5. Quality of interaction **. Correlations are significant at the p<0.01 (2-tailed). Table 4: Correlation analysis of Taiwan’s students’ attitudes Variables 2 3 4 1. Learner autonomy 0.74** 0.71** 0.76** 2. Perceived enjoyment of 0.66** 0.77** multi-format contents 3. Quality of learning functions 0.51** 4. Perceived usefulness of sharing 5. Quality of interaction **. Correlations are significant at the p<0.01 (2-tailed).

5 0.54** 0.59** 0.65** 0.56**

5 0.67** 0.62** 0.83** 0.47**

An independent-samples t-test was conducted to evaluate the hypothesis HB. The results indicate there were significant differences on perceived usefulness of sharing (t(230)=201.75, p=0.000), perceived enjoyment of multi-format contents (t(230)=191.76, p=0.000), quality of learning functions (t(230)=216.95, p=0.000), quality of interaction (t(230)=224.62, p=0.000), and learner autonomy (t(230)=194.26, p=0.002). In other words,

514

S.-S. Liaw and H.-M. Huang / Investigating Learner Autonomy Toward e-Learning

there have statistically significant attitudes’ differences between Taiwan’s and UK’s students. Concerning analytic strategy for assessing the predictive model, multiple regression analysis is an appropriate multivariate analytical methodology for empirically examining sets of relationships in the form of linear causal models. The results of stepwise multiple regressions for the path associated with the variables were presented in Table 5 (UK’s students) and Table 6 (Taiwan’s students). From the results of Table 5, the regression analysis was performed to check the effects of predicted variables (perceived usefulness of sharing, perceived enjoyment of multi-format contents, quality of learning functions, quality of interaction) on learner autonomy. The results showed three independent variables, except perceived enjoyment of multi-format contents, are predictors on learner autonomy (F(3, 98)=115.33, p<0.0005, R2=0.78). These three predictors have 78% contribution for predicting learner autonomy of using e-learning. From the results of Table 6, the regression analysis was performed to check the effects of predicted variables (perceived usefulness of sharing, perceived enjoyment of multi-format contents, quality of learning functions, quality of interaction) on learner autonomy. The results showed three independent variables, except perceived enjoyment of multi-format contents, have positive prediction on learner autonomy (F(3, 126)=113.98, p<0.0005, R2=0.73). These three predictors have 73% contribution for predicting learner autonomy of using e-learning. Table 5: Regression results of UK’s students’ attitudes Dependent variable Independent variables Learner autonomy

Perceived usefulness of sharing Quality of learning functions Quality of interaction

β 0.68 0.52 0.18

Table 6: Regression results of Taiwan’s students’ attitudes Dependent variable Independent variables β Learner autonomy

Perceived usefulness of sharing Quality of learning functions Quality of interaction

0.53 0.27 0.20

R2 Change 0.61 0.15 0.02 R2 Change 0.58 0.14 0.01

p 0.000 0.000 0.010 p 0.000 0.002 0.015

6. Discussions Learning environments are characterized by their place and time independence, their integrated presentation and communication facilities, and their opportunities for re-use of learning technologies in the form of learning objects. Many researchers claim that technology push will enhance the quality of education. The Internet and World Wide Web have provided opportunities of developing e-learning systems. The development of e-learning systems has started a revolution for instructional content delivering, learning activities, and social communication [11]. Clark [12] argues that the question of whether media or technology will ever influence learning remains open to debate. A well-defendable viewpoint lies not in the media or technology used because only positive attitudes toward that media or technology can improve the quality of learning. Thus, understanding users’ attitudes toward learning technology enables us to make learning more efficiency and effectiveness. The present study helps us understand learners’ attitudes, including UK’s and Taiwan’s students, toward e-learning. In light of UK and Taiwan learners’ attitudes, perceived

S.-S. Liaw and H.-M. Huang / Investigating Learner Autonomy Toward e-Learning

515

usefulness of sharing, quality of learning functions, and quality of interaction, these factors demonstrate that e-learning systems can be designed as an environment for improving learner autonomy. From the activity theory approach, individuals actively construct their knowledge within social realms. In other words, learners are not to passively accept information by mimicking the wording or conclusions of others, but instead need to encourage themselves in internalizing and reshaping or transforming information through active consideration. Additionally, based on the high correlation between perceived enjoyment of multi-format contents, it implies when a learner processes information through both verbal and imagery functions, these multi-format approaches to education are thought to be particularly effective for accommodating learners with diverse styles and preferences for learning Acknowledgments: This study was partially supported by NSC95-2520-S-039-001-MY3 and CMU94-050. References [1] Ismail, J. (2002). The design of an e-learning system beyond the hype, Internet and Higher Education, 4, 329-336. [2] McGraw, K. L. (2001). E-learning strategy equals infrastructure (ASTD {online}. Available at http://www.learningcircuits.org/2001/jun2001/mcgraw.html.). [3] Little, D. (2001). Learner autonomy and the challenge of tandem language learning via the Internet. In A. Chambers & G. Davies (Eds.) ICT and languages learning: A European perspective (pp.29-38). Lisse: Swets & Zeitlinger. [4] Ritter, M., Kallenbach, C., & Pankhurst, J. (1999). The all-inclusive tutor – Excluding learner autonomy? ReCall Journal, 11(1), 111-116. [5] Lim, C. P., & Chai, C. S. (2004). An activity-theoretical approach to research of ICT integration in Singapore schools : Orienting activities and learner autonomy, Computer & Education, 43, 215-236. [6] Hurd, S., Beaven, T., & Ortega, A. (2001). Developing autonomy in a distance language learning context: issues and dilemmas for course writers, System, 29, 341-355. [7] Oliver, K., & Hannafin, M. (2000). Student management of web-based hypermedia resources during open-ended problem solving. The Journal of Educational Research, 94(2), 75–92. [8] Schwienhorst, K. (1997). Talking on the MOO: Learner autonomy and language learning in tandem. Presented at CALLMOO: Enhancing Language Learning through Internet Technologies, Norway. [9] Draper, S. W., Brown, M. I., Henderson, F. P., & McAteer, E. (1996). Integrative evaluation: An emerging role for classroom studies of CAL. Computers and Education, 26(1), 17–32. [10] Murray, G. L. (1999). Autonomy and language learning in a simulated environment. System, 27, 295-308. [11] Liaw, S. S., Huang, H. M., & Chen, G. D. (in press). An Activity-Theoretical Approach to Investigate Learners’ Factors Toward E-learning Systems, Computers in Human Behavior. [12] Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21-29.

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

517

Moderating Role of Online Self-Efficacy in Relation between Learning Strategy and Online Performance Peng Huamao1, Wang Ying2, Huang Ronghuai1 Educational Technology School, Beijing Normal University, China 2 Institution of Distance Education Research, China Central Radio and TV University, China 1

[email protected] Abstract: Data derived from 72 students in online English course showed that online self-efficacy was a moderating factor in relation between learning strategy and online performance. In high online self-efficacy group, online performance was predicted by learning strategy positively, which explained 23% of the performance variance. In low online self-efficacy group, online performance was not predicted by learning strategies significantly. Keywords: Online self-efficacy, online performance, learning strategy, moderator

1. Introduction Web-based instruction has become increasingly popular in recent years. It is a type of self-regulated learning. Learners’ motivation and strategy are responsible for their own learning processes and outcomes. Many researchers documented that as learners’ computer or Internet self-efficacy perceptions strengthened, their Web-based learning performance also noticeably improved (Whipp, 2004; Joo, 2000; Al-Khaldi, Al-Jabri, 1998; Hill, Hannafin, 1997). Self-efficacy is people’s judgments of their capabilities to organize and execute course of action required to attain designated types of performance. It is not with the skills one has but with judgments of what one can do with whatever skills one possesses (Bandura, 1986). Self-efficacy exerts effect on performance directly as well as indirectly through various cognitive mechanisms such as learning strategies. Many documents showed that students with a stronger sense of self-efficacy tented to use more strategies, which eventually resulted in better classroom performance (Zhang, Zhang, 2003; Zhou, Zhang, Li et al., 1997; Pintrich, Groot, 1990). It seems to be conclusive that learners use more learning strategies will lead to better online-performance. However, what is role of online self-efficacy in the relation between strategies use and online performance? Is that relation linear with increasingly strong online self-efficacy? Or does that relation change with different online self-efficacy level? We are interested in confirming moderating role of online self-efficacy in the relation between strategies use and online performance.

518

H. Peng et al. / Moderating Role of Online Self-Efficacy

2. Method 2.1 Participants Participants were 72 online learners from Beijing Jiaotong University. The sample consisted of 28 males (39.4%) and 43 females (60.6%). There was 1 missing data of sex. 2.2 Measures Online self-efficacy. Perceived capability to use Internet and computer to complete online learning tasks was assessed by 13 items adapted from the Distance Learning Self-efficacy Scale developed by Peng Huamao, Wang Ying, Huang Ronghuai and Chen Geng (2006). Sample items read: “I can use the links provided by teacher to find learning materials” and “I can’t participated discussions on the BBS about courses.” A response scale ranged from 1(not at all true) to 5(very true).The Cronbach’¢ for this scale was .887. The high score on the scale exhibited high online self-efficacy. Learning Strategy. 67 items developed from the LASSI (Weinsteinˈ1995) assessed learners’ strategy level. Items were revised to adapt to online learning. Sample items read: “I draw pictures or diagrams to summarize materials in the course;” “I always try to relate what I’m learning to my work,” and “I test myself to make sure if I understand what I have learned.” A response scale ranged from 1(not at all true) to 5(very true). The Cronbach’¢ for this scale was .902. The high score on the scale exhibited high strategy level. Online performance. Students’ online performances of English course were measured. Two types performance measure were obtained. One was online unit test. The other performance was performance of online collaborative learning. Two types performance were given equal weight. Therefore, online performance was summation of unit test score and collaborative learning score. 3. Results 3.1 Preliminary Analyses Table 1 presents descriptive statistic of scales and online performance. Table 1 Descriptive Statistic of Scales and Online Performance Male Female Total Scale M SD N M SD N M SD N Online 4.02 .63 28 3.91 .74 43 3.95 .69 72 Self-Efficacy Learning 3.52 .43 28 3.46 .42 43 3.50 .43 72 Strategy Online 60.54 17.92 28 54.15 23.90 43 56.78 21.69 72 Performance A series of t tests was conducted to determinate whether there was any gender difference. No significant gender difference was found (t (71) = .663, p>.05;t (71)= .653, p>.05; t (71) = 1.283, p>.05).

519

H. Peng et al. / Moderating Role of Online Self-Efficacy

3.2 Correlation Analyses Pearson Correlation Coefficients were computed to examine relations among variables. As Table 2 shows, online performance correlated with highly scores of online self-efficacy (r= .283) and learning strategy (r= .348). There was positive correlation between self-efficacy and strategies use (r= .561). Self-efficacy correlated with performance directly, but what was its role in the relation between performance and strategy? It will be concluded through regression analyses. Table 2 Pearson Correlation Coefficients Among Variables Online Learning Self-Efficacy Strategy Online Performance

.283*

.348**

Learning Strategy

.561**

-

Note: **. Correlation was significant at the 0.01 level. *. Correlation was significant at the 0.05 level. 3.3 Regression Analyses The collected data was divided into two groups based on their mean value of online self-efficacy, 3.95. The group of students who scored less than or equal to the mean value were categorized as the group of low self-efficacy (n=36). Otherwise, students were categorized as high self-efficacy (n=36). Further regression analyses of these groups indicated that online self-efficacy was a moderating factor for the relation between online performance and learning strategy. Table 3 shows the results of linear regression used method of least squares. Table 3 Regression Analyses Results of Online Performance Predicted By Learning Strategy Level Group

R2

£

F

Sig.

Low Self-Efficacy

.028

.168

1.014

.321

High Self-Efficacy

.230

.479

9.849

.004

For the high online self-efficacy group, online performance was predicted by learning strategy level (£=.479, p< .01), which explained 23% of the performance variance. However, the indicator for the strength of the correlation dropped in the low online self-efficacy group. The performance was not predicted by learning strategy level significantly (£=.168, p> .05), which explained only 2.8% of the performance variance. In other words, if someone’s online self-efficacy was low, his online performance was not related to his learning strategy level. When his self-efficacy was high, his strategy level would affect performance positively. Therefore, regression analyses results indicated that online self-efficacy was a moderator variable in the relation between performance and strategy level.

520

H. Peng et al. / Moderating Role of Online Self-Efficacy

4. Discussions Online learners in our research are adults’ on-the-job and participating learning through Internet. The learning experiences for these online learners are not only relative to their work, but also relative to their computer skills (or internet skills) during e-learning. In a sense, computer skills are more and more important in e-learning. Some findings indicate that self-efficacy is sensitive to experience, tasks and contexts. And self-efficacy of learners will have effect on latter learning behaviors. We believe if learners have successful experience or substitute experience for computer skills, they can enhance their conception through their own behaviors or others’ behaviors and improve their own self-efficacy. Our research shows that self-efficacy is moderating variable in the relation between performance and learning strategy. In other words, it can improve learning strategy and online performance somehow. Therefore, online tutors should support online learners with opportunities that learners can succeed easily so as to make learners have relevant experience and understand learning tasks. For high self-efficacy learners, online tutors give them more successful experiences, especially give them online learning strategies instructions directly, and help learners improve their online performance. For low self-efficacy learners, online tutors should pay much more attention to learners’ successful experience and not giving them too much learning strategies at first. Enhancing self-efficacy is the first step for low-efficacy learners. They need to increase successful judgments and enough confidence through enough experience of computer skills. Acknowledgments Teaching Reform Project of Beijing Jiaotong University supported the work reported in this paper. We thank the partners as following: Chen Geng, Xu Cheng and Shi Zhiping. References 1. 2. 3. 4.

5. 6. 7. 8. 9.

Al-Khaldi, M. A., and Al-Jabri, I. M. (1998). The relationship of attitudes to computer utilization: New evidence from a developing nation. Computers in Human Behavior, 14, 23-42. Bandura, A. (1986). Social functions of thought and action: A social cognitive theory. New Jersey: Prentice-Hall, Inc. Hill, J. R., and Hannafim, M. J. (1997). Cognitive strategies and learning from the World Wide Web. Educational Technology Research and Development, 45, 5, 37-64. Joo, Y., Bong, M. and Choi, H. (2000). Self-efficacy for self-regulated learning, academic self-efficacy, and Internet self-efficacy in web-based instruction. Educational Technology, Research and Development, 48, 2,5-17. Peng, H., Wang, Y., Huang R., and Cheng, G. (2006). Self-efficacy of distance learning: Structure and related factors. Open Education Research, 12, 2, 41-45. Pintrich, P. R., and De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 1, 33-40. Whipp, J. L and Chiarelli, S. (2004). Self-regulation in a web-based course: A case study. Educational Technology, Research and Development, 52, 4, 5-22. Zhang, L., and Zhang, X. (2003). A Study of the relationships among learning strategy-usingˈself-efficacy, persistence and academic achievement in middle school students. Psychology Science, 26, 4, 603-607. Zhou, G., Zhang P., Li L., and Liu, R. (1997). A study of the relationships among perceived competence, learning strategies and achievement in junior middle school students’ equations learning. Psychology Science,20, 4, 324-328.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

521

Understanding E-Learners' Characteristics and Performance in Online Courses Rowena Santiagoa, Minoru Nakayamab Professor, California State University San Bernardino, USA b Associate Professor, CRADLE, Tokyo Institute of Technology, Japan [email protected] a

Abstract: Learner characteristics have been identified to have a significant influence on student performance in online learning environments. This paper describes the results of a project that surveyed the motivation, personality and thinking styles of e-learners and explored any significant relationship(s) they may have on student performance. Keywords: learner characteristics, e-learning, motivation, personality, thinking styles,

1. Introduction Although Russell [11] reported “no significant difference” in student achievement in traditional instruction and e-learning, research reports often indicate that student success (in terms of performance and drop-out rates) is influenced by factors such as motivation [10], learning styles [2, 5, 13, 15] and self-directive competencies [1]. Analysis of learner characteristics for successful online experience will serve the best interest of students [14] and should be part of any systematic design of instruction [3, 4]. Motivation, personality, and thinking styles have been identified to have a significant influence on student performance in an online teaching environment. McCloy et al. [9] defined motivation as "the combined effect of three choice behaviours: (a) the choice to expend effort, (b) the choice of what level of effort to expend, and (c) the choice to persist in the expenditure of the chosen level of effort". One theory about motivation that is particularly applicable to school performance is the distinction between intrinsic and extrinsic motivation (Kaufman, J., personal communication). Personality is another characteristic that could impact learning. Over the last two decades, a general consensus among personality researchers has emerged that five basic factors underlie traditional personality assessment: extroversion, emotional stability, agreeableness, conscientiousness and openness. Cognitive styles (or thinking styles) are individual consistencies in the ways people characteristically respond to or interpret information or problems presented to them. They are not abilities, but rather reflect stable strategies, preferences, and attitudes of how people choose to use their abilities [12]. 2. Project Purpose The purpose of this paper is to survey the motivation, personality and thinking styles of elearners and explore any significant relationship(s) they may have on student performance. This project is the US-phase of a joint Japan-US survey project on learner characteristics and e-learning.

522

R. Santiago and M. Nakayama / Understanding E-Learners’ Characteristics and Performance

3. Survey and Analysis of Learner Characteristics 3.1. Survey Participants and Instruments Surveys of student motivation, personality, thinking styles were conducted during the first two weeks of classes using the following instruments : KAMS : Kaufman-Agars Motivation Scale [8] (intrinsic and extrinsic motivation) ; Goldberg’s [6,7] International Personality Item Pool (IPIP) inventory of the ‘big five’ personality components (extraversion, agreeableness, conscientiousness, emotional stability, and openness) ; and Sternberg’s [12] Functions of Thinking Styles (legislative, executive, judicial). Survey participants included 86 US students (Florida and California), who were enrolled in three upper division courses: Computer Science (n=18), Health Science (n=21), and Education (n=47). These groups will be referred to as CS, HS, and EDU, respectively. 3.2. Survey Results Initial results (see Table 1) from the motivation scales in all three groups (CS, HS, EDU) reveal consistent and high levels of intrinsic motivation among students, suggesting an overwhelming majority of students are driven by the inherent value of their work. Mean values for extrinsic motivation were not as high as the intrinsic mean values, but could suggest that a relatively equal number of students could be highly driven by external rewards associated with their work. These results indicate that students are intrinsically motivated and can be expected to be driven to learn, but there are some students, or even an equal number of students, who are also extrinsically motivated. Thus, the role of external rewards remains an important consideration when designing e-learning for these students. Table 1: Means Values (with Standard Deviation) for Each Class

Motivation

Personality

Thinking Styles

Constructs Intrinsic Extrinsic Extroversion Agreeableness Conscientiousness Emotional Stability Openness Legislative Executive Judicial

Range 1-10 1-10 1-5 1-5 1-5 1-5 1-5 1-7 1-7 1-7

CS (n=18) Mean (SD) 7.79 (0.82) 5.21 (1.30) 3.16 (0.85) 4.03 (0.53) 3.94 (1.39) 3.38 (0.62) 3.48 (0.60) 5.13 (0.88) 5.24 (0.56) 4.76 (0.73)

HS (n=21) Mean (SD) 7.94 (0.95) 5.60 (1.21) 3.50 (0.95) 4.16 (0.66) 3.81 (0.62) 3.02 (0.70) 3.36 (0.66) 4.71 (1.06) 5.42 (0.89) 4.15 (0.97)

EDU (n=47) Mean (SD) 7.95 (.86) 5.61 (1.14) 3.51 (0.88) 4.46 (0.79) 3.50 (0.72) 3.04 (0.74) 3.57 (0.53) 4.91 (0.84) 5.06 (0.86) 4.08 (0.96)

Initial results based on measures of the five (5) Personality factors show that values of agreeableness were high (above 4.0) for all three groups. Values on conscientiousness, openness and extroversion were moderately high. Lower values for emotional stability indicate greater stability. Personality results suggest that students are extroverted, and open to new experiences (such as e-learning). Online activities that promote social engagement will be favorable elearning experiences for these students. Further, high values for agreeableness suggest that these students place a great deal of importance on cooperation and consideration of others. For conscientiousness, the results suggest that these students value organization,

R. Santiago and M. Nakayama / Understanding E-Learners’ Characteristics and Performance

523

dependability, and discipline. Online group work or discussion, course organization and management will be important to these students. Finally, the results based on measures of Sternberg’s Thinking Styles reveal high values for each of the three thinking styles – legislative, executive, and judicial. Thinking Style results suggest that students in the sample place high value on creativity and self-direction (legislative), following directions and working under structure (executive), and evaluation (judicial). Although this finding seems inconsistent, the fact that students maintain high levels of each style may actually represent substantial flexibility in their approaches to online work. 4. E-Learner Characteristics and Grade Performance In exploring any relationship between e-learner characteristics and grade performance, we must be very cautious in drawing inferences. Given the small sample sizes and the lack of variability in performance outcomes, most relationship values were not interpretable. For the CS group, the only relationship strong enough to recognize is between executive thinking style and grades (r = -.54), which suggests that students who are less rule and structureoriented performed better in this course than those who are more oriented towards an executive style. Table 2: Bivariate Correlations Between Survey Variables and Grades Constructs (Interpretable if value, in either direction, is greater than: Intrinsic Extrinsic Extroversion Agreeableness Conscientiousness Emotional Stability Openness Legislative Executive Judicial

CS 0.43 0.26 0.21 -0.10 -0.35 0.01 0.17 0.24 0.27 -0.54 0.18

HS 0.48 -0.31 -0.20 -0.33 -0.22 0.34 -0.29 0.10 -0.25 -0.06 -0.30

EDU 0.25 -0.10 -0.30 -0.35 -0.11 0.27 -0.03 -0.27 -0.21 0.26 0.03

ALL 0.21) -0.08 -0.14 -0.28 -0.17 0.17 -0.06 -0.03 -0.11 0.02 -0.04

For EDU students, correlation data do suggest relationship between high performance and four individual characteristics: extrinsic motivation (not motivated by external rewards), extroversion (are outgoing and expressive), conscientiousness (place importance on attention to detail and organization), and executive thinking style (prefer to follow directions and work within a structured environment). For the HS group, the small sample and lack of variability in performance outcomes means we cannot interpret these relationships. Overall (n=86) bivariate correlation between study variables and grades resulted in only one interpretable value (values greater than .21 in either direction), and that is Extroversion (-.28). This result could indicate that extroverted students did not perform as well in online environment where there was limited social interaction. 5. Conclusion Survey of learner characteristics can help instructional designers avoid what Dick and Carey [3] described as “Too often, designers make references about the characteristics of the learners without actually verifying them” (p. 85). In this project, survey of learner

524

R. Santiago and M. Nakayama / Understanding E-Learners’ Characteristics and Performance

characteristics verified that learners are intrinsically motivated, value cooperation, and have high levels in each of the thinking styles. Understanding these learner characteristics can help in the design of effective online activities. Exploring any relationship(s) between characteristics and grade performance resulted in very few interpretable data and different relationships for each of the three groups. Gagne [5] points out that there are many traits that can be assessed in so many ways, but nevertheless, “the possibility exists that differences in one or more traits will exhibit an influence on learning that makes desirable the adaptation of instructional approaches to these differences” (p.108). Understanding learner characteristics can provide powerful insights to an instructional designer when integrating online technologies for teaching and learning. Acknowledgments We thank Profs. Danilo Baylen, Marsha Greer and Josephine Mendoza and their students for the data collection; Prof. James Kaufman and Mark Agars for the surveys and statistical analyses. This research is partially supported by the Japan Society for the Promotion of Science (JSPS), Grant-in-Aid for Scientific Research (B16300263), 2004-2006.

References [1] Birch, P.D. (July 2002). E-learner competencies. Learning Circuits – ASTD Online Magazine. http://www.learningcircuits.org/2002/jul2002/birch.html [2] Diaz, D. P., & Cartnal, R. B. (1999). Students' learning styles in two classes: Online distance learning and equivalent on-campus. College Teaching Vol. 47, No. 4, 130-135. [3] Dick, W. & Carey, L. (1985). The Systematic Design of Instruction, 2nd Ed. Glenview, IL: Scott, Foresman & Co. [4] Dick, W. & Carey, L. (1996). The Systematic Design of Instruction, 4th Ed. New York, NY: Harper Collins. [5] Gagné, R.M., Briggs, L.J .& Wager, W.W. (1992). Principles of Instructional Design. 4th ed. Fort Worth: Harcourt Brace Jovanovich College Publishers. [6] Goldberg, L. R. (1999). “A broad-bandwidth, public domain, personality inventory measuring the lowerlevel facets of several five-factor models”. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality Psychology in Europe, Vol. 7 (pp. 7-28). Tilburg, The Netherlands: Tilburg University Press. [7] International Personality Item Pool (2001). “A Scientific Collaboratory for the Development of Advanced Measures of Personality Traits and Other Individual Differences” (http://ipip.ori.org/). Internet Web Site. [8] Kaufman, James C. & Agars, Mark D. (2005) “Kaufman-Agars Motivation Scale”, Unpublished instrument. [9] McCloy, R. A., Campbell, J. P., & Cudeck, R. (1994). “A confirmatory test of a model of performance determinants”, Journal of Applied Psychology, 79, 493-505 [10] Pintrich, P. & Schunk, D. (2002). Motivation in Education: Theory, Research and Applications. Upper Saddle River, NJ: Merrill Prentice Hall. [11] Russell, T.L. (1999). The No Significant Difference Phenomenon. Chapel Hill, NC: Office of Instructional Telecommunications, North Carolina University. [12] Sternberg, Robert J. (1997) Thinking Styles, Cambridge University Press, Cambridge, UK. [13] Terrell, S. & Dringus, L. (1999) An investigation of the effects of learning style on student success in an online learning environment. Journal of Educational Technology Systems, Vol. 28, No. 3, 231-238. [14] Wojciechowski, A. & Palmer, L.B. (Summer 2005). Individual student characteristics: Can any be predictors of success in online classes? Online Journal of Distance Learning Administration, Vol. 8, No. 2. http://www.westga.edu/~distance/ojdla/summer82/wojciechowski82.htm [15] Zhang, L. & Sternberg, R. (2001). Thinking styles across cultures: Their relationships with student learning. In R. Sternberg & L. Zhang (Eds.), Perspectives on Thinking, Learning, and Cognitive Styles. Mahwah, NJ: Lawrence Erlbaum Associates. 197-226.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

525

An Exploratory Study on Teachers’ Perceptions of Game-based Situated Learning Morris S.Y. JONG, Junjie SHANG, Fong-Lok LEE, Jimmy H.M. LEE, Huk-Yuen LAW Centre for the Advancement of Information Technology in Education The Chinese University of Hong Kong Shatin, N.T., Hong Kong [email protected] Abstract: Game-based Situated Learning (GBSL) is a web-based learning paradigm which is a combination of computer game and educational content, in which the learning context is charily designed to put a learner into a game-play environment that is similar or analogous to where the knowledge of the content can be applied in real life. Empirical evidence has shown that teachers’ perceptions are always significant in influencing the success of an educational innovation because they are the ultimate designers of learning and teaching activities in educational processes. The present study aimed to qualitatively analyze the perceptions of the teachers who had participated as facilitators in two comparative research studies on GBSL. Results showed that although the teachers were uncertain on whether GBSL can enable their students to have better learning outcomes, they were very positive towards the educational paradigm of this approach in terms of arousing students’ learning interest and motive. Coincidentally, they suggested a very similar 2-stage “blended” strategy for enhancing the existing GBSL approach. The 1st stage is to let students amusingly learn with a GBSL game in a learner-centred fashion, while the 2nd stage is for a teacher to correct, strengthen and further extend the knowledge that the students have learnt in the game; in fact, this is a process called “debriefing” which allows students to transform their game-play experience into learning experience. All these findings provided vital insights and a basis for further investigating the application and development of GBSL for learning and teaching. Keywords: Educational game, situated learning, game-based learning, web-based learning

1. Introduction The Hong Kong government introduced an ambitious IT in education strategy in 1998 [1] aimed to transform school education from a largely textbook-based teacher-centred approach to a more interactive student-centred approach with information technology, i.e., a “paradigm shift”. Six years after, “collected a rich repository of digital education resources” [2] was one of the remarkable highlights of the achievements made by the government: “Schools, teachers, tertiary institutions, and the private sectors and non-government organizations have produced over 20,000 digital curriculum resources and materials catering for schools’ needs, and many are available for sharing online”. Nevertheless, we should be cautious about confusing quantity with quality, as these resources may just continue to perpetuate teacher-centred approaches, rather than supporting learner-centred approaches. In fact, it is very dangerous in interpreting the full availability of Internet technology and online educational resources as a corresponding increase in ability to enable a paradigm shift to take place. Lee and Lee [3] argued that most

526

M.S.Y. Jong et al. / An Exploratory Study on Teachers’ Perceptions

of existing online learning systems have just been used as a repository of digitized educational materials, without taking the versatile advantages of the immense power of World Wide Web (WWW). With “intrinsic motivation” [4] and “situated learning” [5] as the theoretical basis, they proposed a Game-based Situated Learning (abbreviated as GBSL) approach which is a combination of computer game and educational content, in which the learning context is charily designed to put a learner into a game-play environment that is similar or analogous to where the knowledge of the content can be applied in real life. Does GBSL really work? In fact, this question should be answered in two directions. On one hand, we should measure the students’ learning outcomes and study how they learn with GBSL; on the other hand, it is definitely vital to study the teachers’ perceptions of this approach. Though many factors influence the success of adoption of an educational innovation, “teachers” are always significant throughout the whole process [6], as they are always the ultimate designers of the learning and teaching activities in school education. Kerr [7] asserted that “an educational innovation can whether reach to students, is greatly dependent on their teachers; teachers usually formulate their teaching style by themselves, their perceptions of the innovation should be crucially considered”. The present work aimed to qualitatively analyze the perceptions of 6 secondary school teachers who had respectively participated as facilitators in two GBSL comparative research studies conducted in July 2005 and December 2005 in Hong Kong, and all of the data being analyzed in the study were mainly collected through individual in-depth interviews with the teachers.

2. Web-based Learning Paradigms Since the widespread of the Internet and WWW in early 90’s, web-based learning has become very prevalent and intuitively viewed as a mechanism for empowering improved learning outcomes, increased flexibility of aligning individual needs of learners, and better quality of educational interactions. Plenty of newly made or digitized learning and teaching materials are wholly available on the Internet. It is not hard to imagine that if you make a simple Internet search on an educational topic, thousand or even more links of relevant “educational” resources will be automatically displayed on your computer screen. “Anytime”, “anywhere” and “any pace” are all familiar attitudes inherited to these “read-to-use” web-based resources.

2.1. Conventional Style of Web-based Learning In the early stage of the WWW history, “hypertext” was widely recognized as a powerful medium for disseminating information and enabling non-sequential reading [8]. In order to prevent online readers getting trapped in the hypertext environment, a clear and effective navigation menu bar, headings and sub-headings have been strongly recommended for every learning website. Chunking pieces of text into smaller “self-contained” paragraphs with bullets has also been another vital technique for putting educational content online. On top of that, according to the well-known jargon of “a single picture is worth a thousand words” [9], adding pictorial, schematic, symbolic or figural graphics to enhance the online content always sounds a crucial act. In fact, a lot of web-based educational resources have been designed in the fashion that goes with these “golden” features, and conventionally learning with this sort of resources is conducted by navigating between pages of content within a web browser or a learning management system (LMS), namely “Traditional Web-based Learning” (TWBL) in this paper.

M.S.Y. Jong et al. / An Exploratory Study on Teachers’ Perceptions

527

2.2 Game-based Situated Learning One of the frequently received learners’ negative feedback of conventional style of web-based learning is “hard to stay motivated” [10]. Moreover, the traditional fashion of chunking or fragmenting learning materials may eventually end up with creating unrealistic learning context and depriving the rationale behind the knowledge itself. Lee and Lee [3] also criticized that most of existing web-based learning and teaching approaches have not yet been fully benefited by the pervasive web technologies, and most of existing online learning systems have just been used as a repository of digitized educational materials, without taking the versatile advantages of the immense power of WWW and the frontier of online learning should be further extended. They proposed a Game-based Situated Learning (GBSL) approach, and together with their research team have been conducting various research and development works on this area (e.g., [11], [12], [13]). GBSL is a union of game-based learning [14] and situated learning [5]. As suggested by the name, game-based learning is marriage of a computer game and learning, in which the educational content and context are smartly designed as a game-play environment for a learner to acquire various knowledge and skills in a particular subject domain. It works because computer games are fun but hard, engaging but competitive, pleasurable but challenging, rewarding but risk-taking. Bisson and Lunckner [15] argued that enjoyment and fun are extremely vital when learning new things because learners are relaxed, motivated and more willing to learn; in fact, this assertion aligns with Malone’s “intrinsic motivation” theory [4]. Some empirical evidence also shows that game-based learning is an effective means for empowering learners to understand complex subject matters [16]. Learning is an active process based upon concrete experience [17]. However, Papert [16] argued that the fragmentation of knowledge in school curricula into little pieces with the original intention for making learning easy often ends up with depriving the rationale behind the knowledge itself, creating unrealistic learning contexts and making learning boring. A good game-based learning approach, on the contrary, without chunking or turning educational contents into learning objects or a series of split-screens, it presents near real-life situations for learners. This is situated learning [5], in which the learning takes place unintentionally rather than deliberately in an environment that is similar or analogous to where the knowledge and skills can be applied in the future. With “intrinsic motivation” and “situated learning” as the theoretical basis as well as the merits inherited from game-based learning, Tong Pak Fu and Chou Heung: The Probabilistic Fantasy (TPFCH-PF, http://www.cse.cuhk.edu.hk/~mhp) is a GBSL game for probability learning, developed by the Centre of the Advancement of Information Technology in Education (CAITE), the Chinese University of Hong Kong. The storyboards of TPFCH-PF are based on a well-known Chinese fantastic folklore “Tong Pak Fu and Chou Heung”. Tong Pak Fu was a legendary poet, scholar, painter as well as womanizer in the Ming Dynasty. The folklore started with one day Tong in a busy market square saw Chou who was a maid working in the Wah Mansion. Although Tong had already got eight wives, he still could not resist the “electrified pulse” induced by Chou. He was deeply attracted. After many unsuccessful flirting and dating attempts, Tong finally won the heart of Chou and they fell in love. Tong wanted Chou to be his 9th wife; however, the master of Chou, Mrs. Wah required Tong to take and pass an ultimate exam, otherwise she should have not allowed Chow to marry him. In the exam, Tong had to pick Chou from a group of face-masked brides within a limited number of guessing attempts. The game-play skeleton of TPFCH-PF is composed of 5 different courtship stages of Tong toward Chou. In the game, a learner is situated in these five courtship stages and plays the role of Tong to court and date his beloved lady Chou. In each stage there is one “large” probability problem which is composed of several sub-problems. The “Wise Genie” who is

528

M.S.Y. Jong et al. / An Exploratory Study on Teachers’ Perceptions

one of the important characters in the game who will offer helps to the learner when s/he is being trapped by the problems; nevertheless, the genie will only give advice to the learner rather than explicit solutions. Therefore, s/he has to proactively analyze the perceived content and context in the game, and solve each problem by understanding the concepts underpinned to the solution to that problem, instead of just looking at boring computer displays of information.

3. Design of the Study The present study was a piece of second-tier of qualitative analysis on the perceptional ideas and opinions of 6 secondary school teachers who had respectively participated in facilitating the approximately 3-hour learning experiments in two experimental-control comparative research studies on GBSL and TWBL in Hong Kong. The first comparative research study was conducted in July 2005 which aimed to compare the educational effectiveness of these two web-based learning paradigms for self-directed learning proposes, in which a convenient sample of 158 Secondary 4 (comparably equivalent to K-10) students and 4 Math teachers from 2 secondary schools (indicated as School A and School B) voluntarily participated. The second comparative research study was conducted in December 2005 which aimed to compare the educational effectiveness of these two online learning paradigms for knowledge reinforcement purposes, in which a convenient sample of 72 Secondary 5 (comparably equivalent to K-11) students and 2 Math teachers from a secondary school (indicated as School C) voluntarily participated. In both studies, TPFCH-PF (http://www.cse.cuhk.edu.hk/~mhp) was adopted as the learning material for the experimental group students to perform GBSL; on the other hand, another online learning resource in the fashion of TWBL but covering exactly the same amount of learning content as TPFCH-PF was explicitly created for the control group students to learn with in the studies (http://www.cse.cuhk.edu.hk/~mhp/note). All data being analysed in the study were all qualitative in nature, which were mainly collected through in-depth semi-structured interviews with the teachers within 1 week after they had respectively participated in the 2 comparative studies. The interviews focused on how they perceived the GBSL approach, in terms of its educational value and applicability in learning and teaching, as well as the comparison between GBSL and TWBL. Each interview lasted about 30 minutes and was conducted in Cantonese, audio-recorded and later transcribed for analysis purposes. Table 1 shows the background information of the teachers. Table 1. Background information of the teacher participants School

Teacher

Sex

Age Group

A

Teacher 1 Teacher 2 Teacher 3 Teacher 4 Teacher 5 Teacher 6

F M M M M M

31-40 31-40 41-50 26-30 41-50 41-50

B C

Year of Teaching Experience 16-20 10-15 10-15 5-9 16-20 10-15

Participating in the research conducted in

Teaching Subjects

7/2005 7/2005 7/2005 7/2005 12/2005 12/2005

Math, Computer, Civil Education Math Math Math, Integrated Science Math Math, Computer

4. Findings The “categorizing” strategy advocated by Maxwell [18] was mainly employed in the present study, and the emerged thematic categories were abstraction derived from the data, not the data themselves.

M.S.Y. Jong et al. / An Exploratory Study on Teachers’ Perceptions

529

4.1 Educational Value of GBSL It was found that all of the teachers were quite positive towards the GBSL approach in terms arousing students’ learning interest and motive, especially at the beginning stage of learning a new topic: “The students were quite engaged and devoted when learning with the game. They tried to solve the problems in the game even they were unsure the answers to those probability problems ……” (Teacher 1) “After finishing the game, they used the Internet search engine to search some probability terms such as “conditional probability” …… I was surprised that they were so ‘proactive’. It is a very good learning motivator …” (Teacher 2) “I can assert that it is a very good approach for arousing students’ learning interest, especially when teaching a new topic at the startup stage.” (Teacher 3) “This game is a very good learning motivator, especially for the students who have less motive in learning.” (Teacher 4) “Although the research aim this time was to investigate the use of the game for reinforcement proposes, I think if I use it again next year, I will use it as a learning motivator rather than use it as a reinforcement agent.” (Teacher 5) “The students could be immersed into some interesting scenarios that never appear in textbook-based teaching.” (Teacher 6) Some of the teachers highlighted that GBSL could make learning more authentic: “A very good chance for them to appreciate Math knowledge is not only for answering exam questions, but also for solving real-life problems.” (Teacher 1) “It offered a chance for the students to realize probability learning can be so interesting and the concepts can be applied in their daily life.” (Teacher 5) Nevertheless, they showed their uncertainty on whether it could have enabled their students to gain better learning outcomes: “ I am really not sure how much my students could learn from it .. I am afraid they might have been detracted by the graphics in the game, and forgot that it was a learning activity.” (Teacher 2) “I think the students should be able to learn something, but I am not sure how much the knowledge covered in the game can retain in their memory.” (Teacher 3) “According to my observation, it seemed that some students were just using a ‘trial and error’ strategy to solve the probability problems in the game, without gaining the real understanding of the concepts.” (Teacher 4 and in fact Teacher 5 and Teacher 6 also gave the similar comment) 4.2 Contrasts between GBSL and TWBL The teachers thought that GBSL is a much better approach than TWBL in terms of motivating students to learn: “I don’t think the TWBL approach can arouse the students’ learning interest as GBSL does.” (Teacher 2 and in fact Teacher 1, 4 and 6 also gave the very similar comment) “The TWBL is just like a text-book …… it is lack of interaction and the students can only learn passively…since all students own their text-book already, I don’t think they still need TWBL.” (Teacher 6 and in fact Teacher 3 and Teacher 5 also gave the similar comment)

530

M.S.Y. Jong et al. / An Exploratory Study on Teachers’ Perceptions

Nevertheless, some of them realized that the TWBL material is more flexible and convenient when reviewing or re-studying the learning content is needed: “The flow of the GBSL game is based on situations or scenarios, it is quite inconvenient when a student wants to make an ad-hoc review on specific knowledge points”. (Teacher 1) “The good stuff offered in TWBL is non-linear access of the learning content.” (Teacher 4) “The TWBL material provided flexible navigation … the students could navigate the content very easily, especially when they wanted to review or reinforce the concepts that they just learnt.” (Teacher 5) In fact, all of the teachers did not comment much on TWBL in the interviews. Although some “explicit” questions were being asked (e.g., how would you compare GBSL and TWBL etc.), they just briefly elaborated their ideas. This actually reflected the teachers were not quite enthusiastic towards the TWBL approach. 4.3 Enhancement on GBSL As mentioned in Section 4.1, although all of the teachers very much believed that the GBSL approach could arouse their student’ learning interest and motivate them to learn, they were uncertain whether it could really help their students to have better learning results. Actually in the settings of the comparative research studies, the teachers served mainly as a modulator (or an observer) in the student learning experiments, rather than as a real learning facilitator. The teachers stressed in the interviews that the GBSL approach should not only involve the students, but also themselves: “Even if there is an excellent game for GBSL, I don’t think students can learn all the stuff embedded in the game by their own … just gaining learning motive is not enough, the involvement of the teacher is very crucial.” (Teacher 2 and in fact all other teachers raised the similar idea) They realized that the game works better as a learning motivating agent than as an enhancement agent, and interestingly, all of them suggested a very similar “blended” strategy for enhancing the GBSL approach. The suggested “blended” strategy is composed of 2 stages; the first stage is to let students amusingly learn with the game and acquire knowledge in a learner-centred fashion. The second stage is a reinforcement process in which a teacher should correct, strengthen and further extend the knowledge that the students have learnt in the game: “I prefer to let my students to learn with the game first, and then followed with some reinforcement sessions …… obviously, it is unable to ensure that they can learn everything right in the game. These reinforcement sessions are for me to make sure the students can learn the right things …” (Teacher 1) “I think this game should not be just treated as a supplementary material ... in fact, it is a very good means to empower the students to acquire some basic but vital concepts of probability ….. although they may not be able to fully understand every concept emerged in the game, their learning interest is already aroused. The thing I need to do next is to ‘clear’ up their confusion and misunderstanding, and enhance their understanding …” (Teacher 2) “In fact, after the learning experiment had finished, some students in the experimental group told me that they still had confusion on some probability

M.S.Y. Jong et al. / An Exploratory Study on Teachers’ Perceptions

531

concepts emerged in the game …. this shows that they need reinforcement. I will spend 2 to 3 lessons to reinforce their gained knowledge.” (Teacher 3) “I believe the students should have already learnt something in the game, but obviously, not every student could fully understand the learning content. It is very important to give some reinforcement to the students and this should be done by us …… we act as a safety-net of learning.” (Teacher 4) “I think a teacher is always the best at seeing exactly when, what and why students have difficulties and guiding them to look for possible solutions in the process of learning ….. I think I can play the role of reinforcement agent better than the game.” (Teacher 5 and in fact Teacher 6 also elaborated the similar idea)

5. Conclusion and Discussion Although there has been a great promotion of a shift in education from the traditional, didactic model of instruction to a constructivist model that emphasizes students’ active learning, this new paradigm still strongly advocates the vitality of teachers in the educational setting [19]. “Teachers do things for good reasons only” [20]; in fact, empirical evidence has shown that (e.g., [6], [7]) teachers’ perceptions are always significant in influencing the success of an educational innovation because they are the ultimate designers of learning and teaching activities in educational processes. In the present study of Game-based Situated Learning, all of the participating teachers were quite positive towards this approach in terms of arousing students’ learning interest and motive, although they were skeptical on whether their students could have really gained better learning outcomes with this approach. Some of the teachers also highlighted that a good design of GBSL can provide more authentic learning context for students to learn with. In overall, they valued GBSL much higher than TWBL; nevertheless, they still advocated the flexibly non-linear access feature of TWBL. On the other hand, they believed that teachers are still the best at seeing exactly when, what and why students have difficulties and guiding them to look for possible solutions in the process of GBSL. They suggested a 2-stage “blended” strategy for enhancing the existing GBSL approach. The first stage is to use game-play to scaffold students to acquire some basic knowledge of the topic in a more amusing way. The second stage is for a teacher to correct, strengthen and further extend the knowledge that the students have learnt in the game; this is a process called “debriefing” allowing the students to engage in metacognitive thinking which transforms their game-play experience into learning experience [21], i.e., reflection. Many educational game researchers have asserted that debriefing is the most critical part of the whole game-based learning process (e.g., [14], [21]). In fact, even if students are highly motivated and immersed into the knowledge and concepts underpinned to a very good GBSL game, it is still not always clear that they can leave with exactly the cognitive level that we want them to. To do this they need to reflect, as “experience plus reflection equals learning” [22], and this remains a crucial task for a GBSL teacher who should look for and act on every “debriefable” moment to “lift” students out of particular situations in the game, and empower them to generalize their gained knowledge. Under the research settings of the 2 comparative studies, the learning experiments took only approximately 3 hours. Within the short period of time, it may be hard to let the participating teachers to fully perceive the learning effectiveness of GBSL, and on the other hand to investigate the students’ learning outcomes. In fact, Lave and Wenger [5] suggested that situated learning is a relatively long-term approach which needs time to immerse learners in a learning environment so that they can benefit from the behaviors and activities

532

M.S.Y. Jong et al. / An Exploratory Study on Teachers’ Perceptions

associated with the context. In order to have a closer look on this approach, further development of GBSL resources with a wider range of intra or inter disciplinary topics for carrying out another relatively longer-time experimental study is the next crucial step we should work on. References [1] Education and Manpower Bureau (EMB). (1998). Information technology for learning in a new era: Five-year strategy 1998/99 to 2002/03. HKSAR: EMB. [2] Education and Manpower Bureau (EMB). (2004). Empowering learning and teaching with Information technology. HKSAR: EMB. [3] Lee, J.H.M. & Lee, F.L. (2001). Virtual Interactive Student-Oriented Learning Environment (VISOLE): Extending the frontier of web-based learning. The scholarship of teaching and learning organized by University Grant Council, HKSAR. [4] Malone, T.W. (1980). What makes things fun to learn? A study of intrinsically motivating computer games. Palo Alto: Xerox. [5] Lave, J. & Wenger, E. (1990). Situated learning: Legitimate peripheral participation. Cambridge. UK: Cambridge University Press. [6] Olson, L. (1993). Alliance aims for ‘break the mold systems’, not just schools. Education Week, 12(4), 8-10. [7] Kerr. S. (1996). Visions of sugarplums: The future of technology, education, and the schools. Technology and the Future of Schooling. Chicago, Illinois: University of Chicago Press. [8] Horn, R. (1989). Mapping hypertext. Lexington Institute. [9] Li, W.S. (2001). A single picture is worth a thousand words: The effect of images on online learning content. Proceedings of the First Teaching and Learning Symposium (pp. 173-180). HKUST. [10] Morris, E., & Zuluaga, C. (2003). Educational effectiveness of 100% IT courses. Processing of 20th ASCILITE Conference. Melbourne, Australia. [11] Lee, F.L., Lee, J.H.M., & Lau, T.S. (2002). Fantasy-based learning on the web - Tong Pak Fu & Chou Heung: The probabilistic fantasy. Paper presented at IEEE International Conference on Advanced Learning Technologies. Kazan, Russia. [12] Jong, M.S.Y., Shang, J., Lee, F.L., Lee, J.H.M., & Law, H.Y. (2006). Learning online: A comparative study of a situated game-based learning approach and traditional web-based learning approach. In Z. Pan, R. Aylett, H. Diener, X. Jin, S. Gobel, & L. Li (Eds.), Proceedings of the 1st International Conference of Edutainment 2006: Technologies for E-Learning and Digital Entertainment (pp. 541-551). Lecture Notes in Computer Science, Springer. [13] Shang, J.J., Lee, F.L., Lee, J.H.M., & Chau, K.C. (2005). The intrinsic motivation of the online game and its application in VISOLE. Proceedings of the 9th Global Chinese Conference on Computers in Education. Hawaii, USA. [14] Prensky, M. (2001). Digital game-based learning. New York: McGraw Hill. [15] Bisson, C. & Lunckner, J. (1996). Fun in learning: The pedagogical role of fun in adventure education. Journal of Experimental Education, 9(2), 109-110. [16] Papert, S. (1993). The children’s machine: Rethinking school in the age of the computers. NY: Basis Books. [17] Piaget, J. (1964). Development and learning. Journal of Research in Science and Teaching, 2, 176-186. [18] Maxwell, J.A. (1996). Qualitative research design: An interactive approach. CA: Sage. [19] Howard, J. (2002). Technology-enhanced project-based learning in teacher education: Addressing the goals of transfer. Journal of Technology and Teacher Education, 10(3), 343-364. [20] Miller, L., & Olson, J. (1994). Putting the computer in its place: A study of teaching with technology. Journal of Curriculum Studies, 26, 121-141. [21] Crookall, D. & Saunders, D. (1989). Towards an integration of communication and simulation. Communication and simulation: From two fields to one theme. Clevedon, UK: Multilingual Matters. [22] Dewey, J. (1938). Experience and education. New York: Macmullan.

Visualization

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

535

Visualizing Errors for Self-correcting Discrepancy between Thinking and Writing Hidenobu KUNICHIKA1, Tsukasa HIRASHIMA2, Akira TAKEUCHI3 1 Dept. of Creation Informatics, Kyushu Institute of Technology, Japan 2 Dept. of Information Engineering, Hiroshima University, Japan 3 Dept. of Artificial Intelligence, Kyushu Institute of Technology, Japan [email protected] Abstract: The aim of our research is to realize a system for supporting English composition in second language learning which allows learners to compose some sentences and gain appropriate awareness of their errors in the sentences with indirect information from the system. Learners often make errors, and they can learn from them. In order to learn from the errors, learners need to first be aware of the errors; preferably by themselves or by indirect information. However, learners may fail to notice the errors or they may lack correct awareness of their errors unless they are given good stimuli (i.e., indirect information) from the outside. This paper proposes a method when and how the system provides stimuli to the learners, describes its implementation briefly, and reports validity of the method. We adopt a method of visualizing learners’ errors as indirect information for the learners. Keywords: ICALL, English Composition, Reflection, Error Visualization, Animation

1. Introduction Children make mistakes and learn from them. Learning is much more meaningful if the child is allowed to experiment on his own rather than listening to the teacher lecture. The teacher should present students with materials, situations and occasions that allow them to make their own discoveries [6]. In order to learn from mistakes, learners need to first be aware of the mistakes. Ways to be aware of mistakes are classified into the following three types. x By themselves: Learners reflect what they did and how they did it without stimuli from the outside world and become aware of their mistakes on their own. x By indirect information: Learners are given indirect information (e.g., hints) and from these outside stimuli gain awareness of their mistakes. x By direct information: Learners are given direct information about their mistakes from someone such as teachers or other students, and from this information gain awareness of their mistakes. It is difficult to gain appropriate awareness of mistakes. That is to say, learners may fail to notice mistakes or they may lack correct awareness of their mistakes unless they are given stimuli from the outside. In order to correct misunderstanding or acquire a meta-cognitive ability to be aware of mistakes and correct them, it would be effective to have experiences which learners found and corrected some mistakes. The experiences being aware by themselves and by indirect information will be remembered more easily than being aware by direct information, because when we actively discover something by ourselves, the discovery leaves deep impression for us [6]. Figure 1 shows these relationships.

536

H. Kunichika et al. / Visualizing Errors for Self-Correcting Discrepancy

Correctness low

Ease to remember high

high

low

- By themselves - By indirect information - By direct information

Figure 1. Features of ways to be aware of mistakes The aim of our research is to realize a system for supporting English composition in second language learning which allows learners to gain appropriate awareness of mistakes by giving indirect information. We believe that learners will become aware of their own mistakes by themselves after repeating experiences in finding and correcting their mistakes with stimuli from the outside world. This paper presents a method of error visualization [3] in order to increase awareness of mistakes. In the following sections, we describe error types which our method visualizes and a method of visualization for each type of error. Then, we explain the outline of our system and a method of generating animation for visualization. After that, we describe whether or not subjects have conflict awareness with our method and the applicability of our methods as an evaluation.

2. Our Target and Learning Model 2.1 Our Target The target learners are beginners in English, that is to say, we assume that the vocabulary and grammar used by the learners are basic and taught in Japanese junior high school. Learners make various sorts of errors in English composition. The errors are classified into the following three types: (1) syntactic errors, (2) semantic errors and (3) discrepancy between thinking and writing (DTW). Recent research on computer-assisted language learning (CALL) is focusing on Intelligent CALL (ICALL) systems, which apply artificial intelligence techniques in CALL environments. Natural language processing techniques, intelligent help systems and user models have been integrated into language learning software in order to improve the learning process [1]. Current ICALL systems [e.g., 1, 2, 5] help learners correct error types (1) and (2) by giving hints, example sentences or instructional messages. Type (3) is the sort of error in which a learner writes is different from what she wants to (should) represent. Because it is difficult to identify such DTW errors, current ICALL systems do not support learners in the correction of DTW errors except by informing them that the answers are incorrect. We focus on supporting learners in the correction of DTW errors. A sentence which corresponds to one event1 is composed of objects, their attributes2 and the case elements of the event. Therefore, DTWs appear as differences, i.e. lack, excess and replacement, between the components of a sentence a learner writes and the components of what she wants to represent. That is to say, DTW errors can be farther classified as follows: information lack, information excess and information replacement. The target of our method is to support learners to gain appropriate awareness of these errors by giving animation.

1 2

At present, our system limits that each sentence expresses one event. Our method processes visible events, visible objects and their visible attributes. At present, the events are movement of objects, change of visible attributes of objects and appearance and disappearance of objects.

H. Kunichika et al. / Visualizing Errors for Self-Correcting Discrepancy

537

2.2 Learning Model of Composition and Awareness To give learners stimuli, we have several methods, including giving hints in English or the mother tongue, giving example sentences which contain the same mistakes, and giving correct example sentences. The goal of our method is to emphasize DTW errors by animation visualizing the content of the sentences. We expect that the emphasizing such a discrepancy triggers a conflict awareness enabling the learner to reflect on her activities and revise the sentences in order to reduce the conflict between what a learner wants to represent and an animation corresponding to what she actually wrote. We assume that people have sort of images which correspond to what they want to represent in their mind when they compose sentences. If learners compose sentences intending to represent such images, giving learners feedback in the same form as the images ought to have a strong impact on the learners. Therefore, our method visualizes DTW errors as animation and gives the animation to learners. Figure 2 shows our learning model. First, a learner composes a sentence from images in her mind. Then, our system interprets the sentence and generates animation from the sentence. Next, the learner compares the animation with the images in her mind. After that, she has a conflict awareness, reflects on her activities and revise the sentence if the sentence has any DTW errors. The learner repeats the processes and finally composes a correct sentence.

Figure 2. Learning model

3. Error Visualization for English Composition This section describes different types of DTW errors described in 2.1 and a method of visualizing them. 3.1 Information Lack Information lack describes cases where a learner does not represent the necessary information in the sentences she has composed. This section explains errors and their visualization for each component of the sentence. (1) Objects There are the following two cases. First, although a learner used a pronoun or words together with a definite article, the referents can not be identified because the expressions are inappropriate; second, a learner used no words corresponding to the object. In the former case, our system generates animation which shows that there is no referent. For example, when a dog has not appeared yet and a learner inputs "The dog runs around the tree.", our system projects a "?" moving around the tree because to indicate that there is no referent of "the dog". The latter case is judged as a syntactic error or classified into lack of case elements of events described later. If the lack is syntactic, it is handled by our natural

538

H. Kunichika et al. / Visualizing Errors for Self-Correcting Discrepancy

processing module, and it is beyond the scope of this paper. Learners will be instructed syntactic errors. (2) Attributes of objects There are cases in which attributes of objects should be represented, such as one tall tree and one small tree. If a learner composed a sentence without a necessary attribute of an object, our system generates an animation with particular values of the attribute in order to realize the errors. For example, when a learner should input "a short tree" but instead uses "a tree", our system will show a picture of a tall tree. (3) Case elements of events The system generates animation to make the learner realize errors of necessary case elements. For example, when a learner inputs "The monkey throws the persimmon." for the animation which represents "The monkey throws the persimmon toward the crab", the system generates an animation in which the monkey throws the persimmon in another direction. 3.2 Information Excess The system generates animation which represents the content of a sentence just as learner inputs it, even if the sentence has an excess of attributes for the objects and case elements. For example, when a learner should input "The monkey climbs a tree." but inputs "The monkey climbs a tall tree.", our system generates an animation in which a monkey climbs a very tall tree. 3.3 Information Replacement Information replacement errors occur when a learner uses different objects, attributes, case elements or events from what the learner wants to represent. For these errors, the system generates an animation which represents the content of the sentence the learner has. For example, when a learner wants to refer to the dog which has already appeared and she uses an indefinite article, e.g. "a dog", another picture of a dog appears in the animation, that is, pictures of two dogs are shown against her expectation.

4. The Learning Support System for English Composition We have been constructing a learning support system for English composition. The system gives learners animations reflecting errors in their English sentences in order to make them aware of the errors. Here, we describe the interaction between a learner and the system, and the required functions of the system. 4.1 Learning with the System As Figure 3 shows, our system consists of a natural language processing module [4], a knowledge processing module including a conceptual dictionary, and an animation generation module. The natural language processing module interprets English sentences which are inputted by users, and extracts “case frames” from the sentences. The knowledge processing module, which has a concept dictionary, generates an internal representation called “state transition information”. Then, the animation generation module shows animations to the users following the state transition information.

H. Kunichika et al. / Visualizing Errors for Self-Correcting Discrepancy

539

Figure 3. Outline of our system The system works in two stages: an authoring stage and a learning stage. In the authoring stage, the system helps authors (e.g., instructors, teachers) prepare learning materials. When the authors input natural language sentences (A-1 in Figure 3), the system automatically extracts “original case frame”, generates “original state transition information”, and generates “original animation” (A-2). The authors also specify case elements which learners should express in the sentences. In the learning stage, the system provides a learner images, which she should represent, by animation (L-1 in Figure 3) because the system will not support English composition if a learner composes sentences from her own images. The animation represents the original state transition information. When the learner composes sentences (L-2), the system extracts “learner’s case frames”, identifies DTW errors, and then, generates "learner's state transition information" which reflects the errors. After that, the system shows an animation corresponding to the learner's state transition information (L-3). The animation represents how the sentences are interpreted by the system. The learner is expected to compare the animation with the original animation, have a conflict awareness if the animations have any discrepancy, and reconsider the sentences composed by the learner in order to reduce the conflict. Figure 4 shows a snapshot of the interface in the learning stage. 4.2 Difficulties of Generating Animation and Error Visualization In general, it is difficult to generate animation from natural language sentences even if the sentences are syntactically and semantically correct, because the sentences lack some kinds of information; e.g. common sense, which people can supplement, and attributes of objects such as position and color which are necessary for animation generation. Furthermore, learners’ sentences may have no information which they should represent. That is to say, when a learner composes sentences, there will be two types of information lacks: Lacks which the learner does not need to express in the sentences and lacks of which the learner should be aware such as an error. Moreover, learners will make different types of errors: information excess and information replacement. So the system needs a function to distinguish the lacks and to make an animation when the errors are included. - how to fill the information lacks to generate an animation As shown in Figure 5, a story (i.e., sequence of sentences) is represented as state

540

H. Kunichika et al. / Visualizing Errors for Self-Correcting Discrepancy

Figure 4. A snapshot of the Learning Interface

Figure 5. An example of state transition information transition information which consists of a sequence of scenes. A scene consists of a background, a sequence of events which occur in the background, and snapshots of the scene which is changed by events. State transition information is represented by a frame as shown in Figure 5. Original state transition information is generated from an author’s sentences in the authoring stage and has all the information for generating animation of a story. In the learning stage, in order to complete learner’s state transition information, the knowledge processing module fills lacks of information which the learner does not need to represent by referring to original state transition information if the lacked information is included in it or referring to default values stored in a conceptual dictionary. - how to represent the errors The system generates animations matching the sentences composed by a learner, even if the sentences include some errors. For example, if a learner writes “a tree climbs a monkey”, the system shows an animation: there is a monkey, a tree appears, and the tree moves along the monkey from the ground to the top of the monkey. When the errors included in the sentences are information replacement or information excess, the system works well to represent the errors as they are. When the error is information lack, however, there is a problem in generating an animation. As described in the previous section, the system fills information lacks to generate an

H. Kunichika et al. / Visualizing Errors for Self-Correcting Discrepancy

541

animation. If the system does not distinguish the two types of lacks, the errors of information lacks are also filled and are not represented, and then learners may not be aware of the errors. So, the system detects the errors by confirming whether or not a learner’s sentence has necessary information, that is, case elements specified by the author in the authoring stage, and visualizes the errors. As mentioned before, our system processes events on movement of objects, change of visible attributes of objects and appearance and disappearance of objects. The attributes of events of movement are starting and ending position, and those of objects are position, color and size. The methods of emphasizing errors of each attribute are as follows. x The starting and ending position of the movement and position of an object: the system randomly decides a position which is a particular distance away from the correct position, and uses it. x Color: The system uses the complementary color of the correct color. x Size: The system uses the value which corresponds to the antonym of the correct word.

5. Conflict Awareness and the Applicability of Our System In order to realize learning from errors along the learning model mentioned in the previous section, there are some questions. They can be classified into two categories as follows. (1) Can our system grasp the contents of sentences inputted by learners? x Can learners understand what they should write by watching original animation and compose sentences? x Do learners compose sentences using assumed vocabulary and grammar although ways of expressing a scene are not unique. (2) Can learners be aware of errors? x Can our system generate animations from sentences inputted by learners? x Do visualized errors have tangibility? We have investigated the above two points in order to confirm whether or not our method works well. 5.1 Method We showed each subject original animation which included one event by using our system, and asked her to express the animation in English. If the composed sentence has any DTW errors, she proceeds to the next step. Our system showed the learner’s animation generated from the composed sentence. After that, we asked her to identify the difference between the original animation and the learner’s animation, and to compose a sentence again. We used three animations which corresponds to the following three sentences respectively, and repeated the above process for each sentence. x A monkey climbs the tall tree. x A crab comes. x The monkey throws a green persimmon toward the crab. The number of subjects was 18. They were graduate and undergraduate students. 5.2 Result (1) Can our system grasp the contents of sentences inputted by learners? The subjects composed 90 sentences in total. In the sentences, 83 sentences (92%) are interpreted correctly and the system generated animations from the interpreted sentences.

542

H. Kunichika et al. / Visualizing Errors for Self-Correcting Discrepancy

The rest, that is, 7 sentences, are correct sentences but not interpreted because the sentences have words which are not stored in the conceptual dictionary. All the words which are not stored are verbs. For example, some subjects used “shoot” or “present” instead of “throw”. It seems that such subjects could not understand the appropriate verb to represent an event only from animation. In order to resolve the problem, it is necessary to support such learners; e.g. the system gives the learners choices of verbs if the learners can not use a verb which the system can interpret. (2) Can learners be aware of errors? In the interpreted 83 sentences, 59 sentences have DTW errors. Table 1 shows the numbers of errors and those been aware of for each sort of errors. As shown in the table, the subjects were aware of 66 errors (92%) out of 72 errors which the system visualized. Because almost errors were been aware of, we can say that our method is useful for allowing learners to gain appropriate awareness of errors. The main reason of failing to be aware of errors is that the animation generated from learner’s sentence is very close to the original animation. At present, the system emphasizes errors of information lack in animation, but generates animation which represents the content of a sentence just as learner inputs it for errors of information excess and information replacement. Therefore, there are cases in which errors are not visualized. It is necessary to confirm the visibility of errors of information excess and information replacement. Table 1. The numbers of visualized errors and those been aware of Visualized Errors Errors been aware of (Percentage)

Lack 26 25 (96)

Excess 17 12 (71)

Replacement

29 29 (100)

Total 72 66 (92)

6. Conclusions This paper described a method of visualizing DTW errors in order to allow learners become aware of their own errors by themselves. As the result of our experimentation, we found that our system can grasp the contents of almost sentences inputted by subjects although many sentences were inputted. Furthermore, the subjects were aware of almost DTW errors visualized by animation. Therefore, we can say that our method is useful for visualizing DTW errors. The remaining issues are to investigate the way of giving original animation for learners who do not understand an appropriate verb and to realize a method of confirming the visibility of errors of information excess and information replacement.

References [1] Gamper, J. & Knapp, J.: A Review of CALL Systems in Foreign Language Instruction, Proc. of AIED2001, pp.377-388 (2001). [2] Heift, T. & Nicholson, D.: Web Delivery of Adaptive and Interactive Language Tutoring, International Journal of AIED, Vol.12 (2001). [3] Hirashima, T., Horiguchi, T., Kashihara, A. & Toyoda, J.: Error-Visualization by Error-Based Simulation, International Journal of AIED, Vol.9, pp.17-31 (1998). [4] Kunichika, H., Takeuchi, A. & Otsuki, S.: An Authoring System for Hypermedia Language Learning Environments, and its Evaluation, Proc. of ICCE95, pp.73-80 (1995). [5] Levin, L. & Evans, D.: ALICE-chan: A Case Study in ICALL Theory and Practice. In Holland, V., Kaplan, J., & Sams, M. (Eds.). Intelligent Language Tutors: Theory Shaping Technology. NJ: Lawrence Erlbaum Associates Inc. Ch.5. (1995). [6] Piaget, J.: To Understand Is To Invent. NY: The Viking Press, Inc. (1972).

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

543

Using Systematic Animation to Teach Dynamic Science Concepts Othman Taliba, Shah Christirani Azharb, Nabila Abdullahc Faculty of Educational Studies, University Putra Malaysia, Malaysia Centre of Foundation Studies for Agricultural Science, University Putra Malaysia c Faculty of Education, MARA University of Technology, Malaysia [email protected] a

b

Abstract: It is argued that such animations should gradually facilitate students’ comprehension in stages before they could construct complete mental model of the concepts being explained. Subjects were randomly assigned to an experimental group (N=45) and a control group (N=40). The experimental group received lessons through teacher’s explanation using systematic step-by-step animation whilst the control group received lessons primarily based on teacher’s explanation using prepared transparencies. The pre-test and post-test analyses revealed that students in the experimental group demonstrated better conceptual understanding compared to those in the control group, thus showing evidence of their stronger conceptual understanding in comparison to the students in the control group. Keywords: Animation, electrochemistry

conceptual

understanding,

dynamic

science

concepts,

Introduction This article addresses a research carried out within the field of chemistry education. Specifically, the study looked into the design and development of effective animation to improve students’ conceptual understanding of a specific topic relating to electrochemistry namely that of the dynamic science concepts. It is argued that dynamic science concepts can be better explained using animation because they can bridge students’ existing ideas with new information and help students develop conceptual understanding. However, it is often the case of taking for granted that any animation is always effective over printed material or static illustration. The question now is, “What are the characteristics of animation which could facilitate students’ conceptual understanding of dynamic science concepts?” 1. Computer Animation 1.1 Theoretical Framework According to Paivio’s (1979) dual coding theory, our senses can select, attend to and eventually code two types of stimulus – verbal and visual - simultaneously. Building on this theory, Mayer and Moreno (2003) highlighted that during instruction, a learner mentally arranges selected words and images into coherent mental representations in his working memory. These representations, in verbal and pictorial modes respectively, go through the

544

O. Talib et al. / Using Systematic Animation to Teach Dynamic Science Concepts

process of organizing and re-organizing to form segmented units of information which will become meaningful upon integration with prior knowledge in the long-term memory. The contiguity principle on the other hand states that when imparting information, it is effective to present corresponding words and visuals simultaneously instead of separately (Michas & Berry, 2000). The principle has been found effective for students with minimal existing ideas because those students can only generate their own mental model with the aid of pictorial and visual presentations. In other words, the incorporation of animation in chemistry lessons enhances meaningful learning and deep understanding Mayer (2002) 1.2 Computer-based Animation To date, empirical research in technology-based instruction failed to convince the fact that computer-based animation is superior to static illustration. One of the reasons given is that many animations designed for teaching are limited to attracting students’ attention but not to activate their minds to understand dynamic concepts (Lowe, 2004). It is suggested here that animation should be presented to students through systematic discrete sequences which allow the teacher or instructor to control the flow of animation segment by segment. This can be done through sequential introduction of constructed animation using simple control buttons such as ‘Play’ and ‘Step back’. In the process of electrolysis, for example, students can explicitly see which electrons are donated and which element reduced or oxidized. These processes may be repeated for students to analyze their current understanding at every stage of the process. In addition to the construction of segment by segment animation under teacher control, teacher’s live explanation serves as scaffolds which help students concentrate on a particular stage of the process, thus facilitating conceptual understanding. Explanation must be given accordingly in stages so that students could understand each segment before moving on to the next. 1.3 Development of Systematic Animation The systematic animation program used in this study was originally designed and built by the researcher to teach dynamic concepts of electrochemistry using macromedia Flash MX authoring software. The animations discussed herein also differs from most interactive educational courseware in the sense that it became part of instructional activity rather than functioning only as supplementary material that students engage in isolation. The animations standing in for dynamic concepts in electrolysis, electroplating and salt bridge were designed as part of instructional presentation being integrated with live explanation from the instructor. The instructor used the conversational style (Mayer & Moreno, 2002) of explanation to highlight important points within the topics discussed. Designed in a systematic straightforward orientation, the animations run in stages using the ‘Play’ and ‘Step back’ control buttons as shown in Figure 1.

O. Talib et al. / Using Systematic Animation to Teach Dynamic Science Concepts

Step back button Play button

545

Additional link

Figure 1: Example of systematic animation of galvanic cell

No sounds and unnecessary text movements were incorporated in this prototype model, following Lux and Davidson’s (2003) suggestion whereby a good computer-mediated instruction should avoid the use of excessive sounds and unnecessary graphics. Rather, classroom instruction must emphasize clear, direct and distraction-free presentation. The process of creating animations chiefly involved the process of organizing the movement of atoms, ions and electrons at symbolic, macroscopic and microscopic levels. The organization of these dynamic elements provided highly focused segment by segment chemical processes. For instance, Figure 2 shows six segments of selected snapshots of silver electroplating. It illustrates simple diagrams of silver electroplating apparatus which include a beaker, electrodes, ionic aqueous solution and an electrical circuit. A circular round shape with chemical formula was used to represent a single ion, atom or molecule. Each segment is controlled by clicking the control buttons.

a. Electrons flow from the negative terminal of the cell to the iron spoon.

b. Cation (Ag+ ) moves toward the cathode.

546

O. Talib et al. / Using Systematic Animation to Teach Dynamic Science Concepts

c. The anode dissolves to form Ag+, releasing electrons. The anode becomes thinner.

d. Electrons flow from the anode to the cathode in the external circuit.

e. At the cathode, Ag atom is deposited on the surface of the iron spoon

f. The iron spoon is coated with silver

2. Objectives The study has two aims, namely (a) to develop systematic animation to facilitate conceptual understanding of matriculation science students on a specific topic relating to electrolysis, and (b) to compare the effects of such animation with conventional direct explanation using printed transparencies. The general research question to be addressed is: Do students exposed to systematic animation attain better post-test mean score than those exposed to conventional direct explanation using printed transparencies? 2.1 Method and Sampling Table 1 illustrates the experimental research design used in this study. Subjects were randomly assigned to two different groups; namely the control and experimental groups. The research population of the study was first-semester science matriculation students, between 18 to 19 years of age, at one of the matriculation centre in Malaysia. With respect to academic ability, they were on equal footing as all attained first grade in the Malaysian Certificate of Education examination. All of them had sound knowledge in secondary school level chemistry. One hundred and twenty first-semester science matriculation students were chosen from a potential research population of 250. Each subject was randomly given a number from 1 to 120. They were then assigned to form equivalent experimental and control groups through random allocation. To do this, subjects with odd numbers were assigned to the experimental group whilst those with even numbers allocated to the control group. Both groups consisted of a total of 60 students. However, only 85 students completed all sessions of the study, 45 students in the experimental group and the remaining 40 in the control group.

O. Talib et al. / Using Systematic Animation to Teach Dynamic Science Concepts

547

Table 1: Research Design Group

Illustration of design

Control Group

O1

X1

O2

Experimental Group

O1

X2

O2

Abbreviations and symbols:

X1 treatment for control group X2 treatment for experimental group O1 represents the process of first measurement (pre-test) O2 represents the process of second measurement (post-test) Arrows ( ) indicate the temporal order

2.2 Pre-test and Post-test The pre-test and post-test instruments were designed to evaluate students’ conceptual understanding of the targeted concepts in an electrochemistry topic using open-ended questions. Both tests were identical. The targeted concepts covered include topics of oxidation and reduction reaction, electrolyte, electrolysis and galvanic cell. Below are examples of the post-test questions: • • •

What allows the aqueous solution of CuSO4 to conduct electrical current? Explain your answer. Explain why the cathode in an electrolytic cell of molten PbBr2 is negatively charged whilst the cathode in a galvanic cell of Zn-Cu is positively charged? In a galvanic cell, would a more reactive metal be more likely to become an anode or a cathode? Briefly explain your opinion.

2.3 Procedure Two instructors were involved in the teaching of both groups. One of the Chemistry teachers at the centre volunteered to be an instructor for the control group. Miss Maria (not her real name) is an experienced Chemistry teacher with eight years of experience teaching Chemistry. Meanwhile, the researcher himself conducted classes for the experimental group. The study was conducted in three sessions. In session one, the pre-test was administered to both groups. The test took 40 minutes to complete. This was followed by session two, involving four electrochemistry lessons as treatment. Each lesson was designed as a normal fifty-minute lesson. Session three took place at the end of the treatment sessions. In teaching the CBI group, the instructor used prepared transparencies and presented the lesson in a conventional way through direct instruction approach. She started by giving an introduction to the lesson during the first few minutes of the lesson time. The next 40-50 minutes were the devoted to class activities as stated in the lesson plan. During the lesson, the students’ attention was directed to the illustrations drawn on the transparencies. The transparencies provide complete information of the main concepts as well as illustrations covering the lesson plan. These transparencies contain exactly the same content used for the CAnI group. The lesson ended with a 15 minute question-and-answer session through a teacher-dominated discussion, based only on the content provided to her. During the discussion, the teacher answered all questions by students immediately. She ended the lesson with a conclusion.

548

O. Talib et al. / Using Systematic Animation to Teach Dynamic Science Concepts

Teaching in the CAnI group was based on computer-animated instruction, with the subject matter being presented using a laptop computer and a data projector. At the beginning of the lesson, the teacher spent a few minutes asking questions to probe the students’ existing ideas. The next 40-50 minutes was devoted to the class activities as stated in the lesson plan. The presentation provide complete information of the main concepts as well as animations covering the lesson plan. The presentation contain exactly the same content used for the CBI group. At the end of the lesson, the instructor finished with a conclusion. A series of meetings were held between the researcher and Miss Maria before and after every lesson in order to discuss matters pertaining to the content of lessons planned, manner in which lessons would be carried out and materials provided for the lessons. The main objective of the meetings was to make sure that only the same subject content covered, learning objectives outlined and handouts given during the period of treatment so as to minimize instructor bias. 3. Results The normality of post-test score was tested using the Shapiro-Wilk tests of normality. Results indicated that the scores used for post-test are normally distributed (p>.05). Analysis of covariance (ANCOVA) was used for data analysis due to the presence of an additional variable (pre-test as covariate) and one independent variable. Comparisons made between the experimental and control groups were based on the assumption that both groups were similar with respect to their existing knowledge of the electrochemistry concepts before exposure to treatment. The outputs of independent samples test are shown in Table 2. Table 2: Independent samples test (Pre-test) t-test for Equality of Means

Levene's Test for Equality of Variances .

Equal variances assumed Equal variances not assumed

95% Confidence Interval of the Difference

F

Sig.

t

.532

.468

.254

.251

df

Sig. (2-tailed)

Mean Difference

83

.800

.136

.5349

-.9278 1.2000

73.25

.803

.136

.5429

-.9459 1.2181

Std. Error Lower Difference

Upper

The Levene's test showed no significant difference (t(83) = .254; p>.05), indicating that the assumption of equality of variance for the pre-test scores was met. Results for the independent sample showed no statistically significant difference between the pre-test mean score for the experimental and control groups (t(83)= .254; p>.05). Henceforth, both groups were considered significantly equivalent in their existing knowledge before the treatment.

O. Talib et al. / Using Systematic Animation to Teach Dynamic Science Concepts

549

The homogeneity of regression slopes was first tested to ensure that they meet the assumption that linear relationship between the covariate and the dependent variable should be at the same level for each group. Results of the ANCOVA procedure are shown in Tables 3 and 4. The tables indicate that there was statistically significant difference in the post-test mean scores of the students who were exposed to systematic animation (M=13.6, SD=3.63) in comparison to those who were exposed to the conventional method (M=10.24, SD=3.8895); F(1,80)= 19.70, p<.05, with an effect size of .196 (eta2 =.198). This suggests that 19.6% of the differences in the post-test scores were related to the differences in instructional methods. In other words, students who were exposed to systematic animation achieved a higher post-test mean score than those who were exposed to conventional, direct explanation using prepared transparencies.

Table 3 Descriptive statistics (Dependent Variable: Post-test) Groups Control Experimental Total

Mean 10.250 13.600 12.024

Std. Deviation 3.8895 3.6254 4.0912

N 40 45 85

Table 4: ANCOVA (Dependent Variable: Post-test) Source Corrected Model Intercept pretest group Error Total Corrected Total

Type III Sum of Squares 395.092 3140.506 157.439 246.396 1010.861 13694.000 1405.953

df 2 1 1 1 82 85 84

Mean Square 197.546 3140.506 157.439 246.396 12.328

F 16.025 254.755 12.771 19.987

Sig. .000 .000 .001 .000

Eta Squared .281 .756 .135 .196

The findings also showed that better post-test performance by the experimental group has nothing to do with the differences in pre-test score, as Pearson correlation revealed no significant relationship between the pre- and post-tests. For this data, the experimental conditions yielded a correlation coefficient (r = .18) but it was not significantly correlated (p>.05). This implies no linear relationship between the pre-test mean score and post-test mean score of students who were exposed to systematic animation. Therefore, the main hypothesis, which postulated that systematic animation facilitate students’ conceptual understanding, was supported. 4. Discussion and Implication The goals of this study were to develop systematic animation to facilitate conceptual understanding of dynamic science/chemical concepts and to compare the effects of such animation with conventional direct explanation using printed transparencies. Parametric tests revealed that students exposed to systematic animation demonstrated significantly better post-test mean scores compared to their counterparts in the control group. This

550

O. Talib et al. / Using Systematic Animation to Teach Dynamic Science Concepts

suggests that students in the experimental group experienced improved conceptual understanding better than did their counterparts. One possible explanation for the above outcome is that explanation of dynamic concepts accompanied by a segment of the animation allowed students time to relate important information before the teacher proceeds to the next segment. Here, each segment serves as a chunk in understanding the whole processes. On the contrary, the teacher in the control group tended to incorporate large amount of information and provide lengthy, continuous explanation using a single transparency. Typically, each transparency contains texts, symbols and chemical equations. Due to the substantial amount of facts and concepts introduced at symbolic, macroscopic and microscopic levels presented in one static illustration (on a single transparency), students were found less effective absorbing and had greater problems organizing those information within a short time. The results of this study contributed to the construction of systematic animation in the teaching dynamic concepts. When the goal of science instruction is to facilitate conceptual understanding, animation should be presented to students through systematic yet discrete sequences that allow teacher or instructor to control the flow of animation. References [1] Lowe, R. K. (2004). Interrogation of a dynamic visualization during learning. Learning and Instruction, 14, 257-274. [2] Lux, J. R., & Davidson, B. D. (2003). Guidelines for the development of computer-based instruction modules for science and engineering. Educational Technology & Society, 6(4), 125-133. [3] Mayer, R.E. (2003). The promise of multimedia learning: Using the same instructional design methods across different media. Learning and Instruction, 13, 125-139. [4] Mayer, R., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43-52. [5] Michas and Berry (2000). Learning a procedural task: Effectiveness of multimedia presentations. Applied Cognitive Psychology, 14, 555-575 [6] Paivio, A. A. (1979). Psychological processes in comprehension of metaphor. Cambridge University Press, London.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

551

Creating Animations in SVG format for Visualizing Program Execution Koji KAGAWA RISE, Kagawa University, Japan [email protected] Abstract: Web-based education environments for learning programming have several advantages in assisting learners. Especially, it is easy to use graphics and animations both as teaching materials and as the output format of programs. Novice learners often have difficulties in following the control flow of programs, especially when programs involve complicated control structures such as nested loops and recursion. Animations can help learners understand how computers execute programs by visualizing programs step by step. This paper presents a component to create SVG animations for explaining behaviour of Java programs. This component can be used as a part of a Web-based environment. It allows learners to avoid learning usage of additional software such as debuggers and keeps advantages of Web-based education environments. Keywords: Web, Programming, SVG, Java, Jakarta BCEL

Introduction A Web-based education environment for learning programming has several advantages. It allows learners to avoid inessential difficulties in learning programming such as software installation and configuration. It can keep learner’s record on the server-side and can provide contents according to their proficiency. It can also motivate learners by providing opportunities to communicate and cooperate with teachers and other learners. Finally, it is fairly straightforward to use graphics and animations rather than plain text as the output of programs. There are several graphics and animation formats viewable on Web browsers. Learners can draw simple shapes such as polygons and spirals by their programs and can also describe motions by changing parameters of shapes. Our group has been proposing a Web-based education system[1, 2] that uses a vector graphics and animation formatSVG (Scalable Vector Graphics) [3] as the target graphics format. The system works as follows. It offers simple support libraries, example programs and shows template programs on the Web client-side platform, where learners fill in short fragments of programs. Then, learners compile and run their programs to see the output as animations in the SVG format. Finally, learners upload programs (and animations produced by programs) to the server. Our system consists of an Eclipse plugin that supports creation of the SVG format, a supporting Java library and some server-side programs (JSP/Servlets). Our Eclipse plugin helps users do these protocols that are otherwise tedious and difficult for novices. Using graphics and animations can motive learners, compared to using ordinary ASCII characters. It can also increase quantity and quality of example programs. However, animations can help learners more. Novice learners of programming often feel difficulty in understanding the control flow of programs, especially when they involve complicated control structures such as nested loops and recursions. They often cannot understand how

552

K. Kagawa / Creating Animations in SVG Format for Visualizing Program Execution

programs and generated graphics and animations correspond. Therefore, we can use animations in order to visualize execution of programs step by step, for example, by highlighting the statement being executed in the source code. There are already existing visual debuggers that are useful for inspecting the behaviour of programs. However, it is a burden for learners to install and configure a debugger and to learn how to use the new software. Especially, often debuggers have too many functions for beginners to use since they are mainly designed for professional programmers. We would like to avoid forcing novices to use such software, since it breaks an important benefit of Web-based education environments. In this paper, we will present a component of Web-based education environments that creates animations for explaining behaviour of programs in the SVG format. Producing animations in one of Web standard formats has several benefits. Especially, since SVG is an XML-based format, it is relatively easy to edit and refine animations further both by other programs and by hand. The current target language of the proposed component is Java. It employs byte-code transformation using Jakarta BCEL (Byte Code Engineering Library) [5] in order to retrieve information such as line numbers and live variables. One may wonder why we do not use Java applets and the AWT library for animations. We have already stated the reason several times [2, 4]. Still, it is worth repeating here. x Though it is relatively easy to draw static graphics in AWT, it is not easy for novices to describe animationsin AWT, since it requires advanced notions such as thread. x It is difficult to further process and modify animations in the Java class format. For example, it is difficult to create thumbnails from Java classes. x Moreover, since Java applets are independent programs, Java applets tend to be slow to start up and therefore Web-pages with Java applets are sometimes unreasonably heavy. In contrast, SVG animations are easier to process by programs (and also by hand). Moreover, SVG animations start up reasonably quickly. In the rest of this paper, we first explain the design and implementation of our system (Section 1) followed by examples produced by the proposed component. We compare our work with alternative options (Section 2). Finally, we conclude and give future directions (Section 3).

1. Design and Implementation In this paper, we would like to present a component of a Web-based system that creates animations in the SVG format for explaining behaviour of Java programs. The component consists of two major phases. In the first phase, it converts the target program so that it can produce information about the current method and line number being executed and values of variables at each step, interleaved with the output of ordinary graphics commands. In this phase, we use Jakarta BCEL [5] to manipulate Java class files. Jakarta BCEL is a library to read, manipulate and write Java class files. Java class files have information about line numbers as well as names of local variables. We use the library to extract such information and inserts instructions for showing line numbers and values of variables. In the second phase, it processes both the source file and the result of the first phase. We simply modify the Java 1.5 parser in the examples in JavaCup [6] distribution. Then, it produces a JavaScript animation that highlights the line being executed in the source code (right) and draws shapes produced by the program (left), as is shown in Figure 1. It also shows values of variables and adds some buttons (bottom) for users to control animations such as play, stop and rewind buttons. It represents function calls by stacks of cards on which the program code and the current values of local variables are shown respectively.

K. Kagawa / Creating Animations in SVG Format for Visualizing Program Execution

553

Figure 1. Example (recursive functions) When a method (function) is entered, a new card is put onto the stack with some offset. When the function exits, the corresponding card is removed from the stack. This is for explaining behavior of recursive functions. Learners can view lower (mostly hidden) cards simply by putting the mouse pointer over the visible part of card. In order to cope with cases where the depth of recursion gets larger, we calculate the offset of k-th card when there are n cards in total by a formula U × (2 k -1+1) / 2 n -2 where U is a constant. The maximum offset is bounded to 2U. The rational behind this formula is that we rarely need information of deep (old) cards. Therefore, we assign larger offset for newer cards.

2. Comparison There are some alternative choices for retrieving information about line numbers and method invocation. We examined several options before using byte code transformation. First, we tried source code transformation technique that inserts necessary statements for printing line numbers and for noticing events such as method invocation. This approach is abandoned since the Java syntax is rather complex, it turns out that manipulating byte code is much easier. We also tried using JDI(Java Debug Interface) [7] to retrieve information. Unfortunately, it turns out to be too slow(it takes about 3 minutes for running a program that draws the Sierpinski curve of depth 4 on a PC with Pentium 4 – 3.00GHz CPU.). We have also proposed using mini-languages for learning programming languages in multiple programming paradigms [4]. We have shown that we can extend mini-language interpreters so that they can produce animations to explain execution of programs stepwise. The target languages (original mini-languages) and the implementation language (Objective Caml) are different from the proposal in this paper (Java/Java). Since the main purpose of this previous work is to implement interpreters for (somewhat exotic) mini-languages, functional languages such as Objective Caml are more suitable as the implementation language than Java. In this paper since the target language is the Java byte code, Java as the implementation language is a reasonable option. When we consider cooperation with popular Web platforms such as JSP/Servlet (server-side) and Eclipse (client-side), Java seems much more preferable. However, without generic classes introduced in J2SE 5.0, it

554

K. Kagawa / Creating Animations in SVG Format for Visualizing Program Execution

would be rather tedious to write programs that manipulate symbols like program source code and SVG. The target format of the previous work [4] is SWF, a vector graphics format used by Macromedia Flash. Therefore, comparison of SVG and SWF would be interesting. First, at the time of writing, SWF is more popular than SVG. Macromedia Flash player plugins are installed on almost all Web browsers while SVG viewer plugins are not so widely available. However, recent Web browsers such as Mozilla Firefox 1.5 and Opera 8 support the SVG format natively. Currently, Firefox native support for SVG is not complete and cannot play animations produced by our tool. The situation is, however, likely to improve in future. Second, SVG is an XML-based format while SWF is a binary format. The advantage of adopting XML-based formats is that it is easy to modify later by hand. For example, it is possible to modify an animation for a Java program into one for the C language when it only uses control structures common to both languages. This feature is valuable when improving the SVG generator. We can experiment improvements directly in the generated SVG files and then later feed back into the generator.

3. Conclusions and Future Work We have proposed a component for supporting learners to understand execution of Java programsby creating animations in the SVG format that explain behaviour of Java programs.Though, it does not offer sophisticated interfaces when compared to existing debuggers, it is important to keep burden of learners low. It would be also useful as a tool for teachers to create teaching materials. Since produced animations are in one of Web standard formats, they are freely used in Web pages. We would like to further contrive more intuitive and intelligible ways of presenting information by making good use of the power of the SVG format. For example, it will become more convenient if we can add freehand comments to animated graphics and designate key points for understanding programs.

Acknowledgments This research was partially supported by Japan Society for the Promotion of Science, Grant-in Aid for Scientific Research (B)(17300269).

References [1] Takai, K., Kagawa, K., and Tarumi, H. (2005) A web-based system for learning programming using a graphics format on WWW. The Third International Conference on Active Media Technology, 173–176. [2] Yanagisawa, N., Takai, K., Kagawa, K., and Tarumi, H. (2005) An Eclipse Plug-in for SVG Animations in an Educational System for Programming. The 13th International Conference on Computers in Education (ICCE 2005), 938–941 [3] Ferraiolo, J., Fujisawa, J., Jackson, D. et al. (2003) Scalable Vector Graphics (SVG) 1.1 Specification,. http://www.w3.org/TR/SVG11/. [4] Kagawa, K. (2005) Generating Teaching Materials with Graphical Mini-languages for Multiple Programming Paradigms. World Conference on Educational Multimedia, Hypermedia & Telecommunications (ED-Media 2005), 3469–3474. [5] Apache Software Foundation (2003) Jakarta BCEL (Byte Code Engineering Library) Manual http://jakarta.apache.org/bcel/manual.html [6] Scott Hudson (2006) Cup – LALR Parser Generator for Java™. http://www2.cs.tum.edu/ projects/cup/ [7] Sun Microsystems (2005) Java Debug Interface, http://java.sun.com/j2se/1.5.0/docs/ guide/jpda/jdi/

555

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

Effects of the Voice Recognition on the Writing of Students with Learning Disabilities Hu Lailin Dept. of Education Technology Wenzhou University, Wenzhou Zhejiang (325027) [email protected] Abstract: This paper uses the experiment method to study whether the dictation composition supporting by the voice recognition can improve writing quality of the student with writing disabilities or not. The investigation shows: the primary students are easy to master the voice recognition system. The voice recognition system responds to the demand of the speed that can satisfy dictating composition. Under the Chinese language environment, this method can promote composition level of the students with writing disabilities. Keywords: Voice Recognition; Dictation; Writing Disabilities

1. Introduction The writing disabilities (WD) is one aspect of the students with learning disabilities (LD) [1] .When the students have writing difficulties, the failure of other achievements will be caused. Therefore, many psychologists and educationists pay more attention to this problem. They put forward various solutions and carried on related experiments. The dictating composition is one of effective strategies [2]. Since 90's, along with the development of multi-media computer and the artificial intelligence, there appears the dictation by the voice recognition [3]. This way has not only the advantage of the dictation, but also the advantage of the handwriting. So the effect is the much obvious. How is the validity of dictation by the computer voice recognition under the Chinese language condition? Up to now this research still remains blank in China. So, we suppose the dictation by computer voice recognition can improve writing quality for students with writing disabilities under the Chinese language condition. 2. Method 2.1 Setting and Participants

Table 1: Descriptive Summary of Student Participants Name Sex Birthday IQ Language Rate ˄rank/total˅ Lin Junyi(S1) man ZhouSunguo(S2) man Wang Junyu˄S3˅man

1995.5 101 1995.11 92 1995.3 101

50/53 55/55 48/58

This is a case study. Three participants come from intermediate-grade class of an urban elementary school in Zhejiang province. The criteria for identifying students as writing disabilities are [4]: (a) teacher’s recommendation in their class according to reading and

556

H. Lailin / Effects of the Voice Recognition on the Writing of Students with Learning Disabilities

writing; (b) normal or above-normal intelligence; and (c) discrepancy of two or more grade levels between expected and actual academic performance in the school. The basic information of three participants were showed in table 1. 2.2 Material and Tools 2.2.1 Voice Recognition Software IBM ViaVoice9.1 for Windows software was selected for this study because it was the most affordable among large vocabulary systems available at the time. 2.2.2 The Composition Quality A holistic analytic scale was based on analytic scoring methods described by Zhu Zuoren et al [5]. The scoring categories of the scale included: selected material, ideas, details and brief, beginning and ending, arrangement of ideas and paragraphs, transition and link up. 2.3. Procedures 2.3.1 Preparatory The preparatory included the design of the project, the teachers training, an interview with student and the Chinese language teacher. The project design included: (a) the strategy of the dictation by voice recognition with plan; (b) the researcher to meet each student twice a week for 6 weeks for 40 minutes each time. (c) the instruction of one teacher to one student ; (d) three insulating rooms provided by the primary school for this study. Teachers training included: (a) the installing and the using of the ViaVoice Chinese voice recognition software; (b) how to write descriptive essays; (c) to be familiar with the study plan and how to record the study log. We know three student's characteristics and their writing condition through an interview. 2.3.2 Treatment The whole study included six segments: (a) to use the basic function of the ViaVoice software; (b) to train for the improving system identifying rate; (c) to use the correction procedure (d) to write composition plan; (e) oral rehearsal; and (f) to compose by dictation. Different focus was put in different segment. 2.3.3 Data Collections The records included (a) MP3 recording of the dictation; (b) the time spent each step; (c) students' oral rehearsal before dictation; (d) lists of key words for his writing; (e) the draft plan that student dictated; (f) the interest of student; and (g) storage of students’

H. Lailin / Effects of the Voice Recognition on the Writing of Students with Learning Disabilities

recognition and correction text. 2.3.4 Scoring Procedures Holistic ratings were derived from quick, impressionistic judgments about a writing sample, with no more than two minutes spent rating any given writing sample. 3 Results and Analysis

557

Table 2: Accuracy of Voice Recognition System

Time Word count Correct Correct Rate(%)

second

27

23

85

259

86

forth

134

118

eighth

109

128

sixth

tenth

twelfth M

SD

300

127

454

163

349

73

85

78

77

80.7

5.39

3.1 Mastery of the voice recognition system For intermediate-grade elementary students, it is easy to master the ViaVoice system. There are still two problems: slow typewrite and ignoring to switch on/off the microphone in time. 3.2 Accuracy of Chinese voice recognition system The accuracy of transcription is a key problem. About 80% of the words students dictated were transcribed in the experiment. The rate data are in table 2. 3.3 Writing Quality The study shows that composition levels of three students with writing disabilities all were improved. The composition became longer with more and longer sentences. The composition measured data shows in table 3. In the Paired-Samples T Test for writing length, a significant main effect was found˄df=2, t0.05=5.16> 4.303, p=0.036<0.05˅. Another, in the analysis for general writing quality pre-instruction and post-measurement of all writing conditions, there were significant main effects ˄df=2, t0.05=4.98> 4.303, p=0.038<0.05˅. The composition quality measured data show in table 4. 4. Discuss 4.1 The ViaVoice Identifying Rate

Table 3: Writing Measures Variable PrePostwriting writing number of words S1 157 395 S2 165 283 S3 101 318 Number of sentences S1 7 15 S2 8 14 S3 5 14

Table 4: Quality Sub-scores of Composes Variable prepostwriting writing selected material S1 2.5 3.5 S2 3.0 3.5 S3 2.0 3.25 ideas S1 3.0 3.75 S2 2.5 3.25 S3 2.0 3.0 details and brief S1 2.5 3.75 S2 2.5 3.0 S3 1.5 3.0 beginning and ending S1 3.0 3.75 S2 3.0 3.5 S3 2.0 3.25 arrangement of ideas and paragraphs S1 3.0 4.0 S2 3.0 3.75 S3 2.0 3.25 transition and link up S1 3.0 4.0 S2 2.5 3.25 S3 1.5 3.0

This study shows that ViaVoice system's identifying rate average is 80% approximately. These results are higher than foreign research.

558

H. Lailin / Effects of the Voice Recognition on the Writing of Students with Learning Disabilities

4.2 Composition Quality The results revealed that students using dictation did make improvements in several aspects of their writing. 4.2.1 Why Did Students Write MORE? ViaVoice software allows students to get their ideas down before they are forgotten because of slow handwriting speed. A computer with voice-recognition technology could recognize the student's voice and also transcribe his or her words into text on the screen. Thus, students could not only bypass the writing mechanical difficulties with pen or pencil, but also have the advantage of the exterior vision sense. 4.2.2 Why Did Students Write BETTER? The oral rehearsal taught beforehand is very important to the students with writing disabilities. It can improve ViaVoice system identifying rate by prior inputting some key words to personal dictionary of voice recognition system. In the meantime, the oral rehearsal advances the student ability of talking about focus on subject. This led to the longer composition and better quality. 5. Conclusions The result shows: (a) the primary school students control the ViaVoice without difficulties; (b) the voice recognition system responding speed can satisfy the need of dictation; (c) under the Chinese language environment, this kind of writing method can promote composition of the students with writing disabilities; can increase the length and the quality of the composition; (d) this way also improves writing interest for the students with writing disabilities. The problem of the further research: (a) how to further refine Chinese voice recognition system identifying rate; (b) whether it can lead to long-term effect for the students with writing disabilities; etc. References [1] MacArthur, C. A., R. P. Ferretti, C. M. Okolo, and A. R. Cavalier. 2001. Technology applications for

students with literacy problems: A critical review. The Elementary School Journal 101 (3): 273-301.

[2] Liu Miao. The psychology of compose. [M].BeiJing: Higher Education Press .2001.5˖p88ˈp124.

[3]Reginald J.Roberts(1999).Use of Computer Dictation by Students with Learning Disabilities.Dissertation Abstracts International, Volume: 60-09, Section: A, page: 3272.

[4] Yuehua Zhang. Technology and the Writing Skills of Students with Learning Disabilities. Journal of Research on Computing in Education; Summer2000, Vol. 32 Issue 4, p467, 12p, 1 graph [5]Zhu Zuoren.Scales of pupil’s compose. Xian:ShanXi People Press.1990.

Curriculum

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

561

Designing a Teacher Professional Knowledge Base and Its Operation Model Based on School-Based Curriculum Development Yih-Ruey Juanga, Tak-Wai Chanb, Tzu-Chien Liuc Dept. of Information Management, Jin-Wen Institute of Technology, Taiwan b Graduate Institute of Network Learning Technology, National Central University, Taiwan c Graduate Institute of Learning and Instruction, National Central University, Taiwan [email protected] a

Abstract: Teacher professional development approaches digitalization, combination with practice, and cooperative learning; emphasizes teachers’ growth in subject knowledge and practical knowledge. With the design, practice, and reflection of lesson plans, promotes the teacher’s individual emphasis on professionalism, build a knowledge base through cooperative research between teachers. However, existing teacher professional development systems of network platforms are focused on individual teachers or teacher groups. There is less chance for school curriculum development departments or educational experts to intervene. Thus, while there are many participants, the quality and quantity of knowledge bases both need to be improved. This study proposes a teacher professional development model based on school-based curriculum development, to construct a knowledge base centered on the lesson plan. At the same time, use the operation mechanism 3C-model of creation, collaboration, and communication, to increase efficiency of professional development and the quality and quantity of knowledge bases. Keywords: Teacher professional development, Knowledge Base, School-Based Curriculum Development, Lesson Study

Introduction Teacher professional development refers to the process by which teachers increase professional ability and improve teaching methods to elevate student learning efficiency through various means throughout their career [1]. Traditionally, junior high and elementary school teachers can learn the latest on education through attending various graduate schools or credit classes, or participate in colloquia, symposia, study groups, and workshops. However, these methods do not give the practical experiences of professional development so that teachers do not know how to use what they learned in teaching practice [2]. From a practical perspective, teachers should be seen as professionals, developing individuals, learners and researchers. Teacher professional development should be connected to daily teaching and counseling of students and should be combined with school-based development [3, 4]. In recent years, with a wave after another of educational reform, the school has a broad right to determine content and development of curricula. Teachers’ professional ability is inseparable from the context of curriculum development. Therefore, the teacher professional development model combined with the curriculum research or curriculum development as a basis is increasingly emphasized. The “Lesson study” is a high-profile teacher professional development model that has been practiced for many years in Japan,

562

Y.-R. Juang et al. / Designing a Teacher Professional Knowledge Base and Its Operation Model

and has recently been widely promoted in the United States, China, Sweden, and some developing countries [7]. Besides, there is a tendency towards school-based approach, which views teachers and schools holistically, focusing on schools and not the classroom as the unit for improvement. Further, cooperation between teachers is encouraged to develop school-based professional development [4]. Therefore, it is necessary to integrate professional development of individual teachers and potential advantages of school-based development. From the perspective of information technology support, even though in the past ten years there have been many cases of technological support, such as various software tools, multimedia, and internet, for teacher professional development, and these have advantages of inspiring and developing professional knowledge of teachers [10-17]. But in most cases, only a few teachers interested in the system are willing to participate in experimentation and it is impossible to be widely promoted and utilized. Thus, this study attempts to use school-based curriculum development as a basis to design a lesson plan centered knowledge base framework for teacher professional development. Also, with the performance support of information technology, construct a creative, collaborative, and communicative mechanism — the 3C-model. It appropriately links the components in the knowledge base, exhibits them in curriculum development activities, promotes the construction, refinement, selection, sharing, and dissemination of teachers’ professional knowledge base content, and finally to improve the efficacy of teacher professional development.

1. Practical knowledge is a key point of teacher professional development Many researchers of teacher professional development were very concerned about how to transform education professional theory into instructional practice. Because good educational theory or curricular design does not necessarily produce good instructional practice, practical knowledge (or practitioner knowledge) is the type of knowledge that can cause professional teacher image to stand out. Practical knowledge is linked with practice, detailed, concrete, specific, and integrated [9]. Such practical knowledge is mostly individualized, hidden in the mind of every teacher, and is rarely described or even shared. Partly, this is because the majority of teachers are in their own closed classrooms when they research and conduct instruction, and they are not used to public presentation of their teaching. Additionally, it is also because there is no appropriate form of expression that gives teachers the desire to build practical knowledge; neither is there a culture and system for knowledge-sharing to allow teachers to build practical knowledge of cooperation with other teachers and sharing instruction [6]. Therefore, Hiebert et al. [5] believes that practical knowledge of teachers must be public, storable, sharable, and needs mechanisms for review and improvement. The “lesson study,” or “research lesson,” which has been practiced for many years in Japan, is representative of such teacher professional development models. Its results can be known from the great achievements made by Japanese students in mathematics and sciences. Stigler and Hiebert conducted comparative studies on the teacher instruction and professional development of German, Japanese, and American teachers in The Teaching Gap [7]. The good results of Japanese lesson study attracted attention of American junior highs and elementary schools [8]. The main feature of lesson study is achievement of professional growth through the process of teachers’ planning, implementation, and review of lessons, whereby they research teaching materials and methods to improve professional knowledge and subject knowledge. In other words, the teacher blends the instruction topics that he wishes to study into everyday classroom practices [3, 9].

Y.-R. Juang et al. / Designing a Teacher Professional Knowledge Base and Its Operation Model

563

2. Using information technology to support teacher professional development Many researchers have used information technology to develop various tools or systems that support teacher professional development, to establish a knowledge base which includes multimedia, discussion system, community of practice, lesson planning tool, curriculum development tool, collaborative workspace, and so on. (1) Casebook of Project Practices (CaPPs) [10, 11] is a multimedia application that supports project-based science (PBS). It provides a compendium that allows teachers to design science inquiry programs based on project and question-oriented. The inquiry projects allow students to explore scientific questions while taking the steps specified by the teacher. (2) Student Learning Environment (SLE) [12] is similar to CaPPs, and is a case-based method for teacher learning. Lampert and Ball designed mathematical teaching cases for two different classes and allow the teacher to explore how to apply new technological tools in teaching mathematics. (3) PIViT (Project Integration Visualization Tool) [11, 13] is a productivity tool that helps teachers in the designing of science projects through linear planning procedure. (4) Harvard Graduate School of Education established online teacher professional development websites, Education with New Technologies (ENT) and Active Learning Practices for Schools (ALPS) [14]. Of which, there is an online curriculum development design tool (CCDT) that allows teachers to edit courses based on the processes of Teaching for Understanding (TfU). (5) Inquiry-based Learning Forum (ILF) is a community of practice established by Barab et al. [15] for teachers in using inquiry-based learning in science and mathematics. It provides a discussion system that allows teachers to conduct experience-sharing for issues related to inquiry-based learning. The ILF team can help film teachers’ instruction and exhibit the video clips online with their lesson plans, student works, standards, and resources for reference and critique of other teachers. (6) One of the authors of The Teaching Gap [7], also the founder of LessonLab, James Stigler promoted the use of videotaped lessons as a digital library for teacher professional development. Learning from each other through teaching recording online is more effective than platforms that only provide sharing of lesson plans [16, 17]. Currently on the LessonLab website is a platform for teachers’ professional growth — Visibility PlatformTM. However, the above systems do not meet the actual needs of teachers who not only need such professional development platforms, but even more so to have the support from different levels of education professionals (such as school or grade curriculum development committee) and educational experts. The usages of these systems are not as attractive as expected, the quantity of knowledge bases grow slowly. In some cases, the enthusiasm in discussion systems is also fading, less and less shared knowledge, or sharing even ends with silence. Therefore, we believe that the construction, based on school-based curriculum development, of teacher professional development knowledge base and its operation mechanism is a chance to improve on the disadvantage.

3. Designing teacher professional development knowledge base based on school-based curriculum development Lesson plans can be used as a center of knowledge base for teacher professional development to consider related knowledge content, because in the process of lesson planning, teachers inevitably consider what kind of material to use, which method of instruction to use, which teaching tools to use, which learning resources to provide, and how to evaluate student learning, and other related questions. The teaching entities involved in these questions may include: design of teaching activities (including driving questions and teaching strategies), teaching materials (including learning content, worksheets, and test

564

Y.-R. Juang et al. / Designing a Teacher Professional Knowledge Base and Its Operation Model

questions), student e-Learning projects (or online learning plans, including learning records and learning results), research lessons (including discussion content and reports), and curriculum evaluation (including suggestions from peer assessment and school evaluations). Besides, teachers can upload teaching video clips and observation notes of colleagues to serve as a medium for discussion and reflection after teaching, and then could form a component of the knowledge base as well. Thus, these teaching entities can be centered on the lesson plan and organized into the knowledge base, as shown in Figure 1.

Figure 1. Lesson plan-centered knowledge base framework To incorporate the support from different levels into the knowledge base framework, the diagram showed in Figure 1 is divided into three levels. The curriculum framework belongs to the school level, lesson plan templates belong to the grade level, and lesson plan belongs to the class level. These three layers of knowledge base have a generalization relationship. That is, the curriculum development committee at the school level create the curriculum framework, curriculum development teams at each grade level designs lesson plan template according to the curriculum framework, then teachers of class level use these templates to design complete lesson plans suited to the needs of their class.

4. Three mechanisms to assist in teacher professional development – the 3C-model The proposed knowledge base framework requires an operation procedure, through which teachers can connect the related knowledge components and allow teachers to capture, produce, and utilize knowledge before they are able to share it. In the process of school-based curriculum development, all three levels are involved in the stages of analysis, design, and evaluation. Participants in each level must collaborate, communicate and coordinate to develop curriculum that is consistent from the top down. Therefore, the three support mechanisms of Creation, Collaboration, and Communication, the 3C-model, are the necessary technological functions that support three levels (see Figure 2).

Figure 2. 3C-model

Figure 3. ADE creation mechanism

Y.-R. Juang et al. / Designing a Teacher Professional Knowledge Base and Its Operation Model

565

4.1 Creation mechanism In the 3C-model, creation mechanism form a cycle with the three stages of curriculum development, analysis, design, and evaluation (the ADE mechanism, see Figure 3), and helps in the frequent revisions in creating components of knowledge base. In the analysis stage, it provides functions for different levels of members, such as query and statistics, statistical analysis of evaluation results, and teaching notes search. The query and statistical functions help the user to conduct statistical analysis of specific information in complete and implemented lesson plans, from the three dimensions of school years, subjects, and grades. For example, the usage ratios of each ability indictor, the distribution ratio of each learning field, instruction hours, etc. These functions can help the user to initially understand past course situations and use them to clarify the continuity and sequence of courses. The functions to search evaluation results and teaching notes can be used to help teachers check and analyze the evaluation results of course designers on curriculum framework, curriculum plan blueprints, or lesson plan implementation. In the design stage, creation mechanism focuses on the special needs of different levels in providing tools of design and management to ensure the integration and continuity of courses according to the concept of work procedures. At the school level, this mechanism mainly helps the curriculum development committee to design curriculum frameworks. Curriculum framework refers to the course planning of the entire school, including the interdisciplinary course topics, overall course objectives, and time allocation of curriculum implementation. At the grade level, this mechanism mainly helps the grade representatives and subject representatives to design lesson plan templates. Lesson plan template refers to the simple lesson plan designed by grade level members in reference to curriculum framework. The content includes instructional standards, knowledge map, subjects involved, setup of ability indicators, etc. At the class level, this mechanism helps teachers to design lesson plans where teachers can cite the lesson plan templates with the help of design tools, design a lesson plan suited to the class. In the evaluation stage, creation mechanism provides different rubric tables to help designers self-review curriculum frameworks, lesson plan templates, and lesson plans. Teachers who are not the designers can also use the rubric tables to evaluate other teachers’ lesson plans. Additionally, the query and statistical tool, as provided in analysis stage, can help observe quantitative information of course design on different dimensions such as year, grade, type of lesson plan, and statistic item. 4.2 Collaboration mechanism Collaboration mechanism connects the creation mechanism processes and products of each level, which forms an intimate curriculum development procedure. It not only provides a shared work space, but also connects the collaborative space of curriculum development participants on each level (see Figure 4).

566

Y.-R. Juang et al. / Designing a Teacher Professional Knowledge Base and Its Operation Model

Figure 4. Collaboration mechanism An individual teacher could be the member of a school committee of curriculum development, a grade committee of curriculum development, or an individual teacher at class level. Even though there are different missions for each level, the members of these three levels must work together to complete work for the entire curriculum development. Therefore, work space must be shared and curriculum development flow must be connected. In Figure 4, each level has its own curriculum development internal cycle mechanism (ADE creation mechanism) and output. For example, the output at the school level is the curriculum framework, the product of grade level is lesson plan templates, and the product of class level is the lesson plan. Grade levels can cite or refer to the school level curriculum framework to design lesson plan templates, class levels can cite or refer to the lesson plan templates of grade levels to design lesson plans. Members of each level have the authority to publicly access the output of each level. They can give ratings, opinions, or questions in response to output content. Therefore, teachers can use this mechanism to create collaboratively, share, and cite lesson plans. Also, collaboration mechanism has the function of setting groups, so that teachers interested in working together on lesson study can be set to the same group, and then teachers within the group can collaboratively design, revise, implement, and evaluate the same lesson plan. External cycle mechanism effectively combines internal cycle mechanism (ADE creation mechanism) and implementation to form a reversible top-down procedure. In the process of creation, if there is any need for revision at any level, the procedure can undergo cycles of revision. During curriculum implementation, teachers or observing teachers can record instruction process in the “teaching notes,” which is a tool included by each lesson plan that teachers can use to record class situations or to post questions. If there are video recordings of the class, they can also be uploaded to be included in the lesson plan. Also, each lesson plan has its own online discussion tool that allows collaborating teachers and observing teachers to discuss with each other the issues related to the instruction. This is an important process of lesson study, because a teacher usually cannot observe the performance of each student. Through observation notes of the instruction and student responses, it can provide detailed information for future revision. The summative evaluations of each level are combined to form a bottom-up direction. It is based on the curriculum requirement of different levels to evaluate lesson plan and its related productions. For example, in school level, the emphasis is on whether the lesson

Y.-R. Juang et al. / Designing a Teacher Professional Knowledge Base and Its Operation Model

567

plans conforms to the curriculum framework with the properties of continuity, sequence, and adaptability. In grade level, the emphasis is on whether the lesson plans in the same grade have the properties of integrity and adaptability. The emphasis in class level is on whether lesson plans have adaptability for students in the class. All evaluation contents and revision opinions are recorded in detail as a reference of future revisions for each level. 4.3 Communication mechanism Communication mechanism is an important mechanism that helps school faculty to communicate information and curriculum through group discussion system. It can improve the time consuming meetings and discussions, elevate efficiency of exchanging information and data, and further develop various kinds of learning communities. An individual teacher can use this mechanism to apply or join special interest communities in class level, or participate in regular communities in grade levels such as grade committee and subject committees, or participate in Committee of School Curriculum Development (see Figure 5).

Figure 5. Communication mechanism This mechanism provides a free discussion space for each lesson plan. When one or a group of teachers collaboratively design a lesson plan, other interested teachers can use this specialized discussion space to exchange information about lesson planning, implementation, and evaluation. Such learning communities are called special interest communities. Regular communities are communities established according to school curriculum development strategies. Generally a school has two different types of regular communities, the school committee of curriculum development and the grade or subject committee of curriculum development for each grade. Communication mechanism provides regular communities with the discussion group tool and email. Although these tools are common, the system can provide different related information as hints and references according to different communities, when users post information.

5. Conclusion Teacher professional development is a focus issue imperative in education reform of various countries. The way has been acted from traditional colloquia, workshops, university advanced studies, to digitized, combination with practice, and collaborative learning.

568

Y.-R. Juang et al. / Designing a Teacher Professional Knowledge Base and Its Operation Model

Teachers not only acquire the subject knowledge, but also practical knowledge through various systems and tools. However, the majority of them emphasize individual or groups of teachers, with fewer opportunities for school curriculum development departments or education experts to intervene. This study proposed a teacher professional development model based on school-based curriculum development, in which the teacher can be lead by of curriculum development committee members of various levels or education experts. A lesson plan-centered knowledge base has been established based on the activities of curriculum development. By using the information technology as supporting tools, the operation model 3C-model was presented to provide three mechanisms, the creation, collaboration, and communication. Based on the above concepts, we have constructed a website for teacher professional development (http://eduplans.educities.edu.tw/). Currently, the system has almost sixty thousand members and over thirteen thousand lesson plans, of which nineteen hundred have been published. Detailed system efficacy would still require an experimental study for verification.

References [1] Fullan, M. G. & Hargreaves, A., (1992). Teacher Development and Educational Change. London: The Falmer Press. [2] Fullan, M. G. & Stiegelbauer, S. (1991). The new meaning of educational change (2nd ed.). New York: Teachers College Press. [3] Lewis, C. and Tsuchida, I. (1998). A lesson is like a swiftly flowing river: How research lessons improve Japanese education. American Educator, Winter, 12-17, 50-52. [4] Holly, M. L. H. (1989). Teacher professional development: Perceptions and practices in the USA and England. In M.L. Holley & C.S. Mcloughlin (Eds.), Perspective on the teacher professional development. New York: The Falmer Press. [5] Hiebert, J., Gallimore, R., & Stigler, J. W. (2002). A knowledge base for the teaching profession: what would it look like and how can we get one? Education Researcher, 31(5), 3-15. [6] Hargreaves, D. H. (1999). The knowledge-creating school, British Journal of Educational Studies, 47(2), 122-144. [7] Stigler, J., & Hiebert, J., (1999). The teaching gap. New York: The Free Press. [8] Fernandez, C. & Chokshi, S. (2002). A practical guide to translating lesson study for a U.S. setting. Phi Delta Kappan, 84(2), 128-134. [9] Fernandez, C. (2002). Learning from Japanese approaches to professional development: The case of lesson study. Journal of Teacher Education, 53(5), 390-405. [10] Blumenfeld, P. C., Krajcik, J. S., Marx, R. W., & Soloway, E. (1994). Lessons Learned: How collaboration helped middle grade science teachers learn project-based instruction. Elementary School Journal, 94, 539-551. [11] Marx, R. W., Blumenfeld, P. C., Krajcik, J. S., & Soloway, E. (1998). New technologies for teacher professional development. Teaching and Teacher Education, 14(1), 33–52. [12] Lampert, M., & Ball, D. L. (1998). Teaching, multimedia, and mathematics: Investigations of real practice. New York: Teachers College Press. [13] Borko, H. & Putnam, R. T. (1995). Expanding a teacher’s knowledge base: A cognitive psychological perspective on professional development. In T. R. Guskey & M. Huberman (Eds.) Professional development in education: New paradigms and practices. New York: Teachers College Press. [14] Wiske, M. S., Sick M., & Wirsig, S. (2001). New technologies to support teaching for understanding. International Journal of Educational Research, 35, 483-501. [15] Barab, S., Makinster, J. G., Moore, J. A., Cunningham, D. J., & The ILF Design Team. (2001). Designing and Building an On-line Community: The Struggle to Support Sociability in the Inquiry Learning Forum. Educational Technology Research & Development, 49(4), 71-96. [16] Hiebert, J. & Stigler J. W. (2000). A proposal for improving classroom teaching: lessons from the TIMSS video study. The Elementary School Journal, 101(1), 3-20. [17] Willis, S. (2002). Creating a knowledge base for teaching: a conversation with James Stigler. Educational Leadership, 59(6), 6-11.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

569

Dynamic Composition of Curriculum for Personalized E-Learning1 Yanyan Li, Ronghuai Huang Knowledge Science & Engineering Institute, Beijing Normal University, 100875, Beijing, China [email protected] Abstract: This paper proposes an approach to dynamically compose an adaptive curriculum for e-learning based on a topic-centered resource space. By exploiting the semantic relationships that characterize learning objects (LOs) and learner profile, the approach dynamically selects, sequences, and links learning resources into a coherent, individualized curriculum to address learners’ focused learning needs. Semantic Web (e.g., XML, RDF/s) is used to represent LOs and their relationships, as well as, to support flexible navigation at semantic level. The system has been deployed and evaluated within an academic setting, enabling learners to organize, access, and share the learning resources in a more effective and personalized manner. Keywords: Learning Object, dynamic composition, Knowledge Map, adaptive curriculum

1. Introduction Nowadays, people are often motivated to educate themselves on new topics, but do not necessarily have time to take a full course of instruction. Actually, they often resort to the Web to search and gather enough information about a topic to be able to complete a task or carry on a discussion. However, the rapid growth of information available on the Web has led to people constantly fighting information overload in their pursuit of knowledge. Furthermore, as the information on the web is not effectively organized, the acquired knowledge is often disordered, disconnected, and not effectively integrated to address people’s learning needs. Especially, for people new to a subject, they feel very difficult to find and organize relevant information for effective learning. E-Learning is just-in-time education integrated with high velocity value chains. It is the delivery of individualized, comprehensive, dynamic learning content in real time, aiding the development of communities of knowledge, linking learners and practitioners with experts [5]. But the production of learning content for e-learning is demanding and expensive. It is therefore a necessity to reuse e-learning materials for producing Web-based courses with less time and efforts. Unfortunately, existing electronic courses are seldom reused, as there is usually always a need to change some part for a new course to be held. The notion of learning objects (LOs) has been introduced in the e-learning field to enhance the portability, reusability, and interoperability of learning resources. Learning objects aim to provide self-contained learning materials that once developed can subsequently be exchanged, composed, and reused. Actually, a single learning object of small granularity, in most cases, will not work to achieve a certain learning goal, so several related learning objects are needed to be assembled as a learning curriculum for this purpose. 1

Research work of this paper was supported by National Science Foundation (60402016) Corresponding author: Ronghuai Huang, [email protected]

570

Y. Li and R. Huang / Dynamic Composition of Curriculum for Personalized E-Learning

Ideally, Web-based course developers could quickly and easily assemble learning objects into coherent and effective organization for instruction to address a learner’s focused learning needs [9]. However, current Web-based courses are still developed largely manually. Additionally, they are often designed for general users and thus not adapt to individual learners. Therefore, a more automatic and flexible approach is needed that is sensitive to each learner’s unique needs and context, providing focused and structured learning resources. The literature in education holds that a cognitive structure is defined as a hypothetical construct referring to the organization of relationships among the concepts of long-term memory [8]. A schema is identified as a cognitive construct that allows people to treat multiple elements of information (concepts) as a single element, classified according to the manner it will be used [1]. So, a schema is able to lessen the pressure on working memory and facilitates learning and understanding. By separating the concept-centered knowledge schema from the distributed learning resources, this paper presents an approach generating the adaptive e-learning curriculums to address learners’ individualized learning needs. It is centered on the dynamic selecting, sequencing, and linking of learning resources into a coherent, focused organization for instruction. This approach enables diverse learners to organize, access, and share learning resources in a more effective and personalized manner.

2. Semantic Modeling of Learning Resources The basic idea is to organize the learning resources in a concept space rather than in a page space. By explicitly defining the semantics of the resources and separating the knowledge structure from the learning materials, the distributed learning resources can be organized and reused in a more flexible and interoperable manner. The conceptual model of learning resources organization is shown in Figure 1.

Figure 1. Conceptual model of learning resources organization. Assets refer to any of the media files, such as Web pages, PDF documents, animations, and audios. They can be uniquely identified with URI, and their semantic description is embedded in the corresponding learning objects. The distributed huge amounts of learning assets are stored separately. Learning objects tagged with metadata represent any chunk of learning material regardless of its form, granularity and functionality. The metadata for a generic learning object falls into three broad categories including content, structural, and context. The content description indicates what the learning object is about. The structural description indicates the learning object’s relationship with other learning objects. The context description indicates when to present the learning object, which expresses the pedagogical

Y. Li and R. Huang / Dynamic Composition of Curriculum for Personalized E-Learning

571

information of a learning object. Learning Objects are self-contained learning components that once developed can subsequently be exchanged, retrieved and reused. Knowledge Map (KM) describes the domain-specific knowledge schema (i.e. concepts, semantic relationships), which provides a semantic view of the linking learning objects. For example, it can describe the organization of instruction either for an entire course or for a part of it (e.g. lesson, lecture). Knowledge map can either be designed by the instructor as the universal for each learner’s references or devised by the learners to express their personal knowledge and preferences. We use semantic link network to represent knowledge map [10], which includes nodes and typed semantic links between nodes, encoded as RDF entities and properties. Related assets are grouped together as a learning object, and one or more learning objects about the same subject are linking to the same concept as its instances. Users with different roles (e.g. instruction designer, content provider, and learner) can access the resources via the friendly interface, and the location and format of the learning assets are transparent to users. In this way, users can search and navigate in the resource space in a more flexible and efficient way.

3. Architecture The system architecture is illustrated in Figure 2. The user interface allows the learners to access the system with different privileges by providing role authentication. Each learner is equipped with a profile to describe his personal information and learning history, which can be manually edited and updated by learners through the system's interface at anytime, or the system keeps track of learners’ activities and update it periodically. As for a specific domain, learner profile is actually an overlay of the knowledge map in terms of the knowledge structure where each node is attached with several attribute values (e.g. review times, master level, etc.). After received a learner query, the adaptive composition engine takes charge of selecting and organizing the learning objects according to learner profile, and finally delivering the generated curriculum to the learner. On the other hand, with the support of authoring tool, instructors can design and revise the knowledge map within a specific domain. The execution engine is responsible for executing the authoring operations. The index engine fulfills the task of searching the domain concept that matches the metadata of LOs. At the same time, instructors and learners can freely insert, delete, or modify LOs. The resource space composed of knowledge space (stores the knowledge map, LOs metadata, and pedagogical rules) and information space (stores the learning assets) offers the resource support for adaptive composition engine.

Figure 2. The system architecture.

572

Y. Li and R. Huang / Dynamic Composition of Curriculum for Personalized E-Learning

4. Dynamic Composition of E-Learning Curriculum The e-learning curriculum is generated on the fly to cater for the different learning needs of the learners. The majority of users are accustomed to expressing their learning needs in terms of keywords, and thus we provide the user interface enabling the learners to express their learning needs in terms of keywords during the learning process, but at the same time uses the semantic information regarding the application domain to obtain results that are not possible in traditional information retrieval. In addition to the keywords, the learners may specify other constraints, such as difficulty level, learning time, media type, etc. In this way, learners are able to actively drive the selecting and organizing of learning materials to meet their own learning needs. For the given query proposed by the learner, the curriculum composition is fulfilled by following the five-step process, which is shown in Figure 3.

Figure 3. Process flow of the curriculum composition. Step1: Query annotation. For a given query, the first step is to automatically process the query and annotate the query with possible semantic information to expedite the search for LOs in a large repository. For the case of linguistics ambiguity (e.g. synonym), users’ profiles (e.g. personal information, background knowledge, interests) are taken as the reference to select the proper annotation. Additionally, the search context and the popularity of the term as measured by its frequency of occurrence in a text corpus can also give hints for selecting the proper annotation for the search terms. In addition, other candidate terms are also presented to the learners so that they can select one of them if that is what they intended. Step 2: LOs searching. After the query is processed, the next is to search the LO repository for relevant learning objects based on the keyword matching of the learning objects content and metadata to the query term. The search results are a set of learning objects. Step3: Topic mapping. Identify the target topics by mapping the LOs in the search results to the topics in the knowledge map. The simple way is to follow the links preset by the instructors if there exist links from learning objects to topics, otherwise, use the topic clustering to find the mapping topic based on the metadata of returning learning objects. Step4: Learning syllabus planning. Learning syllabus is represented as a sequence of semantically interrelated topics that a learner can follow to address his focused learning needs. Taken the mapping topics as anchor nodes, the learning syllabus is generated based on the graph traversal approach according to topic relationships. That is, the approach for selecting the target topics, based on the structure of the graph, is to collect the first N triples originated from the anchor topic, where N is the pre-defined traversal constraints. Actually, different learners have different backgrounds and preferences, so the property types of

Y. Li and R. Huang / Dynamic Composition of Curriculum for Personalized E-Learning

573

nodes should not be treated as equally relevant. Therefore, learner profiles are taken into account to select and sequence the necessary topics while ignoring the unfocused semantic relationships. For example, if a learner is interested in a topic, then provide him with the prerequisites that are not studied or not well comprehended. Step5: LOs sequencing. Given the personal syllabus for an individual learner, it is time to substantiate each topic of the syllabus with one or more LOs. Pedagogical rules are used to select and sequence the learning objects about the same topic based on their metadata description. Some example pedagogical rules are as follows: 1) The learning object as the concept part must precede the learning object as the elaboration part; and 2) Regarding the same interesting topic, provide the novice with simple introduction and example, while presenting more detailed and in-depth information to the advanced learner. Additionally, the description about the quality of LOs helps selecting the LOs with high quality. The metadata can be specified by providers during the authoring process, such as best, better, normal recommendation mark, or computed based on the learners’ feedbacks. To the end, serving as components of a learning curriculum, the target learning objects are sequenced and provided for the learners to address their focused and personalized learning needs. Taken a simple example to illustrate this working process, we assume that a novice proposes a query “Lattice homomorphism” but with no other constraints. After annotating the query as a mathematics term, search the the matching learning objects in the LOs repository, and then select the topic corresponding to the matching LOs as anchor node. As the learner has little knowledge on this topic, the generated curriculum should contain the basic content and necessary background knowledge. That is, by using the reasoning rules (e.g. transitivity) on the semantic relationship (e.g. SubtypeOf, Sequential) between topics, its prerequiste topics with linking LOs are all selected and sequenced to compose a learning curriculum. The generated curriculum is depicted in Figure 4. It becomes obvious that different curriculum could be generated to cater for different learners with different backgrounds, capabilities and expectations.

Figure 4. A schematic generated curriculum.

5. Implementation We have developed an e-learning platform WebCL (available at http://www.webcl.net.cn) that has been used in more than twenty universities or high schools, and the total number of registered users exceeds 10000. WebCL provides a set of tools (e.g. course tools, evaluation tools, communication tools) to support cooperative and adaptive learning, while offering a plug-in interface for extension modules.

574

Y. Li and R. Huang / Dynamic Composition of Curriculum for Personalized E-Learning

5.1 Authoring Tool for Instructors An authoring tool has been developed for defining the domain knowledge and structure. Figure 5 shows the graphic user interface for constructing the course-KM. Experts and instructors can click the icons in the top toolbar to add the concepts and relationships, while deleting or editing them in the background graphical interface. Through the dialogs for defining the concept attributes and relationship types, experts and instructors can input the text or select one item from the drop-down list to specify the concept properties and relationship types. Figure 6 gives a graphical view of a part of course-KM. As the figure illustrates, the left frame shows a tree hierarchy of the course concepts on a specific subject, the upper part of the right frame shows the constructed course-KM, and the lower part of the right frame shows the list of properties corresponding to the concepts shown in the upper part.

Figure 5. User interface for constructing course-KM.

Figure 6. Graphical view of a part of course-KM.

Y. Li and R. Huang / Dynamic Composition of Curriculum for Personalized E-Learning

575

5.2 Adaptive Curriculum for Learners The adaptive curriculum application is built on top of the WebCL Workbench that offers a plug-in interface for extension modules. This module is implemented in Java and runs on web application servers compatible with the Servlet 2.0 specification. Web pages are created using JSPs and Learning Object content in XML is transformed to XHTML sing the XSLT style sheet processor. Figure 7 shows the resulting curriculum for the query “Lattice homomorphism” proposed by a novice. As the learner has little knowledge on this topic, the generated curriculum contains the basic and comprehensive content. The componential learning objects are displayed in a sequence with hypertext navigation structure. Each learning object is described with semantic information to guide user’s navigation, such as the pedagogical role (e.g. “concept”, “example”), title, introduction, instructional objective, etc. To the right of each LO title is the learning time of the learning object. More related topics are given at the bottom of the page for learners’ reference, and thus learners can click any topic to get more focused information. Additionally, learners can click the “Advanced Search” to specify more constraints and preferences in addition to the keywords.

Figure 7. The interface for displaying e-learning curriculum.

6. Discussion and Comparison Learning Content Management System (LCMS) focuses on the construction and maintenance of LO repositories for various e-learning applications, examples are MERLOT(http://www.merlot.org/), CAREO(http://www.careo.org), and SMETE(http://www.smete.org/smete/). As [7] pointed out, although they adopt standard e-learning metadata specifications to describe LOs, most of them use full text queries as the only way to access LOs in a disconnected way from the actual learners’ navigation. This functionality is quite different from that of our approach employing the relationship between LOs to select and organize semantic relevant LOs. Considerable work has been conducted on adaptive hypermedia system [2][4], which are considered to be relevant with our approach. But much of the work is to assume that web resources are already richly linked and the job of the adaptive system is to show or hide various links [3]. By contrast, our approach achieves higher level of flexibility and scalability by modeling LOs with associated metadata, and encoding the relationship and sequencing between metadata instead of between learning resources. Some other research groups driven by similar goals have proposed other architectures that match our vision. The CSD-Uoc portal [7] provides an online curriculum on the

576

Y. Li and R. Huang / Dynamic Composition of Curriculum for Personalized E-Learning

Semantic Web by exploiting the semantic relationships established among LOs to generate a learning path. However, it is very difficult to specify the relationship between LOs in a large information space composed of a great deal of LOs. In our approach, we instead construct a domain-specific knowledge map as a basis for users to flexibly specify the relationship between LOs with less time and efforts. [6] presents an approach to assemble learning objects into coherent and focused learning paths from a repository of XML Web resources. It simply relies on the learner’s query to select matching LOs but not consider learner’s learning history and background.

7. Conclusion By incorporating the learning objects and the Semantic Web technologies, this paper proposes an adaptive curriculum composition approach for e-learning. Driven by the specific queries from learners, adaptive e-learning curriculum is dynamically generated by selecting and structuring the semantic relevant learning objects according to knowledge schema, LOs metadata, and learner profile. In this way, the generated curriculum with tailored content and flexible structure caters for different learners with different backgrounds, capabilities and expectations, at different time and venue. We currently provide authoring tools enabling the users to semi-automatically define and annotate knowledge schema and its instance resources with visual operations. Future work will extend this to automatically annotating the relationships between discovered topics or instances. Additionally, we are going to experiment on a large mount of users for widespread use during a long period to test the effectiveness of our proposed approach. References [1] Bagui, S., Reasons for Increased Learning Using Multimedia. Journal of Educational Mutlimedia and Hypermedia 7(1) (1998), 3-18. [2] Conlan, O. et al, An Architecture for Integrating Adaptive Hypermedia Curriculums with Open Learning Environments, in Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications, June, 2002, 344-350. [3] De Bra, P. et al, AHA! The Adaptive Hypermedia Architecture, in Proceedings of the ACM Hypertext Conference, Nottingham, UK, August 2003. [4] Dolog, P. and Henze, N. Personalization Services for Adaptive Educational Hypermedia, in Proceedings Of International Workshop on Adaptivity and User Modelling in Interactive Systems, Karlsruhe, Germany, October 2003. [5] Drucker, P., Need to Know: Integrating e-Learning with High Velocity Value Chains, A Delphi Group White Paper, 2000, http://www.delphigroup.com/pubs/whitepapers/20001213-e-learning-wp.pdf. [6] Farrell, R., Liburd S.D. and Thomas, J.C., Dynamic Assembly of Learning Objects, in Proceedings of The thirteenth International World Wide Web Conference, May, New York, USA, 2004. [7] Kotzinos, D. et al, Online Curriculum on the Semantic Web: the CSD-UoC portal for peer-to-peer e-learning, in Proceedings of the fourteenth International World Wide Web Conference, May, Chiba, Japan, 2005. [8] Shavelson, R.J., Some Aspects of the Correspondence between Content Structure and Cognitive Structure in Physics Instruction. Journal of Educational Psychology, 63(3) (1972), 225-234. [9] Wiley, D.A. (ed.), The Instructional Use of Learning Objects. Agency for Instructional Technology, Bloomington, 2002. [10] Zhuge, H. and Li, Y., Learning with Active E-Course in Knowledge Grid Environment, Concurrency and Computation: Practice and Experience, 2005.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

577

Development of a Photo Management System in Schools Which Ensures Students Appear Equally Kyoko UMEDA, Shinsuke TAKITO, Tetsuro EJIMA, Hironari NOZAKI Department of Education, Aichi University of Education, Japan [email protected] Abstract: Japanese elementary and secondary school teachers create class newspapers that include photos of student activities. Japanese teachers have to treat students equally in the class newspaper. Therefore, the teacher usually counts the number of times a student appears in photos. This method does not take into consideration “presence,” the impact of the face in a photo. We developed a photo management system for class newspapers which consider the “equality” of student’s appearance of newspaper using objective “presence” metadata that we proposed. The results of the evaluation show that this system can support teachers in selecting photos, or can be used in other institutions where there is a need to manage photos. Keywords: photo management system, presence, equally, regression formula, class newspaper

1. Introduction Recently, people can take a lot of photos and save them in a PC easily. However, it is sometimes difficult for people to find the exact photos they want to use. Therefore, a photo management system is needed [1]. This is the situation that schools in Japan find themselves in: teachers can take photos of students’ activities easily, and they are often used in a webpage or class newspaper [2]. Therefore, teachers need a special photo management system that is developed for school use in order to fairly represent students. Many internationally standardized frameworks which use metadata about photos exist. Exif format by JEITA describes the metadata of a digital photo, and gives the parameters at the time of photography, position information, etc [3]. MPEG-7 has specified the framework for describing the contents of the multimedia information about images, sounds or videos [4], and some work has been done on including automatic face recognition in photo management systems [5]. Since both standards are made for flexible use, the description about the contents, like “what the photographic subject is doing,” should be written by a person, and so if two people describe a photo, there may be variation [6]. Moreover, there are approaches which express the metadata for combining existing standards. For example, there are metadata for describing and retrieving photos with RDF and HTTP such as W3C Note [7], for sharing photos with Dublin Core, FOAF, and RSS1.0 [8]. Various existing tools can use these metadata since they have flexibility and high extendibility; however, it is necessary to add or extend the metadata required from the original domain in order to adapt to the new domain. Namely, when developing a metadata type management system, the methods with flexible and extensible standards are used, and adding some metadata which is peculiar to the domain or the purpose is generally required. In this research, we are targeting students’ photos in Japanese schools. Japanese

578

K. Umeda et al. / Development of a Photo Management System in Schools

elementary and secondary school teachers create class newspapers, albums, or webpages that includes photos of student activities. In order to treat students equally, the teacher usually counts the number of times a student appears in the selected photos [9]. However, this does not take into account the fact that a student’s face in a photo of one person has much more impact than the face of a student in a group photo. We call this factor “presence.” This presence does not show up in a simple count of the number of times a student appears in the photos. Therefore, we created metadata which can measure the subjective quality of presence numerically [10]. The purpose of this paper is to describe the development of a photo management system which has built into it presence metadata, and which can support teachers in selecting photos that portray the students equally. In chapter 2, we briefly explain about the presence metadata which we created; in chapter 3 we describe the development of a photo management system; and we evaluate the system in chapter 4.

2. About “Presence” Metadata We define “presence” as the degree of impact of a specific person in a photo, and express presence using the following five levels, going from low presence to high presence. 1. It can be verified only if you observe the photo carefully. 2. It can be understood to exist but you do not feel the strength of its existence. 3. It can give the impression of strength of existence, but it can not say that it is especially conspicuous. 4. It is conspicuous and the eye tends to go there. 5. It is especially conspicuous and it can be said to be the main character of the photo. We tried to specify this subjective presence with objective items, and as a result, a multiple regression formula of presence was made. The procedure is shown as follows. At first, four objective items (and categories) were determined from advanced researches [10-13] as reference: size of photo (640 u 480 pixel, 320 u 240 pixel, 160 u 120 pixel), number of people (individual:1-2, group1:3-6, group2:7-10, party:11 or more), area of photo occupied by the face (big: the face size which surrounded by the rectangle is more than 1/24 of the area of the photo, small: less than 1/24), location of face on the photo (center: divided photo vertical 4 divisions and width 2 divisions, and if more than half of the face is included in the bottom two middle divisions, periphery) (see Fig 1). We conducted an experiment with 97 university students in photos which suited 48 patterns of this category combination (3 u 4 u 2 u 2) was specified, and asked students to judge the presence of people in the photo using the above five levels. A multiple regression analysis (quantification method Σ ) by step-wise selection which uses “presence” as the purpose variable and 11 categories as the prediction variable ex.) size of photo:320*160 pixel number of people:group1

(a) area of face:small location:periphery

(d) area of face:small location:periphery (b) (c) area of face:small location:center

Fig. 1. Example of the “presence” of photographic subjects.

K. Umeda et al. / Development of a Photo Management System in Schools

579

Table 1. Result of the regression analysis 㫀㫋㪼㫄㫏㪈㪈㫏㪈㪉㫏㪈㪊㫏㪉㪈㫅㫌㫄㪹㪼㫉㩷㫆㪽㩷㫇㪼㫆㫇㫃㪼㫏㪉㪉㫏㪉㫀㫅㩷㫇㪿㫆㫋㫆㫏㪉㪊㫏㪉㪋㪸㫉㪼㪸㩷㫆㪽㩷㫇㪿㫆㫋㫆㫏㪊㪈㫏㪊㫆㪺㪺㫌㫇㫀㪼㪻㩷㪹㫐㩷㫋㪿㪼㫏㪊㪉㪽㪸㪺㪼㫏㪋㪈㫏㪋㫃㫆㪺㪸㫋㫀㫆㫅㩷㫆㪽㩷㪽㪸㪺㪼㫏㪋㪉㪺㫆㫅㫊㫋㪸㫅㫋㫏㪈㫊㫀㫑㪼㩷㫆㪽㩷㫇㪿㫆㫋㫆

㪺㪸㫋㪼㪾㫆㫉㫐㪍㪋㪇㬍㪋㪏㪇㪊㪉㪇㬍㪉㪋㪇㪈㪍㪇㬍㪈㪉㪇㫀㫅㪻㫀㫍㫀㪻㫌㪸㫃㪾㫉㫆㫌㫇㪈㪾㫉㫆㫌㫇㪉㫇㪸㫉㫋㫐㪹㫀㪾

㫇㪸㫉㫋㫀㪸㫃㫉㪼㪾㫉㪼㫊㫊㫀㫆㫅㪺㫆㪼㪽㪽㫀㪺㫀㪼㫅㫋㪇㪅㪇㪇㪇㪄㪇㪅㪇㪐㪉㪄㪇㪅㪋㪏㪎㪈㪅㪈㪇㪐㪇㪅㪇㪇㪇㪄㪇㪅㪉㪏㪇㪄㪇㪅㪏㪐㪍㪇㪅㪋㪈㪇

㫊㫄㪸㫃㫃

㪇㪅㪇㪇㪇

㪺㪼㫅㫋㪼㫉㫇㪼㫉㫀㫇㪿㪼㫅㫐

㪇㪅㪋㪋㪊㪇㪅㪇㪇㪇㪉㪅㪏㪐㪋

was conducted. As a result, the 7 variables were selected in table 1 (p<.01). Namely, a multiple regression formula of presence was created: y 0.092 u x12 0.487 u x13 1.109 u x 21 0.28 u x 23 0.896 u x24 0.41u x31 0.443u x41 (adjusted R square=0.461, residual SD=0.879)

Hereafter, the predicted value acquired by this formula is called a “presence point.” To use this presence point for metadata of photo, each photographic subject has a name and a presence point; conversely, one photo has names and presence points for a number of photographic subjects. (For Fig.1, the name of the person is (a) with a presence point of 1.33, names (b) and (c) have 1.78 presence points, and name (d) has 1.33 presence points.) Moreover, in order to examine the validity of the formula, the same experiment was conducted with another group of 57 university students using another 120 photos. Correlation between the observed values which were obtained in this experiment and predicted values from the presence formula was significant (p<.01), correlation coefficient r=0.884. Therefore, it can be said that this multiple regression formula suggests validity.

3. Development of a Management System for Creating a Class Newspaper We developed a photo management system which uses the presence point formula to choose students’ photos used for a class newspaper in a Japanese school. 3.1 Concept of the System For a teacher making a class newspaper, we decided on the system specification that the teacher might use: ࡮ A teacher searches and selects some photos for use in a newspaper and puts them into a “temporary box”. ࡮ After searching the photos, a teacher selects photos from the temporary box for use in the current newspaper. The system alerts the teacher if there are some students who have too many or too few presence points, and shows alternative photos of the student if any. Which photos are actually used is left to the teacher’s judgment. However, since it is too difficult to end up with an exact equal amount sum of presence points for all students, an “equal range” is specified: equal range: average of total presence points of the member of a class r residual SD(0.879) u number of photos

580

K. Umeda et al. / Development of a Photo Management System in Schools

3.2 Features of the System Based on the above specification, we developed a system which uses PHP as the development language, Flash developed by Macromedia/Adobe as a part of the interface, and MySQL as a database. The developed system consists of functions of registration, input of the metadata, and retrieval like other photo management systems. In this section, we describe the feature functions of this photo management system. (1) Management the photos as groups In this system, photos are managed for groups for each activity or each class work, such as an excursion, sports activity, entrance ceremony, etc. Those photos are input into one group, like a concept, as a folder in the PC. Date of the registration of photos, photo resolution, and size are automatically registered in the database. When the user enters keywords for a certain group, those keywords are given as metadata to all photos that belongs to the group. In addition, the user can also give metadata for each photo. (2) Input of presence metadata to photo To input the presence metadata easily, we developed an interface whereby the user drags a subject’s face from the photo using Flash technology. A teacher drags the student’s face on the photo and selects a student name from the name list of class, and repeats this act by the photo subject’s number. By dragging a face, two factors (area of photo occupied by the face and location of face) which constitute presence metadata are attained at once. Also, the number of people and size of photo are added automatically. (Fig.2) (3) Searches a photo by using presence In this system, a search using presence level is possible in addition to the usual retrieval by keyword. (4) Name list of a class which shows presence points. Also, the system generates a name list of the class which indicates the number of previous appearances of the student and the sum of presence points of that student used for the newspaper until now to aid the teacher’s search and selection of photos. (5) Temporary box This system has a temporary box. A teacher searches and selects some photos for use in a

2. Select a name from the list. 1. Drag a face on photo. 3. Check the inputting of the data.

Fig. 2. Window showing inputting of the presence metadata.

581

K. Umeda et al. / Development of a Photo Management System in Schools

newspaper and puts them into a temporary box to work on then later. (Fig. 3) (6) Shows candidate’s alternative photos As mentioned above, in the case there are some students who have too many or too few presence points, the system alerts the teacher, and show the alternative photos of the student if any. An algorithm chooses photos belonging to the same group and includes a student who has few presence points. Also a photo is selected with the fewer number of people in it, because the lower number of people affects other students’ presence point less, and a photo which has few people has more presence points. (Fig. 3) The result of retrieval photos. System shows the candidate’s alternative photos if any.

If you check the photo, the photo moves to a temporary box.

System tells user about photos which have too many presence points.

Fig. 3. The result window of retrieval and temporary box (left), and system alert of too many presence points (right).

4. Evaluation of the System

We asked teachers to evaluate the system and evaluated a piloting of the system. 4.1 Teacher Evaluation of the Usability and Necessity of the System We asked six current Japanese school teachers about the usability of input using metadata, searching the photos, and the necessity of the system. The results for usability in table 2 show that overall opinions about usability were good. However, about the showing of presence point, two teachers answered that the display was difficult to understand because the system just shows the presence points using on a graph, so we learned that it is necessary to explain how to interpret the graph. About the necessity of the system, all teachers answered it is necessary to think about the equal treatment of students and the system was convenient for this purpose. On the other hand, there are still difficult steps to use the system, especially to give a photo metadata, and there are probably some teachers who think it is troublesome. The time and effort to attach Table 2. Results of Questionnaire. Questions about the usability about inputting the metadata Is it easy to input a keywords to each photo? Is it easy to input metadata to the photos? Is it easy to input “presence metadata” by dragging the face of the photo?

Yes No 6 5

0 1

6

0

Questions about the usability about searching the photos Is it easy to search for a photo? Is it easy to select a photo? Is it easy to understand the presence point?

Yes No 6 6 4

0 0 2

582

K. Umeda et al. / Development of a Photo Management System in Schools

the metadata cannot be eliminated completely since those tasks of adding to metadata and the convenience of retrieval have the relation of a trade-off, not only for this system but also for any metadata type picture management system. However, we think there is room to decrease these tasks, so we are trying to improve the steps.

4.2 Evaluation of a Newspaper which Used Photos Chosen by the System To evaluate the system ourselves, we conducted an experiment testing the “equality” of a number of photos which were chosen by the system.

4.2.1 Conditions about Creating a Newspaper To create class newspaper in Japan, we come up with the following conditions for the photographic subjects who appear in the newspapers: ࡮ A group is constituted. (Assumed to be a class in school.) ࡮ The person who belongs to a group appears in a photo-group at least once. (There are no cases whose a certain student does not appear in the newspapers.) To consider these conditions, we created two different newspapers: ࡮ Fair appearance newspaper: appearance of all persons belonging to a group is within “equal range” defined above in chapter 3. ࡮ Unfair appearance newspaper: there are persons whose appearances are not with in the “equal range,” although all persons belonging to a group appear in the newspaper at least one or more times.

4.2.2 Summary of the Experiment (1) Materials We assumed 16 university students belongings to the same university club as a virtual class, and we registered 40 photos which show 16 student club activities, all with resolution 320 u 160. The system selected 16 photos for the fair and 16 photos for the unfair appearance newspaper based on conditions mentioned in 4.2.1. Six of the 32 photos were selected for both groups. Using the 16 photos, we arranged four photos on one sheet of A4 size paper; that is, each newspaper consisted of 4 pages, printed using a color printer. An inoffensive description was attached under each photo. 16 students' presence points for each newspaper can be seen in table 3. Table 3. Sum of “presence points” of 16 photographic subjects. Fair appearance newspaper Unfair appearance newspaper Sum of Sum of Sum of Sum of Number of Number of Number of Number of Person presence Person presence Person presence Person presence appearances appearances appearances appearances points points points points D 5.76 2 8.49 3 6.89 2 14.58 5 6.58 2 5.32 2 H 6.58 2 19.67 6 A 6.17 2 7.8 3 7.3 2 E 14.06 5 C 7.39 3 8.11 3 7.92 3 9.65 3 6.67 2 7.28 2 G 1.91 1 7.69 2 7.67 3 B 9.08 2 F 19.84 5 7.56 2 6.67 2 5.6 2 4.84 2 2.8 1 6.89 2 6.23 2 21.26 6 1.91 1 Average 6.98 Average 9.65 3.47<

Equal range

< 10.5

6.14<

Equal range

< 13.17

K. Umeda et al. / Development of a Photo Management System in Schools

583

(2) Participants Eight of the 17 university students are contained in a virtual class (photographic subjects), and nine students who are outside of the virtual class, but who know the members of the virtual class well are included. The reason we divided the participants was that we thought “equality” might be different whether a person appeared in a photo or not. (3) Procedure At first the participants looked at the four pages of the “fair appearance newspaper.” Participants could look at them any number of times, and then answered some questions. Next, the participants looked at the four pages of the “unfair appearance newspaper.” We asked questions about the “equality level of each newspaper” and “equality level about specific persons” (A-H, in table 3), which were asked using a 4 point scale: “1 equal,” “2 sort of equal,” “3 sort of unequal,” and “4 unequal.”

4.2.3 Results and Discussion (1) Participants are included in the photographic subject or not There was no significant difference between the groups of photographic subjects (N=8) and non-class member (N=9) about all questions. So, we will just explain the results of the combined two groups (N=17). Therefore, we can say that presence point can be one of the objective standards when a teacher chooses photos. (2) Equality about newspaper Table 4 shows the average of “equality level of newspaper.” It is understandable that participants think the “fair appearance newspaper” was “sort of equal,” and the “unfair appearance newspaper” was “unequal.” Also there was significant differences for both newspapers (F(1,15)=18.91, p<.01). Moreover, table 5 shows equality among person A-D were judged “sort of equality” and there was no significant difference as shown by ANOVA (F(3,42)=1.78, n.s.). On the other hand, equality among person E-H were judged “sort of unequal” or “unequal.” Also there were significant differences among them (F(3,42)=19.8, p<.01), and the result of the multiple comparison shows, only the score for H, who was only within the “equality range,” was lower than other participants. The above results show that 16 photographed subjects appear equally in the “fair appearance newspaper” and they do not appear equally in the “unfair appearance newspaper.” Table 4. Average of “equality” 㪤㪪㪛

㪝㪸㫀㫉㩷㫇㪸㫇㪼㫉㪈㪅㪏㪉㪇㪅㪍㪋

㪬㫅㪽㪸㫀㫉㩷㪧㪸㫇㪼㫉㪋㪅㪇㪇㪇㪅㪇㪇

Table 5. Average of “equality” about persons A-H.

㪤㪪㪛

㪝㪸㫀㫉㩷㪸㫇㫇㪼㪸㫉㪸㫅㪺㪼㩷㫇㪸㫇㪼㫉㪬㫅㪽㪸㫀㫉㩷㪸㫇㫇㪼㪸㫉㪸㫅㪺㪼㩷㫇㪸㫇㪼㫉㪘㪙㪚㪛㪜㪝㪞㪟㪈㪅㪏㪉㪉㪅㪉㪐㪈㪅㪐㪋㪈㪅㪐㪋㪊㪅㪎㪈㪋㪅㪇㪇㪊㪅㪐㪋㪊㪅㪇㪇㪇㪅㪏㪈㪇㪅㪐㪉㪇㪅㪎㪌㪇㪅㪌㪍㪇㪅㪋㪎㪇㪅㪇㪇㪇㪅㪉㪋㪇㪅㪍㪈

5. Conclusion

In this research, we developed a photo management system for class newspapers, which considers the “equality” of students’ appearance of newspaper using the presence metadata we proposed. The result of the evaluation was the system usability overall was good. Also, this system could help chose students photos with equality. However, we did not indicate our opinion that a simple count of the number of times a student appears in the photos does

584

K. Umeda et al. / Development of a Photo Management System in Schools

not express adequately the students’ appearance because the greater number of appearance of photographed subjects appear has more presence points in this experiment. Therefore, in the future we will do another experiment increasing the number of photos. In the future, we think we will try to reduce the work required for attaching the metadata, which is a difficult chore especially in the case where there are many subjects in a photo, because the regression formula of presence was most influenced by “number of people.” We showed this photo management system is an effective tool to aid teachers making a class newspaper, but this system could also be used for any institution or individual wanting to make a “fair” publication.

Acknowledgments

This research was partially supported by the Ministry of Education, Science, Sports and Culture, Grant-in-Aid for Young Scientists (B), 16700558, 2006

References [1] Mills, T., et al. (2000) Shoebox: a digital photo management system, Technical Report, AT&T, 10, Laboratories Cambridge. [2] Umeda, K. et al. (2004) The development of SWMS: A System for Easily Creating Dynamic School Websites and Individualized Web Newspaper, Proc of ICCE2004, 1821-1829. [3] WATANABE, M., TSUBAKI, H. (2003) A New Standard for Digital Still Cameras “Exif Version2.2.” Research Report of Fuji Photo Film Co., 48:28-30. [4] SHIBATA, M. (2001) Standardization Activities of Content Description MPEG-7. Journal of Image Information and Television Engineers, 55(3):337-343. [5] Media and Information Research Laboratories, NEC Corp (2004) Development of MPEG7 Face Recognition description for video content retrieval. NEC TECHNICAL JOURNAL Vol.57 No.3. [6] TAKAHASHI N., et al (2002) Introducing and Example of Video Content Management Using MPEG-7. IPSJ SIG Notes (DBS), 41:1-8. [7] Lafon, Y., Bos, B. (2002) Describing and retrieving photos using RDF and HTTP, W3C Note, http://www.w3.org/TR/photo-rdf/. [8] KANZAKI, M(2004)Image data using FOAF and RSS http://www.kanzaki.com/docs/sw/image-rdf.html. [9] NAGAI, M., et al. (1980) Creating a class and a class newspaper in the lower grades. Meijitosho. [10] TAKITO.S, et al. (2005) Development and evaluation about photo management system in school. Master's thesis of Aichi University of Education. [11] NISHIYAMA, H., MATSUSHITA, Y. (1996) An Image Retrieval System Considering Image Composition. Transactions of IPSJ, 37(1):101-109. [12] TAKAHASHI, M., et al. (1999) Human memory. Cognitive Science & Information Processing7, Science. [13] TAKAHASHI, Y., et al. (1990) An Image Retrieval Method Using Inquiries on Spatial Relationships. Transactions of IPSJ, 31(11):1636-1643. [14] TOJIMA, A., HACHIMURA, K. (1999) Extraction of Compositional Features from Paintings and Application for Retrieval. Transactions of IPSJ, 40(3):912-920.

Sense-Making and Facilitation

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

587

Probing Technology as Affordances for Negotiating Meaning in the Elementary Science Classroom ---- A Participation Perspective Fei-Ching Chena, Huo-Ming Jiangb, Jie-Chi Yangc, Yu-Wei Leea a Institute of learning and Instruction,b Institute of Atmospheric Physics, c

Institute of Network Learning Technology, National Central University, Taiwan [email protected] Abstract: Probing technologies are assumed to augment and speed inquiry learning. However, in practice, handheld-based classroom practice is regarded as only having scratched the surface of what is possible. The purpose of this article is to propose regarding the probe as a catalyst for negotiating meaning in inquiry learning activities. Viewed from a participation-framework perspective, we claim that probing technologies trigger the cycle of, and interplay between, reified object generation (numbers, graphs,...) and engaged participation among learners. An analysis of a probe-supported inquiry activity in a grade 5 elementary classroom is conducted. Three episodes in an inquiry classroom reveal the focusing and reified effects of probes in enhancing meaning negotiation, and in supporting student-driven exploration. The function of probes in both emergent and instantaneous aspects of learning is also discussed. Keywords: probing technology, meaning-making, participation

1. Exploring new affordances of probes The computational power and streamlined data acquisition capabilities of probes appears to have been well established. However, the research that resulted in these findings was conducted primarily using pre-post test (Krajcik & Starr, 2001), concept mapping (Novak & Gleason, 2001), and survey (Vahey, Tatar, and Roschelle, in press), or involved establishing impressive learning gains as evidence of the benefit of probing technologies (Tinker, 2000; 2001). Of course, having students see the display change in “real time” is of great educational value, but exactly how learners are able to associate rapidly certain physical changes with changes in the representation of those phenomena remains unclear. We attempt here to reveal with empirical data the sort of meaning relationship that other methods ignore, but which might enrich their analysis through identifying the potential in these technologies for deep learning. Research on alternative affordances and constraints of probes is promising. Vahey, et.al. (In press) propose one particular set of affordances: the ability of students to engage in, and move seamlessly between, private interaction with their computational environment, and public interaction with face-to-face collaboration around the computational environment. Roschelle (2003) also pointed out the challenge of creating a link between the informatic world and the social world, a coupling through which these

588

F.-C. Chen et al. / Probing Technology as Affordances for Negotiating Meaning

powerful probes enable and motivate social interaction. Sharples (2002) also mentioned the importance of learning conversations in externalizing the understanding of the learners. The design of mobile technology for learning must take the affordances of conversation into consideration. Alongside this line of thinking, this study examines the affordances of meaning-making provided by the probes. We introduce the social theory of learning as an analytical framework to perceive probes as catalysts for meaning-making among students and teacher. We then present three episodes of probe use that exploit these unique affordances. In the last section, we discuss how the probe-supported activity design promotes meaning-making learning.

2. Analytical framework of probes in inquiry learning Inquiry learning is generally considered, in school practice, to be a sound demonstration of a set of scientific process skills which include claim forming, hypotheses making, data collecting, and conclusion drawing. However, development in the sociology of science (Roth, 2001) reminds us that scientific knowledge is fundamentally a social process of negotiation. Consistent with Wenger’s communities of practice (Wenger, 1998), these issues can be explored and interpreted further within a participation paradigm using the dual concept: participation and reification. Of particular importance to our study is his argument that meaning is located in a process he termed “negotiation of meaning”, which involves the interplay of two processes: participation and reification. Participation is always based on situated negotiation and renegotiation of meaning in the world, while reification gives form to experience by producing objects that “congeal this experience into thing-ness”(p.58). In the case of an inquiry activity, probes can be configured as a tool for creating reified objectsЁdigital numbers, graphs, and figures. The reification creates points of focus around which the negotiation of meaning becomes organized (Wenger, 1998, p.58). By manipulating the probes, learners involve themselves in hands-on activities. Their discussions are focused on the generated readings from the probes. The whole aggregate of inquiry activities congeals into thing-ness. These points of focus then immediately engage group members in the process, prompting them to participate in the activities of negotiation. It is through this dynamic process that meaning is re-negotiated. These learning technologies play thus a vital role in supporting the pursuit of shared meaning construction. Using that analytical framework, this study is intended to explore in what ways these new tools transform an established inquiry activity.

3. The emergent nature and negotiation in probe-supported inquiry learning We worked for over a year (2003-2004) observing a grade 5 science classroom. The science teacher was our seed member in integrating technology into a science classroom research team. He struggled to introduce the use of probes (Multi-Log) to students in the time allotted. Among the challenges faced were the language barrier, as students sought to use the English interface of the probes, the fact that extra time had to be spent adopting the learning tool and dealing with the resulting tensions of meeting unified schedule of teaching progress (very common in Taiwan), and the various concerns raised by parents. However, near end of the semester, they finally shifted focus from teaching a probe-use skill-set to that of the higher order learning potentials of the tools.

F.-C. Chen et al. / Probing Technology as Affordances for Negotiating Meaning

589

The videotape analyzed in this study was the three-hour chronicle of an inquiry activity in which probes were used to measure water temperature. The teacher created six groups from the class to conduct this experiment with one probe and one tablet PC in each group (see Figure 1). Three episodes were excerpted from the three-hour videotape to illustrate what and how students learn when studying science with the support of probes. The first one involved many-to-many interaction in which meaning is Figure 1. Students concentrate on the readings of the probes and the curve shown on their tablet PC.

generated at the group level; while the remaining two followed a typical classroom pattern (Lemke, 1990), with teacher-interpreted meaning and teacher-centric dialog.

The central concerns in this section are: What do probe-supported activities bring to learning situations that do not exist when teaching science with the use of a traditional thermometer? How do the probes produce objects that “congeal their experience into thingness”? How do real-time readings result in greater student engagement? And, in turn, how does this engaged participation provide students with the opportunity to re-negotiate meaning in a new context?

3.1 The important concept of ‘rate of change’ was instantaneously captured Introduction. The teacher assigned each group leader the task of fetching a cup of hot water from him. He then had each group put the temperature probe into the hot water for 500 seconds with temperature to be recorded each second. He also reminded the class not to stir the water when taking measurements. Here is a transcript of the conversation that took place in one group. episode 1: 1 A: What is the temperature? 2 B: 69..70…. 3 C: It’s going down. 4 B: Really, 63…. 5 B: It’s going up again. ..66…67… 6 B: It’s going down again…64.33 7 C: It’s going down rapidly. 8 D: Is the water very hot? 9 A: What happened to the other groups? Go to check them. 10 B: Wow! It’s going down…50 something… 11 D: It’s so strange. How come it’s going down in a rush way? 12 B: Come on! Don’t stir it. 13 A: Man! Please move a little bit. I can’t see it. 14 C: Didn’t the temperature change rapidly earlier? 15 D: What the temperature is? It’s too weird! 16 A: Watch out! It’s going to be 500th second. You have to push STOP in time. 17 C: Yeh! It stops. It’s in 45 degree. The traditional thermometer serves to measure static equilibrium, which means that the value represents a state in equilibrium. One of the traditional laboratory activities performed by students is, for example, using a thermometer to measure the temperature

590

F.-C. Chen et al. / Probing Technology as Affordances for Negotiating Meaning

over time of a beaker of ice water heated by a Bunsen burner (Staudt, 2001). In this case, students have to concentrate on the tedious task of taking readings from the thermometer at specified intervals. It needs hardly to be said that the tedium of long term experiments taking place over multiple minutes, hours or days is far worse. Also significant is the fact that accurate records of events taking place within very brief periods, (within one second, or 0.01 second, etc.) are not practical. The temperature probe, on the other hand, specializes in measuring dynamic equilibrium and changes of temperature over very short periods of time in an open system. In our study, the constantly updated digitized readings allowed students to become aware of temperature changes between fixed time intervals (one second). As the temperature probe showed on the screen one value each time, students were simultaneously led to compare values in the sequence. In line 2-6, they were very sensitive about the ups and downs with the affordable function of temperature probes. In line 7, 11, 14, and 15, the continuous readings scaffolded them to identify a very important phenomenon: the rate of change during observation. This curiosity prompted them and opened up an opportunity for exploration. Introducing probes into the learning environment also changes the classroom’s attentional affordances (Tatar, et.al., 2003). Analyzing from a participation paradigm perspective, we conceive the probes as a tool not only for effectiveness in observing and collecting data, but as a tool for congealing observed experience into thingness. If attention is a sine qua non during scientific observation, the inscriptions reified through the use of the probes can themselves generate this attention, as can conversations about the readings, and unexpected discoveries made by students during the process. Following this line of thinking, the perspective of “Having a tool to perform an activity changes the nature of that activity” (Wenger, 1998, p. 59) further expands our interpretation. Reifying the concept of temperature may not change its effect on our bodies, but it does change the way we experience the world by focusing our attention in a particular way and enabling new kinds of understanding. The significance of probes is not limited solely to possibly speeding-up student comprehension of this experiment; rather, the simultaneously captured concept “change of rate” provides further opportunity for exploration. “Why does it go down rapidly?” becomes a driving question for the group (and may be common throughout the class). In other words, students’ motivation was triggered by the unique affordances of temperature probes.

3.2. The compelling teachable moments were found Introduction. This episode is taken from a whole-class discussion designed to account for the outcome of an increase followed by a decrease in temperature readings generated during the 500-second observation. Although perceived as a typical classroom pattern, we found that this discussion took place between teacher and students in a very different way. The teacher was initially intent upon getting back to the curiosity generated in many groups about “Why it goes down rapidly?” and anchored the discussion with the ‘WHY’ on the phenomenon of ‘rate of change.’ This appeared to be too difficult for grade 5 students to answer so the teacher instead asked for reasoning to explain the phenomenon of temperature “going up and then down”. Episode 2: 18 Teacher: Does anyone in your group remember the beginning temperature of the water in the cup? 19 Student a: 62

F.-C. Chen et al. / Probing Technology as Affordances for Negotiating Meaning

20 21

591

Student b: Teacher:

63 OK. It doesn’t matter. The beginning temperature is around 60 degree. Let’s look at the curve. Can you see, the temperature was going up and then down, why? If anyone finds the right answer, I will give him/her a reward. 22 Student c: The digital number kept going up till a peak, then the temperature of the water in a cup began cooler, so the digital number was going down too. 23 Teacher: A little bit close. 24 Student d: The temperature probe was originally in the air, and then it was put in the hot water. These two temperatures were different, so when you put it first in the hot water, the digital number would go up from the low position. 25 Teacher: Yes, correctly! Now the number showed in your temperature probe is the same as the temperature of this classroom, which is 19 degree roughly. So the digital number would have to go up till the same as the temperature of the water. The target of the inquiry was to provide insight into the air-water phase change, which was not easy for grade 5 learners. It was probably due to the incentive of a reward, in line 22, that one student responded quickly. Although he did not get the point, he noticed a peak in the curve, which was a key to the correct interpretation of the phenomenon behind the curve. Were traditional thermometers used, the time interval at which temperatures would be recorded could not be brief enough for observers to be sure of detecting the critical peak when dealing with dynamic balance Ё as was here the case. In responding to this inquiry question the teacher posed, readings from the probes never again served as quantitative information for effective learning. Instead, the curves each group had generated became reified objects whose meaning could then be re-negotiated. In line 22, the student’s misconception about the function of the probes (i.e., they overlooked the fact that readings were a transition from measuring the air to the hot water.) was also examined and the meaning behind the peak of the curve was therefore uncovered. From a participation point of view, the interpretation of the readings was designed to rely partly on participation. That is, the emergent opportunity for learning was actually triggered by misconceptions or alternative explanations inherent in the participatory process, and this scaffolded the negotiation of meaning. We will discuss this more in the next episode.

3.3. The negotiation of meaning-making on inscriptions followed Introduction: Near end of the three-hour class, each group drew the curve shown on their tablet PC on the whiteboard for comparison. The teacher raised one question for whole-class discussion. Episode 3: 26 Teacher: Attention! How come the temperature decreased differently in each group? Group 1 decreased slower than the others. Can anyone give us an explanation? 27 Student a: It was possibly that they did something to hold the temperature. 28 Group 1: No, we didn’t. 29 Student b: It’s the place that matters. It is because their group was near the door. 30 Teacher: If so, then the temperature should have been decreased quicker.

592

31 32 33 34 35 36

F.-C. Chen et al. / Probing Technology as Affordances for Negotiating Meaning

Student c: Teacher: Student d: Teacher: Student e: Teacher:

It’s because they sit closer to each other. Is that possible? Yes, but the chances are slim. They use paper cup while we use plastic cup. Sounds interesting too. Teacher, the light! It’s very simple! Right at the beginning of the experiment, the conditions among the groups were not equal. My hint is quite obvious. 37 Student f: The height of the water that you filled in each cup is different. 38 Teacher: Yeh! Maybe I put a little bit more water into group one’s cup. 39 Teacher: That’s why I remind you many times that you have to be careful about all the conditions and their relationships with the data. Having engaged in the use of probes for observation, at end of the class many students were able to make sense out of group 1’s curve. Each student mentioned something s/he felt important to the cause of the particular shape based on his/her own interpretation (see figure 2). Although most of the answers seemed not to satisfy the teacher, it was the students who negotiated meaning with their teacher, and tried to come to a shared interpretation. The teacher, using a probing technology, created points of focus around which the negotiation of meaning become organized. The production of such a reification is crucial to the kind of negotiation that is necessary in order for them Figure 2. A student interprets why to be able to bring together the multiple perspectives, the curve goes up first and then interests, and interpretations that participation entails down. (Wenger, 1998, p.62). Within the frames of Wenger's (1998) social theory of learning, such sharing processes are not merely translations (p.68); they are indeed transformations Ё the production of a new context of both participation and reification, in which the relationships between the tacit and the explicit are renegotiated. The inscriptions (reification) were meaningful to them because, through participation, they experienced the process of generating the curve and came to know exactly how the readings configured this curve. The inscriptions were also new to them because, through participation, they gained new understanding. In other words, they re-negotiated their meaning. These three episodes emphasize the unique attributes probing technologies afford in the science classroom. Indeed, with the focusing and reified effects of the probes, students instantaneously noticed some important concepts beyond the simple act of observation. In this case a teacher-centered dialogue transformed into a student-driven exploration. The more the students negotiate, the more meaningful learning they appropriate.

4. Lessons learned This study proposes a participation framework to uncover the catalytic affordances of probing technologies in support of inquiry learning. We now turn to some implications of this work.

F.-C. Chen et al. / Probing Technology as Affordances for Negotiating Meaning

593

4.1. From computational to catalytic affordances From the designers or manufacturers’ viewpoint, the affordances of the probing technologies are perceived mainly from a technological perspective. Probing technologies are regarded as real-time data acquisition collectors, the major purpose of which, in an inquiry-based learning context, is to enhance and accelerate the learning process. However, in elementary school, some practitioners argue that paying too much attention to the accuracy and real-time functions of the data does not inspire children very much (Chen & Jiang, 2004). Rather, they believe that science education should be focused primarily upon the ontological aspect of scientific phenomena. For example, they think “What does humidity mean?” is a higher priority than determining ‘Is place A more humid than place B?’ It seems that these practitioners have their own perspectives on how a classroom is made effective and how that efficacy can be promoted with the support of technology. In our study, we described how the computational power of probes is transformed. The computational power of the probe did play its role in successfully measuring the temperature of dynamic balance in an open system but, more importantly, the process of data-taking with short intervals (i.e., per second) provided students with higher data resolution with which they were able to make sense of the readings on a deeper level. The three episodes are not excerpted to represent a probe-centered, but rather a probe-supported activity. In other words, it is not our intent to demonstrate how powerful a tool the probe proved to be each episode; instead, we depict the probes as triggers for developing and sustaining a meaningful inquiry activity in the elementary classroom.

4.2. The interplay between the readings and the learners Research on mobile technology (e.g., Roschelle, 2003; Sharple, 2002) is beginning to pay attention to coupling issues. Indeed, the existing enabling technology does not yet couple well with desirable social practices. Also in practice, wireless or internet discussion is not specific enough to describe what sorts of informatic coupling are desirable in mobile learning. It therefore seems that we need more empirical research on the separation of the roles of technology-based communication and non-technology-based interpersonal communication. Based on our findings, it appears that the learning that occurred did not result directly from being mediated by technology. From a participation perspective (Lave & Wenger, 1991), classifying things can only captures surface features of the learning, not its fundamental processes. The duality of participation and reification is not just a distinction between people and things for what it means to be a person and what it means to be a thing both involves an interplay between the two. Through the negotiation of meaning, it is the interplay of participation and reification that makes people and things what they are (Wenger, 1998, p.70). In our study, students engaged in a probe-supported activity and generated readings from the probes. The reified readings have no meaning without a learner’s interpretation. On the other hand, in the process of re-negotiation, learners gained new understanding while the reified readings themselves remained unchanged. The readings and the learners cannot be transformed into each other, yet they transform each other. Therefore, designs for a probe-supported classroom practice must always be distributed between participation and reification – and its practical realization depends on how these two sides fit together. The three episodes in our study demonstrate how teacher and students make meaning of measurements as the result of coupling well between the enabling technology and desirable social practices.

594

F.-C. Chen et al. / Probing Technology as Affordances for Negotiating Meaning

5. Conclusions The purpose of this study is to bridge the gap between the technological affordances perceived by the designers, and the educational affordances perceived by the practitioners. Starting with the concept of Wenger’s duality fundamental to the negotiation of meaning, which involved the interaction of two constituent processes, participation and reification; we illuminate the potentials of probes as catalysts for meaning making in inquiry learning. By proposing this pair of constituent processes for purposes of conceptualization and as an analytical framework for orchestrating classroom activities, we pinpoint a new understanding of the educational affordances of probing technologies in inquiry activities. We are not discounting the contribution of computing and simultaneous data-taking power of probes to student learning, but rather we are reacting to the failure of traditional perspectives to account for the inscriptions and reified objects and the individual interpretation through which engaged and meaning-making participation emerges.

References [1] Chen, F. & Jiang, H.M. (2004). Probing technology in inquiry classroom: A participation perspective. Electronic Journal of Digital Learning, 2. http://www.ael.org.tw/filectrl/2-5.pdf (in Chinese) [2] Krajcik, J.S. & Starr, M. (2001). Learning science content in a project-based environment. In Tinker R. & Krajcik,J.S.(Eds.).Portable Technologies: Science Learning in Context. Netherlands: Kluwer Publishers. [3] Lave, J., & Wenger, E. (1990). Situated Learning: Legitimate Periperal Participation. Cambridge, UK: Cambridge University Press. [4] Lemke, J. L. (1990). Talking science: Language, learning and values. Norwood: Ablex Publishing Co. [5] Novak, A.M. & Gleason, C.I. (2001). Incorporating portable technology to enhance an inquiry, project-based middle school science classroom. In Tinker R. & Krajcik, J.S. (Eds.)(2001). Portable Technologies: Science Learning in Context. Netherlands: Kluwer Publishers. [6] Roschelle, J. (2003). Unlocking the learning value of wireless mobile devices. Journal of Computer Assisted Learning, 19(3), 260-272. [7] Roth, W.-M. (2001). Learning Science through Technological Design. Journal of Research in Science Teaching, 38(7), 768-90. [8] Sharples, M. (2002). Disruptive devices: mobile technology for conversational learning. International Journal of Continuing Engineering Education and Life-Long Learning (IJCEELL), 12, 5/6, 504-520. [9] Staudt, C. (2001). Curriculum design principles for using probeware in a project-based learning setting: learning science in context. In Tinker R. & Krajcik, J.S. (Eds.)(2001). Portable Technologies: Science Learning in Context. Netherlands: Kluwer Publishers. [10] Tatar, D., Roschelle, J., Vahey, P., & Penuel, W. R. (2003). Handhelds go to school: Lessons learned. IEEE Computer, 36(9), 30-37. [11] Tinker, R. F. (2001). Ice machines, steamboats, and education: Handhelds in a wider context. In Tinker R. & Krajcik, J.S. (Eds.)(2001). Portable Technologies: Science Learning in Context. Netherlands: Kluwer Publishers. [12]Tinker, R. (2000). A history of probeware. Retrieved October 22, 2005, from www.concord.org/research/probeware_history.pdf [13] Vahey, P., Tatar, D., & Roschelle, R. (in press). Using handheld technology to move between the private and public in the classroom. In M. A. van 't Hooft & K. Swan (Eds.), Ubiquitous computing: Invisible technology, visible impact. Mahwah, NJ: Erlbaum. [14] Wenger, E. (1998). Communities of Practice: Learning, Meaning, and Identity. Cambridge University Press.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

595

Analysis of Meaning Making in Online Learning Daniel SUTHERS, Nathan DWYER, Richard MEDINA, Ravi VATRAPU Department of Information and Computer Sciences, University of Hawai`i, USA [email protected] Abstract: This paper reports on our efforts to deepen the analysis of online collaborative learning. Most studies of online learning use quantitative methods that assign meaning to contributions in isolation and aggregate over many sessions, obscuring the actual procedures by which participants accomplish learning through the affordances of online media. Methods for studying the interactional construction of meaning are available, but have largely been developed for brief episodes of face-to-face data, and do not scale well to online learning where media resources, time scale, and synchronicity all differ. In order to resolve this tradeoff, we are developing an analytic method that scales up sequential and interactional analysis to longer term distributed and asynchronous interactions. The paper describes applications to data derived from asynchronous interaction of dyads and small groups. Our long-term objective is to obtain a deep understanding of how learning is accomplished in technology-mediated interactions that take place at multiple time scales in different media. Keywords: Collaborative learning, analysis methodology, interactional construction of meaning

1

Introduction

Recent developments have highlighted the ascendancy of online learning [1, 2]. Online collaborative learning brings together social processes of learning and representational aids for this learning, providing a fertile area for research and development while serving an important application. An understanding of how participants appropriate and are influenced by the affordances of the medium is needed to adequately inform the design of the learning experience and the resources that support it [3]. Because learning is largely social, it is also critical to understand the intertwinement of individual and intersubjective trajectories of meaning-making [4]. Yet we do not yet sufficiently understand these areas. Since most online learning has been mediated through text-based tools, we lack intensive study of how richer representations mediate online learning. Moreover, most studies of online learning use quantitative methods that disaggregate interaction into segments and assign meaning to these segments in isolation through coding, losing the interactionally constructed meaning. These methods aggregate over many sessions, obscuring the actual procedures by which participants accomplish learning through the affordances of online media [5]. Methods for studying the interactional construction of meaning are available [6, 7], but have largely been developed for brief episodes of face-to-face data, and do not scale well to online learning where media resources, time scale, and synchronicity all differ. This analytic tradeoff between scalability and fidelity must be resolved in order to inform the design of improved online learning environments and participation structures that engage participants more deeply in intersubjective meaning-making during collaborative inquiry. In this paper we report on our efforts to resolve this tradeoff by scaling up sequential and interactional analysis to longer term distributed and asynchronous interactions while remaining grounded

596

D. Suthers et al. / Analysis of Meaning Making in Online Learning

in participants' use of media. The paper describes applications to data derived from asynchronous interaction of dyads and small groups. Our objective is to obtain a deep understanding of how learning is accomplished in technology-mediated settings when analyzing asynchronous computer-mediated interactions that take place over various durations of time, in different media, among large groups of people. We begin by briefly considering existing analysis paradigms and outlining our approach before providing examples of analysis. 2

Analysis of Online Learning

The experimental paradigm compares an intervention to a control condition in terms of one or more variables. Where process data is considered, it is most often analyzed in the “quantitative” paradigm, in which units such as actions and utterances are annotated under some coding system (e.g., [8]) and then statistical methods are used to compare counts across groups to draw conclusions concerning aggregate (average) group behavior. Such methods are suitable for testing proposed differences between groups. For example, our own work [3, 9] tested hypotheses concerning “representational guidance,” e.g., that users of one version of evidence mapping software will talk more about evidence than users of another version. Yet, coding and counting cannot capture the actual practices of intersubjective meaning-making, and hence the most interesting part of collaborative learning is missed. There are two basic problems. First, the meaning of an act is assigned as an isolated unit, missing the sequential construction of this meaning. Second, when data is aggregated, one loses the actual methods by which individuals appropriated the medium to accomplish collaborative knowledge construction. Without observing how media affordances are used or how opportunities are lost, it may be more difficult to generate design recommendations. Therefore we began explorations in analysis paradigms that better capture intersubjective meaning-making and the role of technology affordances in supporting these processes. Methods of analysis that find the meaning and significance of each act in the context of prior interaction include Conversation Analysis [10], Interaction Analysis [6], and the family of analysis methods loosely classified as “sequential analysis” [11]. Typically, video or transcripts of naturally occurring interactions are studied to uncover the methods by which participants make themselves accountable to each other and accomplish their objectives. For examples applied to the analysis of learning, see Baker [12], Roschelle [13], Koschmann et al. [14], and Koschmann, et al. [5]. This paradigm is becoming increasingly important in computer supported collaborative learning (CSCL) because an approach that focuses on accomplishment through action is necessary to truly understand the role of technology affordances [15]. Yet we also encountered limitations in these methods, mostly due to the assumptions the methods make about the interactional properties of the media they study. Both Conversation Analysis (using audio recordings with Jeffersonian transcripts) and Interaction Analysis (which relies on video recordings) are concerned with face-to-face interaction. The temporality and ephemerality of spoken interactions requires turns [10] and adjacency pairs [16] as the units of analysis. These units of analysis are not as appropriate for computer-mediated communication (CMC) since most online media support parallel production and persistence of contributions. Online media allows multiple participants to produce contributions simultaneously, eliminating the need for turn taking. Furthermore, contributions may become available to other participants in unpredictable orders, may not be immediately available, and because of the medium’s persistence participants may at any time address an inscription that was created much earlier. Online, conceptual coherence is decoupled from temporal adjacency. We cannot simply focus analysis on the relationships

D. Suthers et al. / Analysis of Meaning Making in Online Learning

597

between adjacent events. Nor can we treat CMC as a degenerate form of face-to-face interaction since people adapt to these media attributes and use them to create new forms of interaction [17]. The properties of asynchronous online media require an alternative basic unit for analysis of interaction that accommodates noncontiguous contributions and allows for tracking of availability as a prerequisite to awareness and access. Additionally, this unit of analysis must be applicable to the wide variety of temporal, spatial, and social scales of online activities. Since collaboration is only possible when something is shared and transformed between participants, we began to work with the concept of “uptake”: the event of a participant doing something with previously expressed information, attitudes and attentional orientation. Uptake can incorporate a participant's own prior contribution as well as those of others: by identifying both, we can characterize the mixture of intrasubjective and intersubjective knowledge construction. Uptake is similar to the “thematic connections” of Resnick, et al. [18], but allows for media as well as linguistic relationships. These ideas were originally developed using data from synchronous interaction of dyads [19]. Over the past year we further developed these ideas by analyzing data from an experimental study of pseudo-asynchronous, dyadic collaboration [20]. We found it necessary to separate participants’ media actions from the analysts’ inference of an uptake event. This led to a three-level analysis method: 1. Identification of “Fixed Points.” The first level identifies points where the existence of a conception is empirically grounded in intentional acts of coordination between conceptions and representations [21] (such as editing a shared workspace). We call them “fixed points” because they provide empirically grounded points of departure for further analysis. 2. Identification of Dependencies (Potential Uptake). The second level of the analysis builds a dependency graph of how these acts refer to, manipulate, or otherwise take up previous conceptions. Evidence for dependencies can be roughly divided into media dependencies (e.g., sequences of actions on a representational element) and conceptual dependencies (e.g., reuse of words and phrases). Dependencies are candidates for uptake events. 3. Analysis of Intersubjective Meaning-making. The third level of analysis identifies uptake events, assigning interpretations to sequences of dependencies based on the theoretical phenomena of interest, such as argumentation or collaborative knowledge construction. The dependency graph can serve as a basis for comparison and integration of multiple theoretical interpretations, i.e., a boundary object [22] for the study of collaboration. Our own approach seeks to understand the meaning of an uptake act in terms of how it brings forth and actualizes some aspect of the prior interacction as being significant for this moment and subsequent interaction. 3

Examples

In this section we provide two examples from our exploratory analyses. The first example, based on data from dyads interacting in a laboratory setting, is offered to illustrate intersubjective meaning-making in a highly instrumented asynchronous context. The second example, based on server logs of asynchronous threaded discussions in an online course, is offered to illustrate how our method can be adopted to conventional online learning settings.

598

D. Suthers et al. / Analysis of Meaning Making in Online Learning

3.1 Asynchronous Meaning-Making between Dyads in a Laboratory Setting The data for this example comes from a study in which participants interacted via a shared “evidence mapping” workspace to identify the cause of a disease on Guam (ALS-PD). The update protocol simulated asynchronous interaction [20]. In this setting, rich data including server logs and video capture of the screens are available to us, so we are able to examine the interaction in great detail. Information was distributed across participants such that information sharing was necessary to refute weak hypotheses and construct a more complex hypothesis. At the end, participants wrote individual essays. Our analysis sought to identify whether and how the contents of the essays were accountable to the interactive session, and especially whether intersubjective meaning-making influenced the essays. In brief, how does collaboration lead to learning? We began by tracing back dependencies from the essays of participant 1 (P1) and participant 2 (P2) into the session to identify uptake trajectories that may have led to the essays. In their individual essays, both P1 and P2 mentioned “duration of exposure” as a factor. The example focuses on this convergence. The relevant subgraph is in Figure 1; many fixed points and dependencies are omitted for simplicity. P1’s actions are on the top and P2’s actions on the bottom. In general, time flows left to right, but this being an asynchronous setting we cannot assume that a contribution is available as soon as it is created. The vertical lines in each participant’s half demarcate when that participant’s workspace was updated to display new work by the partner. Numbered nodes represent fixed points, which may include contributions (editing the evidence map) or perception of the partner’s contribution (evidence map objects must be opened to be read). Arrows between the nodes represent dependencies (potential uptake relations). Dotted arrows are intrasubjective and solid arrows intersubjective uptake (always by a perception event).

Figure 1: Intersubjective meaning-making analysis of asynchronous dyads

D. Suthers et al. / Analysis of Meaning Making in Online Learning

599

Node 20 represents a summation of the disease causes expressed by P2 in a note posted to her workspace (but not yet visible to P1). Shortly after that in clock time but asynchronously from the participants’ perspectives, P1 creates a data object derived from an article; node 13 represents the conception expressed by this object. Subsequently, a workspace refresh makes the note expressing conception 20 available to P1: node 20a represents the conception that results from P1’s reading of this note, and the corresponding arrow represents the first example of intersubjective uptake. Sometime later, P1 creates a note indicated by node 10. This node is an uptake of both 20a and 13, as evidenced by the following media-level facts. First, in the interface this note follows that for 20a in a sequential note object: uptake is evidenced by direct media-manipulation. Second, 10 incorporates the concept of “duration of exposure” from 13, expressed as “… time has a factor, the longer you're exposed….” Here, uptake is evidenced by conceptual similarity. Clearly, 10 is an integrative contribution. Let us now examine how information originally available only to P1 (13) and P1’s integration of it (10) become available to P2. Sometime after 13, a refresh makes the corresponding data object available to P2, who accesses it as indicated by fixed point 13a. Subsequently, another refresh makes the response note of 10 available to P2, taken up at 10a. Since P2 has accessed both the data object reporting the “duration of exposure” (13a) and P1’s endorsement of the relevance of duration of exposure (rephrased) (10a), we view P2’s inclusion of this concept in her essay (e3) to be an uptake of both of these conceptions. P1’s essay portion (e48) also evidences uptake of the environmental factors originally expressed by P2 (20). The “round trip” from 20 through 20a, 10 and back to 10a and e3 represents intersubjective meaning-making on a small scale. We cannot rule out that e3 is uptake of only 13a and hence a one-way transfer of information, but nor can we rule out that P1’s endorsement of the importance of the idea in 10, accessed at 10a, also influenced P2’s inclusion of this idea in the essay. It is plausible that both were a factor. In this and other examples, the analysis method enabled us to make sense of the rich data available, examine the meaning-making trajectories of individual learners as evidenced by their manipulations of the media, and identify entwinements of these trajectories in ways that sometimes led to conceptions that could only be understood as a product of intersubjective meaning-making. 3.2 Multiple Participants in an Asynchronous Online Discussion The laboratory setting provided far richer instrumentation than might be expected in typical online learning applications. In order to explore how our method can be adopted to conventional online learning settings and what kinds of analyses are possible with lower resolution data, we analyzed server logs of asynchronous threaded discussions in an online graduate course. The discussion software records message-opening events as well as message postings, but recordings of participants’ manipulations of the screen are not available. In each of the analyses we conducted, we were able to identify a sampling of interaction episodes showing the potential for the method in producing a feature-rich analytical artifact for interpretation. Figure 2 illustrates one example of convergence of two topics across two related online discussion forums involving multiple participants over a seven-day period. One is a subgroup discussion and the other is a large group forum. Students in each subgroup address a set of discussion questions (e.g., q15, q14, q10, q9 in Figure 2). Each group then posts a summary to the class discussion group, and the whole class then has the opportunity to discuss these summaries. The analysis for this episode unfolded by backtracking from a convergent idea in the group forum, represented by node 1 on the far right of Figure 2, through a series of postings that led through two parallel threads and across two discussion forums. The identification of these trajectories relies heavily on a

600

D. Suthers et al. / Analysis of Meaning Making in Online Learning

complementary strategy that includes content analysis and access to server logs. The logs are used to determine and generate media-level fixed points: access to and posting of messages. Content analysis provides the means of transcending the media structure (access logs and reply structure), uncovering trajectories of meaning-making across fragments of discussion threads located in different forums. This ability to identify trajectories that are independent of yet influenced by media structures is an important strength of the method.

Figure 2. Fixed point graph on online interaction episode. Vertical lines represent days, colors represent participants, and arrows represent dependencies. 4

Advantages and Disadvantages of the Method

The dependency graph avoids premature theoretical commitment, so is able to function as a boundary object [22] for multiple theoretical analyses. This representation supports quantitative coding and counting, statistical approaches to sequential analysis, cognitive analysis and ethnographic analysis. If we started with one of these methods, the resulting representations would not support the other analyses. With the dependency graph the opportunity exists to determine how different theories explain the technology affordances for collaborative learning. The dependency graph is media-agnostic, and makes very few theoretical commitments to the nature of cognition or collaboration. It is a record of the multiple personal transformations that took place in an interaction and maps out their interdependencies. However, it is not media ignorant; it can bring in information about the medium. Intersubjective meaning-making can be identified independently of the media but is linked to media by the fixed points, so the relationship between meaning-making and the media can be examined. The analysis can function at multiple levels of detail, although any analytical results will be at the same level of details as the data under analysis. It will also scale according to the quality of the data: incomplete data allows incomplete analysis but does not obstruct analysis completely. Finally, the analysis of entangled personal trajectories does not require a solution to the individual/collective dichotomy. We can separate out individual trajectories and identify when contributions are available to and accessed by each individual, or we can step back and analyze the composite web of interpretations. Collective behavior such as “group cognition” [23] is observable as the result of multiple individuals allowing their individual actions to be influenced by the perception and interpretation of other's behavior. There are presently a few disadvantages of the methodology. The major disadvantage is that it is time consuming to construct a dependency graph. Customized software support can

D. Suthers et al. / Analysis of Meaning Making in Online Learning

601

help address this problem by partially automating data collection and the construction of the graph. The present work develops the specifications for such a tool. A related problem is the difficulty of retrieving information from and obtaining selective views of the dependency graph. Software support will also address this problem by through visualization technologies. In using the graphs for analysis, we have found that one must be careful not to make inferences based on the absence of fixed points and dependencies in the graph. Any graph is partial and can be extended indefinitely due to the continuous nature of human action. Also one must not conduct an analysis entirely by using the dependency graph. In addition to being a structure of interest in its own right, the graph should be used as an index to the original media records. 5

Conclusions

Current methods for analysis of the interactional construction of meaning have largely been developed from brief episodes of face-to-face data, and do not scale well to online learning where media resources, time scale, and synchronicity all differ. While quantitative methods scale well to online learning, they do so by segmenting interaction into units that can be coded and aggregated over many sessions. Consequently, these methods fail to capture the procedures by which participants accomplish learning through the affordances of online media. We are developing an analytical approach that scales up the advantages of sequential and interactional analysis to longer term distributed and asynchronous interactions. The approach has been prototyped on data derived from synchronous and asynchronous interaction of dyads and small groups. Ongoing work is refining the methodology and evaluating its relevance to design. Software support will also be required for this work, for example to view the uptake graph at multiple granularities and through filters, compressing it in time and/or scanning for patterns, and accessing the original data at will until we find an interactionally promising subgraph. Acknowledgments This work was supported by the National Science Foundation under award 0093505. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation. References [1] [2] [3] [4] [5]

[6] [7]

I. E. Allen and J. Seaman, "Growing by Degrees: Online Education in the United States, 2005," Alfred P. Sloan Foundation, Needham, MA November, 2005 2005. P. L. Parker, "Learning Technologies and their Impact on Science Education: Delivering the Promise," Australian Science Teachers Journal, vol. 46, pp. 9-20, 2000. D. D. Suthers and C. Hundhausen, "An experimental study of the effects of representational guidance on collaborative learning," Journal of the Learning Sciences, vol. 12, pp. 183-219, 2003. D. D. Suthers, "Technology affordances for intersubjective meaning-making: A research agenda for CSCL," International Journal of Computers Supported Collaborative Learning, vol. 1, pp. (in press), 2006. T. Koschmann, A. Zemel, M. Conlee-Stevens, N. Young, J. Robbs, and A. Barnhart, "How do people learn: Member's methods and communicative mediation," in Barriers and Biases in Computer-Mediated Knowledge Communication (and how they may be overcome), Computer Supported Collaborative Learning, R. Bromme, F. W. Hesse, and H. Spada, Eds. Amsterdam: Kluwer Academic Press, 2005, pp. 265-294. B. Jordan and A. Henderson, "Interaction Analysis: Foundations and practice," The Journal of the Learning Sciences, vol. 4, pp. 39-103, 1995. C. Goodwin and J. Heritage, "Conversation Analysis," Annual Review of Anthropology, vol. 19, pp. 283-307, 1990.

602

[8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23]

D. Suthers et al. / Analysis of Meaning Making in Online Learning

L. Rourke, T. Anderson, D. R. Garrison, and W. Archer, "Methodological issues in the content analysis of computer conference transcripts.," International Journal of Artificial Intelligence in Education, vol. 12, 2001. D. D. Suthers, C. Hundhausen, and L. Girardeau, "Comparing the roles of representations in face-to-face and online computer supported collaborative learning," Computers & Education, vol. 41, pp. 335-351, 2003. H. Sacks, E. A. Schegloff, and G. Jefferson, "A simplest systematics for the organization of turn-taking for conversation," Language, vol. 50, pp. 696-735, 1974. P. Sanderson and C. Fisher, "Exploratory sequential data analysis: Foundations," Human-Computer Interaction, vol. 9, pp. 251-318, 1994. M. Baker, "Computer-mediated argumentative interactions for the co-elaboration of scientific learning tasks.," in Arguing to Learn: Confronting Cognitions in Computer-Supported Collaborative Learning Environments., Andriessen, Baker, and Suthers, Eds. Dordrecht: Kluwer, 2003, pp. 47-78. J. Roschelle, "Designing for cognitive communication: Epistemic fidelity or mediating collaborating inquiry," in Computers, Communication & Mental Models, D. L. Day and D. K. Kovacs, Eds. London: Taylor & Francis, 1996, pp. 13-25. T. Koschmann and C. LeBaron, "Reconsidering common ground: Examining Clark's contribution theory in the OR," presented at ECSCW 2003: Eighth European Conference on Computer-Supported Collaborative Work, Amsterdam, 2003. G. Stahl, T. Koschmann, and D. D. Suthers, "CSCL: An historical perspective.," in Cambridge Handbook of the Learning Sciences, R. K. Sawyer, Ed. Cambridge, UK Cambridge University Press, 2006. E. A. Schegloff and H. Sacks, "Opening up closings," Semiotica, vol. 8, pp. 289-327, 1973. S. C. Herring, "Interactional coherence in CMC," Journal of Computer Mediated Communication, vol. 4, 1999. L. B. Resnick, M. Salmon, C. M. Zeitz, S. H. Wathen, and M. Holowchak, "Reasoning in conversation," Cognition and Instruction, vol. 11, pp. 347-364, 1993. D. D. Suthers, "A qualitative analysis of collaborative knowledge construction through shared representations " Research and Practice in Technology Enhanced Learning vol. 1, pp. 1-28, 2006. D. D. Suthers, R. Vatrapu, R. Medina, S. Joseph, and N. Dwyer, "Beyond threaded discussion: Representational guidance in asynchronous collaborative learning environments," Computers & Education, (to appear). E. Hutchins, Cognition in the Wild. Cambridge, Massachusets: The MIT Press, 1995. S. L. Star, "The structure of ill-structured solutions: Boundary objects and heterogeneous distributed problem solving," in Distributed Artificial Intelligence, vol. 2, L.Gasser and M. N. Huhns, Eds. San Francisco: Morgan Kaufmann, 1990, pp. 37-54. G. Stahl, Group Cognition: Computer Support for Collaborative Knowledge Building. Cambridge, MA: MIT Press, 2006.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

603

Facilitating Knowledge Construction by Providing Individualized Services Weidong PAN, Igor HAWRYSZKIEWYCZ Faculty of Information Technology University of Technology, Sydney PO Box 123 Broadway, NSW 2007, Australia {wdpan, igorh}@it.uts.edu.au Abstract: In this paper, we propose an agent-based approach to facilitating individual learner’s knowledge construction through individualized services. The services go beyond simply presenting course materials, but supply a wider range of technological facilities, tools and services to support the learning process. A learning process specification language is used to describe rules related to what supportive services should provide for which learners at what moment in a learning process. According to the rules, agents provide individualized services for online learners. Keywords: Online learning, individualized learning, supportive services, knowledge construction, units of learning, software agents

1. Introduction The rapid technological and social change puts forward needs for life long learning. Online learning is an increasingly preferable alternative to conventional classroom learning in order to satisfy such needs. Yet, to date technology is mostly used in the distribution and management of learning materials. Most of today’s online instructional systems just simply deliver course materials over the Internet. They offer a "one size fits all" approach to the delivery of learning materials, the personalized approach to education is sadly lacking from most online systems [2]. In such systems, information is presented to learners through a set of instructional sequences with predetermined outcomes. Learners are directly told about a solution for the problem under study or are taught how to get a solution by a predetermined mode and route. They are denied any opportunity to independently explore possibilities, make their own solutions and actively construct new knowledge in the process. Clearly this online instruction mode cannot satisfy present day’s needs for customizing learning to meet particular demands as it is cost prohibitive to develop customized systems for each particular need. This paper claims that what is needed are generic systems that can be used to easily customize online learning environments by putting predefined components together in different ways to satisfy different learning needs. We propose an approach shown in Figure 1 to identify such components. Here we first look at what is happening in the real world, the current practices in learning and teaching online. Then rather than simply translating each practice to a computer implementation, it develops a general conceptual model that can be used to describe any number of practices. At the same time it identifies the common components that can be applied to need various different learning requirements. This is the first step in introducing the kind of structure needed for implementing flexible computer learning systems. The conceptual model terms can then be used to model particular instances. Since learners often work together with their fellows to

604

W. Pan and I. Hawryszkiewycz / Facilitating Knowledge Construction

collaboratively solve practical problems, collaborative learning conceptual model is used to model the collaboration among learners. A mapping is provided to convert these models to an implementation, usually a learning space. The corresponding services are then developed using technological means such as software agent technology. Real world phenomena

Collaboration meta-model Collaborative concepts

A conceptual model for learning processes

Learning process

Modeling learning instances

Generic agent architecture Agent specifications Collaborative model of the learning process Implementing on a

technology platform

Figure 1. Our approach to developing a flexible learning system

In the paper, we will use this way to analyze the current practices in learning and teaching online and investigate the required services to facilitate individual learner’s learning, and present an agent-based approach to implement these services. We will also explore the description of the rules related to providing what supportive services for different learners under various learning scenarios. 2. An Approach to Constructing a Flexible Online Learning System This section will, by using the approach shown in Figure 1, identify the common components for an online learning process based on the investigation in the current practices, and propose an agent-based approach to constructing a flexible online learning system to facilitate individual learner’s knowledge construction. 2.1 Analyzing the current practices Currently the most common dichotomy in the learning and teaching online is between objectivist and constructivist approaches. The trend is the constructivist approach places more emphasis on providing customizing methods. Objectivist learning assumes that knowledge objectively exists and can be transmitted directly from a teacher to learners. Thus the learners passively receive the information, while the teacher sends the information. Constructivism, however, argues that knowledge cannot be transmitted but must be constructed by learners [5]. The learners are active knowledge-constructors, while the teacher is a cognitive guide who provides learners with individual guidance and scaffolding to support the construction. This subtle difference has profound implications for online learning systems. Because constructivist learning focuses on knowledge construction, learning by adopting a constructivist method can generate more significant results [9]. The prime problem in constructivist learning is it requires learners to possess a certain skill of independent learning. Earlier study has indicated that not all learners are equally capable of adequately constructing knowledge on their own [1]. For instance, some may lack necessary prior knowledge or abilities to independently choose a learning resource and adopt a proper method to conduct a learning process. Consequently, supportive services are necessary to assist learners to study independently. These must include services that go beyond simply presenting course materials, but supply a wider range of technological facilities, tools and services to support the learning process. According to constructivism, knowledge is conducted by learners through actively interacting with and exploring the surrounding environments. They have to actively take their own learning activities to make meaningful understandings of the study theme. To facilitate meaning-building, teachers, tutors or even the online instruction system should not

W. Pan and I. Hawryszkiewycz / Facilitating Knowledge Construction

605

offer them with the solution for a problem under study or impose them to attain a solution using a designed mode. Rather, they should provide a constructivist environment to support the construction. The constructivist learning environment (CLE) is a rich learning environment, where learners may work together and support one another as they use a variety of tools and information resources in their guided pursuit of learning goal and problem-solving activities [9]. Conceptually an online CLE consists of three highly interrelated components: electronic learning spaces, cognitive tools, and supportive services. Online learners interact with the electronic learning spaces to represent and manipulate the study problem, build their own knowledge representations, and communicate with other learners to share and exchange ideas and views. They construct meaningful understandings of the study theme through engagement in reflective, critical thinking and collaboration with others, facilitated by the sophisticated cognitive tools integrated in the learning spaces [4]. In the constructive process, learners are offered cognitive guides by the supportive services to help them construct understandings and develop skills relevant to high-order problem solving. The understandings of an online CLE will be used as a theoretical background for the study presented in this paper. Building upon this background, we will identify the common components in the online learning process and propose an agent-based way to facilitate individual learner’s knowledge construction through various individualized services. 2.2 Identifying the common components To identify common components in an online learning process, we first look at the learning process itself and the kinds of elements or concepts that make up the learning process. The supportive services are then identified based on the requirements for the process steps. The main elements, or the learning process concepts, for a learning process can be summarized as follows (see Figure 2): x learning environment where learning takes place, which refers to the physical as well as to the social environment in which learning takes place and might include physical entities, tools, and people; x learning goal describing what to be achieved through the learning activities; x learning plan defining what learning activities will be carried out and the sequence of the activities to be taken to achieve the learning goal; x learning activity describing the action to be performed for the learning plan, e.g. writing a report, evaluating a problem, etc; x subject metadata providing explicit references to learning resources needed in learning; x learning method defining the way how to conduct learning, including what actions will be carried out in each step of the learning activity; x supportive services describing the services provided for various learning methods respectively. Learning environment provides

sets

Supportive services

Learning goal

use

Learning methods

about

creates follow

Learning activities

contain

Learning plan

Subject metadata

access

Figure 2. Common components for an online learning process

606

W. Pan and I. Hawryszkiewycz / Facilitating Knowledge Construction

This description illustrates a framework for placing the learning activities within a practical context. Thus for example, supportive services depend on the environment in which learning takes place. The learning proceeds in accordance with a plan. How the plan is set depends on the learning environment and the learning goal. The revision of the plan depends on the practical progress of the learning in the environment. Similarly, the method used to support learning activities depends on available supportive services and learning process. It is used to identify the kinds of engagements to be supported. 2.3 Constructing a flexible online learning system Based on the understandings of the CLE, we propose an agent-based way to facilitating individual learner’s knowledge construction through putting the elements and concepts together in different ways. The overall architectural framework of the agent supported learning system (ASLS) is shown in Figure 3. Electronic learning space technology is used to create learning spaces for online learners to manipulate study problems, build knowledge and collaborate with others. The services are implemented by software agents. They are responsible for providing services in individualized ways to satisfy different learning needs. All the rules related to providing what supportive services for which learners at what moment in a learning process are stored in the UOL (unit of learning) database. The UOL database contains a collection of careful designed learning units, each of which defines how to organize learning for different learners to achieve the objective of the unit, including the learning plans, learning materials, assessment methods, case study materials, and so on. According to the rules stored in the UOL database, the agents dynamically organize and generate supportive services for individual learners. The agents observe and monitor individual learner’s learning and evaluate their learning progress. The learner profiles are built and timely updated by the agents through the monitor and evaluation. The agents take the practical learning scenes and the learner profiles as input and generate individualized services for learners. Learners with different learning characteristics are provided distinctive supportive services, and with different presentation modes. Learner profile

UOL database Learning designs

Software agents

Collecting and reasoning

Individualized services

Learning activities

Electronic learning spaces Learners

Monitoring

The agent-supported learning system

Figure 3. Overall architectural framework of the ASLS

3. Implementation of the Individualized Services Quite clearly, the rule stored in the UOL database concerning providing what supportive services for different learners is crucial to make the supportive services be adapted to an individual learner's just-in-time needs and just-for-me needs in online learning. This section will explore the description of the rule so that software agents can use them to provide customized services for learners. A learning process specification language we have developed to describe the rule will be presented and its specific features will be analyzed.

W. Pan and I. Hawryszkiewycz / Facilitating Knowledge Construction

607

3.1 Brief analysis of the earlier work To describe rules related to providing what services for different learners in a learning process is in fact to describe learning activities and processes and the corresponding services to support them. A specification language is thus necessary. Such a specification language has been concerned in earlier researches for different purposes. Recent examples are the investigations for sharing and reusing learning resources, i.e. learning objects, and learning designs. The former, e.g. the IEEE LTSC/IMS standards, SCORM's Content Aggregation Model, etc., define standard ways to describe various learning objects so that they can be shared in online instructional systems. Their main problem is they place too much emphasis on content delivery rather than looking more carefully at what learners do [1]. The latter try to develop a generic language to describe learning designs, i.e. various templates of learning methods, so that they can be shared and reused. The IMS Learning Design specification (IMS-LD) [3] is a typical example. From the constructivist views of learning, it is unsuitable to select one from the encapsulated templates of methods and then impose it to an individual learner because the imposed method is most likely not the optimal one for the learner. Our purpose is to develop a way to describe learning activities and processes so that individual learner’s knowledge construction can be supported and facilitated by supportive services. Building upon the earlier researches on learning objects and learning designs, particular the EML [6], we have developed a learning process specification language (LPSL) to describe and specify learning processes for different learners and the corresponding services to support learning [8]. The LPSL is used to specify learning processes and the corresponding services through a series of UOLs. A UOL is an abstract term used to refer to any delimited piece of education and training [3], such as a course, a lesson, a module, or even a single learning activity such as a discussion to elaborate on some topic. In the following, the role of LPSL is described through a UOL. 3.2 UOL described using LPSL Figure 4 illustrates the basic structure of a UOL when it is described using LPSL. For purposes to increase readability, it is shown in a tree view, only containing a selection of elements and relationships, and no attributes. The major fields and their roles for specifying various learning processes and the supportive services are outlined as follows: (1) Metadata field is to provide a general description of the UOL, including Title, Prerequisites, Learning objectives, etc. Learning objectives describe the overall learning objectives to be achieved by learners who complete the UOL. Each objective has a brief description and a corresponding category. Keywords are the ones extracted from the objective descriptions and are used to match with the learning goal of the learner. (2) Roles field is to specify the intended users of the UOL, such as learners, teachers, etc. They can be further categorized through the Property field, where an identifier of the category is defined and a brief description to the features of the users in the category is provided. This design enables the supportive services to be provided according to the unique features of individual learners. (3) Content field is to describe all the learning resources and all the learning activities related to the UOL. This design is to enable the agents to provide services not only concerning learning resources but also various learning activities, and their combinations. A learning activity is described in an Activity field. Within an Activity, Type specifies the category of the activity (e.g. group activities, etc.), What gives a textual description of what will be done in the activity, Complete indicates the status change after the activity is completed (the status change will affect the sequence of learning activities), and Activity output specifies the artifact files for evaluating the outcomes of the learning activity.

608

W. Pan and I. Hawryszkiewycz / Facilitating Knowledge Construction Id Title

Metadata

Keywords

Type + Learning objective

Learning objectives

Description

Prerequisites

Roles

Type

Learner

+ Property

? Tutor

Description

? Type What

+ Activity

Complete * Activity input

Content

* Activity output

+ Learning resource

Traits

Type

? Content object

? Metadata

? Communication object

Location

? Tool object

UOL

Methods

+ Activity structure

If

Type ? Condition

Activity structure Id

Traits

+ Activity sequence

+ Learning step

* Learning resource ref

Description

* Assessment ref

Then ? Else

Activity ref UOL ref

* Service options

* Case ref Type

? Assessments

+ Assessment Type

? Cases

+ Case

Traits Source

Traits Source Property ref

Plans

+ Plan

Activity structure ref

Legend ? means optional * means zero or more + means one or more represents a sequential list of elements represents a selection of one of the elements

Figure 4. Structure of a UOL described using LPSL

A learning resource is a learning material some learners will use in the learning for the UOL. It can be in any form but mostly is a Web resource. Within a Learning resource, Traits specify the specific features of the resource, Content object indicates the medium type of the resource and its exact location, Communication object illustrates the requirements for the communication facilities, and Tool object specifies the prerequisite tools and facilities for using the resource. (4) Methods field is to define all the learning routes to achieve the objective of the UOL. They are categorized based on the learning characteristics of their targeted learners and divided into different groups accordingly. Each group is put into an Activity structure field. This implies that all the learning methods defined in an Activity structure suit a particular category of learners. An Activity structure may contain multiple Activity sequence fields. An Activity sequence defines a particular learning route in which a sequence of learning activities is specified. This enables agents to provide learners with multiple optional learning methods that suit their learning characteristics. Each Activity sequence can be associated with more than one learning resource, ensuring a learning route can be taken by using different learning resources. Each Activity sequence can be associated with more than one assessment approach, allowing the outcome of learning to be evaluated from different aspects. Also, each Activity sequence can be associated with multiple case study materials, letting agents scaffold knowledge construction through providing multiple optional case study materials. Within an Activity sequence, a learning step can be a reference to a learning activity defined in the UOL or a reference to another UOL. In the latter case, a learning step is the execution of other UOL; this way lets a hierarchical architecture of a module or a subject be defined if required. An Activity sequence may also define a conditional learning path of the given learning steps, allowing sequencing, conditions and repetitions of some learning steps. The Type field is to specify the methodology category of the activity sequence, e.g. knowledge-acquisition, problem-based, project-based, informative testing, etc. Service

W. Pan and I. Hawryszkiewycz / Facilitating Knowledge Construction

609

options declare the tools that are required in the learning sequence, e.g. discussion forums, etc. Traits indicate the particular features of the sequence. (5) Assessments field is to describe all the assessment approaches for the learning of the UOL. Within the description of each assessment approach, Type specifies its category, e.g. question-answering, interactive program, etc.; Traits specify the specific features of the assessment approach; and Source indicates where to find it. (6) Cases field is to describe all the case study materials related to the UOL for scaffolding the learning for the UOL. Within the description of each case study material, Type specifies its category, e.g. document material, interactive program, etc.; Traits specify the specific features of the case study material; and Source specifies where to find it. (7) Plans field is to link a learning method with its targeted learner category. It contains a series of Plan fields, each of which is a pair of a Property ref and an Activity structure ref. The former refers to an ID of the Property defined in Roles whereas the latter refers to an ID of the Activity structure defined in Methods. The field is crucial to dynamically organize a specific learning mode for individual learners and provide associated supportive services for different learning needs. 3.3 Features of LPSL and comparisons with EML As seen while comparing LPSL with EML, the central core of EML is preserved in LPSL and many of its ideas and concepts are also accepted by LPSL. However, LPSL has some specific features that differentiate it from other specification languages, particularly EML and IMS LD. Most of them are to support personalized learning and constructivist learning. First of all, LPSL has different goal with EML/IMS. The latter aim to support the reuse of learning designs [1]. LPSL is aimed to describe the content and process within UOLs from constructivist perspectives of learning so that knowledge construction can be supported and facilitated by supportive services in an individual basis. The centre of EML is on reuse and interoperability of the content and process within UOLs; whereas the centre of LPSL is on the services that support and facilitate learning. The two different goals result in the distinctions between LPSL and EML/IMS in several aspects. The most obvious one is that LPSL is much simpler than EML/IMS. There are substantial differences between LPSL and EML/IMS. The most important two are: x Using LPSL, it is possible to specify multiple learning routes for learners that suit their learning characteristics in an Activity structure through its Activity sequence field. This enables agents to provide learners with multiple optional learning methods, rather than imposing them to accept any particular one. Using EML, however, only one learning method can be specified for a particular category of learners. x In a UOL described using LPSL, each Activity sequence contains a Traits field to indicate its particular features, and each Learning resource contains a Traits field to specify its specific features. This enables the supportive services to be provided for learners in a more personalized way because agents can match them with individual learners to provide the most suitable learning methods and resources. In a UOL described using EML, no such a closed association can be provided. 4. The Implementation Based on the research, a prototype of the ASLS has been developed. To this end, several services have been successfully implemented. The implementation for those relating to the learning process of individual learners, including assisting them to develop personal preferred learning plans, assisting them to get through a learning process, etc., has been reported in [7]. In general, all the supportive services are dynamically generated according

610

W. Pan and I. Hawryszkiewycz / Facilitating Knowledge Construction

to the rules stored in the UOL database and based on the practical learning scene and the learner’s profile. The profile of a learner is characterized by the following three dimensions: 1) the cognition level; 2) the learning style and 3) the preference for learning resources. The cognitive skills are categorized using Bloom’s taxonomy, i.e., knowledge, comprehension, application, analysis, synthesis and evaluation. As an example, the agents assist individual learners to develop personal preferred learning plans to reach their desired learning goal through advising them of several plans for the goal, which include the relevant learning activities to be taken and their conduct sequences. The process for the agents to implement the service for an individual learner can be sketched out as follows. When detecting a learner has set up a learning goal, the agents first retrieve the UOL database to extract a UOL record by matching its objective to the learner's goal. Then all the learning routes defined in the UOL are extracted. Those are all possible learning routes for the UOL; only some suit the learner and some may not suit the learner. Accordingly, the agents next select the suitable ones from those based on the learner's learning characteristics. At the first, the ones that suit the learner in terms of the cognitive level are chosen, and then the remaining ones are further filtered based on the learner's learning styles. Only the learning methods whose Traits mostly match to the learner’s learning styles are left. Finally these learning methods are presented to the learner. All the other services are implemented using a similar way as the example; the agents generate individualized services for learners by using the rules stored in the UOL database and based on the learning activities being conducted and learner’s profile. Due to the limit of space, it is not possible to present a detailed description of the implementation at this paper. 5. Summary The paper proposed an agent-based way to facilitating individual learner’s knowledge construction through a wide range of customized supportive services. It presented a learning process specification language we have developed to describe the rules that software agents use to dynamically organize and generate individualized supportive services based on the practical needs of learners in learning. References [1] Britain, S. (2004) A Review of Learning Design: Concept, Specifications and Tools, A report for the JISC E-learning Pedagogy Programme. Available at: http://www.jisc.ac.uk/uploaded_documents/Review Learning Design.doc (25/08/2004) [2] Cristea, A. (2005) Adaptive Hypermedia Technology: What can it add to Technology-Based Education? The 8th IASTED International Conference on Computers and Advanced Technology in Education (CATE 2005), Oranjestad, Aruba, Aug 29-31. [3] IMS. (2003) IMS Learning Design Specification V1.0. Available at: http://www.imsglobal.org/learningdesign/index.cfm (10/07/2004) [4] Jonassen, D. (2000) Computer as Mindtools for Schools: Engaging critical thinking. Columbus, OH: Prentice-Hall. [5] Jonassen, D. (1999) Designing Constructivist Learning Environments. In C. M. Reigeluth (Ed.), Instructional Design Theories and Models: a New Paradigm of Instructional Theory. MahWah: Lawrence Erlbaum Associates, Publishers, Vol. II. pp. 215-240. [6] Koper, R. (2001) Modeling units of study from a pedagogical perspective: the pedagogical model behind EML. Available at: http://eml.ou.nl/introduction/docs/ped-metamodel.pdf (10/12/2003) [7] Pan, W. and Hawryszkiewycz, I. (2006) Assisting Learners to Dynamically Adjust Learning Processes by Software Agents. International Journal of Intelligent Information Technologies, 2(2). pp. 1-15. [8] Pan, W. and Hawryszkiewycz, I. (2004) A method of defining learning processes. In R. Atkinson, C. McBeath, D. Jonas-Dwyer and R. Phillips (Ed.), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference. Perth, Australia, Dec 5-8. pp. 734-742. [9] Wilson, B. G. (1996) Constructivist Learning Environments: Case Studies in Instructional Design. New Jersey: Educational Technology Publications.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

611

The Role of On-line Facilitators: Types of Collaborative Skills for Effective E-learning Activities Tengku Putri Norishah Tengku Shariman, Habibah Abdul Jalil Faculty of Creative Multimedia, Multimedia University, Malaysia Faculty of Education, University Putra Malaysia, Malaysia [email protected] Abstract: Learning is an active process of acquiring knowledge and skills that humans perform quite naturally, but not all learners are equally capable of effective and efficient learning on their own. Indeed, most will benefit from some level of guidance and support. Collaborative learning encourage discussion among learners and support learners to develop critical thinking skills. It is not, however, guaranteed that such high quality discourse will occur between learners, especially in a computer supported environment. Therefore, this research seeks to identify and examine the role of online facilitators in motivating learners to share and exchange knowledge during computer supported collaborative tasks. This study, based on socio-cultural methods of discourse analysis describes the type of collaborative skills practiced by instructors and peers (who are both considered online facilitators) during computer-mediated communication. Key words: Computer supported collaborative learning, social cultural theory, computer mediated communication, online facilitators, learning community, activity theory

Introduction Computer-supported collaborative learning (CSCL) is increasingly being used a teaching and learning methodology in the e-learning mode. Collaborative learning is defined by Dillenbourg (1999) as “partners doing the work ‘together.’ ” (p.8). He further explains that ‘together’ means “individuals negotiate and share meanings that are relevant to the problem-solving task at hand.” (p.70) As such, CSCL researchers believe that context is essential to many of the theoretical frameworks that guide CSCL practices, such as situated cognition, which is guided by the notion that “…learning, cognition, knowing, and context are irreducibly co-constituted and cannot be treated as isolated entities or processes.” (Barab & Squire, 2004, p. 1). This paper focus on the types of collaborative skills employed by online facilitators (both instructors and peers) through their exchange and exploration of ideas and concepts in a computer mediated communication environment (CMC). 1. Theoretical Framework Globally, universities are adopting various modes of delivery systems for teaching and learning: the adjunct mode, the mixed mode, or the total online mode for teaching and learning purposes (Harasim et. al., 1999). Regardless of the mode of delivery, one of the main aims of universities has remained unchanged, to produce learners who are capable of thinking critically and creatively at a higher level. In Malaysia, universities are also striving to fulfill the Government’s vision of creating a knowledgeable society, and to do

612

T.P.N. Tengku Shariman and H. Abdul Jalil / The Role of On-Line Facilitators

this, true education must go beyond the mere access to information, and involve students in an “engagement with others” to construct and apply knowledge. (Laurillard, 1993) 1.1 Collaborative Learning Collaborative learning requires learners to work together on academic tasks via active interactions. An essential pre-requisite for collaborative learning is the need for reciprocal understanding between collaborators. Such reciprocity involves individuals’ establishing, through the negotiation of meaning, a mutually shared or “common” knowledge (Teasley and Rochelle, 1993). Involvement in this process allows learners to elaborate and reflect upon their knowledge, thereby improving learning and understanding. From the viewpoints stated above, we could say that one of the positive impacts of collaborative learning is the process of interactive knowledge construction in which appropriation of meaning through negotiation plays a central role. 1.2 Social Cultural Theory Dillenbourg and Schneider (1995) believe some factors determine the effectiveness of collaborative learning. The first factor is the social aspect, whereupon students are engaged in establishing relationship roles such as trust-building and conflict-management. The second factor is scaffolding, and the underlying theory is provided by the social-cultural theories of Vygotsky and neo-Vygotskian researchers like Gallimore and Tharpe (1990); and Wertsch (1985, 1991). The role of the tutor and peers is discussed in terms of the Vygotskian conception of the Zone of Proximal Development (ZPD) whereby the process of interactions will include scaffolding. The primary role of the tutor is more a ‘guide on the side,’ whose aim is to encourage self-regulation or autonomous critical thinking by students. The third important factor is the occurrence of a triggering event, or cognitive task. Higher education emphasize the development of critical and creative thinking skills, and to assess if this is achieved, a task which demonstrates an “appreciation of, and commitment to, the value of thinking progressively through an issue or problem, towards a shared resolution” (McManus & Aiken, 1995) should be formulated and applied. In collaborative learning, tasks are structured so that students are dependent upon one another for their personal and group's success to complete an assigned activity and the specified learning outcome. 1.3 Learning Community In traditional communities, interaction arises because of mutual interests, and occurs because members share a physical space, at a specific time. Likewise, in e-learning, CMC acts as the mediating tool to create a learning community -- where social interactions within the community is directed by a structure of collaboration, (Harasim, et. al., 1997) through which people can have discussions and form personal relationships in cyberspace. Simply put, this means that members of a community need not be physically together to become a community. Within this framework of a learning community, the Community of Inquiry Model was developed by Archer, Garrison, Anderson, and Rourke in 2001, to illustrate the interplay of three core elements: social presence, cognitive presence, and teaching presence. Interaction within this community is directed by a structured task, and this result in critical thinking. 1.4 Activity Theory

The theory, known as the Activity theory, spans from the idea put forth by Vygotsky (1962; 1987) that human actions are directed at objects and mediated by artefacts. Hence, the three

613

T.P.N. Tengku Shariman and H. Abdul Jalil / The Role of On-Line Facilitators

form a unit of analysis in understanding cognition and learning. Engestrom (1999) and Cole and Engestrom (1993) elaborated these ideas further to include the community in which the subject (acting agents) operate, the outcomes of the activity, the rules which define the subjects’ relationship within the community, and the division of labour among subjects. Collaborative learning can be regarded as an activity system which encourages learning through mechanisms such as externalizing knowledge and opinions, self explanation, reflecting on each other’s information and reconstructing knowledge through critical discussion. For this purpose, the tutor mediates the collaborative task by scaffolding, thus enabling students to deal with more challenging tasks than they could otherwise handle. Garrison and Anderson (2003) define this scaffolding presence as “the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes” (p.29). Figure 1 below represents the structure of an activity system model for CSCL. Subject: A participant of the collaborative task. As a member of a community, the subject seeks to satisfy the object of the task.

Rules: Conventions of social interaction

Production

Exchange

Object: Collective knowledge production using higher order thinking skills

Distribution

Community: Members of a group socializing as a community

Division of labour: Division based on role type for subject

L E A R N I N G C O M M U N I T Y

[adapted from: Engestrom (1999)]

Research Methodology 2.1 Research Questions This study follows a view of learning as a joint process, mediated by computer-based artefacts, in which participants of a community socially interact with one another to create shared knowledge. Within this view, the aim of the study is to: “Explore and gain understanding of computer supported collaborative learning, during which social interaction and shared learning occur through computer mediated communication.” Associated with this aim, the study seeks to answer the following research questions:

2.2

Do participants create shared knowledge when they interact during CSCL task?

What types of skills are employed by participants during CSCL to foster interactions and exchange of knowledge? Do the levels of skills used vary according to the nature of the CSCL task? How does the creation of shared knowledge occur through time in the CSCL task? Research Context

The data for this study is the asynchronous messages contributed by thirty nine in service teachers during a Masters in Education postgraduate course. The CSCL tasks served to continue or extend the discussions held during face-to-face sessions, and are monitored and

614

T.P.N. Tengku Shariman and H. Abdul Jalil / The Role of On-Line Facilitators

assisted by a tutor and peers. Each group shared the same lecturer who would initially facilitate the discussion. The messages gathered for this study were taken from a series of discussions of three groups of students. The discussions of each group differ in terms of the nature of the initiating CSCL task given by the lecturer in the electronic board. The lecturer has fixed certain cognitive tasks for the students to discuss and collaborate together, in order to achieve specific goals at the end of the task. The first task, which is the Jigsaw task, defined the role of each group member according to information sets. Each group member had only access to a subset of the information necessary to solve a problem. Therefore, no individual can solve the problem alone. The second task, the thematic sheet, involved the creation of a few fact sheets that describe specific themes in educational technology. The group members then discussed the differences and the similarities between the facts sheets, and then proposed a strategy (objectives, actions, and resources) on how to cope with a problem, based on the themes studied. The third task, the debate, created conflict among members and engaged them in interactions to resolve the conflict. The task required members to analyze and argue on a few proposals and then, as a group, they had to choose one proposal that would best resolve the conflict. All of the tasks above were adapted from the CSCL scripts proposed by Koschmann and his team during the CSCL conference held in California in 1999. CSCL script creation is currently one of main research areas of the CSCL community. 2.3 Research Instrument In order to probe the collaborative learning tasks, participants’ messages will be coded using a coding schema to categorize the contributed messages according to the type of collaborative skill. Content analysis of these messages will enable the generalization of the complexity of textual messages, in order to identify the types of collaborative skills and interaction patterns, demonstrated by learners and tutors, during the collaborative learning tasks at different phases. All messages sent through asynchronous communication in the ‘Electronic Board’ in the three CSCL tasks were examined. We use the following definitions of categories of collaborative skills to analyze the message transactions in CSCL, adapted from the work of Gallimore and Tharp (1990) and McManus, M. & Aiken, R. (1995) on electronic interactions. Table 1: Categories of Computer Supported Collaborative Learning Skills CSCL Phases

CSCL Skill

Example of Messages

Initial Phase

1. Coordination: To initiate the discussion or task; explain and clarify tasks , and present required information for coordination purposes

-“This semester’s project will discuss….”

2. Information seeking: To clarify and determine course of action (content, time and roles), questions to trigger the discussion and related to the direct instruction given by the tutor

- “How many points do you want us to come up with?”

Exploration Phase

3. Information sharing: To impart facts and information, exchange notes and discuss individual ideas for in-depth clarification of statements

-“Everyone should at least post 2 messages by every Saturday.”

-“Can I use the Government Smart School Blueprint as a guideline?” - “To date, Internet and mobile technologies usage has increased substantially in major cities.” -“I attended the National ICT in education seminar last year and Prof. Ungku Aziz stressed on……….You can read more in the proceedings available at ………”

T.P.N. Tengku Shariman and H. Abdul Jalil / The Role of On-Line Facilitators

4. Negotiation of meaning: To establish and maintain a shared understanding of meaning

615

-“The cultural element has been well discussed by others in this thread, but I’d like to ask, even if it may seem like an old issue, if the government’s mismanagement really doesn’t play a part in this discussion?” -“The situations you gave are probably just one of the countless cases of white elephants. What do we consider ‘white elephants’ in the education system?”

Integration Phase

5. Reflection: To critically evaluate theories, search for deepening knowledge and generate subordinate questions in order to argue and explain individual solutions

-“I just worry about the educational future of my grandchildren if there is no change in the education system. We need to change the system by………..”

6. Knowledge construction: A coordinated effort to solve the problem together, synthesize, edit and integrate input and ensure consistency of the joint work product

-“I think we should discuss why all of us are quite confused on the direction of this discussion.” -“Bear in mind that Datuk Hishamuddin was speaking from a political perspective, so we should think again about our suggestion that…..” -“To summarize, these are important points which we need to highlight in the report………”

7. Shared Solution: Mutual engagement to formulate and propose a finalized solution with supporting arguments (justification) Assistance given during all phases

2.4

-“I agree with your emphasis on research, but I think we should not include it in the report because……..”

“The original question was ………..and we have finally concluded that……”

8. Scaffolding: To help, guide, and -“Siew Ling, would you like to respond to provide recommendations, that may help Noor’s counterargument to your proposal?” the learner to master the material and move -“Prof. Awang Had Salleh said…….. about to a higher level of learning 10 years ago, and what do you think of his opinion today?” 9. Feedback on performance: To give feedback on specific acts, performance or situation (positive or negative), or acknowledge a contribution in reference to a given set of criteria

-“Looking forward to your findings!!! They are always provocative!”

10. Social interactions: Statements not related to formal content, but provides indirect motivation to encourage discussion; jokes, trust building communication

-“Hi Lim!!!! ☺

-“When you post comments, please remember that there are people of different races in this group, so ….”

- “Everyone….let’s wish Rosmah HAPPY BIRTHDAY.

Mode of Analysis

We used content analysis to identify the existence and frequency of collaborative skills employed by online facilitators in all three CSCL tasks. The steps taken to code the messages are as follows: Step 1: Categorization of existence of collaborative skill. The messages were reviewed to identify the existence of major categories of collaborative skills in terms of the occurrences of meaning in every message.

616

T.P.N. Tengku Shariman and H. Abdul Jalil / The Role of On-Line Facilitators

Step 2: Frequency of collaborative skill. The texts were reviewed to identify frequency counts of each collaborative skill employed i.e. the number of times each category appears in the messages given either by the tutors or peers. Step 3: Mapping of frequency to task type. Number of messages that show the type of collaborative skill used were also counted for comparison between the three types of tasks. These numbers of messages were also used so that they could be compared to the total number of messages in the discussion. 3. Results of Study This study involved a total of forty participants consisting of thirty nine students and one tutor. The participants were grouped according to the type of collaborative task. For the purpose of this research, a total of ten types of collaborative skills were identified. From this, a total of 1,054 messages were analyzed. Out of 1,054 messages, 432 (41%) included collaborative skills contributed during the main (second) phase, 143 (13.6%) during the first phase and the remaining 180 (17.1%) during the final phase. Scaffolding, feedback and social interactions occurred during all phases, 299 (28.3%). Table 2 below represents the number of occurrences of each form of collaborative skill. We found that the most common form of collaborative skill was scaffolding (15.2%). This was followed by information sharing (14.9%), negotiation of meaning (14.7%) and shared solution (11.6%). The types of collaborative skills less used were knowledge construction (5.5%) and feedback on performance (6.0%). From the table, it can also be seen that scaffolding was the most frequently type of assistance given by the participants compared to other types of assistance, where 160 out of 298 incidents of assistance were considered scaffolding. Table 2: Forms of computer supported collaborative skill according to task type No.

Type of CSCL Skill

CSCL Task 1

CSCL Task 2

CSCL Task 3

(%)

1.

Coordination

22

28

18

6.4

2.

Information seeking

26

30

20

7.2

3.

Information Sharing

59

34

64

14.9

4.

Negotiation of Meaning

56

47

52

14.7

5.

Reflection

37

33

50

11.4

6.

Knowledge Construction

19

15

24

5.5

7.

Shared Solution

42

35

45

11.6

8.

Scaffolding

53

55

52

15.2

9.

Feedback on performance

17

25

21

6.0

10

Social Interactions

24

28

23

7.1

Total

355 (33.7)

330 (31.3)

369 (35)

100.00

Table 3 indicates that there is a greater number of collaborative messages at phase two level for all task type. However, the number of messages with assistance is highest for the second task type (108 messages) compared to the other two task type. The second task type also required more explanation, guidance and information (58 messages during phase one). Finally, the third task type had the highest frequency of messages contributed and exchanged by participants (369), and especially during phase two when participants were

617

T.P.N. Tengku Shariman and H. Abdul Jalil / The Role of On-Line Facilitators

busily sharing information, arguing and reflecting on the task at hand. From the table, it can also be seen that assistance were more frequently given by tutors for all CSCL tasks, except for the third task, where more assistance were given by peers. Table 3: Comparison of frequency of messages between CSCL tasks (S=Student; T=Tutor)

No.

Mode of Discussion

Collaborative messages at phase one

Collaborative messages at phase two

Collaborative messages at phase three

Assistance messages at all levels

Total

Online Facilitator

S

T

S

T

S

T

S

T

1.

Task One

28

20

129

23

42

19

45

49

355

2.

Task Two

27

31

85

29

24

26

30

78

330

3.

Task Three

14

24

148

18

53

16

55

41

369

To ensure reliability, two coders independently coded the online postings and their analysis were compared to identify places of agreement and disagreement. Two types of inter-coder reliability co-efficient were calculated: Holsti’s CR and Cohen’s Kappa. As shown in the table below, the means for of inter-coder reliability measures were 0.86 for Holsti’s CR and 0.90 for Cohen’s Kappa, and these figures were satisfactory considering the complexity of the content analysis method using coding systems. Table 4: Inter-coder reliability Type of inter-coder reliability coefficient

CSCL Task 1

CSCL Task II

CSCL Task III

Total

Holsti’s CR

.87

.87

.85

.86

Cohen’s Kappa

.90

.92

.89

.90

4. Discussion One of the main findings of this study is the picture it paints of interactions during CSCL; CSCL is a rich and delicate undertaking, because participants have to create an interactive climate that stimulates the process of active learning, which is the sharing, analysis and development of knowledge, and at the same time, participants are expected to maintain the supportive structures required for learning. It requires a lot of trust and sensitivity, on the part of the learners and especially, the tutors, in order to build a subtle support framework for the learning community. Assistance in the form of scaffolding, feedback on performance and social interactions are important components of this framework. Scaffolding was the most popular one among participants due its direct approach, which was expected from the tutors by students. The results also suggest that students were able to discuss actively once they have understood the task (during phase two of the CSCL task). However, it seems that they were not as successful during the third phase of the CSCL task which required higher order thinking skills. In fact, the least form of collaborative skill used is knowledge construction (5.5%). This could be blamed on the nature of the electronic board that does not promote deeper levels of discussion. It does not provide the advantages of elaborated explanation and immediate feedback -- on any misunderstandings -- that face-to-face interactions could offer

618

T.P.N. Tengku Shariman and H. Abdul Jalil / The Role of On-Line Facilitators

to participants (Brookfield and Preskill, 1999). There is evidence here of collaboration in this research because learners were assisting one another while exchanging ideas and knowledge, and eventually, they were able to produce a shared solution. However, the tutor remains the main source of learning support. Moreover, while both tutors and students offered assistance, with students doing less so, most assistance are in the form of scaffolding. What is of concern is that the more complex forms of assistance, like feedback on performance and contribution, is very minimal. This suggest, at least in this context, that since the assistance offered tends to be simple in nature this could be a reflection of the fact that students find it difficult to agree on a shared solution. Further research in the next phase of this study will generate more conclusive evidence on other factors that also motivate learners to collaborate, including features of the CMC tool, and the rules and conventions of the CSCL task. The use of CSCL, particularly in the online distance learning context, provides a good example of how collaborative learning can used to create a learning community even though the students are separated by time and distance. This study show that the ‘role of tutors as facilitators’ and the ‘social interaction of learners’ are the most important elements to create a learning community. However, the ‘online collaborative learning task’ was necessary to provide a framework or educational context for the community to construct meaning through sustained communication. References [1] Barab, S.A. & Squire, K.D. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences.

[2] Brookfield, S.D., & Preskill, S. (1999). Discussion as a Way of Teaching: Tools and Techniques for University Teachers. Buckingham: Society for Research into Higher Education & Open University Press. [3] Cole, M. & Engestrom, Y. (1993). A cultural historical approach to distributed cognition in G. Salomon (Ed), Distributed Cognitions: Psychological and Educational Considerations. New York: Cambridge University Press. [4] Dillenbourg, P. (1999). What do you mean by ‘collaborative learning’? In P. Dillenbourg (Ed.), Collaborative Learning: Cognitive and Computational Approaches, p. 1-16. Amsterdam, NL: Pergamon, Elsevier Science. [5] Dillenbourg, P. & Schneider, D. (1995). The Mechanisms of Collaborative Learning. Switzerland: University of Geneva. [6] Gallimore, R., & Tharpe, R. (1990). Teaching mind in society: Teaching, schooling, and literate discourse. In L.C. Moll. (Ed.). Vygotsky in Education: Instructional Implications of Sociohistorical Psychology (p.175-205). New York: Cambridge University Press. [7] Garrison, D.R., Anderson, T. (2003). E-learning in the 21st Century: A Framework for Research and Practice. New York: Routledge Falmer. [8] Harasim, L., Hiltz, S.R., Teles, L. & Turoff, M. (1997). Learning Networks: A Field Guide to Teaching and Learning Online. The MIT Press, Cambridge. [9] Koschmann, T. (1999). Toward a Dialogic Theory of Learning: CSCL Scripts Contribution to Understanding Learning in Settings of Collaboration. Paper presented at the CSCL1999 Conference, Palo Alto. [10] Laurillard, Diana (1993. Rethinking University Teaching. London: Oxford Press. [11] McManus, M. M. and Aiken, R.M. (1995). Monitoring Computer Based Collaborative Problem Solving, Journal of Artificial Intelligence in Education , 6(4) , 308-336. [12] Rourke, L., Anderson, T., Garrison, D.R., and Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12, 8-22. [13] Teasley, S., & Rochelle, J. (1993) Constructing a joint problem space: The computer as a tool for sharing knowledge. In S.P. Lajoie, & S.J. Derry (Eds.) Computers as Cognitive Tools (pp. 229-257). Hillsdale, N.J. : Lawrence Erlbaum Associates. [14] Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA.: Harvard University Press. [15] Wertsch, J.V. (1985). Vygotsky and the Social Formation of the Mind. Cambridge, MA: Harvard University Press.

PBL and Test

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

621

Fostering Project-based, Active Learning through Use of Technology Teresa BADER Lead Instructional Technology Specialist, Region #4, New York City Department of Education, USA [email protected] Teh-yuan WAN, Ph. D. Coordinator, Office of Education Technology Programs, New York State Department of Education, USA [email protected] Abstract: In this paper, we present a successful, systemic, and sustainable approach to integrating technology into the K-12 teaching and learning environment. We will discuss obstacles and causes for failure that impeded previous attempts. We will share the vision, methodology and results that have had such a significant impact in New York City classrooms. Teachers have improved their content knowledge, technology and information literacy skills, perceptions about technology, and the use of inquiry to promote understanding. Students are better prepared to access and manage print and electronic information, think critically, solve problems, work on teams, and communicate and collaborate with students around the world. Keywords: Professional Development, information literacy, inquiry, student performance

Introduction After more that twenty years, technology is finally making inroads into the last major vestige of American civilization: Education. Historically, the educational structure evident today was conceived as an organized effort to prepare our youth to be good people and good citizens, by recognizing and developing their innate talents so that they could grow up, get meaningful jobs, and assume their roles as contributing members of society. Today’s world requires another dimension be added to this formula: the development of the skilled researcher, the critical thinker, the problem solver, and the global communicator and collaborator. 1. The Need to Change 1.1 An American Imperative The American government and the business community both consider technology use an undisputed prerequisite for survival in this global economy. In education, there has been no sense of competition, no accountability for success or failure, and no compelling motivation to change. Accepted practices in education were all based on the assumption that the content and context that was taught fifty years ago continues to be relevant and appropriate for today’s generation. Educators can no longer ignore the cadence from

622

T. Bader and T.-y. Wan / Fostering Project-Based, Active Learning Through Use of Technology

businesses, who need better prepared workers, parents, who want their children to be well-informed, competitive and successful, and students, who have to ‘disconnect’ from the real world when they enter their classrooms. 1.2 Causes for previous failures Efforts to provide teachers with the needed technology skills have been unsuccessful for many reasons. Teachers need to develop several competencies in order to make the connections between good content, pedagogies that support understanding and retention, classroom management strategies, and appropriate use of hardware, peripherals, content and tool software. Attempts to focus on any one of these in isolation have been unsuccessful because it is in making the connections between them that teachers get the ‘aha’ moment. Unlike commercial enterprises that encourage a team approach to meeting challenges, most educators plan and teach without the benefit of collaboration with their peers. Administrators were ill-prepared to evaluate the integration of technology and were uncomfortable promoting something they themselves did not understand. Expensive, unreliable computers, complicated software applications, intermittent Internet access, poor fix and repair policies, and trainings that failed to make the connection to what teachers were doing in the classroom, further hindered well-intentioned efforts. 1.3 Education Reforms in New York City In the fall of 2002, the New York City Department of Education came under the jurisdiction of the city mayor, who immediately implemented Phase I of his Restructuring Plan.. Some of the first goals of this initiative were to implement a unified K-12 curriculum, introduce data-driven instruction, and increase accountability at all levels. At about the same time, one of the mandates of the federal No Child Left Behind legislation emphasized the importance of schools and classrooms having equitable access to technology to use as an administrative, teaching, and learning tool. Included in the NCLB legislation was the Title IID Enhancing Education through Technology grant proposal. 2. Region 4 Responds 2.1 The Enhancing Education through Technology Grant Proposal In the fall of 2003, in response to this grant opportunity, the Instructional Technology leadership of one of those regions, Region 4, proposed a new vision for effectively using technology to drive instruction. Since teachers in each grade were finally following the same curriculum, an inquiry project for each grade was designed to support that curriculum and became the vehicle for improving teacher content knowledge, modeling effective instructional practice, and demonstrating the effective use of technology. The leadership examined the performance standards for all content areas and developed several grade specific, curriculum-based, inquiry projects that scaffold nonfiction literacy, information literacy, higher order thinking, problem solving, team-building, and technology skills. All the projects contained adaptations for special education or English Language Learners. These interdisciplinary projects were designed to provide exemplar models of effective technology integration and to improve Literacy and Math test scores. In the spring of 2004, Region 4 was awarded a three year, fifteen million dollar grant, which provided the impetus and the funding to support the changes that followed.

T. Bader and T.-y. Wan / Fostering Project-Based, Active Learning Through Use of Technology

623

2.2 Overcoming Obstacles Despite the obvious obstacles that plague urban school districts, Region 4 is committed to providing all students the opportunity to become self-directed learners, who meet the National Educational Technology Standards, possess a true global work ethic, and are adequately prepared to compete and succeed in the twenty-first century. Educating approximately 105,000 students in 110 Pre-K-12 schools, Region 4 is home to the largest English Language Learner population in New York City, which is the largest school system in the United States. It services families speaking over 108 languages. Many, if not most of the families, live at or below the poverty level. Ongoing job embedded professional development has been the catalyst for ensuring effective implementation, minimizing resistance, and providing hands-on support to struggling teachers. Each school received a rolling laptop cart, printer, scanner and camera and a Technology Integration Specialist, who provided weekly in-class support for teachers throughout the first year of every project implementation. As the staff developers moved up to work in the following grade, they still provided ‘just in time’ support to the previous years’ teachers. Additionally, all project resources were posted online and teachers were provided a variety of online forums, to share best practices, post questions and/or suggestions, discuss educational trends, or examine samples of student work. Over the course of the three years of the grant, internationally known educators, who specialized in curriculum, instructional practice, information literacy, and technology integration provided presentations and hands-on workshops for teachers and administrators. Their work and research were incorporated into the development of the grade projects and they have become major stakeholders in the success of our implementation. 3. Outcomes 3.1 Student Performance After two years of close scrutiny, there is abundant evidence that the introduction of these projects has indeed been the impetus for achieving significant student gains. They have fostered critical thinking and problem solving skills, team work, international collaborations and the mutual exchange of ideas, deeper understandings of content knowledge, and improvement in teacher and student technology competencies. Students are transferring and applying their knowledge and skills to independently extend the use of technology throughout their work. Students with access to technology input information from a variety of sources, visual, auditory, or kinesthetic, and now have the ability to demonstrate their competencies via a multitude of media that best suit their learning style. High school students, who are social by nature, became actively involved in their learning and eagerly embraced the online collaborative learning environments and the sharing of scholarship. Students independently explore topics of interest and have become self-directed learners. Test scores continue to increase and many of the schools that were initially cited for poor performance have been removed from the list of schools in need of improvement. 3.2 Changes in Teacher Practice However, the most significant impact of this effort has been the widespread improvement in teachers’ technology skills, their attitudes toward technology integration, their ability to use inquiry to foster understanding, and the overall transformation of their

624

T. Bader and T.-y. Wan / Fostering Project-Based, Active Learning Through Use of Technology

instructional practice. At the conclusion of the projects, it is not uncommon for teachers to say “I didn’t think my students could do this level of work.” Having a universal grade level implementation provided techno-phobic teachers a degree of comfort and security by using familiar content and limiting their need to master only those technology skills required to implement their grades’ project. They recognized the value of using technology, especially multimedia, to provide differentiated instruction for their students. The results in student performance, the improved level of student engagement, and in-class access to rich online resources are such compelling indicators that teachers are finally clamoring for more access to technology. 3.3 Leaders Are Key Observers to the classrooms where inquiry projects are being implemented immediately see a different kind of instructional environment, where students, not the teachers, are driving the learning. Having spent hours beforehand setting the stage and providing the resources for their students, teachers in these classrooms quietly function in the background supporting and encouraging, asking good questions that stimulate thought, and providing multiple opportunities for deeper, more creative, critical thinking. As the instructional leaders in their school, principals received check lists for each project, describing what they should see and hear when they enter the classroom and what questions they could ask of the teachers and students. Schools, where the principal endorsed the projects, were the most successful and achieved the highest gains in performance 4. Conclusion As a direct result of this project, teachers have transitioned from a ‘sage on the stage’ to a ‘guide on the side’ and technology-rich classrooms are now student-centered rather than teacher-centered. Improved nonfiction literacy and technology skills, students’ abilities to manage and interpret data and effectively collaborate on a team, make this model worth emulating. If we are to successfully prepare out students to be self-directed learners capable of competing and succeeding in this global economy then schools need to embrace the concepts of project based learning and provide our students with the technology skills and access that they need to succeed. Project descriptions and an invitation to collaborate can be viewed at http://www.region4.nycenet.edu/instruction/projects/. References [1] Stephanie Harvey, Nonfiction Matters, Reading, Writing, and Research in Grades 3-8.Stenhouse Publishers, 1998. [2] Alan November, Empowering Students with Technology. LessonLab Skylight Publishers, 2001. [3] Dr. Christopher Tan, Tan, Y.G.C & Kwok, P. (2005). Knowledge Building in Inter-school Learning Communities: Reflections from a Case on Project Learning in Hong Kong. ICCE 2005 Proceeding (Short Paper), Singapore. [6] Dr. Alan Ludman, http://qcpages.qc.cuny.edu/qcglobe/index.htm GLOBE (Global Learning and Observations to Benefit the Environment) [7] Sydney Thornbury, WebPlay http://www.webplay.org/ [8] Dr. Bernie Dodge, WebQuests: A structure for active learning on the World Wide Web. The Distance Educator, (1995).1(2). http://webquest.sdsu.edu/webquest.html [9] Will Richardson, Blogs, Wikis, Podcasts, and Other Powerful Web Tools for Classrooms Corwin Press 2006 http://www.weblogg-ed.com/ [10] Marco Torres. http://www.edutopia.org/php/interview.php?id=Art_994

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

625

Framework for Problem-Solving Based Learning in Nursing Domain – A Practical Study – Yukie Majimaa, Yoichiro Sob and Kazuhisa Setac School of Nursing, Osaka Prefecture University, Japan b Production Systems Research Lab. KOBELCO, Japan c Graduate School of Science, Osaka Prefecture University, Japan [email protected], [email protected], [email protected] a

Abstract: This paper describes a summary and learning effects of e-learning materials based on nursing practice examples for the purpose of cultivating problem-solving ability of nursing students. The materials have a structure that incorporates the relationship between imaging promotion and knowledge that has already been learned. For one subject, learning is carried on in accordance with the following procedures: 1. “Grasp” the subject’s condition. 2. “Analyze” nursing problems that are faced. 3. “Confirm the results” that were analyzed. We put the materials to practical use in whole class teaching and conducted surveys of nursing students. Results indicated remarkable knowledge: it could promote imaging of subjects and nursing problem scenes, it was effective for positive motivation for learning and it makes learners’ self-evaluation criteria get sticker by imaging the real situation. Keywords: problem-solving based learning, nursing education, meta-cognition, imaging, problem-solving ability

1. Introduction In nursing education or learning before going out in the medical sceneΔ practical nursing abilities cannot be obtained merely through mastery of knowledge and nursing skills. In daily learning opportunities, implementation of various simulated problem-solving scenes is required to induce students to put acquired knowledge to practical use in individual nursing practice situations and to thereby solve problems. In contrast, constructing knowledge and skills through trial and error in actual clinical practice might cause suffering to patients, unlike learning with normal coursework. Therefore, educators need to provide opportunities for learners to put knowledge into practice and to exercise skills according to their developmental stages by properly setting up pseudo practices or clinical practices. In this paper, for cultivating nursing problem-solving type thinking, we discuss e-learning materials with a structure to obtain educational effects in terms of “portrayal of a subject’s condition” and “organizing, structuring and associating information”. Furthermore, we report the following: After putting e-learning materials developed in this study to practical use in whole class teaching and carrying out formative evaluation using questionnaires given to nursing students, inference of a subject’s condition was promoted. Moreover, effectiveness for motivation for learning was indicated.

626

Y. Majima et al. / Framework for Problem-Solving Based Learning in Nursing Domain

2. Design Principe of the System We have developed an e-learning framework as shown Fig. 1. It is important to make the design principle of the system explicit for clarifying the evaluation viewpoints as well as continuous improvement of the system. In this section, we describe the design principle and overview our system. We emphasize to realize the learning framework that can enhance the learner to imagine the nursing scenes and oneself as a nursing practitioner and a learner. Thereby, we adopted the problem-based learning style to grow up the ability of breaking down a patient’s condition and nursing practice scenes, breaking down the present situation and self in the future as a nursing practitioner and objectively grasping oneself as a learner based on self-imaging as a nursing practitioner. We aim at providing opportunities for learners to construct knowledge and skills in their respective contexts, along with imaging of nursing practices. We have developed learning material frameworks for promoting imaging of nursing scenes, self-imaging as a nursing practitioner, and self-imaging as a learner, as in the following [1]: 1) Imaging of nursing scenes: Breaking down a patient’s condition and nursing practice scenes. 2) Self-imaging (meta-cognition [2][3]) as a nursing practitioner: Breaking down the present situation and self in the future as a nursing practitioner. 3) Self-imaging (meta-cognition) as a learner: Objectively grasping oneself as a learner based on self-imaging as a nursing practitioner. Figure 1 shows a summary of a learning material framework developed in this study. This learning material frame has accumulated examples according to nursing subjects. As learning modes, the following two are provided: The first mode is a problem-solving learning mode, which enables conductance of problem-solving type learning according to nursing processes by case. For imaging nursing scenes and a patient’s condition minutely, providing multimedia data is expected to be highly efficient in general. We recorded only voice data in the form of a patient talking about a medical condition during interaction with a nurse (a nurse interviewed the patient to obtain necessary information for practicing nursing activities). Lively interactions between a patient and a nurse are shown to learners in each phase that is necessary for proper planning and implementation of a nursing plan: The phase of grasping a subject’s condition, the phase of analyzing nursing practice scenes, and the phase of evaluating analysis results. Grasping the case Analyzing the case Evaluating the case Learning Simultaneously, noteworthy mode 1 points for problem-solving and teachers’ (experts’) suggested answers are presented appropriately so that learners can practice problem-solving learning by referring to them. Learning Providing such an mode 2 environment for learners is intended to promote the following: (i) Comprehension Knowledge learning Skill learning Problem exercise of a patient’s condition and It is characterized by having a structure that enables “study for nursing basic knowledge”, “study for nursing basic skills” and “preparing study for nursing medical nursing scenes. (ii) national exam” through case studies using the e-learning materials. Comprehension of Figure 1: Summary of the learning materials

Y. Majima et al. / Framework for Problem-Solving Based Learning in Nursing Domain

627

(self-observation) one’s own self-image in the future through a nurse who appeared in the narration. In addition, (iii) imagining the situation in which the present self is integrated into the scene and comprehension of (self-observation) the present self as a nursing practitioner. The second mode is a related learning mode. It enables the study of knowledge (e.g., anatomical physiology of human body and pathological physiology of diseases) and nursing skills (e.g., methods of self-blood glucose measurement and method for insulin self-injection) that are necessary for problem-solving in each case, by repeatedly watching figures and illustrations, photographs, and so on. In general, even learned “known” knowledge is not always “constructed as the manageable condition” in a scene to apply. Therefore, we anticipate that providing a system that enables appropriate reference and study of required knowledge and nursing skills in each problem-solving scene by the problem-solving learning mode contributes to acquiring (iv) contexts (meta-knowledge) that put them into practice. Providing two such learning modes is intended to promote self-imaging as a (v) nursing practitioner.

3. Evaluation by teaching practices 3.1 Concerning subjects For second and the third grade students in Nursing School of A University, we conducted teaching practice with corresponding learning material contents in a one hour class of each subject. The classes were held in the information processing room (100 sets installed, Windows 2000 OS; Microsoft Corp.) using one computer for one student. After explaining study goals and learning materials, and requesting research cooperation all together, we conducted the classes in the form of self-learning. 3.2 Method of teaching evaluation Having finished the classes, we carried out self-administrated questionnaires about learning material evaluation for students offering to participate in the study. Evaluation contents included 10 items concerning overall evaluation. We obtained answers from 63 students in the second grade and 68 students in the third grade.

4. Result and Discussion Points that we contrived to develop learning material for specific imaging of subjects and nursing scenes were the following four: Cases were “presented with animation”, “Cases were developed as the interactive type mode”, using voice; “Nursing problem scenes were set by scene.” “Presenting phased thinking points” were related to the problematic point. From 80% to more than 90% answered “Yes” for “I was able to imagine subjects”. In addition, almost all students answered “Good” for points which we contrive to it: “Presented with animation”, “Cases were developed as interactive type mode”, “Nursing problem scenes were set by scene” and “Presenting phased thinking points”. Results indicate that learners positively evaluated the learning materials. In contrast, only 50–60% answered “Yes” for “I understood nursing problems that subjects have”; the evaluation was lower than for other items. In comprehending nursing problems, it seems necessary for teachers to present contents with clear intentions of what they wanted students to learn with the case examples. From the students’ opinions, “I was able to easily imagine the condition by voice and pictures.” “I was able to think on a step-by-step basis.”, lively interactions between a patient

628

Y. Majima et al. / Framework for Problem-Solving Based Learning in Nursing Domain

and a nurse, attention points in problem-solving, and suggested answers by teachers (experts) provided by the learning materials gave learners a system to practice problem-solving learning. That result is considered to engender promotion of intended (i) breaking down of the intended patient’s condition and medical nursing scenes, (ii) breaking down (self-observation) of one’s own self-image in the future through a nurse who appeared in case examples, and (iii) breaking down (self-observation) of the present self as a nursing practitioner. Additionally, from opinions “I felt a need for accurate knowledge” and “I felt a need to think as my own problems”, learners seemed to understand that even previously learned “known” knowledge was not always “constructed as the manageable condition” in the scene. They had to learn while imagining the condition in which the learner encountered a problem-solving scene at that very moment. A system that enables study with appropriately referring knowledge and skills needed for each problem-solving scene in the problem-solving learning mode promotes the recall of previously learned knowledge. Furthermore, it promotes (iv) acquisition of context (meta-knowledge) which puts knowledge to practical use, and (v) self-imaging as a learner. In other words, it is considered to promote grasping one’s own ability based on self-image as a nursing practitioner, and comprehension of study goals (upgrading of evaluation scale from acquisition evaluation of knowledge and skills to management probabilistic assessment in a problem-solving context). In other words, it can prompt the learner’s meta-cognition [2][3]. Prompting meta-cognition is quite important to make the learning activities effective especially in problem-solving oriented learning [4].

5. Conclusions and future assignments In nursing education, the problem-solving learning mode and the related learning mode provided by the learning materials enable a learner to grasp their own ability based on self-image as a nursing practitioner, to comprehend learning goals, and to recognize the present situation. We stress it is meaningful that our framework makes learners’ self-evaluation criteria get sticker by imaging the real situation. It suggests our system enhance the learners’ meta-cognition. Future studies will be undertaken to verify learning effects of the learning materials through various learning scenes more detail. Acknowledgments This study was supported by a grant from Japan Ministry of Education, Culture, Sports, Science and Technology. (No.15390666, Yukie Majima) References [1] Majima, Y., Yoshimine, T. & So, Y. (2005). Evaluation of Instruction by E-learning Materials for Nursing Problem Solving Approach, Proceeding of 30th Annual Conference of Japanese Society for Information and Systems in Education, pp.365-366. (in Japanese) [2] Flavell, J. H. (1976). Metacognitive aspects of problem solving, In L. Resnick (Ed.): The Nature of Intelligence, Lawrence Erlbaum Associates: Hillsdale, NJ, pp. 231-235. [3] Okamoto, M. (1999). The Study of Metacognition in Arithmetic Word Problem Solving. Kazama-shobo Publishing, Tokyo (in Japanese) [4] Seta, K., Tachibana, K., Umano., M., and Ikeda, M. (2005). Human Factor Modeling for Development of Learning Systems Facilitating Meta-Cognition, in Chee-Kit Looi, David Jonassen, Mitsuru Ikeda (Eds): Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences, Frontiers in Artificial Intelligence and Applications, Vol. 133, pp. 396-403, IOS Press, (also Proc. of the International Conference on Computers in Education (ICCE-05), Singapore)

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

629

A Polytomous Computerized-Adaptive Testing that Rewards Partial Knowledge Yung-Chin Yen, Rong-Guey Ho, Li-Ju Chen Department of Information and Computer Education, National Taiwan Normal University, Taipei, Taiwan [email protected]

Abstract Partial knowledge is one of the main factors to take into account when dealing with the improvement of administration of Multiple Choice Questions. Various strategies have been proposed for this factor in traditional testing environment. However, few researches were designed to investigate the impact of partial knowledge on computerized-adaptive testing (CAT). This study proposed a CAT which fits confidence-weighting scheme into graded response model (GRM). The results show the polytomous method is able to assess partial knowledge of examinees with fewer items and higher predictive validity. Besides, the estimated ability is not as likely to be affected by guessing and therefore more consisted with examinee’s true ability. Keywords: partial knowledge, computerized adaptive testing, confidence-weighting, graded response model

1. Introduction Multiple choice questions (MCQ) have been the most widely used assessment method for decades due to their versatility in assessing a wide range of objectives, high reliability, and easy scoring. However, the general acceptance of the MCQ does not imply that its merits are optimal. Comparing with other testing formats, the MCQ test is more easily affected by guessing. One main problem confronted with MCQ test derived from its all-or-none scoring system. The underlying assumption of dichotomous scoring method assumes that an examinee will check the correct answer if s/he has complete knowledge; otherwise s/he will either omit the question, or guess randomly the alternative offered [1][2]. In fact, the examinee who does not know the correct response always makes his/her selection based on partial knowledge [1][3]. As a consequence of conventional dichotomous scoring scheme, it is impossible to distinguish correct answers based on knowledge from those obtained from lucky guesses. Various solutions have been proposed to cope with the problems described above. Marking schemes for MCQ like order-of-preference schemes, answer-until-correct procedures, and confidence-weighting assessment all attempt to diminish the influence of blind guessing and to assess the partial knowledge of examinees. However, these solutions described above are all designed for conventional paper-and-pencil (P&P) or computerbased testing (CBT). Few researches were conducted to investigate the impact of guessing and partial knowledge on computerized-adaptive testing (CAT). The main intention of this study is to examine whether the efficiency and precision of CAT can be improved by applying confidence-weighting scheme. Since the administration of confidence-weighting test is relevant to express one’s appropriate levels of confidence, the relationship between examinee’s ability and partial knowledge is also

630

Y.-C. Yen et al. / A Polytomous Computerized-Adaptive Testing

investigated. This paper first briefly overviews the concept of examinee’s partial knowledge, confidence-weighting test, and the rationale for CAT. Next, we present a polytomous CAT model called Confidence-Weighting Computerized-Adaptive Testing (CWCAT) which combines confidence-weighting scoring scheme with graded response model (GRM) [4]. Finally, an experiment was conducted to evaluate the performance of CWCAT. Emphasis is placed on a comparison between the CWCAT and conventional dichotomous CAT (DCAT) in terms of their test efficiency and test validity. Furthermore, the effect of examinees’ partial knowledge on the validity of CAT was investigated.

2. Brief Survey of Examinees’ Partial Knowledge and Confidence-Weighting Test Formula scoring, as the most widely used method to reduce the effects of guessing— though discourages examinees from blind guessing—may introduce more problems than it solves. A voluminous theoretical and empirical literature was developed to investigate issues such as the impact of test directions on testing scores [5][1][2], test reliability and validity under formula scoring [5][6][7][8][9], and the tendencies to guess of different personality traits and of different lingual cultural backgrounds [2][10]. A large part of these problems lies in the dichotomous nature of the conventional MCQ test. Under dichotomous scoring condition, an examinee is thought to have complete knowledge if s/he checks the correct answer; otherwise, it is considered blind guessing. However, the examinee’s understanding of learning task may fall between these two extremes, i.e., partial knowledge [8][9][11][12][13][14][15]. Partial knowledge can either imply possession of incomplete information that may improve the probability of a successful guess, or indicate the lack of confidence in knowledge one actually has. One representative example comes from Diamond and Forrester [16], who define knowledge as asking the question “What do you know?” followed by the meta-question “How sure are you of the answer to the question about what you know?”. In certain situation, “partial” even refers more to uncertainty than to incompleteness [8]. Confidence-weighting is a model of assessing partial knowledge lies in the degree of examinee’s confidence to their answer. Although confidence testing has a history that dates back to the early part of the 19th century, it first began to be treated as a method for increasing the amount of information in objective tests during the decade of the 1930s [17]. Hevner [18] was one of the first to evaluate this method for true/false tests, and she argued that confidence-weighted numberright scoring can improve test reliability. A similar increase in reliability was found by Soderquist [19]. In an English vocabulary test which asked examinees to mark on a threelevel confidence option (complete confidence, partial knowledge, and random guessing) with each item, Abu-Sayf and Diamond [20] argued that the internal consistency of scores increased with the confidence in the answers while the validity coefficient was highest when partial knowledge was used in the answering process. For a learner, to know his/her own level of knowledge is helpful to control the learning progress. However, few learners are able to accurately assess how much they know. Researches indicate that examinees are often biased and unreliable introspectors into their own subjective states of uncertainty, while the dominant factor that causes this is partial knowledge [12]. The aim of confidence-weighting marking is to encourage reflection, self-awareness, and the expression of appropriate levels of confidence [14].

Y.-C. Yen et al. / A Polytomous Computerized-Adaptive Testing

631

3. Computerized-Adaptive Testing Models CAT is a procedure for administrating tests in which each examinee may receive a different sequence of items that are selected based on current ability to measure their latent trait [21][22]. Hence, unlike the conventional test, items administered in CAT are individually tailored for each examinee. In contrast to traditional testing, CAT not only enables more efficient and precise ability estimation [22][23], but also provides more reasonable interpretation of examinee’s ability [24]. Although the majority of CATs are based on dichotomous item response theory (IRT) models, some researchers have explored the use of polytomous IRT models. In comparison to dichotomous IRT models, polytomous models allow more information about trail level to be extracted from a fixed set of items [25]. Though several IRT models exist for polytomous item response data, GRM was applied in this paper since its ordered categorical responses correspond perfectly with examinees’ different response types in confidence-weighting scheme. The GRM is IRT model appropriate for items having ordered response categories, where higher categories indicate greater ability. The mathematical for of GRM may be expressed as 1.702a i ( b xi ) e Pxj ( ) = (1) 1.702a i ( b xi ) , 1+ e where i is the item number, x =0,1, ... mi is the response category, is the latent trait parameter, ai is the discrimination parameter for item i , bxi is the threshold parameter for category x of item i , and e is constant of 2.718. By definition, Pxj ( ) is the probability of the examinee with ability receiving a category score x i or higher on item i . Under GRM, each item is described by one discrimination, or slope, parameter, and a set of threshold parameters. One goal of fitting the GRM is to determine the location of these thresholds on the latent trait continuum [26]. A typical example of the item response category characteristic curves for a four category item having item parameters (a=1.133, b1=-0.893, b2=-0.052, b3=0.946) is given in Figure 1.

Figure 1: A typical item-characteristic curve for a 4 category

632

Y.-C. Yen et al. / A Polytomous Computerized-Adaptive Testing

4. Method The experiment was organized in two successive phases. In the first phase, a computerbased confidence-weighting test was given to 489 third-year senior high school students. The item bank used by this test consisted of 83 items with four response options drawn from the vocabulary sections of the English Ability Test administered by College Entrance Examinations Center (CEEC) [27]. In this test, examinees were asked to state with each answer their level of confidence (1, 2 or 3) in the correctness of their decision. Corresponding marks of 0.3, 0.6, or 1.2 are awarded if their selection is correct, while 0, 0.3, or -0.6 are awarded (respectively) otherwise. This test was set up for the purpose of calibration of parameter values of two CATs in phase two. Two sets of item parameters applied in phase two were estimated with the marginal maximum likelihood method of PARSCALE [28]. The first dichotomous one was based on the correctness of response regardless of confidence level and was then applied in the dichotomous CAT (DCAT). A description of the item parameters is given in Table 1. The second set of parameter was calibrated for polytomous CAT (CWCAT) according to six possible response types (three confidence levels cross right/wrong feedback). Six categories were classified for each item according to six corresponding response types and, hence, five threshold parameters were calibrated, respectively. Table 2 shows the properties of these parameters. Table 1: The Properties of Three Parameters for Dichotomous CAT (Number of items = 83) Parameter b a c

Mean -0.0855 1.1022 0.2038

Std Dev 0.7678 0.3779 0.0959

Minimum -1.9306 0.1196 0.0012

Maximum 1.6763 1.8821 0.5832

Table 2: The Properties of Parameters for Polytomous CAT (Number of items = 83) Parameter a b1 b2 b3 b4 b5

Mean 0.9338 -0.9725 -0.4490 -0.0917 0.4255 1.0878

Std Dev 0.3068 0.5126 0.2863 0.2648 0.2896 0.6613

Minimum 0.3543 -3.6087 -2.1822 -1.4397 0.1206 0.3841

Maximum 1.5647 -0.3555 -0.1495 0.3845 2.3491 4.8815

During the second phase, two CAT programs were written as follows: one program was based on the dichotomous 3PL model (DCAT), whereas the other was based on the GRM (CWCAT). Both versions of CAT began with an item of medium difficulty to obtain the initial value estimated while employing maximum information item-selection procedure, and both were terminated when a pre-set maximum test length (20 items) were reached or the standard error of estimation (SE) had fallen below 0.3. Seventy-three thirdyear senior high school students participated in this phase. Administration of these two versions of CAT system was counterbalanced to control for possible test order effects by assigning respondents to two groups: Group A (39 students took DCAT first) and Group B (34 students took CWCAT first).

Y.-C. Yen et al. / A Polytomous Computerized-Adaptive Testing

633

5. Results The analyses of results collected were organized according to our intention: (a) To examine whether the efficiency and precision of CAT be improved by handling guessing and assessing partial knowledge. (b) To investigate the relationship between examinee’s ability, self-awareness, and partial knowledge. 5.1 Precision and Efficiency Since investigating the effects of confidence-weighting scheme on CAT is one of the principle aims of this study, it is important to analyze the distributions of the estimated abilities obtained by DCAT and CWCAT. Figure 2 depicts the estimated abilities according to the two systems and it is evident that the distributions are approximately similar. The graph in Figure 2 shows that some lower ability examinees in DCAT get higher ability in CWCAT; however, the difference in these two abilities is not significant for those high ability examinees.

Figure 2: Ability distributions of CWCAT and DCAT Table 3 displays the descriptive statistics of ability estimation, time used per item (in seconds), and test length (number of items) administered in DCAT and CWCAT in detail. The mean ability estimated by CWCAT (CWCAT_A = 0.316) is slightly higher than DCAT version (DCAT_A = 0.212). The mean time used per item in CWCAT (CWCAT_T = 24.427) is marginally longer than DCAT (DCAT_T = 23.853). As expected, the items used in CWCAT (CWCAT_I = 5.411) are significantly less than DCAT (DCAT_I = 11.986) due to the characteristics of polytomous IRT. After further analysis, the differences between ability estimated (DA = -0.104), time used (DT = -0.619) and number of items administered (DI = 6.575) between DCAT and CWCAT also appear in Table 3. A paired t-test is used to determine whether two means are significantly different at a selected probability level (alpha 0.05 is selected in the analysis). The result shows that there is no statistically-significant difference between the mean of time used in two versions of CAT. For the ability estimated and items used, however, the differences are statistically significant.

634

Y.-C. Yen et al. / A Polytomous Computerized-Adaptive Testing

Table 3: Descriptive statistics of ability estimated, time used, and item length administrated and results of t-test Variable

Mean

DCAT_A 0.212 CWCAT_A 0.316 DCAT_T 23.853 CWCAT_T 24.472 DCAT_I 11.986 CWCAT_I 5.411

Std Dev

Means of Differences

SD of Differences

t value

Prob > |t|

-0.104

0.050

-2.09

0.0403

-0.619

1.003

-0.62

0.5391

6.575

0.586

11.22

<.0001

0.084 0.073 0.947 0.973 0.424 0.380

5.2 Predictive Validity One main purpose of this study was to investigate the effect of partial knowledge on CAT. This was performed by comparing the differences in predictive validity between the abilities estimated from two CATs. Two kinds of grades in English Language (Final_Eng) and English Composition (Final_Comp) of the school term were used as predictive indicators. Table 4 presents the statistical distributions of these two factors. The correlations of the two indicators and abilities estimated by two CATs are listed in Table 5. Table 4: The properties of three scores as criteria for predictive validity (Number of examinee = 73) Scores Final_Eng Final_Comp

Mean 72.8904 74.9178

Std Dev 11.0549 8.2357

Minimum 36 55

Maximum 97 95

Table 5: Pearson correlations between ability estimated by two CATs and three criteria (n=73) DCAT_A CWCAT_A *P<0.01; **P<0.0001.

Final_Eng 0.6905** 0.7200**

Final_Comp 0.3622* 0.4013**

As seen in Table 5, Pearson correlations between both CATs and two criteria were all statistically significant. It was found that in both cases the CWCAT_A correlations were higher than the DCAT_A correlations (0.72 vs. 0.6905 and 0.4013 vs. 0.3622, respectively).

6. Discussion The efficiency of this system was examined by comparing the abilities estimated, time used per item, and test length with conventional dichotomous CAT. As expected, the mean time for each item in CWCAT was a little above the average, though not significantly, than in DCAT. The additional time (about 0.6 seconds) required to mark every confidence option is apparently not a drawback of the CWCAT. Besides, the shorter length of CWCAT has obviously reduced the total testing time it needed.

Y.-C. Yen et al. / A Polytomous Computerized-Adaptive Testing

635

The substantial performance increase as indicated by the CWCAT results can be attributed to partial knowledge. Since CWCAT evaluates examinees’ partial knowledge by confidence-weighting, the increment in abilities can be considered as partial knowledge which fails to assess under dichotomous CAT. Consequently, examinees with more partial knowledge than complete knowledge (ie, low ability examinees) may receive higher ability in CWCAT system than in DCAT according to their different degrees of partial knowledge. The smaller deviation also indicates that the CWCAT discriminates well between examinees at different levels of ability. The meaning of partial knowledge can be further examined in the context of validity. According to the analysis result of predictive validity, the CWCAT has greater capacity to predict performance in English Language and English Composition than DCAT. It clearly points out that the ability estimated from CWCAT is more equivalent with the examinee’s true ability than the results from DCAT. This result consists with the finding from AbuSayf and Diamond [20].

7. Conclusions This study represents an effort to explore confidence-weighting scheme for CAT in the context of polytomous IRT. By blending GRM and confidence-weighting scoring, the CWCAT is able to assess partial knowledge of examinees with fewer items and higher validity. Examinees under this test environment are forced to reveal their partial knowledge and reflect their own knowledge state. Furthermore, the ability measured by CWCAT is not as likely to be affected by guessing and therefore more consisted with examinee’s true ability. The idea here is that by investigating strategies used by conventional tests to assessing partial knowledge, we may be able to find a viable and more efficient alternative approach for CAT. It is on these aspects that the importance of this study lies.

References [1] Lord, F. M. (1975). Formula scoring and number-right scoring. Journal of Educational Measurement, 12, 7-22. [2] Rowley, G. L. & Traub, R. E. (1977). Formula scoring, number-right scoring, and test-taking strategy. Journal of Educational Measurement, 14, 15-22. [3] Jackson, R. A. (1955). Guessing and test performance. Educational and Psychological Measurement, 15, 74-79. [4] Samejima, F. (1997). Graded response model. In W.J. van der Linden & R.K. Hambleton (Eds.), Handbook of Modern Item Response Theory (85-100). New York: Springer. [5] Diamond, J. & Evans, W. (1973). The correction for guessing. Reviews of Education Research, 43, 181-191. [6] Bliss, L. B. (1980). A test of Lord’s assumption regarding examinee guessing behavior on multiple-choice tests using elementary school students. Journal of Educational Measurement. 17, 147-152. [7] Alnabhan, M. (2002). An empirical investigation of effects of three methods of handling guessing and risk taking on the psychometric indices of a test. Social Behavior and Personality, 30, 645-652. [8] Burton, R. F. (2002). Misinformation, partial knowledge and guessing in true/false tests. Medical Education, 36, 805-811. [9] Burton, R. F. (2005). Multiple-choice and true/false: myths and misapprehensions.

636

Y.-C. Yen et al. / A Polytomous Computerized-Adaptive Testing

Assessment and Evaluation in Higher Education, 30, 65-72. [10] Gafni, N & Melamed, E. (1994). Differential tendencies to guess as a function of gender and lingual-cultural reference group. Studies in Educational Evaluation, 20, 309-319. [11] Coombs C. H. & Womer F. B. (1956). The assessment of partial knowledge. Educational and Psychological Measurement, 16, 13-37. [12] Budescu, D. & Bar-Hillel, M. (1993). To guess or not to guess: a decision-theoretic view of formula scoring. Journal of Educational Measurement, 30, 277-291. [13] Hammond, E. J., McIndoe, A. K., Sansome, A. J. & Spargo, P. M. (1998). Multiplechoice examinations: adopting an evidence-based approach to exam technique. Anaesthesia, 53, 1105-1108. [14] Gardner-Medwin, A. R. & Gahan, M. (2003). Formative and summative confidencebased assessment. In Proceedings of the 7th International Computer-Aided Assessment Conference (147-155), Loughborough University, UK. [15] Bar-Hillel, M., Budescu, D.,& Attali, Y. (2005). Scoring and keying multiple choice tests: a case study in irrationality. Mind & Society, 4, 3-12. [16] Diamond G. A., & Forrester, J. S. (1983). An epistemologic model of clinical judgment. The American Journal of Medicine, 75, 129-137. [17] Echternacht, G. J. (1971). The use of confidence testing in objective tests. Review of Educational Research, 42, 217-236. [18] Hevner, K. A. (1932). A method of correcting for guessing in true-false tests and empirical evidence support of it. Journal of Psychology, 3. 359-362. [19] Soderquist, H. (1936). A new method of weighting scores in a true-false test. Journal of Educational Research, 30, 290-292. [20] Abu-Sayf, F. K., & Diamond J. J. (1976). Effect of confidence level in multiplechoice test answers on reliability and validity of scores. Journal of Educational Research, 70, 62-63. [21] Lord, F. M. (1980). Applications of Item Response Theory to Practical Testing Problems. Hillsdale, NJ: Eflbaum. [22] Weiss, D. J. (1982). Improving Measurement Quality and Efficiency with Adaptive Testing. Applied Psychological Measurement, 6, 473-492. [23] Wainer, H., Dorans, N., Flaughter, R. Green, B., Mislevy, R., Steinberg, L. & Thissen, D. (1990). Computerized-adaptive testing: A primer. Hillsdale, NJ: Eflbaum. [24] Ho, R. G. (2000). Tailored testing—Adaptive testing, Journal of Testing and Counseling, 157, 3288-3293. [25] Drasgow, F., Levine, M.V., Tsien, S., Williams, B., & Mead, A. (1995). Fitting polytomous item response theory models to multiple-choice tests. Applied Psychological Measurement, 19, 143-165. [26] Cagnone, S. & Ricci, R. (2005). Student ability assessment on two IRT models. Metodoloski zvezki, 2, 209-218. [27] Ho, R. G. & Yen, Y. C. (2005). Design and evaluation of an XML-based platformindependent computerized-adaptive testing system. IEEE Transactions on Education, 48, 230-237. [28] Muraki, E., & Bock, R. D. (1999). PARSCALE. Chicago, IL: Scientific Software.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

637

Research on Algorithm of Computer Adaptive Test Using Optimized MDPLTM Jin-Ling Lia, Feng-lin Wanga, Wang-Xiu Lia a Computer Collage, Nanhua University, China [email protected] Abstract: 2n the basic of the modern education measure theory and computer adaptive test, the paper designed the algorithm of the adaptive test using the new model of item response theory: multi-dimensional polytomous latent trait model. Meanwhile, it tried on individual restrain parameters to optimize the algorithm in order to content the development of an individual’s all-round way abilities. Keywords: Innovation ability; adaptive test; IRT; optimized multi-dimensional polytomous latent trait model ΰOMDPLTMα; difficulty value

Introduction The paper means to realize the test and assessment of learner’s multi-abilities. It was based on the model of the modern educational measurement and the computer adaptive test. And it introduced the new model in IRT named multi-dimensional polytomous latent trait model. Then the paper optimized the algorithm of the adaptive testing through increasing the model’s restrained parameters. At last, a simulate experiment was given out. 1. The Determination of the MDPLTM and the Optimization of the Model 1.1 The Determination of the MKDPLTM Multi-dimensional polytomous latent trait model (abbreviate as MDPLTM) is a multi-dimension test model in IRT. The model could be determined as follows: Pi (θ jq ) =

exp ∑q =1Wqy i (θjq − biqyi ) n

∑

yi

exp ∑q =1Wqz (θjq − biqz ) z =0

mi

n

yi

(1)

Among them, θjq expresses the value of the q ability (q=1,2,,,n) of the j learner; ̕i expresses the learner’s score on the i item (yi=0,1,2,…mi); Wqy yi expresses the fixed integer value of the item i. the item i related to the ability q and reflected as y; biqyi expresses the difficulty parameter of the item i, which related to the ability q and reflected as y; P (θjq) expresses the corrected probability of the item i. the item was given to the learner whose value of the ability q is θjq. 1.2 The Optimization of the MDPLTM The item’s difficulty parameter Βiqy is fixed in advance. For this, the item was lack of relative flexible degree when testing different learners. So the MDPLTM should be

638

J.-L. Li et al. / Research on Algorithm of Computer Adaptive Test Using Optimized MDPLTM

optimized. Here individual restrained parameter was introduced. The relative defines are as follows: Define 1 Preset basic ability of the item θm. It means the ability being according to when the difficulty parameter of each item is preset. Define 2 Difficulty value ρjq. It means the deviate value between the learner’s real difficulty value and the value that θm is corresponding to. According to the define 1 and the define 2, the MDPLTM could be exchanged into the following formula: Pi (θ jq ) = ρ jq =

exp ∑q =1Wqy i (θ jq − biqyi (1 + ρ jq )) k

∑

mi

z =0

θ m − θ jq

yi

exp ∑q =1Wqz (θ jq − biqz (1 + ρ jq )) k

yi

θm

˄2˅

˄3˅

2. The Procedure of the Adaptive Test and the Determination of Certain Algorithms 2.1 The Procedure of the Adaptive Test The test system should have the ability to collect learner’s learning background parameters before testing. And it could offer learner’s ability being accord with to fix the item’s difficulty. Namely it is the preset basic ability of the item θm. The adaptive test procedure is as following figure 1:

Figure.1. the Adaptive Test Procedure The procedure could be described as follows: Individual mining algorithm: Mining out the relative principle between the former learners’ learning background and ability. And compare with the present learners’ learning background. The learners’ original multi-ability θ0 could be gotten. Comparing algorithm: Comparing learner’s multi-ability with θm. The learner’s difficulty value about the test items ρjq could be gotten. Matching algorithm of the maximum information entropy: Getting learner’s difficulty parameters. And the items which have the maximum information could be select. Evolution algorithm: Evaluating learner’s evolved multi-ability according to test result. Adjusting the multi-ability value in order to approach the real ability level further. Ending algorithm: If the ending condition could be satisfied and there is no accident, then the test could be ended. If not, returning to the procedure (3) and continuing the test. If there are accidents happened, then marking the invalid test and withdrawing the test.

J.-L. Li et al. / Research on Algorithm of Computer Adaptive Test Using Optimized MDPLTM

639

2.2 Certain Algorithm of Adaptive Test 2.2.1 Adaptive Mining Algorithm of the Learner’s Original Ability In order to receive learner’s original ability, we could give mining between his former learning background and former ability. And the relative principle could be obtained. If there is a learner entering the system to test, relative learning background would be collected. And it would be mapped to the relative principle. 2.2.2 Adaptive Selecting Algorithm of the Item Here, we introduced the optimized MDPLTM. And the multi-value items were selected as the test items. Their information function is maximum on the original evaluating ability θ0. The test item’s information function could be described as:  P ′ (θ ) mi  im jq  ″ − ∑ Pim (θ jq ) I i (θ jq ) = ∑  P ( ) θ m =1 m =1 im jq 2

mi

˄6˅

In this, the item could be selected to test the individual. The item could satisfy the Ii (θjq) being maximum under the ability value (θ). 2.2.3 Adaptive Evaluating Algorithm of the Evolution Ability The Bayes posterior estimation is introduced to remedy the defect of the maximum likelihood function. the Bayes theorem is: P˄θjq˅= L˄θjq˅*g˄θjq˅/P(Ui) ˄7˅ In this, the Bayes expected posterior estimation could be given. For analyzing deeply into the sub-item, each item is satisfy with the partly independence. So the following form is established: L˄θjq˅=ПPi(θjq)uij * Qi(θjq)1-uij 웉8웊 In it, Ui expressed N*m score matrix (ui1, ui2,,,uij); L(θjq) expresses the conditional likelihood function; g(θjq) expresses the prior distribution of the item’s parameters. In the test system, g ( θjq) could be gotten according to the correct probability of the learner’s preceding test. 2.2.4 Ending Algorithm of the Adaptive Test In the adaptive test, the evaluating standard deviation of the evaluating ability could be used as the ending algorithm. The form is as follows: ∑I˄θi˅-1/2<∂ ˄9˅ When the condition is satisfied, the test could be ended. Considering learner is easy to be reflected by themselves or the outside. And it could cause the great change between the preceding information function and the following one. It could make the unexpected ending. As we know, the result gotten from the test could not be believed. So in the paper, the fixed condition was introduced in the ending condition. In it, β is the preset fixed value. | Ii+1˄θ˅- Ii˄θ˅|<β ˄10˅

640

J.-L. Li et al. / Research on Algorithm of Computer Adaptive Test Using Optimized MDPLTM

3. Simulation and Analysis of the Test Procedure For measuring the ability of the test algorithm based on MKDPLTM. The simulating item base was established. There were 500 items in the base. In it, Wqy yi~ (0.2,2.0) and biqy~(-4,4). In addition, the two-value item base was established for comparing the test efficiency. Through the simulating test, we got the average parameter of the test item satisfying mi=3 as the analyzing target. The comparing figure could be gotten as follows:

Figure.2. Comparing of the Two Item Characteristic Curve Satisfying mi=3 In it, the thin curve expressed the characteristic curve of the MKDPLTM which was not optimized. The number of the item’s reflecting styles were 4. The thick curve expressed the characteristic curve of the optimized MKDPLTM whose difficulty value was 0.2. Dotted curve expressed the characteristic curve of the two-value items. As we could see, as to the characteristic curve of the MKDPLTM, the one of the two-value was not abundant obviously. And it could not reflect learner’s multi-ability well. 4. Conclusion It is possible to realize the adaptive test based on learner’s multi-ability further. But at the same time, the difficulty and the test efficiency of the system would be increased. It involves individual parameters such as individual difficulty value, individual multi-ability, etc. The formulation and the obtaining of the parameters would affect the testing result directly. And many characteristics of the learner are not introduced into the adaptive test. These are the problem we would study in order to establish the individual and adaptive test for all learners. References [1] T. H. Wang, K. H. Wangw, W. L. Wangz, S. C. Huangz & S. Y. Chen, “Web-based Assessment and Test Analyses (WATA) system: development and evaluation”, Journal of Computer Assisted Learning, Vol.20. pp 59–71, November 2004. [2] Adam W. Meade, “Same Question, Different Answers: CFA and Two IRT Approaches to Measurement Invariance”, 19th Annual Conference of the Society for Industrial and Organizational Psychology, pp 56-58, April 2004. [3] Qi Shuqing, Dai Haiqi, Ding Shuliang, Principles of Modern Educational and Psychological Measurement, Beijing, Higher Education Press, Aug. 2002. [4] Dong Min, Huo Jianqing, Wang Xiaopu, “Research on Computerized Test Construction Based on Adaptive Genetic Algorithms”, MINI-MICROSYSTEMS, Vol.25, No.1, pp 82-85, Jan. 2004. [5] Wu Yanwen, Luo Xiaoqiao, “Can Students Grasp Study Points? An Intelligent Evaluation System”, China Distance Education, pp 70-72, Oct. 2004.

Doctor Student Consortium (DSC) Papers

This page intentionally left blank

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

643

Cognitive Maps-Based Student Model Alejandro Peña1,2,3, Humberto Sossa3, Agustín Gutiérrez3 WOLNM1, UPIICSA2& CIC3 - National Polytechnic Institute2,3, Mexico [email protected] Abstract. The main contribution of this PhD thesis is: A proposal for Student Modeling based upon Cognitive Maps (CM’s). Its purpose is to anticipate the effects that a teaching-learning experience will produce on a student before its delivery. Thus, this dissertation focuses on the Student Modeling from the causal-effect perspective. Also, our approach aims at predict outcomes stemmed from teaching-learning sessions delivered by Web-based Education Systems (WBES’s). Wherefore in WBES’s, whose lessons have several sequence strategies and options of content, our Student Model (SM) offers a proactive and adaptive support for a student-centered education. Keywords: Cognitive Maps, Student Model, concepts, relations, causality.

Introduction Student Modeling is the domain of the PhD dissertation resumed in this paper. In a WBES, the SM carries out the generation and maintenance of a mental profile of the individual. The Student Modeling is a challenge, and it could be an intractable problem [1], due to it deals with subjective concepts, partial beliefs and complex tasks. As CM’s are mental models that depict issues from the causal perspective, our thesis sets the teaching-learning process as a cause-effect phenomenon. Wherefore, our project is a pioneer work that introduces the use of CM’s for Student Modeling. Its relevance remains on: A set of theoretical hypothesis and empiric evidence that will be gathered from experimental trials. Thus the overview of the dissertation includes the following subjects: problem definition, solution proposal, conceptual model, and structure of the approach. Finally in the conclusion section is pointed out the contributions of the PhD project and the further work to be done. 1. Overview of the Dissertation 1.1Problem Definition The PhD dissertation focuses on the problems about: How to depict the factors that influence the knowledge acquisition of the student, and how to anticipate their causal effects, before he/she faces a teaching-learning experience? 1.2 Proposal As a solution for the problem early stated the thesis proposes: A Cognitive Maps-based Student Model to sketch and predict the causal phenomenon achieved by a teachinglearning experience delivered by a WBES. The underlying assumption of the thesis is: As a result of identify the factors that bias the teaching-learning, set their cause-effect relationships, and apply a causal-inference mechanism; it is possible to state a SM that anticipates the causal impact on the knowledge acquisition that the student faces.

644

A. Peña et al. / Cognitive Maps-Based Student Model

1.3 Conceptual Model The underlying concepts of our approach remain on: The Activity Theory [2], the Rule-Base Fuzzy Cognitive Maps [3], and the SM foundations [1]. So a profile of the baseline elements is outlined next. The Activity Theory sets five principles: Object-orientedness, hierarchy, tool, externalization–internalization, and development. In regards to the CM’s, they are considered as cause-effect prediction models. The CM’s are based on the philosophy principle about causality that states: For every fact there is at least one cause, and given the same conditions, the same causes bias the same effects. A CM is sketched as a digraph, where nodes correspond to the concepts and links to the causal relationships. Each causal relation is described by a fuzzy rule base. The concepts involved in a CM reveal the beliefs (B) that the WBES (s) holds about the student (U). The beliefs are stated as propositions (p). Thus, the mental model (MM) that the system develops is the set of prepositions that the system believes are believed by the student, as: MM = Bs (U) = { p | Bsp (U) }. 1.4 Structure of the Approach The SM approach is accomplished through three stages: Development, Initialization and Exploitation. In the development stage are fulfilled five components of the SM, as: 1) Module for eliciting the student profile. 2) Tool for authoring teaching-learning experiences. 3) Ontology for depicting the concepts stemmed from the domains. 4) Ontology based mechanism for the automatic generation of the CM’s. 5) Fuzzy causal inference engine. In the initialization stage is carried out four processes: 1) Elicitation of the student profile. 2) Description of the teaching-learning experiences. 3) Ontology administration. In the exploitation stage the teaching-learning cycle is achieved by four steps: 1) Selection of the best option for the teaching-learning experience. 2) Provision of the selected option. 3) Gathering empirical evidence. 4) Tuning the Student Model.

2. Conclusions The contribution of the PhD thesis is to propose a new SM approach causal-centered by means of CM’s. Its theoretical goals are: A set of hypothesis about the impact on the students that experience the support of the Cognitive Maps-based Student Model; and the empirical results gained from the experiments. The further work to be done is: The development of the prototype and the experimentation of the approach.

Acknowledgments The first author states that: This work was inspired in a special way for his Father, his brother Jesus and his Helper as part of the projects of World Outreach Light to the Nations Ministries (WOLNM). Also this work was economically supported by: CONACYT 182329, SEP 2004 C0146805/747, COFAA debec/138/06, Cotepabe 430/05 & Microsoft Mexico.

References [1] Self, J. (1988) Bypassing the Intractable Problem of Student Modeling. Proceedings of ITS´88, 18-34. [2] Leont´ev, A. (1947) Problems of the Development of Mind. English translation. [3] Carvalho, J.P. (2001) Rule-based Cognitive Maps. Qualitative Dynamic Systems Modeling and Simulation. PhD Thesis, Lisboa Technical University, Portugal, October.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

645

Improving Quality of Online Forum Interactions in Distance Higher Education Zhenhong Zhang, Ronghuai Huang Knowledge Science and Engineering Institute, School of Educational Technology, Beijing Normal University, P.R. China [email protected]

As one component in distance education system, interaction plays a key role in facilitating knowledge construction, effective collaboration and offering emotional support for students, which correlates closely to the quality of distance education. Since students may prefer various time and place for learning in distance education, online forum as a major means of asynchronous interaction is adopted by CRTVU (China Radio and Television University) and most of the 67 distance higher education colleges in China. This paper discusses the factors that influence the quality of online forum interactions and make recommendations to e-tutors. Literature examining the use of asynchronous computer-mediated communication for supporting online learning activities cited a number of advantages [1, 2, 3, 4, 5]. Scardamalia and Bereiter analyzed the content of electronic messages of online forum discussions and proved that online discussions support collaborative learning [6]. Wu Cheng-Chih and Lee Greg C. studied BBS as a tool in computer mediated communication in education and pointed out that the key factor influencing students communication was the coordinator’s guidance and evaluation and students should be allowed to discuss topics in a wider range through BBS [7]. To determine the factors that are of significance to effective online forum interactions, research was conducted in the School of Modern Distance Education, Nankai University, P. R. China. 5 e-tutors were in charge of offering learning support to the 183 sophomores in the way of answering students’ questions through Q&A section of the course website, guiding online forum interactions and conduct evaluation of students. Questionnaire surveys and interviews were made at the end of the semester to collect data about students’ perceptions on the factors in online forum interactions. Content analysis of online discussions was also conducted. Questionnaire surveys revealed that online forum interactions were an indispensable part in distance education. E-tutors’ role in guiding online discussions and supporting students enhanced their willingness to participate in the discussions. Students also considered e-tutors’ periodical synthesis valuable to their learning. As to the content and proposition of discussion topics, positive evaluation had been made of their impact on the students’ willingness to participate in and their interest in online forum interactions. To collect more elaborate information about students’ perceptions of the specific factors closely related to quality of online forum interactions, interviews of 18 from the 183 sophomores were made by the lead author. Most interviewees made positive evaluation to online forum interactions, since compared with face-to-face discussions, they had enough time to reflect on their ideas and express them in clearer language, with clearer logic and better structure. Interviewees also mentioned in-depth communication and more control in

646

Z. Zhang and R. Huang / Improving Quality of Online Forum Interactions

learning as advantages of online forum interactions over face-to-face discussions. As to discussion topics, discussion topics should serve as scaffold in students’ knowledge construction. Further, students interviewed showed a general tendency to like those topics that were expressed in simple sentences, easy to comprehend, explained with specific cases, or attached with as few reference materials as possible. As to e-tutor’s role in facilitating forum discussions, it is crucial for e-tutors to participate in and offer timely feedback to students’ discussions. Some interviewees considered most of the postings from peers only personal opinions without much contribution to their knowledge construction and would only like to read e-tutors’ opinions. Also e-tutors’ periodical review and synthesis of discussions help students a lot and their emotional support can increase students’ motivation in participation in online discussions. Analysis was conducted on the 1054 postings in the general forum discussions according to Stacy’s classification of activities of knowledge construction through online collaborative learning into clarification (C), obtaining feedback (O), sharing perspectives (S), and group sharing of resources, new ideas and expert advice (G). Among the four features S and G are on a higher level than C and O in terms of knowledge construction for S and G involve output while C and O involve mainly input of information. The result of analysis showed that e-tutors played a significant role in raising the level of knowledge construction in the process of online discussions. Research results of questionnaires, interviews and content analysis of discussions indicated that online forum interactions contributed to the knowledge construction of students in distance learning. Factors influencing the quality of online discussions include: (1) Content of discussion topics: Topics within students’ perceptions, closer to students’ life experience, having more practical significance, and in the form of case studies are most welcomed. (2) Proposition of topics: Topics should be expressed in simple languages, elaborated with necessary cases, and attached with as few reference materials as possible. (3) E-tutor’s guidance and support: E-tutors should exert more effort in making timely feedback, participating in online discussions, offering emotional support to students by means of encouragement or praise, and making synthesis of discussions regularly. Whether the research results can be of significance to students in other age groups, without previous computer or Internet experience, or with various cultural backgrounds need to be further tested. More research needs to be carried out in this aspect.

References [1] Harasim, L., Hiltz, S. R., Teles, L., and Turoff, M. (1998) Learning networks: A field guide to teaching and learning online. Cambridge, MA: MIT Press. [2] Eastmond, D. V. (1994) Adult distance study through computer conferencing. Distance Education, 15(1), 128-152. [3] Graddol, D. (1989) Some CMC discourse properties and their educational significance. In R. Mason & A. Kaye (Eds.), Mindweave: Communication, computers and distance education. New York: Pergamon Press, 236-241. [4] Hewitt, J. (2001) Beyond threaded discourse. International Journal of Educational Telecommunications, 7(3), 207-221. [5] Mason, R., and Kaye, A. (1990) Towards a new paradigm for distance education. In L. Harasim (Ed.), Online education: Perspectives on a new environment. New York: Praeger, 15-38. [6] Scardamalia, M., and Bereiter, C. (1996) Computer support for knowledge-building communities. CSCL: Theory and Practice of an Emerging Paradigm, ed. T.Koschmann. NJ: Lawrence Erlbaum Associates, 249-267. [7] Wu, C.-C., and G.C. Lee. (1999) Use of BBS to Facilitate a Teaching Practicum Course. Computer & Education, 32: 239-247. [8] Stacy, E. (1999) Collaborative Learning in an Online Environment. Journal of Distance Education, 14(2).

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

647

Applying Weighted Learning Object to Build Adaptive Course in E-learning Anh Nguyen Viet, Dam Ho Si (advisor) College of Technology- Vietnam National University, Hanoi [email protected] Abstract: This research focuses on learning path selection – a part of work in my doctoral research whose goal is to build an adaptive hypermedia. To generate adaptive learning path for each learner, we base on learner’s profile in which learner’s model is evaluated as well as stored. We have supplemented a set of weight for each learning object (LO) which is basic for selecting learning path process. In this paper, we promote a method to select LO in order to create learning path for each learner and summary our experiment results. Keywords: weighted learning objects, learner modeling, and adaptive hypermedia.

Introduction Adaptive course content is an important phase in many processes to build adaptive hypermedia system. Based on evaluated results of learner’s features such as goals, knowledge, background, hyperspace experience, and preferences, adaptive learning path will be created for each learner. There are many methods as well as techniques to build adaptive hypermedia system [1], but a little research is focusing on learning object selection. Some latest researchs, for example, an algorithm for shortest learning path selection only based on time to finish the course was promoted [5. Also based on this algorithm, adaptive learning objects sequencing have been addressed [4]. In addition, focuses on rating as well as selecting learning object has obtained results [2, 3]. In this paper, we address the problem of selecting learning path which is based on all learners’ features (stored in learner’s profile). The main part of the paper describes framework of my adaptive hypermedia system as well as how to select learning path. In this paper we also describe work in progress and methodology. The final part discuses our results, some unsolved problems and proposed future work. 1. Framework Based on method and techniques of adaptive hypermedia that Brusilovsky promoted [1], our designed model includes three modules: learner module, content module and view module. The first module manages learner modeling as well as profile of them. The second module generates suitable learning path for each learner based on learner‘s profile. The last module represents suitable course outline for each learner. In our model, learner’s profile is always updated so the course content as well as structure adaptive through the course. 2. Work in Progress In this phase, we focus on designing the content module. The main point is learning path generating process. To do this, the course’s material such as text, images, video is stored in learning object database. We supplement some attributes for each LO, at this time we added seven attributes. With each attributes we assign a value of [0, 1] so each LO has set of value correspondingly. The value of them has been assign by teacher’s idea. To select learning path, we use a function to evaluate learning object based on value

648

A.N. Viet and D.H. Si / Applying Weighted Learning Object to Build Adaptive Course

of its attribute. Based on function result – a threshold, the LO will be selected or not. This process also includes the evaluation. Some advantages of this design are considering many aspect of the learning object to evaluate so the selection is better than using separate LO attributes. In addition, using a little attribute of LO (around 20 attributes) satisfying real – time selection. The disadvantage, however, selected LO can not match with learner’s profile in some case because using all of LO attributes. We will improve the select function and can use open learner modeling so learners can adjust learning path themselves based on automatically generated learning path. 3. Methodological issues In this area, alongside work previously completed, there is a plenty of literature available from consolidated domains such as Adaptive Hypermedia, Learner Modeling, Knowledge Management, and so on. The project will resource to diverse methodologies, considered suitable for its different features: action–research, case studies, probability statistics model and so on. A prototype system will be developed which will aim to generate a course for learners based on their modeling that used learner modeling theories. Probability statistics model will be used for assigning value for LO. Using Bayesian Belief Networks to decide LO will be selected or not. The test result of system will be interpreted, and recommendation set will be developed. 4. Conclusion This research focused on learning path selection work which based on weight learning object and learner’s modeling. With supplementing some attributes for learning objects, the learning path is so suitable enough for learner participating in. Our project is still at an early stage, we are now improving the select function in order to choose LO which will meet all learner requirements. By the time, this conference takes place we believe there will be some new results to share with the audience. Acknowledgments This work is partly supported by the research project No. QC.06.08 granted by Vietnam National University, Hanoi. Selected References [1] Brusilovsky, P. (1996) Methods and techniques of adaptive hypermedia. User Modeling and User Adapted Interaction 6(2-3), 87 -129. [2] Phythagoras, K. and Samson, D. (2004) Adaptive Learning Object Selection in Intelligent Learning System. Journal of Interactive Learning Research 15(4), 389-407. [3] Kumar, V. (2005) Rating Learning Object Quality with Distributed Bayesian Belief Networks: the why and how. Proceedings of the Fifth IEEE International Conference on Advanced Learning Technologies. [4] Phythagoras, K. and Samson, D. (2006) Adaptive Learning Objects Sequencing for Competence – Based Learning. Proceedings of the Sixth IEEE International Conference on Advanced Learning Technologies. [5] Zhao, C. and Wan, L. (2006) A Shortest Learning Path Selection Algorithm in E-learning. Proceedings of the Sixth IEEE International Conference on Advanced Learning Technologies.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

649

Annotation in Information Research for Decision Making a

Robert Charlesa, David Amosa LORIA - Campus Scientifique, Vandoeuvre-lès-Nancy, France {abiodun-charles.robert}{amos.david}@loria.fr

Annotation tools are very vital in information management and collaborative works. The importance of collaboration for information is vital in research work. Information has to be accessed if it must be used. Annotation provides a means of access to information. One of the earlier proponents of collaborative work emphasized the importance of access to information in a collaborative environment (Vannevar, 1945). Internet technology, which is one of the most outstanding platform for today’s communication is favouring the use of annotation tools to resolve information research problems. Several free annotation tools are available for co-operative work and for personal employment. These tools were developed based on specific local needs or to cater for a generalized application. Annotation can be made to include the parameters of the creator, the document and time. Based on these, our objective is an annotation tool that will permit its exploitation for information research. Several annotation tools and models were considered. These include AMAYA, CritLink, YAWAS, Third voice, CoNote, GrANT and Commentator. The concerns of most of these tools were based on the possibilities and the methodologies involve in annotation creation. The primary objective of most of them is to see how the traditional way of annotating document can be transfer to electronic documents. Some of these tools are platform dependent while others are application dependents. Our concern is not on how annotation can be made but how annotation made on electronic documents can be used to enhance information research in economic intelligence processes (decision making processes). We defined economic intelligence as “all the coordinated actions of collection, processing and distributing of useful information for the economic actors with the aim of its exploitation. These actions are taken legally with all the guarantees of protection necessary for the conversation of the company’s patrimony, in the best conditions of quality, of delay and of cost”. I a simple term, we can see it as information for action. Economic intelligence is similar to other terminologies like “Business intelligence”, competitive intelligence. The difference between economic intelligence and business intelligence is that, from the perspective of economic intelligence, the word “economic” has a broad implication in the sense that information is valuable in all aspects of human endeavors. It then means that economic intelligence is the process of searching for information that can be applied to solve specific problem. In the case of business intelligence, the information sought or “packaged” is meant for business actions. We believe in two main functional actors in economic intelligence. These actors are the Decision maker and information workers. We may be able to subdivide these actors. Our attention is not on their character or task but on their functions in the process of decision making. Our objective is to see how some of the activities between the actors in economic intelligence processes can be performed with the help of an annotation model.

650

R. Charles and D. Amos / Annotation in Information Research for Decision Making

An annotation is seen in with three components: document time and the creator. The creator of an annotation is primarily a document user. These component can be kept is mind as annotation is been made for economic intelligence purposes. A document user is a potential annotation maker. He was characterized from his personal identity and his information needs. Every annotation maker makes annotations based on his own experiences in life and understanding of the document. Document is at the centre of any annotation. A document was defined in its broad sense to mean a container of information. More specifically, we defined it as a trace of human activities. Bibliographic parameters of document were used in our study to characterize documents for annotation. A model AMIE (Annotation Model for Information Exchange) was built based on these parameters. After the modeling, we evaluated annotation based on the predetermined parameters. We compared annotation in a classical system to that conceived for decision making. We found out that, in a classical annotation, context, objective of annotation and document properties are generally used in the production of annotation. Normally annotation resulting from these can be used for evaluation or interpretation of the document. But in a case where annotation is to be used for decision support, document parameters will not be enough to qualify the resulting annotations for strategic decision. In our approach, added to the fact that annotation has a link to the source document, we believe that it also has characteristics that are associated with the maker of the annotation (annotator) and the time when the annotation was made. It is the interaction of the annotation property, time, document and the characteristics of the annotator that makes it viable for use in decision making processes. We implemented the first version of this conception on the Web. We created a MySql database on a MySql server. In the database, we had tables that store information on user, annotation and session. The database was linked to existing bibliographic database in our research team. Our attempt was to use the bibliographic database as document source for annotation. In the implementation, it was still possible to use other documents outside the bibliographic database, for example personal Web pages. Interface to the database was implemented with PHP and HTML. We evaluated the importance of this model with masters’ students in scientific and technical information of the University of Nancy 2, France and bachelor students of computer science from university of Ibadan. We also evaluated the system with the general public in Indonesia, France, Canada, USA, Kuwait, Japan, Nigeria and Canada.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

651

Effects of the Use of Graphic Calculators on Cognitive and Metacognitive Domains in Teaching and Learning of Mathematics Mohd. Tajudin, Na, Ahmad Tarmizi, Rb, Wan Ali, W.Zb, Konting, M.Mb a Universiti Pendidikan Sultan Idris, Perak, Malaysia b Universiti Putra Malaysia, Selangor, Malaysia

Introduction To date, there is a substantial body of research into the use of graphics calculators. However, although this technology has been available for nearly two decades, research on the use of this technology is not robust and there are still inconsistencies in research findings of this area. Its use in secondary classrooms is not well understood, universally accepted, nor well-documented. Further, current literature on the graphic calculator ultimately gave rise to questions and criticisms. For example, critical reviewed by Penglase and Arnold (1996) concerning the methodology used found that the use of experimental and control groups fails to look into issues pertaining to the relationship between the use of the graphic calculator and the important influences such as upon conceptual understanding. In fact, Dunham and Dick (1994) also argue that research on graphic calculator has been descriptive rather explanatory. Recently, a brief overview of literature related to the nature of graphic calculator research by Berger (1998) found that there is a scarcity of research directed towards an explication or interpretation of how the graphic calculator functions as a tool for learning. Furthermore, after surveying more than a decade of mostly positive research on the use of graphic calculators, Milou (1999) found that research into graphing calculators is still in its infancy. He posited that much more research is required to better understand the interaction between graphic calculators and the classroom environment. The usage of graphic calculators in Malaysian schools is still in the early stage. Further, limited studies on using graphic calculators in teaching and learning of mathematics in Malaysian school were done and if any, they were not carried out in depth. Thus, there is a need to further research in this area in the context of teaching and learning of mathematics at Malaysian secondary school level. Purpose of the Study The main purpose of the study is to compare the effect of using graphic calculator strategy(GCS) and conventional instruction method(CIM) in teaching and learning of mathematics on Form 4 secondary school students on the cognitive and metacognitive domains. The cognitive domain comprised of students’ performance. The metacognitive domain comprised of `students’ metacognitive awareness. The study will also compare between GCS group and CIM group in teaching and learning of mathematics with respect to measure of mental effort and instructional efficiency. In addition, the study is also to obtain students’ view on the use of graphic calculators in teaching and learning mathematics. Methodology The research study employed the quasi-experimental, non-equivalent control group design with proxy pretest measures. This design was chosen because only experimental data can

652

N. Mohd. Tajudin et al. / Effects of the Use of Graphic Calculators

conclusively demonstrated causal relations between variables. Data for this study will be collected from a progressive series of three experiments. Since it would not make sense to give the posttest to students who have never had learnt the topic, the researcher used the monthly or performance test as proxies for the pretest. The monthly or performance test was used to determine whether the groups were similar in ability. Otherwise, the proxy pretest will be used as a covariate. The target population for this study was Form 4 students whilst the accessible population was Form 4 students from one selected school in District of Klang, Selangor and one selected school in District of Alor Gajah, Malacca. The sampling procedure used in this study was purposive random cluster sampling. The instructional materials for the experiment were consisted of 15 sets of lesson plans. The main feature of the acquisition phase for the experimental group was that students used “balanced approach” in learning the Straight Lines topic. The “balanced approach” is an appropriate use of paper-and-pencil and calculator techniques on regular basis. The control group’s students were also guided by the same instructional format with one exception. It is a whole-class instruction which will not incorporate the use of TI-83 Plus graphic calculator. The instruments in this study consisted of a Straight Lines Achievement Test (SLAT), a Paas Mental Effort Rating Scale(PMER), a Metacognitive Awareness Survey(MCSS), and a Graphic Calculator usage Survey(GCUS). The SLAT was designed by the researcher to measure students’ performance in the Straight Lines topic. The PMER was used to measure cognitive load. It is a 9-point symmetrical category Likert scale on which subject rates their mental effort used in performing a particular learning task. It was introduced by Paas (1992) and Paas and Van Merrienboer (1994). The MCAS) was designed by the researcher using the State Metacognitive Inventory adapted from O’Neil & Abedi (1996), the metacognitive component of traits thinking questionnaire adapted from O’Neil & Schacter (1997), and the key operations of metacognition by Beyer (1988). The GCUS was also prepared by the researcher to determine students’ of GCS group views about the graphic calculator usage in teaching and learning of mathematics Current Status This doctoral work has been carried out since June 2004 and this work is expected to finish in June 2007. So far, two experiments were carried out. The results of experiments indicated that the use of GC in teaching and learning of mathematics could be helpful in improving students’ performance. However, the results of the analysis of the MCAS indicated that using GC in teaching and learning of mathematics did not significantly improve students’ metacognitive skills. For the second experiment, the researcher emphasized on the cognitive load theory (CLT) (Sweller, 1988; Paas, Renkl & Sweller, 2003) which focuses on the role of working memory in the development of instructional methods. The findings also provided some evidence that the use of GC can be helpful in improving performance in a Straight Lines topic, by reducing the levels of cognitive load. This study provides some considerations on highlighting the concept of CLT when designing the instructional method which takes into account the limitations of working memory. The third experiment will be carried out in January 2007. It replicates the second experiment with several changes made. The methodology of the study will be revised. Further, the materials and instruments used in the study will be altered according to the comments and opinion from the validators.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

653

Improve Effectiveness of Dialogue in Learning Communities Jingyu Yang, Kinshuk Department of Information Systems, Massey University, New Zealand [email protected] Abstract: In a learning community, conventional discussion forums are integral to web-based interventions in traditional classrooms as well as on-line learning environments. Despite popular belief that they are a great success in fostering deep and meaningful discussions and support active learning; research has found that there are millions of messages posted by users to express such an opinion, but it is hard to be directly delivered to all users. Finally there are millions of postings in databases across the country stored away and never reused. This paper introduces a PhD student’s current in-progress research work. It proposes a distributed intelligent discussion forum system dedicated to supporting both students and teachers. The system is developed with the primary goal of reducing the number of problems associated with conventional discussion forum systems in web-based environments and improving the effectiveness of dialogue between students with each other and with teachers so that it can enhance each individual student’s ability to share and learn knowledge. Keywords: Guidelines Distributed Systems, Information Retrieval, Discussion Forum, Leaning Community

Introduction

The use of web-based technologies like asynchronous discussion forums to enhance traditional classrooms and web-based distance learning environments is growing exponentially with no limits in sight. The discussion forums foster a sense of community and can be a powerful resource for learning, if the instructor knows how to encourage thoughtful postings and students are engaged in high quality discussions (Blignaut & Trollip, 2003). This research aims to adopt advanced information retrieval technologies and user-based community theories to develop a value-added distributed interaction environment for students to effectively capture and retrieve knowledge not only from the local learner community but also from the wider Web community. In the DIDF system, when someone is submitting a query, the system will parse the submitted message and use the following three steps to process the most relevant response (Figure 1). In the first step, we use single words as the basic units to represent text. Each text can be pre-processed in the following steps: • Punctuation marks are separated from words. • Numbers and punctuation marks are removed.

654

J. Yang and Kinshuk / Improve Effectiveness of Dialogue in Learning Communities

All words are converted to lower case. Words like prepositions, conjunctions, auxiliary verbs, etc., are removed. A version of stop-list (290 stop words) can be used. The remaining words after pre-processing are potential candidates for use as features, with each word as one feature in the feature vector. In the second step, according to the filtered keywords, the system will adopt the LSI scheme to retrieve all related information from three information resources, the local database, the multiple-institution’s database and the Web community information resources which the URLs are linked by instructors. For the third step, the system will automatically respond all related FAQs and information to learners where the related information is from the local database, distributed remote databases, and the Web community resources. • •

2. Conclusion and Future Work The DIDF is an innovative tool that builds on many existing methodologies and Web technologies. The product has great potential to reduce the numbers of problems associated with conventional discussion forums in a web-based environment and can be a boon to teachers who like to use the previous discussions to frame new and interesting discussions.

Automatic Responses

Posting by Teachers and Students

DIDF User’s Ratings

Evaluate Best Match

Check against 1. Keywords 2. VSM 3. User’s rating

Teacher

Student Store postings in the database Related information from different institution’s’ database and Web communities

Acknowledgement Part of this PHD study has been supported by the eLearning Collaborative Development Fund of Tertiary Education Commission of New Zealand. References [1] Blignaut, S. & Trollip,S.R. (2003). Developing a taxonomy of faculty participation in asynchronous learning environments – an exploratory investigation. Computers and education 41 (2003) 149 -172.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

655

! " # $ &

! "

!

! $ \ ^^ ^$

` | \ !

|

' *+

|

/ 9

`

| !

|

^ !

656 S.A. Noraidah et al. / Validation of the Mathematics Courseware Usefulness Evaluation Instrument

| ^ ^ ! |

$ ;

$

` ` ` ` | | | ^ $ ^$ $ ^$ ^ ! ^ ^ ^ ^ ! $ $! $ $^^ ^$ ^^ ^ ^ ^ $! !! ^ | | ^ ^ | ^ | ` $ ! $ ! ! ` ! | ` | ¡ ` | ` | ` ` ¡

<

| | | |

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

657

Expert Tutoring and Natural Language Feedback in Intelligent Tutoring Systems Xin LU Department of Computer Science, University of Illinois at Chicago, U.S. [email protected]

Abstract: This paper describes a comprehensive study of expert versus non-expert tutoring and a baseline intelligent tutoring system which provides different kinds of feedback. It proposes a method to computationally model expert tutoring and a framework of effective natural language feedback generation with 3-tier probabilistic planning. Keywords: intelligent tutoring system, natural language, expert tutoring

Introduction As computers have become widespread in recent years, people recognize that intelligent tutoring systems (ITSs) can provide the great benefits of one-on-one instruction with lower cost and more flexibility in time and location. However, current ITSs are still not able to provide learning as effective for the users, as expert human tutors do. To bridge the gap between current ITSs and human tutors, previous studies proved that a natural language (NL) interface could be one of the keys[1]. The community is still investigating what type of NL feedback, and when and how to deliver it in ITSs effectively. Two questions still need to be answered: what makes human tutoring effective? what is the most appropriate and convenient way to implement effective tutoring language? This paper aims at answering these two questions.

1. Methods and Results Our tutoring domain concerns extrapolating complex letter patterns[2], such as inferring EFMGHM from ABMCDM. We collected tutoring dialogues with three tutors, one expert, one novice, and one lecturer who is experienced in teaching, but not in one-on-one tutoring. Each subject interacted with one of the tutors before doing two post-test problems. The post-test performance of subjects shows that the expert tutor engenders much better learning outcomes than the non-expert tutors[3]. So investigating what makes expert tutoring more effective would contribute to answering the first question. We transcribed some of the tutoring dialogues and annotated tutor and student moves. After analyzing moves and interaction patterns, we found that some behaviors of our tutors support the predictions on effective tutoring from literatures[4]. For example, subjects with the more experienced tutors explain more than the subjects with the novice tutor. However, there is a finding which contradicts the predictions: the expert prompts his students less than the lecturer does. We also found some significant differences in interaction dialogue patterns between all the tutors. For example, the expert does not answer immediately or directly to a student’s question but the novice tends to immediately deliver the knowledge.

658

X. Lu / Expert Tutoring and Natural Language Feedback in Intelligent Tutoring Systems

In the same domain, I developed four versions of the ITS with different kinds of feedback provided to the student. In the no feedback version, each letter the subject inputs turns blue, with no indication and no message regarding whether it is correct or incorrect; in the neutral version, the only feedback subjects receive is via color coding, green for correct, red for incorrect; in the positive version, they receive feedback via the same color coding, and in addition, verbal feedback on correct responses only; in the negative version, they receive feedback via the same color coding, and in addition, verbal feedback on incorrect responses only. The language in the positive and negative conditions was inspired by (but not closely modeled on) the expert tutor’s language. We ran a between-subject experiment, in which each group of subjects interacted with one of the systems. We found that, even if subjects with the positive version do perform slightly better, these differences are not significantly.

2. Conclusions and Future Research Plans This study has two goals: computationally modeling expert tutoring and effective tutorial feedback generation. For the first goal, I have completed a comprehensive study of the differences between expert and non-expert tutors in effectiveness, tutor and student moves and interaction patterns. However, I am still halfway through a computational model of expert tutoring. To accomplish this goal, I am going to study human tutoring dialogues further and employ a machine learning method -- Classification based on associations[5] -to learn tutorial rules for generating effective NL feedback in ITSs. An association rule is a pattern that states that the features and the targets occur with certain probabilities. For the second goal, I have developed a baseline intelligent tutoring system and evaluated four versions of the system which differ in the types of feedback they provide the student. To generate more sophisticated and effective NL feedback, I am planning to use Midiki (the MITRE Dialogue Kit) as the system shell of my NL feedback generator. Midiki is based on the Information State theory of dialogue management, which identifies the relevant aspects of information in dialogue, how they are updated, and how updating processes are controlled[6]. There will be three modules in the generator: the plan module which includes 3 tiers (plan generation, plan selection and plan monitoring); the update module; the feedback generation module.

Acknowledgments This work is supported by grant N00014-00-1-0640 from the Office of Naval Research.

References [1] Fox, B. (1993) The Human Tutorial Dialogue Project. Lawrence Erlbaum Associates, Hillsdale, NJ. [2] Kotovsky, K. and Simon, H. (1973) Empirical Tests Of A Theory Of Human Acquisition Of Information-Processing Analysis. British Journal of Psychology, 61, 243-257. [3] Di Eugenio, B., Kershaw, T. C., Lu, X., Halpern, A. C. and Ohlsson, S. (2006) Toward a Computational Model of Expert Tutoring: a First Report. 19th International conference of FLAIRS, Melborne Beach, FL. [4] Chi, M. T., Siler, S. A., Jeong, H., Yamauchi, T. and Hausmann, R. G. (2001) Learning From Human Tutoring. Cognitive Science, 25, 4, 471-533. [5] Liu, B., Hsu, W. and Ma, Y. (1998) Integrating Classification and Association Rule Mining. Knowledge Discovery and Data Mining, New York, NY, 80-86. [6] Larsson, S. and Traum, D. R. (2000) Information State And Dialogue Management In The TRINDI Dialogue Move Engine Toolkit. Natural Language Engineering, 6, 3-4, 323-340.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

659

Learning Environment for Designing Physics Experiment: DEEP Takahito TOUMOTOa, Tomoya HORIGUCHIb, Tsukasa HIRASHIMAa and Akira TAKEUCHIc a Faculty of Engineering, Hiroshima University, Japan b Faculty of Computer Science and Systems Engineering, Kobe University, Japan c Graduate School of Computer Science and Systems Engineering, Kyushu Institute of Technology, Japan [email protected] Abstract: In this paper, we explain the importance of the task to design physics experiment in learning of physics. The design and development of a learning environment that supports a learner to design physics experiments, which we call "Designing Experiment Environment for Physics: DEEP", and the preliminary evaluation are also reported. Keywords: learning physics, virtual experiment environment, design experiment

1. Introduction The purposes of experiments in learning physics are categorized in two forms: (1) to let a learner discover (or rediscover) physical knowledge by him/herself, and (2) to let a learner use the knowledge that is already acquired from the class lectures to deal with a concrete situation. In the field of computer based learning, there are many investigations for the former purpose [1, 2], and its importance is obviously. However, although there are few ones for the latter, it is also important. A learner cannot always use physical knowledge to deal with concrete situations even when he/she can use it to solve exercise problems. If he/she cannot use the knowledge in the situations, his/her understanding of the knowledge is not enough. The latter purpose is important as the opportunities to use the knowledge in the situations. In this paper, we explain the task for the latter type of experiment. The design and development of the learning environment and its evaluation are also reported. In the experiment to let a learner use the knowledge in concrete situations, he/she is usually required to derive an attribute value by measuring several attribute values in the situations and calculating the required value by using the measured values with several formulae. In the process, adequate set of formulae is selected from his/her physical knowledge to derive the required value depending on the situation. Besides, methods to measure values are decided depending on the situation. Through this task, it is expected that a learner acquire the way to use his/her knowledge in concrete physical situations. In this paper, we call this task "Designing Physics Experiment: DPE". In the process to carry out this task, a learner may make errors. Because the process is highly individual one, it is difficult to support the errors or impasses adequately in usual experiment in classroom. We, therefore, are investigating a computer-based learning environment for learning by DPE. In the next section, the design and development of the environment are described. We call this learning environment as "Designing Experiment Environment of Physics: DEEP"

660

T. Toumoto et al. / Learning Environment for Designing Physics Experiment: DEEP

2. Overview of DEEP and Experimental Use The architecture of DEEP is shown in Figure 1. We have implemented four kinds of interfaces in DEEP: Problem Presentation, Combining Formulae, Measuring and Substituting. The latter three of them are for the steps of DPE. Problem Presentation Interface presents the definition of problems graphically. DEEP's diagnosis module uses Solution Structure which is automatically generated by inputted measuring tools, a required attribute value, and Phenomenon Structure [3]. Phenomenon Structure presents all numerical relation of attributes in the physical situation. When an error is detected in the steps in DPE, DEEP helps the learner to correct it. Measuring Tool's Skill

Measuring Tools Solver

Usable Measuring Tools Physical Situation

Problem Definition

Phenomenon Structure

Measurable Attribute Values Solution Structure

Solution Generator

Required Attribute Value Diagnosis

Interfaces Problem Presentation

Combining Formulae

Module

Measuring

Substituting

Fig. 1. Architecture of DEEP We let 15 university students use DEEP in 50 minutes. In the experimental use, they designed an average of 2.6 correct physics experiments. In the paper test of DPE carried out before the use, 10 students designed only an average of 1.3 correct experiments in the same time. The environment, therefore, seems to be useful for designing correct physics experiments. Besides, result of questionnaires suggests that most students accepted the environment as effective for physics learning.

3. Conclusions In this paper, we described the design and development of a virtual experiment environment to support learning by DPE, and we reported the result of experimental use of the environment. Although this type of experiment is one of the most important experiments in educational context, the experiment may not be performed sufficiently because of the difficulty and necessary of highly individual diagnosis. Therefore, the adaptive supporting for each learner is an important problem. So we have developed DEEP which is a virtual experiment environment including interfaces and diagnoses for three following steps: (A) combining formulae, (B) measuring values, and (C) substituting the measured values for the formulae. In the result of experimental use of DEEP, students who had used DEEP were able to design the correct experiment more than students who had not use it. The result suggests that the environment is useful for designing correct physics experiments. Besides, the result of questionnaires suggests that the environment is useful for learning physics. References [1] [2]

[3]

White, B. Y. and Frederiksen, J. R. (1990) Causal model progressions as a foundation for intelligent learning environment, Artif. Intell, vol. 42, pp. 99-157. Reimann, P. (1991) Eliciting Hypothesis-Driven Learning in a Computer-Based Discovery Environment, in Intelligent Learning Environment and Knowledge Acquisition in Physics, eds., A. Tiberghin and H. Mandl, pp. 137-150. Hirashima, T., Niitsu T., Kentaro, H., Kashihara, A., and Toyoda, J. (1994) An Indexing Framework for Adaptive Arrangement of Mechanics Problems for ITS, IEICE TRANS. INF. &SYST., VOL. E77-D, NO. 1.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

661

Standards, Adaptation & Pedagogy: Quality Assessment in e-Learning Silvia Sanz-Santamaríaa, Julián Gutiérrez Serranob, José A. Vadillo Zoritab a PhD Student, University of the Basque Country (UPV-EHU), Spain b Full Professor, University of the Basque Country (UPV-EHU), Spain [email protected] Abstract. This is a summary about a paper that presents a new authoring tool for e-learning assessment that lets teachers testing students efficiently. The combination of different research fields inside the tool, namely pedagogy, learner adaptation and standards, makes possible a better and complete assessment adapted to learners’ knowledge and preferences. Keywords: e-Learning, assessment, IRT, standards, pedagogy

Introduction The role of assessment in e-learning is a very important aspect. Assessment is invaluable as a way of affording students the opportunity to learn. In this respect it is certainly more significant for online courses than it is for traditional learning [1]. There are multiple factors that must be taken into account if we want to develop successful evaluations. Pedagogy is one of them [2], but is important to consider other ones like adaptive testing, which provides a lot of advantages in assessment [3, 4], or standards, another outstanding area if we talk about e-learning tools. This paper presents a summary about an e-learning assessment authoring tool that is being implemented in a PhD. Thesis. This authoring tool lets teachers develop items and tests taking into account pedagogy, adaptation and standards, the three factors commented before that influence on e-learning assessment.

1. An Authoring Tool for e-Learning Assessment Before working on the implementation of the authoring tool, we carried out a review work about other assessment tools (MicroCAT, <e-aula>, TestEditor, The KOD Project, Moodle, WebCT, and more). We focus the research in 3 relevant factors related to e-learning assessment: adaptation, standards and pedagogy. This review shows that the existing tools have developed different advances in some of the aspects commented before, but there is no one that combines all of them. Our work consist on covering this hollow developing an item/test authoring tool for generating adaptive assessments, using standards and paying special attention to Pedagogy. The next paragraphs explain briefly the characteristics of the new authoring tool in relation with these three aspects. The authoring tool we are developing makes possible the coexistence of classic tests (predefined and dynamics) with adaptive ones. We want that teachers profit from IRT [5] advantages without deep knowledge about it. We apply IRT with three parameters

662

S. Sanz-Santamaría et al. / Standards, Adaptation & Pedagogy

(difficulty, discrimination and pseudo-guessing) because the use of four parameters does not cause a big improvement in the adaptation level [6]. Once a teacher has developed some items/test, or even for checking the tool, it could be very interesting for teachers to share their items/tests with other colleagues, as well as import items/tests from them. We offer this possibility by using the IMS QTI Standard [7]. We chose QTI because is the ‘de facto’ standard for assessment in e-learning. In addition, the last version (2.0) allows the definition of multiple types of items, necessary for a complete assessment. Finally, with the aim of provide the tool with a pedagogical component we are working in the development of an on-line help guide that lets teachers to design good items to achieve the pedagogical objectives fixed. Also, we want to profit from the structure of the pedagogical domain in which the assessment takes part to achieve a better assessment. In this respect, we are working on relating items with: links (relations between concepts) and composed nodes (a group of nodes that are related). Even more, we want to develop different items for the same concept so that each item works different cognitive levels [8] about that concept. In short, the tool we are implementing lets teachers working with a generic tool (applicable to any domain) for testing learners. They can easily develop classic and adaptive tests without deep knowledge about pedagogical adaptive theories (IRT), just filling a form. It is possible to create and update items and tests for fitting them to the necessities of each moment. Also, and thanks to the IMS QTI standard, teachers can import and export items and tests from/to other systems and tools that follow the same standard, adjusting some parameters of them if it was necessary. The tool is prepared to house multiple item types for a better adaptation to the learner knowledge and preferences. Even, the authoring tool is going to provide a guide for helping teachers to decide which type of item is better for achieving some skills and how to develop good assessments.

References [1] Macdonald, J. (2004) Developing competent e-learners: the role of assessment. Assessment & Evaluation in Higher Education, 29, 2, 215-226. [2] Govindasamy, T. (2002) Successful implementation of e-Learning Pedagogical considerations. Internet and Higher Education, 4, 287-299. [3] Olsen, J. B., Cox, A., Priece, C., and Strozeski, M. (1989) Development, implementation and validation of a predictive and prescriptive test for statewide assessment. AERA Meeting, San Francisco. [4] Hambleton, R. K., Swaminathan, H. and Rogers, H. J. (1991) Fundamentals of Item Response Theory, volume 2. Kluwer Academic Publishers: Norwell, Massachusetts (USA). [5] Weiss, D. J., and Yoes, M. E. (1990) Item Response Theory. Advances in Educational and Psychological Testing, Kluwer Academic Publishers: Dordrecht (The Netherlands), 69-95. [6] Barton, M.A., and Lord, F. M. (1981) An upper asymptote for the three.parameter logistic item-response model Research Bulletin 81-20. Princeton, New Jersey: Educational Testing Service. [7] IMS Question & Test Interoperability Specification, http://www.imsglobal.org/question/index.html. [8] Bloom, B. S. (1956) Taxonomy of Educational Objectives, Handbook I: Cognitive Domain. David McKay Company Inc., New York, USA.

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding R. Mizoguchi et al. (Eds.) IOS Press, 2006 © 2006 The authors. All rights reserved.

663

Author Index Abdul Azim, A.G. Abdul Jalil, H. Abdullah, N. Ahmad Tarmizi, R. Akahori, K. Alwi, N.H. Amos, D. Au, W.K. Bader, T. Bourdeau, J. Brine, J.W. Brudvik, O.C. Calinger, M. Chan Mow, I. Chan, T.-W. Chang, C.-Y. Chang, N. Charles, R. Chee, Y.S. Chen, F. Chen, F.-C. Chen, L.-J. Chen, M.C. Chen, N.-S. Chen, W. Chen, Z. Chenyuan, T. Cheung, K.K.F. Cheung, S.H. Chung, T. Concannon, F. Dai, H. dan Aida Suraya, M.Y. Davies, C. de Jong, T. Deng, L. Dillenbourg, P. Dung, J.-J. Dwyer, N. Ejima, T. Eto, K. Feng, J. Gongyi, Y.

655 611 543 651 81, 171 443 649 55 621 37 341 133, 141 89 55 561 491 335 649 133, 141, 495 7 587 629 257, 327 267 149 225 225 503 349 457 357 409 655 283 3 473 v 267 595 577 179 383 7

Gotoda, N. 477 Guangzuo, C. 7 Guo, L. 141 Gutiérrez, A. 643 Hamzah, A. 443 Han, K. 71 Hanafusa, Y. 499 Harada, M. 383 Hasan, S. 655 Hawryszkiewycz, I. 603 Hayashi, T. 499 Hayashi, Y. 37 Hirashima, T. 127, 483, 535, 659 Hirayama, K. 275 Hishiyama, R. 363 Ho Si, D. 647 Ho, R.-G. 629 Hoe, W.-M. 417 Hong, K.S. 133, 141, 495 Hongtao, S. 409 Horiguchi, T. 659 Hsu, J.-M. 201 Hu, C. 7 Huang, G. 447 Huang, H.-M. 509 Huang, R. 217, 229, 379, 517, 569, 645 Hughes, S. 451 Hung, V. 389 Hwang, W.-Y. 267 Ikeda, M. 29 Inoue, T. 499 Ip, W.-h. 439 Ishida, T. 363 Ishigami, M. 315 Ishima, N. 115 Isotani, S. 193 Itoh, Y. 115 Iwane, N. 275 Jeffery, L. 157 Jiang, H. 185 Jiang, H.-M. 587 Jiang, L. 405

664

Jiuling, G. Jong, M.S.Y. Juang, Y.-R. Kabasawa, Y. Kagawa, K. Kakehi, M. Kanbe, A. Kanenishi, K. Kang, M.J. Kasai, T. Kashihara, A. Kato, H. Kato, S. Kato, Y. Khandaker, N. Kikukawa, I. Kim, B. Kinshuk Kitazawa, T. Kojima, K. Kojiri, T. Kong, S.C. Konishi, T. Konting, M.M. Ku, H.-H. Kume, I. Kunichika, H. Kwok, P.L.Y. Lai, Y.-S. Lailin, H. Langlotz, A. Law, H.-Y. Le, N.-T. Lee, E. Lee, F.-L. Lee, J.H.M. Lee, Y. Lee, Y.-W. Leng, J. Leong, M.K. Leung, H. LeVoi, M. Li, C. Li, J.-L. Li, K. Li, M. Li, S.S.-c. Li, W.-X. Li, Yanyan

7 503, 525 561 179 551 237 275 413, 477 383 397 465 81 171 171 185 289 89 157, 653 81 123 237 241, 439 115 651 423 75 535 349, 431 201 555 119 525 63 71 209, 439, 503, 525 503, 525 71 587 229 457 107 357 45 637 353 233 439 637 217, 229, 569

Li, Yongna Liao, J. Liaw, S.-S. Lin, X. Littleton, K. Liu, C.-M. Liu, F. Liu, I-F. Liu, T.-C. Liu, Y. Liu, Z. Loo, J.P.L. Looi, C.K. Lu, Xin Lu, Xueming Luk, E.T.H. Majima, Y. Matsubara, Y. Matsui, T. Matsuura, K. McConnell, D. Medina, R. Menzel, W. Miell, D. Mitsuhara, H. Miwa, K. Miyadera, Y. Miyake, N. Miyata, H. Mizoguchi, R. Morimoto, Y. Nagai, M. Nagano, K. Nagaoka, K. Nakaya, M. Nakayama, M. Nguen Viet, A. Niki, K. Nishihori, Y. Nishinaga, N. Nitta, N. Nor, H. Mohd. Noraidah, S.A. Nozaki, H. Okamoto, M. Ota, K. Othman, T. Ou, Y. Pan, W.

323 217, 229 509 97 357 201 49 257, 327, 331 561 495 175 457 149 657 323 503 625 275 179 477 205 595 63 357 413 123 289 15 315 v, 37, 193, 397 289 81 397 383 465 521 647 477 383 383 75 443 655 577 127 465 543 209 603

665

Peña, A. 643 Peng, H. 517 Pun, S.-w. 309, 439 Qi, L. 167, 405 Rao, L. 323 Reese, D.D. 89 Santiago, R. 521 Sanz-Santamaría, S. 661 Sasaki, S. 261 Sato, H. 383 Schiltz, G. 119 Serrano, J.G. 661 Seta, K. 29, 625 Shah, C.A. 543 Shang, J. 503, 525 Shen, L. 233 Shen, R. 383 Smestad, Ø. 157 So, W. 389 So, Y. 625 Soh, L.-K. 185 Sossa, H. 643 Sun, Y. 257, 327 Suthers, D. 21, 595 Tajudin, N. Mohd. 651 Takemura, Y. 75 Takeuchi, A. 127, 483, 535, 659 Takito, S. 577 Tan, I. 417 Tanaka, K. 383 Tang, K.-T. 107 Tengku Shariman, T.P.N. 611 Tominaga, H. 499 Toumoto, T. 659 Tse, K.-l. 309 Tse, W.C.A. 209 Turk, D.C. 341 Ueda, T. 115 Ueno, M. 289 Umeda, K. 577 Umetsu, T. 483 Vadillo Zorita, J.A. 661 Vass, E. 357 Vatrapu, R. 595 Wan Ali, W.Z. 443, 651 Wan, L. 167, 405 Wan, T.-y. 621 Wang, B. 323 Wang, F.-l. 637

Wang, H. Wang, J. Wang, Lu Wang, Lianghui Wang, R. Wang, Xiaodong Wang, XiaoYuan Wang, Ying Wang, Yonggu Wang, Youmei Watanabe, H. Watanabe, S. Watanabe, T. Wong, M.K.H. Wong, Y.F. Wu, H.-C. Wu, Y. Xie, Y. Xu, Y. Yamaguchi, H. Yamamoto, Y. Yamasaki, T. Yang, B. Yang, J. Yang, J.-C. Yang, Y.-L. Yano, Y. Yasuda, T. Yates, G.C.R. Yen, Y.-C. Yeung, Y.-y. Yin, R. Yip, W. Yokoi, S. Yokoyama, S. Yokoyama, T. Young, S.S.-C. Yu, P.-T. Yu, Y. Yuan, F. Yuan, J. Yueliang, Z. Yuen, A.H.K. Zhang, H. Zhang, JingJing Zhang, JinBao Zhang, X. Zhang, Y. Zhang, Z.

45 217, 229 409 97 413 233 447 517 353 297 261 225 237 503 431 491 167 305 447 397 383 499 323 157, 653 587 267 413, 477 371 55 629 249 305 389 371 289 127 331, 423 201 167 413 413 97 473 45 283 447 323 157 379, 645

666

Zhao, C. Zhao, J. Zhao, M. Zhao, W.

405 205 405 233

Zheng, J.J. Zhou, W. Zhou, Ying Zhu, Z.

233 371 217 v

This page intentionally left blank

This page intentionally left blank

Artificial Intelligence in Education (Frontiers in Artificial Intelligence and Applications)

Artificial Intelligence Research and Development (Frontiers in Artificial Intelligence and Applications, Vol. 146) (Frontiers in Artificial Intelligence and Applications)

Understanding and Facilitating Adult Learning

Information Modelling and Knowledge Bases XIX: Volume 166 Frontiers in Artificial Intelligence and Applications (Frontiers in Artificial Intelligenece and Applications)

Ontology and the Semantic Web: Volume 156 Frontiers in Artificial Intelligence and Applications (Frontier in Artificial Intelligence and Applications)

Algorithms and Architectures of Artificial Intelligence (Frontiers in Artificial Intelligence and Applications)

Advances in Logic, Artificial Intelligence and Robotics: Laptec 2002 (Frontiers in Artificial Intelligence and Applications, 85)

Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms (Frontiers in Artificial Intelligence and Applications)

Modular Ontologies (Frontiers in Artificial Intelligence and Applications, WoMO 2011)

Approximation Methods for Efficient Learning of Bayesian Networks: Volume 168 Frontiers in Artificial Intelligence and Applications (Frontiers in Artifical Intelligence and Applications)

Artificial Intelligence in Education: Building Technology Rich Learning Contexts that Work (Frontiers in Artificial Intelligence and Applications)

Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms (Frontiers in Artificial Intelligence and Applications) (Frontiers in Artificial Interlligence and Applications)

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding - Volume 151 Frontiers in Artificial Intelligence and Applications

Artificial Intelligence in Education (Frontiers in Artificial Intelligence and Applications)

Artificial Intelligence Research and Development (Frontiers in Artificial Intelligence and Applications, Vol. 146) (Frontiers in Artificial Intelligence and Applications)

Understanding and Facilitating Adult Learning

Artificial Intelligence Research and Development: Volume 163 Frontiers in Artificial Intelligence and Applications

Artificial Intelligence Research and Development: Volume 163 Frontiers in Artificial Intelligence and Applications

Information Modelling and Knowledge Bases XIX: Volume 166 Frontiers in Artificial Intelligence and Applications (Frontiers in Artificial Intelligenece and Applications)

Ontology and the Semantic Web: Volume 156 Frontiers in Artificial Intelligence and Applications (Frontier in Artificial Intelligence and Applications)

Ontology and the Semantic Web: Volume 156 Frontiers in Artificial Intelligence and Applications (Frontier in Artificial Intelligence and Applications)

Algorithms and Architectures of Artificial Intelligence (Frontiers in Artificial Intelligence and Applications)

Algorithms and Architectures of Artificial Intelligence (Frontiers in Artificial Intelligence and Applications)

Advances in Logic, Artificial Intelligence and Robotics: Laptec 2002 (Frontiers in Artificial Intelligence and Applications, 85)

Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms (Frontiers in Artificial Intelligence and Applications)

Modular Ontologies (Frontiers in Artificial Intelligence and Applications, WoMO 2011)

Active Mining (Frontiers in Artificial Intelligence and Applications, 79)

Active Mining (Frontiers in Artificial Intelligence and Applications)

Formal Ontologies Meet Industry (Frontiers in Artificial Intelligence and Applications)

Approximation Methods for Efficient Learning of Bayesian Networks: Volume 168 Frontiers in Artificial Intelligence and Applications (Frontiers in Artifical Intelligence and Applications)

Approximation Methods for Efficient Learning of Bayesian Networks: Volume 168 Frontiers in Artificial Intelligence and Applications (Frontiers in Artifical Intelligence and Applications)

Artificial Intelligence in Education: Building Technology Rich Learning Contexts that Work (Frontiers in Artificial Intelligence and Applications)

Artificial Intelligence in Education: Building Technology Rich Learning Contexts that Work (Frontiers in Artificial Intelligence and Applications)

Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms (Frontiers in Artificial Intelligence and Applications) (Frontiers in Artificial Interlligence and Applications)

Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms (Frontiers in Artificial Intelligence and Applications) (Frontiers in Artificial Interlligence and Applications)

Applications of Data Mining in E-Business and Finance (Frontiers in Artificial Intelligence and Applications)

Computational Models of Argument: Proceedings of COMMA 2006, Volume 144 Frontiers in Artificial Intelligence and Applications

Artificial General Intelligence (Cognitive Technologies)

The Mentor's Guide: Facilitating Effective Learning Relationships

Artificial General Intelligence (Cognitive Technologies)

Self-Organization and Autonomic Informatics (I): Volume 135 Frontiers in Artificial Intelligence and Applications

Information Modelling and Knowledge Bases XVIII - Volume 154 Frontiers in Artificial Intelligence and Applications

Information Modelling and Knowledge Bases XX - Volume 190 Frontiers in Artificial Intelligence and Applications

Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding - Volume 151 Frontiers in Artificial Intelligence and Applications

Artificial Intelligence in Education (Frontiers in Artificial Intelligence and Applications)

Artificial Intelligence Research and Development (Frontiers in Artificial Intelligence and Applications, Vol. 146) (Frontiers in Artificial Intelligence and Applications)

Understanding and Facilitating Adult Learning

Artificial Intelligence Research and Development: Volume 163 Frontiers in Artificial Intelligence and Applications

Artificial Intelligence Research and Development: Volume 163 Frontiers in Artificial Intelligence and Applications

Information Modelling and Knowledge Bases XIX: Volume 166 Frontiers in Artificial Intelligence and Applications (Frontiers in Artificial Intelligenece and Applications)

Ontology and the Semantic Web: Volume 156 Frontiers in Artificial Intelligence and Applications (Frontier in Artificial Intelligence and Applications)

Ontology and the Semantic Web: Volume 156 Frontiers in Artificial Intelligence and Applications (Frontier in Artificial Intelligence and Applications)

Algorithms and Architectures of Artificial Intelligence (Frontiers in Artificial Intelligence and Applications)

Algorithms and Architectures of Artificial Intelligence (Frontiers in Artificial Intelligence and Applications)

Advances in Logic, Artificial Intelligence and Robotics: Laptec 2002 (Frontiers in Artificial Intelligence and Applications, 85)

Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms (Frontiers in Artificial Intelligence and Applications)

Modular Ontologies (Frontiers in Artificial Intelligence and Applications, WoMO 2011)

Active Mining (Frontiers in Artificial Intelligence and Applications, 79)

Active Mining (Frontiers in Artificial Intelligence and Applications)

Formal Ontologies Meet Industry (Frontiers in Artificial Intelligence and Applications)

Approximation Methods for Efficient Learning of Bayesian Networks: Volume 168 Frontiers in Artificial Intelligence and Applications (Frontiers in Artifical Intelligence and Applications)

Approximation Methods for Efficient Learning of Bayesian Networks: Volume 168 Frontiers in Artificial Intelligence and Applications (Frontiers in Artifical Intelligence and Applications)

Artificial Intelligence in Education: Building Technology Rich Learning Contexts that Work (Frontiers in Artificial Intelligence and Applications)

Artificial Intelligence in Education: Building Technology Rich Learning Contexts that Work (Frontiers in Artificial Intelligence and Applications)

Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms (Frontiers in Artificial Intelligence and Applications) (Frontiers in Artificial Interlligence and Applications)

Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms (Frontiers in Artificial Intelligence and Applications) (Frontiers in Artificial Interlligence and Applications)

Applications of Data Mining in E-Business and Finance (Frontiers in Artificial Intelligence and Applications)

Computational Models of Argument: Proceedings of COMMA 2006, Volume 144 Frontiers in Artificial Intelligence and Applications

Artificial General Intelligence (Cognitive Technologies)

The Mentor's Guide: Facilitating Effective Learning Relationships

Artificial General Intelligence (Cognitive Technologies)

Self-Organization and Autonomic Informatics (I): Volume 135 Frontiers in Artificial Intelligence and Applications

Information Modelling and Knowledge Bases XVIII - Volume 154 Frontiers in Artificial Intelligence and Applications

Information Modelling and Knowledge Bases XX - Volume 190 Frontiers in Artificial Intelligence and Applications

Recommend Documents