The Role of Technology in CSCL: Studies in Technology Enhanced Collaborative Learning (Computer-Supported Collaborative Learning Series)

The Role of Technology in CSCL

COMPUTER-SUPPORTED COLLABORATIVE LEARNING VOLUME 9

Series Editor: Pierre Dillenbourg, Swiss Federal Institute of Technology, Lausanne, Switzerland Editorial Board: Michael Baker, CNRS & Université Lumière Lyon, France Carl Bereiter, Ontario Institute for Studies in Education, Canada Yrjö Engeström, University of Helsinki, Finland Gerhard Fischer, University of Colorado, U.S.A. H. Ulrich Hoppe, University of Duisburg-Essen, Germany Timothy Koschmann, Southern Illinois University, U.S.A. Naomi Miyake, Chukyo University, Japan Claire O’Malley, University of Nottingham, U.K. Roy Pea, SRI International, U.S.A. Clotilde Pontecorovo, University ‘La Sapienza’, Italy Jeremy Roschelle, SRI International, U.S.A. Daniel Suthers, University of Hawaii, U.S.A.

The Computer-Supported Collaborative Learning Book Series is for people working in the CSCL field. The scope of the series extends to ‘collaborative learning’ in its broadest sense; the term is used for situations ranging from two individuals performing a task together, during a short period of time, to groups of 200 students following the same course and interacting via electronic mail. This variety also concerns the computational tools used in learning: elaborated graphical whiteboards support peer interaction, while more rudimentary text-based discussion forums are used for large group interaction. The series will integrate issues related to CSCL such as collaborative problem solving, collaborative learning without computers, negotiation patterns outside collaborative tasks, and many other relevant topics. It will also cover computational issues such as models, algorithms or architectures which support innovative functions relevant to CSCL systems. The edited volumes and monographs to be published in this series offer authors who have carried out interesting research work the opportunity to integrate various pieces of their recent work into a larger framework.

The titles published in this series are listed at the end of this volume.

H. Ulrich Hoppe H. Ogata A. Soller (Editors)

The Role of Technology in CSCL Studies in Technology Enhanced Collaborative Learning

H. Ulrich Hoppe University of Duisburg-Essen Duisburg 47057 Germany [email protected]

Hiroaki Ogata University of Tokushima Tokushima 770-8506 Japan [email protected]

Amy Soller Institute for Defense Analyses Alexandria, VA 22311 USA [email protected]

Series Editor: Pierre Dillenbourg Swiss Federal Institute of Technology Lausanne, CH-1015 Switzerland

Library of Congress Control Number: 2007923578 ISBN 978-0-387-71135-5

e-ISBN 978-0-387-71136-2

Printed on acid-free paper. © 2007 Springer Science+Business Media, LLC. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC., 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. 9 8 7 6 5 4 3 2 1 springer.com

Contents

Contributors............................................................................. vii Acknowledgements.................................................................. xi Introduction 1. H. Ulrich Hoppe Educational information technologies and collaborative learning ………………………………………….…………………......... 1

PART I. Design, Modeling, and Analysis of Collaborative Learning 2. A. Soller, H. Ogata, F. Hesse Design, modeling, and analysis of collaborative learning ....................... 13 3. M. Wessner, H.-R. Pfister Points of cooperation: Integrating cooperative learning into web-based courses ...............................................…………..…….. 21 4. K. Nakakoji, M. Ohira, A. Takashima, Y. Yamamoto A computational tool for lifelong learning: Experiencing breakdowns and understanding situations ...……….………………..... 47 5. A. Soller, A. Lesgold Modeling the process of collaborative learning ....................................... 63 6. M. Constantino-González, D.D. Suthers An approach for coaching collaboration based on difference recognition and participation tracking ..................................................... 87

vi

Contents

PART II. Collaborative Tools in Educational Practice 7. A. Soller, A. Lesgold Collaborative tools in educational practice ............................................ 117 8. H. U. Hoppe, A. Lingnau, F. Tewissen, I. Machado, A Paiva, R. Prada Supporting collaborative activities in computer-integrated classrooms - the NIMIS approach .................... 121 9. M. F. Verdejo, B. Barros, T. Read, and M. Rodriguez-Artacho Designing a CSCL environment for experimental learning in a distance learning context.....................................................…........ 139 10. C. Kynigos, E. V. Dimaraki, E. Trouki Pupil communication during electronic collaborative projects: Integrating communication tools with communication scenarios ......... 155 11. H. Ogata, K. Matsuura, Y. Yano Supporting awareness in distributed collaborative learning environments………….………….....................…................................ 173

Index.………………………………………………………..…….……..193

CONTRIBUTORS

María de los Ángeles Constantino-González ITESM Campus Laguna, Torreón, Coah. 27250, MÉXICO Beatriz Barros Departamento de Lenguajes y Sistemas Informáticos UNED Ciudad Universitaria s/n, 28040 Madrid, Spain [email protected] Evangelia V. Dimaraki Educational Technology Lab, School of Philosophy, University of Athens 11 Akteou and Poulopoulou St., GR-11851 Athens, Greece [email protected] Friedrich Hesse Institut für Wissensmedien, University of Tuebingen. Konrad-Adenauer-Str. 40. D-72072 Tuebingen, Germany [email protected] Heinz Ulrich Hoppe University of Duisburg-Essen Lotharstr. 65, D-47048 Duisburg, Germany [email protected]

viii

Contributors

Chronis Kynigos Educational Technology Lab, School of Philosophy & CTI, University of Athens 11 Akteou and Poulopoulou St., GR-11851 Athens, Greece [email protected] Alan Lesgold School of Education, University of Pittsburgh Pittsburgh PA 15260, USA [email protected] Andreas Lingnau University of Duisburg-Essen Lotharstr. 65, D-47048 Duisburg-Essen, Germany [email protected] Isabel Machado ISCTE, University of Leeds & INESC-ID. Rua Alves Redol, 9 1000 - 029 Lisbon, Portugal Kenji Matsuura Center for Advanced Information Technology, Tokushima University 2-1, Minami-josanjima, Tokushima-shi, 770-8506, Japan [email protected] Kumiyo Nakakoji Research Center for Advanced Science and Technology, University of Tokyo 4-6-1 Komaba, Meguro, Tokyo, 153-8904, Japan [email protected] Hiroaki Ogata Faculty of Engineering, Tokushima University 2-1, Minami-josanjima, Tokushima-shi, 770-8506, Japan [email protected] Masao Ohira Graduate School of Information Science, Nara Institute of Science and Technology 8916-5 Takayama, Ikoma, Nara, 630-0192, Japan [email protected] Ana Paiva NESC-ID, Rua Alves Redol, nº 9, sala 630, 1000-029 Lisboa, Portugal [email protected]

Contributors Hans-Rüdiger Pfister Institute of Experimental Industrial Psychology (LueneLab) University of Lüneburg Wilschenbrucher Weg 84, D-21335 Lüneburg, Germany [email protected] Rui Prada NESC-ID / Instituto Superior Técnico - Tagus Park Av. Prof. Dr. Cavaco Silva, 2780-990 Porto Salvo, Portugal Sala 2-N7.19 [email protected] Timothy Read Departamento de Lenguajes y Sistemas Informáticos UNED Ciudad Universitaria s/n, 28040 Madrid, Spain [email protected] Miguel Rodriguez-Artacho Departamento de Lenguajes y Sistemas Informáticos UNED Ciudad Universitaria s/n, 28040 Madrid, Spain [email protected] Amy Soller Institute for Defense Analyses Alexandria, VA,22311 U.S.A. [email protected] Daniel D. Suthers University of Hawai`i at Manoa Honolulu, HI 96822, USA [email protected] Akio Takashima Graduate School of Information Science and Technology, Hokkaido University [email protected] Frank Tewissen University of Duisburg-Essen Lotharstr. 65, D-47048 Duisburg-Essen, Germany Evie Trouki Educational Technology Lab, School of Philosophy, University of Athens 11 Akteou and Poulopoulou St., GR-11851 Athens, Greece [email protected]

ix

x

Contributors

M. Felisa Verdejo Departamento de Lenguajes y Sistemas Informáticos UNED Ciudad Universitaria s/n, 28040 Madrid, Spain [email protected] Martin Wessner Fraunhofer IPSI Dolivostr. 15, D-64293 Darmstadt, Germany. [email protected] Yasuhiro Yamamoto SRA Key Technology Laboratory Inc., Japan 3-12 Yotsuya, Shinjuku, Tokyo 160-0004, Japan [email protected] Yoneo Yano Faculty of Engineering, Tokushima University 2-1, Minami-josanjima, Tokushima-shi, 770-8506, Japan [email protected]

ACKNOWLEDGEMENTS

This book resulted from the planning efforts at the international workshop on New Technologies for Collaborative Learning on Awaji Island, Japan, in November 2000. This workshop was supported by The Telecommunications Advancement Foundation and the Japan Chapter of the IEEE Education Society. We appreciate the contributions of the workshop organizers and all the participants: G. Ayala, Y. Hada, Y. Hayashi, F. Hesse, M. Ikeda, A. Inaba, K. Ito, T. Kasai, F. Kusunoki, K. Matsuura, H. Masukawa, N. Miyake, R. Mizoguchi, M. Muehlenbrock, K. Nakabayashi, K. Nakakoji, Y. Ochi, H. R. Pfister, H. Shirouzu, M. Sugimoto, D. Suthers, and Y. Yano. The editors also acknowledge the intellectual contributions from the organizers and participants of the 1st and 2nd International Workshops on Designing Computational Models of Collaborative Learning Interaction

Chapter 1 EDUCATIONAL INFORMATION TECHNOLOGIES AND COLLABORATIVE LEARNING Introduction H. Ulrich Hoppe University of Duisburg-Essen

1.

WHERE IS THE TECHNOLOGY IN CSCL?

The workshop on the island of Awaji-shima (Japan) from which this volume originates, was motivated by taking a technology perspective on collaborative learning. This was based on the (subjective) observation and experience that at other conferences and workshops on Computer-Supported Collaborative Learning (CSCL) technology appeared to be a secondary theme. Contributions focusing on system development and computational aspects related to CSCL would usually attract much less participants than conceptual and/or empirical presentations from an educational point of view. The idea of this workshop was to take the role of computational technologies in CSCL seriously. Of course this would not imply that only system-oriented papers were to be presented. Different roles of computational technologies should be elaborated, exemplified and discussed. Among these are the following prototypical roles as they appear not only in CSCL but also in other types of technology enhanced learning environments: – facilitation and enabling, i.e., either technology facilitates known types of learning in new or alternative ways, or it enables new kinds of learning experiences; – integration, i.e., technology is used to integrate learning activities and learning results and thus allow for a smooth “learning flow”;

2

H.U. Hoppe

– modeling, i.e., computational techniques are used to model or formally describe (collaborative) learning processes; – analysis, i.e., interaction traces or situational data from learning environments are analyzed using computational techniques. All these roles of technology will be reflected in some of the articles in this book. However, we also have to consider wider notions of technology: The material technologies we refer to and start from are information and communication technologies in a broad sense, including not only computers and networks of different types and reach, but also new electronic devices and digital media in general. This notion of “material” technology is to be distinguished from “social technologies” in a sense of methodological approaches of designing and shaping social situations to meet certain pragmatic goals or requirements. Although the notion of “social technology” is clearly relevant to the organization and “orchestration” of collaborative and other learning situations, this book does not presuppose a specific shared understanding of what social technology is and if it is at all desirable, or if it is even impossible. Related aspects are rather taken up in a more specific way, namely from a design perspective. Throughout this book, the “new” information and communication technologies (ICT) are conceived or construed in their specific functions related to learning and teaching. This is reflected in the term “educational information technologies” which is a simplification in so far as not all relevant types of learning may be subsumed under the term “education”. Beyond terminology, this is also a mission statement: We are interested in ICT as they are appropriated for and/or by educational or learning communities. Appropriating the material technology for a specific purpose is a dialectic process in which the meaning of technology may be augmented – a medium in a specific context may have a richer and more clearly defined message (cf. McLuhan, 1964) – or it may be modified in that the inherent logic or message of the medium is changed. The use and interpretation of technical media depends on the context and culture into which it is embedded. On the other hand, the culture of a (learning) community into which new “expressive” technologies, such as interactive media, are inserted will also be modified under the influence of these media. This dialectic has recently been described in more detail as a modern interpretation of Dewey’s originally esthetically oriented notion of media (cf. Dewey, 1934, as adopted by Vogel, 2001). It is beyond the horizon of this book to dive deeper into these philosophical issues. From a practical point of view, it should be noted that an educational appropriation of ICT does not necessarily go along with

Educational Information Technologies

3

re-inventing the basic technology in a more specific form: We have seen special-purpose authoring systems for learning material in the spirit of programmed instruction being replaced by general-purpose multimedia authoring tools with an even richer variety of presentation and interaction techniques. These general-purpose tools have been appropriated for educational purposes creating a specific culture of “educational multimedia”. We would categorize consumer-oriented educational multimedia as established technologies, not of primary importance for innovative approaches to collaborative learning. Internet connections in schools and academic institutions are no longer a new feature, though the learning culture originating from this technology may still lack a sufficiently rich definition. The technological “hot spots” of interest in this book are in turn: groupware or multi-user technologies such as group archives or synchronous co-construction environments, embedded interactive technologies in the spirit of ubiquitous computing, and modeling tools based on rich representations. Important features of these technologies are: – the move from individually oriented software tools to multi-user tools providing group awareness as well as facilities for the co-construction of external representations; – a definition of software use beyond a single piece of software towards multiple applications or tools which are not only technically interoperable but also task and role compliant in a social situation (social interoperability); – high interactivity and creative potential with high productive activity and initiative on the part of the user (as opposed to the receptive scheme of usage of many educational multimedia applications); – peripherals supporting ubiquitous computing and augmented reality (Weiser, 1993; Ishii & Ullmer, 1997) which allow for redefining the borderline between physical action on the one hand and virtual or symbolic on the other. How about “intelligent support”? Although research into intelligent tutoring systems has not led to a new class of computer-based learning environments in wider practical use, it has brought forth modeling and description techniques which may also support awareness and systeminitiated scaffolding also in collaborative learning. Whereas the development of intelligent tutoring systems could be guided by the idea of analyzing and directing the complete learning situation, presumably in the closed world of a dyadic relation between a single user-learner and a computer program, this approach is unconceivable in collaborative learning environments. Free linguistic communication, either using spoken language in face-to-face or

4

H.U. Hoppe

virtual tele-presence environments, or relying on synchronous teletype style dialogues such as chats, combined with the flexible composition of learning groups and varying task and role assignments all together define a degree of complexity which is far beyond the analytic potential of machine computation. Thus, intelligently supported collaborative learning environments have to be conceived as open worlds with only partial understanding or insight on the part of the machine. Giving up the “closed world/full control” assumption opens a new perspective towards symbiotic types of support and scaffolding in which roles are distributed between humans, e.g., as peer helpers, and machines, for example as watchdogs for critical group situations which require external support (cf. Hoppe & Plötzner, 1999). As a basis for these kinds of symbiotic mechanisms, we need to develop a rich repertoire of analytic methods making maximum use of the computational potential without striving for automated control. Existing “systems that support the management of collaborative interaction” have been compared and classified by Jerman et al. (2001). Whereas the practical benefit of analysis-based symbiotic support mechanisms is still subject to speculation, the development of analytic methods is in itself a theoretical contribution to the advancement of modeling and understanding the structure and process of collaborative learning.

2.

THE DIALECTICS OF COLLABORATIVE LEARNING AND TECHNOLOGY

The combination of computer support with collaborative learning has created a community of researchers, developers, and practitioners with roots in the humanities as well as in computer science and technology. In this combination, collaborative learning has become a trendy theme, and this is most likely due the technological ingredient in several respects: It is a common pattern that everything around the Internet and digital media tends to be perceived as innovative and “hot”. If there is a new kind of application of (supposedly) innovative technologies such as computer-mediated communication, conferencing and application sharing environments, this transfers interest from the technology to the application – in this case, to collaborative learning. This increased or renewed interest is appreciated and taken up not only by the technologists but also by experts from the application field – in this case, for example, educational scientists or social and pedagogical psychologists. On the other hand, the “collaborative move” has also opened new perspectives for the more technology-oriented research in educational computing: As mentioned above, the “closed world/full control” assumption,

Educational Information Technologies

5

which was predominant not only in ITS research but also in the development of computer-based learning and training environments in a wider sense, had turned out to be neither realistic from a practical point of view nor theoretically appealing in the light of active learner-centered educational methods. Koschmann (1996) even sees CSCL as a new paradigm in educational technology, i.e., as a scientific paradigm shift in the sense of Kuhn (1972). Yet, the “collaborative move” does not take place in a welldefined scientific community as did, e.g., the relativistic move in the physics community in the early 20th century. The CSCL community is methodologically heterogeneous with varied backgrounds between humanities on the one hand and formal sciences and technology on the other. If CSCL is at all considered a field of scientific study, then it is of the type “design science” or “science of the artificial” (Simon, 1969) in that it does not describe, analyze, and explain given phenomena but creates its own objects of study. This is clearly the case for both the technology-oriented engineering as well as the normative pedagogical approaches. And even from a psychological perspective it is almost impossible to set up computerized collaborative learning experiments without implicitly or explicitly designing and inventing a specific setting. Irrespective of common theoretical foundations currently sought in social constructivism, going back to Vygotsky (1978), and activity theory (c.f. Cole & Engeström, 1993), there is no way of deterministic or taxonomic reasoning that would guide us from these foundations directly to the concrete design of artifacts and scenarios for collaborative learning. These theories are much more reflections of certain innovative practices than they are themselves generators of innovation. In the understanding which underlies this book, the “innovative tension” is primarily allocated between emerging technology and educational practice. Irrespective of CSCL being considered as a new “paradigm” or not, the heterogeneity of scientific and methodological backgrounds in the CSCL community is itself a potential source of creative conflict which may lead to innovation through interdisciplinary synergy. First, there is a need for “common ground” on a conceptual level. This has led to a discussion about the meaning of the basic vocabulary. In his introductory chapter, Dillenbourg (1999) tries to distinguish but also to narrow down the variety of meanings for “learning” and “collaboration”. Particularly, he claims that truly collaborative situations should be based on a symmetry of action, knowledge, and status between the collaborators. Not only should they have shared goals but also mutual awareness of these common goals. This is more a normative statement about how things should be than it is an empirical statement about the reality of learning which takes place in group settings and involves group interaction. For the purposes of this book a wider

6

H.U. Hoppe

understanding of collaborative learning, maybe just as “learning in groups with a high degree of interaction between the group members in a defined setting”, appears to be more suitable. In the pedagogical literature, we find quite pragmatic definitions of these basic notions. For instance, Cohen (1994) approaches collaborative learning specifically as “classroom groupwork” from an educational design perspective. This pragmatic type of view comes quite close to what we suggest for defining and elaborating an innovative technology-oriented perspective on group learning. Although the current interest in CSCL and in collaborative learning in general is nurtured by technological innovation and the search for new applications, the different perspectives on CSCL lead to very different attitudes with respect to the role of technology. We can distinguish the following prototypical attitudes related to the new “material” technologies: 1. Technologies, namely ICT, are taken as given. They are used, mainly for text-based communication and archiving in group learning scenarios. Typically, the technologies used are “consumer-ready” services on Internet and browser platforms such as electronic messaging, chats, threaded discussion tools, group archives, conferencing facilities, etc. The perspective on technology is a consumer-user perspective; the technology is not modified or subject to a re-design process. It is also not questioned in the sense that experience is evaluated and analyzed to suggest improvements of technology. 2. Existing technology is “appropriated” in a collaborative learning setting in that its functions and its way of embedment in the learning context is subject to design and investigation. The technology itself may be unchanged or only superficially modified by using available parameterization or configuration mechanisms. Often, also the interaction and “social interoperability” of different systems or tools is investigated. The experience is evaluated in such a way as to help re-designing technology and/or its forms of usage. 3. New technologies, such as software tools or communication mechanisms, are designed and developed to support certain forms of cooperation and particularly of collaborative learning. Practical testing and evaluation is part of an iterative redesigning process in which technological and social, organizational, or particularly educational aspects are closely intertwined. It is evident that, based on what has been said about the role of technology so far, we would favor research of type 2 and 3 over 1. Also, we do not accept in our context such types of merely technology-driven research in which unspecific system mechanisms are developed or improved and for which educational applications are sought “ex post”. Imagine a new

Educational Information Technologies

7

compression algorithm which improves video transmission bandwidth is available, and now applications applying this technology in distance learning are suggested without clearly elaborated educational goals. According to current standards in the CSCL community, this latter type of research would hardly be accepted as relevant, whereas acceptance of type 1 research is not a rare case. This is a subjective statement and view. It may or may not be shared by other contributors to this book. Yet, the obvious consequence that technology should, among other aspects, be considered as a genuine subject of CSCL research in its own right is part of the common orientation that motivated us to publish this collection of articles.

3.

THE REST OF THE BOOK

The rest of this book is organized in two parts, the first one dealing with the modeling, analysis, and design of collaborative learning and the second dedicated to studies of collaborative tools in educational practice. Each theme has its particular “rhetoric”, and this is what is to be explained in this general introduction, not a content summary of the articles. Among other themes, Part I introduces analytic approaches using computational methods as a type of “basic research” in CSCL. It is one of the primary functions of any piece of groupware to provide “group awareness”, i.e., rich mutual information about the current situation and structure of the group and of its individual members. In conventional groupware, awareness is an implicit result of computer-mediated communication and of the design of information access functions and displays. There is actually no awareness of the group situation on the part of the system. It has been argued above that new types of scaffolding and group management support may arise from having more of this “awareness information” explicitly represented and processed inside the machine. To achieve this, we need representations of group constellations and situations, including descriptions of roles, goals and tasks, as well as diagnostic mechanisms to analyze and interpret the interactions in a learning group. Most analytic approaches in CSCL have been based on discourse or dialogue, yet not all of these are “computational” in the sense of a completely automated interpretation. One way of making the linguistic material machine-interpretable is to pre-structure the dialogue on the interface level, e.g., based on a pre-categorization of speech acts or types of contributions. More recently, user operations executed on the machine have been used for “action-based analysis”. This approach is particularly well suited for co-constructive environments with formally defined representations of objects and operations. A variant of this action-based

8

H.U. Hoppe

approach does not primarily rely on operations (i.e., on process features) but on an analysis of the problem state. All these basic aspects mentioned so far are covered by articles in Part I, yet, as for the first theme, an integration of these approaches still defines a challenge. The aspect of designing and providing external representations for knowledge sharing and co-construction is shared between parts one and two. An important difference lies in origin of the representations: The representation can be the material which is directly used in a co-constructive process as is the case in scientific modeling using diagrams. Here, the concrete visual representation is the object and external medium of the creative process. The interactions materialize in the medium, and thus the inherent structure and logic of the external medium shapes or guides the learning process. Yet, learning is not only and not always based on construction “from scratch”. Learning materials and presentations may be fed into the learning process from an external source or from a teacher. The structure of this material is important from an initially receptive learner’s perspective: Can it be flexibly accessed and easily appropriated? But also: can it be post-processed, annotated, shared in such a way as to facilitate the interaction between co-learners in synchronous and asynchronous settings? Finally, the external representation can be used to encode meta-level information on the learning process. This is the case with interaction histories or sociograms. Such combinations of information on learning objects, participant behavior and attitudes over time are a valuable resource for reflection. The integration of these different representational support functions is an important challenge for future work. The second part on collaborative tools in educational practice goes beyond the design perspective in that it focuses on the introduction of collaborative learning technologies into real-life educational scenarios. Hence, “ecological validity” is a central challenge. However, the question is not only if the technology or methodology “works” and fits in with the reallife situation, but also if it creates innovation. Can (and does) the educational community appropriate the proposed technology in way that generates new rich forms of interaction and qualitatively new learning arrangements? In a sense, the new technology functions as a “perturbation” of an existing scenario from which we expect an evolution which hopefully leads to a better adaptation to the external (e.g., societal) and individual demands. Of course, this perturbation is not a random one, but was originally inspired by a socio-technical rationale such as, for instance, a purposeful educationally oriented design. At some later point, we will have to assess if the change is a change for better. However, we should avoid a “measurement overkill” by demanding empirical “proofs” at a very early stage of innovation. Interestingly, this second part which has the closest connection to practice

Educational Information Technologies

9

also features the most innovative examples of technology in terms of XMLstructured “active documents” and embedded interactive devices and tangible interfaces. This may indicate that there are ways to fruitfully combine technological innovation with the quest for a change in educational practice.

REFERENCES Cohen, E. G. (1994). Designing Groupwork: Strategies for the Heterogeneous Classroom. New York: Teachers College Press. Cole, M. & Engeström, Y. (1993). A cultural-historical approach to distributed cognition. In G. Salomon (Ed.), Distributed Cognitions (pp. 1-46). Cambridge, MA: Cambridge University Press. Dewey, J. (1934). Art as Experience. Reprinted 1980. New York: Perigee Books. Dillenbourg, P. (Ed.) (1999). Collaborative learning: Cognitive and computational approaches. Amsterdam et al.: Elsevier Science. Dillenbourg, P., Baker, M., Blaye, A., O’Malley, C. (1995). The evolution of research on collaborative learning. In H. Spada and P. Reimann (Eds.), Learning in Humans and Machines (pp. 189-211). Amsterdam et al.: Elsevier Science. Jermann, P., Soller, A., Mühlenbrock, M. (2001). From mirroring to guiding: A review of state of the art technology for supporting collaborative learning. Proceedings of the First European Conference on Computer-Supported Collaborative Learning (324-331). Maastricht, The Netherlands. Hoppe, H. U. & Plötzner, R. (1999). Can analytic models support learning in groups? In Dillenbourg (Ed.) (1999), pp. 147-168. Ishii, H. & Ullmer, B. (1997). Tangible Bits: Towards Seamless Interfaces between People, Bits and Atom. In Proceedings of ACM CHI '97 (pp. 234-241). ACM Press. Koschmann, T. (1996). Paradigm shifts and instructional technology: an introduction. In T. Koschmann (Ed.). CSCL: Theory and Practice (1-23). Hillsdale (NJ): Lawrence Erlbaum. Kuhn, T. S. (1972). The Structure of Scientific Revolutions (2nd ed.). Chicago: University of Chicago Press. McLuhan, M. (1964). Understanding Media – The Extensions of Man. Reprinted 1994. Cambridge, MA: MIT Press. Simon, H. (1969). Sciences of the Artificial. Cambridge (MA): MIT Press. Vogel, M. (2001). Medien der Vernunft – Eine Theorie des Geistes und der Rationalität auf der Grundlage einer Theorie der Medien. (Media of Reason – A Theory of Mind and Rationality based on a Theory of Media.) Frankfurt a. M. (Germany): Suhrkamp. Vygotsky (1978). Mind in Society. Cambridge, MA: Harvard University Press. Weiser, M. (1993). Some computer science issues in ubiquitous computing. Communications of the ACM Vol. 36, No. 7, 75-84

PART I DESIGN, MODELING, AND ANALYSIS OF COLLABORATIVE LEARNING

Chapter 2 DESIGN, MODELING, AND ANALYSIS OF COLLABORATIVE LEARNING Introduction to PART I Amy Soller1, Hiroaki Ogata2, Friedrich Hesse3 1

Institute for Defense Analyses, Alexandria, VA,22311 U.S.A.

2

Faculty of Engineering, Tokushima University, 2-1, Minami-josanjima, Tokushima-shi, 770-8506, Japan 3

Applied Cognitive Psychology and Media Psychology. University of Tuebingen. KonradAdenauer-Str. 40. D-72072 Tuebingen

Over the past 20 years, computer-based training software has become increasingly successful at addressing the learning needs of individuals. Yet, the problems we face in meeting the needs of learning groups continue to be a challenge, both on line and in the classroom. As Webb and Palincsar (1996) explain, studying group learning involves much more than studying a synthesis of individual behaviors: “Consider the numerous intraindividual factors (e.g., prior knowledge, motivation, language) that influence the learning of one child in “individualistic” activity. Place this learner in a group context, and not only does one have to contend with all the issues that attend this interaction among the group members (from the very mundane resource issues to the more lofty issues of attaining intersubjectivity), but in addition, other intraindividual factors that may have receded into the background when considering individualistic activity now emerge as salient, indeed critical (e.g. the learner’s gender and social status).” (p. 867) Just as supporting individual learning requires an understanding of individual thought processes, supporting group learning requires an understanding of the processes of collaborative learning. These processes are shaped by the group members’ individual behaviors, and the dynamics of

14

A. Soller et al.

their interaction. The chapters in Part I of this book bring together cognitive, social, and computational perspectives to evolve advanced methods for designing, modeling, analyzing, and evaluating online collaborative learning activities. To be consistent with the contributions that follow, we limit our discussion to collaborative learning activities that occur at a distance, over a computer network, although many of these ideas may be derived from, or may also pertain to face-to-face collaborative learning. Guidelines for studying the collaborative learning process are by no means straightforward, however the pay off tends to be quite attractive. The collaborative learning experience has the potential to motivate students to seek new insights and perspectives, ask questions openly, and practice explaining difficult concepts, thereby gaining a better understanding of the domain (Doise, Mugny, & Perret-Clermont, 1975). The extent to which these benefits are realized depend largely on the effectiveness of the group interaction. The overall goal of the approaches described in this section is to help students interact effectively, so that they may maximize their potential learning gain. Many different factors may influence group dynamics, which in turn influence student learning. Some of these factors include group composition and cohesion, group size, task structure, student and teacher roles, discourse styles, nature of facilitation, rewards or incentives, training in communication skills, group processing, and the learning environment (Levine & Moreland, 1998; Webb & Palincsar, 1996). In the interest of positively influencing the process of collaborative learning through computational means, Part I of this book views these factors in terms of those that must be decided before the students begin collaborating (e.g., group composition, rewards), and those that may be altered as the collaboration progresses (e.g., roles, facilitation methods). The chapters that follow cover two fundamental approaches to promoting effective group interaction. The first approach varies the assortment and intensity of external environmental factors such as the group’s composition or the learning context. For example, a (human or computer) facilitator might construct a learning group for a specific task by selecting members with the most compatible knowledge, skills, and behaviors in anticipation that this will create the dynamics needed to produce effective learning. The second approach focuses on the modeling and diagnosis of internal group interaction factors by analyzing the group interaction after the students have begun an assigned task. In this case, the facilitator might study the progression of the group conversation or the development of the group’s shared solution. By applying a combination of these approaches, the system may glean enough information from the analysis to dynamically facilitate the interaction, propose new problem sets that target specific skills, or alter the environment to adapt appropriately to the students’ changing needs.

Design, Modeling, and Analysis

15

Part I begins with a chapter by Wessner and Pfister in which they discuss the effect of both external environmental factors, such as group formation, and internal group interaction factors, such as the structuring of the learners’ communication and collaboration processes, on web-based cooperative learning. They introduce the notion of “points of cooperation” that describe opportunities to cooperate within specific learning contexts, and they extend this discussion to explain how activities may vary in the degree to which they are integrated in the web-based course design. For example, generic cooperation activities may be less integrated than spontaneous or intended cooperation activities. Special attention is given to the “intended points of cooperation”, because these represent the optimal degree of logical (in relation to other parts of the course) and didactical (dependent on the type of instructional content or media) integration. Intended points of cooperation include cooperative learning methods such as pro/contra-disputes or brainstorming. These may be defined during course authoring and treated as course units. To illustrate these ideas, Wessner and Pfister describe a learning environment developed for the project “L3–Lifelong learning as a basic need.” Different group formation criteria are considered depending on the learning mode (class vs. individual) and the cooperation mode (synchronous vs. asynchronous). Depending on the learning mode, the group formation may be accomplished either manually or automatically. Management tools assist the tutor in manually constructing the group by considering the constraints of the intended cooperative activity (e.g., number of participants required, knowledge preconditions for the task). When no tutor is available, the automatic group formation algorithm enables the learning system to automatically extract the information needed to select and group participants. The system supports collaboration during intended interactive activities by providing group members with the information and tools they need (e.g., the topic to be learned or discussed, number of participants, duration, additional information for discussants, cooperation scripts) to initiate and manage the cooperative learning process. The second chapter in Part I, authored by Nakakoji, Ohira, Takashima and Yamamoto, focuses on a computational environment that supports “breakdowns” as opportunities for lifelong learning. Winograd and Flores explain that a breakdown is, “not a negative situation to be avoided, but a situation of non-obviousness, in which the recognition that something is missing leads to unconcealing (generating through our declarations) some aspect of the network of tools that we are engaged in using (Winograd & Flores, 1986, p. 165).” Nakakoji and colleagues propose two approaches to support lifelong learning; (1) experiencing a breakdown, and (2) asking for information relevant to the breakdown. Knowledge-based critiquing systems

16

A. Soller et al.

have been studied to support these processes by monitoring human performance, identifying potentially problematic situations, alerting the users about potential problems, and providing explanations for the criticism and information relevant to the problem. Although they have been found to be effective, such systems do not support synchronous collaborative learning among practitioners. Nakakoji and colleagues complement the knowledgebased critiquing approach with their presentation of EVIDII (Environment for Visualizing Individual Differences of Impressions), a system that helps group members visualize the differences between set associations (e.g., pictorial images and words). While interacting with EVIDII, users experience breakdowns when they encounter unexpected associations made by other group members and are encouraged to ask the other members about the association. These activities are intended to prompt further communication and knowledge construction among group members. Throughout the case studies, the authors observed that conversations often started with phrases such as “Really?” indicating that users did experience breakdowns while interacting with the system, and most such breakdowns occurred when participants discovered differences or seemingly conflicting associations. Methods for dynamically analyzing peer interaction after the students have begun to collaborate form the basis for the approaches to promoting effective group interaction described in the next two chapters in Part I (Soller & Lesgold, and Constantino-Gonzalez & Suthers). These authors describe computational methods can be applied to model and analyze different aspects of group interaction. In general, a student’s understanding of a concept is reflected in his actions, and his explanations of these actions. In a one-on-one tutoring environment, this information is available, and in most cases, straightforward to analyze. The system would typically watch the student solve a problem, perhaps ask pointed questions to evaluate the student’s understanding of key concepts, and once in a while, interrupt him if remediation is necessary. Evaluating the learning of a group of students solving the same problem, however, presents a few new challenges. If one student solves the problem successfully while explaining his actions, and his teammates acknowledge and agree with his actions, to what degree should we assume his teammates understand how to solve the problem themselves? If a student is continually telling her partner what to do, and her partner is simply following her instructions without questioning her, who should get credit for solving the problem? The only way to know for certain which group members understand which material is to have some knowledge about how the group conversation relates to the student actions.

Design, Modeling, and Analysis

17

Unfortunately, introducing natural language understanding technology means introducing its underlying issues of ambiguity in language, increasing the complexity of the problem substantially. There have been a few different approaches to dealing with this issue. The software may restrict the students’ natural language to a formal language (e.g., Tedesco and Self, 2000), or it may structure the students’ language by having them select opinion buttons (e.g., “OK”, “I agree”), or begin their utterances with sentence openers (e.g., “I think”, “Do you know”). A combination of these approaches may also be used. For example, Soller and Lesgold present an approach that assesses group interaction by analyzing students’ communication patterns, in the form of speech act sequences (e.g., Request, Inform, Acknowledge) and performs a coarse-grained analysis of student workspace actions. The approach in Constantino-Gonzalez and Suthers’ article combines an analysis of the students’ participation trends, and student opinions about problem solving actions in private and shared workspaces, to guide the interaction. In Soller and Lesgold’s approach, a machine learning algorithm is used to train a computer to generate a model of knowledge sharing between peers. Soller and Lesgold identify knowledge sharing as a critical aspect of collaborative learning, since it initiates the questioning, explaining, and critical discussion that often follows the exchanging of new concepts and ideas. Their system learns by iteratively constructing a probabilistic statebased model that generalizes classified examples of knowledge sharing interaction. Sequences of knowledge sharing interaction are coded by the system using conversational acts (such as “Request”, “Acknowledge”, or “Motivate”) that represent the sentence openers the students may choose to begin their utterances. Soller and Lesgold use this system to (1) identify the student playing the role of knowledge “sharer” during knowledge sharing conversation, and (2) determine the effectiveness of the interaction. Distributing the knowledge needed to solve a problem among the group participants enabled Soller and Lesgold to capture and study the social process of information sharing. Specialized knowledge distribution, however, may have the effect of distributing task roles, creating a local expert effect in which each student independently applies his or her knowledge to the problem (Stasser, 1999). When this happens, it may inhibit the group’s ability to collaboratively construct new knowledge. One way of dealing with this problem is to create a private workspace that students can use to individually solve the problem, or try out solutions before proposing them to the group. Constantino-Gonzalez and Suthers describe COLER, a system that builds on this idea to help students learn Entity-Relationship modeling, a formalism for conceptual database design. COLER’s private individual workspaces help students independently develop their ideas, while its shared group workspace enables students to

18

A. Soller et al.

jointly construct a shared representation. Decision trees drive the system’s back end by analyzing both task-based and conversational interaction, and dynamically generating recommendations for improving group problem solving. Students are required to express their agreement or disagreement (by clicking on “Agree”, “Disagree”, or “Not Sure” buttons) each time an item is added or changed on the group’s shared workspace. This information, along with student participation statistics, and differences between students’ private and group workspaces, is used by COLER’s personal coaches to dynamically facilitate the group. For example, Jim’s COLER coach might observe his teammate adding a node to the group’s shared diagram, and might notice that this node is missing in Jim’s private diagram. If Jim disagreed with his teammate’s new addition, his coach might then recommend that the two students discuss a few alternatives so that they may learn from each other, and perhaps come to consensus. Modeling and analyzing collaborative learning means accounting for the spectrum of activities that groups engage in while learning. Separating a student’s participation from the quality of his contributions, or studying discourse and action separately, may produce an inadequate understanding of the group activity. The articles in this section should be viewed as corresponding to pieces of a pie that represents a comprehensive model of group interaction and learning. For example, Wessner and Pfister focus on group composition within the context of specific learning opportunities, Nakakoji and colleagues focus on addressing learning breakdowns, and Soller and Lesgold focus on knowledge sharing dialog. Finally, ConstantinoGonzalez and Suthers specifically study the interaction between student participation, opinions, and differences in structured representations. These articles should not be viewed independently, but rather as a toolbox of methods and strategies for understanding and supporting various aspects of online collaborative learning behavior. This toolbox reflects the perspectives of both the software designer and the educational practitioner, enabling the marriage of theory and implementation. Modeling collaborative learning activities means modeling both verbal and nonverbal interactions, and both task and social aspects of group learning. Studying these aspects separately allows researchers to deal with difficult issues (such as natural language understanding), while controlling for the variability inherent in collaborative learning. Future research along these lines should help to develop a more complete toolbox of methods for computationally analyzing collaborative learning activities. With a more complete toolbox at hand, researchers may be better suited to adopt holistic views of supporting collaborative learning communities. Knowledge about how students interact is useful to a system only if it can apply this knowledge to recognize specific situations that call for intervention.

Design, Modeling, and Analysis

19

Classroom teachers learn to analyze and assess student interaction through close observance of group interaction, trial and error, and experience. Developing a system to analyze group conversation, however, poses its own challenges. Focused research in computational modeling of peer interaction will help in making the transition from understanding how to mediate learning groups to understanding how to train a system to mediate learning groups. The many factors that influence collaboration often interact with each other in unpredictable ways, making it very difficult to measure learning effects (Dillenbourg, Baker, Blaye, and O’Malley, 1995). This may be one reason why the focus of collaborative learning research shifted in the nineties from studying group characteristics and group products to studying group process. With an interest in having an impact on the group process, the focus has recently shifted again – this time from studying group processes to identifying computational strategies that positively influence group learning (Soller, 2001). Furthermore, since the choice of mediation strategy must be based on an analysis of the group’s needs, there is a need for the integration of evaluation in the modeling and analysis cycle. Ideally, the system would model and analyze the group process, and then select and apply one or more mediation strategies. The next logical step would be to evaluate the effect of the mediation with respect to the group process and product. This evaluation would then, in turn, be used to modify the group process model, which would then be used to analyze the group process, and so on (Jermann, Soller, and Muehlenbrock, 2001). Few systems have achieved this, although some of the systems described in this section have taken steps in this direction. For example, COLER was evaluated based on the appropriateness of the computer coach’s advice (as judged by a domain expert) and the students’ reactions to the advice. Future research should build upon these notions to not only develop computational methods for identifying and analyzing group interaction needs, but also link these needs to suggested facilitation strategies (which are grounded in psychological literature), and evaluate the utility of this process for supporting on line collaborative learning. Although computer-based approaches to individualized instruction have met with great success, many unresolved issues still exist in the realm of computer-supported group learning. The processes underlying peer interaction are complex, and not yet fully understood by practitioners in the educational and social sciences (Dillenbourg, 1999; Levine & Moreland, 1998) – a challenge that presents opportunities for new technologies to help in understanding and supporting this rich source of learning, but one that introduces uncertainty in the theoretical foundations of the technology. Because of this uncertainty, computational methods for analyzing peer interaction tend to focus on key aspects of the process that are thought to

20

A. Soller et al.

influence learning outcomes. The four articles in this section cover factors such as group composition, participation, individual and group problemsolving actions, socio-cognitive conflict, and knowledge sharing. Many of these issues are new to computer-based instruction, and because of this, the authors have been careful to ground their computational approaches in existing research on collaborative learning and group dynamics wherever possible. This section aims to further our understanding of a few group specific issues, so that we may better support the process of on line collaborative learning in the future. We hope that you will find the ideas and methods presented in the following four articles informative and helpful in furthering your own research program.

REFERENCES Dillenbourg, P. (1999). Collaborative learning: Cognitive and computational approaches. Elsevier Science. Dillenbourg, P., Baker, M., Blaye, A., & O’Malley, C. (1995). The evolution of research on collaborative learning. In H. Spada and P. Reinmann (Eds.), Learning in Humans and Machines, Elsevier Science. Doise, W., Mugny, G., & Perret-Clermont, A. (1975). Social interaction and the development of cognitive operations. European Journal of Social Psychology, 5(3), 367-383. Jermann, P., Soller, A., & Muehlenbrock, M. (2001). From mirroring to guiding: A review of state of the art technology for supporting collaborative learning. Proceedings of the First European Conference on Computer-Supported Collaborative Learning, Maastricht, The Netherlands, 324-331. Levine, J. M. & Moreland, R. L. (1998). Small groups. In D. Gilbert, S. Fiske, & G. Lindzey (Eds.), The Handbook of Social Psychology (4th ed.) Boston, MA: McGraw-Hill, 415-469. Soller, A. L. (2001). Supporting social interaction in an intelligent collaborative learning system. International Journal of Artificial Intelligence in Education, 12(1), 40-62. Stasser, G. (1999). The uncertain role of unshared information in collective choice. In L. Thompson, J. Levine, and D. Messick (Eds.), Shared Knowledge in Organizations (pp. 4969). Hillsdale, NJ: Erlbaum. Tedesco, P. & Self, J. A. (2000). Using meta-cognitive conflicts in a collaborative problem solving environment. Proceedings of the 5th International Conference on Intelligent Tutoring Systems, Montreal, Canada, 232-241. Webb, N. & Palincsar, A. S. (1996). Group Processes in the Classroom. In D. Berlmer & R. Calfee (Eds.), Handbook of Educational Psychology (pp. 841-873). New York: Simon & Schuster Macmillan. Winograd, T. & Flores, F. (1986). Understanding Computers and Cognition: A New Foundation for Design, Ablex Publishing Corporation, Norwood, NJ.

Chapter 3 POINTS OF COOPERATION: INTEGRATING COOPERATIVE LEARNING INTO WEB-BASED COURSES

Martin Wessner1, Hans-Rüdiger Pfister2 1

Fraunhofer IPSI. Dolivostr. 15, D-64293 Darmstadt, Germany [email protected] 2 Institute of Experimental Industrial Psychology (LueneLab), University of Lüneburg Wilschenbrucher Weg 84, D-21335 Lüneburg, Germany [email protected] Abstract:

1.

Current web-based courses provide only limited means to support cooperative forms of learning. In this paper we introduce the notion of “Point of Cooperation (PoC)” to classify a wide range of cooperative learning activities in a networked learning environment. With respect to the extent a cooperative activity is integrated into a web-based course we distinguish between Generic (GPoC), Spontaneous (SPoC), and Intended (IPoC) Points of Cooperation. The PoC concept is compared with traditional web-based learning environments. We outline how PoCs are integrated into a course from the course author's point-of-view, and we describe how PoCs are handled during the ongoing learning process, including the management of PoCs, the execution of PoCs, the integration of communication and cooperation tools, and the management of the cooperation’s results. We then describe the project “L³: Lifelong learning as a basic need”, a German federally funded project which serves as a use case for the PoC approach. Finally, the process of learning group formation as a prerequisite for cooperative learning is analysed in more detail.

INTRODUCTION

Currently, we observe an increasing demand for web-based training on all levels, universities, continued education, and training within organisations and companies. Especially, virtual organisations consisting of geographically distributed teams feel the need to provide learning on

22

M. Wessner and H.-R. Pfister

demand and training opportunities for distributed learners via inter- and intranets. Team learning, i.e., learning together as a group and thereby sharing and building knowledge and skills, provides a means to learn in a motivating as well as efficient way. This is an application domain for what has been called cooperative learning (Slavin 1995), and modern web-based environments provide the technology to establish various forms of computersupported and net-based cooperative/collaborative learning (CSCL; O’Malley 1994, Koschmann et al., 1996). However, we also observe a discrepancy between the development and use of web-based courses and the incorporation of cooperative learning methods. Practically all web-based courses are constructed according to the paradigm of individual learning, at best enhanced by a collaborative environment which enables users to communicate with each other in a generic way. For example, a typical webbased course might consist of an elaborated hypermedia structure, i.e., a set of linked (HTML) pages with interactive multimedia demonstrations and simulations. This structure is then embedded into a technical environment which provides general tools for communication, e.g., chat-rooms or video conferencing. However, both aspects – content and cooperation – are not systematically integrated; learners and tutors are able to communicate about the learning material, but are not able to learn the content in a genuinely cooperative manner. We argue that a tight and systematic integration of content, including learning goals, and cooperation, i.e., of the knowledge to be acquired and the cooperative didactical procedures employed, is necessary to warrant success for cooperative web-based learning. Note that we use the terms cooperative learning and collaborative learning interchangeably. We use a very simple definition of cooperative learning: Two or more people communicate and cooperate with respect to the common goal of knowledge acquisition, they are willing to share their knowledge and experience and to support each other. The primary means of cooperative learning is a common discourse about the learning domain guided by the learning goal, given the constraints of available communication and cooperation tools in a computer-supported environment. This definition is fairly general, neglecting issues such as how much guidance and structuring should be provided or how groups of learners should be formed. The notion of Points of Cooperation introduces several systematic ways cooperative learning could be realised in an integrated learning environment. In the following, we first briefly comment on some popular learning platforms and derive requirements with respect to cooperative learning. Then, the notion of Points of Cooperation (PoC) is introduced and a taxonomy including three types of PoCs is proposed, according to the extent to which a cooperative activity is incorporated into a course: generic,

Points of Cooperation

23

spontaneous, and intended cooperation. Especially, the concept of intended cooperation (IPoC) is outlined in more detail as a core component of integrated cooperative learning. Intended cooperation involves the application of didactical considerations throughout the learning process, from authoring to execution. Then, we describe how PoCs are realised in a German federal research project called “L3 – Lifelong learning”. Eventually, we elaborate the process of group formation as an essential aspect of webbased cooperation of distributed learners.

2.

COOPERATIVE LEARNING IN WEB-BASED ENVIRONMENTS

An efficient CSCL environment should enable learners to cooperate in a variety of ways, depending on learning goals, group size, group structure, and other factors (Dillenbourg, 1999). Two basic questions are: With whom will I learn together? How will we learn together? The first question refers to the problem of group formation, the second question refers to the problem of communication and cooperation structuring. For both issues, the learning environment should provide support. For example, tools are needed which provide awareness of potential cooperation partners and which provide information about their relevant characteristics in order to support the search for partners. Sometimes cooperation is based on the learning context the learner is currently in. Here, the CSCL environment must provide additional information about the context, e.g., the specific course a learner has selected, the group a learner is part of, and knowledge about the specific situation, e.g., what content is on the learner’s screen at the moment (see Brown et al., 1989). Furthermore, tools should be available which enable, support and possibly guide the cooperation. This refers to the application of scripted cooperation as a kind of implemented didactics (Dansereau 1988, Wessner et al., 1999); in this case, the environment has to provide cooperative learning methods and guide the group through a cooperative learning process. Finally, these different cooperation modes need to be integrated in one homogeneous learning environment in order to enable smooth transitions between these modes. The integration includes the “look and feel” for possibilities to contact peer learners, the management of cooperation sessions, group formation, and the creation of persistent artifacts as input for and output of cooperative learning processes. To what extent do existing systems fulfill these requirements? An increasing number of learning platforms are commercially available which aim at supporting cooperative training on the Web. These platforms, for example TopClass, WebCT, or Blackboard, can be characterised as learning

24

M. Wessner and H.-R. Pfister

management systems enriched by communication facilities. In these environments, cooperation on demand is supported but the cooperation is not integrated into the courses or into the logic of the course material. For example, it is up to the sender of a message to provide the necessary context; or, the receiver has to reconstruct the needed context information. The type of communication is completely arbitrary with respect to course content and structure. Table 1 provides a short overview of some of the common cooperation features available in commercial learning platforms (state: end of 2000; based on Landon, 2000). Table 1. Cooperation Tools of Learning Platforms.

Web CT Black board Top Class Learning Space Learn Linc

Asynchronous Synchronous Cooperation Communication Communication EBulNews Chat Au- Vi- White Appli- Shared mail letin Groups dio deo board cation BrowBoards Sharing sing x x x x x x x x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

Virtual Spaces

x

x

x

x

x

x

x

x

x

x

x

x

x

x

These platforms provide a rich menu of general synchronous and asynchronous tools, embedded on different levels into the course management system. However, none of these tools is part of a course itself, and none is integrated into the didactical approach according to which a course has been designed. Different from these learning management systems are cooperation platforms which are more tailored (or tailorable) to the purpose of learning and teaching, such as TeamWave or VITAL. However, the learning process and the cooperation process, i.e., the content and cooperation about the content, are not systematically integrated. Some research prototypes aim at a theoretically founded integration of cooperation into the learning environment, such as WebCaMILE (Guzdial et al., 1997) or WebGuide (Stahl 1999). Both systems provide support for the anchored cooperation approach (CTGV, 1994), e.g., by linking web pages to threads of discussions, but both are limited to a small set of types of cooperative activities. The FITS/CL framework (Supnithi et al., 1999; Inaba et al., 2000)

Points of Cooperation

25

represents another line of research in CSCL which evolved from the artificial intelligence field. This approach aims at detecting appropriate situations for collaboration, forming effective learning groups supported by intelligent agents, and facilitating the group interaction based on an elaborated knowledge model of learning goals as well as of learner characteristics. In sum, learning platforms can be classified along two dimensions, yielding a two-by-two matrix: the integration of cooperation into the learning environment can be high or low, and the support for cooperative learning methods can also be high or low (Wessner, 2001). Table 2 shows some examples for each combination. A system in the low/low field is VITAL (Pfister et al., 1998). In VITAL, learners work, learn and communicate in different types of virtual rooms, for example, there are specific private study rooms and auditoriums for lectures and presentations. Hence, the environment is basically a virtual space people “live” in, but their learning activities have to be organized and structured by themselves. Other examples of this type are BSCW and TeamWave. A high/low type of learning sytem is Blackboard or any of similar large scale platforms such as WebCT or TopClass. Communication and cooperation is integrated into an overall management structure of learning activities: emails can be send to selected tutors of groups, bulletins boards and newsgroups are available with sophisticated access control. A low/high type of sytem is CLARE which supports the collaborative construction of representations for scientific discourse. Another example is CaMILE, based on the specific didactical model of anchored collaboration; though support for cooperative learning is restricted to this one method, support for it is high. Finally, the learning platform L3, described in section three, aims at combining support for various cooperative methods with a high integration of cooperation into the learning environment and the learning process. Table 2. Classification of Learning Systems. Support for cooperative learning methods

Integration into the learning environment

low

high

low

VITAL, BSCW, TeamWave

CLARE, CaMILE

high

WebCT, BlackBoard, TopClass

L3

26

3.

M. Wessner and H.-R. Pfister

POINTS OF COOPERATION (POC)

In this section we introduce the notion of Points of Cooperation (PoC) and compare it with traditional web-based training. Logically, a Point of Cooperation (PoC) constitutes an opportunity to cooperate, given a specific learning context. From the course author’s point of view, a PoC denotes a logical locus within the learning activities emerging from taking a course, characterised by specific cooperation functionalities. Pragmatically, a PoC denotes the starting point as well as the process of cooperation within a specific context. From the learner’s point of view, a PoC is the specific user interface which symbolizes the activiation of a PoC as well as the actual sequence of cooperative activities via a specific tool.

3.1 Types of Points of Cooperation Extending the above definition of cooperative learning, we call the extent to which a cooperative activity is incorporated into a specific course the degree of contextuality of a cooperative activity. Low contextuality means that cooperation, i.e., when it occurs, who participates, what the topic is, etc., is completely arbitrary and not even necessarily bound to a specific course; high contextuality means that cooperation is a necessary and didactically justified component of the learning process and directly associated with specific course units and intermediate learning goals. Hence, context includes all cognitive, didactic, social and organizational information that has or could have an impact on the communication itself. So far, we make no attempt to quantify contextuality in any way. Qualitatively, we differentiate between three types of cooperative activities with respect to contexuality: (i) Generic cooperation, (ii) Spontaneous cooperation, and (iii) Intended cooperation.

3.2 Generic Cooperation and GPoCs A cooperative activity is a generic cooperation if it is not integrated into the course the learner is enrolled in or currently working on. A CSCL environment might offer a wide range of cooperation facilities which can be used to cooperate with other users of the system. For example, a mail/news interface provides access to mail and news facilities for arbitrary exchange of messages. We call elements of the learning environment which offer functionality to start a generic cooperation Generic Points of Cooperation (GPoC). Depending on the concrete implementation, GPoCs can be offered after selecting cooperation partners (“start audio conference with ”, “send email to ”), or the user first selects the specific tool (“News”,

Points of Cooperation

27

“Application Sharing”) and only then he or she selects and invites one or more cooperation partners. Generic Points of Cooperation do not need information about the learning context. Learners are free to use them whenever they want and for whatever purposes.

3.3 Spontaneous Cooperation and SPoCs A cooperative activity is a spontaneous cooperation if it is integrated into a specific course but not to a specific unit of this course. We call elements of the learning environment which offer functionality to start a spontaneous cooperation Spontaneous Points of Cooperation (SPoC). SPoCs are used for cooperative activities which are not strictly limited to specific content elements of the course, but relate to the course as a whole, e.g., asking a tutor for help, finding peer learners to discuss problems. In contrast to GPoCs, some basic context information is needed for the handling of SPoCs. For example for the execution of a SPoC “send message to tutor” or “ask a peer learner” the learning environment needs organisational information in order to select an appropriate tutor or peer learner of the course the user is currently working on. Sufficient context has to be provided for all participants to enable them to discuss the problem at hand; for example, they could be provided with a screen shot of the problematic course unit.

3.4 Intended Cooperation and IPoCs A cooperative activity is an intended cooperation if it is logically and didactically integrated into a course and located at a specific position or learning unit of the course. We call the components in the learning environment which offer the functionality to start an intended cooperation Intended Points of Cooperation (IPoC). Logical integration implies that IPoCs are connected to other parts of the course by various relations. For example, an IPoC could require some other parts of the course to be learned as a prerequisite for its execution. In the simplest case, an IPoC is just another “page” in the sequence of pages of a linear course and has to be completed before the learner can switch to the next page or section; in a hypertext structure, the way the IPoC is linked with other units defines the requirements to perform the IPoC. Didactical integration implies that the type of cooperative activity is didactically justified with respect to the type of knowledge to be learned and with respect to the position in the course. For example, a group discussion among three learners about the global consequences of the greenhouse effect might be justified only after having acquired a certain amount of basic knowledge. Or let us look at another IPoC type, the Pro/Con-dispute. Here, two learners take the roles of supporters of

28

M. Wessner and H.-R. Pfister

opposing points of view with respect to some controversial topic. The context consists of the acquired knowledge pertaining to that topic, and information about the constraints of the dispute (e.g., role, partner, time). The didactical rationale consists of two goals, one is to increase the ability to understand multiple perspectives about the chosen issue, and another is to practice the ability to defend one’s opinion with good arguments. The learning group here is minimal, i.e., it is a dyad, which makes the group formation fairly easy. Still, a pair of learners has to be found that fulfills all the contextual requirements. If the Pro/Con-dispute is controlled by the system, this could involve a strict floor control by strictly switching pro- and con-arguments, as well as a forced typing of messages according to a given schema. For example, the discussants have to indicate the type of message they are currently delivering: a supportive argument, a challenge, a critique, or something else.

PoC

GPoC

GPoC: Generic Cooperation • independent of course • arbitrary content • initiated by participants

SPoC

SPoC: Spontaneous Cooperation • associated with a course • arbitrary content • initiated by participants

IPoC

IPoC: Intentional Cooperation • logically and didactically incorporated in course • initiated by system

Figure 1. Overview of PoC-Types.

Whereas GPoCs and SPoCs are learner driven activities, IPoCs are system driven. With IPoCs the course author can define “when”, i.e. at which point in the locigal course structure, “what” cooperative activity should be performed. A cooperative method is defined by a set of parameters such as: – Group size: the minimum and a maximum number of learners to participate in the cooperative activity – Duration: a minimum and a maximum duration for the learning group’s cooperation for this IPoC

Points of Cooperation

29

– Instructions: a set of appropriate text pages with instructions how to perform the IPoC – Learning material: a set of documents (text, diagramms, etc.) the group uses as a starting point for the activity – Tools: a set of communication and cooperation tools; this includes tools specifically designed for a certain cooperative learning method as well as general-purpose tools – Structure: a system-controlled cooperative learning method, i.e., a socalled learning protocol (Pfister & Mühlpfordt, 2002; Wessner et al., 1999) based on the scripted cooperation approach (Dansereau 1988), which guides the group through the cooperation. In sum, an IPoC is an integrated unit of a course which can be realized by different tools. Several types of IPoCs have been developed, though the set of potential IPoC types is infinite. For example, an IPoC such as “group discussion” denotes an activity to be performed at a certain point involving certain participants and referring to a specific topic. The actual discussion could then be realized either by a chat tool, an audio-conferencing tool, or something else. Furthermore, a learning protocol can be used to support a specific IPoC. For example, in an IPoC type “pro/contra-dispute” two participants discuss a critical topic. Each participant is assigned either the role of the proponent, generating arguments in favor of the position, or the role of a contrahent, generating arguments that challenge the position. A learning protocol can be used to structure the process. For example, an enhanced chat-tool can instantiate a learning protocol by assigning the roles (pro, con, observer) and guiding the discourse by floor control and by defining the type of message to be exchanged. Table 3 shows some examples for IPoC types together with values for selected parameters: Table 3. IPoC Types. IPoC Type Small Group Discussion Brainstorming Pro / Contra Dispute

Min/Max No. Documents / of Learners Material 3/5 Text 2/10 2

Topic, List Of Seed Words Positions, Background Info

Tools

Other Parameters

Chat, Audio/Video Conferencing Shared Whiteboard, Brainstorming Tool Audio / Video Conferencing, Chat

Moderated By Tutor Send Results To The Tutor

30

M. Wessner and H.-R. Pfister

3.5 The Berlin Model of Didactics As a side-step, we will briefly discuss the relation between the PoC concept and traditional Web-based learning using the Berlin model of didactics as a frame of reference. According to the Berlin model of didactics (Heimann 1962; Schulz 1965) didactical activities are characterised by four fields which define the range of necessary or possible didactical decisions: Intentions, content, methods, and media; all fields are interdependent in intricate ways (Figure 2). Furthermore, didactical activities are constrained by two general conditions: anthropological-psychological and socio-cultural conditions. They characterize the individuals participating in the activity and the socio-cultural situation in which these individuals are embedded, ranging from the small groups to society as a whole. Finally, each didactic activity has consequences for the participating individuals (anthropologicalpsychological consequences, e.g., enhancing one’s competence), as well as for the group or the society (socio-cultural consequences, e.g., changes in attitudes or values). The Berlin model of didactics was developed with a twofold intention: (i) It provides dimensions to describe didactical activities and thereby serves as an instrument for the structural analysis of the decisions made as a basis for the reflection on the given conditions; (ii) it provides a schema to plan didactical activities in a structured way, moving from the conditions to the decision fields. This model now serves to describe and compare the two approaches, the traditional web-based training (WBT) and WBT enriched with Points of Cooperation (PoCs).

Points of Cooperation Socio-cultural constraints

Anthropologicalpsychological constraints

Intentions

Content

Methods

Media

Socio-cultural consequences

31

Anthropologicalpsychological consequences

Figure 2. The Berlin Model of Didactics.

3.6 Traditional Web-based Training Traditional web-based training (WBT) is defined by the use of web-based media as a means to communicate the intentions, contents and methods between the author and the learner. This includes various file formats for texts, pictures, sounds, videos and animations, and increasingly interactive elements (e.g., realised via Java-Applets). Due to its individual setting, the range of applicable didactical methods is limited to individual activities, i.e., the consumption of learning material and interaction with its content. The content of a WBT is solely provided by the author. Contributions of learners are not visible for the author or other learners. With respect to intentions, it is widely accepted that WBT has its strength in conveying factual knowledge. Due to its media restriction and individualistic usage, other intentions cannot be adequately addressed, e.g., to play an instrument, to perform a dialogue in a foreign language or to acquire a certain social skill.

3.7 WBT with PoCs Cooperative WBT, i.e., WBT augmented with PoCs, is characterized by the usage of web-based presentation and interaction media as well as communication facilities to communicate the intentions, contents and methods of the didactic activity. With these communication facilities and the

32

M. Wessner and H.-R. Pfister

availability of other persons as contributors the range of applicable methods increases enormously: In addition to the individual acquisition of knowledge by interacting with the content the learner can process the content by interacting with other persons (learners, tutors, and others) and thereby get a deeper understanding of the content. The learner also can create content individually or cooperatively and discuss it with other participants, e.g., a tutor or an expert. Based on the richer media facilities, additional methods, and the possibility to actively construct and communicate knowledge, cooperative WBT can exceed traditional WBT with respect to the intentions that can be achieved. Communication and cooperation allow pursuit of goals which depend on the involvement and/or judgement of other persons, such as conversation abilities or social competences. Especially, highly complex problems are now open to discourse, which might be the only efficient way to learn and understand highly complex subject matter (Feltovich et al., 1996). The Berlin model of didactics evolved in the context of classroom based synchronous teaching. For cooperative learning in general, e.g., asynchronously or geographically distributed teams, more fields could be added to describe cooperative activities such as date and time of the cooperation (fixed, spontaneous, on a regular basis; synchronously or asynchronously), technical or content-specific preconditions for a cooperation, recommendations with respect to cooperation partners etc. Further research is needed in this area. One should also note that the Berlin model can be used not only to analyse but also to plan traditional and cooperative WBT. Taking a look at cooperative activities, the decision for a specific activity which is - at first glance - a decision in the method field, also constrains the other fields: it is useful for specific types of goals, specific kinds of content and needs specific kinds of media. For example, when defining a brainstorming exercise, a list of seed words or a topic is provided (content), the brainstorming is defined according to a specific goal (evoke knowledge on a specific topic) and only some tools such as a shared whiteboard can be used (media).

4.

THE L3 PROJECT: INTEGRATION OF POCS IN A WEB-BASED LEARNING ENVIRONMENT

The L3-project, which stands for ‘Lifelong learning – continued training as a basic need’, was funded by the German Ministry of Education and Research (funding period was 1999 – 2002). It was a joint project involving twenty organisations ranging from research organisations, universities to companies and content providers. In the project a large scale integrated

Points of Cooperation

33

internet-based learning infrastructure for continued training and education has been developed. L3 courses are delivered by a central Service Center to Learning Centers where learners can learn individually, in groups within one Learning Center as well as in groups across Learning Centers. Courses are constructed in a modular way, consisting of a number of knowledge units. The L³ learning environment proposes a navigation sequence which follows one of several available learning strategies that can be adapted individually to each learner. Especially, a knowledge unit can be an IPoC, as explained previously. Figure 3 shows an overview of the components of L3 and the locus of IPoCs within the L3-approach. The authoring process consists of designing the course including the definition of IPoCs. Course design in L3 is strictly separated from media production, i.e., the logical-didactical structure of the course involves the definition of knowledge units (labeled with meta-data information) and the definition of relations among the units. The actual content is produced and managed independently, and can be in any format (HTML, video, etc.); the content files are then linked with the knowledge units during run-time. Courses are distributed via a centralized Service Center to the Learning Centers, where learners order and carry out the courses. Learning starts with the selection of a learning strategy, then, learners individually navigate through the content units. If they run into an IPoC, the system tries to establish an appropriate cooperation activity. Finally, after completion of the course, learners receive a certificate and are billed.

Authoring

course design

Distribution

media production

Service Center

Learning Centers

definition of IPoCs

Testing / Certification Accounting / Billing

Learning

strategy selection

individual navigation

Figure 3. Overview of the L3 Approach.

execution of IPoCs

cooperation

34

M. Wessner and H.-R. Pfister

The L3-project serves as a large use case for the PoC approach. The L³ learning environment provides GPoCs and SPoCs automatically during the course design. Since IPoCs are defined as knowledge units by the author during the authoring process, cooperative learning becomes an integrated part of each course. However, the actual instantiation of a Point of Cooperation is a non-trivial task: participants need to be selected according to appropriate criteria, matching communication tools have to be activated, the PoC execution has to be controlled in keeping with time and budget constraints, and the results of the cooperation processes need to be handled. We will now discuss some details of the L3 learnflow with a focus on the management of IPoCs.

4.1 Authoring IPoCs From the author’s perspective, an IPoC is formally equivalent to any other knowledge unit, except that it implies some kind of group process, whereas other knowledge units only imply the delivery of content. Once the author has decided that an IPoC of a specific type fulfills a didactical intention, he or she has to do two things: First, the relations to other units of the course have to be defined, and second, the specific parameters of the IPoC have to be determined. The relationships define the preconditions for the IPoC, i.e., what the learners need to have learned to be able to perform the IPoC, and the paramters define the operational constraints during runtime. Common parameters are the topic of the IPoC, additional information for discussants, e.g., arguments in a pro/contra-dispute IPoC, the number of required participants (minimum, maximum, recommended), and durations for the different phases of an IPoC. As a special authoring tool an IPoCEditor is available to enter the necessary parameters. Figure 4 shows an IPoC-Editor with some information about a ‘Brainstorming-IPoC’.

Points of Cooperation

35

Figure 4. IPoC-Editor for Brainstorming. (Thema=topic; Zeitlimit=time limit; Teilnehmeranzahl=number of participants; Startwörter=seed words; Ausführungshinweise=instructions).

4.2 Learning with PoCs During runtime, the L3 course engine generates a proposed navigation sequence according to the selected learning strategy. Currently, there is a choice of inductive, deductive or linear strategies. From the learner’s perspective, PoCs are contained in a so-called PoC-Pool, i.e., they are depicted as icons in a separate window. The PoC-Pool provides information on each PoC (e.g., type, topic, expected duration) and about the current state of the PoC (e.g., ready for execution, running, completed). IPoCs are automatically added to the learner’s PoC-Pool as soon as the learner runs into an IPoC as defined by the course author. The IPoC can be activated by the learner as soon as the required group for that IPoC has been formed by the system. The system provides all necessary information as well as manages all necessary tools to proceed through the cooperative learning process. When the IPoC has been processed, this is automatically reflected in the PoC-Pool. The behaviour of a PoC after activation in the PoC-Pool depends on the PoC’s state: If the PoC is ready for execution and there are enough other

36

M. Wessner and H.-R. Pfister

learners available who could serve as group members as required by the particular PoC and a tutor is available (if required by the PoC definition) the group formation process is activated. If the group formation is successful the PoC becomes active, otherwise it has to wait for more learners to arrive. If the PoC is active at the time of the PoC activation the corresponding cooperation tool is launched and fed with the information of the PoC definiton and all necessary connections to other group members are established. If the PoC activated in the PoC-Pool is already terminated the user can look individually at the result of the cooperative activity.

Figure 5. Context: A Cooperative Text Processing IPoC During Learning.

Figure 5 shows the interface of yet another IPoC type called Cotext, which is a method for cooperative text processing adopted from the scripted cooperation procedure introduced by Dansereau (1988). A group of learners work through a text consisting of several sections; for each section, one of the learners constructs a summary and the other learners comment on it. Roles of summarizer and commenter switch for each section. The Cotext window shows the current section of the original text in the left pane, the current summary in the middle pane, and the participants in the right pane (also indicating their current role). A simple chat area in the lower right part of the window provides a communication channel to discuss the summary and propose changes.

Points of Cooperation

37

The results of cooperative activities (summaries, diagrams, etc.) can be stored by the participating users in their private repositories in order to work on them later, individually or cooperatively, by using the result as an input for another PoC. There is a trade-off between using a rich file format preserving as much as possible of the PoC process with the result and using a wide-spread general-purpose file format, such as HTML, in order to enable a maximum of compatibility between the tools used for various PoC types.

5.

GROUP FORMATION

In this last section, we will elaborate on the group formation process, which is one of the critical steps in a distributed cooperative learning environment. We illustrate our approach to group formation as it is designed and implemented in the L³ learning environment; for simplicity, we just sketch the overall concept, a lot of special cases and additional complexities will not be mentioned. We distinguish two learning modes: In class mode learners are assigned to classes, usually a set of persons who signed up for the same course. All learners in a class start the course at the same time and synchronize their learning processes, according to a predefined schedule or according to a tutor or administrator. On the other hand, in individual mode all learners work self-controlled and self-paced. There is no common start or end times nor are there planned synchronization points. Not surprisingly, each learning mode requires a different approach to group formation. With respect to the mode of cooperation we make the usual distinction between synchronous and asynchronous IPoCs. For synchronous IPoCs, it is necessary that all participants are online at the same time, for asynchronous IPoCs there is some leeway as to when a learner actively contributes to the cooperative activity. Each combination of learning mode and collaboration mode demands different criteria which need to be taken into account for the group formation (see Table 4). For a synchronous IPoC the only requirement in class mode is that participants be online. Since in class mode all members are synchronized with respect to their position in the course material, for the instantiation of an IPoC it is guaranteed that all obligatory course units have been completed by the participating learners. For asynchronous IPoCs, even the online requirement can be dropped. Group formation is much more complex for participants in individual mode. Here, the chances that at a certain point in time, when an individual learner runs into a synchronous IPoC, a sufficient number of peer learners are online and waiting, are quite low. Appropriate members of a learning group need not only be online, they also have to fulfill the constraint that the obligatory course units have been

38

M. Wessner and H.-R. Pfister

studied. Since it is very likely that learners fulfill the necessary conditions to different degrees, the notion of learning distance has been introduced to quantify the proximity of a learner to the IPoC currently under consideration. Only learners within a reasonable distance are then selected as members of the learning group for the IPoC. Table 4. Modes of learning and collaboration. collaboration mode learning mode class

individual

synchronous

asynchronous

online

-

online requirements learning distance

requirements learning distance

The actual process of group formation can be done either manually or automatically. With manually we mean that a tutor, supported by appropriate management tools, forms learning groups following the constraints given by the IPoC (e.g., number of participants required). Automatic group formation means that the learning system extracts and manages all necessary information, selects a suitable set of participants, and eventually initiates the cooperative activity without help from a human tutor or administrator. Manual group formation makes most sense in class mode, where participants synchronize their learning processes often, know each other, and the tutor – being familiar with the members of his “class” - can base his formation on knowledge about the learners which is not modeled in the learner profile. Depending on the availability of the tutor and characteristics of the course and class, automatic group formation as described below could sometimes also be usefull in class mode. Automatic group formation is the standard in individual mode, when no tutor is responsible and potential group members are distributed unsystematically across space and time.

5.1 Manual Group Formation For manual group formation, the tutor uses a group formation tool (Figure 6) which presents all IPoCs for a given course and class and displays a list of all learners in the class. Information is displayed for the tutor about who completed an IPoC, how often an IPoC has been completed, who is waiting to perform an IPoC. The assignment of learners to learning groups is

Points of Cooperation

39

done by the tutor according to the information he or she receives from this overview; basically, the tutor is free to form groups any way he believes to be reasonable. When a learner activates an IPoC, the system assembles the group according to the assignments made by the tutor. The formation tool could include support mechanisms to propose segmentations of the class into groups of equal size or following any other criteria taken from the learner profiles.

Figure 6. Manual group formation. (Large window: overview; medium-sized window: assigning users to groups; small window: warning – not all learners assigned).

In the scenario illustrated in Figure 6 the tutor already did the group formation for the IPoCs 1 and 3. Only two groups were formed for the IPoC 2. As it can be seen in the medium-sized window for assigning users to group, the tutor can be assigned to groups as well, e.g., to fill up groups too small. For the same reasons, learners can be assigned to more than one group. The tutor needs additional functionality to handle IPoCs flexibly, e.g., for marking an IPoC as done for a learner if the execution of the IPoC is not possible or desired by the tutor.

40

M. Wessner and H.-R. Pfister

5.2 Automatic Group Formation The automatic group formation in the class mode works in principle as described above for the manual group formation. For each IPoC, the class is segmented in groups and learners are assigned to the groups. The assignment of learners to groups is done on demand, i.e., whenever a learner reaches an IPoC or activates an IPoC in his PoC-Pool. Given an IPoC that has to be performed in groups of at least min and at most max learners (according to the IPoC definition), the segmentation algorithm tries to find groups of equal size from the set of n available learners in the class according to:

groupsize = round ( 12 (max+ min)) If n mod groupsize <> 0, the remains are spread randomly and evenly across the other groups. In individual mode, learners usually do not know each other and do not share a common background as learners in a class often do. The collaborative activities have a different quality than in the class mode, the coherence of groups are much lower, and the tutor, if available at all, has little background information for forming sound groups. Especially, since learners in individual mode are not synchronized, i.e., their current learning positions can range across the entire course, it is a fairly complex task for a tutor to integrate the different preconditions of all learners. Hence, in individual mode groups need mostly be formed automatically. In individual mode no segmentation is possible because the total number of learners in that course can change at any time. As described above an IPoC is automatically added to the PoC-Pool as soon as the learner runs into a cooperative unit as defined by the course structure. When a learner wants to execute the IPoC he or she initiates the group formation process. The system then identifies potential partners which satisfy the requirements for participation. The requirements can be defined as being in the same course and having the same IPoC in the PoC-Pool but not yet executed. Depending on the cooperation mode an additional requirement might be to be online at the moment. If a sufficient number of participants is detected, they are invited by the system to participate in the IPoC activities. If there aren’t enough potential peer learners, group formation is quite difficult, for example when a user waits for other learners to “arrive” in order to cooperatively work on an IPoC. It is not a priori clear, at which time in the future enough potential peer learners will be available to perform the cooperative activity. Figure 7 shows the PoC-Pool interface of the L3system; two IPoCs are marked as completed, one is marked as active, i.e.,

Points of Cooperation

41

the learner is currently performing the cooperative learning process, and four are new, i.e., waiting to be performed by the learner. In the L³ implementation, activating a new IPoC in the PoC-Pool starts the automatic group formation as described above; the manual group formation tool is not used.

Figure 7. PoC-Pool with completed, active, and new points of cooperation.

Group formation in individual mode implies a number of complexities which mainly arise from the independent movements of individual learners through their course pages. The group formation algorithm has in principle to be able to deal with any distribution of learners across the course units. Above we introduced the notion of learning distance to accomodate with that problem. Put simply, the learning distance measures the distance between a learner and an IPoC by counting the number of intervening units; since the course structure is not necessarily linear, the minimal path is calculated. A more sophisticated distance measure could include further information on the learning history and learning styles of the learners. However, a learner who initiates an IPoC will move away from that position while the system tries to form the learning group. So he or she might be too far away from the original IPoC to participate in the cooperation. The learning distance could be visualized to the learner in order to give him an awareness of the other learners and let him estimate when they will “arrive”. Another use of the learning distance is to prioritise the list of potential peer learners. For

42

M. Wessner and H.-R. Pfister

example, the system could ask those learners first to join the group who have a distance to the IPoC similar to the learner initiating the collaboration.

6.

CONCLUSION

This paper presents a novel approach to classify cooperative learning activities and to integrate cooperation systematically into web-based courses. The notion of Points of Cooperation (PoC) is introduced, and especially Intended Points of Cooperation (IPoCs) are analysed in more detail. The comparison with traditional web-based training shows a significant increase in possibilities to achieve and convey knowledge, and to apply different methods and media in web-based learning scenarios. We have argued that cooperative learning should be integrated as part of web-based courses throughout the complete learnflow, i.e., from authoring to execution and beyond. Authors of web-based courses should be able to define specific cooperative learning activities as part of the course design. An authoring tool which supports authors and course designers has to provide a selection of IPoCs fitting different didactical needs. It is also desirable if authors get some advice on the question when an IPoC is didactically reasonable to include in a course; for example, depending on the type of knowledge (factual, procedural) to be acquired or depending on the type of media available (text chat, video conferencing) appropriate IPoCs could be proposed. In the L³ project we have developed a series of IPoC types and corresponding tools, including the IPoC-Editor to support the authoring process, the PoC-Pool to initiate and manage cooperative activities, and the tools to perform the cooperative activities as defined by the IPoC type. A large scale evaluation has to show the effectiveness and acceptance of IPoCs by authors, tutors, and learners. In another project, we plan to experimentally test the effects of two IPoC types in small and medium sized groups of learners. Further empirical evaluations will show whether the postulated improvement of web-based training by incorporating PoCs - so far only based on theory - can be observed in lab experiments and in the field as well.

7.

LATER WORK

Several lines of research originated from the work described in this chapter. First of all, the L³ platform has been completed and evaluated in a variety of settings (see Wessner 2005a). The need of mechanisms to secure a critical mass of users in the system, help for authors to select appropriate

Points of Cooperation

43

cooperation methods, increased flexibility to tailor the system to different learning contexts, and improved technical frameworks to minimize the effort for developing new cooperation tools are some of the most important insights gathered there. Another line of research focused on what we call learning protocols, i.e., IPoCs as a computer-controlled cooperative learning method. Learning protocols are a variant of scripted cooperation tailored to text-based synchronous learning discourses on the macro- as well as on the micro-level (Pfister, 2005; Pfister & Mühlpfordt, 2002; Pfister, Mühlpfordt & Müller, 2003). On the macro-level, learning protocols determine the sequence of discourse types such as ‘giving explanations’ or ‘pro/contra-argumentation’, and assign respective roles to the participants; on the micro-level, learning protocols provide support for referencing and typing of contributions (see Soller & Lesgold, this book). It has been shown that learning protocols can under certain circumstances increase learning performance (Oehl & Pfister, 2005; Pfister & Mühlpfordt, 2002), change communication behaviour (Mühlpfordt & Wessner, 2005), and can also be used for re-learning and recapitulation (Pfister, Müller & Holmer, 2004). Haake and Pfister (2007) discuss how simple learning protocols, i.e., elementary IPoCs, can be connected to larger composite IPoCs, depending on the learning goal and the needs of the learners. A further line of research looks at modelling, capturing and using the cooperation context in order to support successful cooperation (Wessner, 2005a, Wessner, 2005b; see also Ogata, Matsuura & Yano, this book). The concept of IPoCs, i.e., the integration of cooperative learning in noncooperative learning material was also applied in the area of lecture-support (Wessner et al., 2003). Here, short cooperational learning activities are integrated in traditional powerpoint files or html pages. Students in the lecture hall use Personal Digital Assistants (PDAs) to participate in these cooperative episodes. Five years later (the original contribution was written in 2000/2001) not only research on computer supported collaborative learning has made much progress (Bromme, Hesse & Spada, 2005; Koschman, Hall & Miyake, 2002), but also efforts to integrating cooperative learning into web-based courses has seen some progress. Tools building on standards such as EML or IMS-Learning Design allow specification of a learning design that includes cooperative learning methods as integral components of the learning process and the learning environment (Koper, 2006; Koper & Tattersall, 2005). The IMS-LD specification includes the option to define learning design patterns, which can be adopted especially for collaborative designs; however, how collaboration tools and activities are instantiated during learning is still under discussion. An advanced tailorable system that interprets IMS-LD has

44

M. Wessner and H.-R. Pfister

been developed by Bote-Lorenzo et al. (2004), and an authoring tool called Collage which is specialized for the design of what the authors call collaborative learning flow patterns is described by Hernández-Leo et al. (2006).

ACKNOWLEDGEMENTS The concept of PoCs and the tools mentioned above were developed at GMD-IPSI (since 2001: Fraunhofer IPSI). We thank all our colleagues for valuable contributions. The L³ project was funded by the German Ministry of Education and Research under the grant 21B8196B. Work on learning protocols has been funded by the Deutsche Forschungsgemeinschaft (DFG) under research grant PF330/1-2 to Hans-Rüdiger Pfister.

REFERENCES Bote-Lorenzo, M.L., Vaquero-González, L.M., Vega-Gorgojo, G., Dimitriadis, Y., AsensioPérez, J.I., Gómez-Sánchez, E., Hernández-Leo, D.A. (2004). Tailorable Collaborative Learning System that Combines OGSA Grid Services and IMS-LD Scripting. Proceedings of the X International Workshop on Groupware, CRIWG 2004, Heidelberg: Springer, 305321. Bromme, R., Hesse, F., & Spada, H. (Eds.) (2005). Barriers and biases in computer-mediated knowledge communication - and how they may be overcome. New York: Springer. Brown, J.S., Collins, A., & Duguid, P. (1989) Situated cognition and the culture of learning. Educational Research, 18, 32-42. CTGV Cognition and Technology Group at Vanderbild (1994). The relationship between situated cognition and anchored instruction. Educational Technology, 34, 28-32. Dansereau, D.F. (1988). Cooperative learning strategies. In C.E. Weinstein, E.T. Goetz & P.A. Alexander (Eds.), Learning and study strategies: Issues in assessment, instruction, and evaluation (pp. 103-120). San Diego: Academic Press. Dillenbourg, P. (1999). Introduction: What do you mean by collaborative learning? In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 1-19). Amsterdam: Elsevier. Feltovich, P.J., Spiro, R.J., Coulson, R.L., & Feltovich, J. (1996). Collaboration within and among minds: Mastering Complexity, individually and in groups. In T. Koschmann (Ed.), CSCL: Theory and practice of an emerging paradigm (pp. 25-44). Mahwah, NJ: Erlbaum. Guzdial, M., Hmelo, C, Hübscher, R., Nagel, K., Newstetter, W., Puntembakar, S., Shabo, A., Turns, J., & Kolodner, J.L. (1997). Integrating and Guiding Collaboration: Lessons learned in computer-supported collaboration learning research at Georgia Tech. Proceedings of CSCL'97 (pp. 91-100). Toronto, Ontario, Canada. Haake, J., & Pfister, H.-R. (2007). Flexible scripting in net-based learning groups. In F. Fischer, I. Kollar, H. Mandl & J.M. Haake (Eds.), Scripting computer-supported cooperative learning. Cognitive, computational, and educational perspectives. New York: Springer.

Points of Cooperation

45

Heimann, P. (1962). Didaktik als Theorie und Lehre. Die deutsche Schule, Heft 9/1962. Hernández-Leo, D., Villasclaras-Fernández, E.D., Asensio-Pérez, J.I., Jorrín-Abellán, I.M., Ruiz-Requies, I., & Rubia-Avi, B. (2006). COLLAGE: A collaborative learning design editor based on patterns. Educational Technology & Society, 9, 58-71. Inaba, A., Supnithi, T., Ikeda, M., Mizoguchi, R., & Toyoda, J. (2000). How can we form effective collaborative learning groups? In Gauthier, G., Frasson, C., & VanLehn, K. (Eds.), Intelligent Tutoring Systems. Proceedings of the ITS 2000 (pp. 282-291). Berlin: Springer. Koper, R. (2006). Current research in learning design. Educational technology & Society, 9, 13-22. Koper, R. & Tattersall, C. (2005). Learning Design. A handbook on modelling and delivering networked education and training. Heidelberg: Springer. Koschmann, T., Hall, R., & Miyake, N. (Eds.). (2002). CSCL 2: Carrying forward the conversation. Mahwah, NJ: Erlbaum. Koschmann, T., Kelson, A.C., Feltovich, P.J. & Barrows, H.S. (1996). Computer-supported problem-based learning: A principled approach to the use of computers in collaborative learning. In T. Koschmann (Ed.), CSCL: Theory and practice of an emerging paradigm (pp. 83-124). Mahwah, NJ: Erlbaum. Landon, B. (2000). Comparative analysis of online educational delivery applications. http://www.ctt.bc.ca/landonline/ (February 28, 2001). Mühlpfordt, M. & Wessner, M. (2005). Explicit Referencing In Chat Supports Collaborative Learning. Proceedings of the CSCL 2005. O’Malley, C. (Ed.) (1994). Computer-Supported Collaborative Learning. New York: Springer. Oehl, M. & Pfister, H.-R. (2005). Using learning protocols for knowledge acquisition and problem solving with individual and group incentives. In P. Kommers & G. Richards (Eds.), Proceedings of the ED-MEDIA 2005 World Conference on Educational Multimedia, Hypermedia & Telecommunications (pp. 2097-2103). Norfolk, VA: Association for the Advancement of Computing in Education (AACE). Pfister, H.-R. (2005). How to support synchronous net-based learning discourses: Principles and perspectives. In R. Bromme, F. Hesse & H. Spada (Eds.), Barriers and biases in computer-mediated knowledge communication (pp. 39-57). New York: Springer. Pfister, H.-R. & Mühlpfordt, M. (2002): Supporting discourse in a synchronous learning environmment: The learning protocol approach. In G. Stahl (Ed.), Proceedings of the CSCL2002 Conference on Computer Supported Collaborative Learning (581-589). Hillsdale: Erlbaum (e-document). Pfister, H.-R. Mühlpfordt, M., & Müller, W. (2003). Lernprotokollunterstütztes Lernen – ein Vergleich zwischen unstrukturiertem und systemkontrolliertem diskursivem Lernen im Netz. Zeitschrift für Psychologie, 211, 98-109. Pfister, H.-R., Müller, W., & Holmer, T. (2004). Learning and re-learning from net- based cooperative learning discourses. In L. Cantoni & C. McLoughlin (Eds.), Proceedings of ED-MEDIA 2004 World Conference on Educational Multimedia, Hypermedia & Telecommunications (pp. 2720-2724). Norfolk, VA: Association for the Advancement of Computing in Education (AACE). Pfister, H.-R., Schuckmann, C., Beck-Wilson, J., & Wessner, M. (1998). The metaphor of virtual rooms in the cooperative learning environment CLear. In N. A. Streitz, S. Konomi & H. Burkhardt (eds.), Cooperative Buildings – Integrating Information, Organization, and Architecture (pp. 107-113). Berlin: Springer.

46

M. Wessner and H.-R. Pfister

Schulz, W. (1965). Unterricht - Analyse und Planung. In Heimann, P., Otto, G., & Schulz, W. (Eds.), Unterricht. Analyse und Planung. Hannover: Schroedel. Slavin, R.E. (1995). Cooperative learning: Theory, research, and practice. Needham Heights, MA: Allyn and Bacon. Stahl, G. (1999). Reflections on WebGuide: Seven issues for the next generation of collaborative knowledge-building environments. http://www.cs.colorado.edu/~gerry/ publications/conferences/1999/cscl99/. Supnithi, T., Inaba, A., Ikeda, M., Toyoda, J., & Mizoguchi, R. (1999). Learning Goal Ontology Supported by Learning Theories for Opportunistic Group Formation, Proceedings of AI-ED’99 (pp.67-74). Le Mans, France. Wessner, M. (2001). Software für e-learning: Kooperative Umgebungen und Werkzeuge. In R. Schulmeister, Virtuelle Universität - Virtuelles Lernen (pp. 185-210). München, Wien: Oldenbourg. Wessner, M. (2005a). Kontextuelle Kooperation in virtuellen Lernumgebungen. Lohmar: Eul. Wessner, M. (2005b). Chat Communication in Context. In T. Okamoto, D. Albert, T. Honda, F.W. Hesse (Eds.) The 2nd Joint Workshop of Cognition and Learning Through MediaCommunication for Advanced e-Learning (JWCL), September 28-30, 2005, Tokyo, Japan, pp. 34-39. Wessner, M., Dawabi, P., Fernández, A. (2003). Supporting Face-To-Face Learning With Handheld Devices. In: In B. Wasson, S. Ludvigsen, & H.U. Hoppe (Ed.): Designing for Change in Networked Learning Environments, Proceedings of the International Conference on Computer Support for Collaborative Learning 2003., pp. 487-491, Dordrecht, Kluwer Wessner, M., Pfister, H.-R., & Miao, Y. (1999). Using learning protocols to structure computer-supported cooperative learning. Proceedings of the ED-MEDIA'99 - World Conference on Educational Multimedia, Hypermedia & Telecommunications (pp. 471476). Seattle, Washington.

URLs for systems mentioned in the text: System URL Blackboard www.blackboard.com BSCW www.orbiteam.de CaMILE / WebCaMILE www.cc.gatech.edu/gvu/edtech/CaMILE.html CLARE csdl.ics.hawaii.edu/Research/CLARE/CLARE.html Learning Space www.lotus.com/learningspace (URL no longer valid; the product is no longer supported) LearnLinc www.learnlinc.com TeamWave Workplace http://www.markroseman.com/teamwave/ TopClass www.wbtsystems.com VITAL www.darmstadt.gmd.de/concert/projects/clear (URL no longer valid) WebCT www.webct.com WebGuide http://www.cis.drexel.edu/faculty/gerry/webguide/

Chapter 4 A COMPUTATIONAL TOOL FOR LIFELONG LEARNING: EXPERIENCING BREAKDOWNS AND UNDERSTANDING SITUATIONS

Kumiyo Nakakoji1,4, Masao Ohira2, Akio Takashima3, Yasuhiro Yamamoto1 1

Research Center for Advanced Science and Technology, University of Tokyo 4-6-1 Komaba, Meguro, Tokyo, 153-8904, Japan tel/fax: +81-3-5452-5286 2 Graduate School of Information Science, Nara Institute of Science and Technology 8916-5 Takayama, Ikoma, Nara, 630-0192, Japan tel: +81-743-72-5312; fax: +81-743-72-5319 3 Graduate School of Information Science and Technology, Hokkaido University West8, North13, Kita-ku, Sapporo Hokkaido, 060-8628, Japan tel: +81-11-706-7250; fax: +81-11-706-7808 4 SRA Key Technology Laboratory Inc., Japan email: {kumiyo, yxy}@kid.rcast.u-tokyo.ac.jp, [email protected], [email protected] Abstract:

Our approach to supporting lifelong learning involves regarding a breakdown as an opportunity for learning. In this approach, systems for lifelong learning need to support a learner in: (1) experiencing a breakdown, and (2) asking for information relevant to the breakdown situation. Knowledge-based critiquing systems, which have been studied to support these processes, monitor human performance within its computational environment, identify a potentially problematic situation, alert the user about the situation to make the user aware of the potential problem, and provide explanation for the criticism and information relevant to the problem. Although found to be effective, such critiquing systems depend on knowledge-bases that need to be built by knowledge engineers prior to the use of the system. Thus, such systems cannot afford synchronous collaborative learning among practitioners. To complement this aspect, this paper presents a new system, EVIDII (Environment for Visualizing Individual Differences of Impressions), which visualizes differences among associations made by individual group members between two sets, for instance, pictorial images and words. By interacting with EVIDII, a user is encouraged to experience a breakdown by encountering an unexpected association made by other group members, and to ask for more information about the association from the other members. This embraces

48

K. Nakakoji et al. further communication and knowledge construction among group members. Scenarios are provided to illustrate how learning takes place when using EVIDII.

1.

SUPPORT FOR LIFELONG LEARNING

As technologies evolve and the world rapidly changes, lifelong learning is no longer a luxury but has become a necessity (Fischer, Nakakoji 1997). Different from in-class learning, lifelong learning within the working context is characterized as follows: •

Lifelong learning has no curricula of what to learn. There is no list of items that learners have to address.

•

Although lifelong learning is recognized as crucial in improving and maintaining quality of work, learning can also be avoided. In many cases, practitioners have a way to deal with most situations with their existing skills and knowledge, even if they are aware of the existence of a better way if they would acquire new knowledge.

•

Most learning styles are constructive rather than instructive. Because lifelong learning has no pre-determined agenda, instruction materials cannot be prepared a priori. Instead, a practitioner learns in a constructive manner, by identifying a problem, collecting information relevant to the problem, and constructing a solution to the problem. Collected information, approaches used to frame the problem, as well as problem-solution mapping are learned by the practitioner and become part of the practitioner’s knowledge.

•

Such learning, therefore, is mostly situated. Learning takes place within the context of the learner’s problem at hand. Due to these characteristics, most traditional intelligent tutoring system approaches (Frasson, Gauthier, 1990), which presuppose a set of predefined curricula, are not applicable to support lifelong learners. To support lifelong learners, computational tools need to be applied within the work context of these learners. With this goal in mind, we have taken an approach that takes breakdowns as opportunities for learning (Fischer 1995). A breakdown is “not a negative situation to be avoided, but a situation of non-obviousness, in which the recognition that something is missing leads to unconcealing (generating through our declarations) some aspect of the network of tools that we are engaged in using” (Winograd, Flores, 1986, p.165).

Computational Tool for Lifelong Learning

49

According to Maturana and Varela (1998), when an organism learns, the structure of the organism changes. Such a change is triggered by perturbations caused by interacting with external environments. How the structure changes are not determined by the perturbations themselves, however, but by the structure itself; what these authors call a structuredetermined system. Our approach is to provide a computational environment that helps people experience breakdowns (Schegloff 1991). Breakdowns are triggers that serve as perturbations that cause structural changes of an organism, or knowledge construction. Our approach is based on the following claims: •

It is possible for a computational mechanism to make users experience breakdowns.

•

Experiencing a breakdown is a trigger that “may” cause structural changes. What is needed for ensuring structural changes, and thereby constructing knowledge, is to provide information relevant to the experienced breakdown.

•

Providing information relevant to the breakdown can be achieved through communication, either with computers or with other people. This chapter presents our approach to supporting lifelong learning within the working context through experiencing breakdowns, and through understanding their situations. To do so, computational tools need to support a learner (1) when experiencing breakdowns by encouraging the learner to encounter them, and (2) when understanding the situations by providing information through communications. Note that this type of communication is not merely “transmitting information;” It cannot be adequately described in terms of the communication-through-tube metaphor. “The phenomenon of communication depends on not what is transmitted, but on what happens to the person who receives it. And this is a very different matter from ‘transmitting information’” (Maturana, Valera 1998; p.196). Information provided through such communication needs to be made relevant to the learner’s problem situation where the breakdown was experienced.

50

2.

K. Nakakoji et al.

COMPUTATIONAL TOOLS FOR LIFELONG LEARNING

2.1 Computational Critiquing Systems We have studied computational critiquing systems in a variety of domains; KID for kitchen designs (Nakakoji, Fischer 1995), eMMaC for designing colors in computer graphics (Nakakoji et al., 1995), and IAMeMMa in image selections (Nakakoji et al., 2000). A computational critiquing system (critics, for short) is a knowledge-based human-computer cooperative problem-solving system. Such a system monitors human performance within its computational environment, identifies a potentially problematic situation, alerts the user to the situation to make the user aware of the potential problem, and provides an explanation for the criticism and information relevant to the problem as requested (Fischer et al., 1998). Our studies (Nakakoji, Fischer 1995; Nakakoji et al., 2000) have identified that critics support lifelong learning by making breakdowns as opportunities for learning. A breakdown is a problematic situation that a critic identifies. Two types of learning have been evoked by using critics (Nakakoji, Fischer 1995). First, the user learns when the critic explains why it found a certain situation to be problematic. It provides argumentation about the situation, pros and cons of alternative solutions, and possible effects. Second, the user learns by arguing back against the critic’s explanation. An expert, when being critiqued, usually argues against the system’s behavior, trying to justify why the user’s action should not be critiqued. This serves as a knowledge elicitation mechanism (Nakakoji, Fischer 1995; Nakakoji et al., 1998) as the user articulates otherwise tacit arguments regarding the situation. This often allows the user to reflect on what has been done and helps the user to externalize knowledge relevant to the situation. Having observed these situations, critics have been found to be very effective in supporting lifelong learners (Fischer 1995; Fischer et al., 1998). However, the obvious shortcoming of this approach is that critics work only when they have predefined critiquing rules, or knowledge on which the critics can base their behavior. The system can identify possible breakdowns only if the system has a prior understanding about how to analyze the task. The arbitrary identification of a trivial problem without using knowledgebases has already been found less effective in supporting learning (Owen, 1986). Although we have built a mechanism to allow a user to add and modify critiquing rules as the user uses the system (Nakakoji, Fischer 1995),

Computational Tool for Lifelong Learning

51

the core knowledge must be fed by system builders prior to the usage of the critiquing system.

2.2 A Complement to Critiquing Systems: The EVIDII System To complement this knowledge-based critiquing approach, we have built a computational environment with which practitioners can talk about breakdowns they experience. Using the system, communication among the practitioners serves as critics. That is, by using the computational environment, a practitioner identifies a breakdown, and by talking about the breakdown, individual practitioners are given opportunities to learn. EVIDII (an Environment for Visualizing Individual Differences of Impressions) is an interactive tool that visualizes differences of individual associations between two data sets on multiple two-dimensional spaces (Ohira et al., 1999; Ohira et al., 2000; Ohira 2003). EVIDII first asks collaborating group members to associate each object in one set with object(s) in another set. Then, the system provides interactive interfaces that visualize the relationships between the two sets of data—objects in one set and those in the other set—in terms of persons. Group members operate the EVIDII system together during their collaborative task. Communications are encouraged among the members based on what they find interesting and question-provoking using EVIDII. The visualization causes breakdowns during a group discussion, making the group members acquire knowledge about one another and resulting in new understanding about the problem, solutions, and the language the members use to communicate (Clark 1991). In the next section, we present scenarios of how EVIDII is used among practitioners and how learning takes place.

3.

SCENARIOS: USING EVIDII

In the scenarios to be discussed here, five kitchen designers use EVIDII to talk about a new kitchen design targeted for young couples. The scenarios are observed in actual user session studies using EVIDII. We have conducted the studies to understand how EVIDII encourages the designers’ learning processes through collaboration (Ohira 2003). In the scenarios, the designers have eight kitchen example pictures and twenty affective adjectives that illustrate aspects of various personal preferences and lifestyles, such as urban, casual, pretty, rich, or cool.

52

K. Nakakoji et al.

Each of the five designers associates one or more word with each kitchen picture. Figure 1(a) illustrates how one of the designers, Shingo, associates words from a list on the right to each of the eight kitchen pictures listed on the left. The other four designers make associations in the same manner. When all designers are done making associations, they sit together in a meeting room and the EVIDII system is shown on a computer-projected large screen. One designer acts as the operator of EVIDII, and the other four look at EVIDII’s visualizations of associations (Figures 1(b), (c) and (d)). In each of the visualization windows, a set of words is positioned in a two-dimensional space (see, for example, the middle window of Figure 1(b)). This positioning is done by one of the designers, who puts words with similar meanings close to each other. The operator clicks on the kitchen image icons in the upper-right of Figure 1(b) one by one. The system shows who associated which kitchens with each word by displaying kitchen images on top of assigned words. By looking at Figure 1(b), the designers become aware that only Shingo (represented with a face icon with yellow hair) associated a certain kitchen to a different word than the rest of the designers. Selecting a different kitchen image changes the visualization as EVIDII displays who selected which word for this kitchen (Figure 1(c)). Selecting a face icon in the lowerright corner makes EVIDII display which kitchen designs are associated with which word by this particular designer (in this case, by Shingo) (Figure 1(d)).

Figure 1. The EVIDII System.

Computational Tool for Lifelong Learning

53

The two scenarios below are based on parts of the actual transcripts we have taken from user studies (Ohira et al., 2000). (1) Kitchen #1 is associated with seemingly opposite words. Shingo associates it with “cool” whereas Tomoka associates it with “warm.” When this difference is discovered in viewing EVIDII visualizations, Tomoka asks a question to Shingo: Tomoka: Why do you think this kitchen [kitchen #1] is “cool?” Shingo: Why? Because it is cool. Tomoka: I found kitchen #1 is kind of warm and has a family-oriented atmosphere. Shingo: Oh that’s why you said it’s “warm.” You see, this picture [kitchen #1] has shadow in it. I think everything that has shadows and dark places is “cool.” In this scenario, Shingo and Tomoka discover their conflicts in how they perceive kitchen #1. By communicating about this breakdown, however, they can describe how they use words such as “cool” and “warm.” This results in their mutual understanding of how they use those two words. Through this conversation, Tomoka learns that having shadows in a picture may cause a person to think that it is “cool.” Not only Tomoka, but all the other designers learn that “cool” and “warm” are not necessarily conflicting associations. (2) When kitchen #2 is examined, Yasuhi finds that Tomoka associates two words with the kitchen: “family-oriented” and “commercial.” Yasuhi thinks that those two words are conflicting, and associating those two words to a single kitchen is not understandable. Yasuhi: Wait, wait, wait, ... (pause) ... don’t you think that Tomoka [’s association] is a little strange that he associated this kitchen [kitchen #2] with both “family-oriented” and “commercial,” which are two contradictory words? Tomoka: I like the color of this kitchen [kitchen #2] because it provides a family-oriented atmosphere. However, the shape of this drawer handle is kind of commercial-like, and I do not like the shape. In this scenario, the experienced breakdown also involves conflicting words, but this time they are associations of a single person, Tomoka. Yasuhi, who experiences this breakdown, poses a question to the group. Then the designers all discover that the supposed conflict is due to different aspects Tomoka looked at. For Tomoka, its color is family-oriented but the shape of drawer handles is commercial-like. Yasuhi has never paid attention to the shape of drawer handles until this moment, and Yasuhi learns the

54

K. Nakakoji et al.

existence of a new aspect based on the communication. Tomoka, who made the two seemingly conflicting associations, has not thought that these associations conflict with each other because it was so obvious to him. Tomoka learns by himself and becomes able to articulate why he made certain associations. The next section discusses underlying models of the learning supported by EVIDII.

4.

DISCUSSION

The scenarios provided in the previous section illustrate how the EVIDII system supports lifelong learners in experiencing breakdowns and in understanding the situations. This section illustrates the EVIDII approach with computational critiquing systems by using the model of a lifelong learning process. We then discuss crucial design elements provided and not provided in the current EVIDII.

4.1 Model of a Learning Process Figure 2 illustrates how lifelong learning takes place through xperiencing breakdowns and understanding the situations. First, a learner experiences a breakdown. This makes the learner aware of the need for more information relevant to the situation that has caused the breakdown. By obtaining the information, or knowledge, the learner may be convinced of why the breakdown happened and become able to integrate newly acquired information with pre-existing knowledge about the domain. This is one type of learning. Once given the information, the learner may not agree with the information (i.e., the way the breakdown situation was described) and may argue about the information provided. This is another type of learning. If the arguments articulated by the learner had not been explicitly mentioned before, then this is a situation when the learner’s tacit knowledge is partially externalized (Polanyi 1966). The experiential knowledge of the learner becomes the reflective knowledge through the process of learning (Norman 1993). Based on the model depicted in Figure 2, as we argued in the first section, computer environments can help lifelong learners in the process of breakdown-based learning in two ways: (1) when experiencing a breakdown, and (2) when requesting relevant information.

Computational Tool for Lifelong Learning

55

experiences a breakdown asks for more information

explanations

agrees and understands

disagrees and argues against

Figure 2. Lifelong Learning through Experiencing Breakdowns and Understanding the Situations.

Critiquing systems support both aspects for learning (Figure 3(a)). Critics monitor a user making a move in a solution space. The system continuously analyzes each move made by the user by using its knowledge-base, and as soon as it detects a problematic situation, the system presents a critiquing message to the user. This causes the user to experience a breakdown. When the user clicks on the critiquing message, the system can provide the user with explanations of why the system fired the critiquing message; that is, which rules were used to identify possible problematic situations. The user may be able to obtain further relevant argumentation about the critiquing rule. When presented with such relevant information, the user is either convinced and understands the situation, or the user argues against the system’s behavior, often justifying why the situation should not have been judged as problematic. The user learns about the situation and is given an opportunity to reflect on otherwise tacit knowledge used in solving the problem.

56

K. Nakakoji et al.

makes a move

makes associations visualizes the associations

analyzes explores the visualization

critiques Knowledge Base

experiences a breakdown asks for more information

experiences a breakdown asks for more information

retrieves relevant information understands / argues

makes associations

articulates / explains understands / argues

Figure 3. Support for Lifelong Learning: left: (a) the Critiquing Systems Approach; right: (b) the EVIDII Approach.

The EVIDII system also supports both aspects for learning (Figure 3(b)). However, the role distribution between people and the computer is a little different from that of the critiquing systems. With EVIDII, each of users make associations among two sets; for instance, as in the scenarios, between a set of words and a set of kitchen pictures. EVIDII then provides visualizations of collected associations. When a user interacts with the visualization, the user may experience a breakdown; for instance, by finding that a certain user made an association that is very different from those of the rest of the users. Then the user asks for information relevant to the breakdown situation by directly asking the group member who made the interesting association. The user being asked, as well as other participating users, will talk about why such interesting phenomena have been created. The user is given an opportunity to become aware of new perspectives for the domain or unfamiliar opinions about the domain that are being talked about by the participating users. This gives the user an opportunity to learn about the domain as well as to learn about the other participating users. Thus, they all become collaborative learners. The key difference between the critiquing system approach and the EVIDII approach is twofold. First, the way that a user is helped in experiencing a breakdown is different. With critics, the system continuously monitors the user’s performance and actively notifies the user of potential

Computational Tool for Lifelong Learning

57

breakdowns. In contrast, with EVIDII, the system simply provides visualizations, and it is the user who needs to interact with the visualization and to discover a breakdown situation by him/herself. The EVIDII approach needs the user to take a more active role in experiencing a breakdown. With critics, the user can stay in a more passive mode in terms of experiencing a breakdown. Second, the relevant information provided by the system is different. Most of critics depend on the existence of pre-constructed knowledge-bases. With critics, it is not the critiquing message itself that the user learns; rather, it is information retrieved relevant to the situation (Fischer et al., 1998). Such knowledge-bases need to be built by domain engineers prior to the use of the system. In this sense, the user of critics cannot obtain information that is not included in its knowledge-base unless the user him/herself comes up with new thoughts or becomes able to articulate a part of previously tacit knowledge. With EVIDII, the relevant information provided to the learner is coming not from a computer, but from other users using EVIDII at the same time. It embraces a live exchange of knowledge among the participants. The use of EVIDII often encourages people to articulate knowledge and thoughts that have never been externalized before.

4.2 How EVIDII Helps Learners Experience Breakdowns In the previous section, we argued that the critiquing approach and EVIDII nicely complement each other. While critics embrace a user to learn pre-stored knowledge through asynchronous communication, EVIDII embraces a user to learn “live knowledge” through synchronous face-to-face communication. Critics actively help a user experience a breakdown, whereas EVIDII needs a user to actively experience a breakdown. It is a challenging task to design a system that motivates a user to actively interact with the system and actively experience a breakdown. We have chosen an approach to use interactive visualizations of individual differences with other users to motivate the user to participate in the discovery process. Association is a simple but powerful scheme to illustrate individual differences. Figure 4 illustrates how associations are used in EVIDII. Take an association between pictures and words, for instance. There are two cases when a difference emerges. First, two users may experience the same picture differently. Second, two users may use the same word in a different manner, even if the two experienced the picture in the same manner. In the beginning of our studies with EVIDII, we anticipated that only the first type of difference would be discussed. In the studies, however, interesting discussions and more learning opportunities were identified when users

58

K. Nakakoji et al.

found that different people used the same word in different manners. The scenarios illustrated above provide nice examples. It is not that designers experienced the same kitchen design in different ways but that designers used the same word quite differently, resulting in different associations between a kitchen design and a set of words. It turns out that the use of EVIDII helps users develop shared ontology among group members.

Figure 4. Individual Differences of Impressions.

If associations are such an interesting scheme, we could just use a simple table format to illustrate associations and give the table to the users. A table is a simple, compact, yet precise format to illustrate associations. However, we argue that interactive visualization of associations is another essential element of the design of EVIDII. EVIDII provides many views in visualizing associations. In our observations, we have found that users of EVIDII always browse through different visualizations by clicking on each of the users, pictures, or words, one by one in order. By repeating this process, users often found something visually interesting and stimulating. In the current EVIDII, we use face-like icons to represent users. In one of our former user studies, we used simple numbers to represent different users, and not much lively discussion took place. Although we have not identified what factors were critical to make people really involved in interacting with visualizations and experiencing breakdowns, subtle design decisions, such as using friendly face-like icons rather than numbers to designate users, do matter. Representational issues need more careful attention in supporting this type of learning (Yamamoto et al., 2000; Yamamoto, Nakakoji 2005; Nakakoji 2005).

5.

CONCLUSION

In summary, through our user studies, we have discovered various effects of EVIDII, including:

Computational Tool for Lifelong Learning

59

•

Users become aware of other aspects/perspectives; their perspectives have been widened.

•

The system prevents users from suffering from having “tunnel vision” or fixed viewpoints.

•

Users have learned how to look at a certain picture or an artifact.

•

The use of system evokes a user’s mental simulation by making the user try to understand why a particular person has a different point of view. Throughout the case studies, we have observed situations similar to the scenarios presented above. Such conversations often started with phrases such as “Really?” or “Wao,” which indicates that there has been a breakdown experienced by the users while interacting with the system. Most such breakdowns were identified when a subject found differences or seemingly conflicting associations among the user and the other participants, among multiple participants, or even by a single participant. We have not observed any cases where the participants could not describe (verbalize) why a certain association was made. Once a breakdown was observed, communication regarding the breakdown was embraced, which triggered the participants to rationalize and verbalize why a certain action was taken, or association was made. It is through this rationalization process that the structural change occurs in a person’s mind. Once rationalized and verbalized, then such knowledge can be shared among participating collaborators and, most of all, the user him/herself is helped to have better understanding about how he/she has seen and will see the problem and solution. Although our approach has been discussed in the context of lifelong learning support, it is also applicable to classroom situations in which emphasis is placed on students’ self-paced constructive learning. We plan to continue the application of our breakdown-oriented learning support in a wider context and further refine the process model.

ACKNOWLEDGMENTS We would like to thank Kimihiko Sugiyama and Shingo Takada for early development of the concept presented in this chapter. We would also like to thank Gerhard Fischer and Center for LifeLong Learning and Design at University of Colorado for helping us develop a conceptual framework for the approach.

60

K. Nakakoji et al.

REFERENCES Clark, H. (1991). Arenas of Language Use, The University of Chicago Press & Center for the Study of Language and Information. Fischer, G. (1995). Learning Opportunities Provided by Domain-Oriented Desgin Environments, Computers and Exploratory Learning, A. A. diSessa, C. Hoyles, and R. Noss (Eds.), Springer-Verlag, New York, NY. 463-480. Fischer, G., Nakakoji, K., Ostwald, J., Stahl, G., Sumner, T., (1993). Embedding Critics in Design Environments, Readings in Intelligent User Interfaces, M. Maybury, W. Wahlster (Eds.), Morgan Kaufman Publishers, San Francisco, CA. 537-561, 1998 (reprint from Knowledge-Engineering Journal. Fischer, G. and Nakakoji, K. (1997). Computational Environments Supporting Creativity in the Context of Lifelong Learning and Design, Knowledge-Based Systems Journal, 10(1), 21-28. Frasson, C. and Gauthier, G. (Eds.), (1990). Intelligent Tutoring Systems: At the Crossroad of Artificial Intelligence and Education, Ablex Publishing Corporation, Norwood, NJ. Maturana, H.R. and Varela, F.J. (1998). The Tree of Knowledge: The Biological Roots of Human Understanding, Shambhala Publications, Inc., Boston, MA. Nakakoji, K. and Fischer, G. (1995). Intertwining Knowledge Delivery, Construction, and Elicitation: A Process Model for Human-Computer Collaboration in Design, KnowledgeBased Systems Journal: Special Issue on Human-Computer Collaboration, 8(2-3), Butterworth-Heinemann Ltd, Oxford, England, 94-104. Nakakoji, K., Reeves, B.N., Aoki, A., Suzuki, H. and Mizushima, K. (1995). eMMaC: Knowledge-Based Color Critiquing Support for Novice Multimedia Authors, Proceedings of ACM Multimedia’95, ACM Press, San Francisco, CA, 467-476. Nakakoji, K., Yamamoto, Y., Suzuki, T., Takada, S., Gross, M.D. (1998). Beyond Critiquing: Using Representational Talkback to Elicit Design Intention, Knowledge-Based Systems Journal, Elsevier Science, Amsterdam, 11(7-8), 457-468. Nakakoji, K., Ohira, M., Yamamoto, Y. (2000). Computational Support for Collective Creativity, Knowledge-Based Systems Journal, Elsevier Science, 13(7-8), 451-458. Nakakoji, K. (2005). Humane Requirements for Enabling and Nurturing Collective Creativity, Proceedings of the HCI International Conference (HCII), Las Vegas, CD-ROM. Norman, D.A. (1993). Things That Make Us Smart, Addison-Wesley Publishing Company, Reading, MA. Ohira, M., Yamamoto, Y., Takada, S., and Nakakoji, K. (1999). EVIDII: An Environment that Supports Understanding Differences Among People, Proceedigns of International Conference on Cognitive Science 99 (ICCS 99), 466-471. Ohira, M., Yamamoto, Y., Nakakoji, K. (2000). EVIDII: A System that Supports Mutual Understanding through Visualizing Differences of Impressions, Journal of Information Processing Society of Japan (in Japanese), 41(10), 2814-2826. Ohira, M. (2003). EVIDII: An Environment for Mutual Understanding and Idea Elicitation in Face-To-Face Inter-Cultural Communication, Doctoral Dissertation (in Japanese), Graduate School of Information Science, Nara Institute of Science and Technology. Owen, D. (1986). Answers First, Then Questions, User-Centered System Design, D.A. Norman, S.W. Draper (Eds.), New Perspectives on Human-Computer Interaction, Lawrence Erlbaum Associates, Inc., Hillsdale, NJ, 361-375. Polanyi, M. (1966). The Tacit Dimension. Doubleday, Garden City, NY.

Computational Tool for Lifelong Learning

61

Schegloff, E. (1991). Conversation Analysis and Socially Shared Cognition, Perspectives on Socially Shared Cognition, L.B. Resnick, J.M. Levine, and S.D. Teasley (Eds.), Chap. 8, pp. 150-171, American Psychological Association. Winograd, T. and Flores, F. (1986). Understanding Computers and Cognition: A New Foundation for Design, Ablex Publishing Corporation, Norwood, NJ. Yamamoto, Y., Nakakoji, K., Takada, S. (2000). Hands-on Representations in a TwoDimensional Space for Early Stages of Design, Knowledge-Based Systems Journal, Elsevier Science, 13(6), 375-384. Yamamoto, Y., and Nakakoji, K. (2005). Interaction Design of Tools for Fostering Creativity in the Early Stages of Information Design, International Journal of Human-Computer Studies (IJHCS), Special Issue on Creativity, L. Candy, E. Edmonds (Eds.), 63(4-5), 513535.

Chapter 5 MODELING THE PROCESS OF COLLABORATIVE LEARNING

Amy Soller1, Alan Lesgold2 1

Institute for Defense Analyses, Alexandria VA 22311

2

University of Pittsburgh School of Education, Pittsburgh PA 15260

Abstract:

1.

Supporting group learning activities requires an understanding of the process of collaborative learning. This process is complex, coupling task-based and social elements. We present a view of the collaborative learning process from multiple perspectives, highlighting those that drive explaining, criticizing, sharing, and motivating behaviors. Modeling and supporting these processes requires a fine-grained sequential analysis of the group activity and collaboration. The selection of a computational approach to perform this analysis should take into account the chosen perspective and the desired goal: to better understand the interaction, or to provide advice or support to the students. Examples of five different computational approaches for modeling collaborative learning are discussed: Finite State Machines, Rule Learners, Decision Trees, Plan Recognition, and Hidden Markov Models. We illustrate the Hidden Markov Modeling approach in detail, showing that it performs significantly better than statistical analysis in recognizing the knowledge sharer, and the knowledge recipients when students exchange new knowledge during learning activities.

INTRODUCTION

The key to modeling, analyzing, and understanding computer-supported collaborative learning lies in understanding the rich interaction between individuals (Dillenbourg, 1999). These interaction patterns contain information about the students’ roles, understanding of the subject matter, engagement, degree of shared understanding, and ability to follow and contribute to the development of ideas and solutions. A collaborative learning environment that can analyze sequences of learning interaction may

64

A. Soller and A. Lesgold

be able to determine, for example, when a student is falling behind in the group, and why. Analyzing sequences of collaborative learning interaction however, is not without its own challenges. Collaborative learning researchers face many of the same challenges that intelligent tutoring systems researchers have faced over the past 30 years, such as student, domain, and pedagogical modeling, knowledge representation, diagnosis, and remediation. The collaboration factor however also means that inferring the meaning and intention behind individuals’ problem solving actions may not be enough (especially when students are instructing each other), and that assessing students’ understanding may require at least some partial analysis of the rich conversation between group members. Each of the three steps of transcription, coding of speech acts and student actions, and classification of interaction patterns presents difficulties. The popularity and acceptance of internet-based text chat has eliminated the many hours that researchers spent in the past on transcription, before they could begin analysis. Various different schemes for coding dialog exist, and selecting one that fits the bill is also a challenge. This has led many researchers to develop their own schemes to meet the needs of their projects, only to find out that developing a dialog coding scheme is itself a research project! The third challenge, identifying patterns of coded interaction indicative of effective group learning, remains to be an impressively difficult area of research. This is due to the many interacting high-level factors (such as group cohesion, participation, and student role playing) that are involved in determining whether or not a group is effective (see the introduction to Part I), and the undefined process of mapping a series of low level student actions and conversational exchanges to these high-level group interaction behaviors. In this chapter, we explain why it has been so difficult to understand the process of collaborative learning, discuss an array of strategies for analyzing this process, and walk through a detailed analysis of one key aspect of this process – knowledge sharing. Knowledge sharing events are initiated when a student shares or exchanges new knowledge with his peers. How this new information is received and assimilated into the group activity depends on the sort of questioning, explaining, and critical discussion that may or may not follow the sharing event. We have found that Hidden Markov Models, a technology well known in the speech recognition community, are effective at analyzing coded sequences of knowledge sharing interaction. In the last part of this chapter, we discuss the underlying algorithm, and provide an example in which this technique is used to (1) identify the student playing the role of knowledge “sharer” during knowledge sharing conversation, and (2) determine the effectiveness of the interaction.

Modeling the Process of Collaborative Learning

2.

65

UNDERSTANDING THE PROCESS OF COLLABORATIVE LEARNING

Performing team tasks well means not only having the skills to execute the task, but also collaborating well with teammates. Collaborating well means, among other things, asking questions to gain a better understanding of key concepts, sharing and explaining ideas, and elaborating and justifying opinions. When group members’ combined skills suffice to complete the learning task, effective group work may result in greater overall achievement than individual learning (Doise, W., Mugny, G., and Perret-Clermont, 1975; Heller, Keith, and Anderson, 1992; Joiner, 1995). Students learning in effective teams benefit through both enhanced learning of the task, and improvement in the social interaction skills they need throughout their lives. Soller (2001) describes a comprehensive model of collaborative learning (summarized in Table 1) that compiles research ideas from educational psychology, computer-supported collaborative learning, and small group dynamics. The model describes potential indicators of effective collaborative learning teams, and proposes strategies for promoting effective peer interaction in an intelligent collaborative learning environment. Table 1. The Collaborative Learning Model elements, and strategies for supporting each element of interaction (Soller, 2001). Name of Model Element Participation (Jarboe, 1996) Social Grounding (Teasley & Roschelle, 1993) Active Learning (Brown & Palincsar, 1989) Performance Analysis & Group Processing (Johnson & Johnson, 1991) Promotive Interaction (Webb, 1992)

Description of Element All students share their ideas openly Students establish and maintain a shared understanding Students achieve a high quality of communication by practicing explanation, justification and motivation Students individually and collectively assess their progress Team members promote each other’s success by helping each other

Strategies for Promoting Effective Behavior Facilitate round-robin brainstorming sessions Choose roles to assign to students, and rotate roles at appropriate times Have computer or student play devil’s advocate to encourage critical thinking Provide feedback on group/ Individual performance

Help students recognize their peers’ needs and train them to respond by composing highquality explanations

The strategies outlined in Table 1 (such as assigning roles to students, or facilitating brainstorming sessions) describe actions that a computer could carry out to facilitate learning teams. Each strategy addresses a different

66

A. Soller and A. Lesgold

aspect of effective interaction. How does the system know which strategies to employ, and when? Answering this question requires the ability to dynamically analyze the group interaction based on an understanding of the collaborative learning process. Hence, we turn to a discussion of the different ways one might view this process. Construction. Many educators and philosophers believe that collaboration facilitates learning. There are multiple reasons for this belief. Social constructivists (e.g., Rorty, 1979), arguing that all knowledge is constructed by those who have it, go on to assert that learning is essentially an initiation into the belief (i.e., knowledge) system of a group. By this account, individual learning consists of noticing aspects of group activity and assimilating them. A more complete level of learning might, from this viewpoint, involve some questioning by the learner and some explanation from others in response to those questions. By this view, collaborative learning involves a combination of the learner noticing group activity that imparts new knowledge, and the group explaining its actions and thinking when the learner is curious. Criticism. A second tradition goes back to the dialectic of Aristotle and even some of his predecessors and contemporaries (Owen, 1968). Aristotle tended to begin arguments by noting existing knowledge, assertions, and phenomena that seemed relevant to an issue. Inevitably, he would discern apparent contradictions in these data and attempt to understand how to reconcile these contradictions. He also introduced the idea that one person could help another person better understand the world by asking questions that exposed the apparent contradictions. Much of the history of logic is the evolution of ideas about how central contradictions are to complete understanding, but for our purposes, the central point is that people can help each other learn through some process of critique, exposing the apparent contradictions and incompleteness of each others’ thoughts. By this view, collaborative learning consists of examining each others’ assertions and challenging any apparently contradictory claims. Accumulation. A third view of collaborative learning has to do with the practicalities of accumulating knowledge. In this view, the job of the learner is to track down important knowledge and assimilate it (Herbart, 1896). It seems plausible (though not all social psychological studies confirm this) that two people searching for bits of knowledge will find more than one. By this view, successful collaborative learning involves participants making public what they have figured out – sharing knowledge. Motivation. A final view is that collaborative learning works because it is motivating. Festinger (1954) argued that people have a deep need to match their activity to that of others, a process he called social comparison. One aspect of motivation in group activity is simply that each person sees the

Modeling the Process of Collaborative Learning

67

others engaged in the learning task at hand and is thus motivated, via social comparison, to keep working himself. Beyond that, words of encouragement can pass from one person to another and provide motivation more explicitly. Any intelligent effort to contribute to collaborative learning by participating in conversations among learners will need to include the ability to recognize the likely presence or absence of one of these four possible group activities (explaining, criticizing, sharing, and motivating) and to offer suggestions based upon their presence or absence. For example, a system might note the absence of explanation activity and suggest that learning will improve if people explain ideas to each other. We suspect that intelligent coaching of collaborative learning will need to attend to a higher level of analysis than individual actions or speech acts such as explanation or assertion. Rather, effective collaborative learning is likely to involve a higher level unit of conversation, such as asking a question and then receiving an explanation, or making an assertion and hearing a criticism. For this reason, we are attempting to determine whether there are sequences of speech acts in learning collaborations that signal coherent and effective instances of explaining, criticizing, sharing, and motivating, and others that signal incomplete or less effective instances.

3.

DIALOG-BASED INTERACTION ANALYSIS

Coding and analyzing sequences of conversational interaction by hand, termed interaction analysis, helps us understand the patterns of conversation that lead to key learning events. Various coding schemes (e.g., Katz, O'Donnell, and Kay, 2000; Pilkington, 1997) have been developed for studying different aspects of interaction (e.g., identifying grounding behaviors, or achieving educational goals). Although coding schemes help in segmenting and analyzing dialogs, translating from one coding scheme to another sometimes requires the entire dialog to be re-coded. This, in effect, means that findings using one coding scheme may not transfer well to other schemes. Understanding and explaining hand-coded sequences of interaction is one thing – automating the identification of such sequences in collaborative learning environments is another. One clear constraint is that involving computer understanding of natural language. Natural Language Understanding technologies are rapidly advancing, yet they continue to be errorprone, computationally intensive, and time consuming. Cahn and Brennan (1999) explain that a system can represent or model a dialog using only the “gist” of successive contributions; a full account of each contribution, verbatim, is not necessary. The gist of a contribution can

68

A. Soller and A. Lesgold

often be determined by the first few words, or the sentence opener. Sentence openers such as “Do you know”, “In other words”, and “I agree because”, suggest the underlying intention of a statement. Associating these sentence openers with conversational acts such as Request Information, Rephrase, or Agree, and requiring students to use a given set of sentence openers, allows a system to automatically code dialog without having to rely on Natural Language parsers. Previous work has established promising research directions based on approaches that adopt this idea. Most approaches make use of a structured interface, comprised of organized sets of sentence openers (see Figure 1 in section 5 for an example). Students typically select a sentence opener from the interface to begin each contribution. One of the first systems to adopt this approach was McManus and Aiken’s (1995) Group Leader. Their system compared sequences of students’ conversation acts to those allowable in a four finite state machines developed specifically to monitor discussions about comments, requests, promises, and debates. The next section describes this system in more detail. Recently, several researchers have been interested in the tradeoffs involved in requiring students to use sentence openers to communicate. Baker and Lund (1997) compared the problem solving behavior of student pairs as they communicated through both a sentence opener interface and an unstructured chat interface. They found that the dialogue between students who used the sentence opener interface was more task focused. Jermann and Schneider’s (1997) subjects could choose, for each contribution, to type freely in a text area, or to select one of four short cut buttons, or four sentence openers. Jermann and Schneider discovered that, in fact, it is possible to direct the group interaction by structuring the interface, as Baker and Lund suggest. Furthermore, they found that the use of the sentence openers was more frequent overall than that of the free text zone (58% vs. 42%). Soller (2001) found that the types of conversation acts that group members use may indicate the quality of interaction. Their work suggests that conversations of effective groups include a balance of different conversational acts, and in particular an abundance of questioning, explaining, and motivation, whereas ineffective groups tend to show an imbalance of conversation acts, with an abundance of acknowledgement. Interaction Analysis research is rich with examples of dyads working together to solve problems (e.g., Joiner, 1995; Baker and Lund, 1997), however no comparable body of research exists for groups of three or more (but see Constantino-Gonzalez and Suthers, 2000, and Inaba and Okamoto, 1997). Research involving groups of three or more students may be applied more easily to groups of 4, 5 and 6, however research involving groups of only two may apply only to groups of 2. For example, a question asked by a student working in a dyad is understood to be directed toward his partner. In

Modeling the Process of Collaborative Learning

69

a group of 3, a student can put a question “on the table”. Depending on the level of activity in the group, and the students’ abilities to answer the question, it may or may not get answered, whereas in the dyad case, the question recipient is expected to respond. Determining which students understand the concepts that are being discussed is easier in the dyad condition because of the rigid turn-taking structure, and more difficult when more participants weaken the link between participation and understanding. Learning conversations involving 3 or more participants lack the tight logical sequencing of dyad conversations. These dialogs are full of gaps and overlaps. For example, Persons A and B might be conducting an instance of effective critique, but Person C might interject a comment in between the assertion and the criticism. Or, Person A might assert and Person B critique, but other things may intervene before Person A demonstrates some level of synthesis of the criticism into his own knowledge. In essence, the job of recognizing when to coach collaborative learning can be seen as one of figuring out how to detect meaningful speech act sequences that are embedded in longer sequences, and that are not necessarily contiguous in those longer sequences. The next section takes a look at a few different methods for detecting and analyzing such sequences.

4.

APPROACHES TO ANALYZING SEQUENCES OF COLLABORATIVE LEARNING INTERACTION

An array of different computational approaches to analyzing group learning activity is available (see Soller, Martínez-Monés, Jermann, & Muehlenbrock, 2005, for a review). Each approach targets a different type of interaction (e.g., action sequences, dialog sequences), or a different analysis perspective (e.g., construction, criticism – see section 2). The selection of an analysis method should be driven by the desired outcome: to better understand the interaction, or to provide advice or support to the students. In the interest of impacting the group learning process, the result of the analysis should reveal occurrences of events that the system knows how to target. We describe four different computational approaches for analyzing group learning interaction below, and a fifth approach in section 5. In some cases, the approach requires that the system designer adopt a particular knowledge representation. Knowledge representations may be used to describe systems, constrain users’ choices, or analyze users’ actions; we focus here on approaches aimed at analyzing the interaction.

70

A. Soller and A. Lesgold

4.1 Finite State Machines The Coordinator (Flores, Graves, Hartfield, and Winograd, 1988) was one of the first systems to adopt the finite state machine approach. In Flores et al.’s view, conversations represented intentions to take actions in an organization. Users sent messages to each other by choosing conversational acts (such as Request or Promise) from menus set up by the system. The system dynamically generated these menus based on a state transition matrix of “sensible next states”, displaying only those actions that would direct the conversation toward completion of action. The Coordinator was intended to create organizational change by making the structure of conversation explicit. Consequently, the first versions were often regarded as overly coercive. McManus and Aiken’s (1995) Group Leader system compared sequences of students’ conversation acts to those allowable in a four finite state machines developed specifically to monitor discussions about comments, requests, promises, and debates. The Group Leader was able to analyze sequences of conversation acts, and provide feedback on the students’ trust, 1 leadership, creative controversy, and communication skills . For example, the system might note a student’s limited use of sentence openers from the creative controversy category, and recommend that the student, “use the attribute of preparing a pro-position by choosing the opener of ‘The advantages of this idea are’”. The Group Leader received a positive response by the students, and paved the way for further research along these lines. Inaba and Okamoto (1997) describe a model that draws upon the ideas of finite state machines and utility functions. They used a finite state machine to control the flow of conversation and to identify proposals, while applying utility functions to measure participants’ beliefs with regard to the group conversation. For example, the utility function for evaluating a student’s attitude took into account the degree to which his teammates agreed with his proposals. Hybrid approaches such as this are key, as they broaden our ability to analyze interaction in new ways. Barros and Verdejo’s (2000) asynchronous newsgroup-style environment enables students to have structured, computer-mediated discussions on-line. Users must select the type of contribution (e.g., proposal, question, or comment) from a list each time they add to the discussion. The list is determined by the possible next actions given by a state transition graph, which the teacher may specify before the interaction begins. In this case, the state transition graph provides a mechanism to structure, rather than to 1

These four categories were proposed by Johnson and Johnson (1991), and are intended to define the skills involved in small group learning.

Modeling the Process of Collaborative Learning

71

understand, the conversation. Evaluating the interaction involves analyzing the conversation to compute values for the following four attributes: initiative, creativity, elaboration, and conformity. For example, making a proposal positively influences initiative and negatively influences conformity. These four attributes, along with others such as the mean number of contributions by team members and the length of contributions, factor into a fuzzy inference procedure that rates student collaboration on a scale from “awful” to “very good”. This work is seminal in combining a finite state approach with fuzzy rubrics to structure and understand the group interaction. A closer look at interaction sequences containing both task and conversational elements may help in composing rubrics for dynamically evaluating learning activity, enabling a facilitator agent to provide direction at the most appropriate instances.

4.2 Rule Learners Katz, Aronis, and Creitz (1999) developed two rule learning systems, String Rule Learner and Grammar Learner, which learn patterns of conversation acts from dialog segments that target particular pedagogical goals. The rule learners were challenged to find patterns in the hand-coded dialogs between avionics students learning electronics troubleshooting skills and expert technicians. The conversations took place within the SHERLOCK 2 Intelligent Tutoring System for electronics troubleshooting (Katz et al., 1998). The String Rule Learner, which searches for patterns common to a training set, discovered that explanations of system functionality often begin with an Identify or Inform Act. The Grammar Learner, which develops a probabilistic context-free grammar for specified conversation types, learned that explanations of system functionality not only begin with an Inform statement, but may go on to include a causal description, or another Inform 2 Act followed by a Predict Act . Rule learning algorithms such as these hold promise for classification and recognition tasks, and may prove useful tools for assisting in the sequential analysis of learning conversations.

4.3 Decision Trees and Plan Recognition Constantino-Gonzalez and Suthers (2000) system, COLER, coaches students as they collaboratively learn Entity-Relationship modeling, a formalism for conceptual database design. Decision trees that account for

2

The coding terminology used here has been altered from the original for brevity and clarity.

72

A. Soller and A. Lesgold

both task-based and conversational interaction are used to dynamically advise the group. This approach is described in the next chapter in this book. Muehlenbrock (2001) takes a plan recognition approach to analyzing collaboration processes. In their approach, the system maps actions taken on a shared workspace to steps in a partially ordered, hierarchical plan. This approach enables the system to identify sequences of student activity that suggest coordination or task-related conflicts without imposing user interface constraints on the collaborative interaction. Both Constantino-Gonzalez and Suthers, and Muehlenbrock and Hoppe have implemented novel ways to analyze group members’ actions on shared workspaces, and have successfully inferred domain independent behaviors from information based on the frequency and types of domain related actions. Combining such approaches with an analysis of the rich conversation between peers as they discuss their problems, ask questions, and probe their teammates for explanations will enable computer tutors to help instructors more fully address the pedagogical and social needs of the learning group. More work is needed to understand how students communicate, and to apply this knowledge in developing computational methods for determining how to best support and assist the process of collaboration. In the next section, we describe an approach to analyzing collaborative learning using Hidden Markov Models. In section 2, we discussed four main processes involved in collaborative learning conversation: explaining, criticizing, sharing, and motivating. Here, we focus on the process of knowledge sharing.

5.

EXAMPLE: MODELING KNOWLEDGE SHARING USING HIDDEN MARKOV MODELS

At the beginning of this chapter, we described knowledge sharing as one way to view the process of collaborative learning. In fact, the way in which knowledge is shared, and the parties involved (the knowledge sharer and the knowledge recipients), determine to a large extent whether or not that knowledge will be critiqued, and how it will be constructed or changed, and assimilated. For this reason, we have chosen to take a closer look at how effectively learners exchange the knowledge that they bring to the table during a collaborative session. We define a knowledge sharing episode as a series of conversational contributions (utterances) and actions (e.g., on a shared workspace) that begins when one group member introduces new knowledge into the group conversation, and ends when discussion of the new knowledge ceases. New

Modeling the Process of Collaborative Learning

73

knowledge is defined as knowledge that is unknown to at least one group member other than the knowledge sharer. Determining the effectiveness of a knowledge sharing episode involves the following three steps: 1. Determining which student played the role of knowledge sharer, and which the role(s) of receiver 2. Analyzing how well the knowledge sharer explained the new knowledge 3. Observing and evaluating how the knowledge receivers assimilated the new knowledge In this section, we describe an experiment for assisting in the identification and assessment of knowledge sharing episodes. We introduce Hidden Markov Models (HMMs), and walk through an example in which we successfully used them to accomplish step (1) above. We then briefly discuss ongoing work on steps (2) and (3), in which we have begun to train HMMs to successfully evaluate the effectiveness of knowledge sharing episodes. In our experiment, the team knowledge sharing process was analyzed by comparing the dialog segments in which students shared new knowledge with the group to the group members’ performance on pre and post tests. These tests targeted the specific knowledge elements to be shared and learned during the experiment. To ensure that high-quality knowledge sharing opportunities exist, each group member was provided with a unique piece of knowledge that the team needed to solve the problem. By artificially constructing situations in which students are expected to share knowledge, we single out interesting episodes to study, and more concretely define situations that can be compared and assessed. Experiments designed to study how new knowledge is assimilated by group members are not new to social psychologists. Hidden Profile studies (Lavery, Franz, Winquist, & Larson, 1999; Mennecke, 1997), designed to evaluate the effect of knowledge sharing on group performance, require that the knowledge needed to perform the task be divided among group members such that each member’s knowledge is incomplete before the group session begins. The group task cannot be successfully completed until all members share their unique knowledge. Group performance is typically measured by counting the number of individual knowledge elements that surface during group discussion, and evaluating the group’s solution, which is dependent on these elements. Surprisingly, studying the process of knowledge sharing has been much more difficult than one might imagine. Stasser (1999) and Lavery et al. (1999) have consistently shown that group members are not likely to discover their teammates’ hidden profiles. They explain that group members tend to focus on discussing information that they share in common, and tend

74

A. Soller and A. Lesgold

not to share and discuss information they uniquely possess. Moreover, it has been shown that when group members do share information, the quality of the group decision does not improve (Lavery et al., 1999; Mennecke, 1997). There are several explanations for this. First, group members tend to rely on common knowledge for their final decisions, even though other knowledge may have surfaced during the conversation. Second, “if subjects do not cognitively process the information they surface, even groups that have superior information sharing performance will not make superior decisions (Mennecke, 1997).” Team members must be motivated to understand and apply the new knowledge. At least one study (Winquist and Larson, 1998) confirms that the amount of unique information shared by group members is a significant predictor of the quality of the group decision. More research is necessary to determine exactly what factors influence effective group knowledge sharing. One important factor may be the complexity of the task. In Lavery et al.’s (1999) task, subjects were given hypothetical situations in which pregnant high school girls dropped out of high school, and were asked to determine what percentage of girls like the hypothetical cases would eventually return to finish school. Mennecke’s (1997) task involved prioritizing a set of 5 student applicants and selecting the best candidate for admission. These tasks are straightforward, short-term tasks that subjects may perceive as artificial. They require group members to aggregate information, but do not require a deep understanding of the knowledge involved. Tasks that require subjects to cognitively process the knowledge that their teammates bring to bear may reveal the importance of effective knowledge sharing in group activities. In the next section, we describe one such task.

5.1 Experimental Method Five groups of three were asked to solve one Object-Oriented Analysis and Design problem using a specialized shared workspace, while communicating through a sentence opener interface (section 3), containing sets of phrases organized in intuitive categories. Sentence openers provide a natural way for users to identify the intention of their conversational contribution without fully understanding the significance of the underlying communicative acts. The sentence opener interface is shown on the bottom half of Figure 1. The categories and corresponding phrases on the interface represent the conversation acts most often exhibited during collaborative learning and problem solving in a previous study (Soller, 2001). The communication interface features the following functionality:

Modeling the Process of Collaborative Learning

75

– Students can refer back to statements in the dialogue history by selecting the appropriate line (shown in bold in the figure). – Students may direct their comments to a particular team member by clicking on his picture. – A separate agenda window (not shown) enables students to keep track of their work. The current discussion item from the agenda is shown in the “current discussion” notepad above the discussion window. – While students are typing a contribution, before they hit <Enter> to send their comment to the group discussion window, a small cloud appears on their pictures to let their teammates know they are about to make a contribution – The picture of the last person to contribute to the discussion appears in a red box.

Figure 1. The shared OMT workspace (top), and sentence opener interface (bottom).

76

A. Soller and A. Lesgold

The specialized shared workspace is shown on the top half of Figure 1. The workspace allows students to collaboratively solve object-oriented design problems using Object Modeling Technique (OMT) (Rumbaugh, Blaha, Premerlani, Eddy, & Lorensen, 1991). Object-Oriented Analysis and Design was chosen because it is usually done in industry by teams of engineers with various expertises, so it is an inherently collaborative domain. An example of an OMT design problem is shown below. Exercise: Prepare a class diagram using the Object Modeling Technique (OMT) showing relationships among the following object classes: school, playground, classroom, book, cafeteria, desk, chair, ruler, student, teacher, door, swing. Show multiplicity balls in your diagram. The shared OMT workspace provides a palette of buttons down the lefthand side of the window that students use to construct objects, and link objects in different ways depending on how they are related. Objects on the shared workspace can be selected, dragged, and modified, and changes are reflected on the workspaces of all group members. Subjects. Fifteen subjects (five groups of three students each) volunteered to participate in the study. The subjects were undergraduates or first-year graduate students majoring in the physical sciences or engineering, none of which had prior knowledge of Object Modeling Technique. The subjects received pizza halfway through the four hour study, and were paid afterward. Procedure. The five groups were run separately. The subjects in each group were asked to introduce themselves to their teammates by answering a few personal questions. Each experiment began with a half hour interactive lecture on OMT basic concepts and notation, during which the subjects practiced solving a realistic problem. The subjects then participated in a half hour hands-on software tutorial. For the sentence opener portion of the tutorial, the subjects were told that they have decided to be roommates, and are moving into their new three bedroom apartment. The subjects used the sentence opener interface to answer questions such as, “How would you ask your roommates if it is ok to put the TV in the hallway?”, and “How would you apologize for the mess you made in the kitchen?”. During the tutorial, the subjects were introduced to all 36 sentence openers on the interface. They also learned how to use the shared OMT workspace, and practiced by drawing the examples from the lecture. The subjects were then assigned to separate rooms, received their individual knowledge elements, and took a pre-test. The three individual knowledge elements, distributed among the three group members, addressed key OMT concepts, for example, “Attach attributes common to a group of subclasses to a superclass.” Each knowledge element was explained on a separate sheet of paper with a

Modeling the Process of Collaborative Learning

77

worked-out example. The pre-test included one problem for each of the three knowledge elements. It was expected that the student given knowledge element #1 would answer only pre-test question #1 correctly, the student given knowledge element #2 would answer only pre-test question #2 correctly, and likewise for the third student. To ensure that each student understood his or her unique knowledge element, an experimenter reviewed the pre-test question pertaining to the student’s knowledge element before the group began the main exercise. The subjects were not told specifically that they hold different knowledge elements, however they were reminded that their teammates may have different backgrounds and knowledge, and that sharing and explaining ideas, and listening to others’ ideas is important in group learning. All groups completed the OMT exercise on-line within about an hour and fifteen minutes. During the on-line session, the software automatically logged the students’ conversation and actions (see Figure 2). After the problem solving session, the subjects completed a post-test, and filled out a questionnaire. The post-test, like the pre-test, addressed the three knowledge elements. It was expected that the members of effective knowledge sharing groups would perform well on all post-test questions.

Figure 2. The student action log dynamically records all student actions and conversation.

Most groups showed both instances of effective knowledge sharing and instances of ineffective knowledge sharing. In order for a knowledge element to be effectively shared, three things must happen: (1) the individual sharing the new knowledge (the “sharer”) must show that she understands it by correctly answering the corresponding pre and post test questions, (2) the concept must come up during the conversation, and (3) at least one group member who did not know the concept before the collaborative session

78

A. Soller and A. Lesgold

started (as shown by his pre-test) must show that he learned it during the session by correctly answering the corresponding post-test question (F. Linton, personal communication, May 8, 2001). Since there were 15 subjects, there were 30 opportunities for effective knowledge sharing. Ten of these were effective (i.e., they met all 3 criteria listed above), and four did not meet criteria (1). The sequences of coded dialog in which students shared and discussed their individual knowledge elements were extracted from the logs, and for each sequence, the student playing the role of knowledge sharer was identified. The sequences were then classified as either effective or ineffective, and used to train a system to analyze and classify new instances of knowledge sharing (see Soller, 2004a, for more detail). The next section elaborates on the training algorithm.

5.2 A Brief Introduction to Hidden Markov Models Hidden Markov Models (HMMs) were used to model the sequences of interaction present in the knowledge sharing episodes from the experiment. HMMs were chosen because of their flexibility in evaluating sequences of indefinite length, their ability to deal with a limited amount of training data, and their recent success in speech recognition tasks. In this section, we briefly explain the basics of the Hidden Markov Modeling approach. Markov Chains are similar to finite state machines, except that each arc from one state to another stipulates the probability of taking that arc, and all arcs leading out of a state must sum to one. The probability of taking a particular path through the model is then the product of all the probabilities along the path. Given a set of example (training) sequences, one can imagine constructing a Markov Chain describing all the different types of transitions that occur in those sequences. The main limitation of such a model is that it does not generalize well to new examples, even when an abundant amount of training data is available (Charniak, 1993). Hidden Markov Models were developed specifically to deal with the problem of sparse training data. Hidden Markov Models generalize Markov Chains in that they allow several different paths through the model to produce the same output. Consequently, it is not possible to determine the state the model is in simply by observing the output (it is “hidden”). Markov models observe the Markov assumption, which states that the probability of the next state is dependent only upon the previous state. This assumption seems limiting, however efficient algorithms have been developed that perform remarkably well on problems similar to that described here. Hidden Markov Models allow us to ask questions such as, “How well does a new (test) sequence match a given model?”, or, “How can we optimize a model’s parameters to best describe a

Modeling the Process of Collaborative Learning

79

given observation (training) sequence?” (Rabiner, 1989). Answering the first question involves computing the most likely path through the model for a given output sequence; this can be efficiently computed by the Viterbi (1967) algorithm. Answering the second question requires training an HMM given sets of example data. This involves estimating the (initially guessed) parameters of an arbitrary model repetitively, until the most likely parameters for the training examples are discovered. The explanation provided here should suffice for understanding the analysis in the next section. For further details on HMMs, see Rabiner (1989) or Charniak (1993). Student Subskill A Request

Attribute Opinion

C B C

Discuss Request Discuss

Doubt Elaboration Agree

A

Inform

Explain/ Clarify

A

Maintenance Apologize

Actual Contribution (not seen by HMM) Do you think we need a discriminator for the car ownership I’m not so sure Can you tell me more about what a discriminator is Yes, I agree because I myself am not so sure as to what its function is Let me explain it this way - A car can be owned by a person , a company or a bank. I think ownership type is the discrinator. Sorry I mean discriminator.

Actual HMM Training Sequence A-Request-Opinion C-Discuss-Doubt B-Request-Elaboration C-Discuss-Agree A-Inform-Explain A-Maintenance-Apologize Sequence-Termination Figure 3. An actual logged knowledge sharing episode (above), showing system coded subskills and attributes, and its corresponding HMM training sequence (below).

5.3 Using Hidden Markov Models to Select the Knowledge Sharer The software logs (e.g., Figure 2) from the five experiments were parsed by hand to extract the dialog segments in which the students shared their unique knowledge elements. Fourteen of these knowledge sharing episodes

80

A. Soller and A. Lesgold

were identified3. The segments varied in length from 5 to 62 contributions, and contained both conversational elements and action events. The sequences of conversation acts within the extracted episodes were used to train a Hidden Markov Model to identify the knowledge sharer. These conversational sequences ranged from 2 to 50 contributions. Figure 3 shows an example of one such sequence. The sentence openers, which indicate the system-coded subskills and attributes, are italicised. For each episode, the system was tasked to select one of the three participants as the knowledge sharer. Hidden Markov Models, however, are designed to output the probability that a particular sequence matches a trained model. The knowledge sharer role was therefore held consistent throughout the training data, and each test set was reproduced twice such that three test sequences were obtained, each featuring a different participant playing the role of knowledge sharer. Because of the small dataset, we used a 14-fold cross validation approach, in which we tested each of the 14 examples against the other 13 (training) sets, and averaged the results. Given the choice of three possible knowledge sharers, the 5 node HMMs chose the correct student as knowledge sharer for all 14 experiments, achieving a 100% accuracy. The baseline comparison is chance, or 33.3%, since there is a 1/3 chance of arbitrarily choosing the right student as knowledge sharer. The next best comparison is to count the number of Inform conversation acts each participant uses during the knowledge sharing episode, and select the student with the highest number in each test set. This strategy produces a 64.3% accuracy. The results are summarized in Table 2. This analysis shows that determining which participant is sharing new knowledge involves more than simply determining who is doing all the informing. The next question, then, is, “What exactly are knowledge sharers doing if they are not primarily providing information?” Table 2. Accuracy of three methods for selecting the group member playing the role of knowledge sharer. Selection Method 5 Node Hidden Markov Model Participant with Greatest Number of Inform Acts Baseline (Chance)

Accuracy (over 14 cross validation trials) 100% 64.3% 33.3%

A closer look at the trained HMM provides clues about why this approach works so well. Figure 4 shows one of the five node HMMs trained 3

These do not correspond directly to the 10 students who effectively learned during the session, as described in section 5.1, since one episode may result in 2 students learning, or one student may learn across several episodes.

Modeling the Process of Collaborative Learning

81

using thirteen conversational sequences totalling 180 contributions (outputs). The test sequence for this model is shown in Figure 3. The most probable path for this output sequence starts at state 5 (A-Request-Opinion, with .12 probability), and proceeds through states 5 (C-Discuss-Doubt, .04), 2 (BRequest-Elaboration, .03), 1 (C-Discuss-Agree, .09), 4 (A-Inform-Explain, .05), and 3 (A-Maintenance-Apologize, .06), ending in state 5 (SequenceTermination). This sequence is seen by the model as more likely than a sequence in which the knowledge sharer expresses doubt, and one of the other participants provides an elaborated explanation, for obvious reasons. The model in Figure 4 describes the possible ways that student A might share new knowledge with his teammates, and the possible ways that his teammates’ might react. The model is therefore a sort of compiled conversational model, and should be analyzed in the context of the sorts of examples it embodies. This analysis shows that Hidden Markov Models (HMMs) can effectively learn to recognize the knowledge sharer, and the knowledge recipients when new knowledge is shared during learning activities. The Hidden Markov Model approach was shown to perform significantly better than a statistical analysis approach. In a similar investigation, we trained HMMs to determine the effectiveness of knowledge sharing episodes. An episode is considered effective if one or more students learn the shared knowledge (as shown by a difference in pre-post test performance), and ineffective otherwise. The 6 node HMMs for determining effectiveness considered sequences including both task and conversational events, correctly classifying 92% of these sequences.

82

A. Soller and A. Lesgold

17% B-Inform-Elaborate 11% A-Acknowledge-Accept

15% B-Discuss-Doubt C-Acknowledge-Accept 11% C-Request-Information B-Discuss-Agree

.68

14% B-Acknowledge-Accept 13% A-Inform-Suggest

2

1

3 .55

.69

.99 .31

.34

.44

.32

4

5

.58

.08 14% C-Motivate-Encourage 13% B-Request-Opinion 12% A-Inform-Assert 11% A-Inform-Suggest

12% C-Inform-Suggest A-Request-Opinion B-Inform-Suggest

Figure 4. A summary of the five states HMM, trained using 13 conversational sequences in which A is the knowledge sharer. Outputs for each state that exceed an 11% threshold are shown in boxes.

The next section discusses the results of a follow-on study in which we analyzed the knowledge sharing effectiveness of an additional seven collaborative learning groups. We also point to a few articles that describe how the computational methods were extended, and how this research was grounded in the educational research literature.

6.

SUMMARY & RECENT WORK

The difficulties encountered in analyzing the process of collaborative learning can be attributed to the complex nature of group interaction, the limitations of computer-based natural language understanding, and the coupling of task-based and social elements that factor into collaborative activities. To help explain the complex nature of group interaction, we offered a multiple perspective view of the collaborative learning process in section 2, highlighting the perspectives that drive explaining, criticizing, sharing, and motivating behaviors. To address the limitations of natural language technology, we described an approach in section 3 that makes use of key phrases to help students identify the intentions of their contributions. In

Modeling the Process of Collaborative Learning

83

sections 4 and 5, we discussed examples of five different computational approaches that can be used to analyze and assess the collaborative learning process through a fine-grained sequential analysis of the group interaction in the context of the learning goals. These computational methods include Finite State Machines, Rule Learners, Decision Trees, Plan Recognition, and Hidden Markov Models. Since this article was first written, several other new approaches have been proposed, implemented, and evaluated. For example, Goodman, Linton, Gaimari, Hitzeman, Ross, and Zarrella (2005) used neural networks to train an agent-based system to recognize when students are experiencing trouble related to specific aspects of interaction. Their approach involved training neural networks with a combination of contextualized features such as segmented, coded (speech act) student dialog, conversation timing and delay, speaker identification, and utterance length. Analysis and assessment of collaborative learning should take into consideration the affordances, features, and constraints of the collaborative environment. For example, a distributed collaborative environment can impacts a group’s metacognitive processes simply by showing the members a graphical representation of their interaction. Jermann (2004) evaluated a system that displayed a color-coded model of actual and desirable participation rates to collaborators as they solved a traffic light tuning problem. For the condition in which the color-coded models suggested that chatting with teammates was relatively more desirable than taking simulation-based actions, students not only chatted more but also engaged in more precise planning activities. Soller, Martínez-Monés, Jermann, and Muehlenbrock (2005) reviewed a selection of environments that support collaborative learning interaction, including Jermann’s (2004) system. They characterized the systems within a simple classification framework that distinguishes between mirroring systems, which display basic actions to collaborators, metacognitive tools, which model and analyze the state of interaction, and coaching systems, which assess the collaborative learning interaction and offer advice. The research described in this chapter has been recently extended to develop a framework for understanding and assessing the effectiveness of distributed knowledge sharing, in particular why students experience knowledge sharing breakdowns. Soller (2004b) describes the framework, experimental design, and computational approach in detail. Soller (2004a) explains the output of the machine learning algorithm in the context of educational research and instructional theory, and describes how the findings can be applied in the distance learning classroom. For example, models of ineffective knowledge sharing include models in which new knowledge is not effectively conveyed, and models in which new knowledge is not

84

A. Soller and A. Lesgold

effectively assimilated. A system that can help an online instructor differentiate between these cases may also help him or her identify which strategies will best support distributed student collaboration and knowledge sharing.

ACKNOWLEDGMENTS We are grateful to Patrick Jermann and Kwang-Su Cho for helping to run the experiments, and for their suggestions and insights. Special thanks to H. Ulrich Hoppe and Frank Linton for fruitful discussions on the ideas presented here. Thanks also to Martin Muehlenbrock and Angeles Constantino-Gonzalez for helpful comments on a previous draft of this chapter. Amy Soller is appreciative of the Telecommunications Advancement Foundation in Japan for helping to fund her participation in the NTCL Workshop in Awaji-shima, Japan. This research was supported by a U.S Department of Education, grant R303A980192, and an Andrew Mellon Predoctoral Fellowship.

REFERENCES Baker, M. & Lund, K. (1997). Promoting reflective interactions in a computer-supported collaborative learning environment. Journal of Computer Assisted Learning, 13, 175-193. Barros, B. & Verdejo, M.F. (2000). Analysing student interaction processes in order to improve collaboration. The DEGREE approach. International Journal of Artificial Intelligence in Education, 11, 221-241. Brown, A. & Palincsar, A. (1989). Guided, cooperative learning and individual knowledge acquisition. In L. Resnick (Ed.), Knowledge, learning and instruction (pp. 307-336), Lawrence Erlbaum Associates. Cahn, J. & Brennan, S. (1999). A psychological model of grounding and repair in dialog. Proceedings of the AAAI Fall Symposium: Psychological Models of Communication in Collaborative Systems, Cape Cod, MA, 25-33. Charniak, E. (1993). Statistical Language Learning. Cambridge, MA: MIT Press. Constantino-Gonzalez, M. & Suthers, D. (2000). A coached collaborative learning environment for Entity-Relationship modeling. Proceedings of the 5th International Conference on Intelligent Tutoring Systems, Montreal, Canada, 324-333. Dillenbourg, P. (1999). What do you mean by “Collaborative Learning”. In P. Dillenbourg (Ed.) Collaborative Learning: Cognitive and Computational Approaches (pp.1-19). Amsterdam: Elsevier Science. Doise, W., Mugny, G., & Perret-Clermont A. (1975). Social interaction and the development of cognitive operations. European Journal of Social Psychology, 5(3), 367-383. Festinger, L. (1954). A theory of social comparison processes. Human Relations, 7, 117-140. Flores, F., Graves, M., Hartfield, B., & Winograd, T. (1988). Computer systems and the design of organizational interaction. ACM Transactions on Office Information Systems, 6(2), 153-172.

Modeling the Process of Collaborative Learning

85

Goodman, B., Linton, F., Gaimari, R., Hitzeman, J., Ross, H., & Zarrella, G. (2005). Using dialogue features to predict trouble during collaborative learning. User Modeling and User-Adapted Interaction: The Journal of Personalization Research, 15 (1), 85-134. Heller, P., Keith, R., & Anderson, S. (1992). Teaching problem solving through cooperative grouping. Part 1: Group versus individual problem solving. American Journal of Physics, 60(7), 627-636. Herbart, Johann Friedrich (1896). Herbart’s ABC of sense-perception, and minor pedagogical works, translated by William J Eckoff. New York: D. Appleton and Company. DeGarmo, C. (1979). Herbart and the Herbartians. Folcroft, PA: Folcroft Library Editions. (Original work published 1896.) Inaba, A. & Okamoto, T. (1997). Negotiation Process Model for Intelligent Discussion Coordinating System on CSCL Environment. Proceedings of the 8th World Conference on Artificial Intelligence in Education, Kobe, Japan, 175-182. Jarboe, S. (1996). Procedures for enhancing group decision making. In B. Hirokawa & M. Poole (Eds.), Communication and Group Decision Making (pp. 345-383). Thousand Oaks, CA: Sage Publications. Jermann, P. (2004). Computer Support for Interaction Regulation in Collaborative ProblemSolving. Doctoral Dissertation, University of Geneva. Jermann, P. & Schneider, D. (1997). Semi-structured interface in collaborative problem solving. Proceedings of the First Swiss Workshop on Distributed and Parallel Systems, Lausanne, Switzerland. Johnson D. & Johnson, R. (1991). Learning together and alone. Englewood Cliffs, NJ: Prentice Hall. Joiner, R. (1995). The negotiation of dialogue focus: An investigation of dialogue processes in joint planning in a computer based task. In C. O’Malley (Ed.) Computer Supported Collaborative Learning (pp. 203-222). Berlin: Springer-Verlag. Katz, S., Aronis, J., & Creitz, C. (1999). Modelling pedagogical interactions with machine learning. Proceedings of the Ninth International Conference on Artificial Intelligence in Education, LeMans, France, 543-550. Katz, S., Lesgold, A., Hughes, E., Peters, D., Eggan, G., Gordin, M., Greenberg, L. (1998). Sherlock II: An intelligent tutoring system built upon the LRDC Tutor Framework. In C.P. Bloom & R.B. Loftin (Eds.), Facilitating the Development and Use of Interactive Learning Environments (pp. 227-258). New Jersey: Lawrence Erlbaum. Katz, S., O’Donnell, G. & Kay, H. (2000). An approach to analyzing the role and structure of reflective dialogue. International Journal of Artificial Intelligence in Education, 11, 320-343. Lavery, T., Franz, T., Winquist, J., Larson, J. (1999). The role of information exchange in predicting group accuracy on a multiple judgment task. Basic and Applied Social Psychology, 2(4), 281-289. McManus, M. & Aiken, R. (1995). Monitoring computer-based problem solving. Journal of Artificial Intelligence in Education, 6(4), 307-336. Mennecke, B. (1997). Using group support systems to discover hidden profiles: An examination of the influence of group size and meeting structures on information sharing and decision quality. International Journal of Human-Computer Studies, 47, 387-405. Muehlenbrock, M. (2001). Action-based Collaboration Analysis for Group Learning. Amsterdam: IOS Press. Owen, G. E. L. (ed.). (1968). Aristotle on Dialectic: The Topics. Proceedings of the Third Symposium Aristotelicum. Cambridge: Cambridge University Press.

86

A. Soller and A. Lesgold

Pilkington, R. (1997). Analysing educational discourse: The DISCOUNT scheme. (CBLU Technical Report No. 019703). The University of Leeds, Leeds, UK: Computer Based Learning Unit. Rabiner, L. (1989). A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257-286. Rorty, R. (1979). Philosophy and the Mirror of Nature. Princeton: Princeton University Press. Rumbaugh, J., Blaha, M., Premerlani, W., Eddy, F., & Lorensen, W. (1991). Object-Oriented modeling and design. Englewood Cliffs, NJ: Prentice Hall. Soller, A. (2001). Supporting social interaction in an intelligent collaborative learning system. International Journal of Artificial Intelligence in Education, 12(1), 40-62. Soller, A. (2004a). Computational Modeling and Analysis of Knowledge Sharing in Collaborative Distance Learning. User Modeling and User-Adapted Interaction: The Journal of Personalization Research, 14 (4), 351-381. Soller, A. (2004b). Understanding knowledge sharing breakdowns: A meeting of the quantitative and qualitative minds. Journal of Computer Assisted Learning, 20, 212-223. Soller, A., Martínez-Monés, A., Jermann, P., & Muehlenbrock, M. (2005). From Mirroring to Guiding: A Review of State of the Art Technology for Supporting Collaborative Learning. International Journal of Artificial Intelligence in Education, 15 (4), 261-290. Stasser, G. (1999). The uncertain role of unshared information in collective choice. In L. Thompson, J. Levine, & D. Messick (Eds.) Shared Knowledge in Organizations (pp. 4969). Hillsdale, NJ: Erlbaum. Teasley, S. & Roschelle, J. (1993). Constructing a joint problem space. In S. Lajoie & S. Derry (Eds.), Computers as cognitive tools (pp. 229-257). Hillsdale, NJ: Lawrence Erlbaum. Viterbi, A. (1967). Error bounds for convolutional codes and an asymptotically optimal decoding algorithm. IEEE Transactions on Information Theory, 13, 260-269. Webb, N. (1992). Testing a theoretical model of student interaction and learning in small groups. In R. Hertz-Lazarowitz & N. Miller (Eds.), Interaction in Cooperative Groups: The Theoretical Anatomy of Group Learning (pp. 102-119). New York: Cambridge University Press. Winquist, J. R. & Larson, J. R. (1998). Information pooling: When it impacts group decision making. Journal of Personality and Social Psychology, 74(2), 371-377.

Chapter 6 AN APPROACH FOR COACHING COLLABORATION BASED ON DIFFERENCE RECOGNITION AND PARTICIPATION TRACKING

María de los Ángeles Constantino-González, Daniel D. Suthers ITESM Campus Laguna, Torreón, Coah. 27200, MÉXICO; University of Hawai`i at Manoa, Honolulu, HI 96822, USA Abstract:

1.

This chapter describes a new approach to coaching collaboration in a synchronous computer mediated learning context. Prior work on supporting collaboration has relied largely on comparing student discourse to models of collaborative discourse. Comparison of student work to expert solutions is prevalent in individual coaching paradigms. Although these approaches are valuable, our approach evaluates the potential contribution of tracking student participation during group problem solving and comparing students' individual and group solutions. Our theoretical motivation is that conflicts between individual and group solutions constitute learning opportunities, provided that students recognize and address these conflicts. The coach encourages such negotiation when differences are detected, and also encourages participation in other ways. Our evaluation relied primarily on expert judgement and secondarily on student reactions to the coach. Results show that the quality of the generated advice was good; however, other knowledge sources should be consulted to improve coverage of advice to a broader range of situations and advice types. This coaching approach could be applied in those learning tasks oriented towards the solution of a problem and in which structured representations of problem solutions exist.

INTRODUCTION

Today’s organizations require individuals who can work together to solve complex problems and share and negotiate their own knowledge and experiences with others (Senge, 1994). Therefore, higher education should

88

M. Constantino-González and D.D. Suthers

provide opportunities to learn and practice collaborative skills, such as communication and conflict management. Opportunities to practice collaborative skills are especially needed in computer mediated distance learning environments, in which individuals who are working or geographically isolated seek higher education through the Internet and the World Wide Web (WWW). These learners may be isolated and have few opportunities to collaborate. Although they sometimes work in groups, there is little evaluation of the collaboration process and the students’ collaborative skills. Abrami and Bures (1996) state “...with social and intellectual isolation, students may fail to develop and refine those cognitive an interpersonal skills increasingly necessary for business and professional careers.” Many computer-mediated collaborative learning environments have been developed (Soller, Martinez, Jermann & Muehlenbrock, 2005; Dimitracopoulou, & Petrou, 2005). We believe that the design of these kinds of environments should consider theoretical foundations of collaborative learning and results from classroom-based collaborative learning studies. These studies have shown that properly designed collaborative learning techniques help students to improve their achievement and develop their critical thinking and cooperative behavior (Gokhale, 1995; Slavin, 1995; Johnson & Johnson, 1994). Nevertheless, it is known that interaction by itself does not ensure effective collaboration (Brown & Palincsar, 1989). Facilitators should monitor students’ teamwork in order to guide participants in the application of collaborative skills. However, it is difficult for a human facilitator to provide guidance when many teams have to be monitored and students are located in remote places. An intelligent software system could be helpful to overcome these problems. Some computer-mediated environments have integrated intelligent techniques in some way to encourage participation and facilitate group discussion. Most of them, such as C-CHENE (Baker & Lund, 1996), Group Leader Tutor (McManus & Aiken, 1995), iDCLE’s Expert System Coordinator (Okamoto, Inaba & Hasaba., 1995), and DEGREE (Barros & Verdejo, 2000) use restricted menu-driven or sentence-opener interfaces in order to understand students’ interaction, and give guidance based on an ideal model of dialogue or interaction. Similarly, Soller and Lesgold’s system, described in the previous chapter, analyzes coded interaction based on conversational acts and uses machine learning to generate a model of knowledge sharing between peers. Dialogue-based support provides several advantages, such as potential applicability to any subject matter area, automated interpretation of students’ interactions, and restriction of discussion moves and learning interactions to those believed to be productive for learning. However, these systems present some disadvantages, such as

An Approach for Coaching Collaboration

89

restricting the type of communicative acts, slowing the communication process, and misinterpreting students’ dialogue when students use the interface buttons incorrectly. Therefore, it is worthwhile to investigate other ways to support collaboration that may complement previous research in this area. One example is Muehlenbrock’s system (Muehlenbrock & Hoppe, 1999), which provides automated support to monitor group interaction within shared workspaces based on action-based collaboration analysis. Another example is GRACILE (Ayala & Yano, 1996) which gives help based on Vygotsky’s zone of proximal development concept. This chapter describes the design and implementation of a computer coach that supports collaboration. Our theoretical motivation is based on pedagogical theories that explain how social interaction mediates learning. According to the Socio-Cognitive Conflict Theory (Doise & Mugny, 1984), students learn from disagreements when they identify and resolve conflicts in their viewpoints, present alternatives, and request and give explanations. Cognitive Dissonance Theory (Festinger, 1957) states that the existence of disagreement among members of a group produces cognitive dissonance in the individual, who experiences pressure to reduce this dissonance, leading the individual to a process of social communication and revision of his position. The value of the disagreement depends less on the correctness of the opposing position than on the attention, thought processes and learning activities it induces. A similar approach based on differences is taken by Nakakoji, Ohira, Takashima and Yamamoto (this volume) in the EVIDII system in which they provide synchronous collaborating learners with visualization that enables learners to uncover the differences between their viewpoints and discuss these differences. When “breakdowns” are experienced, learners may engage in critical examination and negotiation of their concepts. This gives learners an opportunity to learn about the domain as well as about each other. COLER uses both methods discussed by Nakakoji et al., software coaching that critiques individual solutions, and visualization in a shared workspace where learners can detect conflicts between their solutions. Our work seeks to provide automated facilitation of effective collaborative learning interactions, particularly with respect to the recognition and resolution of conflicts between students’ problem solutions, with minimal reliance on restricted communication devices such as sentence openers. In this chapter, we evaluate the feasibility of generating advice based primarily on comparing students’ individual and group solutions and on tracking student participation (contributions to the group diagram). Unlike previous work in this area, which has focused on supporting interaction by analyzing collaborative dialogue or tracking shared working

90

M. Constantino-González and D.D. Suthers

spaces, we focus on encouraging learning interactions by detecting conflicting situations based on a comparison of individual and group work and tracking participation. Previous studies have also used automated coaches to give advice when a student’s solution differs from an expert’s solution (Burton & Brown, 1982; Corbett & Koedinger, 1997). In contrast, our work evaluates the possibility of giving advice without comparing student work to an expert solution. This coaching approach could be applied in those learning tasks oriented towards the solution of a problem and in which structured representations of problem solutions exist. The computer coach has been implemented as part of COLER (Constantino-Gonzalez & Suthers, 2000), a web-based computer-mediated collaborative learning environment that was developed to enable students construct together Entity-Relationship (ER) diagrams as solutions to database modeling problems. Students begin by constructing their ER diagrams alone. Later they work in small groups to agree upon a group solution. The software coach is not an ER modeling tutor, such as KERMIT (Suraweera & Mitrovic, 2004), but a collaboration advisor. This coach tries to lead learners into particular kinds of interactions expected to lead to learning by identifying differences between group and individual solutions based on a minimal understanding of the ER and application domains. Entity-Relationship modeling, a critical phase in the development of information systems, was selected as an appropriate task for this research due to several reasons: (1) Database design is a collaborative activity in the real world: designers and database users collaborate to produce a conceptual schema that meets the information needs of an organization (Gordon & Hall, 1998; Batini, Ceri & Navathe, 1992; Ram & Ramesh, 1998). (2) Different solutions are possible in this task due to different assumptions or misconceptions. Therefore, students may have genuine differences to discuss. (3) Research in the instruction of database design has found that ER Modeling is a complex task for novices (Batini, Ceri & Navathe, 1992; Gordon & Hall, 1998). A good proportion of novice errors are due to students’ acceptance of the initial solution without considering alternatives (Batra & Antony, 1994). Besides the automated coaching facility, COLER demonstrates several integrated design elements that facilitate remote collaboration and promote students’ participation and discussion of discrepancies (ConstantinoGonzalez & Suthers, 2003). According to Suthers (2005), the shared visual representations that are modifiable by participants (e.g., the diagrams, chat, pencil and opinion buttons) become resources for conversation in which constructs their knowledge. COLER can be used in a distant or local context as part of an assigned task of a course. Petrou and Dimitracopoulou (2003) state that local

An Approach for Coaching Collaboration

91

application of synchronous computer mediated problem solving is possible and that it had positive effects on teachers’ strategies, providing teachers with tools to analyse students’ activity (both on content and collaboration). The remainder of this chapter describes this environment, the computer coach, and an evaluation of its performance and of the knowledge role in coaching collaboration.

2.

COLER

COLER (COllaborative Learning environment for Entity Relationship modeling) is a Web-based environment in which students can solve database-modeling problems while working synchronously in small groups. It provides student and professor interfaces.

2.1 COLER’s Student Interface COLER provides four different modes of operation according to the type of user (student or professor) and the selected type of session (individual or group). COLER is designed for sessions in which students first solve problems individually and then join small groups to develop group solutions. The initial problem solving helps ensure individual participation and provides differences (conceptual conflicts) between students’ solutions that form the basis for discussion. The individual session or the private workspace in the collaborative session also enables students to try solutions they are uncertain of without feeling they are being watched. When a student is ready to work in group, she enters into the collaborative session. COLER’s student collaborative interface is shown in Figure 1. The problem description window (upper center) presents an entity-relationship modeling problem. Students construct their individual solutions in the private workspace (upper right). They use the shared workspace (lower center) to collaboratively construct ER diagrams while communicating largely via the chat window (lower right). Students can use a HELP button (upper left) to review ER Modeling concepts. A team panel (middle left) shows the teammates already connected. Only one student, the one who has the pencil, can update the shared workspace at a given time. The floor control panel (bottom left) provides two buttons to control this workspace: ASK/TAKE PENCIL and LEAVE PENCIL. Additionally, this panel shows the name of the student who has the control of this area and the students waiting for a turn.

92

M. Constantino-González and D.D. Suthers Public Transport Problem Monterrey city has acquired all bus lines to implement a very efficient bus service, managed by the government. You have been assigned to build a data model to support the implementation of this system according to the following requirements:

has is offered CONTROL-POINT

DRIVER Valeria ROUTE

SCHEDULE

ROUTE

is assigned

DEPART-DATE

has SCHEDULE Valeria Ruben

Coach: Welcome to this collaborative session Valeria: What do you think of adding Route+Schedule relationship?

Figure 1. COLER Collaborative Student Interface.

An opinion panel (middle right) shows teammates’ opinions on a current issue. This area contains three buttons: OK to indicate total agreement, NOT to show total or partial disagreement, and ? to indicate uncertainty. When a button is selected, students have the option of annotating their selection with a justification. Opinion button selections are displayed in the chat area (along with any optional justifications) in order to correlate these opinionexpressing actions with the chronology of the chat discourse. Opinion button selections are displayed in the opinion panel to provide students with a persistent summary of their teammates’ current opinions. A personal coach (upper left) gives advice in the chat area based on the group dynamics: students’ participation and group diagram construction. Although several suggestions may be computed at a certain time, only one is shown in the chat area. The others may be given on demand by pressing the SUGGESTIONS button, which is disabled if the coach does not have any advice to offer. When all team members are ready to work as a group they can begin to place components of their solutions in the workspace. This may be done either with COPY/PASTE from private workspaces or by making new structures in the shared workspace. After each change to the workspace, the changed object is highlighted in yellow. Then, students are required to express their opinions using the OK/NOT/? buttons before making subsequent use of the shared workspace. The group phase of COLER is an “Intended point of Cooperation” (IPoC) in Wessner and Pfister’s (this volume) terms: it is logically and didactically integrated into a course

An Approach for Coaching Collaboration

93

(Databases) and located at a specific position or learning unit of the course (Conceptual Database design). COLER’s IpoC is of a different type than those presented by Wessner and Pfister. The task is not merely to discuss but to come to a joint conception of the problem domain. A joint conception is essential to the problem domain: database modelling requires a schema that serves all stakeholders’ needs. COLER is based on the open architecture for collaborative learning systems designed by Suthers and Jones (1997) and originally used for the implementation of the Belvedere software for collaborative critical inquiry (Suthers, Toth & Weiner, 1997). COLER was developed using an iterative design technique based on previous experience in teaching ER Modeling and a review of the Computer Supported Cooperative Work (CSCW) and Computer Supported Collaborative Learning (CSCL) literature, and emphasizing the cognitive conflict-based social learning theories. Subsequently, several empirical studies influenced COLER's design. Details of architecture and formative design are given in Constantino-González and Suthers (2000) and Constantino-González (2000).

2.2 COLER’s Professor Interface COLER’s professor interface offers tools to define the glossary that students will consider while creating ER diagrams for a given problem, as well as to weight typical ER differences between diagrams to indicate which ones are worth discussing. The glossary usually includes the names of entities and attributes mentioned in the database problem students have to solve. It may contain not only correct names, but also names that might correspond to students’ mistakes. The types of ER differences were identified through a review of database literature and with the guidance of a database expert. A weight is pre-assigned to each difference depending on its impact on solution quality, but can be modified by instructors. Instructors can also participate during the collaborative session. The instructor is able to observe one student’s individual diagram at a time, follow the chat verbal discussion and the group interaction, observe coach messages and send individual or group messages to facilitate collaboration.

3.

COLER’S COACH

COLER’s personal coach is a pedagogical agent that encourages students to discuss and participate during collaborative problem solving. Each student is assigned a personal coach. The student that the personal coach monitors is

94

M. Constantino-González and D.D. Suthers

called MyCoachedStudent (MCS). COLER constantly observes actions of this student in the learning environment. It also observes teammates’ participation in the shared workspace and in chat discussions (no natural language interpretation is attempted), and teammates’ selections of opinion buttons. COLER applies event-driven reasoning to this information from the environment, applying knowledge of how to detect learning opportunities and to coach collaboration to decide whether or not to give advice and what advice to give. The coach consists of several modules that are assigned specific functions. A description of the coach’s internal architecture, the knowledge it requires and the reasoning it follows is given below.

3.1 Personal Coach’s Architecture The coach consists of several modules that interact and cooperate to decide when and what advice to give based on the current situation (Figure 2). Input (perception) Communications Module

Output (actions)

Internal model of the environment

Goals

Diagram Analyzer

Collaboration Supervisor Advice Generator

Advice Selector

Differences Recognizer Participation Monitor

Blackboard

Figure 2. Coach Architecture.

The Communications Module is in charge of getting real time information from the environment and communicating a coach’s advice when it is required. The executive component, the Collaboration Supervisor, maintains an internal model of the environment. It operates in two phases: Advice Generation and Advice Selection. The Advice Generator computes the set of appropriate advice for a given situation, while the Advice Selector chooses the most appropriate advice from this set based on control strategies. These processes are described in detail below. The Collaboration Supervisor draws upon three knowledge sources: a Diagram Analyzer, Differences Recognizer and Participation Monitor. The Differences Recognizer detects opportunities for students to collaborate by finding semantically significant

An Approach for Coaching Collaboration

95

differences between individual and group ER diagrams. The Differences Recognizer can either find differences specifically related to the currently added object, or find all “extra work” that the student can contribute to the group. The Participation Monitor attends to the activity in the group diagram. If nobody has worked in the group diagram for a period of time, it reports this event. It also monitors whether each student is participating too much or too little. The Diagram Analyzer is a simple module that identifies participation opportunities based on the detection of problems in the quality of the ER group diagram. It uses syntactic and semantic information. Each one of COLER’s reasoning modules contributes distinct expertise, and is implemented as a Java thread. Coach-applets communication and inter-applet communication are implemented in two different forms: (1) Using static variables stored in the shared working memory and (2) direct communication between applet instances. The reasoning modules have different temporal extents and communicate in different ways. The types of messages they use in the communication are presented in Figure 3. Diagram Analyzer

create(ANALYZE_DIAGRAM)

Differences Recognizer

startParticipationMonitor setTimerControlGroupArea

create(CURRENT-OBJECT), create(ADDITIONAL-WORK)

o ibuti ontr

setDifferencesList(list) setAdditionalIndividual Work(list)

setHaveWorked(true) setDifferencesList(null) setAdditionalIndividualWork(null) setGroupDiagramOpportunities(null)

Blackboard

response

Collaboration Supervisor

setGroupDiagram Opportunities(list)

c

ns()

Participation Monitor

eC utiv s ec e Con espons any r M k he c

getHaveWorked() getDifferencesList() getAdditionalIndividualWork() getGroupDiagramOpportunities() getParticipation() isPartMonitorFinished()

setHaveWorked(false) setParticipation(value) setPartMonitorFinished(boolean)

Figure 3. Communication between COLER coach modules.

The Collaboration Supervisor is active during the complete group session. It starts the Participation Monitor (e.g., startParticipationMonitor()) when an “InitGroupSession” message is received from the Team/FloorControl Applet, and asks it to initialize some variables, such as the coached student, team members, individual and group diagrams, and timer for group work. The Participation Monitor thread stays alive until the session ends, so it is able to respond to specific requests from the Collaboration Supervisor at a given time, such as checking MCS’s consecutive contributions (e.g., checkManyConsecutiveContributions()) or setting

96

M. Constantino-González and D.D. Suthers

the timer for the control of the group area (e.g., setTimerControlGroupArea() ). In contrast, the Differences Recognizer and the Diagram Analyzer are created only when the Collaboration Supervisor wants to detect specific learning opportunities for discussion or participation (e.g., createDiffRecognizer (currentObject), createDiffRecognizer(additional Work), createDiagramAnalyzer (analyzeDiagram) ). When these modules finish their work, they store their results in a blackboard (e.g., setGroupDiagramOpportunities(list), setDifferencesList(list), setAdditional IndividualWork(list)). A blackboard is a common work area to exchange information and data. This data can be used later by the Collaboration Supervisor (e.g., getGroupDiagramOpportunities(), getDifferencesList(), getAdditionalIndividualWork()). Once the Collaboration Supervisor reads this data, these modules are disposed. The Participation Monitor also communicates its results to the Collaboration Supervisor via the blackboard. This blackboard enables the Collaboration Supervisor to check for the occurrence of a specific situation at a given time and to reason about it when such a situation occurs.

3.2 Personal Coach’s Knowledge COLER contains knowledge to recognize learning opportunities and to give collaborative advice based on a set of control strategies. Learning opportunities are recognized by evaluating a number of syntactic dissimilarities between individual and group ER diagrams and by tracking participation in the group workspace. COLER requires minimal domain and problem-specific knowledge for the detection of diagram differences. The domain knowledge that we deemed important includes heuristic knowledge about significant differences in ER diagrams and the procedures for identifying these differences. Significant differences were defined based on four different information sources: (a) domain expert suggestions obtained by personal meetings with our colleague, Dr. Icaza of ITESM; (b) the first author's previous experience in teaching data modeling; (c) analysis of 13 students’ solutions for a specific ER problem provided by the domain expert; and (d) common errors in ER modeling reported in the literature (Batra & Antony, 1994; Shanks, 1996). A weight was assigned to each one of these differences depending on its impact (e.g., missing entity, weight = 0.7). This weight is used to decide when to give advice. We also defined some common errors in the quality of ER diagrams. The main idea to include them was to see how they could be used to encourage participation. Problem-specific knowledge consists of a problem-specific glossary of terms. The glossary is used to match diagrams and identify differences, compensating for the coach’s lack of natural

An Approach for Coaching Collaboration

97

language understanding capability. The glossary includes the names of entities and attributes mentioned in the database problem, as well as some names that might correspond to students’ mistakes. The singular form of the name is used to match entities. Relationships instead are matched using dynamically generated internal names constructed from the associated entities (e.g., Employee-Project). Table 1. Coach’s Advice Types. Category Discussion

Participation

Feedback Self-Reflection ER Modeling

Welcome Goodbye

Abbreviation ED AE AJ GE GJ EU AA RA GC SC CT GP LO LP IP LM LC AF GF CD ER

RW IW GW IG GG

Description Express Disagreement Ask for Explanation Ask for Justification Give Explanation Give Justification Express Uncertainty Analyzing Alternatives Reflect with teammates about... General Contribution Specific Contribution Continue working on Task Explain, in general, the importance of participation Listen to Others Let Others Participate Invite others to Participate Listen to Others, Mandatory Ask a teammate to let you contribute Ask for Feedback Give Feedback Check Own Discrepancies Entity-Relationship Modeling: Connect a disconnected Entity, draw a relationship, add an entity or attribute, define a key. Review Work Completeness Individual Welcome Group Welcome Individual Goodbye Group Goodbye

COLER coaches collaboration by generating a set of suggestions and selecting the advice to give according to some control strategies. COLER’s advice is expressed as suggestions or questions intended to encourage students to discuss and participate. They are not imperative, so students should feel free to follow the advice or discard it when they believe it to be inappropriate. Advice types and categories were defined based on the

98

M. Constantino-González and D.D. Suthers

collaborative learning literature and Wizard of Oz formative studies in which the human expert coached through the chat interface: see ConstantinoGonzález and Suthers (2000) or Constantino-González (2000). Seven advice categories were defined in the present version of COLER (Table 1). The first two categories, Discussion (in chat) and Participation (in the group workspace), are the main categories related to coaching collaboration. Feedback messages are related to students’ pressing of COLER opinion buttons. The ER Modeling category includes suggestions related to some common errors in the domain. The Self-Reflection category consists of suggestions that individuals think about a problem or situation. Besides using advice from these categories, COLER can use messages for welcoming and saying goodbye. Several advice types were defined and classified according to each of these categories. For each advice type, several advice templates were defined using different wording to provide linguistic variety. They can be put in context by binding variables to specific values taken from the current situation, such as the student’s name, the object type (e.g., entity, relationship), the object’s name and the problem type. Control strategies were specified to define COLER’s general behavior concerning when and what advice to give. These strategies are computed based on a set of parameters: Category Preferences, Collaborative Session Phases, Discussion Intensity, Participation Balance, Time on Task, and Waiting for Feedback. A description of these parameters is given below. Category Preferences are used to select between alternate advice categories such as promoting discussion of differences, balance in participation, use of feedback, self-reflection and ER modeling achievement. Default preferences are assigned to the advice categories when the session starts, so the coach can process them in accordance to their weighting. These preferences can change during the group session according to the group dynamics. Collaborative Session Phases: During the collaborative session, students have a time limit to work and learn together and to generate a group solution. According to the progress in the group diagram (number of objects) and the elapsed time, the group session was divided into eight phases: Init, Waiting, Ready, Started, Middle, Verification, TimeFinishing and End. Advice patterns with different semantics were defined for each phase, so it is possible to give suggestions according to the current phase. These phases and how it is possible to move from one phase to another are described in Constantino-González (2000). The evolution for the Middle, Verification, Time-Finishing and End sessions depends on five parameters that should be provided by the professor. Two are time-related: the time in seconds the group session will last (TimeLimit) and the time in seconds assigned for

An Approach for Coaching Collaboration

99

reviewing almost at the end of the session (ReviewingPeriod). The other three parameters are associated with the group diagram: the expected number of objects of the problem solution (TotalNodesApproximate) and two percentages to indicate the progress in the process of diagram construction (MiddlePercentage, VerificationPercentage). Discussion Intensity: Advice is not necessarily given every time a difference is found. COLER uses three parameters to define the desired discussion intensity and then decide when to interrupt students for discussion: ThresholdImportantDifference, ThresholdHighTotalWeight and ThresholdMediumTotalWeight. ThresholdImportantDifference indicates the value that should be considered to decide whether a difference is important when a single difference exists. Its value ranges from 0 to 1. The last two parameters are applied when multiple differences occur. ThresholdHighTotalWeight indicates the value of the sum of several differences that should be considered as high. ThresholdMediumTotalWeight indicates the value of the sum of several differences that could be considered as of medium importance. These values should be greater than zero. If they are small, discussion is encouraged more; otherwise the coach remains mostly silent. The values defined for these parameters should be in concordance with the weights defined for each type of difference in the model analyzed. Participation Balance: COLER uses three parameters to monitor group dynamics and to enable the professor to define the desired balance in participation: o MaximumStandardDeviation, o MaximumConsecutiveContributions, and o MinimumListenAdvice. MaximumStandardDeviation (MSD) is used to determine the desired level of participation of each student compared with his/her teammates. If the value of this parameter is high (e.g., MSD > 1.4), the coach will encourage students to participate only when a significant difference exist in their participation levels. On the other hand, if this value is too small (MSD < 0.5), the coach will interrupt students almost after every action they do. MaximumConsecutiveContributions indicates the maximum number of continued contributions that MyCoachedStudent can do before COLER suggests that he/she let others participate. MinimumListenAdvice indicates the minimum number of “listen or take note” advices (e.g., Listen to Others, Let others Participate) that should be used to encourage MyCoachedStudent to let others participate before taking control of the group area from him. Soller & Lesgold’s (this volume) method of applying hidden Markov model analysis could also be applied to identify and therefore coach the roles students are playing in the participation.

100

M. Constantino-González and D.D. Suthers

Time on Task: One parameter was defined to monitor students’ time on task: TimeoutNoAction. This parameter refers to the maximum period of inactivity in the shared window that COLER considers before suggesting continue working on task. Every time an action is performed in the group diagram (e.g., add, delete, change object), a timeout is set to verify that students are not just chatting or discussing for a long time, but also working on the construction of the group diagram. This parameter should be defined according to the total time assigned to the group session. If this value is too small, the coach will constantly pressure students to work in the group area, with almost no time to discuss anything. If this value is too big, the student might not realize how the time is going and spend a lot of time chatting without any alert message from COLER. Waiting for Feedback: Two parameters were defined to enable COLER say something related to the student use of the opinion buttons: TimeoutTeammateAction and TimeoutMyStudentAction (time in seconds). A “Give Feedback” suggestion is considered when MyCoachedStudent has not pressed any opinion button (“OK,” “not OK,” or “unsure”) after a teammate has performed an action in the shared area (add, delete, update) and the TimeoutTeammateAction has passed. An “Ask For Feedback” suggestion is considered when MyCoachedStudent has not received feedback from his/her teammates and TimeoutMyStudentAction has passed. The control strategies defined above are used by the coach during the event-driven reasoning. This reasoning is described in the following section.

3.3 Coach’s Reasoning COLER uses event-driven inference to generate advice. Three main types of events are attended to: (a) Time-triggered events, such as inactivity in the group diagram (b) Activity in group and individual diagram events, for example, the addition, change or removal of an object, and (c) Voting events, for instance, the receiving and giving of feedback. The Collaboration Supervisor then analyzes the situation and takes an action if it is required. Each event type has an AND/OR tree that generates advice for events of that type, as illustrated in Figure 4.

An Approach for Coaching Collaboration

101

Event Identify Event

EventType Event -driven Reasoning (And / Or Tree)

Advice Types GC, GP SC

ER

RA

Figure 4. Advice Generation.

Every branch of the tree represents a possible set of suggested advice. Several suggestions might be generated for any given event because several leaves may be reached at once via the “and” arc of the tree. Also, many of the leaves of each tree generate multiple advices, and trees for different events may be invoked at the same time. The AND/OR decision tree describing the coach’s reasoning for the “Node added in group diagram” event has two main “or” branches, depending on who performed the action. If a teammate added the object (left hand branch), processes to evaluate (1) participation, (2) feedback, and (3) discussion are executed (branches connected by an “and” arc). If MyCoachedStudent added the object (right hand branch), processes to evaluate (4) discrepancy checking, (5) received feedback and (6) participation are executed. Chat information is considered in different situations to decide the kind of advice to give. Chat information includes who has chatted and the chat messages’ length. See Constantino-González, Suthers and Escamilla (2003) for further details of the advice generation process. Once advice has been generated a selection process chooses the advice to give. This process is illustrated in the following example.

3.4 Example Scenario An example of the coach's operation is presented below. Figure 5 shows the diagram constructed by the team formed by David, George and Frank. Figure 6 illustrates George’s diagram.

102

M. Constantino-González and D.D. Suthers

David

David CENTER id researcher

David controls N

1 1

David contorls N

George ITESM building

David

RESEARCHER SSN name tel-home tel-office

PROJECT id description responsible

N Frank participate 1

Figure 5. Group Diagram.

REPORT number topic abstract title N

CENTER id researcher 1

generates

1 manages 1

1 PROJECT id responsible description

controls

N N

participates M

N

has

1

ITESM building

1 controls N

RESEARCHER SSN name tel-home tel-casa

Figure 6. George’s Diagram.

Frank, George’s teammate, has just added the PROJECT+RESEARCHER relationship (participates) to the group diagram. Then, George pressed OK button to indicate his agreement. George’s Coach’s reasoning is based on the AND/OR decision tree presented mentioned earlier. It considers the results obtained by the Participation Monitor and the Differences Recognizer. The Participation Monitor concludes that George, who has one contribution, has not participated enough. The Difference Recognizer compares George’s and the group’s diagrams, finding that George defined PROJECT+RESEARCHER relationship with cardinality N to M instead of 1 to N, and that he labeled the relationship with participates instead of participate. Since the action was performed by a teammate, the left hand side of the AND/OR decision tree is analyzed. Then, the appropriate branches of this side are analyzed and the following advice types (Table 1) are generated: General Contribution (GC) General Participation (GP) Specific Contribution (SC): SC(CENTER+RESEARCHER:manages), SC(CENTER+ITESM:has), SC(REPORT) Entity-Relationship Modeling (ER): ER(ITESM disconnected), ER(CENTER,key). One of the SC types is selected according to a contribution selection algorithm. Objects that are connected to entities that are also in the group diagram are preferred (e.g., Relationship, Entity connected to current node, entity connected to any other entity group object). Then the ones with a name or key included in the group diagram are favored. This relates MyCoachedStudent’s work to others’ work. If there is no such object, then COLER prefers objects that are connected to other objects in MyCoachedStudent’s diagram. This means that MyCoachedStudent has done some thinking about it and has related it to another object. The ones that have more objects connected are preferred. This algorithm returns a

An Approach for Coaching Collaboration

103

vector with the best objects to suggest. If several objects are returned, the object to suggest is selected randomly from this vector. In this example, one is selected randomly between SC(CENTER+RESEARCHER:manages) and SC(CENTER+ITESM:has), which will be the best objects to suggest. SC(Report) has less priority. Considering that the feedback given was OK and there was no extra evidence of disagreement, additional advice types are generated by other branches of the tree: Ask for Explanation (AE) Analyze Alternatives (AA) Express Uncertainty (EU) Reflect About (RA) Advice Selection eliminates the type of the advice most recently given (AA in the present example), and then randomly selects one advice type from each leaf of the AND/OR tree. In our example, the result is as follows: GP; SC(Center+ITESM:has); ER(Center, key) and RA. Examples of linguistic realizations of the advice (advice templates with variables bound) are shown below: GP: George, participation is a learning opportunity. I suggest that you leverage it. Come on, participate! : ) SC: George, you could share your work with your teammates by adding CENTER+ITESM relationship to the diagram ER: George, you could define the key of the CENTER entity. RA: George, PROJECT+RESEARCHER relationship has been just added by Frank. What do you think about it? Is it correct? I suggest that you discuss it with your teammates. Before this list is sorted according to the preferences, the group’s dynamics (participation) is evaluated to adjust the preference priorities. Since there is a problem in participation, the priority of a Participation preference is increased. The following order is produced: SC, RA, ER, GP. Therefore the coach then gives the SC advice shown above. The rest of the advice patterns generated are stored for future use and made available for advice on demand.

4.

EVALUATION

COLER’s coach was evaluated in several different ways. The evaluation procedures reported here address expert judgment of advice quality, students’ reactions and opinions to advice, and contributions of knowledge sources. Detailed evaluation of advice generation and selection algorithms can be found in Constantino, Suthers and Icaza (2001) and Constantino and Suthers (2001).

104

M. Constantino-González and D.D. Suthers

4.1 Summary of Method and Procedure This laboratory evaluation of COLER involved participants who had the right level of domain knowledge for using the system: ITESM undergraduate students taking a database course. Our domain expert, a computer science professor, was also present in two sessions. A pilot session was run to test COLER’s usability and functionality. Then, five sessions were conducted to generate data and scenarios for the different types of evaluations. In each of these sessions, three students were presented with a simple database design problem. They first solved the problem individually, and then convened to construct a group solution. Software instrumentation recorded all of the activities of the students and of COLER's coach. The pilot study and the two sessions in which the Expert was present were used for preliminary evaluation, detecting some problems in COLER's user interface and coach algorithms. The last three sessions, in which the expert was not present, were used to evaluate COLER’s algorithms and the quality of its advice. These sessions involved a total of 72 advice events. Of these, 34 were Participation, 23 Discussion, 6 Self-Reflection, and 9 Feedback advices.

4.2 Expert Judgement Results Documents were generated to describe the chronological sequence of events of the collaborative session in reference to a specific student, and the context (current state of the environment) associated with each event. The expert evaluated each of the three sessions that he did not witness by analyzing the documents generated for each student. These documents provided the expert with precisely the same information that was available to COLER's coach. For example, the documents included the existence of chat contributions but not their contents. The expert first indicated the advice he would give in each situation. He then ranked the advice generated by COLER and indicated whether this advice was “reasonable,” “so-so,” or “not worth saying.” He was not told the actual advice selected by COLER until his judgments were complete. Of the advice actually selected by COLER, the expert judged 71% as “reasonable” and 16% as “so-so.” Advice was judged as “so-so” primarily for reasons of inappropriate wording used in COLER’s advice patterns. Reviewing the wording with the expert could solve this problem. The nonreasonable advice (13%) was attributed to two main problems: changes in the environment that made the advice inapplicable, and spurious mismatches due to spelling errors. Defining the conditions for each specific advice type that should be reviewed before giving the advice could solve the first problem. Spelling errors could be managed by devaluing the importance of

An Approach for Coaching Collaboration

105

differences in relationship’s names for generating “Check discrepancy” advice, or by using a distance metric between the spellings. Turning to coverage, 67% of the advice given by the expert was not given by the coach. Of this advice, 69% would require new advice types and new branches in the AND/OR situation tree, 21% involved situations already defined in the AND/OR tree but requiring new advice types attached to them, and 10% involved advice that COLER could give with minor adjustments to parameters. These differences between the expert and the coach are mainly because of the following reasons: (1) the coach’s knowledge considers not only expert’s knowledge, but also advice types suggested by the collaborative learning literature; (2) the expert, as a human, has a bigger repertoire of advice; (3) the coach’s knowledge is limited and did not consider “social” messages or specific contexts. Results show that a new category of advice, “Social Interaction,” is needed to establish a closer relationship between COLER coach and the student. This category could include different advice types such as thanking the student for listening to advice, and otherwise commenting on student actions. Additionally, some existing advice types need to be extended to mention a specific context, such as suggesting that students reflect on a specific difference or inviting someone in particular to participate. The findings also suggested situations in which a new “Self-Reflection” advice type could be given.

4.3 Students’ Reactions and Opinions An analysis of the chat transcripts and videotapes provided information about the effects of COLER’s advice on students’ behavior. Findings indicated that 40% of the total advice instances were taken by the students; 28% were applicable but ignored; 21% were no longer needed due to changes in the situation; and student response to 11% of the advice could not be determined. Specific evaluation on students’ learning outcomes has not yet been performed. However, it was observed in all the sessions that the final group diagram was better than the individual diagrams. Information on students’ opinions on COLER coach was obtained by administering questionnaires after the collaborative problem-solving phase to the 13 students in all 5 sessions (including those viewed by the expert). Responses indicated that students found several advice types to be useful, while several other types of advice were found sometimes useful and sometimes irrelevant, such as the “Continue Working on Task” and the “Review Work” suggestions. Almost all students believed that advice was given at appropriate times since it occurred during or immediately after the events. However, some students indicated that some advice was given just after the action suggested was performed, and sometimes the advice

106

M. Constantino-González and D.D. Suthers

interrupted chat continuity. Concerning advice frequency, 69% of students think it was appropriate, 23% think it was low and 8% think it was high. Most of the students thought that the presence of a coach during the session helped guide and coordinate the collaborative session and establish the group dynamics required in collaborative learning. Most students said they reaffirmed their ER knowledge and learned about collaboration during the session. Students suggested additional types of messages and indicated that sometimes advice should be given to the whole group instead of just to individuals. Regarding students’ outcomes and the use of COLER we have not yet any results. A pre-test and post-test experiment design should be run to evaluate COLER’s impact on students learning.

4.4 Roles of Knowledge Sources Several type of knowledge sources were considered to generate advice. Some of them were related to domain knowledge, such as the significant differences found between ER diagrams, the problem glossary defined and the common problems found in this type of diagrams. Others were related to strategic session control, such as parameters used to control balance in participation, time on task, discussion intensity, advice category and sort preferences. The last ones are related to the collaborative interface tracking, such as each individual use of chat, pencil and feedback buttons. In order to determine which knowledge sources are more important, we identified the contribution of these knowledge sources to generation of advice, and ranked the knowledge sources by the percentage of “reasonable” advice they helped to generate. The knowledge sources rank as follows: Feedback Tracking and Feedback timeout (49%), Participation Balance (48%), Significant Differences and Glossary (41%), Time on Task (40%), Chat Tracking (37%), Discussion Intensity Parameters (29%) Category and Sort Preferences (22%), Pencil Tracking (14%) and Common Problems in ER diagram (2%). Some knowledge sources were used to generate multiple categories of advice, such as Discussion or Participation, while others were used only in a specific advice category. The generation of reasonable advice (mostly Discussion and Participation) in this study required the conjunction of several types of knowledge (e.g., Significant Differences and Glossary, Participation Balance) and confirmed our hypothesis that information about problem solving activity could be used to generate collaboration advice. We evaluated the role of natural language understanding and expert domain knowledge by having the expert read the students’ chat transcripts at the end of the session and review his suggestions for a second time. (Chat contents were not available during his initial assessments.) He indicated that

An Approach for Coaching Collaboration

107

there wouldn’t be significant changes in what he said as a coach because he and COLER emphasize collaboration and participation, because ER is a complete modeling technique to represent the design, and because the problem type considered in these sessions clearly specified the user requirements. If the problems had been less well specified, natural language understanding might have been necessary in order to track the substantial discussion required to negotiate agreement.

5.

SUMMARY AND FUTURE WORK

This work is part of a research agenda that seeks to characterize the knowledge needed to facilitate collaborative learning processes. The work follows an incremental research strategy by implementing and evaluating a small number of simple knowledge sources to understand their value before incorporating additional or more complex knowledge sources. We focused on how much leverage can be gained by a basic ability to detect semantically interesting differences between two representations of problem solutions, coupled with simple tracking of each individual's quantity of participation and discussion. We showed that reasonable collaboration advice could be generated without the need for expert solutions or discourse understanding. (The addition of these knowledge sources would improve the quality and range of advice generated and selected by the system, at the cost of considerable additional knowledge engineering.) This approach should generalize to all domains in which students construct formal representations of problem solutions that can be compared for significant differences. We encourage others to test the value of the approach in other domains that have this property. The coach has been implemented as a personal assistant in each client. By viewing a given student’s private workspace and the shared workspace, the coach helps to prevent missed opportunities for collaborative learning (Baker & Bielaczyc, 1995). Future work may investigate a single global coach endowed with the ability to inspect all students’ private workspaces as well as the shared workspace. Such a coach would be able to identify conflicts between solutions in private workspaces and encourage the students to share the relevant part of their solutions, thereby creating conflict opportunities for collaborative learning. Little additional knowledge engineering would be required. A greater investment in domain-specific knowledge would enable the coach to compare student solutions to expert solutions in order to (a) guide advice selection (e.g., encourage students to share solutions that are correct), and (b) comment directly on the correctness of solutions in the manner of traditional intelligent tutoring systems. Other

108

M. Constantino-González and D.D. Suthers

extensions can be envisioned that require models of discourse and natural language understanding.

6.

LATER WORK

Besides the evaluation presented previously we have also evaluated the coordination between COLER’s agents. Agent coordination is especially important in computer-mediated learning environments in which personal agents support collaboration between individual students. Advice given by these agents should be complementary in some way. Agent coordination was obtained in COLER based on the local information that each personal coach had available in the shared workspace. We found that some coordination between the advice types given was present, and in fact the coaching agents gave complementary advices (for example, one coach would suggest that aggressive student let others participate around the same time as another coach encouraged its passive student to participate more). There was no communication between the agents: coordination emerged from purposeful reactions of homogeneous agents to a mutually shared situation (Les, Cumming & Finch, 1999). It would be interesting to investigate how much coordination is possible without explicit negotiation between the agents. A new design for the instructor interface has been developed as part of an ITESM master’s degree thesis (Coronado, 2000). It provides tools for reviewing all team’s individual diagrams simultaneously using a “zoom” tool, and to enable the human facilitator make the software coach send a specific message in a particular situation. Results from the evaluation of this new version will be presented in a future publication. A new version of COLER is being generated (Castaños, Morales, Tun & Constantino, 2005). It includes support for Extended Entity-Relationship (EER) Modeling, a modeling technique similar to UML class diagrams. EER modeling adds to the basics concepts of the ER model (entities, relationships and attributes), the concepts of superclass, subclass, specialization, generalization and its constraints (completeness and specialization type). Considering this model, the range of possible differences between students’ individual diagrams is greater than the range in the basic ER model, which generates more opportunities for learning interactions. The change has been applied both in the Coach program that supports collaboration, and in the graphics area, by including the icons for hierarchies: superclasses, subclasses and their restrictions. A similar environment is being developed by Baghaei and Mitrovic (2005), but using constraint-based intelligent tutoring systems to support individual and collaborative learning of UML class diagrams.

An Approach for Coaching Collaboration

109

This new version also considers different ways to try improving collaboration: a) Awareness of elements of students' problem solution. COLER’s interface now includes an alternative representation of individual and group ER diagrams as a list of objects categorized by their type: entities and relationships. These lists are updated each time the student performs an action (add, delete) over an object in his/her individual diagram or in the group diagram, so students can visualize all diagram objects in a list although they cannot see complete diagrams in the individual and shared modeling workspaces. The idea is to see what these lists add to group awareness and interaction and to help students identify differences between their work. For example, they can see the names of entities/relationships that appear in the group diagram but are not included in their individual diagram or vice versa. Students could also select a name object from the list to highlight, update or delete it in the diagram. b) Awareness of other students’ solutions. Enabling a student to see his/her teammates’ solutions, either in a graph or list format, at a specific phase during the construction of the group diagram. c) Awareness of group activity expressed in aggregate terms (parameters). COLER includes a new window to provide students with visual feedback (bar or pie charts) on the collaboration process based on basic collaboration parameters computed during the group session. The information the user may select to view includes number of contributions to the group diagram; number of chat messages or elapsed time with the pencil. The idea is to make students think about their participation and compare it with their teammates’. This graph could also be used to provide teachers with some way to analyze group dynamics and assist students, in a similar way to the instructional assistant agent included in the FLE3 system (Dolonen, Chen & Morch, 2003) or the CAF (Collaborative Activity Function) tool proposed by Fesakis, Petrou and Dimitracopoulou (2004). d) New suggestions from the software coach related to problem solving elements and activity. Some Affective advice has been added, such as Give Recognition (GR): suggesting students to recognize their teammates' work. Other advice addresses Awareness of group activity, such as Processing Group (PG): asking students to reflect on their collaboration process; and Share Understanding (SU): requesting that students think about their teammates' level of understanding/learning of the modeling technique. Other advice focuses on the Problem, such as Assessment messages assuming the coach can check an “expert solution diagram”: Appropriate Solution (AS) and No Appropriate Solution (NS). A qualitative study is being prepared to evaluate this version and examine how students respond to each of the additions above.

110

M. Constantino-González and D.D. Suthers

ACKNOWLEDGMENTS We thank our colleagues at the Learning Research and Development Center of the University of Pittsburgh for their support; in particular, Alan Lesgold for hosting the first author as a visiting scholar and the second as his research associate, Sandy Katz for discussions about collaborative learning, and Kim Harrigal and Dan Jones for their assistance with the implementations. We also thank Jose G. Escamilla of ITESM for his advisorship of the first author, Jose I. Icaza of ITESM for acting as our domain expert, Moraima Campbell of ITESM for her support in user interface design, visiting students Vincent Trifot and Justin Peltier for implementing part of the personal coach and the Professor module, and the Center for Artificial Intelligence of ITESM for providing research facilities. During this research, Constantino-González was funded by ITESM Campus Laguna and by CONACYT, and Suthers was funded by the Presidential Technology Initiative (while at LRDC) and by NSF’s Learning and Intelligent Systems (while at the University of Hawai`i).

REFERENCES Abrami, P. C., & Bures, E. M. (1996). Computer Supported Collaborative Learning and Distance Education, Reviews of lead article. The American Journal of Distance Education, 10(2), 37-42. Ayala, G., & Yano, Y. (1996). Learner models for supporting awareness and collaboration in a CSCL environment. Proceedings of the Third International Conference of Intelligent Tutoring Systems (ITS’96), Montreal, Canada, 158-167. Baghaei, N., & Mitrovic, A. (2005). COLLECT-UML: Supporting Individual and Collaborative Learning of UML class diagrams in a Constraint-Based Intelligent Tutoring System. Proceedings of the 9th International Conference on Knowledge-Based & Intelligent Information & Engineering Systems, Melbourne, Australia, 458-464. Baker, M. J., & Bielaczyc, K. (1995). Missed opportunities for learning in collaborative problem-solving interactions. In J. Greer (Ed.), Proceedings of the AI-ED 95–World Conference on Artificial Intelligence in Education (pp. 210–217), Washington, DC, August 16-19. AACE. Baker, M. J., & Lund, K. (1996). Flexibly structuring the interaction in a CSCL environment. In P. Brna, A. Paiva & J. Self (Eds.), Proceedings of the Euro-AIED Conference (European Conference on Artificial Intelligence and Education) (pp. 401-407), Lisbon, October. Barros, B., & Verdejo, F. M. (2000). Analyzing student interaction processes in order to improve collaboration. The DEGREE approach, International Journal of Artificial Intelligence in Education, 11, 221-241. Batini, C., Ceri, S., & Navathe, S. B. (1992). Conceptual Database Design: An EntityRelationship Approach. Benjamin/Cummings, Redwood City, California. Batra, D., & Antony, S. R. (1994). Novice errors in conceptual database design. European Journal of Information Systems, 3(1), 57-69.

An Approach for Coaching Collaboration

111

Brown, A. L., & Palincsar, A. S. (1989). Guided, cooperative learning and individual knowledge acquisition. In L. Resnick (Ed.), Knowing, Learning and Instruction: Essays in Honor of Robert Glaser. Hillsdale, NJ: Lawrence Erlbaum Associates. Burton, R. R., & Brown, J. S. (1982). An investigation of computer coaching for informal learning activities. In D. Sleeman, & J. S. Brown (Eds.), Intelligent Tutoring Systems (pp. 79-98). London: Academic Press. Castaños, J. M., Morales, L. C., Tun, J. A., & Constantino, M. A. (2005, October). A coached Collaborative Learning Environment for Entity-Relationship Modeling: COLER, Poster Session presented at XXI Simposio Internacional de Computación en la Educación SOMECE 2005, Hermosillo, Sonora, México. Constantino-González, M. A. (2000). A Computer Coach to Support Synchronous ComputerMediated Collaborative Learning. Unpublished doctoral dissertation, Monterrey Institute of Technology (ITESM) Campus Monterrey, Monterrey, N. L., México. Constantino-González, M. A., & Suthers, D. D. (2000). A Coached Collaborative Learning Environment for Entity-Relationship Modeling. In G. Gauthier, C. Frasson, & K. VanLehn (Eds.), Intelligent Tutoring Systems, Proceedings of the 5th International Conference ITS2000 (pp. 324-333). Berlín: Springer-Verlag. Constantino-González, M. A., Suthers, D. D., & Icaza, J. (2001). Designing and Evaluating a Collaboration Coach: Knowledge and Reasoning. In J. D. Moore, C. L. Redfield, & W. L. Johnson (Eds.). Artificial Intelligence in Education: AI-ED in the Wired and Wireless Future, Proceedings of the 10th International Conference on Artificial Intelligence in Education (pp. 176-187). Amsterdam: IOS Press. Constantino-González, M. A., & Suthers, D. D. (2001). Coaching Collaboration by Comparing Solutions and Tracking Participation. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.) European Perspectives on Computer-Supported Collaborative Learning, Proceedings of the First European Conference on CSCL (pp. 173-180). Netherlands: Universiteit Maastricht. Constantino-González, M. A., & Suthers, D. D. (2003). Automated Coaching of Collaboration based on Workspace Analysis: Evaluation and Implications for Future Learning Environments. In Proceedings of the 36th Annual Hawaii International Conference on System Sciences (HICSS-36), January 6-9, Wakoloa, Hawaii (CD-ROM), Institute of Electrical and Electronics Engineers, Inc. (IEEE). Constantino-González, M. A., Suthers, D. D., & Escamilla, J. G. (2003). Coaching Webbased Collaborative Learning based on Problem Solution Differences and Participation. International Journal of Artificial Intelligence in Education, 13(2-4), 263-299. Coronado-López, F. J. (2000). Diseño y desarrollo del módulo del profesor para COLER, Unpublished master’s thesis. Monterrey Institute of Technology (ITESM) Campus Monterrey, Monterrey, N. L. México. Corbett, A. T., Koedinger, K. R., & Anderson, J. R. (1997). Intelligent tutoring systems. In M. G. Helander, T. K. Landauer, & P. Prabhu (Eds.), Handbook of Human-Computer Interaction (2nd ed., pp. 849-874). Amsterdan, The Netherlands: Elsevier Science. Dimitracopoulou, A., & Petrou, A. (2005). Advanced collaborative distance learning systems for young students: Design issues and current trends on new cognitive and metacognitive tools, THEMES in Education International Journal . Dolonen, J. A., Chen, W., & Mǿrch, A. I. Integrating software agents with FLE3. In Proceedings of the International Conference on Computer Support for Collaborative Learning 2003 (CSCL 2003) (pp. 157-161). Dordrecht, The Netherlands: Kluwer Academic Publishers.

112

M. Constantino-González and D.D. Suthers

Fesakis, G., Petrou, A., & Dimitracopoulou, A. (2004). Collaboration activity function: an interaction analysis tool for computer supported collaborative learning activities. Proceedings of the 4th IEEE International Conference on Advanced Learning Technologies (ICALT 2004), 196-200. Gokhale, A. A. (1995). Collaborative Learning Enhances Critical Thinking, Journal of Technology Education, 7(1), 22-30. Gordon, A., & Hall, L. (1998). A Collaborative Learning Environment for Data Modeling. In Proceedings of the Eleventh International Florida Artificial Intelligence Research Symposium Conference (pp. 158-162). Menlo Park, Calif.: AAAI Press. Johnson, D. W., & Johnson, R. T. (1994). Learning Together and Alone, Englewood Cliffs, NJ: Prentice Hall. Les, J., Cumming, G., & Finch, S. (1999). Agent systems for diversity in human learning. In S. P. Lajioe, & M. Vivet (Eds.) Artificial Intelligence in Education: Open Learning Environments: New Computer Technologies to Support Learning, Exploration and Collaboration: Proceedings of the AI-ED 99 Conference (pp. 13-20). Amsterdan: IOS Press. McManus, M. M., & Aiken, R. M. (1995). Monitoring Computer Based Collaborative Problem Solving, Journal of Artificial Intelligence in Education, 6(4), 308-336. Muehlenbrock, M., & Hoppe, H. U. (1999). Computer supported interaction analysis of group problem solving. In C. Hoadley, & J. Roschelle (Eds.) Proceedings of the Conference on Computer Support for Collaborative Learning CSCL-99 (pp. 398-405), Palo Alto, CA. December. Mahwah, NJ: Lawrence Erlbaum Associates. Okamoto, T., Inaba, A., & Hasaba, Y. (1995). The Intelligent Learning Support System on the Distributed Cooperative Environment. In J. Greer (Ed.), Proceedings of Artificial Intelligence in Education AI-ED’95, August, Washington, D.C., 588. Petrou, A., & Dimitracopoulou, A. (2003). Is synchronous computer mediated collaborative problem solving ‘justified’ only when by distance? Teachers’ point of views and interventions with co-located groups during every day class activities. In H.U. Hoppe (Ed.), Proceedings of the Computer Support for Collaborative Learning Conference: Designing for Change in Networked Learning Environments, CSCL 2003, June, Bergen, Norway, 449-455. Ram, S., & Ramesh, V. (1998). Collaborative Conceptual Schema Design: A Process Model and a Prototype System, ACM Transactions on Information Systems, 16(4), 347-371. Senge, P. (1994). The Fifth Discipline: The Art and Practice of the Learning Organization, Currency Doubleday. Slavin, R. E. (1995). Cooperative Learning (2nd ed.). Allyn and Bacon. Soller, A., Martinez, A., Jermann, P., & Muehlenbrock, M. (2005). From Mirroring to Guiding: A Review of State of the Art Technology for Supporting Collaborative Learning. International Journal of Artificial Intelligence in Education, 15, 261-290. Suraweera, P., & Mitrovic, A. (2004). An Intelligent Tutoring System for Entity Relationship Modelling, International Journal of Artificial Intelligence in Education, 14(3-4), 375-417. Suthers, D. D., & Jones, D. (1997). An Architecture for Intelligent Collaborative Educational Systems. In Proceedings of the 8th World Conference on Artificial Intelligence in Education (AIED’97), August, Kobe, 55-62. Suthers, D. D. Toth, E. E., & Weiner, A. (1997). An integrated approach to implementing collaborative inquiry in the classroom. In Proceedings of the 2nd International Conference on Computer Supported Collaborative Learning (CSCL’97), Toronto, 272-279. Suthers, D. (2005). Collaborative Knowledge Construction through Shared Representations. In Proceedings of the 38th Hawai’i International Conference on the System Sciences

An Approach for Coaching Collaboration

113

(HICSS-38), January, Waikoloa, Hawai’i (CD-ROM), Institute of Electrical and Electronics Engineers, Inc. (IEEE).

PART II COLLABORATIVE TOOLS IN EDUCATIONAL PRACTICE

Chapter 7 COLLABORATIVE TOOLS IN EDUCATIONAL PRACTICE Introduction to Part II Amy Soller1, Alan Lesgold2 1

Institute for Defense Analyses, Alexandria VA 22311

2

University of Pittsburgh School of Education, Pittsburgh PA 15260

Research efforts in designing, developing, and evaluating collaborative learning technology pay off when this technology can be integrated into educational practice, creating new or enhanced opportunities for learning. This section discusses a few successful examples of this integration process. The tools and environments described over the next four chapters put the lessons learned from CSCL research into practice in the classroom, while taking care to address the dynamics and constraints of existing classroom processes. We begin with two chapters that discuss technological design considerations for the collaborative learning classroom, followed by three chapters that describe the possibilities afforded by integrating collaborative learning technology into educational practice. The design of technology for the (face-to-face or virtual) classroom should take into consideration both the physical learning environment, and the existing curriculum, social norms, and teachers’ experience. These factors influence students’ and instructors’ perception of the technology and their ability to accept and use the technology productively. Studying educational practice, classroom culture, and even the spatial organization of the educational environment helps us understand what kind of technology might be useful, and how it might be used (and by whom) in the classroom. For example, the first two chapters in Part II of this book describe examples in which students learning in an exploratory science classroom

118

A. Soller and A. Lesgold

benefit from an open, learner-centered design laboratory environment, while students in math and computer classes are motivated by gaming-oriented environments. In developing the NIMIS collaborative learning desktop (chapter 8), Hoppe and his colleagues first analyzed the collaborative learning practices and social norms that were already established in one classroom. The NIMIS environment was then designed to specifically address these classroom needs. For example, the environment facilitated the collaboration within the face-to-face small groups to which the students were already accustomed by enabling the group members to log into virtual learning spaces simultaneously. Personalized virtual companions were added to keep track of each student’s learning activities and provide tailored advice. The integration of this new technology created new learning opportunities in the form of new teacher, student, and computer roles in the classroom. This project was successful, not only because the development team was committed to considering the physical, social, and educational constraints in their design, but also because they were able see these constraints as opportunities to develop new educational practices. The integration of collaborative tools into educational practice does not necessarily mean that the technology is used solely to augment classroom activity. Significant effects may also be achieved by augmenting peripheral educational activities such as homework and lab preparation. In chapter 9, Verdejo, Barros, Read, and Rodriguez-Artacho describe a system that students used from home as they prepared for their in-class laboratory activities. The students logged on to the virtual pre-lab environment, and collaborated with their lab partner (who was working from home as well). The environment was open-ended, encouraging the students to generate, share, and record their ideas. It also provided configurable help and advice to complement the existing laboratory guidance. The system kept track of the questions that the students asked the teachers online before coming to lab, automatically generating a glossary of frequently asked questions and answers that could serve as an evolving reference for all the students. This technology created educational change by enabling a type of lab preparation that did not exist at all before the technology was introduced. Verdejo et al., explain that before the technology was available, students did not prepare at all before coming to the lab; they did not even look at the documentation or guidelines that they were given. As technology becomes an integral part of the classroom activity, it may expand the means by which students learn, for example by introducing new roles for the teachers and students, or enabling new mechanisms for knowledge sharing and creation. These new processes may impact

Collaborative Tools in Educational Practice

119

educational practice both positively and negatively, and therefore should be evaluated and adapted continually to meet the needs of this evolving process. In chapter 10, Kynigos, Dimaraki, and Trouki describe two different scenarios in which the technology impacted the way in which students presented, received, negotiated, and interpreted information, which in turn impacted the researchers’ assessment and development of the technologyenhanced environment. In one scenario, students collaborated at a distance to plan student exchange programs; in another, students worked in groups to build models of bridges, taking account of cultural, historical, structural, and architectural factors. During these activities, the teachers and students learned to play different roles in the classroom. While the teachers learned to play the roles of learning guides and partners, rather than being strictly sources of information, the students (who were accustomed to producing learning products and submitting them for evaluation) learned that in the technology-supported collaborative learning classroom, the process by which the learning products are developed is more important. For example, they learned how to analyze information from different sources, and discern valid information from hearsay or opinion. Kynigos et al., also explain how computer-based communication helped the students maintain a record of their ongoing interaction, raising their awareness of the communication, and facilitating the assimilation of ideas over time. Their system encouraged the students to devise methods for establishing and maintaining communication with peers at a distance, implicitly raising the students’ awareness of their communication. Ogata, Matsuura, and Yano (chapter 11) take this idea one step further by introducing technology that is specifically designed to raise students’ awareness and curiosity about their peers’ knowledge. Their Knowledge Awareness Map provides students with visualizations that show the relationships between their peers, and the knowledge they hold. The software also includes a mediator agent that makes recommendations about which peers might be the most appropriate collaborators for various tasks. Such recommendations might assist teachers in constructing effective collaborative learning groups, or selecting peer facilitators and helpers. Awareness can also mean contextual awareness. The chapters in this section provide compelling evidence that, in designing new collaborative learning tools and integrating them in existing educational practices, the seamlessness of the integration and the transparency of the technology in the existing learning environment may be just as important as the new capabilities the technology has to offer. The introduction of the new technology in the classroom will provide new affordances for learning, giving rise to new learning mechanisms and processes. For example, Kynigos, Dimaraki, and Trouki explain how their

120

A. Soller and A. Lesgold

collaborative distance learning technology afforded students the ability to collaborate with peers located around the world, creating opportunities for learning about different cultures, languages, and perspectives. We see change in educational practice when these new affordances are accommodated by continuous cooperation among technology developers, classroom instructors, professional developers, and technology support staff, and when this process is guided by a comprehensive evaluation plan that supports the evolving educational practice through software re-design and integration.

Chapter 8 SUPPORTING COLLABORATIVE ACTIVITIES IN COMPUTER INTEGRATED CLASSROOMS – THE NIMIS APPROACH

H. Ulrich Hoppe1, Andreas Lingnau1, Frank Tewissen1, Isabel Machado2, Ana Paiva2, Rui Prada2 1

University of Duisburg, Germany IST, Technical University of Lisbon and INESC, Portugal

2

Abstract:

1.

This chapter presents the concept of a collaborative computer integrated classroom (CiC) specially designed to achieve a unique combination of interactive and collaborative software with spatial arrangements, special furniture, and new peripherals including furniture (“roomware”). Although, technologically innovative, the CiC approach respects grown pedagogical traditions and classroom procedures. In-line with the notion of ubiquitous computing it tries to augment the real classroom instead of defining a virtual learning environment. Based on these principles, the European NIMIS project has put into practice a specific classroom environment for early learning with general tools and specific applications supporting literacy-related activities. In addition to the collaborative nature of the classroom scenario as such, specific mechanisms for co-construction in shared workspaces are provided.

INTRODUCTION

The background of this chapter is a European research program called “Experimental School Environments” (ESE). This program was based on the challenge of providing “early learners” (4-8 years) with innovative learning environments using novel interaction techniques and interfaces. It is part of the challenge that these users cannot be expected to know how to read and write and thus need specialized interfaces, especially for document archiving and retrieval. Among the different ESE projects, the specialty of the project NIMIS (Networked Interactive Media in Schools, 1998-2000) was its focus

122

H.U. Hoppe et al.

on conceiving and orchestrating an integrated whole classroom environment and not just one specific interface. The notion of a “computer integrated classroom” or CiC had already been introduced by Hoppe et al. (1993), but only through the NIMIS project it could be brought from the laboratory to school practice. The basic idea is to enrich and support classroom practice through a set of embedded and networked hardware and software tools. Grown classroom procedures shall be facilitated and not be obstructed, new productive processes may be constituted. As part of the NIMIS project, three CiC’s were established in the UK, Portugal and Germany. According to the differing curricular and institutional backgrounds (obligatory public schools in UK and Germany vs. a supplementary private school in Portugal, different grades within the age range of 4-8), the different NIMIS sites focused on different applications. This article concentrates on the Portuguese and the German experiences with their respective applications supporting enacted story telling with virtual characters and “reading through writing”, a phonetic approach to learning how to read and write. Before these two applications are described, basic and shared features of the NIMIS CiC are introduced.

2.

A COLLABORATIVE COMPUTER INTEGRATED CLASSROOM FOR EARLY LEARNING

2.1 Ubiquitous Computing in the Collaborative Classroom The notion of “ubiquitous computing” has been developed in the 1980s at Xerox Parc (Weiser, 1993). Re-conceptualized in today’s terms, it could be characterized as using special purpose networked digital devices (rather than uniform computing equipment) embedded in natural environments, be it at home, at work, or in educational settings. The NIMIS project aims at developing a special version of ubiquitous computing for a “computer integrated” primary school classroom. This is in-line with recent integrative approaches to designing new interfaces and interactive devices, e.g., under the notions of “The Invisible Computer” (Norman, 1998), “Tangible Bits” (Ishii and Ullmer, 1997) or “Roomware” (Streitz et al., 1998). A first version of the CiC as a general concept and first prototypical implementation are described in (Hoppe et al., 1993).

Supporting Collaborative Activities

123

As it is the case with “roomware, the CiC approach lets the traditional computers go more to the background and invents new specialized devices and interfaces driven by appropriate software architectures. Such integrated solutions take aspects of software, hardware and furniture into account when designing interactive media environments. To be successful in terms of usability for and acceptance by the envisaged users, the construction of these environments has to take into account existing social interactions and needs. Evolving from the specific application for early learning, there is a clear general need of explicitly defining the share of responsibilities and roles in the interaction of humans and machines in CiCs. In this concrete design for early learners we follow these principles: – provide uniform access to multiple representations of media and use a variety of information sources; – do not let the technology “get in the way”; – facilitate existing classroom procedures; and – use a modular software architecture which allows e.g., plugging in intelligent modules.

Figure 1. Use of pen-based devices.

Special hardware components used in the CiC comprise a specially designed big interactive screen which is height-adjustable and has a touchsensitive glass surface, as well as an interactive pen-based LC displays integrated with the children’s tables as the primary user input device. The computers as such are invisible (i.e., concentrated in a separate room and not operated directly during a lesson) and integrated in a LAN. Fig. 1 gives an impression of the embedded classroom installation with integrated

124

H.U. Hoppe et al.

hardware components, particularly the flat interactive tablet displays, and the big interactive screen. To make sure that new media technology supports learning and does not redefine well suited pedagogical procedures, we studied existing interactions and curricular activities in the classrooms of today’s primary schools. The CiC instances that exist now in schools in Germany, England and Portugal, have been set up in close cooperation with teachers and test groups of young children. Although new technology should not redefine pedagogy, new roles may evolve from these special environments for the teachers and the children. Teachers act as information managers, and thus have to learn about new ways of accessing information and to judge and select from qualitative new kinds of information. The same is true for the children: Without explicitly mentioning the computer as a topic, the children get used to managing data and information and working with different devices and in different group constellations supported by the technology.

2.2 Classroom and School Environments GGS Kirchstrasse and O Nosso Sonho are two of the three schools collaborating in the NIMIS project having installed the CiC in one of their classrooms. GGS Kirchstrasse in Duisburg is a normal curricular primary school. In general, the children are taught in individual work phases in which the teacher organizes the surrounding learning environment. The structuring of activities (what has to be done when) is done by the children themselves. Most of the time the teachers act as moderators and decide what the line of work will be for the school day. Recently, the school has adopted a new method for teaching reading and writing. The traditional way of letter-byletter or whole-word learning has now been replaced by the method called “Reading through Writing”, a phonetics based approach originally proposed 3 by Reichen (1991). This method is now supported by the T application specially designed for the NIMIS CiC (see Tewissen et al., 2000a). The NIMIS classroom in Duisburg is equipped with 8 computers, situated in a separate room. In the classroom there are six tables in three groups, with built in WACOM tablets, loudspeakers and an optional keyboard and a large interactive screen for finger based input (see Fig. 2).

Supporting Collaborative Activities

125

Figure 2. The NIMIS classroom in Duisburg.

O Nosso Sonho is a school located in the suburbs of Lisbon, which covers a wide variety of learning stages and activities: kindergarten, free time occupation, psychological support and professional training. Further, O Nosso Sonho is not a curricular school, and its pedagogical approach aims at providing the children with the possibility to choose freely the daily activities and this way promoting the acquisition of mature decisions by the children. The pedagogical strategy begins with the spatial organization of the school. So, the learning activities are distributed in different rooms with different goals and aptitudes. For example: (1) in the Dramatic Room, children can do dramatic performances of fairy tales or free theme stories, and (2) in the Intellectual Room is a place for the children to write, play with the computers (games, puzzles, educational software, etc.) and with traditional educational games. The CiC is installed in one corner of the Intellectual Room and this new setting allows the children to continue doing their usual activities as well as having the opportunity to bring the dramatic performances to the computerized environment and to act out their own stories together.

126

3.

H.U. Hoppe et al.

GENERAL TOOLS FOR CLASSROOM MANAGEMENT

3.1 The NIMIS Desktop To provide a platform for networked interactive working and collaboration usable by early learners, a genuine NIMIS Desktop has been developed as a multifunctional tool. It can be used by pupils and teachers cooperatively. The basic functionality is shown in Fig. 3. 3.1.1 Companions and archiving As a child-oriented metaphor for handling and visualizing different objects and media and for providing consistent orientation and guidance, we have introduced a companion as a virtual representative of the child. The child logs in to the computer by calling the companion from a window on the desktop. The companion appears and shows the child’s own data in the form of small preview images (see Fig. 3, in the left). The children’s data view consists of a list of symbolically labeled general purpose and specialized (e.g., incoming mail) folders. Older children may invent their own organization using special folders and drag and drop operations to arrange their data.

Figure 3. Elements of the NIMIS Desktop.

Supporting Collaborative Activities

127

3.1.2 Login procedures and information flow To cope with the special requirements of its user group, NIMIS has designed and implemented new policies for desktop operations which differ from the usual conventions. To support the physical cooperation of children sharing one tablet on a table, the NIMIS desktop allows for two children to log in at the same time on one machine and work together at their desktop. Both have their own companion to which they can hand over documents they have produced. Joint products can be copied using a desktop “copier” and may be archived individually. When a child logs out, the companion disappears and goes to sleep. This is also true, when a child logs in on a different machine: The child is automatically logged out in the former desktop, i.e., at each point in time one child’s companion has one defined location. The companion metaphor together with the previously explained archiving principle is also used to facilitate access to public resources such as the scanner: For example, a child may go to the scanner, put a paper sheet onto it, and, by logging in on the machine next to it, call his or her companion. The companion would now appear on the screen next to the scanner and shows the newly scanned image. Returning to the workplace the child would call the companion again. Accordingly, the companion would appear at the workplace’s interactive display, carrying the scanned image. The metaphor of letting a virtual companion carry and collect the child’s data turned out to be a very natural way of facilitating archiving and document management for first graders.

4.

FORMS OF COLLABORATION IN THE NIMIS CLASSROOMS

4.1 General Features The general design of a NIMIS classroom is focused on supporting the ordinary classroom activities. These are typically multi-threaded in the way that pupils work alone, in pairs or in larger groups on potentially different topics or assignments. A main function of the teacher is to stimulate, coordinate, and, if necessary, control these multiple individual and collaborative activities. In this sense, the teacher can be seen as a manager of classroom activities and information resources. To support this, the NIMIS classroom provides a supervision interface which displays the location of

128

H.U. Hoppe et al.

logged-in children in the classroom. The interface also allows for sending assignments to the children and observing their current state of work. In addition to supporting general classroom information management, NIMIS develops specific applications in the area of early literacy, including the acquisition of initial reading and writing skills as well as aspects of story creation (writing, enacting/playing and watching). Two example applications are described in the following sections. All NIMIS tools and applications can be run in shared workspace or synchronous communication mode, supported by the Java MatchMaker communication mechanism (see section 4). This allows for co-constructive activities in different areas of literacy and narrative expression. However, again, NIMIS is not focused on remote communication and children may also just cooperate sitting side by side on the same table. This kind of collaboration is supported by the specific login and archiving mechanisms described above. Generally speaking, NIMIS conceives collaboration as a ubiquitous activity in the classroom, which is only partially mapped onto technically mediated communication.

4.2 The case of T3 The T3 application (T3 stands for “Today’s Talking Typewriter”) supports the phonetic method of “Reading through Writing” (Reichen, 1991). The method is based on the principle that children first start to write phonemes which they derive from decoding spoken words. In T3, this is done using a “phoneme table”, from which they drag letters into a workspace to form phonetically (not necessarily orthographically) correct words. Immediate feedback is given by a Text-To-Speech (TTS) system so that the children can listen to the word they have composed (see Fig. 4). In the non-computerized classroom practice, the feedback cycle starts with an exact pronunciation of the writing provided by the teacher. Using T³, children can press the “speak” button whenever they wish to hear the complete text or a single word that they were most recently working on. Certain modifications to the TTS system were necessary to comply with the requirements of dealing with non standard writings: While the TTS system in normal working mode would spell orthographically incorrect words letter by letter, our system needs to generate continuous phonetic output also for mis-spelt words which might e.g., consist only of consonants. To achieve this, all input sequences of letters are converted by a Prolog algorithm into a phonetic transcription according to the internal encoding of the TTS system. With this input, the TTS system will read out any letter sequence continuously as desired. This can now replace the teacher’s feedback.

Supporting Collaborative Activities

129

Figure 4. The T3 application.

Using a database with the most likely target words for first graders, the children can also pre-select a word they would like to write by choosing its pictorial representation from a separate “theme page”. This selection enables the children to compare their own writing with the correctly pronounced target word by pressing a corresponding button in the workspace and listening to both, the word they wrote and the target word they selected.

Figure 5. Visualization of individual results.

The T3 application has been enhanced with intelligent support. This support is not confined to individual feedback but also includes support for collaboration following a “peer helper” principle (Hoppe and Plötzner,

130

H.U. Hoppe et al.

1999). Take, e.g., a situation in which one child is working on a word which has been written by other children before: The work of the child will be supervised by an implicit agent. If the agent detects that there were other children who had problems related to a similar word, a selection of these children will appear (see Fig. 5) and the child has the choice to hear the words of the other children, working with the word written by another child, or to initiate a collaboration session for working together on this word. 3 A collaborative session can be initiated with two or more T applications coupled. The children virtually share one and the same workspace so that everyone will see all changes made by the other without loosing local data. In the current implementation, the phoneme table will be split so that one child gets the vowels, the other children the consonants. While working together on a word one child can now act as a “vowels advocate” and supervise the using of vowels in a word (see Fig. 6). Indeed, it can be observed that children tend to leave out vowels rather than consonants.

Figure 6. Collaboration with T3.

4.3 The Case of Teatrix Teatrix is a collaborative virtual environment for story creation which aims at providing effective support for young children (7-8 years old). It supports different stages of development: (1) the basic understanding of narrative, through the dramatization of different situations; (2) social relations, through the interaction by means of their characters; and (3) the ability to take different perspectives across a wide range of situations.

Supporting Collaborative Activities

131

4.3.1 Collaborative story creation Narrative and storytelling start to make part of our lives since early childhood, when children start to hear the first stories through their parents and siblings’ voices. After having explored the world that surrounds them, children start to acquire their first concepts about the objects and events of the world, begin to construct more decentralized plays and start also to include the others in their make-believe world. At this point they also start to use common objects in such a way that they become magical and powerful props in their stories, for example: a stick that becomes a horse (Singer and Singer, 1990). Following Piaget, this evolution of the make-believe activities allows children to perform different roles, gain control of the course of the action and acquire the skills to organize the sequence of a play and most importantly to project these experiences into the cognitive and social requirements of the real world. Based on these evidences we decided to bring the make-believe activities to the computerized environment and to provide the means for children to collaborate in the story creation process and implicitly promoting social interactions among them. 4.3.2 Design of Teatrix Before designing Teatrix as a CiC application, a set of informal experiences was run in the Portuguese school “O Nosso Sonho”. In these informal experiences, we observed children of several ages performing fairy tales in different settings: theatre and puppet scenes. The results of such observations showed that: – during the story performance, younger children (4-6 years old) needed more scaffolding from the teacher than the oldest ones (7-8 years old); – children spent a long period of time to characterize (in order to better “incarnate”) their characters; – the older the children the better they followed their roles in the play, and stayed in character throughout the story progression; – at acting time, older children coordinate between themselves in order to ensure that the story was being conveyed to the audience in the meaningful way; – the teacher usually played the role of the narrator and acted also as a mediator. Based on these results we designed Teatrix (Prada et al., 2000) which follows a theatrical metaphor and targets public school children between 7 and 8 years. Similar decisions and results concerning the children ages were

132

H.U. Hoppe et al.

found in the ESE project Kidstory (Benford et al., 2000). The story creation environment is divided in three steps, strongly related to theatrical performances: 1. The Backstage – this option offers the children the possibility to prepare the scenes, props and characters for each story. 2. On Stage – this mode provides the children with the possibility to initiate a story and to start the acting. Before starting the acting activity, each child has to choose a character to control. In this environment, there is also the possibility to have system-controlled characters After having chosen the characters, children are ready to start their performances which take place in a collaborative 3D world (see Fig. 7). From the story creation process a “film”-like object is created. This “film”-like object offers the children a product they can analyze and even reconstruct in future performances. 3. The Audience or Public - is based on the artefact produced from the story creation process. In this phase, children can be the audience of their own performances and watch their previous performances and also have the possibility to write about those stories. 4.3.3 Collaborative story creation in Teatrix Teatrix can be considered as a system that enables collaboration since it provides the children with the necessary means to act autonomously, but at the same time being aware of the presence of others and having freedom to coordinate efforts in order to achieve a collaborative story. The On Stage option is a collaborative tool that allows several children to work simultaneously on the same story. In this case, virtual reality technology plays an important role because it provides the children with the means to explore the scenes during the story creation (Roussos et al., 1996).

Figure 7. Teatrix: On Stage Option - two children playing with two different characters.

Supporting Collaborative Activities

133

The story evolves whilst the children work together to achieve a common goal: their story. Furthermore, the children get much more from an interaction or experience if in the end they will create a meaningful artifact they can exhibit as a proof of their personal or collaborative work (Papert and Harel, 1991). Each child acts in the story by means of a character, her/his adopted character. Each character in Teatrix has a role to play, for example: a villain, a hero, a magician, etc. The role definition was based on the seminal work done by Propp (1968) on one hundred Russian folktales (see Machado and Paiva, 1999 for more details on the characters definition), which define and establish the functional role of an agent by means of the specification of the actions and roles for it. During the story creation process, each child sees the actions performed not only by her/his character but also by the characters in the same scene. The children can talk to each other in order to coordinate their activities as well as collaborate through their controlled characters. For example: the magician (controlled by one child) may need to give a magic element to the hero (controlled by another child) for him to defeat the villain.

5.

IMPLEMENTATION AND TECHNICAL PLATFORM

The NIMIS desktop, the T3 and Teatrix applications were developed using the Java programming language. The communication and collaboration is technically based on a Windows network and the Java MatchMaker (JMM) software library and server (Tewissen et al., 2000b). Every Desktop is connected via JMM to a central Classroom Service, which supports centralized access to – a database in which the core information about the users is stored (e.g., to which group using the classroom a specific child belongs, or which image is used for the child’s companion, etc.), – central logging of children’s activities within the classroom (e.g., starting an application, writing a word with the T3 application), and – monitoring central control facilities for the teacher (e.g., to show who is logged in where, and what applications are currently in use). T3 and Teatrix use a general Java-based XML format for storage. The audio feedback is provided by an external Text-To-Speech (TTS) system. The T3 application uses SWI-Prolog to internally transform words which are written phonetically, but not in correct orthographical form, into special

134

H.U. Hoppe et al.

phoneme control sequences to prevent the TTS system from spelling every single letter in a word. The Teatrix application, in addition to the standard Java Development Kit, also uses the Java 3D API for the implementation of the 3D environment (world, characters and props) and the Java Expert System Shell (JESS) for the development of agent reasoning modules.

6.

FIRST EXPERIENCES

For the teachers and children in Duisburg, the use of the NIMIS classroom was a seamless transition from the normal classroom to the computerized environment. During one school year, the NIMIS classroom has been used every day for one hour by half of the group for reading through writing. At the same time, the other half stayed in normal classroom and practiced reading through writing in conventional form with a second teacher. After a short period of initial guidance, most of the pupils were able to use the NIMIS environment autonomously, i.e., particularly they were able to write using the letter palette without help of the teacher. This enabled the teachers in the NIMIS classroom to use their time more efficiently to support children with special needs, e.g., children with German as their second language.

Figure 8. Visual log from NIMIS activities.

One important result was that the quantity and quality of writing products was much higher in the NIMIS classroom, i.e., by using the T3 application, than in the normal classroom situation. Especially those children who were scared of writing took an advantage by using the T3 application and its

Supporting Collaborative Activities

135

immediate TTS feedback of the written content. This allowed them to try out the composition of phonemes independently without the direct observation of the teacher (as it happens automatically in the normal classroom situation in which the teacher reads out the writing to the child). In the NIMIS classroom teachers and evaluators are provided with detailed data logs of all the activities. Fig. 8 shows an example of composing a sentence in the T3 workspace which is visualized as an HTML document. This view gives the teacher a comprehensive impression of ongoing activities and enables the monitoring of the children’s achievements. In O Nosso Sonho, the CiC environment with Teatrix has been installed and the first results showed that children were very keen to use it and got very skilful with it. These opinions are expressed by their comments about Teatrix: – It’s funny, instead of doing the drawings to use in the story we can pick them from a list of characters, props and scenes and in the end we build the story – It’s a fantasy of heroes and princesses. It’s entertainment in the computer! – Teatrix is a like a theatre, where we can play together. What I like most is the feeling of being inside the characters. In Teatrix we can do things that all others can watch, and that is very important because by this way everybody can participate in the stories. The Teatrix experience is not meant to replace “real” bodily enactment, but it has an interesting additional potential for reflection and exploration which is taken up by the children in their creative production.

7.

MORE RECENT WORK AND CONCLUSIONS

Meanwhile, the above mentioned findings about the superiority of writing products and learning progress in the NIMIS classroom with T3 have been confirmed by a more thorough empirical study (Lingnau et al., 2003). In several aspects, this study relied on standardized test for reading/writing related abilities and on standardized criteria for the classification of reading/writing products. A prognostic test of the learning group’s prior abilities (Bielefeld screening) which was conducted at the beginning of the school year and observation period yielded an extraordinarily high share of seven out of 24 children (29%) showing a high risk of developing literacy problems. In spite of this problematic finding, literacy levels after 6 and 11 months were within the normal range of 16% (20%) low performers (according to the standardized test “Hamburger Schreibprobe”). Due to the

136

H.U. Hoppe et al.

continuous switching between the traditional and the NIMIS classroom, the quality of writing products could be compared between the two environments for the same group of students using a standardized classification. Here, the NIMIS condition showed more productivity, higher overall quality, and more transitions to new quality levels. Children did not only show a good mastery and flexible use of the NIMIS tools with audio feedback on demand, but also showed a productive and organized behavior in collaborative sessions which were specifically arranged and observed. Of course, the claims associated with the CiC idea are more general than what can be corroborated by these specific findings. The main claim is that networked interactive media and new peripherals provide an added value in making the learning in the classroom more productive and creative as well as in making the products of learning more tangible and re-usable. This can, in a way, also be formulated as a postulate (or even an imperative): If networked interactive media provide new and better ways of intellectual production and expressivity, educational researchers and teachers are obliged to appropriate this potential for the benefit of their students! This postulate was a guiding principle for a later European project (SEED) which aimed at empowering teachers to enhance and enrich their own classroom practice. The basic ideas of the transition from NIMIS to SEED are explained in Hoppe (2004) under the notion of “collaborative mind tools”. From a technological perspective, the design choices of the NIMIS project have to be revised: The most evident point is probably the availability of wireless connectivity and of mobile devices. Tablet PCs provide more or less the same form factor as the Wacom tablets used in NIMIS, but they allow for more flexible use. (It is, however, questionable if the possibility of moving the whole equipment between classrooms is more than a provisional solution.) Also other devices such as PDAs or simple feedback systems for question answering or voting can be integrated. Still in line with the original CiC ideas, a range of possibilities using these new technologies is currently being explored in Taiwan in realistic school settings (Liu et al., 2002). After all, the NIMIS experience has practically demonstrated the viability of new forms of integrative technological support in a whole classroom, yet subordinate to grown pedagogical methods. NIMIS supports and propagates a mix of natural and technologically mediated forms of collaboration and of classroom information management in which the role of teachers is explicitly defined and particularly reflected in the software tools.

Supporting Collaborative Activities

137

ACKNOWLEDGEMENTS Parts of this work refer to the Esprit project No. 29301, “NIMIS”. We thank our NIMIS partners for the good and constructive cooperation, and especially the teachers and pupils of the associated schools for constructive discussions and creative input.

REFERENCES Benford, S., Bederson, B., Akesson, K., Bayon, V., Druin, A., Hansson, P., Hourcade, J. P., Ingram, R., Neale, H., O’Malley, C., Simsarian, K., Stanton, D., Sundblad, Y., Taxén G. (2000). Designing storytelling technologies to encourage collaboration between young children. In Proceedings of ACM CHI 2000, ACM Press, pp. 556- 563. Hoppe, H. U., Baloian, N., Zhao, J. (1993). Computer support for teacher-centered classroom interaction”. In Proceedings of ICCE 1993, Taipei (Taiwan), pp. 211-217. Hoppe, H. U., Plötzner, R. (1999). Can analytic models support learning in groups? In P. Dillenbourg, (ed.). Collaborative Learning - Cognitive and Computational Approaches. Amsterdam (The Netherlands): Pergamon, pp. 147-168. Hoppe, H. U. (2004). Collaborative mind tools. In M. Tokoro & L. Steels (eds.). A Learning Zone of One’s Own. Amsterdam (The Netherlands): IOS Press, pp. 223-234. Ishii, H., & Ullmer, B. (1997). Tangible Bits: Towards seamless interfaces between people, bits and atom”. In Proceedings of ACM CHI '97, ACM Press, pp. 234-241. Lingnau, A., Hoppe, H. U., Mannhaupt, G. (2003). Computer supported collaborative writing in an early learning classroom. Journal of Computer Assisted Learning, vol. 19 no. 2, 186194. Liu, T. C., Wang, H. Y., Liang, J. K., Chan, T.W., Yang, J.C. (2002). Applying wireless technologies to build a highly interactive learning environment. In Proceedings of WMTE 2002, Vaexjoe (Sweden), pp. 63-70. Machado, I., & Paiva, A. (1999). Heroes, villains, magicians, …: believable characters in a story creation environment”. In Proceeding of the AI-ED 1999 Workshop on Life-like Pedagogical Agents, Le Mans (France). Norman, D. A. (1998). The Invisible Computer. Cambridge (MA): MIT Press. Papert, S., & Harel, I. (1991). Situating Constructionism. Norwood (NJ): Ablex Publishing. Prada, R., Machado, I., & Paiva, A. (2000). Teatrix: Virtual environment for story creation. In Proceedings of ITS 2000, Montreal (Canada), Springer LCNS 1839, pp. 464-473. Propp, V. (1968). Morphology of the Folktale. Austin: University of Texas Press. Reichen, J. (1991): Lesen durch Schreiben. (German teacher’s guide to “Reading through Writing”). Heinevetter Lehrmittel Verlag. Roussos, M., Johnson, A., Leigh, J., Vasilakis, C., Moher, T. (1996). Constructing collaborative stories within virtual learning landscapes. In Proceedings of EuroAI-ED, Lisbon (Portugal), pp. 129-135. Singer, D., & Singer, J. (1990). The House of Make-Believe. Harvard University Press. Streitz, N., Geißler, J., & Holmer, T. (1998). Roomware for cooperative buildings: integrated design of architectural spaces and information spaces. In Proceedings of CoBuild 1998, Darmstadt (Germany), pp. 4-21.

138

H.U. Hoppe et al.

Tewissen, F., Lingnau, A., & Hoppe, H. U. (2000a). Today’s Talking Typewriter - supporting early literacy in a classroom environment. In Proceedings of ITS 2000, Montreal (Canada), Springer LCNS 1839, pp. 252-261. Tewissen, F., Baloian, N., Hoppe, H. U., Reimberg, E. (2000b). “MatchMaker” synchronising objects in replicated software architectures. In Proceedings of CRIWG 2000, Madeira (Portugal). IEEE Computer Society Press, pp. 60-67. Weiser, M. (1993). Some computer science issues in ubiquitous computing. Communications of the ACM, vol. 36, no. 7, 75-84.

Chapter 9 DESIGNING A CSCL ENVIRONMENT FOR EXPERIMENTAL LEARNING IN A DISTANCE LEARNING CONTEXT

M. Felisa Verdejo, Beatriz Barros, Timothy Read, Miguel Rodriguez-Artacho Departamento de Lenguajes y Sistemas Informáticos UNED Ciudad Universitaria s/n, 28040 Madrid, Spain {felisa, bbarros, tread, martacho}@lsi.uned.es Abstract:

1.

This paper deals with the design of collaborative support for experimental learning, focusing on the articulation of actions in the lab, either real or virtual, and argumentation. Our approach is based upon a distance learning context where we distinguish between three phases: pre-lab, lab and post-lab. We elaborate on the pre-lab phase. The goal of this phase is to provide students with motivation and context for the lab phase, in order to situate theory and experimentation. We provide an environment where individual and remote collaborative activities are combined. Activities are structured to focus student attention on the issues they should learn about: content-related and problemsolving techniques as well as interpersonal skills. We propose to characterize the collaborative support in terms of the type of learning tasks in order to help designers define the kinds of mediational tools best suited for an experimental learning activity.

INTRODUCTION

A growing number of studies have been undertaken regarding the design of computer support for learning collaboratively scientific reasoning. Discussion is highlighted as a primary medium for knowledge building in science. A variety of means have been proposed to promote and reify scientific dialogue in learning situations: typed and structured spaces for argumentation to be used synchronously either by face to face interaction or

140

M.F. Verdejo et al.

via distributed access (Wan & Johnson, 1994) (Suthers & Jones, 1997) (Nakamura et al., 1999) (Hoppe et al., 2000); semi-structured hypermedia databases allowing distant students to create notes, choose and assign labels to them as well as establishing connections with others author’s notes in order to build a communal database by an incremental, asynchronous and distributed process (O´Neill & Gomez, 1994) (Bell et al., 1996) (Scardamalia & Bereiter, 1994); conversational tools to structure students interactions (Baker & Lund, 1997) (Ravenscroft & Hartley, 1999). This chapter deals with the design of collaborative support for learning science, focusing on the articulation of actions in the lab, either real or virtual, and argumentation. Drawing on previous experience (Verdejo et al., 1999) (Barros et al., 1998) we have developed a system which enables students to perform tasks collaboratively and facilitates the analysis of student’s actions and interactions, in order to provide them with feedback to enhance their learning process. Our aim is to support learners in complex learning tasks, involving collaborative experimentation in the lab. The learning environment has to provide articulation between the experimentation space and the argumentation space, offering facilities to handle different types of information (text, video, graphics) generated by a variety of tools such as: drawing tools, databases, simulators, different editors, personal and shared notepads, cognitive and communication tools. This paper is organized as follows: section 2 elaborates the motivation for our proposal and briefly describes the current experimental context; while section 3 outlines the new framework we have developed. The design for the computer support for the pre-lab phase will be detailed and discussed in section 4. We finish by pointing out some issues for future work.

2.

MOTIVATION

In distance education institutions, such as the UNED 4 , the study of experimental subjects requiring lab work is organized in turns, where students come to the University to follow an intensive lab stage of three to six days in the middle or at the end of the academic year. They receive a handout with guidelines on how to perform the experiments before coming to the lab, and they have to write a report at home after performing the experiments. The lab is an interesting experience for students even if not satisfactory connected with their individual study throughout the academic year. 4

UNED is the Spanish Open University, operating worldwide with about 200.000 students

Designing a CSCL Environment

141

Networked technologies open the way to create new lab frameworks for science education in a distance learning context. Physical presence and manipulation still remain crucial but this lab work should be better intertwined with the rest of the learning period. There are many possibilities for new design, some of them mentioned in the introduction. Our approach is to start from the current situation looking for opportunities to improve the process, with a very realistic and learner-centred perspective. First we have created a pilot lab course, a scalable model, taking into account the constraints of the social context where we are. This case study scenario is been developed in connection with the DiViLab5 project. We have selected Chemistry as the domain application, a subject offered in the second year at the Industrial Engineering School. First of all we have analysed the current scenario, following the observation of a series of student lab sessions in March 2000, and subsequent in depth discussion with the teaching staff. As a result, we pointed out a couple of major problems: 1. Students are provided with documentation about the lab work in advance but they do not work with or even look at the guidelines before coming to the sessions. The tight schedule of the lab sessions does not favour thinking and reflection. This, together with the fact that they have no previous preparation, means that students in the lab need to focus on figuring out what to do, making interpretations on the fly about the procedures outlined in the guidelines; like following a recipe. The result is a very poor articulation between the theoretical knowledge students could have and the practical manipulations they are carrying out. 2. Experiments are performed in groups of two. They have no previous experience either of collaboration or collaborative support for the work they have to perform together in the lab and subsequently. For instance, each student uses personal notepads to write observations and his/her own copy of the guidelines to make annotations during the lab period. Usually they do not check whether their notes are complete, complementary or inconsistent. These sketchy notes are used afterwards for writing the final report, which is written from scratch. To cope with these problems, and taking into account that lab schedules can not be changed for organizational purposes at institutional level, we have proposed the inclusion of a pre-lab period where students, at home, at their own pace, could carry out virtual lab activities in collaboration. Next we 5

DiViLab is a project funded by the EC under the IST- 5th framework program. The consortium includes Archimed, Aveiro University, Duisburg University, France Telecom, INESC, UNED, and UST Lille

142

M.F. Verdejo et al.

outline the new global framework and then we will focus in the design of the pre-lab phase.

3.

THE NEW FRAMEWORK

We consider three phases, as illustrated in figure 1. For each phase, a computer-supported environment offers a structured scenario with functionality for carrying out individual and collaborative activities. A variety of mediational tools are available, some for the entire period such as structured glossaries, others specifically intended for a particular phase, for instance simulation models. At PRELAB students at home carry out problem solving tasks to develop a working understanding of the subject matter; At the LAB, students work in a real lab and focus on manipulation with chemical tools and chemical processes, the system in this phase supports data collecting as well as some collaborative modelling; and at POSTLAB, students should reflect together and articulate in depth theoretical background and the experimental work carried about in the lab.

3.1 Pre-Lab The goal here is to provide motivation and context for the lab phase, in order for the student to integrate theory and experimentation. We provide an environment where individual and remote collaborative activities are combined. Activities are structured in order to focus student attention on the issues they should learn about: content-related and problem-solving techniques as well as interpersonal skills. The environment enables students to undertake a “simulated” lab experience. An experiment is presented as a space of related tasks where, the computer definition includes all the possible steps or paths that could happen during experimentation. Students have to collaboratively decide which subtask has to be performed next for a particular problem-solving situation. Direct observations (for instance the colour of a composite) and results (for example the boiling point when the composite is heated), are provided by simulation techniques. The environment is seen by students as a personal structured notepad, where the performed tasks are recorded as well as the outcome obtained from the use of different tools. This information can be consulted or reused in further tasks.

Designing a CSCL Environment Developing working knowledge

Carrying out experiments

PRE-LAB

Reflecting to consolidate learning

LAB

REMOTE VIRTUAL COLABORATION

FACE - TO - FACE COLLABORATION

asynchronous

synchronous

POST-LAB REMOTE VIRTUAL COLABORATION

asynchronous synchronous

task Chemical tools

task Goal (final state) List of possibilities to the goal Individual work + Argumentative discussion

computer

Lab notes Data Lab process

143

Subtasks

Material for reflecting

Argumentative discussion and collaborative reporting

Final report Discusion process

Simulation Actions and Decision Processes

Figure 1. The three stages.

3.2 Lab Students carry out experiments in the lab in pairs. A computer is available in each lab workplace. The support environment is similar to the pre-lab, but the contents and functionality are enriched, for instance help about how to do a particular procedure is provided on demand (this kind of advice is some times in form of animations). Furthermore, input data from a variety of devices is stored. Collaboration here is important, and our expectation is that collaborative practice in the previous phase could facilitate patterns for this second situation where students meet for the first time and work together for three days in the same place. Hypothesis formation in the pre-lab phase should help students to handle the experimental space. The notepad is shared here. The environment provides export/import facilities from their notepads for a variety of tools. Collected data about the experiments can have a variety of formats, for example pictures taken by the students of the processes and intermediate results. Students are encouraged to annotate, comment and discuss data together A modelling tool (Hoppe et al., 2000) will provide structural and visual support for building scientific argumentation, in real-time, in the same working space.

144

M.F. Verdejo et al.

3.3 Post-Lab Students will work together from home through remote collaboration to prepare their final reports. Two configurations will be supported, either asynchronously using a collaborative learning tool which allows argumentative discussion or synchronously with a multi-user editor. These reports should include elaborated explanations i.e., students should go from a general description of what happened, to a causal explanation (why did it happen?), and to a justification of the causal explanation (what the evidence supports?). Some questions proposed by teachers at the lab will help them in this task. Assistants are also actors in the learning community; the system supports their tasks in the post-lab phase. Each assistant has to mark, comment upon and assess a group of students. There is a tool to read and annotate these reports as soon as students decide to submit them. Students automatically receive a notification once this feedback is available. Based on these premises, we will focus now on the rationale for the design of the PRELAB phase.

4.

PRE-LAB DESIGN

Students have to solve similar problems as those which will be proposed in the lab, but in this phase they perform simulations, not real experimental procedures. In this phase students should become aware of the kinds of experiments and mental techniques they are going to work with and will therefore feel more familiar with in the lab where they can pay more attention to the details of the real manipulations. Hopefully they will have more time for thinking and reflection while doing so. The goal of the pre-lab phase is twofold following a knowledge coconstruction perspective: 1. To improve student background knowledge, using a problem solving approach. Students are asked to solve similar cases, as they will in the real lab, although the physical and chemical experiments used to obtain data are simulated. In this pre-lab phase the focus is placed on acquiring the underlying reasoning process. A mechanism is provided to explicitly represent tasks and their structure, including workspaces where data can be collected from a variety of tools. For each task students have to express their contribution using categorical labels, sometimes individual work, others times as the result of peer negotiation. 2. To enable students to work in collaboration, promoting participation in task related activities such as discussing and reaching a decision,

Designing a CSCL Environment

145

arguing, explaining or justifying a particular proposal, asynchronous collaboration is provided through a combination of working spaces and computer mediated conversation. Next we will elaborate the pre-lab scenario and outline the functionality of the computer environment supporting this phase. However, we will not describe the technical details of the underlying system architecture.

4.1 Scenario Description Students work at home, using a virtual environment. This environment offers an integrated space of resources where students have to solve a set of problems. The environment includes general domain knowledge, such as a glossary, and specific content for each kind of problem to be solved, for example: z The objective of the virtual experiment z A minimal theoretical background z A description of the experiment in term of a set of tasks. An experiment usually involves several tasks. Thus an experimental description is divided into several subtasks and for each one there are some indications about the particular constraints as well as the different possible methods available to perform them. In more abstract terms, we can say that the problem is represented as a search in a task space. Not all the tasks need to be done for a particular case. Each pair of students will solve a different case, selecting the subtasks and the steps to be performed for a particular problem. The student environment for a particular task contains links to: z the multimedia glossary as well to other sources of information. z workspaces, either personal or shared. z a set of domain tools such as specialized editors, simulators of databases z a semi-structured conversational tool The glossary contains domain knowledge: concepts, properties, instruments, procedures …. defined or explained in different structured ways. The rationale here is to promote an active, goal-oriented approach for providing information. As it has been pointed out “people learn best when engrossed in the topic, they are motivated to seek out new knowledge because they need it in order to solve the problem at hand ” (Norman & Spohrer, 1996).

146

M.F. Verdejo et al.

The set of workspaces in the environment can be viewed as being a personal notepad, structured and related to the subtasks each student has selected to carry out a particular problem. The environment supports multimedia objects with different representations. For instance, an appropriate word processor for chemistry is available to build formulae as graphics and the results are pasted into the notepad. Furthermore, the outcome of a search in a database of infrared spectrum can also be stored. All the tools are integrated into the environment, so that students can easily use them and paste the outcome in their notepad. To solve a problem, students have to adopt a search strategy for the task space. For each candidate task students have to z discuss and take a decision on whether to do it or not, justifying their choice. z in the case of performing a task, i.e. when carrying out a test to collect evidence, they have to explicitly identify their initial hypothesis, the collected data and the conclusions reached. In the pre-lab phase students obtain the data either by running a simulation, or by querying a multimedia database. To facilitate peer collaboration in this problem-solving process a semistructured conversational tool is provided, based on DEGREE (Barros & Verdejo, 1999). Figure 2 illustrates the student environment in the pre-lab phase. In the upper part of the main window the title of the current task appears: Identification of an unknown organic compound. In the left part of this window, indicated by label 1, there is an index with direct access to the subtasks for the virtual experiment at hand. The current status of the student task space is visualized by clicking on the task list button (2) where for each possible essay, we know whether or not it has been considered (submitted or not submitted) and a procedure performed. Attached to decision there is a textual co-constructed knowledge item: decision and justification, and attached to procedure, as shown in figure 2, three more: hypothesis, data, and conclusion. These annotations reflect the outcome of the collaborative discussion and are an inspectable shared context for each group of students.

Designing a CSCL Environment

147

Figure 2. Student environment in the pre-lab phase.

The role of the teacher is firstly, preparatory, to build the set of cases to be proposed for the pre-lab phase, as well as to organize the groups. Secondly, in the pre–lab phase, it includes monitoring the work of students, with some input and support from the automatic tracking performed by the system. Furthermore, students have the possibility of asking the teacher questions related to a particular subtask. These questions and the answers (a kind of FAQ) are dynamically linked to the glossary. In this way the glossary evolves with the use of the system, and it is a student’s contribution to a kind of “organizational memory” for future students. Some advice is available about particular choices. This advice is configurable, using two parameters, one fixed by the teacher to specify whether or not it should be available for a particular group of students, and another based upon the previous student behaviour, to offer different levels

148

M.F. Verdejo et al.

of help. For this purpose, the system builds a simple student model recording student actions.

4.2 Learning Design Different collaborative strategies could have been implemented. For the first pilot study, based on our previous experiences in the context of distance learning (Verdejo et al., 99), we have selected the following one: Students start solving a simple case individually, in order to became familiar with the environment, and then in pairs for a more complicated case. Typically an experiment is made up of a set of related essays together with some modelling or interpretation of the data collected. For the essays, students are asked (1) to explicitly discuss and negotiate some critical decisions: for example which tests should be undertaken and in which order different ones should be performed. (2) to identify hypothesis, data and conclusions for each essay they decide to carry out and (3) to comment on the results obtained. For the pre-lab scenario, we propose the use of prescriptive conversation to accomplish the collaborative task of taking decisions and developing justifications. Conversation consists of turn taking, where each contribution both specifies some content and grounds it (Bobrow, 1991). The type of contributions and their combination can be defined to establish an explicit structure for a conversation. Our underlying model for the conversational structure is a labelled oriented graph (Barros & Verdejo, 1999). This mechanism allows structuring a group conversation in a generic way. Figure 3 shows a snapshot of the student interface with a set of deployed windows corresponding to a discussion point with the collaborative tool. In the upper part appears the main window, with the title Virtual Chemical Lab. In this window, indicated by label 1, students have selected to decide whether to undertake the nitrogen essay, and the conversational tool is opened. The discussion process is as follows: a user adds, modifies or comments upon the colleagues contributions with a new contribution following a conversational graph. Each new contribution is visible in a workspace (label 3) for all group members and will be part of the discussion process, represented as a node in a discussion-tree (label 2). This tree is automatically built by the system. During the discussion process a user can complement or reply to any previous contribution by clicking a node in the tree. The selected contribution appears on the right side, and depending on its type, all the possible options are displayed. For instance to a proposal, one could add a question, a comment, a counterproposal or an agreement. The user selects a type and elaborates his/her contribution. Once submitted, this new contribution is linked automatically to the selected contribution, and

Designing a CSCL Environment

149

the discussion tree updated. Only some of the contributions will form part of the final document (label 4), and they can be chosen with an agreement method through voting. In this conversation processes, proposing, answering, refining, comparing, negotiating and coming to an agreement are combined, in an ordered and alternated way. We mean this process of interchanging and co-construction of common ideas as argumentative discussion. The same mechanism applies for establishing the hypothesis, data, and conclusion for a selected essay.

1

2

Discussion process

5

Pre-lab task

3

Discussion tree

4

Result 6

Figure 3. Collaborative tool for group discussion integrated in the active document.

A summary of the problem solving process i.e., current task space exploration is always available and inspectable (label 6) by the students. This explicit representation permits further metacognitive learning tasks,

150

M.F. Verdejo et al.

such as enabling students to reflect and compare different problem-solving approaches. The environment described is generic in the sense that it provides a representation for learning activities in terms of a structured problem-solving space of tasks. Learning is viewed as a collaborative exploration mediated by argumentative discussion. Apart from writing essays, students also have to perform other kinds of generic tasks such as modelling, for example in the case of functional analysis they have to collaboratively annotate the relevant peaks of the infrared spectrum with the components they have previouslsy guessed. This is a different situation requiring another kind of collaborative support. For this type of modelling task a collaborative tool combining discussion and direct manipulation on the spectrum is best suited. Figure 4 shows an example, where two students, each one using his/her own window but sharing the same content, have identified and marked two areas of their infrared spectrum. The tool provides a shared blackboard, with two separated areas, one for annotating the spectrum, the other, on the bottom, to comment on the actions performed.

Figure 4. Collaborative tool for co-drawing and discussing in groups.

Designing a CSCL Environment

5.

151

CONCLUSION

The pre-lab phase is a multiple-possibility problem space that can be explored by the students. Each student can work on different aspects of a given task at his/her own pace, but at certain points in time they have to decide what the next action they have to do is. The purpose of obligatory points is to synchronise the students and guide them when working collaboratively. A tool is provided for argumentative discussion allowing an explicit and categorised representation of common decisions and knowledge constructs. Some tasks also need other kinds of external representations to support collaboration, a subject of further study in other chapters of this book. The scenario discussed in this chapter is currently under evaluation. This particular application offers interesting aspects to study and analyse collaboration through different mediating tools. Firstly, we can represent the process of moving through the problem-space. This representation will show the actions taken for solving the task. Secondly, we have also categorised information related to each task, as well as the asynchronous-communication structured log and the nature of the problem-space. Our aim is to make an analysis based on interaction (Barros & Verdejo, 2000) combined with an analysis based on actions (Mühlenbrock & Hoppe, 1999) to better understand the relationships between actions and interactions in a collaborative framework. Collaborative learning support is an open subject in many aspects. From the designers point of view a characterization in terms of tasks and suitable tools would greatly help to the selection of the type of computer support for a concrete learning situation.

6.

FURTHER WORK

The work presented in this paper corresponds to the first pilot developed in the project. Subsequently, based on an analysis of feedback obtained from a study, a new system was designed (Verdejo et al., 2002), and the framework was extended to cover the lab and the post-lab phases (Verdejo et al., 2003). Furthermore, the system was successfully used for second year students in the UNED Industrial Engineering School, for two academic courses.

ACKNOWLEDGEMENTS This research was supported in part by the DiViLab project, from the IST-5th framework program of the European Community. The ideas

152

M.F. Verdejo et al.

presented here however do not represent the official approach of the consortium.

REFERENCES Baker, M., & Lund, K. (1997) Promoting reflective interactions in a CSCL environment, Journal of Computer Assisted Learning, 13(3), 175-193. Barros, B., Rodríguez-Artacho, & Verdejo, M.F. (1998). Towards a model of collaborative Support for distance learners to perform joint tasks. The Virtual Campus: Trends for Higher Education and Training. M.F. Verdejo and G. Davies (Editors). Chapman and Hall. 155-168. Barros, B., & Verdejo, M.F. (1999). An approach to analyse collaboration when shared structured workspaces are used for carrying out group learning processes. Proc AI&ED’99. S. Lajoie and M. Vivet (Editors). IOS Press. 449-456. Barros, B., & Verdejo, M.F. (2000). Analysing students interaction process for improving collaboration. The DEGREE approach. International Journal of Artificial Intelligence in Education. 11, 221-241. Bell, P., Davis, E.A., & Linn, M.C. (1995). The Knowledge Integration Environment: Theory and Design. Proceedings of the 1995 Computer Supported Collaborative Learning, 14-21. Bobrow, D. (1991). Dimensions of Interaction. AAAI-90 Presidential Address, AI Magazine, 12(3), 64-80. Hoppe, H.U., Gabner, K., Mühlenbrock, M., & Tewissen, F. (2000). Distributed visual language environments for cooperation and learning: applications and intelligent support, Group Decision and Negotiation 9, 205-220, Kluwer Academic Publishers. Mühlenbrock, M., & Hoppe, H.U. (1999). Computer Supported Interaction Analysis of Group Problem Solving, Proc. CSCL-99, (C. Hoadley & J. Roschelle,Editors), 398-405. Mahwah, NJ: Erlbaum. Nakamura, M., Hanamoto, K. and Otsuki, S. (1999). Assistance and visualization of discussion for group learning, Proc AI&ED 99. S.Lajoie, & M.Vivet (Editors). IOS Press, 465-472. Norman, D.A., Spohrer, J.C. (1996). Learner-Centered Education. Communications of the ACM, Vol 39, No.4, April, ACM Press, 24-27. O´Neill, D.K., & Gomez, L.M. (1994). The collaboratory notebook: a networked knwoledgebuilding environment for project learning. In Proc. ED-MEDIA’94. 416-423. Ravenscroft, A., & Hartley, R. (1999). Learning as Knwoledge refinement: designing a dialectical pedagogy for conceptual change, Proc AI&ED 99. S.Lajoie, & M.Vivet (Editors). IOS Press, 155-162. Scardamalia, M., & Bereiter, C. (1994). Computer Support for Knowledge-building communities, Journal of the Learning Sciences, 3(3), 265-283. Suthers, D., & Jones, D. (1997). An architecture for intelligent collaborative educational systems, Proc AI-ED’97. B. du Boulay, R.Mizoguchi (Editors). IOS Press, 55-62. Verdejo, M.F., Rodríguez-Artacho, M., Mayorga, J.I., & Calero, Y. (1999). Creating Webbased scenarios to support distance learners, Building University Electronic Educational Environments. S.S. Franklin, & E.Strenski (Editors). Kluwer Academic Publishers. Verdejo, M.F., Barros, B., Read, R., & Rodríguez-Artacho, M. (2002). A System for the Specification and Development of an Environment for Distributed CSCL Environments, Proceedings of 6th International Conference on Intelligent Tutoring Systems,(Cerri, S., Gouardères, G., Paraguaçu, F.) LNCS, Springer-Verlag. 139-148.

Designing a CSCL Environment

153

Verdejo, M.F., Barros, B., Gómez Antón, R., & Read, T. (2003). The design and implementation of experimental collaborative learning in a Distance Learning context, Proceedings of 4th international conference on Information Technology Based Higher Education, in cooperation with the Institute of Electrical and Electronics Engineers (IEEE) Education Society and with the United Nations Educational, Scientific and Cultural Organization (UNESCO) Wan, D., & Johnson, P. (1994). Experiences with CLARE: a computer-supported collaborative learning environment. Journal of Human-Computer Studies. 41, 851-859.

Chapter 10 PUPIL COMMUNICATION DURING ELECTRONIC COLLABORATIVE PROJECTS: INTEGRATING COMMUNICATION TOOLS WITH COMMUNICATION SCENARIOS

Chronis Kynigos1, Evangelia V. Dimaraki2, Evie Trouki2 1

Educational Technology Lab, University of Athens – School of Philosophy & CTI

2

Educational Technology Lab, University of Athens – School of Philosophy

Abstract:

1.

The rapid expansion of computer-mediated communication (CMC) into classrooms has nurtured expectations that Communication Technology (CT) will support pupils’ construction of shared knowledge by articulating their thoughts and reflecting on their activity. In this paper we argue that the connection between CMC and such learning experiences is far from selfevident. The paper investigates how the coupling of CMC tools with educational ‘scenarios’ can structure communication learning activities in the classroom. We describe the design aspects of two different scenarios, such as activities, communication need, roles, layers and channels. We also provide qualitative empirical evidence on educational potential for information handling, reflection and communicative awareness. Implications for whether a communication setting ensures meaningful communication are discussed.

INTRODUCTION

In this chapter we argue for the importance of developing descriptions of educationally principled activity plans (we call them ‘scenarios’), rather than restricting focus mainly on collaborative tools and the types of activities encouraged through their use. These plans, which we call ‘communication scenarios’, take into account collaborative tools, but also other types of tools which may have not been designed for collaboration per se but are used in a collaborative setting. Furthermore, they also equally take into account the

156

C. Kynigos et al.

orchestration and definition of an activity where communication and collaboration make sense and have some functional purpose. Traditional classroom paradigms are built on one – way communication aiming to transfer established information from teacher or resource to student and to guide the student to rote learning without much meaning or deep understanding of the significance or the concepts underlying that information. Communication Technology (CT) seems a good lever for generating change in this paradigm and building what Mercer has called a social mode of learning (Mercer, 1995), where communication is used for the co-construction of meaning. The rhetoric advocating the use of CT for collaborative learning, however, still seems rather fragmented from what we know about learning processes in individual and face-to-face social contexts. This chapter aims to contribute to a growing volume of research adopting an integrated perspective to design educational environments (Garcia & Jacobs, 1998, Sfard, 2002). For instance, there is a large volume of research within the constructionist paradigm, involving students in small face-to-face collaborative groups working with computational tools designed for experimentation, expression, exploration, handling data and manipulating models (Cobb, 1995, Kynigos & Theodosopoulou, 2001). The type of learning encouraged in such environments can be enhanced by the CSCL idea, and the collaboration from a distance movement can build on knowledge about learning developed by the constructionist movement. In this paper, we argue of the importance of developing communication scenarios as frameworks for building collaborative learning environments. We describe two such scenarios we developed and implemented one involving structured collaboration in student groups and the other openended cooperation in a discussion forum. Our intention in both was to illustrate how they structure aspects of the learning situation such as: •

how communication need emerges,

•

what communication goals are set by the participants (teachers & pupils),

•

what are the sources of communication content,

•

what is the status of information,

•

how the integration of communication with information promotes knowledge-building,

•

how communicative awareness is expressed.

Pupil Communication During Electronic Collaborative Projects

2.

157

THEORETICAL FRAMEWORK AND RELEVANT RESEARCH

We draw upon Vygotsky’s notion that the interaction with others promotes learning process through the interplay between internalization and externalization (Vygotsky, 1978). This interplay has been usefully, though implicitly, appropriated for the design of learning environments in the idea of constructionism (Papert & Harel, 1991) having emphasized the importance of the social interaction for the learning process through the notion of shared artifacts as “social constructs” (Shaw, 1996). Drawing from research involving face-to-face peer interaction in computer-based environments we see that even in the case of small collaborative groups, quality of peer talk does not come about automatically and its relations with the quality of the learning process is by no means unproblematic (Hoyles, Healy & Sutherland, 1991, Light & Mevarech 1992, Mercer, 1995, Kynigos, 1999b). What is at stake in situations such as these is the implicit change in discourse norms of presenting, receiving, sharing, controlling and negotiating knowledge. This change is related to challenges posed by the authority imbalance between teacher and pupils and the rigid classroom rituals in the traditional paradigm (Edwards and Mercer, 1987, Kynigos and Theodossopoulou, 2005). Computer networks drive research beyond face-to-face collaborative learning, creating new opportunities for learning within distributed environments, in “coordinated network-enhanced classrooms” (Songer, 1996). CT may encourage discourse patterns having more immediate and natural extensions to the real world than classroom-bred discourse patterns (Scardamalia and Bereiter, 1996). Still, according to related research (Kupperman et al., 1997, Lipponen and Hakkarainen, 1997) the opportunity for knowledge sharing in itself seems not to facilitate learning, as participants do not take automatically knowledge-oriented actions. Further thought must also be given to the use of CT tools regarding the question of what kind of information is pedagogically valuable to share and for what purpose. Therefore, we believe that a focal area of concern in promoting learning through CT tools is the design of communication scenarios, in the sense that these activities entail the tasks, the priorities of goals, the role of CT tools and the status of information in new CT-supported learning environments. The communication scenarios we built in this study theoretically draw upon the distributed constructionism perspective (Resnick, 1996) whereby pupils’ constructionist activity can be embedded within network communities. However, there has yet been little study on learning environments such as the ones reported here, where there is an integrated use

158

C. Kynigos et al.

of computational and communication tools to support learning by doing and communicating (see for instance, Resnick, 1996, Salomon, 1994). Our research moves in this direction of examining how dimensions of the learning context shape communication. Taking also into consideration that research based on the information ecology perspective doesn’t inform on whether learning is actually happening (Guzdial, 1997), we attach special importance on the substantive nature of computer-mediated interaction and we focus on the analysis of the discourse produced by groups of pupils while communicating with their “pen-pals” electronically. We believe that Computer Supported Cooperative Learning cannot be restricted to research on communication tools, without the study of communication scenarios, which weave together tools and learning process, thus our research addresses this need.

3.

ALTERNATIVE COMMUNICATION SCENARIOS

The need for designing the communication scenarios described in this paper stems from our concept about enhancing teaching and learning process with CT. Communication among remotely situated classrooms implies not only their equipment with communication tools but also a rationale for communicating. This is the function of the scenario idea. Within such a framework, each scenario is in effect a communication milieu, the important structural dimensions of which can be specified and conceptually described. In this section we present two scenarios, Communicating to Meet (CtM) and Bridges of Europe (BoE)6, the former involving structured collaboration in student groups and the latter open-ended cooperation in a discussion forum. In the Communicating-to-Meet (CtM) scenario remotely situated classrooms collaborate to plan for the exchange of pupil visits. According to the scenario pupils write a joint proposal to a funding agency. The proposal should include a rationale justifying the visits, detailed schedules, financing and activity plans. For all sections, exchange of information and opinions is necessary. In the Bridges of Europe (BoE) scenario, pupils select bridges, build their models, search and use cultural, historical, structural and architectural information on selected bridges and, finally, communicate about their

6

I.M.E.L.: Intercultural Microworld courseware for Exploratory Learning, European Commission, Socrates, Open and Distance Learning, 1996-1998. NETLogo: The European Educational Interactive Site. European Community, Educational Multimedia Taskforce, #MM1020, Joint Call in Educational Multimedia, 1998-2000.

Pupil Communication During Electronic Collaborative Projects

159

projects, via a web-site which includes services such as forums and libraries for uploading work. Drawing from the basic premise that “common ground” makes communication possible, we implemented the above two scenarios to organize communication among remotely situated classrooms. The core of both scenarios consists of specific activities-tasks, reasons for communication, roles, layers of communication and channels/tools of communication. The implementation of these scenarios reveals what happens when electronic communication is taking place according to the alternative features presented in the next table in which the middle column consists of the elements the two setups have in common. Table 1. Alternative communication setups.

CtM scenario Specific characteristics Structured communication Pupils in different locations write a joint proposal to some agency to fund the exchange of visits. They construct active maps and trip costs representations Collaboration The need for communication is carefully in-built Exchange information during their project in order to complete it Between specified groups Consistent messagereply flow Participants from 1 country

Joint characteristics of both scenarios Using a scenario as a point of reference Combine construction with communication

Communication goal to provide and request information

Communication via e-mail

BoE scenario Specific characteristics Open-ended communication Pupils collect information about local bridges, build models of them and communicate about their projects Cooperation The common activity is a trigger for communication Exchange information on their project In a forum Not every message gets a reply Participants from three countries Use of English

160

C. Kynigos et al.

Use of mother language

4.

METHODOLOGY

We designed our communication scenarios and carried out research on classroom implementation adopting a socio-cultural perspective (Wertsch, 1991). To investigate the incorporation of remote communication in the teaching-learning process and the classroom culture, we adopted a research methodology allowing the observation of human activities in real time (Cobb & Yackel 1996, Goetz & LeCompte 1984, Kynigos 1999b). We drew from the following sources of data: Observational data of classroom activity: during CtM activity the last two lessons in each school (8 hours) were videotaped; during BoE activity 3 lessons in each school (24 hours) were videotaped. The pupils’ e-mail messages: during CtM activity pupils exchanged 26 messages; during BoE activity pupils exchanged 30 messages. Pupils and Teachers Interviews: the interviews were open-ended, focusing on teachers’ and pupils’ opinions about the aims, the nature and the success of the project. During CtM activity 4 teachers and 6 pupils were interviewed; during BoE activity 4 teachers and 12 pupils were interviewed. The 6th grade of two primary schools participated in the CtM project: School K., which is an private school of Larissa, the primary school of Kalloni (School L.), a village in Lesbos. The 6th grade of four primary schools participated in BoE project: School A and School B (private schools in Athens), School C (private school of Larissa) and School D (private school in London). The observation study was carried out through video-recording and transcription of spoken language in the classroom settings 7 . The videorecording included: (a) a wide view of the classroom climate and activity with occasional focus on screens and pupils; and (b) a specific view on 7

We used the following observation equipment to produce our recordings: a video-camera, an external microphone and a sound-mixer to send sound to both the videotape and an audiotape.

Pupil Communication During Electronic Collaborative Projects

161

groups of pupils working. The relation between pupils, teachers and researchers carrying out the observation was collaborative and built previously. Where possible, all electronic data was captured and saved for analysis, especially those concerning communication. We analyzed the data using a bottom-up, generative process, drawing from ethnographic and discourse-analytic approaches (Mercer, 1995). In this paper we focus on how the new communicative demands were shaped by each scenario, and how these demands interacted with established classroom norms. We, thus, concentrated on pupils’ work around e-mail exchanges, which we attempt to illuminate by commenting on excerpts we selected as representative of the kind of issues emerging from the analysis. Table 2. Communication modes.

Traditional classroom communication

Joint construction communication

Communication is between teacher and pupils.

The teacher is not the center of communication.

Pupils aren’t allowed to talk with each other.

Pupils are allowed to talk to each other.

Teacher-pupil interaction consists of the sequence QuestionAnswer- Feedback. Teacher poses questions to which he already knows the answer in order to evaluate pupils’ knowledge.

Teacher does not have total control over the flow of communication in the class. Pupils express ideas, share information, argue, pose questions to each other, evaluate each other’s ideas.

The teacher is the main source of information in the classroom. Teacher transmits knowledge to pupils.

Pupils are encouraged to make experiments and find resources other than the teacher. Teacher is the coordinator of knowledge construction.

162

5.

C. Kynigos et al.

EMPIRICAL FINDINGS FROM CLASSROOM IMPLEMENTATION

5.1 A New Communication Context: Doing and Communicating In a traditional classroom, the mode of communication accompanies a view of knowledge as static and certain and a view of the learning process as an one-way transfer of accurate knowledge from teacher to pupils. Communication takes a different meaning when pupils learn in an exploratory way while working collaboratively; in this context, knowledge is something that the learner constructs together with others. The distinctive characteristics of these communication modes are presented in the table 2. The activity which occurred in the classroom during the CtM and BoE communication scenarios is very different from what takes place in a regular classroom regarding two main aspects: teacher role and role of information as a school commodity. The teachers felt challenged to change their role in the classroom into a learning partner. At the same time, personal experience became “informational material”. Significant effort went into students’ selecting of information for use by others and into processing the information received from their peers. However, the kind of educational practice to which the students were used to did not encourage them to talk about what they do and explain it to their peers. The following episode illustrates pupils’ difficulty in dealing with such communication goals. The researcher approaches a group of pupils who are working on composing a message and makes a suggestion to include in the message the construction process they followed: R: You could also tell them how you constructed your bridge. Following which process? They use Logo too. P: Yes, but we haven’t finished yet. Could we tell them the procedure we have made? R: Ok. How you’re thinking of constructing it. P: It is difficult however, because how they’re going to understand it? R: They will ask whatever they don’t understand, won’t they? P: All right, it’s just a bit difficult for them to understand the way we thought. R: Ok, but think of it. I mean along with other information tell them how you constructed it, since in due time you will send your project too. (BoE activity, Classroom Video-recording, Group, School A (5), 24/5/99)

In the above case pupils are taken aback by the suggestion to make their work the object of their communication. In response to the researcher

Pupil Communication During Electronic Collaborative Projects

163

suggestion, the pupil expresses his hesitation first indirectly, coming up with a number of rather trivial obstacles, and then directly admitting that he thinks it is a difficult task. The exchange of information as a communication goal interwoven with constructing personal artifacts is at the heart of both communication scenarios. Construction is the ground for communication and communication promotes construction. This interplay has implications for what information is communicated, from what sources and for what purpose. This, in turn, raises new cognitive demands for dealing with information in the classroom. Another set of new demands, which we describe with the term “communicative awareness” revolve around the act of communicating per se.

6.

THE STATUS OF INFORMATION

In traditional classrooms, information has an ontology of its own. It is considered as authentic and unquestionable, not a ground for argument or for expressing one’s own opinion. There is also a connotation of certainty, in the sense that being uncertain about a piece of information is an undesirable state of affairs and means that the information is not yet retained by the student well enough for them to be certain about it. In the communication scenarios in this study there were grounds for handling and conveying different kinds of information, that which is factual (like the names of places or a bridge’s architectural style) and that which is derived from the students’ experience (like the way in which they built a model of a bridge or the parts of the local park which they thought were interesting).

6.1 Informational Sources Both scenarios were conducive to an open–ended learning situation and a communication situation in which pupils gathered and conveyed information themselves using various sources. External sources and personal experience were complementary, without replacing the teacher as source of information. 6.1.1 External sources The first external source that the pupils turned to was, quite naturally, their parents. Later, pupils looked up a range of information sources outside the ones they traditionally accepted in class: travel agents, relevant books and web-sites. When the pupils were confronted with questions that made them realize that teachers can’t have all the answers, the teacher sometimes

164

C. Kynigos et al.

assumed the role of a resource person, who suggested possible sources of information. 6.1.2 Personal experience Faced with the task to look up and provide information, pupils also started to use their personal experience as a source, i.e., knowledge gained both in and out of school though their own experience. Pupils began to realize that what they had seen or done gave them knowledge that allowed them to inform others who did not have the same experience. In the CtM activity, personal experience had to do with general world knowledge. The personally meaningful element of the activity was its anchoring on pupils’ travel and locality. In the BoE activity, locality did not take central stage in the communication even though some elements were also respectively designed into the activity. For the most part the personal experience communicated had to do with pupils’ work on model constructions: their own work and the review of others’ work. In the following exchange, the element of personal knowledge based on locality enters into the students’ communication. This exchange of messages was motivated when two groups from different schools in different countries discovered that they were engaged in constructing the same bridge. A group of Greek pupils asked their British partners for information about the Tower Bridge that they were constructing. Hi Musa and John-Michael We are from Doukas School in Greece. Our team’s name is “THE BEST”. We are four kids Alex, Anthony, Maria and Antiope. We make the “TOWER BRIDGE” of London. Have you ever visited “TOWER BRIDGE”? If yes, could you tell us some information about it? Friendly “the best”

They received the following reply: Hi the Best, I’m afraid were not Musa and John-Michael. But we are working on Tower Bridge too. We are Ben, Oliver and Shuaib and we come from All Souls School in London. Tower Bridge is both a suspension bridge and a cantilever bridge, which makes it very special, and it is also special because it opens up in the middle to let ships through. Tower Bridge was opened in 1894. When we find some more information we will send it through to you. If you have any special questions we will be willing to try and answer them. There is also a funny true story about Tower bridge that when the bascules were open a bus drove over and was really lucky to land safely on the other side. Hope to hear from you soon. The TOWER’S.

Pupil Communication During Electronic Collaborative Projects

165

There is an interesting blend of ‘official’ and ‘personal’, informal knowledge in this message. Further, though Greek pupils might have encountered a description of the bridge as a “suspension” and a “cantilever” bridge, they probably could not have the vivid description of the bridge “opening up a in the middle to let ships through” or the “funny but true story”. In some cases, information was treated as an untouchable and valid object for transmission, leading to modes of communication that resemble traditional classroom communication. In other cases, working with information and on information from various sources revealed information as malleable and negotiable with respect to its fallibility and its relevance to the task at hand. In the BoE activity, communication was mainly oriented to personal constructions, which allowed the exchange of information that was negotiable (e.g., personal knowledge from performing experiments). However, within the same activity there were exchanges involving factual information, which was treated as fixed. We found the distinction between factual and procedural information useful in gaining some insight into the reasons why students considered pieces of information in these two different ways.

6.2 Informational Content: Factual vs Procedural Information Factual information was common in both communication setups, but took different forms in each. Procedural information was present mostly in BoE exchanges. In the BoE activity, exchanged factual information was mostly about the bridge each group chose to construct. On the other hand, in the CtM activity factual information was often drawn from personal experience. The exchange of procedural information in the BoE activity was related to the process of students’ model construction. Procedural accounts were rendered in two different ways: in a linear-procedural manner and in a problemsolving manner. In the following example the pupils describe their problem-solving experimentation that led to a solution by an equation using analogies: At first we formed a bridge with a given base distance and we tried to fit the equivalent arch, using the “Slider”. Next we created a second bridge with another width. Using analogies, we solved a math equation. In this investigation, we were helped a lot by the “Slider”, which let us find suitable arches by making experiments. (BoE activity, School C 18.05.99)

166

C. Kynigos et al.

The strategy pupils described in this excerpt is trial-and-error used in an early stage of their work, in order to reach a solution. These rather fragmentary, not fully and comprehensively developed articulations are, nevertheless, clearly attempts at problem-solving accounts. In summary, we see that, while both communication scenarios encourage the use of information in ways that depart from classroom-bred patterns, they do so in different ways.

7.

THE COMMUNICATION AWARENESS

One of the main challenges the students found in expressing themselves was the need for “de-centering of expression” 8 . Each time they wrote a message, they had to take into consideration the recipients’ needs, interests and experience. This demand was novel for the students, since in their traditional class, they rarely wrote for a recipient other than the teacher. Communicative awareness is related to concerns about both appropriate content and appropriate form.

7.1 Communication Aspects of Content Negotiations over the content of the outgoing message were often a sign of the pupils’ increasing preoccupation with the communicative aspects of the task. This negotiation was also connected with their desire for personal communication. The episode below is such an example, in which the pupils began to explore possible forms of questions: S3: Miss, should we write questions that we want to pose to the kids? T: Yes S3: But what questions, Miss? T: Well, you yourselves have to think what else you would like to ask. S3: Can we ask about their lives? T: About their lives, about their school. (CtM activity, classroom recording, group GB, Karavanas School, Larisa, 16/12/98)

The tension between school and personal communication is revealed in the teacher’s effort to re-focus the questions on school issues. Student discussion returns to the same issue later despite the teacher’s suggestion. 8

The term “decentering”, coined by Piaget, refers to the cognitive ability to take the perspective of others. We borrow it to refer to perspective taking during message composition.

Pupil Communication During Electronic Collaborative Projects

167

S2: Let’s write something about us, how we live. S3: No, let’s ask them. We are the ones sending a message to them. S2: Yes but we should tell them too. (CtM activity, classroom recording, GB Group, Karavanas School, Larissa 16/12/98)

Here, the development of a de-centered perspective, is expressed not only in anticipating the information needs of the other partners in communication, but also in considering the norms of e-mail communication.

7.2 Communication Norms: Rules of Form and Patterns of Practice In both scenarios the communication situation in which the pupils were introduced involved the exchange of information related to shared constructive activity. However, communication was differently organized in each case since the CtM activity involved structured communication between specified groups while in the BoE activity communication was unstructured and in a forum. As a result different communication patterns emerged. During the CtM activity, a “question-answer game” seemed to be at the center of the activity, both within and between classrooms. In reviewing both the e-mail messages exchanged and the transcripts of the classroom recordings one gets the impression that everything must be said in the context of a question or in the context of an answer. Thus, fairly formal and traditional rules of communication were established. On the other hand, the open structure of the communication in BoE activity seemed to permit variety in the communication pattern. There was no special in-built expectation beyond “informing others on our project” and “keeping in mind that others too are working on the same idea”, as one of the teachers put it. To provide a better sense of this “communication openness”, the following e-mail requests were respectively extracted from four different groups participating in BoE activity. Have you ever visited Tower bridge. If yes, could you tell us some information about it? We are looking forward to seeing your bridge. Drop us a line about your project. Can you send us a copy of the Greek alphabet?

168

C. Kynigos et al.

The lack of a ritual like the question-answer game coupled with the lack of a specific recipient for the message due to the forum set-up, did not guarantee a response. The increased awareness of the need to evoke a response, on behalf of the pupils was expressed in various ways; simple statements of anticipation of receiving an answer or specific requests. The following message contains a different but creative communication technique to address it: HI!!! This is “The Chelsea Gang” Can somebody send me a message please !!! I am begging you please. I have sent 3 messages already please reply. P.S Hope to hear from you. BYE!!!!! P.P.S I have an easy way to make an arch here it is: to arcl :a :b repeat :a [fd :b*11/630 lt 1] end OR to arcr : a :b repeat :a [fd :b*11/630 rt 1] (BoE activity, School D, student message, 10/5/99)

In this message the Logo code written by the students was used as a hook: they offered a Logo routine to attract correspondence. It is interesting to ponder on a communication interpretation of the fact that there are no explanations accompanying the provided code. In the CtM activity, appropriate forms of polite communication complemented the question-answer game in the organization of messages. Pupils never started their message by asking for information, but they always wrote an introductory sentence referring to their communication so far, responding to the requests of others or thanking for the information already received. During the BoE activity, a demanding task in terms of polite form was composing messages based on “trying out” the work of others to communicate criticism electronically. The next episode shows how teachers of school A introduced their class to this task: T: If you are going to criticize someone what you have to do is you have to do it in a positive manner, and you always have to back it up with your own (…) (CtM activity, Classroom Video-recording, Group, School D (5), 8/6/99)

Other aspects of communicative awareness have to do with considering their message in the context of the sequence in which it belongs. Pupils always checked on what they had written in previous messages, to avoid unnecessary repetitions and they tried to think of new things to say. They

Pupil Communication During Electronic Collaborative Projects

169

also compared messages and assessed the sufficiency of their response after writing it.

8.

DISCUSSION AND FUTURE RESEARCH ISSUES

Through the implementation of the CtM and BoE scenarios, our analysis illustrates ways in which they played a central role as a common referent that helped sustain communication, negotiate its ground rules and develop a dynamic that goes beyond school-like ways of relating and dealing with information. The aspects of classroom practice specific to working with these scenarios, which we identified here as educationally promising, were: •

the breakdown of the aspect of the teachers’ role as the only source of information,

•

the changing status of information,

•

the legitimization of communicating personal experience,

•

developing an awareness of the communicative aspects of expression,

•

the challenging of traditional communication rituals in classroom norms.

The teachers were challenged to change roles from that of the global expert to that of a learning partner, while sources external to the curriculum and pupils’ personal experience came to the surface. We observed a distinction in the perception of information. In some cases, information was treated as an untouchable and valid object for transmission, which occasionally led to reverting to traditional modes of communication. In other cases, information was treated as fallible and exchange was on personal views and descriptions of that information. Personal experience had to do either with pupils’ general world knowledge (e.g., pupils’ traveling and locality) or pupils’ work on model constructions. An important question arising from the analysis is why electronic messages are so emphatically assumed by pupils to be means of spreading information rather than personal thoughts. An explanation comes from the obvious consistency of this assumption with most current educational practice: pupils learn by getting information from a valid source - which is the teacher or the book - and reproducing it. Lack of previous experience

170

C. Kynigos et al.

with CT may lead some teachers to preserve or even revert to a restrictive, traditional mode while guiding pupils’ activity. Pupils also find it easier to reproduce information rather than produce original content in their messages, because this is what they are used to in their daily classroom work. Furthermore, classroom discussion and electronic messages offered evidence of a nascent awareness of written expression as an act of communication, an awareness that is generally hard to cultivate in school. Electronic communication created for the pupils the need to develop practices that allowed them to establish and sustain communication. The above analysis demonstrated pupils’ potential for sophisticated communication in their writing, as they negotiated the content of their messages and communication norms through the interplay of traditional and innovative communication practices. We also saw that the generation of communicative learning processes and the adoption of a social mode of thinking did not come about in an unproblematic way. Far from it, there seemed to be some rather deeply engrained obstacles: the pupils seemed to have a general lack of familiarity with social modes of learning. It seems that there is a lot more communicative potential that remains unrealized, and could be supported by variations in these scenarios. For example, in the BoE activity, pupils could have engaged in exchange of tips, ideas, Logo code and the like, aiming to construct two different models of the same or similar bridges. The scenario could also be turned from a cooperative to a collaborative one by having groups engage in a joint construction, each group constructing different parts of the bridge and joining them together in one model. Pupils would be expected to share ideas, try out and use others’ code, provide feedback, learn from each other and broaden their experience. A trade-off of such extensions to the scenario would be increased complexity, thus framing a need for specific CT tools to support connected communities of learners around the design and construction of the common artifacts. We can therefore see how a scenario through its empirically - based refinement can serve as a vehicle for research into the teaching-learning practice and the design of software. Our research focused on the ways the students jointly created norms for their activity when presented with a new communicative and instructional task and we documented its pedagogical potential. In future implementations of these scenarios, research should attend to leveraging this potential to avoid the danger of stagnation in the student communication, by guiding them to a more substantive involvement with the scenario and its collaborative challenges.

Pupil Communication During Electronic Collaborative Projects

9.

171

CONCLUSION

The discussion aimed to highlight several aspects of communication that are of educational value. At the same time we aimed to demonstrate several challenges in orchestrating communication of high educational quality. Our findings reveal how existing perceptions and practices of classroom communication are extended to the CT-supported environment and how there are challenged and at times changed. They also raise questions about whether communication can take place automatically or there are certain presuppositions. Electronic interaction may not be a meaningful exchange unless interlocutors share the same concept on what communication is and succeed to articulate their thoughts and put them on a common ground. Given that communication scenarios can be the “red thread’’ engaging classrooms in communication, they may have an important role in encouraging certain structures of electronic communication, and consequently they may prove to be an important issue on the effectiveness and the educational value of on-line classroom.

REFERENCES Cobb, P. (1995). Mathematical learning and small-group interaction: Four case studies. In P. C. H. Bauersfeld (Ed.), The emergence of mathematical meaning: Interaction in classroom cultures (pp. 25-129). Mahwah, NJ: Lawrence Erlbaum Associates. Cobb, P., & Yackel, E. (1996). Constructivist, emergent, and sociocultural perspectives in the context of developmental research, Educational Psychologist, Vol. 31(3-4), 175-190. Edwards, D., & Mercer, N. (1987). Common Knowledge: The development of understanding in the classroom. London: Methuen. Garcia, A., & Jacobs, J. B. (1998). The interactional organization of computer mediated communication in the college classroom. Qualitative Sociology, 21(3), 299-317. Goetz, J., & LeCompte, M. D. (1984). Ethnography and Qualitative Design in Educational Research. London: Academic Press. Guzdial, M. (1997). Information ecology of collaborations in educational settings: influence of tool. In R. Hall, N. Miyake, N. Enyedy (Eds.), Proceedings of the CSCL Conference (83-90). Hoyles, C., Healy, L., Sutherland, R. (1991). Patterns of discussion between pupil pairs in computer and non-computer environments, Journal of Computer Assisted Learning, 7, 210-228. Kupperman, J., Wallace, R., Bos, N. (1997). Ninth Graders’ use of a shared database in an Internet research project: issues of collaboration and knowledge-building. In C. Hoadley and J. Roschelle (Eds.), Proceedings of the CSCL Conference (157-163). Kynigos C. (1999). Perspectives in analyzing classroom interaction data on colalborative computer based mathematical projects. In C. Hoadley and J. Roschelle (Eds.), Proceedings of the CSCL Conference (333-341).

172

C. Kynigos et al.

Kynigos, C., and Theodosopoulou, V. (2001). Synthesizing Personal, Interactionist and Social Norms Perspectives to Analyze Student Communication in a Computer - Based Mathematical Activity in the Classroom. Journal of Classroom Interaction, 36.2., 63-73. Light, P. H., & Mevarech, Z. R. (1992). Peer-based interaction at the computer: Looking backward, looking forward, Learning and Instruction, 2, 275-280. Lipponen, L., & Hakkarainen, K. (1997). Developing culture of inquiry in computersupported collaborative learning. In R. Hall, N. Miyake, N. Enyedy (Eds.), Proceedings of the CSCL Conference (164-168). Mercer, N. (1995). The Guided Construction of Knowledge: Talk amongst teachers and Learners. Clevedon: Multilingual Matters Ltd. Papert, S., & Harel, I. (1991). Constructionism. US: Ablex Publishing Corporation. Resnick, M. (1996). Distributed Constuctionism. Proceedings of the International Conference on the Learning Sciences. Association for the advancement of Computing in Education. Northwestern University. Salomon, G. (1994). Distributed Cognition. Cambridge UK: Cambridge University Press. Scardamalia, M., & Bereiter, C. (1996). Computer Support for Knowledge-Building Communities. In T. Koschman (Ed.), CSCL: theory and practice of an emerging paradigm. USA: Lawerence Erlbaum, NJ, 249-268. Sfard, A. (2002). There is more to discourse than meets the ears: Looking at thinking as communicating to learn more about mathematical learning. In C. Kieran, E. Forman & A. Sfard (Eds.), Learning discourse: Discursive approaches to research in mathematics education (pp. 13-57). Dordrecht, Netherlands: Kluwer. Shaw, A. (1996). Social Constructionism and the Inner City: Designing Environments for Social Development and Urban Renewal. In Y. Kafai and M. Resnick (Eds.), Constructionism in Practice: Designing, Thinking and Learning in a Digital World. NJ USA: Lawerence Erlbaum. Songer, N. B. (1996). Exploring Learning opportunities in Coordinated Network-Enhanced Classrooms: A case of Kids and Global Scientists, The Journal of The Learning Sciences, 5(4), 297-327. Vygotsky, L. S. (1978). Mind in Society. London: Harvard University Press. Wertsch, J. V. (1991). Voices of the Mind: a Socioculutral Approach to Mediated Action. Cambridge, MA: Harvard University Press.

Chapter 11 SUPPORTING AWARENESS IN DISTRIBUTED COLLABORATIVE LEARNING ENVIRONMENTS

Hiroaki Ogata, Kenji Matsuura, Yoneo Yano Faculty of Engineering, Tokushima University 2-1, Minami-josanjima, Tokushima-shi, 770-8506, Japan ｛ogata, matsuura, yano}@is.tokushima-u.ac.jp Abstract:

This article focuses on distributed collaborative learning environments where it is very difficult for the learners to understand the other learners’ knowledge and activities. For example, since the learner cannot be aware of the

other learners’ knowledge and activities in using a web based collaborative learning system, it is very difficult to find suitable partners at the beginning of collaboration. Therefore, this chapter proposes knowledge awareness map to visualize the relationship between the shared knowledge and activities of other learners. For example, a learner may easily approach to the peer who is interested in the same knowledge. It plays a very important role in finding peer helpers, and inducing collaboration and knowledge sharing. In this map, a mediator agent recommends suitable collaborators who can help solving the problem. The prototype system, SharlokII has been developed and used experimentally. Moreover, knowledge awareness in mobile learning environment is described as a recent work.

1.

INTRODUCTION

Researchers in the educational systems area are attempting to provide technological support for cooperative and collaborative learning advocated by educational theories (Slavin, 1990; Webb & Palincsar, 1996; O’Malley, 1994). This chapter focuses on distributed collaborative learning environments, similar to systems such as CoVis (Edelson et al., 1996), KIE (Linn, 1996), CSILE (Scardamalia & Bereiter, 1996), WebCamile (Guzdial et al., 1997) and Belvedere (Suthers & Jones, 1997). CoVis focuses on making a

174

H. Ogata et al.

collaboration process visible. KIE succeeded in helping the students to link, connect, distinguish, compare, and analyse their repertoire of ideas. CSILE and WebCamile support knowledge building for the creation of knowledge. Moreover, Belvedere, which is a networked software system, was implemented to provide learners with shared workspaces for coordinating and recording their collaboration in scientific inquiry. In such environments, distributed expertise and multiple perspectives enable learners to accomplish tasks and develop understandings beyond the possibilities of one learner,, and it is therefore important for learners to collaborate with each other. However, little attention has been given to the technical support for inducing collaboration in distributed learning spaces connected via the internet. In computer supported cooperative work (CSCW), awareness is one of the most interesting topics, which can increase communication opportunities in distributed workspaces. Dourish and Bellotti (1992) defined awareness as “understanding of the activities of others, which provides a context for your own activity.” In CSCL (computer supported collaborative learning), Knowledge Awareness (KA) has been proposed to bridge learners who are interested in the same knowledge and to create effective collaboration in a distance learning environment (Ogata et al., 1996a; Ogata & Yano, 1997). KA provides information about other learners’ activities in a shared knowledge space. Sample messages could be, for instance: “Someone is looking at the same knowledge that you are looking at” or “Someone changed the knowledge that you have inputted.” These messages of KA encourage collaboration by exciting the learner’s curiosity and byactiviting the learning. Sharlok (Sharing, Linking and Looking-for Knowledge) has been developed as a testbed of the KA (Ogata & Yano, 1996b). Sharlok is a distributed collaborative learning environment, and it integrates a knowledge building tool with a collaborative interface tool. The system allows the learners: (1) to share their respective knowledge in its shared knowledge space, and to explore this knowledge space freely, (2) to make hypertext links between relevant knowledge, and (3) to collaborate about shared knowledge in an ad-hoc group in real time. Evaluation of Sharlok showed that KA encouraged collaboration by exciting learner’s curiosity and that KA effectively induced collaboration (Ogata & Yano, 1998). However, the problem arises that it is very difficult for learners to understand the relationships between the other learners and knowledge, because KA is provided by text messages. A large number of educational systems have been developed on the World Wide Web (WWW). Generally, educational facilities are provided within these systems and the most notable common funcion across all these systems is the facility to make available the presence and action of other

Supporting Awareness

175

users available across the WWW. A variety of systems are tackling this core problem in a number of different ways; for example, Palfreyman and Rodden (1996) developed an open awareness protocol for use with the WWW. Also see WebVis (Pitkow & Bharat, 1994), WAVE (Kent & Neuss, 1994), CGV (Girardin, 1995) and Footorints (Alan & Maes, 1997). This chapter proposes the use of a Knowledge Awareness Map (KA map) that visualizes KA information on the Web. The map helps the learner to mediate and recognize collaborators in the shared knowledge space. On this map, the Mediator Agent identifies learning companions who can help solving a problem. The characteristics of the map are: 1. Visualization of the hyperlinks and categorization of pages on the Web as educational materials, 2. Visualization of the relationships between pages and learners to induce collaboration, 3. Recommendations of appropriate collaborators on the KA map to help finding suitable partners. SharlokII is a prototype system for the KA map, it enables users to share individual knowledge and to learn by collaboration on WWW pages. This chapter is organized as follows. Section 2 describes what the awareness concept is and the past related works. The KA map is proposed in section 3 and section 4 is described the implementation of SharlokII. Finally, KA in a mobile learning environment is mentioned in section 5 and the concluding remarks are given in section 6.

2.

AWARENESS

In CSCW (computer supported cooperative work), a collaboration process consists of four processes (Malone, et al., 1994); co-presence, awareness, communication, and collaboration. Co-presence gives the user the feeling that s/he is in a shared workspace with someone else at the same time. Awareness is the process by which users recognize each other’s activities on the premise of co-presence, for example, “What are they doing?” or “Where are they working?”. The other processes explain how the user collaborates on the specific task with other users and accomplishes the task and common goals. Awareness is fundamental to social and collaborative activities. While observing people in real life situations, the ability to sense or become aware of others is the first step towards any kind of interaction. Failure to become aware of others before engaging in activities leads to confusion and is often the cause of accidents (mistakes). Awareness of others also enables communication with them, which in turn enables collaboration. Before

176

H. Ogata et al.

considering the particular case of the WWW, it is worthwhile looking at the notion of awareness in general settings. The concepts of awareness are classified into four categories: group, workspace, contextual, and peripheral awareness (Liechti, 2000). Note that these categories are not mutually exclusive, and that they are sometimes combined in certain systems.

2.1 Group Awareness The goal of group awareness is to convey some information about the state and activities of people in a team. This information does not have to be very precise: many tools only give a general idea about what is going on (e.g., “Are there people around?”, “Is it appropriate to contact them?”). One of the first group awareness tools was the Portholes system (Dourish and Bly, 1992) developed at Xerox PARC. Other media spaces (Bly et al., 1993) and more lightweight tools (Greenberg, 1996) also fit into this category. One issue raised by group awareness tools is privacy, since the activity of people has to be recorded and published. This problem has been discussed, and solutions have been proposed in (Hudson & Smith, 1996; Lee et al., 1997). While the CSCW literature obviously mainly focuses on workplace applications, group awareness systems have a huge potential in social settings (Prinz & Graether, 2000).

2.2 Workspace Awareness Workspace awareness is the up-to-the-minute knowledge about other learners’ interactions within a shared workspace. It applies both to shortterm tasks, performed synchronously, and long-term tasks, performed asynchronously. Gutwin et al. (1996), for example, implemented this awareness using GroupKit (Roseman & Greenberg, 1992), a system that supports workspace awareness using multi-user scrollbars. A framework for workspace awareness has been described in Gutwin et al. (1998).

2.3 Contextual Awareness Contextual awareness relates to different application domains, not only to CSCW. For instance, the idea of adapting the behavior of a service to the current ‘context’ plays an increasingly important role in ubiquitous computing systems (Salber et al., 1999). Groupware systems that support some form of awareness (e.g., group or workspace awareness) can in addition implement some form of contextual awareness. In this case, they can adapt their behavior to the current situation. This is essential to determine a) what

Supporting Awareness

177

information users should be made aware of, and b) how they should be made aware of it. Making appropriate decisions in these two cases helps to address the tradeoff between awareness and disturbance that is often encountered in awareness systems (Hudson & Smith, 1996).

2.4 Peripheral Awareness The notion of peripheral awareness denotes the way in which some systems present information to their users, i.e., without focusing on their attention. The notion of periphery is central to what Weiser and Brown (1996) describe as ‘calm’ technologies. Their definition for a calm technology is “one that will move easily from the periphery of our attention, to the center, and back”, and was illustrated with the “Dangling String”. AROMA (Pedersen, 1998) is a particularly interesting media space, because it was designed with peripheral awareness being the foremost goal. The system introduced the ideas of abstract representation of activity across multiple sites. AROMA does not rely on traditional audio or video links to convey explicit awareness information. Instead, the information gathered by sensors on the capture site (e.g., sound level, user identification) is abstracted and then communicated to the display site, where it is finally synthesized by output devices (e.g., screen, tangible artifacts). The Internet is not only a network of TCP/IP protocols, but also a community of people who communicate with each other. The WWW itself is becoming an activity space, of which people should be made more ‘aware’. For instance, information consumers should be able to ‘see’ and meet each other when they visit related places on the Web. Browsing the web as a social, collaborative or shared activity is more entertaining, productive and valuable than browsing alone. If we intend to increase the value of the internet for social and collaborative activities, we will need to continue to design, develop and explore the use of awareness in web applications. One key goal should be done is to develop a greater understanding of when and how much awareness is suitable for a given application, activity or situation. That is, the challenge of future computer systems is not to provide information anytime and anywhere, but to “say the right thing at the right time in the right way”. That can be done with the analysis of the user’s activities.

3.

KNOWLEDGE AWARENESS MAP

The KA map visualizes the strength of the relationships between the shared knowledge and the learners. The Mediator Agent of each learner

178

H. Ogata et al.

acquires and analyses the learner’s profile, then recommends a suitable partner for collaboration in the KA map.

3.1 What is knowledge awareness? KA is defined as awareness of the use of knowledge. In a distancelearning environment, it is very difficult for the learner to be aware of the use of other learners’ knowledge because the learner might not understand their actions in the remote site beyond the Internet. KA messages inform a learner about the other learners’ real-time or past-time actions: for example, when a learner looks at, changes, and discusses knowledge with which another learner was or is presently engaged. KA messages are, for example, “Someone is changing the same knowledge that you are looking at” or “Someone discussed the knowledge which you have inputted.” These messages make the learner aware of someone 1. who has the same problem or knowledge as the learner; 2. who has a different view about the problem or knowledge; 3. who has potential to assist solving the problem. Therefore, these messages which are independent of the domain, can enhance collaboration opportunities in a shared knowledge space, and make it possible to shift from solitary learning to collaborative learning in a distributed learning space. KA messages are classified along two dimensions (figure 1): time and knowledge separation. KA of the same time type (ST) informs the learner that other learners are doing something at the same time the learner is using the system. By using learners’ past actions, KA of the different time type (DT) provides encounters beyond time. Examples of KA of the same knowledge type (SK) include messages about other learners’ activities regarding the same knowledge that the learner is looking at, e.g., discussing or changing the knowledge. This type is useful for learners in findding partners who have the same problem or knowledge. KA of the different knowledge type (DK) enhances collaboration possibility with another learner (1) who has something to do with the learner’s interests; or (2) who has different expertise from the learner’s interests. In this way, KA facilitates peer review of the shared knowledge. For example, the message of the STSK type “Who is looking at the knowledge?” shows the learners who are looking at the knowledge that the user is also looking at. By this message, the user may start to discuss the knowledge together. Likewise, the message of the DTSK type “Who changed the knowledge since I have looked at?” facilitates the start of discussion on the knowledge transformation. Moreover, the STDK message

Supporting Awareness

179

“What knowledge are they discussing?” is useful for the learner in deciding whether or not to join the discussion. Same knowledge Who looked at the knowledge? Who changed the knowledge? Who discussed the knowledge?

Who is looking at the knowledge? Who is changing the knowledge? Who is discussing the knowledge?

Different time

DTSK

STSK

Same time

(Asynchronous)

DTDK

STDK

(Synchronous)

What knowledge did they look at? What knowledge did they change? What knowledge did they discuss?

What knowledge are they looking at? What knowledge are they changing? What knowledge are they discussing?

Different knowledge

Figure1. Classification of knowledge awareness.

KA is related to the learner’s curiosity. Hatano and Inagaki (1973) identified two types of curiosity: particular curiosity (PC) and extensive curiosity (EC). EC occurs when there is a desire for learning and makes the learner’s stock of knowledge well balanced by widening the learner’s interests. PC is generated by the lack of sufficient knowledge, and it is very useful for the learner because he can acquire more detailed knowledge. KA of the SK type excites PC, and KA of the DK type satisfies EC. For example, a STDK message stirs up the learner’s EC by attracting the learner to the particular knowledge when the learner focuses on nothing. Moreover, the STDK message about the knowledge leads the learner to the collaboration by arousing the learner’s PC. In this way, KA induces collaboration by exciting the learner’s curiosity.

3.2 Knowledge Awareness map In our previous research, Sharlok presented KA information as a text message. From this message, however, it is very difficult for the learner to understand in which way the other learners are interested in the knowledge. Another learner may be a vital helper who can assist the learner to understand the knowledge deeply, or just looking at the knowledge. Therefore, we propose the Knowledge Awareness Map that graphically displays KA information. This map provides the learner with a clear grasp of other learners working on the knowledge. With this, the learner can seek the discussion companions interactively. To build a KA map, we propose the Mediator Agent (MA) which supports learners in finding suitable collaborators concerning the knowledge that is in focus.

180

H. Ogata et al.

As for the visualization of the WWW space, an awareness support system (Palfreyman & Rodden, 1996) was developed in order to display the links between WWW pages and the users who are accessing the pages, in both 2D and 3D representation. This system displays to the user information about the links of the pages, and who is referring those pages in real-time. In addition, the KA map shows the degree of the learner’s interests and recommends adequate collaborators.

3.3 Mediator Agent (MA) The Mediator Agent bridges a learner who has a question and appropriate companions for creating effective collaboration. The information that the MA deals with is shown below: 1. Link information: This is a link between web pages as a shared knowledge; 2. KA information: This is information about the current actions of learners, e.g., a learner is logged-in and looking at some knowledge; and the degree of the learners’ interests, e.g., the learner is often looking at and discussing some knowledge; 3. Discussion information: This is information about the participants of discussions. KA information depends on a learner’s actions during the learning. For providing the KA map, learners’ actions can be classified into the following: 1. Exploration on the web: The learner is looking at a web page or looking for a page. 2. Requesting collaboration: Learner is asking to join a discussion. 3. Being requested for collaboration: Learner decides whether he or she accepts a request for discussion. 4. Discussing: Learner is communicating with other learners. 5. Idle: This state means that the learner is doing nothing. The MA manages and provides information about whether or not a learner is using the same knowledge as other learners. Moreover, it watches and stores the current actions of the learner into his or her profile, and the MA detects the degree of understanding and interest of the knowledge using the learner’s profile.

3.4 Learner’s profile The MA collects learner’s profile applying two techniques: 1. The action log of the learner: e.g., access times to WWW pages;

Supporting Awareness

181

2. The explicit registration by the learner. The actions of a learner in a web-based learning environment can be classified as following: (A) creating a category, (B) creating knowledge, (C) creating links to WWW pages, (D) asking a question, (E) answering a question, (F) modifying knowledge, (G) participating in discussion, and (H) looking at knowledge. These eight actions of a learner are used to generate the learner’s profile. However, it is difficult to detect the knowledge and the interest of the learner from learner’s actions only. Therefore, it is necessary that the learner supplies his or her interests about the knowledge. SharlokII implements the registration of the interests of the learner using footprints. A footprint is an explicit flag that shows the learner's interest on shared knowledge.

3.5 Strategy for Recommending Peer Learners When the learner asks a question and looks for a helper, the MA recommends one to three persons. The types of the participant in the collaboration are shown below: 1. Questioner: This learner often asks questions and requires collaboration. 2. Answerer: This learner is often requested to start or to join discussions. 3. Collaborator: This learner often joins the discussion halfway. S/he might easily start collaboration. The MA recommends answerers who can help solving a problem; and some collaborators to a questioner. It selects a learner using the following information: 1. The login situation of learners: Because of real-time discussion, the MA selects only logged users as collaborative candidates. 2. The footprints of each learner: Participants are selected from those whose footprint is set in the knowledge (page) of the question. 3. The profile of each learner: Although the profile consists of the number of actions done on the knowledge, the MA has to evaluate it according toa total scale. If the total of (A)-(D) actions of a learner is larger than that of (E)-(H) actions, then MA considers the learner as an answerer. Otherwise, the learner is a collaborator. The larger the total of a learner’s actions, the more the learner is preferred to join into the collaboration. 4. The current action of a learner: The MA gives high priority to learners who are doing nothing (idle) in the learning environment. This

182

H. Ogata et al.

consideration activates passive learners by stimulating their intellectual curiosity. This research proposes the level of interest (LOI) as follows:

LOI = {F +

⎫ 1 The number of the learner' s actions to the page ⎬× The max number of other learner' s actions to the page ⎭ 2

Variable F shows the footprint given by learner. The value of F is 1 if the learner has set a footprint on the page and 0 otherwise. The range of LOI is from 0.00 to 1.00.

3.6 Visualization of KA A link in the KA map shows the relationship between web pages and learners. The length (L) of a link means the strength of the relationship which is calculated by the following equation:

L = D (2 - LOI ) where D is a default value of link length. The range of L is from D to 2D. If a learner is very interested in a page, then the link length (L) will be short and close to D.

4. IMPLEMENTATION OF KNOWLEDGE AWARENESS MAP IN SHARLOKII SharlokII is a web-based collaborative learning environment, which consists of a shared knowledge space and a collaborative learning environment. SharlokII has been developed with Java1.2 and Perl5.0 on a WWW Server.

4.1 A Web-Based Collaborative Learning Environment This chapter focuses on the WWW as a place where the above environment can be realized. Moreover, the WWW provides a lot of rich

Supporting Awareness

183

information that many learners can easily access. The characteristics of the SharlokII are shown below. 1. Learners can participate in learning with SharlokII at any time and any place though web. 2. The shared knowledge space of learners is extensible to the web. 3. Learners collaborate about the shared knowledge, and each learner can access the collaboration results via web.

4.2 System Configuration Figure 2 shows the system architecture of SharlokII. The system consists of a server and a client. The server includes a WWW server, the mediator module, a map server and four data categories. The shared knowledge space stores HTML (Hyper Text Markup Language) formatted files as learning materials. Learner’s actions, e.g., clicking a link on a web page, are saved into the history database of the learner’s activities. The Learner’s profile stores personal information, e.g., major field of study. The map data file is used for data exchange between the client and the server. Client Web browser

Server WWW Server Shared Knowledge Space

UI

Mediator module Knowledge Awareness Map

Applet

Map Client

Map Server

History of Learners' Activities Learners' Profile Map Data

Figure 2. System configuration.

4.3 Collaborative Learning Environment SharlokII provides a shared knowledge space, where learners provide and share their own knowledge. They collaboratively collect and use the knowledge on WWW pages in order to solve problems. When collecting available pages into the shared knowledge space, users have to appropriately classify pages into the appropriate categories that their teachers have created. These categories help learners to find the desired information. SharlokII also facilitates collaboration between learning about the shared knowledge.

184

H. Ogata et al.

Synchronous collaboration can be realized with a chat tool, and asynchronous collaboration can be performed with email and a bulletin board system. The learner solves problems and to acquires new knowledge through collaboration, and the results are accumulated into the shared knowledge, accessible for learners. Questions about technological engineering fundamental information were stored as initial data. There are some categories such as data structure, algorism, flow chart, program languages, etc. For example, a question is “please show the decimal number given by the hex number 0.248.” The learners collect and share the related web pages and work it individually. However, if s/he cannot answer it, s/he tries to find peer learner and work out the solution to the question with other learners. After answering the question, s/he may receive the KA information if someone answers it in different way or if someone gives a wrong answer.

Figure 3. Collaborative learning environment in SharlokII.

The interface of the collaborative learning environment of SharlokII is shown in figure 3. Screen (a) is the main window of SharlokII. The learner

Supporting Awareness

185

can access knowledge (web page) in the right frame of window (a). The left frame of window shows the user’s current status and selection of categories in SharlokII, e.g., the message is that “you selected programming using Perl language category.” The right frame shows the pages about Perl. For example, the user can see the question if s/he selects the link “execution control of Perl”. Also there is a discussion log about this question. Window (b) automatically appears if the user selects “execution control of Perl”. The discussion can be requested by clicking the ‘start collab.’ button. Then, window (c) is shown, where the MA recommends suitable collaborators for the question, and the learner can also arrange other learners who want to join into the discussion. The request status is displayed on the window (d). If the other user agrees to start discussion, then the discussion window will open. After the discussion, the log can be stored as a new web page, which is similar to the web page and can start collaboration. In this case, the log will be linked to “execution control of Perl” web page.

Figure 4. Knowledge awareness map.

4.4 The KA Map The KA map is shown in figure 4. The map is displayed when opening SharlokII. For example, when the user ‘imai’ is looking at the page (X) “Processor”, window (a) appears automatically. In the map of window (a), the rectangles represent the web pages categories, e.g., “Memory” and “Computer”; the diamonds represent web pages belonging to the same

186

H. Ogata et al.

category, e.g., “Computer” in the map (a); and the circles represent learners who are accessing or have accessed the same page “Processor” before. The circle colour shows either the user is logging into the system, or not. Moreover, when the user wants to request discussion, the MA categorizes learners into four types of learners; questioners, answerers, collaborators and others. In map (b), the user “sharlok” is an answerer and “ogata” is a collaborator. The length of the line between a user and a page is given by the equation L, which means the strength of the learner’s interest in the page. It means that, the shorter the length between the user and the page, the more interesting the page is to the user. For example, when the user looks for learning companions about a topic, the user can use this map to find peer learners who have strong similar interests. User can also use this graphical map to discover appropriate peers who can answer their questions. Moreover, when receiving an invitation for discussion from another learner, the user can learn about the backgrounds of the inviting learner. By clicking a person node, both the personal information of the learner and his or her past actions about peripheral knowledge, are displayed.

5.

MORE RECENT WORK

The research on ubiquitous computing (Abowd & Mynatt, 2000) has recently been accelerated by improved wireless telecommunications capabilities, open networks, mobile devices, continuous increase in computing power, improved battery technology, and the emergence of flexible software architectures (Lyytinen & Yoo, 2002). Therefore, we are investigating on how those technologies can support collaborative learning in the daily life. Especially, we are focusing on language learning because language is strongly influenced by situations. Also KA map has been implemented in a mobile CSCL system, which is called CLUE (Collaborative-Learning support-system with a Ubiquitous Environment) (Ogata & Yano, 2004a, 2004b, 2005).

Supporting Awareness

187

Figure 5. Screenshots of CLUE.

Interface of the collaborative learning environment of CLUE is shown in figure 5. The map window (B) shows the current location of each learner who has a PDA with GPS. The face icon on the map means the learning status of each learner. For example, if a learner has a problem or question, the face turns into a fad one. By clicking the face icon, it is possible to send a message to the learner corresponding to the icon. In addition, a rectangle icon on the map shows a landmark where a teacher or a learner gives some expressions, or where they communicate with each other. If a learner enters an expression at one place for the first time, then a new landmark is created in the map. By cling the rectangle icon, the user can see the web page of the place (e.g., the hospital), the expressions that are used in the place, or the communication logs about either the expressions or the place. Users can also register their positions at any time if GPS does not work. For example, it might come out when big buildings surround them, or when they are inside a building.

188

H. Ogata et al.

If the learner approaches certain place, the window (C) will appear, which shows a useful expression for that place. If the current user has already learnt all the expressions for that place, the expressions do not appear. If the learner can correctly answer the questions given by other learners or teachers, another question will appear. Otherwise, the learner will be given the same expression at the next time when he/she comes to the place later. If the learner has a question about the expressions, the window (D) will appear. It shows the relation between expressions and other learners. The color of an oval icon shows the level of difficulty given by a teacher for one expression. Moreover, the color of a rectangle icon shows the level of proficiency of the learner. The more correct-answers the learner gives, the higher is the level. From this KA map, the learner can find a suitable person to ask the question. If the learner has a question about one place, the window (E) shows other people who have visited it, and the window (F) shows the relation between people and all the places on the map. The initial experimentation was conducted with six overseas students to learn Japanese language. 89 sentences were given related to some places around the University, where outside wireless antennas were established. Each student walked during a week through the campus with a PDA with a wireless LAN and a GPS. As shown in figure 5(A), the learner from France on the left side entered a ticket shop in the University in order to buy a bus ticket. Then, in window (C) CLUE automatically showed him some Japanese expressions that mean “do you have a ticket for Osaka? I would like to buy a one way ticket.” The learner read the expressions, and could smoothly communicate with the shop clerk. CLUE was very useful because most of the shop clerks and the office staffs in the university cannot understand English conversation easily and they hesitate to try to speak English. But it is not so easy for foreigners to speak Japanese so proficiently. In such a situation, CLUE helps “learning in the real world.” After a while, the learner can review and brush up the expressions that the learner used before. In terms of KA map, most of the users stated that KA could be provided in the appropriate way. One of the learners commented that KA map is easy enough to understand. Through discussions that are created using KA map, users were able to teach and learn from each other, and most learners replied that they had a feeling of achievement and more interests in learning Japanese with CLUE. After one week from the experimentation, some users commented out “it was very useful for me to be provided the useful expression at the current location with PDA.” This comment is simple, but it seems to indicate the effectiveness of CLUE. Moreover, some users said that CLUE helped to link

Supporting Awareness

189

the expressions and the corresponding spots. As for KA map, they commented that KA map was very helpful to find a peer helper.

6.

SUMMARY

This article proposed the KA map that supports learners to find appropriate companions for their problem solving in distributed collaborative learning environments. Also, the Mediator Agent was proposed for recommendation of suitable partners. To evaluate the KA map and MA, SharlokII was developed on the WWW. The evaluation showed the following results: 1. It seemed to be possible to discuss with the appropriate companion by KA map. 2. With the KA support, the learner could perform lively discussion. 3. Most of the learners remembered the contents of the discussions that were created by KA map. In addition, KA map has been developed for mobile language learning. KA map could be useful in mobile language learning contexts rather than in web-based learning because knowledge for language changes often over time. KA map might facilitate to collect and refine dynamically-changing knowledge through peer reviews

ACKNOWLEDGEMENTS This contribution is a revised version of an article previously discussed in a workshop (NTCL’2000, held in Awaji-island, Japan, Nov., 2000). We thank all the participants of that, reviewers, and all the colleagues in the Yano laboratory, the University of Tokushima. This work was supported by the Grant-in-Aid for Scientific Research No. 11780125 and No. 12558011 from the Ministry of Education, Science, Sports and Culture in Japan, the Okawa Foundation for information and telecommunications.

REFERENCES Abowd, G.D., & Mynatt, E.D. (2000). Charting Past, Present, and Future Research in Ubiquitous Computing, ACM Transaction on Computer-Human Interaction, Vol. 7, No.1, 29-58.

190

H. Ogata et al.

Alan, W. & Maes, P. (1999). Footprints: history-rich tools for information foraging, Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, 270-277. Bly, S., Harrison, S. & Irwin, S. (1993). Media Space: Bringing People Together in a Video, Audio and Computing Environment, Communications of the ACM, 28-47. Dourish, P. & Bellotti, V. (1992). Awareness and coordination in shared workspaces, Proceedings of Computer Supported Cooperative Work ‘92, ACM Press, 107-114. Dourish, P. & Bly, S. (1992). Portholes: Supporting Awareness in a Distributed Work Group. Proceedings of the ACM SIGCHI, Conference on Human Factors in Computing Systems (CHI'92), 541-547. Edelson, D., Pea, R., & Gomez, L. (1996). The collaboratory notebook, Communication of the ACM, Vol.39, No.4, 32-33. Girardin, L. (1995). Cyberspace geography visualization, http://www.girardin.org/luc// cgv/report Greenberg, S. (1996). Peepholes: Low Cost Awareness of One's Community. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, 206-207. Gutwin, C., Greenberg, S. & Roseman, M. (1996). Workspace awareness support with radar views. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, 210-211. Gutwin, C. & Greenberg, S. (1998). Design for individuals, design for groups: tradeoffs between power and workspace awareness. Proceedings of the ACM Conference on Computer supported cooperative work (CSCW'98), 207-216. Guzdial, M., Hmelo, C. Hübscher, R., Nagel, K., Newstetter, W., Puntembakar, S., Shabo, A., Turnst, J. & Kolodner, J. (1997). Integrating and Guiding Collaboration: Lessons learned in computer-supported collaboration learning research at Georgia Tech. Proceedings of Computer Supported Collaborative Learning. 91-100. Hatano, G. & Inagaki, K. (1973). Intellectual curiosity, Chuo Koronsya Press. (in Japanese) Hudson, S.E. & I. Smith. (1996). Techniques for Addressing Fundamental Privacy and Disruption Tradeoffs in Awareness Support Systems. Proceedings of the ACM Conference on Computer supported cooperative work (CSCW’96), Cambridge, MA. ACM Press, 248257. Kent R., & Neuss, C. (1994). Creating a Web Analysis and Visualization Environment, Computer Networks and ISDN Systems, Vol. 28, 109- 117. Liechti, O. (2000). Awareness and the WWW: an Overview, in proceedings of the International Workshop on Awareness and the WWW, ACM Proceedings of CSCW’2000, 3-12. Linn, M. (1996). Key to the information highway, Communication of the ACM, Vol. 39, No. 4, 34-35. Lee, A., Girgensohn, A. & Schlueter, K. (1997). NYNEX portholes initial user reactions and redesign implications. Proceedings of the Conference on Supporting Group Work, Phoenix, AZ. ACM Press. 385-394. Lyytinen, K. & Yoo, Y. (2002). Issues and Challenges in Ubiquitous Computing, Communications of ACM, Vol. 45, No. 12, 63-65. Malone, T. & Crowson, K. (1994). The interdisciplinary study of coordination, ACM Computing Surveys, Vol. 26, No. 1, 87-119. Ogata, H., Matsuura, K. & Yano, Y. (1996a). Knowledge awareness: Bridging between shared knowledge space and collaboration in Sharlok, Proceedings of Educational Telecommunications ‘96, 232-237.

Supporting Awareness

191

Ogata, H., Matsuura, K. & Yano, Y. (1996b). Sharlok: Bridging learners through active knowledge awareness, Proceedings of IEEE Systems Man & Cybernetics, Vol.1, 601-601. Ogata, H., & Yano, Y. (1997). Knowledge Awareness Filtering: toward Efficient Collaborative Learning, Proceedings of Artificial Intelligence in Education, 207-214. Ogata, H., & Yano, Y. (1998). Knowledge Awareness: Bridging learners in a Collaborative Learning Environment, International Journal of Educational Telecommunications, AACE, Vol. 4, No. 2/3, 219-236. Ogata, H., Liu, Y., Ochi, Y., & Yano, Y. (2000). Agent-Mediated Language-Learning Environment Based on Communicative Gaps, in Gauthier, G., Frasson, C., VanLehn, K. (Eds.), LNCS1839, Intelligent Tutoring Systems, Springer, 454-463. Ogata, H., & Yano, Y. (2004a). Knowledge Awareness Map for Computer-Supported Ubiquitous Language-Learning, Proceedings of IEEE WMTE2004 (International Workshop on Wireless and Mobile Technologies in Education), 19-26. Ogata, H., & Yano, Y. (2004b). Context-Aware Support for Computer Supported Ubiquitous Learning, Proceedings of IEEE WMTE2004 (International Workshop on Wireless and Mobile Technologies in Education), 27-34. Ogata, H. & Yano, Y. (2005). Knowledge awareness for a computer-assisted language learning using handhelds, International Journal of Continuous Engineering Education and Lifelong Learning, Vol. 14, Nos. 4/5, 435-449. O’Malley, C. (1994). Computer supported collaborative learning, NATO ASI Series, F: Computer & Systems Sciences, Vol. 128. Palfreyman, K., & Rodden, T. (1996). A Protocol for User Awareness on the World Wide Web, Proc. of CSCW’96, 130-139. Pitkow, J.E. & Bharat, K.A (1994). Webviz: A tool for WWW access log analysis, Proc. of WWW. 271-277. Pedersen, E.R. (1998). People Presence or Room Activity: Supporting Peripheral Awareness over Distance. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI’98), 283-284. Prinz, W. & Graether, W. (2000). The Social Web Cockpit: Awareness and Mobile Portal for Web-Communities. Proceedings of the International Workshop on Awareness and the WWW, ACM CSCW’2000 Conference, 34-40. Roseman, M. & Greenberg, S. (1992). GroupKit: A groupware toolkit for building real-time conferencing applications, Proc. of CSCW ‘92, 43-50. Salber, D., Dey, A.K. & Abowd, G.D. (1999). The Context Toolkit: Aiding the Development of Context-Enabled Applications. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI’99), 434-441. Scardamalia, M., & Bereiter, C. (1996). Student communications for the advancement of knowledge, Communication of the ACM, Vol. 39, No. 4, 36-37. Slavin, R. (1990). Cooperative Learning: Theory, Research and Practice, Allyn and Bacon. Suthers, D. & Jones, D. (1997). An Architecture for Intelligent Collaborative Educational Systems. Proceedings of 8th World Conference on Artificial Intelligence in Education (AIED’97), 55-62. Webb, N. & Palincsar, A. (1996). Group processes in the classroom, In Berliner, D & Calfee, R, (eds.), Handbook of Educational Psychology, Macmillan, 841-873. Weiser, M. & Brown, J.S. (1996). Designing Calm Technology. Power Grid Journal, Version 1.01. Available at http://nano.xerox.com/hypertext/weiser/calmtech/calmtech.htm

INDEX

action-based analysis, 7 activity theory, 5 ambiguity, 17 breakdowns, ii, 15, 18, 48, 49, 50, 51, 54, 57, 58, 59, 83, 86, 89 co-construction, 3, 8, 121, 144, 149, 156 cognitive dissonance, 89 communication scenarios, iii, 155, 156, 157, 158, 160, 163, 171 complexity, 4, 17, 74, 170 computer integrated classroom, 121, 122 constructionism, 157 contextual awareness, 176 cooperation scripts, 15 cooperative learning, ii, 15, 21, 22, 23, 25, 26, 29, 32, 34, 35, 37, 41, 42, 43, 44, 45, 46, 84, 111 critiquing system, 15, 47, 50, 54, 56 CSCW, 93, 174, 175, 176, 190, 191 curiosity, 119, 174, 179, 182, 190 Decision Trees, 63, 71, 83 diagnosis, 14, 64 distance learning, 7, 83, 88, 111, 120, 139, 141, 148, 174 educational multimedia, 3 external representations, 3, 8, 151 face-to-face, 3, 14, 57, 117, 118, 156, 157 fine-grained sequential analysis, 63, 83 group awareness, 3, 7, 109, 176 group composition, 14, 18, 20

group formation, 15, 21, 23, 28, 36, 37, 38, 39, 40, 41 groupware, 3, 7, 191 Hidden Markov Model HMM, 63, 64, 72, 73, 78, 79, 80, 81, 83, 86 HTML, 22, 33, 37, 135, 183 intelligent tutoring system ITS, 3, 64, 107, 108 interaction analysis, 67, 112 interactive visualization of associations, 58 internal group interaction, 14, 15 ITS, 5, 24, 45, 110, 111, 138 knowledge awareness, 119, 174, 175, 177, 179, 182, 191 knowledge construction, 16, 48, 49, 161 knowledge representation, 64, 69 learning communities learning community, 2, 18 learning companion, 175, 186 learning flow, 1, 44 learning objects, 8 lifelong learning, 15, 21, 23, 32, 48 machine learning, 17, 83, 85, 88 matchmaker, 128, 134, 138 metacognitive learning, 149 mobile CSCL, 186 motivation, 66 multiple perspectives, 28, 63, 174

194

Index

multi-user tools multiuser tool, 3 object modeling technique OMT, 76 pedagogical agent, 93 peripheral awareness, 176, 177 personal coach, 18, 92, 93, 108, 110 plan recognition, 63, 71, 83 points of cooperation, 15, 41 roomware, 121, 123 scaffolding, 3, 7, 132 sentence openers, 17, 68, 70, 76, 80, 89 social constructivism, 5 social constructivists, 66 social interoperability, 3, 6 social technologies, 2

social technology, 2 socio-cognitive conflict, 20, 89 tele-presence, 4 text to speech, 129, 134 ubiquitous computing, 3, 9, 121, 122, 138, 176, 186 uncertainty, 19, 92 virtual experiment, 145, 146 visualization, 130, 175, 182, 190 web-based course, ii, 21, 22, 42, 43 web-based training WBT, 21, 26, 30, 31, 42 workspace awareness, 176, 190 XML, 9, 134 zone of proximal development ZPD, 89

COMPUTER-SUPPORTED COLLABORATIVE LEARNING

1. 2. 3. 4. 5. 6. 7. 8. 9.

Arguing to Learn J. Andriessen, M. Baker, D. Suthers (eds.) ISBN HB 1-4020-1382-5 Designing for Change in Networked Learning Environment B. Wasson, S. Ludvigsen, H.U. Hoppe (eds.) ISBN HB 1-4020-1383-3 What We Know About CSCL J.-W. Strijbos, P.A. Kirschner, R.L. Martens (eds.) ISBN HB 1-4020-7779-3 Advances in Research on Networked Learning P. Goodyear et al. (eds.) ISBN HB 1-4020-7841-2 Barriers and Biases in Computer-Mediated Knowledge Communication: And How They May Be Overcome R. Bromme, F.W. Hesse, H. Spada (eds.) ISBN HB 0-387-24317-8 Scripting Computer-Supported Collaborative Learning F. Fischer, I. Kollar, H. Mandl, J.M. Haake ISBN HB 978-0-387-36947-1 Dialogic, Education and Technology: Expanding the Space of Learning Rupert Wegerif ISBN HB 978-0-387-71140-9 The Teacher’s Role in Implementing Cooperative Learning in the Classroom R.M. Gillies, A.F. Ashman, J. Terwel ISBN HB 978-0-387-70891-1 The Role of Technology in CSCL: Studies in Technology Enhanced Collaborative Learning H.U. Hoppe, H. Ogata, and A. Soller ISBN HB 978-0-387-71135-5