Solutions and Innovations in Web-Based Technologies for Augmented Learning: Improved Platforms, Tools, and Applications Nikos Karacapilidis University of Patras, Greece
Information science reference Hershey • New York
Director of Editorial Content: Director of Production: Managing Editor: Assistant Managing Editor: Typesetter: Cover Design: Printed at:
Kristin Klinger Jennifer Neidig Jamie Snavely Carole Coulson Larissa Vinci Lisa Tosheff Yurchak Printing Inc.
Published in the United States of America by Information Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue, Suite 200 Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail:
[email protected] Web site: http://www.igi-global.com and in the United Kingdom by Information Science Reference (an imprint of IGI Global) 3 Henrietta Street Covent Garden London WC2E 8LU Tel: 44 20 7240 0856 Fax: 44 20 7379 0609 Web site: http://www.eurospanbookstore.com Copyright © 2009 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identi.cation purposes only . Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark. Library of Congress Cataloging-in-Publication Data Solutions and innovations in web-based technologies for augmented learning : improved platforms, tools, and applications / Nikos Karacapilidis, editor. p. cm. -- (Advances in web-based learning) Includes bibliographical references and index. Summary: "This book covers a wide range of the most current research in the development of innovative web-based learning solutions, specifically facilitating and augmenting learning in diverse contemporary organizational settings"--Provided by publisher. ISBN 978-1-60566-238-1 (hardcover) -- ISBN 978-1-60566-239-8 (ebook) 1. Education--Computer network resources. 2. Internet in education. 3. Organizational learning--Computer assisted instruction. 4. Distance education. I. Karacapilidis, Nikos. LB1044.87.S619 2009 371.33'44678--dc22 2008023189 British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book set is original material. The views expressed in this book are those of the authors, but not necessarily of the publisher. If a library purchased a print copy of this publication, please go to http://www.igi-global.com/agreement for information on activating the library's complimentary electronic access to this publication.
Advances in Web-based Learning Series (AWBL) ISBN: 1935-3669
Editor-in-Chief: Nikos Karacapilidis, University of Patras, Greece Web-Based Education and Pedagogical Technologies: Solutions for Learning Applications Liliane Esnault; EM Lyon, France
IGI Publishing • copyright 2008 • 364 pp •H/C (ISBN: 978-1-59904-525-2) • US $89.95 (our price) The rapid development and expansion of Web-based technologies has vast potential implications for the processes of teaching and learning world-wide. Technological advancements of Web-based applications strike at the base of the education spectrum; however, the scope of experimentation and discussion on this topic has continuously been narrow. Web-Based Education and Pedagogical Technologies: Solutions for Learning Applications provides cutting-edge research on such topics as network learning, e-learning, managing Web-based learning and teaching technologies, and building Web-based learning communities. This innovative book provides researchers, practitioners, and decision makers in the .eld of education with essential, up-to-date research in designing more effective learning systems and scenarios using Web-based technologies.
Solutions and Innovations in Web-Based Technologies for Augmented Learning: Improved Platforms, Tools, and Applications
Nikos Karacapilidis, University of Patras, Greece
Information Science Reference • copyright 2009 • 374 pp •H/C (ISBN: 978-1-60566-238-1) • US $195.00 (our price) The proper exploitation of Web-based technologies towards building responsive environments that motivate, engage, and inspire learners, and which are embedded in the business processes and human resources management systems of organizations, is highly critical. Accordingly, the research field of technology-enhanced learning continues to receive increasing attention. Solutions and Innovations in Web-Based Technologies for Augmented Learning: Improved Platforms, Tools, and Applications provides cutting-edge research on a series of related topics and discusses implications in the modern era’s broad learning concept. Addressing diverse conceptual, social, and technical issues, this book provides professionals, researchers, and practitioners in the field with up-to-date research in developing innovative and more effective learning systems by using Web-based technologies.
The Advances in Web-based Learning (AWBL) Book Series aims at providing an in-depth coverage and understanding of diverse issues related to the application of web-based technologies for facilitating and augmenting learning in contemporary organizational settings. The issues covered address the technical, pedagogical, cognitive, social, cultural and managerial perspectives of the Webbased Learning research domain. The Advances in Web-based Learning (AWBL) Book Series endeavors to broaden the overall body of knowledge regarding the above issues, thus assisting researchers, educators and practitioners to devise innovative Web-based Learning solutions. Much attention will be also given to the identification and thorough exploration of good practices in developing, integrating, delivering and evaluating the impact of Web-based Learning solutions. The series intends to supply a stage for emerging research in the critical areas of web-based learning to further expand to importance of comprehensive publications on these topics of global importance.
Hershey • New York Order online at www.igi-global.com or call 717-533-8845 x100– Mon-Fri 8:30 am - 5:00 pm (est) or fax 24 hours a day 717-533-7115
Associate Editor Liliane Esnault, E.M.LYON, France
Editorial Review Board Agostinho Rosa, Technical University of Lisbon, UK Amaury Daele, University of Fribourg, Switzerland Amita G. Chin, Virginia Commonwealth University, USA Andy Koronios, University of South Australia, Australia Anil Aggarwal, University of Baltimore, USA Anthony Norcio, University of Maryland at Baltimore County, USA Antonio Cartelli, University of Cassino, Italy Brian Corbitt, Deakin University, Australia Carol Lerch, Daniel Webster College, Nashua, NH, USA Cesar Alberto Collazos O., University of Cauca, Popayán, Colombia Cesar Garita, University of Amsterdam, The Netherlands Chris Zhang, University of Saskatchewan, Canada Colin McCormack, University College Cork, Ireland Danièle Herrin, Univeristé de Montpellier, Montpellier, France David McConnell, Lancaster University, UK David Taniar, Monash University, Australia Dimosthenis Anagnostopoulos, Harokopio University of Athens, Greece Emmanuel Fernandes, Université de Lausanne, Suisse Eugenia Ng, Hong Kong Institute of Education, China Fuhua Oscar Lin, Athabasca University, Canada George Ghinea, Brunel University, UK Gord McCalla, University of Saskatchewan, Saskatoon, Canada Holly Yu, California State University, USA Ikuo Kitagaki, Hiroshima University, Japan Jan Frick, Stavanger University College, Norway Janine Schmidt, McGill University, Canada John Lim, National University of Singapore, Singapore Katia Passerini, New Jersey Institute of Technology, USA Katy Campbell, University of Alberta, Canada Khaled Wahba, Cairo University, Egypt
Larbi Esmahi, Athabasca University, Canada Li Yang, Western Michigan University, USA Mahesh S. Raisinghani, Texas Woman’s University, USA Mara Nikolaidou, University of Athens, Greece Maria Manuela Cunha, Polytechnic Institute of Cavado, Portugal Marisa Ponti, IT Universitetet, Sweden Martin Crossland, Oklahoma State University, Tulsa, USA Martin Gaedke, Universität Karlsruhe, Germany Mehdi Ghods, The Boeing Company, USA Miguel-Angel Sicilia, University of Alcalá, Spain MinnieYi-Min Yen, University of Anchorage, Alaska Moez Limayem, University of Arkansas, USA Murali Shanker, Kent State University, USA MV Ramakrishna, Monash University, Australia Nikos Karacapilidis, University of Patras, Greece Patrice Sargenti, University of Monaco, Monaco Patrick van Bommel, University of Nijmegen, The Netherlands Philippe Koch, IBM Learning Solutions, France R. Subramaniam, Nanyang Technological University, Singapore Romain Zeiliger, Gate-CNRS, Ecully, France Ron Vyhmeister, Adventist International Institution of Advanced Studies, Philippines Roy Rada, University of Maryland at Baltimore County, USA Sergio Lujan-Mora, Escuela Politecnica Superior IV Universidad de Alicante, Spain Sherry Y. Chen, Brunel University, UK Sree Nilakanta, Iowa State University, USA Sue Tickner, University of Glasgow, UK Tak-ming Law, Institute of Vocational Education, HK Tang Changjie, Sichuan University, China Tanya McGill, Murdoch University, Australia Terry Ryan, Claremont Graduate University, USA Timothy Shih, Tamkang University, Taiwan Vivien Hodgson, University of Lancaster, UK VP Kochikar, Infosys Technologies, India Werner Beuschel, University of Applied Science, Brandenburg, Germany Witold Abramowicz, Poznan University of Economics, Poland Yair Levy, Nova Southeastern University, USA
Table of Contents
Preface.................................................................................................................................................. xix Section I Augmenting Learning Chapter I The Role of Learner in an Online Community of Inquiry: Responding to the Challenges of First-Time Online Learners................................................................................................................. 1 Martha Cleveland-Innes, Athabasca University, Canada Randy Garrison, The University of Calgary, Canada Ellen Kinsel, Odyssey Learning Systems, Canada Chapter II Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning................................................................................................................................................. 15 Xinchun Wang, California State University, Fresno, USA Chapter III Cognition, Technology, and Performance: The Role of Course Management Systems........................ 35 Teresa Lang, Columbus State University, USA Dianne Hall, Auburn University, USA Chapter IV The Role of Organizational, Environmental and Human Factors in E-Learning Diffusion................................................................................................................................................ 53 Kholerile L. Gwebu, University of New Hampshire, USA Jing Wang, Kent State University, USA Chapter V Distance Education: Satisfaction and Success....................................................................................... 71 Wm. Benjamin Martz, Jr., Northern Kentucky University, USA Morgan Shepherd, University of Colorado at Colorado Springs, USA
Chapter VI Group Support Systems as Collaborative Learning Technologies: A Meta-Analysis........................... 79 John Lin, National University of Singapore, Singapore Yin Ping Yang, National University of Singapore, Singapore Yingqin Zhong, National University of Singapore, Singapore Section II Design, Modeling, and Evaluation Issues Chapter VII Knowledge Flow and Learning Design Models towards Lifewide E-Learning Environments....................................................................................................................................... 110 M.C. Pettenati, University of Florence, Italy M.E. Cigognini, University of Florence, Italy Chapter VIII An Agent-Based Framework for Personalized E-Learning Services................................................... 130 Larbi Esmahi, Athabasca University, Canada Chapter IX Supporting Evolution of Knowledge Artifacts in Web Based Learning Environments...................... 142 Dimitris Kotzinos, Institute of Computer Science, FORTH-ICS and Department of Geomatics and Surveying, TEI of Serres, Greece Giorgos Flouris, Institute of Computer Science, FORTH-ICS, Greece Yannis Tzitzikas, University of Crete and Institute of Computer Science, FORTH-ICS, Greece Dimitris Andreou, Institute of Computer Science, FORTH-ICS, Greece Vassilis Christophides, University of Crete and Institute of Computer Science, FORTH-ICS, Greece Chapter X Interface and Features for an Automatic ‘C’ Program Evaluation System.......................................... 168 Amit Kumar Mandal, IIT Kharagpur, India Chittaranjan Mandal, IIT Kharagpur, India Chris Read, Kingston University, UK Chapter XI Evaluating Computerized Adaptive Testing Systems.......................................................................... 186 Anastasios A. Economides, University of Macedonia, Greece Chrysostomos Roupas, University of Macedonia, Greece
Chapter XII Technology Integration Practices within a Socioeconomic Context: Implications for Educational Disparities and Teacher Preparation........................................................................... 203 Holim Song, Texas Southern University, USA Emiel Owens, Texas Southern University, USA Terry T. Kidd, University of Texas School of Public Health, USA Chapter XIII Utilizing Web Tools for Computer-Mediated Communication to Enhance Team-Based Learning.......................................................................................................................... 218 Elizabeth Avery Gomez, New Jersey Institute of Technology, USA Dezhi Wu, Southern Utah University, USA Katia Passerini, New Jersey Institute of Technology, USA Michael Bieber, New Jersey Institute of Technology, USA Chapter XIV Accessible E-Learning: Equal Pedagogical Opportunities for Students with Sensory Limitations............................................................................................................................. 233 Rakesh Babu, University of North Carolina at Greensboro, USA Vishal Midha, University of North Carolina at Greensboro, USA Section III Tools and Applications Chapter XV Supporting Argumentative Collaboration in Communities of Practice: The CoPe_it! Approach........................................................................................................................ 245 Nikos Karacapilidis, University of Patras and Research Academic Computer Technology Institute, Greece Manolis Tzagarakis, Research Academic Computer Technology Institute, Greece Chapter XVI Personalization Services for Online Collaboration and Learning........................................................ 258 Christina E. Evangelou, Informatics and Telematics Institute, Greece Manolis Tzagarakis, Research Academic Computer Technology Institute, Greece Nikos Karousos, Research Academic Computer Technology Institute, Greece George Gkotsis, Research Academic Computer Technology Institute, Greece Dora Nousia, Research Academic Computer Technology Institute, Greece
Chapter XVII Computer-Aided Personalised System of Instruction for Teaching Mathematics in an Online Learning Environment.................................................................................................... 271 Willem-Paul Brinkman, Delft University of Technology, The Netherlands Andrew Rae, Brunel University, UK Yogesh Kumar Dwivedi, Swansea University, UK Chapter XVIII Social Software for Sustaining Interaction, Collaboration, and Learning in Communities of Practice...................................................................................................................... 300 Sandy el Helou, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland Denis Gillet, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland Christophe Salzmann, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland Yassin Rekik, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland Chapter XIX Multimedia Authoring for Communities of Teachers.......................................................................... 317 Agnès Guerraz, INRIA Rhône-Alpes, France Cécile Roisin, INRIA Rhône-Alpes, France Jan Mikáč, INRIA Rhône-Alpes, France Romain Deltour, INRIA Rhône-Alpes, France Compilation of References................................................................................................................ 334 About the Contributors..................................................................................................................... 367 Index.................................................................................................................................................... 375
Detailed Table of Contents
Preface.................................................................................................................................................. xix Section I Augmenting Learning Chapter I The Role of Learner in an Online Community of Inquiry: Responding to the Challenges of First-Time Online Learners................................................................................................................. 1 Martha Cleveland-Innes, Athabasca University, Canada Randy Garrison, The University of Calgary, Canada Ellen Kinsel, Odyssey Learning Systems, Canada Learners experiencing an online educational community for the first time can explain the adjustment required for participation. Findings from a study of adjustment to online learning environments validate differences found in 3 presences in an online community of inquiry. Using pre- and post-questionnaires, students enrolled in entry-level courses in 2 graduate degree programs at Athabasca University, Canada, describe their adjustment to online learning. Responses were analyzed in relation to the elements of cognitive, social, and teaching presence, defined by Garrison, Anderson, and Archer (2000) as core dimensions of learner role requirements in an online community of inquiry. Five areas of adjustment characterize the move toward competence in online learning: interaction, self-identity, instructor role, course design, and technology. Student comments provide understanding of the experience of first-time online learners, including the challenges, interventions, and resolutions that present themselves as unique incidents. Recommendations for the support and facilitation of adjustment are made. Chapter II Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning................................................................................................................................................. 15 Xinchun Wang, California State University, Fresno, USA Although the pedagogical advantages of online interactive learning are well known, much needs to be done in instructional design of applicable collaborative learning tasks that motivate sustained student participation and interaction. In a previous study based on a Web-based course offered in 2004, Wang
(2007) investigated the factors that promote sustained online collaboration for knowledge building. By providing new data from the same Web-based course offered in 2006 and 2007, this study investigates students’ attitudes toward process- and product-oriented online collaborative learning. The analysis of 93 post course survey questionnaire data show that the overwhelming majority of students have positive experience with online collaborative learning. Data also suggest that students are more enthusiastic about process-oriented tasks and their attitudes toward product-oriented collaborative learning tasks are mixed. Chapter III Cognition, Technology, and Performance: The Role of Course Management Systems........................ 35 Teresa Lang, Columbus State University, USA Dianne Hall, Auburn University, USA Development and sale of computer-assisted instructional supplements and course management system products are increasing. Textbook sales representatives use this technology to market textbooks, and many colleges and universities encourage the use of such technology. The use of course management systems in education has been equated to the use of enterprise resource planning software by large businesses. Research findings about the pedagogical benefits of computer-assisted instruction and computer management systems are inconclusive. This study describes an experiment conducted to determine the benefit to students of using course management systems. The effects of cognition, learning styles, and computer attitude were considered and eliminated to better isolate any differences in performance. Student performance did not improve with the use of the technology. Chapter IV The Role of Organizational, Environmental and Human Factors in E-Learning Diffusion................................................................................................................................................ 53 Kholerile L. Gwebu, University of New Hampshire, USA Jing Wang, Kent State University, USA Improvements in technology have led to innovations in training such as Electronic Learning (E-learning). E-learning aims to help organizations in their training initiatives by simplifying the training process and cutting cost. It also attempts to help employees in their learning processes by making learning readily accessible. Unfortunately, the diffusion of this innovation has not been as successful as was initially predicted. In this paper we explore the drivers behind the diffusion of e-learning. Apart from the factors investigated by previous research, we believe that one more dimension, -human factors- should be taken into account when evaluating the diffusion of a training innovation, since learners are, to a large extent, the central issue of training. In the case of e-learning we believe that motivation plays a key role in the diffusion of e-learning. Chapter V Distance Education: Satisfaction and Success....................................................................................... 71 Wm. Benjamin Martz, Jr., Northern Kentucky University, USA Morgan Shepherd, University of Colorado at Colorado Springs, USA
• •
Almost 3.5 million students were taking at least 1 online course during the fall 2006 term. The 9.7 % growth rate for online enrollments far exceeds the 1.5 % growth of the overall higher education student population. (Allen and Seaman, 2007)
By 2006, the distance education industry was well beyond $33.6 billion (Merit Education, 2003). As with most markets, 1 of the keys to taking advantage of this growing market is customer satisfaction. Therefore the greater the student satisfaction in a distance program, the more likely that program will be successful. This paper identifies 5 key components of satisfaction for distance education programs through a student satisfaction questionnaire and factor analysis. A questionnaire was developed using these variables and administered to 341 distance students. The results revealed 5 constructs for student satisfaction in a distance education program (Martz and Reddy, 2005; Martz and Shepherd, 2007). Using these factors as guidance, this paper extends those findings to provide some operational and administrative implications. Chapter VI Group Support Systems as Collaborative Learning Technologies: A Meta-Analysis........................... 79 John Lin, National University of Singapore, Singapore Yin Ping Yang, National University of Singapore, Singapore Yingqin Zhong, National University of Singapore, Singapore Computer-based systems have been widely applied to support group-related activities such as collaborative learning and training. The various terms accorded to this research stream include virtual teams, ecollaboration, computer-supported collaborative work, distributed work, electronic meetings, and so forth. A notable and well-accepted aspect in the information system field is group support systems (GSS), the focus of this chpater. The numerous GSS studies have reported findings which may not be altogether consistent. An overall picture is much in want which attends to the synthesizing of the findings accumulated over decades. This chapter presents a meta-analysis study aimed at gaining a general understanding of GSS effects. We investigate 6 important moderators in GSS experimental research: group outcomes, namely group size, task type, anonymity, time and proximity, level of technology, and the existence of facilitation. The results point to important conclusions about the phenomenon of interest; in particular, their implications vis-à-vis computer-supported collaborative learning technologies and use are discussed and highlighted along each dimension of the studied variables. Section II Design, Modeling, and Evaluation Issues Chapter VII Knowledge Flow and Learning Design Models towards Lifewide E-Learning Environments....................................................................................................................................... 110 M.C. Pettenati, University of Florence, Italy M.E. Cigognini, University of Florence, Italy
This chapter considers the affordances of social networking theories and tools in building new and effective e-learning practices. We argue that “Connectivism” (social networking applied to learning and knowledge contexts) can lead to a re-conceptualization of learning in which formal, non-formal, and informal learning can be integrated so as to build potentially lifelong learning activities which can be experienced in “personal learning environments”. In order to provide a guide for the design, development, and improvement of e-learning environments, as well as for the related learning activities, we provide a knowledge flow model and the consequent learning design model, highlighting the stages of learning, the enabling conditions, and possible technological tools to be used for the purpose. In the conclusion to the chapter, the derived model is applied in a possible scenario of formal learning in order to show how the learning process can be designed according to the presented theory. Chapter VIII An Agent-Based Framework for Personalized E-Learning Services................................................... 130 Larbi Esmahi, Athabasca University, Canada This paper provides an overview of personalized e-learning services and related technology and presents a multi-agent system for delivering adaptive e-learning. We discussed the main issues related to personalization in e-learning: technology advancement and the shift in perception of the learning process, one size fits all versus personalized services, and the adaptation process. The paper provides also an overview of most known implemented systems for adaptive e-learning, as well as detailed description of the architecture and components of the proposed multi-agent framework. Finally, the paper concludes with some comments about the dimensions to consider for implementing personalization.. Chapter IX Supporting Evolution of Knowledge Artifacts in Web Based Learning Environments...................... 142 Dimitris Kotzinos, Institute of Computer Science, FORTH-ICS and Department of Geomatics and Surveying, TEI of Serres, Greece Giorgos Flouris, Institute of Computer Science, FORTH-ICS, Greece Yannis Tzitzikas, University of Crete and Institute of Computer Science, FORTH-ICS, Greece Dimitris Andreou, Institute of Computer Science, FORTH-ICS, Greece Vassilis Christophides, University of Crete and Institute of Computer Science, FORTH-ICS, Greece The development of collaborative e-learning environments that support the evolution of semantically described knowledge artifacts is a challenging task. In this chapter we elaborate on usage scenarios and requirements for environments grounded on learning theories that stress on collaborative knowledge creation activities. Subsequently, we present a comprehensive suite of services, comprising an emerging framework, called Semantic Web Knowledge Middleware (SWKM), that enables the collaborative evolution of both domain abstractions and conceptualizations, and data classified using them. The suite includes advanced services for ontology change, comparison and versioning over a common knowledge repository offering persistent storage and validation.
Chapter X Interface and Features for an Automatic ‘C’ Program Evaluation System.......................................... 168 Amit Kumar Mandal, IIT Kharagpur, India Chittaranjan Mandal, IIT Kharagpur, India Chris Read, Kingston University, UK A system for automatically testing, evaluating, grading, and providing critical feedback for submitted ‘C’ programming assignments has been implemented. The interface and key features of the system are described in detail along with some examples. The system gives proper attention towards the monitoring of a student’s progress and provides complete automation of the evaluation process, with a fine-grained analysis. It also provides online support to both the instructors and students and is designed for serviceoriented integration with a course management system using Web services. Chapter XI Evaluating Computerized Adaptive Testing Systems.......................................................................... 186 Anastasios A. Economides, University of Macedonia, Greece Chrysostomos Roupas, University of Macedonia, Greece Many educational organizations are trying to reduce the cost of exams, the workload, delay of scoring, and the human errors. Also, organizations try to increase the accuracy and efficiency of the testing. Recently, most examination organizations use Computerized Adaptive Testing (CAT) as the method for large scale testing. This chapter investigates the current state of CAT systems and identifies their strengths and weaknesses. It evaluates 10 CAT systems using an evaluation framework of 15 domains categorized into 3 dimensions: Educational, Technical and Economical. The results show that the majority of the CAT systems give priority to security, reliability, and maintainability. However, they do not offer to the examinee any advanced support and functionalities. Also, the feedback to the examinee is limited and the presentation of the items is poor. Recommendations are made in order to enhance the overall quality of a CAT system. For example, alternative multimedia items should be available so that the examinee would choose his preferred media type. Feedback could be improved by providing more information to the examinee or providing information anytime the examinee wished. Chapter XII Technology Integration Practices within a Socioeconomic Context: Implications for Educational Disparities and Teacher Preparation........................................................................... 203 Holim Song, Texas Southern University, USA Emiel Owens, Texas Southern University, USA Terry T. Kidd, University of Texas School of Public Health, USA With the call for curricular and instructional reform, educational institutions have embarked on the process to reform their educational practices to aid the lower SES student in their quest to obtain quality education with the integration of technology. The study performed was to examine the socioeconomic disparities of teachers’ technology integration in the classroom as it relates to implementing technology
interventions to support quality teaching and active student learning. This chapter provides empirical evidence of whether these disparities continue to exist, and their effects on student achievement in the classroom. Chapter XIII Utilizing Web Tools for Computer-Mediated Communication to Enhance Team-Based Learning.......................................................................................................................... 218 Elizabeth Avery Gomez, New Jersey Institute of Technology, USA Dezhi Wu, Southern Utah University, USA Katia Passerini, New Jersey Institute of Technology, USA Michael Bieber, New Jersey Institute of Technology, USA Team-based learning is an active learning instructional strategy used in the traditional face-to-face classroom. Web-based computer-mediated communication (CMC) tools complement the face-to-face classroom and enable active learning between face-to-face class times. This paper presents the results from pilot assessments of computer-supported team-based learning. The authors utilized pedagogical approaches grounded in collaborative learning techniques, such as team-based learning, and extended these techniques to a web-based environment through the use of computer-mediated communications tools (discussion web-boards). This approach was examined through field studies in the course of two semesters at a US public technological university. The findings indicate that the perceptions of team learning experience such as perceived motivation, enjoyment and learning in such a web-based CMC environment are higher than in traditional face-to-face courses. In addition, our results show that perceived team members’ contributions impact individual learning experiences. Overall, Web-based CMC tools are found to effectively facilitate team interactions and achieve higher-level learning. Chapter XIV Accessible E-Learning: Equal Pedagogical Opportunities for Students with Sensory Limitations............................................................................................................................. 233 Rakesh Babu, University of North Carolina at Greensboro, USA Vishal Midha, University of North Carolina at Greensboro, USA The transformation of the world into a highly technological place has led to the evolution of learning from the traditional classroom to e-learning, using tools such as course management systems (CMS). By its very nature, e-learning offers a range of advantages over traditional pedagogical methods, including issues of physical access. It is particularly useful for people with sensory limitations as it offers a level playing field for them in learning. This study examines the accessibility, usability, and richness of CMS used for e-Learning in institutions of higher education. A model is proposed that underscores the influence of accessibility, usability, and richness of the CMS, coupled with learning motivation on the learning success as perceived by students with sensory limitations. The model is tested by surveying university students with sensory limitations about their views on the course management system used. The results suggested that accessibility and usability of a CMS have a positive influence on the learning success as perceived by students with sensory limitations.
Section III Tools and Applications Chapter XV Supporting Argumentative Collaboration in Communities of Practice: The CoPe_it! Approach........................................................................................................................ 245 Nikos Karacapilidis, University of Patras and Research Academic Computer Technology Institute, Greece Manolis Tzagarakis, Research Academic Computer Technology Institute, Greece Providing the necessary means to support and foster argumentative collaboration is essential for Communities of Practice to achieve their goals. However, current tools are unable to cope with the evolving stages of the collaboration. This is primarily due to the inflexible level of formality they provide. Arguing that a varying level of formality needs to be offered in systems supporting argumentative collaboration, this chapter proposes an incremental formalization approach that has been adopted in the development of CoPe_it!, a Web-based tool that complies with collaborative principles and practices, and provides members of communities engaged in argumentative discussions and decision making processes with the appropriate means to collaborate towards the solution of diverse issues. According to the proposed approach, incremental formalization can be achieved through the consideration of alternative projections of a collaborative workspace. Chapter XVI Personalization Services for Online Collaboration and Learning........................................................ 258 Christina E. Evangelou, Informatics and Telematics Institute, Greece Manolis Tzagarakis, Research Academic Computer Technology Institute, Greece Nikos Karousos, Research Academic Computer Technology Institute, Greece George Gkotsis, Research Academic Computer Technology Institute, Greece Dora Nousia, Research Academic Computer Technology Institute, Greece Collaboration tools can be exploited as virtual spaces that satisfy the community members’ needs to construct and refine their ideas, opinions, and thoughts in meaningful ways, in order to suc-cessfully assist individual and community learning. More specifically, collaboration tools when properly personalized can aid individuals to articulate their personal standpoints in such a way that can be proven useful for the rest of the community where they belong. Personalization services, when properly integrated to collaboration tools, can be an aide to the development of learning skills, to the interaction with other actors, as well as to the growth of the learners’ autonomy and self-direction. This work pre-sents a framework of personalization services that has been developed to address the requirements for efficient and effective collaboration between online communities’ members that can act as catalysts for individual and community learning.
Chapter XVII Computer-Aided Personalised System of Instruction for Teaching Mathematics in an Online Learning Environment.................................................................................................... 271 Willem-Paul Brinkman, Delft University of Technology, The Netherlands Andrew Rae, Brunel University, UK Yogesh Kumar Dwivedi, Swansea University, UK This paper presents a case study of a university’s discrete mathematics course with over 170 students who had access to an online learning environment (OLE) that included a variety of online tools, such as videos, self-tests, discussion boards, and lecture notes. The course is based on the ideas of the Personalised System of Instruction (PSI) modified to take advantage of an OLE. Students’ learning is initially examined over a period of 2 years, and compared with that in a more traditionally taught part of the course. To examine students’ behaviour, learning strategies, attitudes, and performance, both qualitative and quantitative techniques, were used as a mixed methodology approach, including in-depth interviews (N=9), controlled laboratory observations (N=8), surveys (N=243), diary studies (N=10), classroom observations, recording online usage behaviour, and learning assessments. In addition, students’ attitude and performance in 2 consecutive years where PSI was applied to the entire course provides further understanding that is again in favour of PSI in the context of OLE. This chapter aims to increase understanding of whether PSI, supported by an OLE, could enhance student appreciation and achievement as findings suggest. Chapter XVIII Social Software for Sustaining Interaction, Collaboration, and Learning in Communities of Practice...................................................................................................................... 300 Sandy el Helou, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland Denis Gillet, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland Christophe Salzmann, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland Yassin Rekik, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland The École Polytechnique Fédérale de Lausanne is developing a Web 2.0 social software called eLogbook and designed for sustaining interaction, collaboration, and learning in online communities. This chapter describes the 3A model on which eLogbook is based as well as the main services that the latter provides. The proposed social software has several innovative features that distinguish it from other classical online collaboration solutions. It offers a high-level of flexibility and adaptability so that it can fulfill the requirements of various Communities of Practice. It also provides community members with ubiquitous access and awareness through its different interfaces. Finally, eLogbook strengthens usability and acceptability thanks to its personalization and contextualization mechanisms.
Chapter XIX Multimedia Authoring for Communities of Teachers.......................................................................... 317 Agnès Guerraz, INRIA Rhône-Alpes, France Cécile Roisin, INRIA Rhône-Alpes, France Jan Mikáč, INRIA Rhône-Alpes, France Romain Deltour, INRIA Rhône-Alpes, France One way of providing technological support for communities of teachers is to help participants to produce, structure and share information. As this information becomes more and more multimedia in nature, the challenge is to build multimedia authoring and publishing tools that meets requirements of the community. In this paper we analyze these requirements and propose a multimedia authoring model and a generic platform on which specific community-oriented authoring tools can be realized. The main idea is to provide template-based authoring tools while keeping rich composition capabilities and smooth adaptability. It is based on a component-oriented approach integrating homogeneously logical, time and spatial structures. Templates are defined as constraints on these structures. Compilation of References................................................................................................................ 334 About the Contributors..................................................................................................................... 367 Index.................................................................................................................................................... 375
xix
Preface
This volume of Advances in Web-based Learning (AWBL) Book Series, entitled “Solutions and Innovations in Web-Based Technologies for Augmented Learning: Improved Platforms, Tools, and Applications”, includes a wide range of the most current research in the development of innovative Web-based learning solutions. It aims at providing an in-depth coverage and understanding of issues related to the implementation and application of Web-based technologies for facilitating and augmenting learning in diverse contemporary organizational settings. In our present era, Web-based technologies offer numerous means for facilitating and enhancing learning in various contexts. Their exploitation should be in accordance with a series of technical, pedagogical, cognitive, social, cultural, and managerial parameters. This volume will assist researchers, educators, and professionals in understanding the necessary components for Web-based learning technologies and how to best adopt these elements into their own contexts, wehter being in classrooms, workgroups, communities, or world-wide organizations. Chapter I, “The Role of Learner in an Online Community of Inquiry: Responding to the Challenges of First-time Online Learners” by Martha Cleveland-Innes, Athabasca University (Canada), Randy Garrison, The University of Calgary (Canada), and Ellen Kinsel, Odyssey Learning Systems (Canada), reports on findings from a study of adjustments to online learning environments. Using pre- and postquestionnaires, students enrolled in entry-level courses in two graduate degree programs at Athabasca University, Canada, and described their adjustments to online learning. Responses are analyzed in relation to the elements of cognitive, social, and teaching presence, which have been defined as core dimensions of learner role requirements in an online community of inquiry. Student comments provide understanding of the experience of first-time online learners, including the challenges, interventions, and resolutions that present themselves as unique incidents. Recommendations for the support and facilitation of adjustment are also made. Chapter II, “Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning” by Xinchun Wang, California State University at Fresno (USA), focuses on the instructional design of applicable collaborative learning tasks that motivate sustained student participation and interaction. By providing data from a Web-based course offered in 2006 and 2007, this chapter investigates students’ attitudes toward process and product oriented online collaborative learning. The analysis of 93 post course survey questionnaire data show that the overwhelming majority of students have positive experience with online collaborative learning. Data also suggest that students are more enthusiastic about process oriented tasks and their attitudes toward product oriented collaborative learning tasks are mixed. Chapter III, “Cognition, Technology, and Performance: The Role of Course Management Systems” by Teresa Lang, Columbus State University (USA) and Dianne Hall, Auburn University (USA), describes an experiment conducted to determine the benefit to students of using course management systems. The effects of cognition, learning styles, and computer attitude were considered and eliminated to better
xx
isolate any differences in performance. The data collected in this study supported the hypothesis that cognition influences performance. Learning style was found not to influence performance. Cognition and learning styles are frequently cited in the literature as influencing performance both with technology and without technology, so this finding partially supports the literature. Moreover, empirical evidence to support the benefit of course management systems in the learning process was not found in this study. Chapter IV, “The Role of Organizational, Environmental and Human Factors in E-Learning Diffusion” by Kholekile Gwebu, University of New Hampshire (USA) and Jing Wang, Kent State University (USA), explores the factors that influence e-learning diffusion in contemporary corporations. Through the application of concepts from the literature on organizational change, innovation diffusion, and motivation, authors attempt to assess the factors that influence the diffusion of e-learning in organizations. Among these factors are organizational variables such as Organizational Complexity (specialization, function differentiation, professionalism) and Bureaucratic Control (formalization, centralization, vertical differentiation). Much attention is given to the factor of employee motivation. Chapter V, “Distance Education: Satisfaction and Success” by Benjamin Martz, Northern Kentucky University (USA) and Morgan Shepherd, University of Colorado at Colorado Springs (USA), identifies five key components of satisfaction for distance education programs through a student satisfaction questionnaire and factor analysis. A questionnaire was developed using these variables and administered to 341 distance students. The results revealed five constructs for student satisfaction in a distance education program. Using these factors as guidance, this chapter extends those findings to provide some operational and administrative implications. Chapter VI, “Group Support Systems as Collaborative Learning Technologies: A Meta-Analysis” by John Lim, Yin Ping Yang, and Yingqin Zhong, National University of Singapore (Singapore), presents a meta-analysis study aimed at gaining a general understanding of Group Support Systems (GSS) effects. Six important moderators in GSS experimental research are investigated: group outcomes, namely group size, task type, anonymity, time and proximity, level of technology, and the existence of facilitation. The results point to important conclusions about the phenomenon of interest. Their implications with regards to computer-supported collaborative learning technologies and use are discussed and highlighted along each dimension of the studied variables. Chapter VII, “Knowledge flow and learning design models towards lifewide e-learning environments” by Maria Chiara Pettenati and Elisabetta Cigognini, University of Florence (Italy), considers the affordances of social networking theories and tools in building new and effective e-learning practices. In order to provide a guide for the design, development and improvement of e-learning environments, as well as for the related learning activities, this chapter proposes a knowledge flow model and the consequent learning design model, highlighting the stages of learning, the enabling conditions, and possible technological tools to be used. The proposed model is applied in a scenario of formal learning. Chapter VIII, “An Agent-Based Framework for Personalized E-Learning Services” by Larbi Esmahi, Athabasca University (Canada), provides an overview of personalized e-learning services and related technology, and presents a multi-agent system for delivering adaptive e-learning. The author discusses the main issues related to personalization in e-learning: technology advancement and the shift in perception of the learning process, one size fits all vs. personalized services, and the adaptation process. The chapter also provides an overview of the most known implemented systems for adaptive e-learning, as well as a detailed description of the architecture and components of the proposed multi-agent framework. Chapter IX, “Supporting Evolution of Knowledge Artifacts in Web-Based Learning Environments” by Dimitris Kotzinos, FORTH-ICS and TEI of Serres (Greece), Giorgos Flouris, FORTH-ICS (Greece), Yannis Tzitzikas, University of Crete and FORTH-ICS (Greece), Dimitris Andreou, FORTH-ICS (Greece), and Vassilis Christophides, University of Crete and FORTH-ICS (Greece), elaborates usage scenarios
xxi
and requirements for e-learning environments grounded on learning theories that stress on collaborative knowledge creation activities. Subsequently, this chapter presents a comprehensive suite of services, comprising an emerging framework, called Semantic Web Knowledge Middleware, that enables the collaborative evolution of both domain abstractions and conceptualizations, and data classified using them. The proposed suite includes advanced services for ontology change, comparison, and versioning over a common knowledge repository offering persistent storage and validation. Chapter X, “Interface and Features for an Automatic ‘C’ Program Evaluation System” by Amit Kumar Mandal, IIT Kharagpur (India), Chittaranjan Mandal, IIT Kharagpur (India), and Chris Reade, Kingston University (United Kingdom), reports on an implemented system for automatically testing, evaluating, grading, and providing critical feedback for submitted ‘C’ programming assignments. The interface and key features of the system are described in detail along with some examples. The system gives proper attention towards the monitoring of a students’ progress and provides complete automation of the evaluation process. It also provides online support to both the instructors and students and is designed for service-oriented integration with a course management system using Web services. Chapter XI, “Evaluating Computerized Adaptive Testing Systems” by Anastasios Economides and Chrysostomos Roupas, University of Macedonia (Greece), investigates the current state of Computerized Adaptive Testing (CAT) systems and identifies their strengths and weaknesses. More specifically, this chapter evaluates ten CAT systems using an evaluation framework of 15 domains categorized into three dimensions: Educational, Technical, and Economical. The results show that the majority of the CAT systems give priority to security, reliability, and maintainability. However, they do not offer to the examinee any advanced support and functionalities. Also, the feedback to the examinee is limited and the presentation of the items is poor. Recommendations are given in order to enhance the overall quality of a CAT system. Chapter XII, “Technology Integration Practices within a Socioeconomic Context: Implications for Educational Disparities and Teacher Preparation” by Holim Song, Texas Southern University (USA), Emiel Owens, Texas Southern University (USA), and Terry Kidd, University of Texas (USA), reports on a study that was performed in order to examine the socioeconomic disparities of teachers’ technology integration in the classroom as it relates to implementing technology interventions to support quality teaching and active student learning. This chapter provides empirical evidence of whether these disparities continue to exist and if so, their effects on student achievement in the classroom. Chapter XIII, “Utilizing Web Tools for Computer-Mediated Communication to Enhance TeamBased Learning” by Elizabeth Avery Gomez, New Jersey Institute of Technology (USA), Dezhi Wu, Southern Utah University (USA), Katia Passerini, New Jersey Institute of Technology (USA), and Michael Bieber, New Jersey Institute of Technology (USA), presents the results from pilot assessments of computer-supported team-based learning. Authors utilized pedagogical approaches grounded in collaborative learning techniques, such as team-based learning, and extended these techniques to a Webbased environment through the use of computer-mediated communications (CMC) tools. Their approach was examined through field studies duirng the course of two semesters at a U.S. public technological university. The findings indicate that Web-based CMC tools effectively facilitate team interactions and achieve higher-level learning. Chapter XIV, “Accessible E-Learning: Equal Pedagogical Opportunities for Students with Sensory Limitations” by Rakesh Babu, University of North Carolina at Greensboro (USA), and Vishal Midha, University of North Carolina at Greensboro (USA), examines the accessibility, usability, and richness of course management systems (CMS) used for e-learning in institutions of higher education. A model is proposed that underscores the influence of accessibility, usability, and richness of the CMS, coupled with learning motivation on the learning success as perceived by students with sensory limitations. The
xxii
model is tested by surveying university students with sensory limitations about their views on the course management system used. The results suggest that accessibility and usability of a CMS have a positive influence on the learning success as perceived by students with sensory limitations. Chapter XV, “Supporting Argumentative Collaboration in Communities of Practice: The CoPe_it! approach” by Nikos Karacapilidis, University of Patras and Research Academic Computer Technology Institute (Greece), and Manolis Tzagarakis, Research Academic Computer Technology Institute (Greece), argues that a varying level of formality needs to be offered in systems supporting argumentative collaboration. The chapter accordingly proposes an incremental formalization approach that has been adopted in the development of CoPe_it!, a Web-based tool that complies with collaborative principles and practices, and provides members of communities engaged in argumentative discussions and decision making processes with the appropriate means to collaborate towards the solution of diverse issues. Chapter XVI, “Personalization Services for Online Collaboration and Learning” by Christina Evangelou, Informatics and Telematics Institute (Greece), Manolis Tzagarakis, Research Academic Computer Technology Institute (Greece), Nikos Karousos, Research Academic Computer Technology Institute (Greece), George Gkotsis, Research Academic Computer Technology Institute (Greece), and Dora Nousia, Research Academic Computer Technology Institute (Greece), focuses on the integration of personalization services to collaboration support tools, the aim being to advance the development of learning skills, the interaction with other actors, and the growth of the learners’ autonomy and self-direction. This chapter presents a framework of personalization services that has been developed to address the requirements for efficient and effective collaboration between online communities’ members that can act as catalysts for individual and community learning. Chapter XVII, “Computer-Aided Personalised System of Instruction for Teaching Mathematics in an Online Learning Environment” by Willem-Paul Brinkman, Delft University of Technology (The Netherlands), Andrew Rae, Brunel University (United Kingdom), and Yogesh Kumar Dwivedi, Swansea University (United Kingdom), presents a case study of a disrete university mathematics course with over 170 students who had access to an online learning environment that included a variety of online tools, such as videos, self-tests, discussion boards, and lecture notes. Students’ learning is initially examined over a period of two years, and compared with that in a more traditionally taught part of the course. To examine students’ behaviour, learning strategies, attitudes ,and performance, both qualitative and quantitative techniques were used as a mixed methodology approach, including in-depth interviews, controlled laboratory observations, surveys, diary studies, classroom observations, recording online usage behaviour, and learning assessments. Chapter XVIII, “Social Software for Sustaining Interaction, Collaboration, and Learning in Communities of Practice” by Sandy El Helou, Denis Gillet, Christophe Salzmann, and Yassin Rekik, École Polytechnique Fédérale de Lausanne – EPFL (Switzerland), presents a Web 2.0 social software, called eLogbook, which has been designed for sustaining interaction, collaboration, and learning in online communities. This chapter describes the 3A model on which eLogbook is based as well as the main services that the latter provides. The proposed social software has several innovative features that distinguish it from other classical online collaboration solutions. Among others, it offers a high-level of flexibility and adaptability so that it can fulfill the requirements of various Communities of Practice. It provides community members with ubiquitous access and awareness through its different interfaces, and strengthens usability and acceptability thanks to its personalization and contextualization mechanisms. Chapter XIX, “Multimedia Authoring for Communities of Teachers” by Agnès Guerraz, INRIA Rhône-Alpes (France), Cécile Roisin, INRIA Rhône-Alpes (France), Jan Mikáč, INRIA Rhône-Alpes (France), and Romain Deltour, INRIA Rhône-Alpes (France), proposes a multimedia authoring model and a generic platform on which specific community-oriented authoring tools can be realized. The main
xxiii
idea is to provide template-based authoring tools, while keeping rich composition capabilities and smooth adaptability. The proposed model is based on a component-oriented approach integrating logical, time and spatial structures, while templates are defined as constraints on these structures. The proper exploitation of Web-based technologies towards building responsive environments that motivate, engage, and inspire learners, and which are embedded in the business processes and human resources management systems of organizations, is highly critical. Accordingly, the research field of technology-enhanced learning receives a continuously increasing attention. “Solutions and Innovations in Web-Based Technologies for Augmented Learning: Improved Platforms, Tools and Applications” provides cutting-edge research on a series of related topics and discusses its implications in the modern era’s broad learning concept. Addressing diverse conceptual, social, and technical issues, this book provides professionals, researchers, and practitioners in the field with up-to-date research in developing innovative and more effective learning systems by using Web-based technologies. Nikos Karacapilidis Editor-in-Chief Advances in Web-based Learning Book Series
Section I
Augmenting Learning
Chapter I
The Role of Learner in an Online Community of Inquiry:
Responding to the Challenges of First-Time Online Learners Martha Cleveland-Innes Athabasca University, Canada Randy Garrison The University of Calgary, Canada Ellen Kinsel Odyssey Learning Systems, Canada
Abstract Learners experiencing an online educational community for the .rst time can explain the adjustment required for participation. Findings from a study of adjustment to online learning environments validate differences found in 3 presences in an online community of inquiry. Using pre- and post-questionnaires, students enrolled in entry-level courses in 2 graduate degree programs at Athabasca University, Canada, describe their adjustment to online learning. Responses were analyzed in relation to the elements of cognitive, social, and teaching presence, defined by Garrison, Anderson, and Archer (2000) as core dimensions of learner role requirements in an online community of inquiry. Five areas of adjustment characterize the move toward competence in online learning: interaction, self-identity, instructor role, course design, and technology. Student comments provide understanding of the experience of first-time online learners, including the challenges, interventions, and resolutions that present themselves as unique incidents. Recommendations for the support and facilitation of adjustment are made. Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.
The Role of Learner in an Online Community of Inquiry
Introduction The move to online delivery in post-secondary education institutions has increased exponentially over the last decade. Early concerns were raised about the extent to which students would embrace online education. However, recent evaluation of student enrolment in online courses indicates much willingness to engage; optimistic online enrolment projections are now a reality and there are implications that growth will continue. “Online enrolments continue to grow at rates faster than for the overall student body, and schools expect the rate of growth to further increase.” (Allen & Seaman, 2004, Introduction, 3rd para.). As growth continues, more and more students will experience online education. Students will have to develop new skills required to be competent online learners, and will modify behaviours from classroom learning to fit the online environment. The details of this adjustment process for learners to this new delivery method is still under explored; “there is also (sic) a need for better understanding of students’ adaptation to online learning over time.” (Wilson, et al., 2003). Adaptation to the role of online learner can be understood by looking at the structure of the online pedagogical environment, or community of inquiry (Garrison, Anderson & Archer, 2000) and tenets of role theory (Blau & Goodman, 1995) and how role change occurs (Turner, 1990). The integration of new behaviours into one’s role repertoire (Kopp, 2000) occurs in a context (Katz & Kahn, 1978) and through an intricate process of role taking, role exploration and role making (Blau & Goodman, 1995). As the context of teaching and learning in online environments is very different from long standing classroom structure, and will act as a catalyst for role adjustment for individual satudents moving online. This paper outlines the character of adjustment made by such students, determined from a study of novice online learners. Students responded to open-ended questions before and after (pre
and post) their first online experience; responses were coded and categorized according to adjustment to cognitive, social and teaching presence. With each of the presences, responses formed a pattern around activities and outcomes in the following thematic areas: interaction, instructor role, self-identity, course design, and technology. In addition, a process of meeting challenges presented by this new environment is outlined. This data provides understanding of the experience of first-time online learners. Recommendations are made for incorporating this understanding into instructional design and facilitation in order to ease adjustment for learners new to the online environment.
LITERATURE REVIEW Online Community of Inquiry The community of inquiry model, originally proposed by Garrison, Anderson and Archer (2000), served as the conceptual framework around which to study online learning and learner adjustment. The theoretical foundation of this framework is based upon the work of John Dewey (1938). At the core of Dewey’s philosophy are collaboration, free intercourse, and the juxtaposition of the subjective and shared worlds. This is the essence of a community of inquiry. Consistent with his philosophy of pragmatism, Dewey (1933) viewed inquiry as a practical endeavour. Inquiry emerged from practice and shaped practice. Dewey’s work on reflective thinking and inquiry provided the inspiration for operationalizing cognitive presence and purposeful learning in the community of inquiry framework (Garrison & Archer, 2000). The other elements of the community of inquiry model, social presence and teaching presence, were derived from other educational sources, but are consistent with Dewey’s philosophy and the framework of a community of inquiry (Garrison & Anderson, 2003).
The Role of Learner in an Online Community of Inquiry
The community of inquiry framework has attracted considerable attention in higher education research. In particular, it has framed many studies of online learning. This speaks to both the importance of community in higher education as well as the usefulness of the framework and how the elements are operationalized. Moreover, the structural validity of the framework has been tested and confirmed through factor analysis (Arbaugh & Hwang, 2006; Garrison, ClevelandInnes & Fung, 2004; Ice, Arbaugh, Diaz, Garrison, Richardson, Shea, & Swan, 2007). A review of the research using the framework and the identification of current research issues is provided by Garrison and Arbaugh (2007). An online community of inquiry, replete with interaction opportunities in several places of ‘presence’ (Garrison & Anderson, 2003) provides a supportive context for the re-development of the role of learner. The relationship among these dimensions is depicted in Figure 1. These are the core elements in an educational experience and key to understanding role adjustment. Cogni-
tive, social and teaching presences represent the primary dimensions of role in an educational context; it has a character of its own in an online environment. Changes in cognitive, social and teaching presence, as a result of a new context and communication medium, will necessitate role adjustments by the learners. Cognitive presence is defined “as the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse …” (Garrison, Anderson & Archer, 2001, p. 11). Role adjustment here reflects the nature of the communication medium: spontaneous, verbal communication is supplanted by a reflective, textbased medium. This represents a radical departure from classroom interaction. A more precise and recorded form of communication, the text-based medium has the potential to support deep and meaningful learning outcomes. Social presence is defined as “the ability of participants to identify with the community (e.g., course of study), communicate purposefully in a trusting environment, and develop inter-personal
Figure 1.
The Role of Learner in an Online Community of Inquiry
relationships by way of projecting their individual personalities” (Garrison, in press). In addition to the general challenge of asynchronous written communication and its lack of non-verbal cues, is the challenge of group identity. An essential characteristic of online learning is open communication and group cohesion. Social presence provides the capacity to communicate and collaborate. This requires that members identify with the group or class (Rogers & Lea, 2005). Since the educational experience is a social transaction, special consideration must be given to the social interactions and climate. Interpersonal and emotional communication should build over time. Social presence represents a major role adjustment in moving from a real-time face-to-face classroom experience to a virtual community. Teaching presence is defined as “the design, facilitation and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes” (Anderson, Rourke, Garrison, and Archer, 2001). It is what binds all the elements together in a purposeful community of inquiry. The properties of the online community also necessitate significant design changes and role adjustment for the teacher. Teaching presence must recognize and utilize the unique features of the medium and structure and model appropriate learning activities. This translates into an experience and role that may not be at all familiar to the learner.
“process by which people learn the characteristics of their group … (and) the attitudes, values and actions thought appropriate for them” (Kanwar & Swenson, 2000, p. 397). Under conditions of long-standing roles, individuals engage in ‘role-taking’ behavior, where observation and mimicry of role models allow those new to the role to ‘practice’ appropriate role behaviors. ‘Role making’ occurs as individuals construct aspects of the role with their own individual meanings and satisfying behaviors attached. This occurs under social conditions where such individual autonomy is allowed. It also occurs where role models are not readily available, and construction of the role is required. Such is the case for becoming an online learner. An adjustment from the more generalized role of learner, the responsibilities and requirements of working online are not readily apparent to those new to the role. The transition to, and adjustment in, the role of online learner, is part of the current social climate in online learning. While maintaining the usual expectations and privileges attached to the role of learner, online learners add such things as: • •
•
knowledge about, skill with and acceptance of the technology, new amounts and modes of communication with instructors, peers and administrators, increased levels of learner self-direction, and a new ‘place’ for learning in time (anytime, usually determined by the learner and their life circumstances) and space (anywhere, dependent upon equipment requirements).
Role Adjustment
•
‘Role’ is used here as a sociological construct, defined as a collection of behavioral requirements associated with a certain social position in a group, organization or society (Kendall, Murray & Linden, 2000). At its most general level, role expectations are dictated by the social structure. Individuals who engage in the role are guided, through a process of socialization, to appropriate role performance. Socialization then refers to the
An online community of inquiry is a distinct personal and public search for meaning and understanding. New roles are necessitated in an online community by the nature of the communication which compels students to assume greater responsibility for and control over their learning.
The Role of Learner in an Online Community of Inquiry
As McLuhan observed, “each form of transport not only carries, but translates and transforms the sender, the receiver and the message” (McLuhan, 1995, p. 90). An asynchronous and collaborative learning community necessitates the adoption of personal responsibility and shared control. This goes to the heart of an online learning community and represents a significant shift from the information transmission of the lecture hall and the passive role of students. Thus, online learning communities demand role adjustments. This brings another need: to understand changes in responsibilities and roles. Differences in the required activities of online learning, in comparison to classroom based faceto-face, result in new, required expectations and behaviors for learners. These new activities cluster into a pattern that is seen as the ‘role’ of online learner. The term role refers to the expected and generally accepted ways of behaving, acting and interacting (Knuttila, 2002). Taking on a role (e.g. teacher, mother, learner) involves learning what the expected behaviors are through a process of observation and trial and error attempts at the role (Collier, 2001). While the adoption and enactment of social roles is a standard, commonplace element of everyday experience, becoming an online learner has a unique characteristic. For many learners, role models for learning the required and expected activities are not present until one is already engaged in an online course (Garrison & Cleveland-Innes, 2003). Role acquisition is part of individuation in the experience of working online. Each online learner engages in the experience of learning online and the process of role taking and role making occurs concurrently within the learning experience. From the perspective of the individual, learning online requires the development of competencies in the role of ‘online learner’. As a new social role, the pathway to competence will occur over time as the role becomes prevalent and normalized. In this early stage, online communities will contribute to the socialization process for those engaging in
this new role. The result is a new role and a new identity for learners.
METHODOLOGY Sample Students participating in this study were enrolled in two graduate programs at Athabasca University. Students came from 19 distinct course groups over four terms. Two-hundred and seventeen students from core courses normally taken early in each program, courses purposively selected in order to include the greatest number of novice online learners, agreed to participate. Of the 217 students consenting to participate, 150 returned both questionnaires; 33% male and 58% female (9% did not indicate gender). Respondents indicated their age as follows: 20-29 years – 10%, 30-39 years – 24%, 40-49 years – 43%, 50+ years – 23%. All courses were delivered using a combination of print and electronic media and online conferencing. The online conferencing component provided the opportunity for student engagement and group interaction. Required conference participation was used for assessment in some courses while it remained a voluntary activity in others.
Data Collection This study used an instrument validated by Garrison, Cleveland-Innes & Fung (2004), based on the concepts of the community of inquiry model, to measure the extent of student identification with the behaviours, expectations and requirements of the role of online learner. Identical questionnaires were sent by email during the first two weeks of each term and again during the final two weeks of each term. The questionnaire collected both quantitative and qualitative data. Quantitative data was generated from 28 Likert-type scaled responses to statements derived from the community of inquiry model. After the scaled items,
The Role of Learner in an Online Community of Inquiry
seven open-ended questions about the adjustment to online learning were presented. These questions were written and pre-tested with a select sample of students and faculty familiar with online learning. This qualitative data was analyzed and summarized in relation to the adjustment process to the role of online learner, and is the evidence used for this discussion. Forty-six percent of the participants reported this as their first experience in an online learning environment (n=73). This group is the focus for this paper, as the other respondents would have experienced their primary adjustment to online learning in previous courses. Written, detailed responses were gathered from open-ended questions related to activities and outcomes, becoming part of the online learning community, and the design and facilitation of online learning.
Data Analysis The constant comparison method was used to code responses to questions; open, axial and confirmatory coding progression was employed. Five themes emerged from this process: interaction, instructor role, self-identity, course design, and technology. Definitions of these constructs are outlined below. Interaction: Respondents identified issues such as quality, quantity, and value of dialogue with classmates and instructors, often comparing it to interaction in the face-to-face learning environment and describing their transition from verbal to written communication. Instructor Role: Respondents commented on the visibility of the instructor in the conference forums and the quantity, quality and timing of feedback. Self-identity: Respondents showed evidence of reflection on self-concept, learning style, personal needs, and increasing responsibility and ownership for learning.
Design: Respondents commented on the effectiveness of course structure and delivery and the availability of institutional support. Technology: Respondents pointed out technology issues that may affect participation in the community of inquiry and slow their adjustment to the role of online learner.
FINDINGS The purpose of this study was to assess the experiences of first-time online learners and their perceptions of the adjustment to online learning. Their responses to the open-ended questions reflect varying aspects of adjustment clustering around the emergent themes of interaction, instructor role, self-identity, course design, and technology. These themes are explored in relation to cognitive, social and teaching presence in the online environment. Cognitive Presence. Table 1 provides sample comments from first-time online learners regarding adjustments in cognitive presence in an online community of inquiry specific to each of the themes (numbers indicate respondent ID codes). Learners voiced concern regarding their adjustment to contributing to online content discussions that lack the visual cues available in face-to-face interaction. Some mentioned their fear of being misunderstood or saying something wrong. First-time online learners also reported an adjustment to assuming more responsibility for their own understanding of the material without direct instruction from the professors. Concern was voiced that without more direction from the instructor, it became necessary to rely on fellow students for interpretation, and this could lead to uncertainty or dissatisfaction with learning outcomes. Several learners commented that their participation in online discussions was greater than in a traditional classroom where they
The Role of Learner in an Online Community of Inquiry
Table 1. Adjustment in cognitive presence Interaction
At first, I hesitated in fear of saying something wrong (similar feelings in F2F situations as well). However, after receiving feedback from other colleagues, the online conference engagements became enjoyable and valuable from a learning outcome perspective. (#195)
Instructor Role
I have found that it is more difficult to be sure that you understand the material in the online learning because there is little discussion with the prof. The prof seems to set up the lecture and then let us talk amongst ourselves with no interaction to let us know if we are on the right track. (#331)
Self-identity
I feel that I don’t have as much to offer as others, either because I have had a more limited scope of learning or life experiences or because I can be intimidated by huge thoughts from bright people. (#250)
Course Design
Gaining equal participation and a common understanding in group work was a challenge. At the same time, it led to bonding between some group members. Group assignments early in the class helped to get us started quickly. (#335)
Technology
I like asking questions, but I rarely do on-line. I like clarifying things, but I rarely do on-line. I like to participate in class, but I’m a slow typist, so I rarely do on-line. (#390)
were often shy and reluctant to speak up, while others reported a feeling of intimidation when they perceived that classmates had a greater understanding of the concepts or dominated the forum discussions. Social Presence. In terms of social presence, first-time online learners expressed a need for time to feel comfortable communicating in a text only environment and to adjust to expressing emotion and communicating openly in an environment that lacks visual or other non-textual cues that provide context to communications in a face-to-face setting. Some appreciated opportunities to connect with one or a few other learners in a small group activity while others found this difficult to manage, particularly when one group member was domi-
nant in the group. One learner expressed concern that “This mode of learning, however convenient for my schedule, is dehumanizing the learning process for me, and I’m not sure I’m happy with that” (#279). Sample comments on adjustment in social presence in an online community of inquiry are shown in Table 2. Teaching Presence. Table 3 includes sample comments from first-time online learners regarding their adjustment to a changed teaching presence from past experience in face-to-face learning environments. Many indicated that a more visible teaching presence at the very beginning is desired to ease the adjustment from traditional learning environments to the online environment where the instructor is more of a facilitator and guide. Some
Table 2. Adjustment in social presence Interaction
I did notice my emotional, social ability to communicate became easier and I felt more relaxed as the course progressed. (#197)
Instructor Role
The only aspect (once again) that I found challenging was that I didn’t really feel that I got a sense of ‘knowing’ the instructor, nor did he really get to ‘know’ me. (#421)
Self-identity
I find that I am much more open and interactive on-line than I am in person… I am not able to “hide in the corner” as I could in a live class. (#58)
Course Design
I found the use of small working groups to be a positive way of getting people to interact with one another, allowing me to project myself as a “real” person. . .It may be tougher to do this in the context of the larger class (i.e. those in other working groups.) One does not have the same degree of back and forth “organizational” communication with these other people. I think I may be less of a three dimensional person to these other people. (#284)
Technology
Not ever having learned how to type may also be a factor as I [consider it] to be a handicap the same way someone who has difficulty expressing themselves verbally would. (#37)
The Role of Learner in an Online Community of Inquiry
Table 3. Adjustment to teaching presence Interaction
Once we were comfortable with his role as more of a guide and facilitator than an omni-present being, we were able to take more ownership for our role in the program and for our own investment in the course. (#407)
Instructor Role
I’m certain that [the professor] reviewed the discussion threads regularly but he seemed more like the virtual “fly on the wall” than an active participant. (#204)
Self-identity
I personally felt that a little more input and guidance from the instructor might have removed some anxiety and stimulated some interaction on my part. (#197)
Course Design
I think the instructor needs to be a very active participant at the beginning of the course. Everyone seems eager to talk to each other at the beginning (how many times did I log in on the first day to see if there was anything new?), and the instructor should tap into that by starting to focus that energy on the content. (#211)
Technology
Most emails sent by my instructor disappeared and I did not know what I had to do. (#146)
reported that they adjusted by assuming more responsibility for their own learning outcomes while others expressed concern that the learners were left to discuss content on their own without assistance from the instructor to let them know if they were on the right track. All three types of presence addressed in the online community of inquiry model are evident in the responses from new online learners. Adjustments were demonstrated in the identification of things that were unexpected or new, and the response to that newness. Five components of the online environment emerged as themes in the adjustment process, separately for all three presences. Performing the first analysis, an interesting pattern was noted in the data. Respondents provided enough detail to demonstrate the process of adjustment in some answers. A second analysis was performed, using pre-identified codes of challenge, intervention and resolution. This coding structure identified any challenges reported in each presence, the intervening action on the part of student or others, and the result that followed. Constant comparison, with inter-rater confirmation, was employed as a process. Specific challenges and the resolution of these challenges provide another window on the adjustment process. Challenges are identified as those things students find difficult, uncomfortable or in any way problematic regarding the online learning environment. Interventions are any occurrence
that ensued after the challenge, either deliberately or incidentally. Resolution refers to what the students describe as happening after. In some cases, the result was a positive one. In other cases, what happened resulted in an unsatisfactory outcome. Table 4 provides examples of the adjustment process, by presence.
DISCUSSION It is clear that an adjustment is taking place for students engaging in online learning, and that students can articulate the processes of this adjustment when asked. Using the community of inquiry model provided a valuable heuristic device for organizing the character of this adjustment. In each element of presence, the same five themes identified areas where change is taking place. Evident in student descriptions of their experience is a unique orientation to five thematic areas, which together embody the experience of being an online learner. These five thematic areas do not act in isolation. Consider that technology use occurs within a particular course design and is more or less optimized by the role of the instructor. WebCT, for example, provides the opportunity to chat synchronously; if the course design doesn’t require it, or the instructor doesn’t provide time to use it, students may or may not experience this technological opportunity. As another example,
The Role of Learner in an Online Community of Inquiry
Table 4. Adjustment process by presence Challenge Cognitive presence
At first, I hesitated in fear of saying something wrong
I have found that it is more difficult to be sure that you understand the material online ….
Intervention
Result
… after receiving feedback from other colleagues …
…. online conference engagements became enjoyable and valuable from a learning outcome perspective. (#195)
…. the prof seems to set up the lecture and then let us talk amongst ourselves …
I feel that I don’t have as much to offer as others, either because I have had a more limited scope of learning or life experiences or because I can be intimidated by huge thoughts from bright people. (#250) Gaining equal participation and a common understanding in group work was a challenge … I like asking questions, I like clarifying things, I like to participate in class, but I’m a slow typist Social presence
My emotional, social and ability to communicate
.. .led to bonding between some group members …
…. with no interaction to let us know if we are on the right track. (#331)
(such that) group assignments early in the class helped to get us started quickly. (#335)
…. so I rarely do on-line. (#390)
as the course progressed
. became easier and I felt more relaxed (#197)
The only aspect (once again) that I found challenging was that I didn’t really feel that I got a sense of ‘knowing’ the instructor, nor did he really get to ‘know’ me. (#421) I am not able to “hide in the corner” as I could in a live class.
…. I find that I am much more open and interactive on-line than I am in person. (#58) Small working groups to be a positive way of getting people to interact with one another …..
allowing me to project myself as a “real” person. (#284)
Not ever having learned how to type may also be a factor as I [consider it] to be a handicap the same way someone who has difficulty expressing themselves verbally would. (#37)
continued on following page
The Role of Learner in an Online Community of Inquiry
Table 4. continued Teaching presence
Instructor role as more of a guide and facilitator than an omni-present being
was something we had to get more comfortable with
Once we were, we were able to take more ownership for our role in the program and for our own investment in the course. (#407)
a little more input and guidance from the instructor
might have removed some anxiety and stimulated some interaction on my part. (#197)
I think the instructor needs to be a very active participant at the beginning of the course.
Everyone seems eager to talk to each other at the beginning (how many times did I log in on the first day to see if there was anything new?), and the instructor should tap into that by starting to focus that energy on the content. (#211)
Most emails sent by my instructor disappeared.
I did not know what I had to do. (#146)
interaction can be fostered or hindered by the instructor’s ability to invite learners to participate, the learners’ sense of competence regarding the presentation of ideas in print, the technological possibilities for interaction, and the design of the course in question. In other words, all five themes can be examined separately as they relate to online learning, but must be considered in comprehensive relation to each other if we are to illuminate the student experience online.
This is also the case as we examine all five themes in each area of presence. What emerges is a multidimensional perspective that must guide thinking as we design online courses to engage and support students as they adjust to, move into and become competent performing in the online community of inquiry. Technology, for example, has a unique role to play in each of cognitive, social and teaching presence. Social presence becomes possible as learners use the technology
Figure 2.
Community of Inquiry Social Presence
Interaction Challenges Interventions Results
10
Cognitive Presence
s elf Identity Challenges Interventions Results
Instructor Role Challenges Interventions Results
Teaching Presence
c ourse Design Challenges Interventions Results
t echnology Challenges Interventions Results
The Role of Learner in an Online Community of Inquiry
to present themselves as individuals through the written word or verbal language. This use of the technology overlaps with, but is unique from cognitive presence, where intellectual reasoning is presented as a portion of individual identity. Teaching presence emerges where the technology allows for presentation of material, directions from self to others and the interpretation of material. The importance of teaching presence is demonstrated in differences noted across instructors. Evidence here affirms Garrison & ClevelandInnes’ (2005) premise that “teaching presence must be available, either from the facilitator or the other learners, in order to transition from social to cognitive presence.” (p. 16). Without adequate support from the instructor, adjustment occurs without a clear point of reference to expectations. This lack may create a situation fails to sustain interest and engagement. This supports previous evidence that learners without guidance operate remotely: “without instructor’s explicit guidance and ‘teaching presence,’ students were found to engage primarily in ‘serial monologues’” (Pawan, et al., 2003, p. 119). Teaching presence supports sustained, beneficial academic interaction, movement within the presences of online community and, for first-time online learners, points of reference regarding expectations in the adjustment to the online environment. Specific challenges and the resolution of these challenges demonstrate places where students are not prepared, and must respond or change to manoeuvre the online environment. Challenges are any requirement that learners find difficult, uncomfortable or in any way problematic regarding the online learning environment. Interventions ensued after the challenge, deliberately or incidentally. Resolution happens afterwards, in some cases but nor all. Challenges may resolve themselves (#195), require intervention from the instructor (#331) or remain a challenge (#390); the latter, in particular, need attention by instructional designers or instructors.
CONCLUSION AND RECOMMENDATIONS The adjustment process to the characteristics and requirements of online learning is not merely a matter of comfort or student satisfaction. It has practical and pedagogical implications as well. Much research demonstrates the authenticity of social, cognitive and teaching presence online (see, for example, Meyer, 2003; Shea, Pickett, & Pelz, 2004 and Swan, 2003). These elements are unique to the medium and will require established roles for learners and instructors. Competent online learners are essential to creating community and contributing to higher-order learning activities. “Balancing socio-emotional interaction, building group cohesion and facilitating and modeling respectful critical discourse are essential for productive inquiry” (Garrison, 2006, p. 7). Evidence supports the premise that students experience a dynamic adjustment to the role of online learner, made up of particular ways of behaving, acting and interacting (Knuttila, 2002). As the role of online learner is still undefined, students grapple with requirements, looking to their own reasoning, other students and the instructor for direction about the right things to do. Adjustments occur in all three areas of presence, and each presence is both constrained and enabled by course design, technology, the instructor, personal self identity and the interaction within the community. Attention to these online elements in relation to the ‘getting up to speed’ or adjustment for learners, each time they join an online community, will smooth this move to competence. In order to become present in the important functionalities of an online community of inquiry, adjustment must occur. Without adjustment to competence as an online learner, the learning process may be hindered. Support for students to move to a place of comfort and sense of competence is of value. Based on the comments of first-time online learners describing their adjustment to the online
11
The Role of Learner in an Online Community of Inquiry
community of inquiry in terms of cognitive presence, social presence, and teaching presence, we recommend the following be incorporated into the instructional design and delivery of online courses in order to ease the adjustment to the role of online learner and enhance the elements that contribute to an effective community of inquiry. •
•
•
•
12
Acknowledge and make explicit the initial adjustment process, and provide a venue to identify challenges, consider interventions and ensure resolution. Professional development opportunities for instructors should focus on techniques for easing learner adjustment to the online learning environment. Instructors should learn to recognize indicators of adjustment and be prepared to suggest appropriate support services if required.. Provide ample opportunity for those unfamiliar with the technology to gain skills and feel comfortable so they don’t feel they are at a disadvantage vis à vis their classmates. An orientation conference forum moderated by experienced online learners and facilitators should be available well in advance of the course start date. Encourage participants to separate content related dialogue from socializing. This can be accomplished by providing a café-style conference for those who enjoy chatting while those who have limited time or prefer not to socialize can focus on content. Include greater instructor involvement at the beginning of introductory level courses, but judiciously thereafter. This will ease the adjustment for first-time online learners as they assume greater responsibility for meeting learning outcomes, gain comfort in contributing to discussions often dominated by more experienced online learners, and become more confident in their new role
•
•
•
without the immediate feedback from the instructor that occurs in the face-to-face classroom. Too much online intervention by the instructor can be intimidating and may decrease engagement. Request that participants limit the length of their conference postings to one screen and insist that discussions remain focused on content other than in the online café. Time is an issue for distance learners, and adjusting to the role of online learner includes learning to balance that role with others in adult life including work, family, and community. Provide the opportunity to establish group rules of netiquette. Ask that individuals to make explicit his or her strategies for use of emoticons and other expressions so everyone has access to the same tools. Request, through private email, that learners dominating online dialogue limit their postings to avoid intimidating novice conference participants. At the same time, private email can be used to encourage the less active participants to contribute. One effective technique is to include learners’ names in conference postings and replies in order to draw them further into the conversation. This is the equivalent of being called on in a face-to-face classroom.
Changed practice implies role adjustment for the instructors as well as the learners. Professional development activities that focus on the affective components of course delivery will enable instructors to ease the adjustment of the learners to online learning as well as increase their own comfort level and effectiveness.
Limitations Participants in this study were new to online learning, but most were new to graduate study as well. The adjustment to online learning would occur in conjunction with adjustment to graduate study;
The Role of Learner in an Online Community of Inquiry
this needs to be considered in interpretation of findings. In addition, awareness of the requirements of online learning may have been created by completing the pre-questionnaire. This may have affected the adjustment process and student response to it.
FuTUure RESEear This research program continued with a review of conference transcripts. This demonstrated adjustment to conference behaviour from course commencement to completion. This view of student activity provided general confirmation that what students say happened did happen. Further research will clarify the stages of adjustment for first-time online learners, and similarly for experienced online learners each time they begin a new course. Challenges and appropriate interventions in each of social, cognitive and teaching presence must be made explicit through research such that responses may be recommended. These responses will ultimately identify what must be in place to ensure complete and competent engagement for online students. It is this latter understanding that is a critical conclusion to this work on learner adjustment in online environments.
REFERENC Allen, I. E., & Seaman, J. (2004). Entering the mainstream: The quality and extent of online education in the United States , 2003 and 2004. Needham , MA : Sloan-C. Retrieved December 4, 2005, from http://www.sloan-c.org/resources/ entering_mainstream.pdf Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5(2), Retrieved
September 2005 from: http://www.aln.org/alnweb/journal/jaln-vol5issue2v2.htm Arbaugh, J. B., & Hwang, A. (2006). Does “teaching presence” exist in online MBA courses? The Internet and Higher Education, 9(1), 9-21. Blau, J.R., & Goodman, N., (Eds.) (1995). Social roles & social institutions. New Brunswick: Transaction Publishers. Collier, P. (2001). A differentiated model of role identity acquisition. Symbolic Interactionist, 24(2), 217-235. Dewey, J. (1933). How we think (rev. ed.). Boston: D.C. Heath. Dewey, J. (1938). Experience and education (7th printing, 1967). New York: Collier. Garrison, D. R., (2006). Online community of inquiry review: Understanding social, cognitive and teaching presence. Invited paper presented to the Sloan Consortium Asynchronous Learning Network Invitational Workshop, Baltimore, MD, August. Garrison, D. R., & Anderson, T. (2003). E-Learning in the 21st Century: A framework for research and practice. London: Routledge/Falmer. Garrison, D. R., & Arbaugh, J. B. (2007). Researching the community of inquiry framework: Review, issues, and future directions. Internet and Higher Education, 10(3), 157-172. Garrison, D. R., & Archer, W. (2000). A transactional perspective on teaching-learning: A framework for adult and higher education. Oxford, UK: Pergamon. Garrison, D.R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. Internet and Higher Education, 11(2), 1-14. Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence and computer
13
The Role of Learner in an Online Community of Inquiry
conferencing in distance education. American Journal of Distance Education, 15(1), 7-23.
Katz, D., & Kahn, R. (1978). The social psychology of organizations. New York: Wiley.
Garrison, R., & Cleveland-Innes, M. (2005). Facilitating cognitive presence in online learning: Interaction is not enough. American Journal of Distance Education, 19(3), 133-148.
Knuttila, M. (2002). Introducing sociology: A critical perspective. Don Mills, Ontario: Oxford University Press.
Garrison, D.R., & Cleveland-Innes, M. (2003). Critical factors in student satisfaction and success: Facilitating student role adjustment in online communities of inquiry. In Elements of Quality Online Education: Into the Mainstream, Vol. 4 in the Sloan-C Series, J. Bourne and J. Moore (Eds.), 29-38. Needham, MA: Sloan-C. Garrison, R., Cleveland-Innes, M., & Fung, T. (2004). Student role adjustment in online communities of inquiry: Model and instrument validation. Journal of Asynchronous Learning Networks, 8(2), 61-74. Retrieved December 2004 from http://www.sloan-c.org/publications/jaln/ v8n2/pdf/v8n2_garrison.pdf. Ice, P., Arbaugh, B., Diaz, S., Garrison, D. R., Richardson, J. Shea, P., & Swan, K. (2007). Community of Inquiry Framework: Validation and Instrument Development. The 13th Annual Sloan-C International Conference on Online Learning, Orlando, November. Kanwar, M., & Swenson, D. (2000). Canadian sociology. Iowa: Kendall/Hunt Publishing Company. Kendall, D., Murray, J., & Linden, R. (2000). Sociology in our times. (2nd ed.). Ontario: Nelson Thompson Learning.
14
Kopp, S. F. (2000). The role of self-esteem. LukeNotes, 4(2). Retrieved September, 2005 from http://www.sli.org/page80.html McLuhan, M. (1995). Understanding media: The extensions of man. Cambridge: The MIT Press. Meyer, K. A. (2003). Face-to-face versus threaded discussions: The role of time and higher-order thinking. Journal of Asynchronous Learning Networks, 7(3), 55-65. Rogers, P., & Lea, M. (2005). Social presence in distributed group environments: The role of social identity. Behavior & Information Technology, 24(2), 151-158. Shea, P., Pickett, A., & Pelz, W. (2004). Enhancing student satisfaction through faculty development: The importance of teaching presence, Elements of Quality Online Education: Into the Mainsteam, Needham, MA: SCOLE (ISBN 0-9677741-6-0). Swan, K. (2003). Developing social presence in online discussions. In S. Naidu (Ed.), Learning and teaching with technology: Principles and practices (pp. 147-164). London: Kogan Page. Turner, J. (1990). Role change. Annual Review of Sociology, 16, 87-110. Wilson, D., Varnhagen, S., Krupa, E., Kasprzak, S., Hunting, V. & Taylor, A. (2003). Instructors’ adaptation to online graduate education in health promotion: A qualitative study. Journal of Distance Education, 18(2), 1-15.
15
Chapter II
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning Xinchun Wang California State University, Fresno, USA
Ab Although the pedagogical advantages of online interactive learning are well known, much needs to be done in instructional design of applicable collaborative learning tasks that motivate sustained student participation and interaction. In a previous study based on a Web-based course offered in 2004, Wang (2007) investigated the factors that promote sustained online collaboration for knowledge building. By providing new data from the same Web-based course offered in 2006 and 2007, this study investigates students’ attitudes toward process- and product-oriented online collaborative learning. The analysis of 93 post course survey questionnaire data show that the overwhelming majority of students have positive experience with online collaborative learning. Data also suggest that students are more enthusiastic about process-oriented tasks and their attitudes toward product-oriented collaborative learning tasks are mixed.
INTRODUCTION The pedagogical advantages of student interaction in collaborative construction of knowledge are grounded in the social constructivist perspective of learning. From the social constructivist
perspective, all learning is inherently social in nature. Knowledge is discovered and constructed through negotiation, or collective sense making (Duin & Hansen, 1994; Kern, 1995; Wang & Teles, 1998; Wu, 2003). Pedagogically sound tasks in an online learning environment should, therefore, reflect social learning and facilitate
Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
interactive learning and collaborative construction of knowledge.
Interactive Learning and Factors Influence Online Collaboration From a student’s perspective, online interaction in learning takes place at two different levels: interaction with the contents, including interactive computer software and multimedia system, and interaction with instructors and between peers (Evans & Gibbons, 2007; Gao & Lehman, 2003). There is evidence that pedagogically well-designed interactive learning tasks actually increase rather than decrease student access to instructors; increase interactions between instructors and among students; and increase students involvement of course content as well (Lavooy & Newlin, 2003; Mouza, Kaplan, & Espinet; 2000; Wu, 2003). Interactive learning tasks also promote greater equality of participation (Mouza, Kaplan, & Espinet, 2000), more extensive opinion giving and exchanges (Summer & Hostetler, 2002), empower shy students to participate, and promote more student-centered learning (Kern, 1995; Wang & Teles, 1998) At the level of interaction with content, students benefit more from producing explanations than receiving explanations. Such proactive learning engages students in a higher level of thinking than the reactive type of learning (Gao & Lehman, 2003; Wu, 2003). Additionally, students who reported high levels of collaborative learning in an online course tend to be highly satisfied with their learning and they also tend to perceive high levels of social presence in the course (So & Brush, 2007). Despite these advantages, research also indicates that online interactive learning and collaboration are not always sustainable and students’ participation in Computer Mediated Communication (CMC) tasks may wane after the assessed tasks that require the postings are completed (Macdonald, 2003). In a survey on
16
college student’s attitudes toward participation in electronic discussions, Williams & Pury (2002, p.1) found that “contrary to much literature on electronic collaboration suggesting students enjoy online collaboration, our students did not enjoy online discussion regardless of whether the discussion was optional or mandatory.” Like any other form of learning, learning collaboratively in an online course is also characterized by individual differences. Collaboration as a process of participating to the knowledge communities is not an equal process to all the members of the community (Leinonen, Järvelä, & Lipponen, 2003). Much needs to be done to explore factors that promote sustained student interest in online interactive learning and collaboration. One challenge for developing sustainable online collaborative learning tasks lies in the nature of the CMC system itself. Although CMC supports interaction and collaborative learning, it also has inherent shortcomings. Disadvantages include the time it takes to exchange messages and the increased difficulties in expressing ideas clearly in a context reduced learning environment and the difficulty in coordinating and clarifying ideas (Sumner & Hostetler, 2002). The increased time it takes to reach consensus and decisions (Kuhl, 2002; Sumner & Hostetler, 2002) and to produce a final product (Macdonald, 2003). Given all these difficulties students need to overcome in order to collaborate effectively in interactive learning environment, online instructors need to address these obstacles with careful instructional design and provide support for collaborative learning with appropriate interactive learning tasks. Research has also shown that computer mediated communicative tasks require more active role of students than traditional instruction in the face-to-face environment does (Wang & Teles, 1998). Students need to be willing to send a formal written question rather than have a casual conversation with peers or with the instructor in order to have their questions answered (Kuhl, 2002). To communicate effectively with peers
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
and the instructor, students need to create the context through written messages, which requires the writing skills to identify their problems and express them precisely in order to have the questions answered. Team work and negotiation for meaning are necessary skills in CMC that cannot be assumed. Students need to learn to be familiar with the discourse of the discipline and academic genre for an online synchronous and asynchronous forum (Kuhl, 2002; Macdonald, 2003). In addition to negotiation skills online, previous research has identified a number of other factors that influence student participation and interaction in a web-based learning environment. Among others, the assessment of collaborative learning tasks plays a crucial role in ensuring student participation (Kear, 2004; Kear & Heap, 1999; Macdonald, 2003) and it directly influences the level of participation (Wang, 2007). In general, assessed collaborative learning tasks attract student participation at the cost of unassessed tasks. Furthermore, grade for discussion was also positively related to students’ perceived learning (Jiang & Ting 2000). The structure of discussion in CMC is found to be another important factor in ensuring the amount of participation and level of interaction and collaboration among the peers. Such structure includes the size of the discussion groups, the nature and types of discussion topics (Williams & Pury, 2002), and whether the collaboration emphasizes on the process of learning or the end product of such collaboration, or both (Kear, 2004; Kear & Heap, 1999; Macdonald, 2003, Wang, 2007). Research also indicates that student facilitators play an important role in attracting their peers’ participation in online group discussions and the success of such roles are closely related to the depth of the discussion threads that often lead to more than six or more rounds of student postings (Hew & Cheung, 2007). The interaction level between the students and the teacher and among the students was found to be a significant factor
in determining the effectiveness of the teaching method (Offir, Lev & Bezalel, 2007). In addition, class size and level of participation in terms of note writing and reading is also found to be related. Data show that large classes are related with an increase in the number of notes written, a decrease in average note size, and an increase in note scanning rather than reading (Hewitt & Brett, 2007). To summarize, although the pedagogical advantages of online collaborative learning are commonly recognized, such learning is sometimes difficult to sustain due to a number of factors. Among others, online negotiation skills, the direct link between collaborative tasks and assessment, the structure of online discussions such as the nature and types of discussion topics, the size of the group, and the differences between process and product oriented collaborative tasks are some of the factors that influence student participation, interaction, and collaboration.
Process and Product Oriented Online Collaborative Learning Online collaboration can be either process or product oriented. Forum discussions regarding course contents or related issues are commonly process oriented as the sharing of ideas help learners understand the issues without necessarily leading to a final product. Students are assessed individually based on their participation and quality of their contributions. Alternatively, online interaction and collaboration may lead to a final product such as an essay, a project, or a webpage, etc. There can be two assessment elements to such tasks, a common grade for the group for the overall quality of the collaborative product and individual grades for the contribution of each individual to the collaborative endeavor (Kear, 2004; Kear & Heap, 1999; Macdonald, 2003). The similarities and differences of process and produce oriented online collaborative learning tasks are summarized in Table 1.
17
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
Table 1. Similarities and differences between process and product oriented online collaborative learning tasks Process oriented tasks
Product oriented tasks
Exchange of views to share ideas that may or may not lead to agreements
Exchange of views that are consensus building to reach agreements
No end product
End product: a project, report, etc.
Relatively easy to interact and share views
Difficult to reach agreement by a time line
Individual grade
Common and/or Individual grade
For a product oriented collaboration, simply assigning learners to work on a group project does not necessarily mean that they will work collaboratively. Learners tend to use a task specialization approach where tasks are divided among group members. Therefore, learners may not look for opportunities to develop mutual engagement, knowledge and skill exchange, and interpersonal communication skills (So & Brush, 2007). Therefore, the instructional design of product oriented online collaborative learning tasks need to take measures to ensure real collaboration among peers.
The Study In a previous study based on a web-based course offered in 2004, Wang (2007) investigated the factors that promote sustained online collaboration. The current study extends the previous study by providing new data from the same web-based course offered in 2006 and 2007. I addition to investigating the factors that promote sustained collaborative learning in a web-based course using asynchronous conferencing system as its main learning tasks (Wang 2007), this study further investigates the students’ attitudes toward collaborative learning. In particular, it investigates the learners’ perspectives of process and product oriented online collaborative learning tasks. Both types of interactive learning activities were implemented as the main learning tasks of the web-based course under study.
18
The research questions are: 1.
2.
3.
What are students’ overall attitudes toward collaborative learning as the main learning tasks in a web-based course? Are there any differences in student attitudes toward process and product oriented online collaborative learning tasks? If so, what are the factors that influence students’ different perspectives toward such tasks? What pedagogical implications do the findings have?
COURSE INFORMATION AND DATA CLLECTION Course Information The course under study is an upper division general education course on Bilingualism and Bilingual Education delivered entirely on Blackboard at a state university in California. In a previous study, (Wang 2007), data were collected from a post course questionnaire in Spring and Fall 2004 semesters when this web-based course was first offered. A total of 60 students, 22 in the Spring Semester class, and 20 and 18 students in the two Fall Semester classes completed the course. A total of 53 of the students completed the post course survey questionnaire and the results were reported in the previous study (Wang, 2007). The current study further investigates students’ atti-
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
tudes toward online collaborative learning with additional 93 survey questionnaires completed by the students who took the same course during the Fall 2006 and 2007 semesters. Course structure remained unchanged for the most part. Forum discussions on course readings and related issues formed the core interactive learning activities that were 45% of the course grade. These were process oriented interactive learning tasks for which individual grades were assigned for each student based on their quantity and the quality of postings in the forums. Small groups of 4-6 people were formed at the beginning of the semester for the weekly asynchronous group forums. During the 16 week semester, a total of 18 discussion forums were completed in each online group. For each forum, the instructor assigned a reading chapter along with comprehension questions and discussion topics to help the students to grasp the contents. Students divided the reading questions among themselves in their groups and posted the answers to each question for the first round of postings. They were also required to make comments on at least one peer’s answers in the second round of postings to carry on the discussions. To ensure participation, strict deadlines for each round of postings were enforced and each student’s answers to the questions and comment messages were assessed by the instructor who assigned up to 3% of the course grade for participation of each discussion forum. The other major collaborative task was a product oriented group project that constituted 12% of course grade for which all the students in the same group received a common grade based on the level of collaboration and the quality of the final written report. There was no individual assessment component for the group project. The interdependent grading (a common grade for all members of a group only) was aimed at promoting more collaboration among the peers to produce a true collaborative product with individual contributions. The group project was closely related
to one of the course themes on types of bilingual education programs. Each student was required to visit a local school to interview a bilingual teacher to gain first hand information about bilingual education programs implemented in California. Students then shared and synthesized the interview data to produce a group report. They were not required to meet face-to-face for the group project but they exchanged information in an online forum that was mostly procedural to plan, negotiate, to reach agreement and to produce the final product. The process of planning and producing the project required negotiation, cooperation, and collaboration among peers to actually arrive at consensus to produce a report. Though not graded, the progress of each group in the online forums was closely monitored by the instructor. The only deadline for submitting the group project was imposed to ensure the completion of the work for the first two semesters when the course was offered in 2004 (Wang, 2007). During Fall 2006 and 2007 semesters when the course was offered (two classes for each semester), strict intermittent deadlines for each step (such as the completion of the interview, the posting of interviews in the Group Forums, and the completion of the first draft of the group project) were enforced in addition to the final deadline for the submission of the Group Project. These measures were taken to ensure that the “slow” students must stay on the course and complete their tasks according to the schedule at every step. Other course activities included two individual written assignments (8%) and three online exams (35%) that assessed the learning outcomes of the course readings and group discussions. Table 2 summarizes the course activities and grading.
Data Collection: Post Course Survey Data
At the end of the semester, an online survey was administered in each class to collect information about students’ learning experience and
19
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
Table 2. Course activities and grading Activities
Grading
Weekly group forums
45%
Structured discussions on course readings
Weekly class forums
0%
Required postings of moderator’s summaries from each weekly group forum (Spring Semester class only)
Group project
12%
Final product graded interdependently (same grade for each member of the group)
Individual assignments
8%
No interaction among students required
Three exams
35%
Online exams on course contents to assess outcome of learning
their attitudes toward the course, in particular, their experience with online collaboration in both the weekly conference discussions and the group project. The survey questionnaire, which consisted of 17 multiple choice questions and 4 open-ended questions (see Appendix) was uploaded to the survey area of the course on Blackboard. Students were able to access and complete the survey questionnaire anonymously during the week after the final exam. Blackboard automatically calculated the results of the multiple choice questions in percentage. The transcripts of the survey responses for all three classes were printed out for analysis. Ninety – three students completed the survey questionnaire. Therefore, the analysis of the survey data is based on the 93 completed questionnaires. These new data were also analyzed along with the 53 survey questionnaire data reported in Wang (2007) study that were based on the same course offered in the Spring and Fall 2004.
Description
RESLTS Students’ Attitudes Toward Cllaborative Learning Table 3 presents students’ responses to the question “what are your thoughts about the structure of the course?” Students took this web-based course overwhelmingly preferred the interactive learning structure of the course to the weekly quizzes if they were given the choices. Although the percentage of students who preferred weekly quizzes based on the readings increased from 8% in 2004 to 24% in 2006 and 2007, the overall majority of them still preferred the current course structure. An additional question about the learners’ overall experience with this web-based course was included in the survey for the 2006 and 2007 classes and the results are summarized in Table 4. Ninety-eight percent of the students reported that their experience with this web-based course was “very positive” and “positive”. Therefore, even
Table 3. Students’ responses to “what are your thoughts about the structure of the course?” Choices
20
2004, N= 53
2006-2007, N= 93
I like the way the course is structured in terms of forum discussions because we learn from each other.
92.5%
76%
I prefer weekly quizzes based on the readings rather than answering questions and joining the group discussions.
7.5%
24%
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
Table 4. Students’ responses to the question “my experience with this web-based course:” (N=93) Choices
2006-2007, N=93
Is very positive
53%
Is positive
44%
Is negative
1%
Is very negative
1%
though about a quarter of the students preferred ‘weekly quizzes’ type of less interactive learning if there were given the choices, they still reported that their overall experience with the course was “positive” and “very positive”. What factors encouraged students to participate in this form of active and interactive learning throughout the semester? Did the students really think that they learned from building on each other’s insights? What were the effects of such learning as reflected by students’ responses in the survey data? The survey questionnaire addressed these issues in a number of questions. Table 5 summarizes students’ responses to the effectiveness of group discussions. Chi Square analyses of students’ responses to the questions in Table 5 along the scale of strongly agree to strongly disagree were all sig-
nificant beyond 0.0001 level. About 90% of the students agreed or strongly agreed that answering questions and participating in discussions helped them understand the readings better and that online discussion was helpful because they collaborated more and learned more from each other. Additionally, 72% of the students responded that they learned more from online discussions than they would have learned from the lectures. Furthermore, 89% of the students responded that group cohesion and mutual trust was an important factor in their group.
Level of Participation Students’ participation in group discussions was not only required but was also directly linked to the assessment of their postings in the forums. The
Table 5. Students’ views about group discussions (N=53) Survey Questions
% Responses Strongly agree
Agree
Disagree
Strongly disagree
Chi²
My answers to the questions and comments on peers’ messages help me to understand the readings better.
30%
62%
8%
0%
49.717
My peers’ answers/comments helped me understand the readings better.
32%
57%
11%
0%
39.453
I learned more from online discussions than I would have learned from lectures.
25%
47%
25%
2%
21.792
The online discussion is helpful because we collaborate more and learn from each other more.
38%
55%
6%
2%
41.415
The group cohesion and mutual trust is an important factor in our group.
53%
36%
11%
0%
36.132
21
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
required postings and their assessment appeared to play an important role in motivating the students to participate the discussions. Table 6 summarizes students’ responses to the level of participation in their group discussions if the postings were not required and graded. The new data from the 2006 and 2007 classes were more or less the same as the previous data when the course was offered for the first time in 2004. Overall, 45% -50% of the students responded they would post some but not as many messages, 21% said they would post very few and 6%-8% responded they would not post any messages at all! Only 21% -28% (an increase of 7% in the new data) responded they would post the same number of messages. One might arguer that the survey data may not reflect the real level of participation in discussions if the postings were not required or assessed because all the postings in this course were actually required and assessed. Therefore, a firm claim of the effect of assessment on forum contributions must be tested with a treatment group whose postings in forums were assessed and compared with a
control group whose postings in forums were optional and unassessed. Nevertheless, students’ responses to this survey question still reflect the “if not” situation because they had just completed the weekly postings for the entire semester and such learning experience would certainly affect their responses. Therefore, the “if not assessed” situation was contrasted against the real situation of “assessed’ postings. In addition to the number of postings, the amount of time spent on reading peers’ messages in an asynchronous forum also reflects the level of participation of collaborative learning. When the course was offered in 2006 and 2007, the survey questionnaire asked an additional question about the reading of peers’ messages in the forums, also an important indication of interactive learning. The results are summarized in Table 7. Thirty-seven percent of the students responded that they read all the messages posted by the peers while 47% of the students read most but not all the messages. Adding the answers to these two items up, the overall majority of the students, 84%, read
Table 6. Students’ responses to “would you post the same number of messages as you actually did over the semester if these postings were optional, not required or graded?” Choices
2004, (N= 54)
2006-2007, (N=93)
Yes, I will post the same number of messages
21%
28%
I will post some messages but not as many
51%
45%
I will post very few messages
21%
21%
I will not post any messages
8%
6%
Table 7. Students’ responses to “In our group discussions,” (N=93) Choices I read all the messages posted by my peers
37%
I read most but not all the messages by my peers
47%
I read some but not all the messages
10%
I do not read these messages often
22
2006-2007, N=93
6%
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
most if not all the postings in their groups. Still, 10% of the students read only some messages while 6% of them responded that they did not read these messages often. Unlike the required postings by the deadlines in an asynchronous forum, it is relatively more difficult to enforce the reading of each posting in such learning environment. Therefore, the request to make comments on peers’ messages for a required second round of posting is the only direct measure to enforce the reading. Still, 16% of the students read only some of the messages. It is very likely that these students chose to read one or two peers’ postings on which they made comments. Previous studies have examined the relationship between group size and the amount of reading of postings of online asynchronous forums. It is important to point out that group size and number of postings in a forum may have a direct impact on the number of messages the students actually chose to read. The students’ self-reported amount of reading of messages will be further discussed in the next session.
Group Formation
Table 8 summarizes students’ responses to the question on group formation. The 2004 data were based on the 37 questionnaires from the Fall semester students as the Spring 2004 semester course did not have this question in the survey. Not much noticeable differences were observed
from the new data when the course was offered in 2006 and 2007. There is a slight increase of percentage of students, from 8% to 15%, who wanted to work with different people in a group every few weeks. Almost the same percentage of students, 30% vs. 31% for the two sets of data, responded that it made no difference for them to work with the same or different people in a group throughout the semester. More than half other students responded that they preferred to work with the same people for their group discussions because they knew each other better and the number was even higher (62%) for the Fall 2004 class data. It appears that the group as a community for online learning established deep roots in this course. Except for some course related general forums in which questions regarding course activities were exchanged, students generally did not have access to the majority of the fellow students in their class. It would not have been surprising if students had expressed their desires to learn the discussions in other groups through some form of exchanges on a class level, or, through reshuffling groups. Yet, the survey responses suggest that at least two thirds of the students did not express the need to work outside their fixed small groups. It is important to note that the survey data reflected the students’ views towards their working groups that were fixed for the entire semester. If they actually had the chance to work in different groups in this online course, they might have
Table 8. Students’ responses to “what is your view about group formation?” Choices
Fall 2004 N=37
2006-2007, N=93
I want to work with the same group members the way it is now because we know each other better.
62%
I want to work with different people in a group every few weeks because we will learn from other students we never meet.
8%
15%
It will not make a difference to me working with the same people or different people in a group.
30%
31%
53%
23
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
Table 9. The group project about bilingual programs in our local schools Multiple choices (choose all that apply)
2004, N=53
Is a good assignment and I learned a lot through doing the project. Makes the course readings more meaningful and more relevant to me.
2006-2007, N=93
70%
48%
68%
52%
Is a good assignment but takes too much time to complete.
17%
20%
Could be an individual assignment focusing on one school rather than a group project that involves more collaboration.
30%
24%
4%
2%
Is not very important for this course.
different views. To explore the advantages and disadvantages of fixed or dynamic small groups in a web-based course that uses weekly forum discussions, both group types need to be included in the data in future studies.
Process vs. Product Oriented Collaboration Table 9 presents students’ responses to a question that allowed for multiple choices about the group project. In this “choose all that apply” multiple choice question, the first two choices were aimed at assessing whether the assignment itself was important for the course in the students’ eyes because the importance of the group project may affect their overall performance, or, vice
versa. As seen in Table 9, 70% of the students from the 2004 classes felt that the group project was a good assignment and agreed they learned a lot through doing it. Sixty-eight percent of the students also responded that the project made the course readings more meaningful and relevant. About half of the student population, 48% -52%, in the 2006 and 2007 classes viewed the group project as an important task, a noticeable decline for the previous semesters in 2004. The Fall 2004 semester post course survey asked an additional question about their experience with the group project and the responses are summarized in Table 10. (This question was not included in the Spring 2004 post course survey.) New data from the 2006 and 2007 classes were also included in Table 10.
Table 10. Students’ response to the group project Choices
2006-2007, N=93
I prefer individual work leading to a project of my own even though I only have information about one school.
32%
27%
I prefer to collaborate with peers the way it is now because it is not a problem with me to collaborate.
24%
24%
24%
24%
22%
25%
I prefer to collaborate with others for a group project but I do not like to depend on other people’s schedule because some just do not get their work done on time. Even though it is hard to collaborative for the group project, it is still worth doing it because we learn more about our bilingual programs in different schools through doing it together.
24
Fall 2004, N=37
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
As seen from Table10, the new data of 2006 and 2007 were almost identical to the Fall 2004 class, despite the fact that the number of students who perceived the importance of this group project actually went down for the students in the 2006 and 2007 classes. While 24% of the students preferred to work with peers because they had no problems to collaborate, exactly another 24% of them did not like to depend on other peoples’ schedules because some just did not get the work done on time. It is surprising that the percentage of responses to these two choices were identical for the two sets of data, even though more intermittent deadlines were imposed at different stages for the group project in 2006 and 2007 classes as new measures to facilitate the collaboration among peers. Twentyfive percent of the students in the 2006 and 2007 classes felt it worthwhile to collaborate for the group project despite the fact that it was difficult, only a slight increase of 3% from the Fall 2004 class. Similarly, 27% of the students preferred individual work leading to a project of their own even though they would not accomplish as much, a slight decrease of a 5% from the previous data. Compared to the overall positive responses toward collaboration in forum discussions (see Table 3 and Table 4), students’ attitudes toward online collaboration in producing the group project were mixed. Such differences were also reflected in some student comments on the group project in the open-end questions. One student wrote “I think it’s too inconvenient to try and get a group project together online. I also don’t like having someone’s performance affect my grade. I would rather do the project on my own.” It appears that the end product type of collaborative tasks demands more consensus-building collaboration. When students were timed for such intensive interaction and collaboration, they became less enthusiastic about it.
DISCUSSION Students’ Overall Attitudes Toward Collaborative Learning and Their Level of Participation The new data based on the 2006 and 2007 classes support the earlier findings about students’ attitudes toward online collaborative learning. Ninety-eight percent of the students reported that they were “positive” and “very positive” with this web-based course. The overall majority of them also stated that they preferred the forum discussions to weekly quizzes as the main learning tasks, if they were given the choices. As summarized in Wang (2007) findings, a number of factors contributed to the sustained small group discussions in this course. One of the factors may be the structure of the forums that required two rounds of postings. Students not only always had “something to say” in each forum but knew exactly what specific questions they were expected to answer in advance. These written exercises required in the first round of postings kept each individual student accountable for knowing the contents through reading. Therefore, students’ interaction with the course readings, the first level of interaction with the material, was enhanced by producing written answers to be commented by peers in the group forums. The enthusiasm in group discussions never waned forum after forum because each forum focused on a new reading chapter. Furthermore, the comment messages required students to exchange information by building on each other’s ideas to negotiate for meaning and to collaboratively construction knowledge. Such interaction between peers and between students and instructors provided another level of interaction for learning. Students’ positive experience with the semester long forum discussions was related to the benefits of proactive learning and learning from each other for knowledge construction. While the advantages of online interactive learning
25
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
have long been proved in previous studies (Kern, 1995; Lavooy & Newlin, 2003; Mouza, Kaplan, & Espinet; 2000; Summer & Hostetler, 2002; Wang & Teles, 1998; Wu, 2003), this study provided new data for the use of small group discussion as the core interactive learning tasks through the application of carefully prepared discussion questions that elicits proactive learning and through peer interaction and collaboration. When online collaborative learning tasks become main course pedagogy, such interactive learning is likely to be more sustainable and effective. With regard to levels of participation in collaborative learning, previous research indicates that the size of online learning community affects the level of comforts which influences the level of participation (Hewitt & Brett, 2007; Williams & Pury, 2002). Hewitt & Brett (2007) reported that large classes were related with an increase in the number of notes (messages) written, a decrease in average note size, and an increase in note scanning rather than reading. The current finings suggest that students were comfortable with their peers in a group of 4-6 members and group cohesion and mutual trust was an important element for their collaborative learning. However, data also show that such trust and comfort with a smaller group size is no guarantee of semester-long sustainable interactive learning in the asynchronous forums. Overall, only 28% of the students responded that they would post the same number of messages in their forums if the postings were not required and graded. Forty-five percent of the students responded they would post some but not as many messages and 21% percent of the students said they would post very few messages. What is more, 8% of the students responded they would not post any messages at all. These numbers from the new data were almost identical to the previous data. Taken together, these data support the previous research findings that the assessment of collaborative learning tasks plays a crucial role in ensuring student participation. Macdonald (2003) reported that students actively contributed to the discussions
26
when the tasks were assessed but participation of discussions waned when the postings became optional. Grade for discussion was also positively related to students’ perceived learning (Jiang & Ting 2000). Apparently, any optional interactive learning tasks would not have sustained for the entire semester. Another indication of the level of participation is the amount of reading of messages posted by the peers in their group forums. Thirty-seven percent of the students reported that they read all the messages posted by the peers while 47% of them read most but not all the messages. However, 16% of the students responded that they read only some messages or did not often read the messages posted by the peers. Unlike the required postings by the deadlines in an asynchronous forum, it is relatively more difficult to enforce the reading of each posting in such learning environment. It is important to note that the size of the group (4-6 students) is relatively small for asynchronous forums and it was assumed that such a size would generate sufficient responses from each other. On the other hand, the number of messages produced by each member was manageable and easy to keep track of. Future studies need to investigate the level of participation of collaborative learning with different group sizes and with different learning tasks. With regard to group formation, the majority of students reported that they preferred working with the same members of the group for the entire semester rather than rotating the peers. Obviously, it takes time to establish such mutual trust, even in a small group of 4-6 members. Therefore, it is very likely that the group cohesion and mutual trust comes from the semester long of interaction, cooperation and collaboration online. The new data showed a slight increase of the percentage of students who expressed the desire to work with different peers during the semester. Future studies need to investigate the benefits and disadvantages of dynamic group formations in which students
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
are given the chance to work with different online peers during the semester.
Students’ Attitudes Toward Process and Product Oriented Interactive Learning Very few studies have dealt with the differences between process and product oriented interactive learning tasks and how these differences influence peer interaction and collaboration (Kear, 2004; Kear & Heap, 1999; Macdonald, 2003; Wang, 2007). This web-based course applied both process and product orientated interactive learning tasks that required different types and levels of interaction and collaboration. As discussed earlier, in the weekly group forums, the debate and exchange of ideas focused on the process of learning that did not lead to a final product. In contrast, the group project was a product driven collaborative task in that the interaction and collaboration among the peers through sharing and exchange of ideas and negotiation must help to reach certain consensus to produce a group report. Survey data suggest that students were more enthusiastic about process oriented group discussions than the group project. In the previous study based on the same course offered in 2004, Wang (2007) found that among others, the main reasons for students’ frustration about the group project were the difficulties in reaching agreement according to a time frame, especially in the online environment. The differences in working pace and conflicts of schedules, and, perhaps more importantly, differences in level of devotion to the collaborative task in online environment made it more difficult for the peers to reach consensus in the process of doing the group project. The early birds who preferred to start and complete their parts of the work in a timely fashion conflicted with those who procrastinated in getting the work done. As peers in the same group would receive a common grade only for their project, there was pressure for them to
compromise to reach agreements in completing the project. In order to reduce the frustration caused by the schedule conflicts between the peers, a few intermittent deadlines were imposed when the course was offered in 2006 and 2007 years. Students had to meet each deadline (individual interviews, sharing the interview summaries, and draft the project) at each step to avoid the last minute rush that usually delayed the progress of the group work. The new data about the group project from 2006-2007 classes were collected after these new measures were taken. As seen from Table 9 and Table 10, these new measures did not appear to address these challenges the groups faced in completing the projects. In fact, only 24% of the students from the current data, (exactly the same 24% of the students in the previous study) responded that they preferred to collaborate with others for a group project but they did not like to depend on other peoples’ schedule because some just did not get their work done on time. Similarly, 24% of students in the current data (exactly the same percentage as found in the previous study) responded that they preferred to collaborate with peers because it was not a problem for them to collaborate. Only a slight increase of the number of students, from the previous 22% to the current 25%, responded that even though it was hard to collaborative for the group project, it is still worth doing it because they learned more about the bilingual programs in different schools through doing it together. There was also a slight decrease of number of students, from the previous 32% to the current 27%, who stated that they preferred individual work leading to a project of their own even though they only have information about one school. These data suggest that, overall, students’ attitude towards the group project was mixed. There was less enthusiasm for the group project than for the weekly forum discussions. In addition to the differences in working schedules and level of devotion to the group project, the common grade assigned to students for their
27
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
group project appeared to be another factor that caused the frustration for this collaborative task. Although the common grade can be used as a useful instructional strategy to implement end product driven collaborative tasks to encourage collaboration, the frustration and stress caused by the schedule conflicts and different levels of devotion toward such collaboration calls for more careful instructional design of such tasks. Perhaps some form of individual grading in addition to the interdependent grading are necessary to measure each individual student’s efforts and contribution. In fact, Kear & Heap (1999) reported that students expressed a preference for a higher individual grade component when both common and individual grades were assigned for their group project. It is important to balance the level of collaboration among the students and the individual flexibility of online learning. Future studies need to address the pedagogical design of end product driven collaborative tasks in web-based courses.
CONCLUSION The new data in the current study further supported the earlier findings (Wang, 2007) about students’ positive attitudes toward collaborative learning as main learning tasks in a web-based course. The overall majority of them also stated that they preferred the forum discussions to weekly quizzes as the main learning tasks, if they were given the choices. Among others, the structure of the online discussion, group cohesion, direct link of the interactive learning tasks to the assessment, and strictly imposed deadlines are some of the important factors that influence students’ learning experience and level of participation in collaborative learning. The differences in process and product driven interactive learning tasks also have a different impact on student online collaboration. In general, students were more enthusiastic about process oriented than
28
product driven collaborative tasks. Despite the new measures taken in the form of imposing more intermittent deadlines for the preparation of the group project, a product orientated collaborative task, a substantial number of students still preferred to do an individual project on their own, if they were given the choices. Many of them expressed their concerns about their grades being affected by their peers’ work. It appears that assigning a common grade to all the members of the group may not be the best way of assessing such a product oriented collaborative task. Some element of individual assessment component might be necessary to reflect the different level of devotion of the students. Finally, as the current data are based on one web-based course that was mainly a reading course, the findings may not be generalized into a broad scope. Because of this limitation, the current findings may not be directly applicable to other courses that have a different online pedagogical approach. Yet, a few recommendations may be made for designing and implementing similar interactive learning activities to promote sustained and effective online collaboration. •
•
• •
Although a very good tool for promoting interactive learning and collaboration, online discussion is not always sustainable if not well planned and structured. It is recommended that instructors carefully design each forum discussion with direct involvement of course contents with predetermined specific questions to engage students in a high level of thinking through providing written answers to the topics for which peer critiques are required. To continue to motivate the students, link the assessment with all interactive learning tasks utilizing specific grading scales. Impose strict deadlines for each round of postings in each discussion forum. Form small groups of 4-6 as learning communities for discussions so the peers will
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
•
•
have sufficient input from each other yet still find it easy to keep track of all the postings in each new thread. Use process oriented interactive learning tasks to facilitate continuous online interaction and collaboration and yet still give each student sufficient amount of freedom in completing the assessed learning tasks. When design product oriented interactive learning tasks, much care needs to be taken in order to prepare the students to reach consensus. Give sufficient time for completing such learning assignment. Incorporate both common and individual grades in grading an group project.
REFERENCES Duin, H., & Hansen, C. (1994). Reading and writing on computer networks as social construction and social interaction. In Selfe, C. & Hilligoss, S. (Eds.) Literacy and computers: The complications of teaching and learning with technology, (pp. 89-112). New York: The Modern Language Association. Evans, C., & Gibbons, N. J. (2007). The interactivity effect in multimedia learning. Computers & Education 49, 1147–1160. Gao, T., & Lehman, J., D. (2003). The effects of different levels of interaction on the achievement and motivational perceptions of college students in a Web-based learning environment. Journal of Interactive Learning Research, 14(4), 367-387. Hew, K. F., & Cheung, W.S. (2007). Attracting student participation in asynchronous online discussions: A case study of peer facilitation. Computers & Education (doi:10.1016/ j.compedu.2007.11.002). Hewitt, J., & Brett, C. (2007). The relationship between class size and online activity patterns in asynchronous computer conferencing environ-
ments. Computers & Education 49, 1258–1271. Jiang, M., & Ting, E. (2000). A study of factors influencing students’ perceived learning in a Web-based course environment. International Journal of Educational Telecommunications, 6(4), 317-338. Kear, K. (2004). Peer learning using asynchronous discussion systems in distance education. Open Learning, 19(2), 151- 164. Kear, K., & Heap, N. (1999). Technology-supported group work in distance learning. Active Learning 10, 21-26. Kern, R. (1995). Restructuring classroom interaction with networked computers: Effects on quantity and characteristics of language production. The Modern Language Journal, 79, 457- 476. Kuhl, D. (2002). Investigating online learning communities. U.S. Department of Education Office of Educational Research and Improvement (OERI). Lavooy, M. J., & Newlin, M. H. (2003). Computer Mediated Communication: Online instruction and interactivity. Journal of Interactive Learning Research, 14(2), 157-165. Leinonen, P., Järvelä, S., & Lipponen, L. (2003). Individual Students’ Interpretations of their contribution to the computer-mediated discussions. Journal of Interactive Learning Research, 14(1), 99-122. Macdonald, J. (2003). Assessing online collaborative learning: process and product. Computers and Education, 40, 377-391. Mouza, C., Kaplan, D., & Espinet, I. (2000). A Web-based model for online collaboration between distance learning and campus students (IR020521): Office of Educational Research and improvement, U.S. Department of Education. Offir, B., Lev, Y., & Bezalel, R. (2007). Surface and deep learning processes in distance
29
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
education: Synchronous versus asynchronous systems. Computers & Education, doi:10.1016/ j.compedu.2007.10.009. So, H.-J., & Brush, T. A. (2007). Student perceptions of collaborative learning, social presence and satisfaction in a blended learning environment: Relationships and critical factors. Computers & Education, doi:10.1016/j.compedu.2007.05.009. Sumner, M., & Hostetler, D. (2002). A comparative study of computer conferencing and face-to-face communications in systems design. Journal of Interactive Learning Research, 13(3), 277-291. Wang, X. (2007). What factors promote sustained online discussions and collaborative learning in a Web-based course? International Journal of Web-Based Learning and Teaching Technology, 2(1), 17-38.
30
Wang, X., & Teles, L. (1998). Online collaboration and the role of the instructor in two university credit courses. In. Chan, T. W., Collins, A. & Lin, J. (Eds.), Global Education on the Net, Proceedings of the Sixth International Conference on Computers in Education, 1, 154-161. Beijing and Heidelberg: China High Education Press and Springer-Berlag. Williams, S., & Pury, C. (2002). Student attitudes toward participation in electronic discussions. International Journal of Educational Technology, 3(1), 1-15. Wu, A. (2003). Supporting electronic discourse: Principles of design from a social constructivist perspective. Journal of Interactive Learning Research, 14(2), 167-184.
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
APPENDIX: SSurvey QUESTIionnaire 1.
Is this your first web-based (entirely online) course? a. b. c. d
Yes. No, I already took one entirely online course before this one. No, I took two or more other entirely online courses before this one. I took one or more web-enhanced course (partially online) before this web-based (entirely online course). e. No, I have never taken any web-based nor web-enhanced course. 2.
This reading course is structured on group discussions with individual and group assignments. What are your thoughts about the structure of the course? a. I like the way the course is structured in terms of forum discussions because we learn from each other. b. I prefer weekly quizzes based on the readings rather than answering questions/joining group discussions.
3.
Will you post the same number of messages as you actually did over the semester if these postings were optional, not required and graded? a. b. c. d.
4.
Yes, I will pos the same number of messages. I will post some messages but not as many. I will post very few messages. I will not post any messages.
Please circle one answer for each of the following: a. In our group forums, my answers to the questions and comments on peers’ messages help me to understand the contents/readings of the course better.
strongly agree
agree
disagree
strongly disagree
b. My peers’ answers/comments helped me to understand the readings better.
strongly agree
agree
disagree
strongly disagree
c. I learned more through online discussions than I would have learned from the lectures.
strongly agree
agree
disagree
strongly disagree
d. The online discussion is helpful because we collaborate more with each other and support each other.
strongly agree
agree
disagree
strongly disagree
31
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
e. The group cohesion and mutual trust is an important factor in our group forums.
strongly agree
agree
f.
I prefer individual work to group work and would have done better if I did not have to collaborate with my peers in my group for discussions.
strongly agree
agree
disagree
disagree
strongly disagree
strongly disagree
g. I prefer individual work to group work and would have done better if I did not have to collaborate with my peers in the group for the . nal project.
strongly agree
agree
disagree
strongly disagree
h. The deadlines for the readings and postings in each forum are very important because they help me to complete the readings and the course.
5.
strongly agree
agree
i.
The overall course contents are interesting and I have learned a lot about bilingualism and bilingual education from taking this course.
strongly agree
agree
disagree
disagree
strongly disagree
strongly disagree
Choose one of the following: a. I wanted other group members to read our group discussions and I also missed the discussions in other groups. b. Every group should have summarized their forum discussions each week and post it to a general forum so that interested students could comment on the discussions in other groups. c. Reading and responding to peers’ messages in our own group discussions is sufficient for me to understand the course contents. It would take too much time to read and respond to summary messages from other groups.
6.
What is your view about group formation? a. I want to work with the same group members the way it is now because we know each other better. b. I want to work with different people in a group every few weeks because we will learn from other students we never meet. c. It will not make a difference to me working with the same people or different people in a group.
32
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
7.
The pace of the course, including readings and postings a. b. c. d.
8.
Is neither too fast nor too slow for me. Is too fast for me because I always try to catch up with the readings. Is too slow for me and we could have read more chapters. Should be OK for a course like this but I found it too fast for me because I work many hours a week and have limited time for course work.
Course documents: a. I printed out all the lecture notes and review guides (or some of them) because they are helpful. b. I read the lecture notes and the review guides online but did not print them all. c. I never printed out nor read the lecture notes and the review guides because they are not essential for me.
9.
The videos on reserve in the music library are used in all other face to face sessions of the same course. I found these videos a. worth seeing because they are informative and very relevant to the course content. b. relevant to the course content, but it is hard for me to make special trips to the university to watch them all. c. are not relevant to the course content and can be omitted.
10. You took all the three exams online in this semester. Do you think the online exam should be kept the way they are now, or do you prefer to take these exams in a classroom on a certain date? a. I prefer online exams the way they are now. b. I prefer to come to a classroom to write the exams. c. I have no preference. 11. Exam format: a. I prefer multiple choice exams. b. I prefer essay question type of exams. c. It does not make a difference for me. 12. The group project about bilingual programs in our local schools (circle all the answers that apply to you) a. b. c. d.
Is a good assignment and I learned a lot through doing the project. Makes the course readings more meaningful and more relevant to me. Is a good assignment but takes too much time to complete. Could be an individual assignment focusing on one school rather than a group project that involves more collaboration. e. Is not very important for this course.
33
Students’ Attitudes toward Process and Product Oriented Online Collaborative Learning
13. For the group project: a. I prefer individual work leading to a project of my own even though I only have information about one school. b. I prefer to collaborate with peers the way it is now because it is not a problem with me to collaborate. c. I prefer to collaborate with others for a group projected but I do not like to depend on other people’s schedule because some just do not get their work done on time. d. Even though it is hard to collaborative for the group project, it is still worth doing it because we learn more about our bilingual programs in different schools through doing it together. 14. Overall, my experience with this web-based course a. b. c. d.
Is very positive. Is positive. Is negative. Is very negative.
15. Experience with the Blackboard and the online forums: circle all apply to you. a. I found it challenging at the beginning but quickly picked up and like it now. b. The interface is straightforward and easy to learn, although I was not very experienced with any online courses. c. It was never a problem for me because I am good at technology. d. It was a plus because I learned the technology as well as the course contents 16. If I have the choice in future, a. I will take a similar web-based course. b. I will not choose to take a similar web-based course. c. It will not make a difference, web-based or face-to-face version. 17. Would you recommend a friend to take this web-based course? a. Yes b. No c. Not sure 18. Please take some time to answer the following questions: a. Please describe your experience with the forum discussion part of the course. (positive, negative, expectation, effect on learning, etc. anything you think is relevant) b. What do you like the most, or dislike the most about this course? c. In your opinion, what are the most important elements for a web-based course like this to be successful? d. To improve the course for future students, what changes do you recommend?
34
35
Chapter III
Cognition, Technology, and Performance:
The Role of Course Management Systems Teresa Lang Columbus State University, USA Dianne Hall Auburn University, USA
AbSTRACT Development and sale of computer-assisted instructional supplements and course management system products are increasing. Textbook sales representatives use this technology to market textbooks, and many colleges and universities encourage the use of such technology. The use of course management systems in education has been equated to the use of enterprise resource planning software by large businesses. Research findings about the pedagogical benefits of computer-assisted instruction and computer management systems are inconclusive. This study describes an experiment conducted to determine the benefit to students of using course management systems. The effects of cognition, learning styles, and computer attitude were considered and eliminated to better isolate any differences in performance. Student performance did not improve with the use of the technology.
Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.
Cognition, Technology, and Performance
CogniTehnology, and Performan: TThe of Cour Managemen Advances in technology have become marketing tools in our society. Cell phone providers offer text messaging, photo capability, and Internet connectivity to entice consumers to purchase their service over that of the competition. Businesses offer online bill paying, discounts for Internet orders, and auction derived pricing opportunities. Publishers of today’s college textbooks offer a variety of computer-based supplements and computer-based course management tools to accompany their textbooks. Textbook sales representatives use these technologies as a strategic marketing tool when approaching college faculty about textbook alternatives and adoption. Faculty members are promised easy implementation and it is implied that the technologies will lead to improved learning for students. Today’s society expects to receive things on demand. Purchase software and download it immediately, follow the stock market up to the minute, take a test and get the results immediately. Course management systems make course materials and student performance available continuously and as fast as the instructor can post the information. The expanded use of the Internet in our society has changed the standard method of transmitting information in education (Aggarwal & Legon 2006). The academic community uses networks and the Internet for communication and conferencing, and for information access, retrieval, and use. Previous studies confirm that connectivity is beneficial to pedagogy when interaction, discussion, research, or transmission of information is involved; however, there have been few experiments to determine how Internet-enhanced courses affect student learning and understanding (Agarwal & Day, 1998). Publishers are responding to apparent changes in demand for technology while managing prof-
36
itability. Colleges are attempting to restructure to balance resources with changing demands, faculty members are encouraged to learn and incorporate technology into the curriculum with the belief that students will experience improved performance. However, research findings about the pedagogical benefits of computer assisted instruction and computer management systems are inconclusive (Russell, 2002; Morgan, 2003). This paper describes an experiment to determine whether publisher provided textbook technology simply provides accessibility or provides true pedagogical benefits to the student.
Course Technologies There are many terms used in the literature over the past several years to describe the use of computers and technology in education. Computer-based education (CBE) and computer-based instruction (CBI) describe the use of computers for drill and practice, tutorials, simulations, instructional management, supplementary exercises, programming, database development, writing using word processors, and other applications. Computerbased training (CBT) refers to self-paced tutorials frequently used in industry. Computer-assisted instruction (CAI) refers to drill-and-practice, tutorial, or simulation activities offered either by themselves or as supplements to traditional teacher directed instruction (Cotten, 1991). Course management systems (CMS) were developed in the mid-1990’s as distance education developed and expanded. Course management systems are also called learning management systems and virtual environments and may operate on the university’s network or use the Internet to operate from the publisher’s network (Simonson, 2007). These systems are used in higher education and include applications for course content organization and presentation, communication tools, web pages, and course management functions such as materials and activities.
Cognition, Technology, and Performance
Figure 1. Example of applications available with WebCT
Syllabus
Quizzes
Practice Material
Content
Links
Gradebook
Communication Capabilities
CMS (On local server or publisher’s server)
WebCT or Vista and Blackboard are the most commonly used proprietary course management systems in higher education today and are described as the academic system equivalent of enterprise resource planning (ERP) systems (Morgan, 2003). There are also open source or free CMSs such as the Sakai Project and Moodle (Simonson, 2007). These course management systems provide a structured format with builtin applications to facilitate faculty adoption of technology into the course curriculum. Textbook publishers develop packages for use in conjunction with these CMS platforms. The package generally contains all of the instructor tools available from the publisher plus grade book capabilities and communication applications such as chat rooms. Students can access their grades from the CMS and faculty can develop and use some of their own material in conjunction these applications. Figure 1 shows the breadth of common components inherent in a course management system.
Cognitive Skills and Learning Styles The learning process includes three main aspects: cognitive (how knowledge is assimilated), conceptualization (how information is processed), and affective (the influence that motivation, decisionmaking styles, values, emotional responses and preferences has on the individual). The combination of these three aspects define the learning style of an individual (Litzinger & Osif, 1993).
Cognitive styles and learning styles are terms used interchangeably in many situations. Typically, cognitive style refers to more theoretical situations, such as academic research, and learning style refers to practical applications. Measurement of cognitive/learning styles falls somewhere between the measurement of aptitude and the measurement of personality (Liu & Ginther, Fall 1999) . There are several cognition and learning style models. The National Association of Secondary School Principals (NASSP) Student Learning Style Task Force developed a detailed outline of the elements of learning style that effectively summarizes the fundamental aspects of the best known learning style and cognition models (Keefe & Monk, 1990). Table 1 presents a summary of these elements. A learner’s ability to concentrate, learn, and remember is affected by how the learner perceives, interacts with, and responds to the learning environment. These are multidimensional processes affected by various cognitive, perceptual, and environmental elements (Keefe, 1985, 1987).
Cgnition Cognitive styles are information-processing habits, and are both genetic and learned. The measurement of these skills attempts to capture the learner’s processing inclinations and the learner’s ability to identify problems, visualize
37
Cognition, Technology, and Performance
Table 1. Elements of a learning style Styles Cognitive Styles
Element Analytic processing
Able to perceive the elements of problem as separate from the overall problem. Field independent.
Spatial
To identify geometric shapes, rotating them in the mind; to construct objects in mental space.
Discrimination
To find the important elements of a task; to focus attention on required detail and avoid distraction.
Categorization
To use reasonable criteria for classifying information; to form accurate, complete, and organized categories.
Sequential Processing
To process information sequentially or verbally; to understand information presented in a linear fashion.
Simultaneous Processing
To grasp visual spatial relationships; to sense an overall pattern from the relationships of the component parts.
Memory
To retain distinct vs. vague images in repeated tasks; to detect and remember subtle changes in information.
Visual
Prefer information input via visual experience.
Auditory
Prefer information input via auditory experience.
Kinesthetic
Prefer information input via “hands-on” or manipulative experience.
Emotive
Prefer information input via emotional experience.
Study/ Instructional
Persistence
Willingness to work at a task until completed.
Verbal Risk
Willingness to express opinions, speak out, etc.
Preferences
Verbal-Spatial Preference
Prefer verbal or nonverbal activities.
Perceptual Response
Study Time
Early Morning
Prefer to study in the early morning.
Late Morning
Prefer to study in the late morning.
Afternoon
Prefer to study in the afternoon.
Evening
Prefer to study in the evening.
Grouping
Prefer whole class or small group or pair grouping.
Posture
Prefer formal or informal study arrangements.
Mobility
Prefer taking frequent breaks or working until done.
Sound
Prefer quiet study areas or some background sound.
Lighting
Prefer bright or lower lighted study areas.
Temperature
Prefer studying in a cool or a warm environment.
Adapted from (Keefe & Monk, 1990)
38
Description
Cognition, Technology, and Performance
solutions, determine what tasks are required to accomplish the solution, and to select, categorize, and mentally manipulate the data required for those tasks (Keefe, 1985). Ausburn and Ausburn (1978) describe three properties of cognitive styles. The first property is that there is generality and stability across tasks and over time, making individuals resistant to training and change. The second property is relative independence of cognitive styles from traditional measures of general ability. The third property is cognitive styles’ relationships with some specific abilities, characteristics, and learning tasks. These researchers assert that cognitive styles have either positive or negative relationships with academic achievement depending on the nature of the learning task (Ausburn & Ausburn, 1978). Previous research indicates there is a substantial positive relationship between cognitive ability and task performance, and cognitive ability helps individuals adapt to new situations, prioritize rules and regulations, and deal with unexpected problems (Klausmeier & Loughlin, 1961; Hunter, 1986; Ackerman, 1987). Other researchers assert that cognitive styles have either positive or negative relationships with performance depending on the nature of the learning task (Ausburn & Ausburn, 1978; Hall, Cegielski & Wade, 2006). Thus we posit: Hypothesis 1: Cognitive ability influences performance.
Perceptual Response Perceptual response elements indicate the sensory mode that the learner will routinely employ when taking in information. Visual information is the primary sensory mode for some learners, while other learners rely on auditory information, and others on information acquired through physical activity. Brain function, perceptual skills,
training, and habit come together to determine a learner’s perceptual responses. Students strong in analytical thought processing process information logically and sequentially, and generally achieve higher grades on objective tests. Visual processors process information nonlinearly and holistically, and achieve lower grades than analytical students (O’Boyle, 1986; Sonnier, 1991; Gadzella, 1995). Individual preferences may change over a lifetime, but remain constant over shorter periods of time (Fleming & Bonwell, 2002). According to Dunn and Griggs (1989), everyone has a unique, specific learning style, and instruction should be designed to best accommodate different methods of learning (Dunn & Griggs, 1998). Learning styles are a multidimensional construct and CMS has the capacity to engage the user at different levels such as visual, aural, reading/writing, or kinesthetic thereby possibly helping users learn, retain and enjoy the process (Dunn, 1996). Thus, we posit: Hypothesis 2: Learning style influences performance. Cognition and learning styles may affect the student’s ability to learn, which will influence how beneficial the CMS is for the student as measured by performance. After controlling for the differences in students’ cognitive skills and learning styles, a more accurate evaluation of the effect of CMS is possible, and may better explain the inconclusive prior research findings about technologies in education in general. We posit: Hypothesis 3: There is a difference in student performance between students in classroom learning environments using CMS and students in classroom learning environments not using CMS after controlling for cognition and learning style.
39
Cognition, Technology, and Performance
Computer Attitudes Prior research suggests that individual attitudes toward computers may affect computer use (Ajzen & Fishbein, 1980; Rainer & Miller, 1996; Al-Khaldi & Al-Jabri, 1998; Chau, 2001; Chang & Lim, 2006). Some research finds that negative attitudes toward computers may also affect individual performance (Eason & Damodaran, 1981). Computer attitudes are pertinent to this study because if individuals have a positive attitude toward computers, they may be more satisfied with computer assisted instruction and computer management systems than students with negative computer attitudes. Any difference in computer attitude may be reflected in student performance. Some students are comfortable using computers and expect a positive outcome from the interaction with a computer based instructional method, while others are anxious when using computers and do not expect to perform well in a computer environment. Students enrolled in classes using CMS may expect to perform poorly because of their negative computer attitudes. This expectation may then negatively affect their performance because they do not fully utilize the CMS. Thus, we posit:
Hypothesis 4: Computer attitude influences performance. and Hypothesis 5: Students with higher computer attitude will perform better than students with lower computer attitude when enrolled in classroom learning environments using CMS. This study examines the effect that CMS have on performance. Learning style and cognition are included as control variables in order to separate the effects of CMS from other variables believed to affect performance. Computer attitude is introduced and is evaluated as a potential relevant variable when evaluating CMS results. Combining the above, we posit: Hypothesis 6: There is a difference in student performance between students in classroom learning environments using CMS and students in classroom learning environments not using CMS after controlling for cognition, learning style, and computer attitude. Table 2 presents a summary of our hypotheses.
Table 2. Hypotheses tested Hypothesis 1: Cognitive ability influences performance. Hypothesis 2: Learning style influences performance. Hypothesis 3: There is a difference in student performance between students in classroom learning environments using CMS and students in classroom learning environments not using CMS after controlling for cognition and learning style. Hypothesis 4: Computer attitude influences performance. Hypothesis 5: Students with higher computer attitude will perform better than students with lower computer attitude when enrolled in classroom learning environments using CMS. Hypothesis 6: There is a difference in student performance between students in classroom learning environments using CMS and students in classroom learning environments not using CMS after controlling for cognition, learning style, and computer attitude.
40
Cognition, Technology, and Performance
METHODOLOGY
of students into the two treatment groups and the two control groups was based on enrollment in particular classes.
In today’s technological era, computers are being used extensively as educational tools and instructional devices. The success or failure of computer-assisted instruction (CAI) and course management systems (CMS) depend on a number of variables, including the user’s computer attitude, cognition, and learning style. This research examines the effect of a CMS on performance after controlling for cognition and learning style. The effect of computer attitude on performance when CMS is used is also of interest (see Figure 2). This experiment involves two approximately equally sized groups of students: a control group using no computer provided course material, and a treatment group using WebCT (a popular course management system) as provided by Wiley Publishing. Participants consisted of the students enrolled in four different sections of a class in Principles of Financial Accounting at a large, southeastern university. One instructor taught all four sections. Two sections combined make up the CMS treatment, and two other sections combined make up the control section. Placement
Participants The computer management system used in this study is the WebCT package sold by the textbook publisher to accompany the textbook. The publisher provided the software free of charge to the students and faculty member in support of the experiment. This process avoided any bias the CMS treatment group may have had because of increased textbook costs related to the CMS software. The package included self-help multiplechoice quizzes, grade posting capabilities, communication capabilities, and graded homework and graded quiz capabilities. The treatment group submitted all homework using the CMS and took quizzes using the software. The control group had in class quizzes and submitted homework on paper. Quizzes were timed equally both on-line and in class. The communication capabilities included four chat rooms and the ability to email the course instructor. The course syllabus and
Figure 2. Extension of Keefe model including computer attitudes Cognition
Perceptual Visual Aural Reading/Writing Kinesthetic
General Reasoning
Learning Styles
Computer Attitude CMS
Environment
41
Cognition, Technology, and Performance
the course homework schedule were added to the course material available via WebCT. All WebCT capabilities were available to the students during the semester. Although it was available, students did not use the communication capabilities provided with WebCT.
Materials The non-computer (traditional) students did not use computer aids and served as the control group. All materials available to the computer group were made available to non-computer students in paper format. The traditional students had email access along with both of the treatment groups because this is the official method of communication at this university. All students were required to purchase a note pack that provided notes for the entire semester. The note pack was not available from WebCT to avoid potential bias between groups caused by the control group purchasing copies of the note pack, while the treatment group could have downloaded their note pack. The note pack included outlines of each chapter. A common syllabus was used for the sections, as well as common exams, homework, project, and quizzes. The same textbook was used for all classes.
Data Collection Two Internet accessible surveys were used to collect data for the study. The first survey included demographics, learning styles, and computer attitude. This survey was completed after the first exam but before mid-semester. The second survey included cognition, and questions about usage of the CMS, and was administered the last two weeks of the semester. The usage questions asked about the students’ knowledge of the treatments used in other classes. For example, “Did you share your password to use WebCT with students from other classes that did not use WebCT?” Open-ended
42
questions were also included. For example, “How could the course be improved?” The responses to these questions were used to evaluate cross talk between students in different groups – no appreciable evidence exists to suggest that the treatment and control groups were aware of the experimental condition. Ten points were awarded for completing each of the two surveys, and attendance was collected. Students that did not want to participate were allowed to complete additional homework assignments instead of the surveys. Performance was collected for each exam, each quiz, and each homework assignment. The scores collected were raw scores without any curve. For consistency, one graduate student graded all homework and quizzes. The number of times students accessed the CMS was accumulated at the end of the semester. WebCT has a tracking feature built in to track usage of the CMS. This data was used to confirm which students actually received the treatment.
Instruments The cognition instrument used in this study is a tested and validated instrument from a Kit of Factor-Referenced Cognitive Tests developed by the Educational Testing Service in Princeton, New Jersey. The general reasoning measure was selected as a general measurement of cognition for this study. This instrument scores the individual’s the ability to select and organize relevant information in order to solve a problem (Ekstrom, 1999). An example question follows: “If a man earns $5.75 an hour, how many hours should he work each day in order to make an average of $46.00 per day? a. b. c. d.
subtract divide add multiply
Cognition, Technology, and Performance
Individuals are scored based on the number of questions they get correct minus one-half the number they get wrong. Higher scores represent higher cognitive ability and lower scores represent lower cognitive ability relative to the sample population. The results were categorized into very low, low, average, high, and very high for comparison. This instrument is well established and previously validated (Ekstrom, French, Harman & Dermen, 1999). A learning styles instrument developed and copyrighted by Neil Fleming was selected to categorize students by perceptual response (Fleming & Mills, 1992). The instrument (VARK) categorizes students as visual, aural, reading/writing, kinesthetic, or multimodal and has 13 nonquestions with non-mutually exclusive answers corresponding to one of the four categories. The answers are accumulated by category and the highest score determines whether the student is visual, aural, reading/writing, kinesthetic, or multimodal. If the highest score is shared by two or more categories the student is considered multimodal. Multimodal students may use more than one category equally well. Validity and reliability concerns regarding learning style measurement instruments has been identified as a problem within cognition/learning style research (Cox & Gall, 1981; Ferrell, 1983; Sewall, 1986; Drummond, 1987; Moran, 1991; James & Blank, 1993). The validity and reliability of the majority of the measurement instruments commonly cited in the literature have been questioned, including the LSI, LSQ, and Embedded Figures Test (Sims, Veres, Watson & Buckner, 1986; Tennant, 1988; Allinson & Hayes, 1990). Despite its lack of published reliability, the VARK is appropriate in this study because the instrument is incorporated into the textbook used in the classes included in the study (Kimmel, Weygandt & Kieso, 2004). The VARK appears at the beginning of the textbook with instructions to the student to use it to determine their learning style. Study strategies
for the different learning styles are then outlined that are specific to the textbook. For example, if a student is categorized as a visual learner, they are encouraged to: a. b. c. d.
pay close attention to charts, drawings, and handouts; underline; use different colors; and use symbols, flow charts, and graphs. Visual students are encouraged to:
a. b. c.
recall your “page pictures”; draw diagrams where appropriate; and practice turning your visuals back into words.
In addition, throughout the textbook, icons appear in the margins to guide students’ study of the material based on their learning style determined using the VARK instrument. The ‘Computer Attitude Scale’ (CAS) is a previously validated instrument that measures computer attitude (Nickell & Pinto, 1986). Harrison and Rainer (1992) found that the computer attitude construct has three components: optimism, pessimism, and intimidation. Subsequent research supported the use of the optimism component as a measure of computer attitude (Chau, 2001). The degree of optimism an individual has toward computers is determined based on responses ranging from strongly agree to strongly disagree on a 7-point Likert-type scale. For example, “The use of computers is enhancing our standard of living.” Higher scores indicate more optimism and lower scores indicate lower optimism.
RESULTS The student participants in this study were enrolled in Principles of Financial Accounting. Three hundred and thirty-five students completed
43
Cognition, Technology, and Performance
both surveys, a 99% participation rate. Therefore, non-response bias is not considered to be a problem. Of the 335 students that completed both surveys, five students did not take the final exam and therefore did not complete the class. The analysis was done on the remaining 330 students after looking for but finding no outliers or other incomplete information. This convenience sample provided a sufficient sample size to test the dependent variable, performance, because there are more observations in each cell than there are dependent variables included in that cell (Hair, Anderson, Tatham & Black, 1998). Most (95%) of the participants were between 19 and 22 years of age, in their sophomore or junior year, and full-time students. There were more males (55%) than females (45%), and 54% of the participants’ majors were undeclared or other business majors.
Data Analysis The data was analyzed using the SPSS statistical program. Analysis of variance was used because the dependent variable is continuous and the independent variables are categorical and ordinal (Leeper, 2004). Analysis of variance and analysis of covariance procedures are valid only if the dependent variable(s) are normally distributed, the variances are equal for all treatment groups, and the observations are independent. These analysis procedures are robust with regard to these assumptions except in extreme cases. Violation of the normality assumption has little impact with large sample sizes, such as the one used in this study. However, no such violations were found and performance (the dependent variable) is normally distributed. A violation of the equal variance assumption has minimal impact if the groups are of approximately equal size. Groups are considered approximately equal if the largest group size divided by the smallest group size is less than 1.5 (Hair et al., 1998). This study had a group size quotient of 1.2,
44
well within guidelines. Performance had equal variance, and independence of the observations is evaluated as part of each data analysis. Linearity and multicollinearity among the dependent variables were evaluated using scatter plots, correlation matrix, and variance inflation factors. No problems were identified with linearity or multicollinearity. The relationships between the demographic variables, age, gender, major, and class, and the dependent variables, and performance were evaluated. There were no significant differences between the different intervals of demographic variables and performance. For example, there were no significant differences in performance between students majoring in accounting and students majoring in management. The cognition instrument used in this study is a tested and validated instrument from a Kit of Factor-Referenced Cognitive Tests developed by the Educational Testing Service in Princeton, New Jersey (Ekstrom et al., 1999). The Cronbach alpha for CMS was .85, and for the control group was .85. The learning styles (perceptual response) instrument used in this study is the VARK (Fleming & Bonwell, 2002). The Cronbach alpha for the population was computed for visual (.39), aural (.48), read/write (.41), and kinesthetic (.36). The reliabilities for the various multimodal combinations would fall within this range. This level of reliability is common with these types of instruments (James & Blank, 1993). The ‘Computer Attitude Scale’ (CAS) is a previously validated instrument that measures computer attitude (Nickell & Pinto, 1986). The optimism component has four questions with responses ranging from strongly agree to strongly disagree on a 7-point Likert-type scale. Higher scores indicate more optimism The SPSS statistical program was used to normalize the scores into five intervals to be consistent with the other measures used in the study. The Cronbach alpha was .80 for the population, .81 for CMS, and .78 for control group.
Cognition, Technology, and Performance
Hypothesis Testing Hypothesis 1: Cognitive ability influences performance. An ANOVA was performed to determine the relationship between cognitive ability and performance. The SPSS statistical program was used to normalize the cognition measures into five intervals. Interval one designates very low cognitive ability and interval five designates very high cognitive ability. The analysis revealed that students with higher cognitive ability performed significantly better than students with lower cognitive ability. This finding was expected based on the literature and supports hypothesis 1. Further, the results support using cognitive ability as a covariate in relation to performance in this study to isolate the treatment effect. The findings for hypothesis one and the remaining hypotheses are summarized in Table 3 at the end of this section. Hypothesis 2: Learning styles (perceptual response) influences performance. An ANOVA was performed to determine the relationship between learning style and performance. The analysis revealed that performance did not differ significantly across the different learning styles. Hypothesis 2 is not supported. Learning style (perceptual response) did not influence performance. Therefore, the inclusion or absence of learning style as a covariate in relation to performance will not change the results. Hypothesis 3: There is a difference in student performance between students in classroom learning environments using CMS and students in classroom learning environments not using CMS, after controlling for cognition and learning style. An ANCOVA was performed using performance as the dependent variable, treatment as the independent variable, and cognition as a covariate. Cognition was used as covariate to isolate the treatment effect. The analysis revealed that student performance was not significantly different between students in the CMS group and
students in the control group, after controlling for cognition, failing to support this hypothesis. Hypothesis 4: Computer attitudes influence performance. An ANOVA was performed to determine the relationship between computer attitude and performance. The SPSS statistical program was used to normalize the computer attitude scores into five intervals. Analysis revealed that students’ performance did not differ significantly across the five intervals of computer attitude. Computer attitude did not significantly influence performance. Therefore, hypothesis 4 is not supported. Hypothesis 5: Students with higher computer attitude (more optimism) will perform better than students with lower computer attitude (less optimism) when enrolled in classroom learning environments using CMS. An ANCOVA was performed using performance as the dependent variable, computer attitude as the independent variable, and cognition as a covariate. Cognition was used as covariate to eliminate the effect of this variable and isolate the effect that computer attitude has on performance. The analysis revealed that students with higher computer attitudes (more optimism) did not perform significantly better than students with lower computer attitudes (less optimism). This finding is consistent with the results from hypothesis 4 and does not support hypothesis 5. Students with higher computer attitude (more optimism) do not appear to perform better than students with lower computer attitude (less optimism) when using CMS. Hypothesis 6: There is a difference in student performance between students in classroom learning environments using CMS and students in classroom learning environments not using CMS, after controlling for cognition, learning style, and computer attitude. An ANCOVA was performed using performance as the dependent variable, treatment as the independent variable, and cognition, and computer attitude as covariates. Cognition and computer attitude were used
45
Cognition, Technology, and Performance
Table 3. Summary of results No.
Hypothesis
1
Cognitive ability influences performance.
2
Learning style influences performance.
3
There is a difference in student performance between students in classroom learning environments using CMS and students in classroom learning environments not using CMS after controlling for cognition and learning style.
pvalue
4
4.689
.001
4
.137
.969
2
1.083
.299
4
Computer attitude influences performance.
4
.387
.818
5
Students with higher computer attitude will perform better than students with lower computer attitude when enrolled in classroom learning environments using CMS.
4
.183
.947
6
There is a difference in student performance between students in classroom learning environments using CMS and students in classroom learning environments not using CMS after controlling for cognition, learning style, and computer attitude.
3
1.054
.305
as covariates to eliminate the effect of these variables and further isolate treatment effects. The analysis revealed that student performance was not significantly different between treatments after controlling for cognition, and computer attitude. Hypothesis 6 is not supported.
DiscSCssion The results from hypothesis 1 testing confirm there is a relationship between cognition and performance. This finding further confirms prior literature and supports the position taken in this study for the need to control for this variable in order to isolate the impact CMS may have on performance (Holliday, 1976; Winn, 1982; Keefe, 1987). Based on the results from testing hypothesis 2, learning styles do not influence performance. The lower Cronbach alphas for the learning styles instrument, although not uncommon for this type of instrument, may have influenced the detection of this relationship in the analysis.
46
F-Statistic
df
Results of hypothesis 3 testing indicate no difference in students’ performance after controlling for cognition and learning style. The hypothesis is not supported. Hypotheses 4 and 5 do not indicate a relationship between performance and computer attitude regardless of the testing environment. Hypothesis 6, which analyzed differences in performance between CMS and non-CMS classrooms after controlling for potential influencers, is also not supported. Course management system software, while perhaps facilitating classroom management, does not appear to improve student performance regardless of the individual’s cognitive level, learning style, or computer attitude. If performance is not improved by the addition of course management technology to a course, what are other potential reasons that an adoption decision is made? First, colleges and universities across the nation are confronting difficult factors, including higher student costs, lower state funding, and increased diversity of the student body. Research
Cognition, Technology, and Performance
related to these problems suggests fundamentally restructuring the teaching and learning process through the integration of information technologies (Burke, 1994; Doucette, 1994; Guskin, 1994; Issroff & Scanlon, 2002; Morgan, 2003). College and university faculty and administrators hope to use technologies such as CMS to balance decreased funding with increasing student demands. In this respect, CMS may be able to provide a foundation that allows for more efficient teaching of higher numbers of students while reducing cost associated with traditional paper-based notes, handouts, and exams. Second, many textbook sales are pushed with technology in mind. As a result, when adopting a new textbook today, college faculty members must not only review and select among the numerous textbooks generally available, but must also be knowledgeable enough with the technology to select, implement, and teach students how to use the technology products they select, or in many cases come automatically prepackaged with the book. The technology is often used to promote new textbooks with the stated or implied notion that the technology will improve faculty efficiency and/or improve student performance. Third, some faculty may adopt CMS technology with a misunderstanding that such is also CAI technology. Most CMS systems, however, are not designed to provide this specialized information delivery without substantial work and innovation by the instructor. However, it may be that CAI technology, when embedded with CMS technology, will lead to improved student performance. Thus far, research on CAI technology generally supports this assertion. For instance, past research has found CAI typically produces greater levels of learning when combined with traditional instruction than does traditional instruction alone; however, much of this research was conducted in K-12 (Luyben, Hipworth & Pappas, 2003). Computer-assisted instruction has been found to improve college students’ attitudes toward the topic they are
studying as well as improve student achievement. Instructional time required to teach students using CAI was significantly lower than the instructional time required using conventional instruction, and retention of material learned improved when CAI was used (Kulik, Kulik & Cohen, 1980; Kulik, Kulik & Shwalb, 1986). Nontraditional adult learners have demonstrated improved performance, decreased instructional time, better retention, and improved attitude about the topic under study when CAI was used; however, the size of the effect was related to the type of CAI (Kulik et al., 1986). As an example, distance learning is a primary user of computer applications such as CAI and CMS. The majority of studies comparing traditional face-to-face instruction to distance learning find no significant difference in student satisfaction, learning, or performance (Russell, 2002; Bonds-Raacke, 2006). Prior research advocates the need for studies to determine the connection between cognition, student learning, technology, and general education before adopting electronic media (Kaput, 1992; Morgan, 2003). Technologies like CMS provide more timely feedback, allow for individualized pace and focus of learning, interactive exercises, access to up-to-date information, and the opportunity for drill and practice (Roy & Elfner, 2002) but do not provide the process of reflection on material and interaction with it as is usually the case with CAI. It is this interaction that create an understanding (Brown, Bransford, Ferrara & Campione, 1983; Farnham-Diggory, 1990). However, if use of CMS technology reduces an instructor’s time spent in course management, then perhaps more time will be spent creating a more cohesive and supportive learning environment. Study/instructional preferences are indicators of the learning environment in which a learner feels most comfortable. These preferences include physical parameters such as noise level, light level, and room temperature, as well as the time of day during which the learner learns best. Social as-
47
Cognition, Technology, and Performance
pects are also included in this category, such as the learner’s willingness to speak in public and size of the learning group in which the learner feels most comfortable. The learner’s gender, mental and physical health, and habits form the learner’s study/instructional preferences (Keefe & Monk, 1990). Method of presentation of instructional material and cognitive styles has a significant influence on learning performance among secondary and college level students. Colleges and universities typically present information in lecture format, emphasize reading, and require evidence of learning in written form. This format is most well suited for read/write and aural individuals. Imager (visual) students do best when pictorial presentations are used in educational material (Holliday, 1976; Winn, 1982). Verbal thinking is generally emphasized in research and intelligence testing with masculinity related to visual thinking, and femininity related to verbal thinking (Smith, 1964). Imagers perform better in text-plus-picture conditions, and verbalizers are better in text-plustext conditions where material is presented in only text form. Imagers use diagrams to illustrate their answers more often than verbalizers (Riding & Buckle, 1990; Riding & Cheema, 1991). It has been estimated that 20%-30% of American students are auditory, approximately 40% are visual, and the remaining are some combination (Bissell, White & Zivin, 1971; Dunn & Dunn, 1979). Lessons may be learned from this and related research to establish classrooms that are comfortable learning environments. There is a common belief in higher education that students drive faculty and institutions to use technology; however, in a recent study at a large university faculty reported that students actually discouraged them from using CMS because the students had difficulty gaining access to the CMS and were uncomfortable with technology in general (Morgan, 2003). Faculty members reported they were not confident about the reliability of CMS and were doubtful about the extent to which CMS use improves pedagogy. 48
A study on the effects of CMS found that a paperless disbursement of information was possible. The textbook, syllabus, grade book, and assignments were available through a CMS and were available twenty-four hours a day, seven days a week. The researchers posit the continuous availability and nontraditional format would improve student performance. They were disappointed to find that student performance did not improve and, in fact, several students either did not access the site or did not read the instructions carefully enough to fulfill course requirements. Many students did not look at or follow the web page links to access resources available on the site (Foreman & Widmayer, 2000). Motivation is an important variable in learning and academic performance (Lang & Hall, 2006). The students did not demonstrate the motivation to use the materials provided. The students in our research appear to have followed this same pattern, accessing the CMS only as required by the course despite content being available for their voluntary use. It is possible that the CMS may have had a greater effect on performance if students had been more proactive in its use. Reliability and boundaries of the system were a problem in this experiment. The servers or access to the servers was unavailable a few times. This occurred once when homework was due, and was confirmed with the publisher. Students were given an extension of time to complete the homework. This was frustrating for the students because of their concern about possibly losing points and wasted time. This was frustrating to the professor because of the technical problems of extending the time after the original due date had passed to submit homework. Subsequently, students reported the homework was not available, but this could not be confirmed with the publisher. Further, homework solutions required specific formats such as decimals and capitalization, which further frustrated students. These additional frustrations experienced by the students using WebCT may have influenced their
Cognition, Technology, and Performance
willingness to use the technology and thereby their performance. In all, this and other research indicates that the verdict on the effectiveness of CMS to improve student performance is still not clear. While research on CAI implies the opposite, this research shows that CMS is not effective. Given that most students prefer the classroom experience to straight computer-assisted learning, we assert that course management technology by itself may not improve performance, but that features inherent in CAI technology may be a beneficial extension of both CMS and this research. By combining the opportunity to interact and work with course material through technology with other desirable classroom environmental factors (class size, time temperature, etc.), CMS may prove beneficial both to instructor effectiveness and student performance.
CONCLUS ION The purpose of the experiment was to evaluate whether student performance was improved when CMS was used, after controlling for cognition, learning style, and computer attitude. Computer attitude was introduced as a variable that may influence the learning process and, therefore, the relationship between CMS and performance. The data collected in this study supported the hypothesis that cognition influences performance. Learning style was found not to influence performance. Cognition and learning styles are frequently cited in the literature as influencing performance both with technology and without technology, so this finding partially supports the literature. The hypothesis that CMS influences performance was not supported. Cognition and learning styles were believed to influence performance based on prior literature. The question addressed in this study was whether or not CMS would influence performance after controlling for the cognition and learning style.
The standard belief in higher education is that CMS is beneficial in the learning process. While the empirical evidence to support such benefit is not found in this study, there may be potential to derive efficiencies from course management systems that would indirectly improve the education process. If the use of these systems gives faculty more time away from mundane tasks in order to further develop curriculum, this could indirectly improve the performance and satisfaction of the students. The training required in integrating CMS and the course development issues encountered by faculty to use the technology may cause them to rethink approaches used in the classroom, thus improving the learning process. The learning process may improve because of more student/faculty interaction using communication capabilities, faster performance feedback, and potential tracking of student effort by monitoring the number of computer accesses and duration of use. By considering the effects of cognition and technology acceptance in conjunction with CMS, instructors may be better able to develop courses and instructional techniques that improve the ultimate outcome for the students.
ReferenCES Ackerman, P. L. (1987). Individual differences in skill learning: An integration of psychometric and information processing, perspectives. Psychological Bulletin, 102(3-27). Agarwal, R., & Day, A. E. (1998). The Impact of the Internet on Economic Education. Journal of Economic Education, Spring, 99-110. Aggarwal, A. K., & Legon, R. (2006). Case study Web-based education diffusion. International Journal of Web-Based Learning and Teaching Technologies, 1(1), 49-72. Ajzen, I., & Fishbein, M. (1980). Understanding Attitudes and Predicting Social-Behavior, Prentice Hall, Englewood Cliffs, NJ.
49
Cognition, Technology, and Performance
Al-Khaldi, M. A., & Al-Jabri, I. M. (1998). The relationship of attitudes to computer utilization: New evidence from a developing nation. Computer in Human Behavior, 14(1), 23-42.
Drummond, R. J. (1987). Review of Learning Styles Inventory, in Test Critiques, Sweetland, D. J. K. R. C. (Ed.) Test Corporation of America, Kansas City, Missouri, 308-312.
Allinson, C. W., & Hayes, J. (1990). The validity of the Learning Styles Questionnaire. Psychological Reports, 67, 859-866.
Dunn, R. (1996). How to implement and supervise a learning style program, Association for Supervision and Curriculum Development, Alexandria, VA.
Ausburn, L. J., & Ausburn, F. B. (1978). Cognitive styles: Some information and implications for instructional design. Educational Communication and Technology, 26(4), 337-354. Bissell, J., White, S., & Zivin, G. (1971). Sensory modalities in children’s learning, in Psychology and educational practice, Lesser, G. S. (Ed.) Scott, Foresman, & Company, Glenview, IL. Brown, A. L., Bransford, J. D., Ferrara, R. A., & Campione, J. C. (1983). Learning, remembering, and understanding, in Handbook of Child Psychology: Cognitive Development, Wiley, 77-166. Burke, J. (1994). Education’s new challenge and choice: Instructional technology--Old byway or superhighway? Leadership Abstracts, 7(10), 22-39. Chau, P. Y. K. (2001). Influence of computer attitude and self-efficacy on IT usage behavior. Journal of End User Computing, 13(1), 26-33. Chang, K. T., & Lim, J. (2006). The role of interface elements in Web-mediated interaction and group learning: Theoretical and empirical analysis. International Journal of Web-Based Learning and Teaching Technologies, 1(1), 1-28. Cotten, K. (1991, 8/31/01). Computer Assisted Instruction. Retrieved June 11, 2003, from http:// www.nwrel.org/scpd/sirs/5/cu10.html Cox, P. W., & Gall, B. G. (1981). Field dependenceindependence and psychological differentation, Educational Testing Service, Princeton. Doucette, D. (1994). Transforming teaching and learning using information technology. Community College Journal, 65(2), 18-24. 50
Dunn, R., & Griggs, S. (1998). Learning styles: Quiet revolution in American secondary schools, National Association of Secondary School Principals, Reston, VA. Dunn, R. S., & Dunn, K. J. (1979). Learning styles/teaching styles: Should they. Can they be mateched? Educational Leadership, 36, 238244. Eason, K. D., & Damodaran, I. (1981). The needs of the commercial users, in Computer Skills and the User Interface, Atly, M. J. C. J. I. (Ed.) Academic Press, New York, NY. Ekstrom, R. B., French, J. W., Harman, H. H., & Dermen, D. (1999). Manual for Kit of FactorReferenced Cognitive Tests 1976, Educational Testing Service, Princeton, NJ. Farnham-Diggory, S. (1990). “Schooling”: The developing child, Harvard University Press, MA, Cambridge. Ferrell, B. G. (1983). A factor analytic comparison of four learning-style instruments. Journal of Educational Psychology, 75, 33-39. Fleming, N. D., & Bonwell, C. C. (2002). VARK pack Version 4.1. Fleming, N. D., & Mills, C. (1992). Helping students understand how they learn, Magma Publications, Madison, Wisconsin. Foreman, J., & Widmayer, S. (2000). How online course management systems affect the course. Journal of Interactive Instruction Development, Fall, 16-19.
Cognition, Technology, and Performance
Gadzella, B. M. (1995). Differences in academic achievement as a function of scores on hemisphericity. Perceptual and Motor Skills, 81, 153-154.
Keefe, J. W. (1985). Assessment of learning style variables: The NASSP task force model. Theory Into Practice 24, 138-144.
Guskin, A. E. (1994). Reducing student costs & enhancing student learning, Part II: Restructuring the role of faculty. Change, 26(5), 16-25.
Keefe, J. W. (1987). Learning style theory and practice, National Association of Secondary School Principals, Reston, VA.
Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1998). Multivariate Data Analysis, Prentice-Hall, Inc., Upper Saddle River, New Jersey.
Keefe, J. W., & Monk, J. S. (1990). Learning style profile examiner’s manual, National Association of Secondary School Principals, Reston, VA.
Hall, Dianne J., Cegielski, Casey G., Wade, James N. (2006). Theoretical value belief, cognitive ability, and personality as predictors of student performance in object-oriented programming environments. Decision Sciences Journal of Innovative Education 4(2), 237-257. Harrison, A. W., & Rainer, R. K. J. (1992). The influence of individual difference on skill in end-user computing, Journal of Management Information Systems, 9(1), 93-111. Holliday, W. G. (1976). Teaching verbal chains using flow diagrams and text. AV Communications Review, 24(1), 63-78. Hunter, J. E. (1986). Cognitive ability, cognitive aptitude, job knowledge, and job performance. Journal of Vocational Behavior, 29, 340-362. Issroff, K., & Scanlon, E. (2002). Using technology in higher education: An activity theory perspective. Journal of Computer Assisted Learning, 18, 77-83. James, W. B., & Blank, W. E. (1993). Review and critique of available learning-style instruments for adults. New Directions for Adult and Continuing Education, 39(Fall). Kaput, J. (1992). Technology and mathematics education. In Handbook or research on mathmematics and teaching and learning (D. A. Grouws ed., pp. 515-556). New York, NY: Macmillan Publishing Company.
Kimmel, P. D., Weygandt, J. J., & Kieso, D. E. (2004). Financial accounting tools for business decision making, John Wiley & Sons, Inc., New York, NY. Klausmeier, H. J., & Loughlin, L. J. (1961). Behaviors during problem solving among children of low, average, and high intelligence. Jouranl of Educational Psychology, 52, 148-152. Kulik, C.-L. C., Kulik, J. A., & Shwalb, B. J. (1986). The effectiveness of computer-based adult education: A meta-analysis. Journal of Educational Computing Research, 2(2), 235-252. Kulik, J. A., Kulik, C. L. C., & Cohen, P. A. (1980). Effectiveness of computer-based college teaching: A meta-analysis of findings. Review of Educational Research, 50(4), 525-544. Lang, T. K., & Hall, D. (2006). Academic motivation profile in business classes. Academic Exchange Quarterly, Fall 2005, 145-151. Leeper, J. D. (2004). Choosing the correct statistical test. Retrieved February 27, 2004. Litzinger, M. E., & Osif, B. (1993). Accommodating diverse learning styles: Designing instruction for electronic information sources, in What is good instruction now? Library instruction for the 90s, Shitaro, L. (Ed.) Pierian Press, Ann Arbor, MI, Pierian Press Liu, Y., & Ginther, D. (Fall1999). Cognitive styles and distance education. Online Journal of Distance Learning Administration, II(III).
51
Cognition, Technology, and Performance
Luyben, P. D., Hipworth, K., & Pappas, T. (2003). Effects of CAI on academic performance and attitudes of college students. Teaching of Psychology, 30(2), 154-158.
Roy, M. H., & Elfner, E. (2002). Analyzing student satisfaction with instructional technology techniques. Industrial and Commercial Training, 34(7), 272-277.
Moran, A. (1991). What can learning styles research learn from cognitive psychology? Educational Psychology, 11(3/4), 239-246.
Russell, T. (2002). The “No Significant Difference Phenomenon” Website. Retrieved September 14, 2003, from http://teleeducation.nb.ca/nosignificantdifference
Morgan, G. (2003). Faculty use of course management systems, University of Wisconsin System, Boulder, CO. Nickell, G. S., & Pinto, J. N. (1986). The computer attitude scale. Computers in Human Behavior, 2, 301-306. O’Boyle, M. W. (1986). Hemispheric laterality as a basis for learning: What we know and don’t know., in Cognitive classroom learning: Understanding, thinking, and problem solving, Andre, G. D. P. T. (Ed.) Academic Press, New York, 21-48. Bond-Raacke, J. (2006). Students’ attitudes towards introduction of course Web site. Journal of Instructional Psychology, 33(4), 251-255. Rainer, R. K. J., & Miller, M. D. (1996). An assessment of the psychometric properties of the computer attitude scale. Computers in Human Behavior, 12(1), 93-105. Riding, & Buckle, C. F. (1990). Learning styles and training performance, Training Agency, Sheffield. Riding, & Cheema (1991). Cognitive styles -- An overview and integration. Educational Psychology, 11(3), 193-215.
52
Sewall, T. J. (1986). The measurement of learning styles: A critique of four assessment tools. ERIC Document 261 247 Simonson, M. (2007). Course Management Systems. The Quarterly Review of Distance Education, 8(1), xii-ix. Sims, R. R., Veres, J. G., Watson, P., & Buckner, K. E. (1986). The reliability and classification stability of the learning styles inventory, in Educational and Psychological Measurement, 753-760. Smith, I. M. (1964). Spatial ability, Knapp, San Diego, CA. Sonnier, I. L. (1991). Hemisphericity: A key to understanding individual differences among teachers and learners. Journal of Instructional Psychology, 18(1), 17-22. Tennant, M. (1988). Psychology and adult learning, Routledge, London. Winn, W. D. (1982). The role of diagrammatic representation in learning sequence, identification, and classification as a function of verbal and spatial ability. Journal of Research in Science Teaching, 19(1), 79-89.
53
Chapter IV
The Role of Organizational, Environmental and Human Factors in E-Learning Diffusion Kholekile L. Gwebu University of New Hampshire, USA Jing Wang Kent State University, USA
AbSTRra Improvements in technology have led to innovations in training such as Electronic Learning (E-learning). E-learning aims to help organizations in their training initiatives by simplifying the training process and cutting cost. It also attempts to help employees in their learning processes by making learning readily accessible. Unfortunately, the diffusion of this innovation has not been as successful as was initially predicted. In this paper we explore the drivers behind the diffusion of e-learning. Apart from the factors investigated by previous research, we believe that one more dimension, -human factors- should be taken into account when evaluating the diffusion of a training innovation, since learners are, to a large extent, the central issue of training. In the case of e-learning we believe that motivation plays a key role in the diffusion of e-learning.
INTRODUCTION With the rapid improvement in technology and the growing demand for a knowledge- based labor force, the demand for e-learning has grown considerably over the past few years. E-learning has provided organizations and employees with
tremendous advantages over traditional training (Li & Lau, 2006). It transcends the limitation of time and space and has been reported to provide companies with time and cost saving benefits in the long run (Li & Lau, 2006; Ong, Lai, & Yishun, 2004; Zhang, 2004). In a recent report, Deloitte and Touche (2002) spell out some of the major
Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.
The Role of Organizational, Environmental and Human Factors
advantages of e-learning including: increased volume of training, geographic distribution and reusability of content. E-learning gives organizations the ability to simultaneously train a larger percentage of employees than does traditional classroom-based training as employees can be trained anytime from anywhere. Moreover, it permits large dispersed organizations to train all their employees with homogeneous content. This is extremely useful for organizations that want to ensure that employees gain standardized skills and knowledge. Furthermore, it has the power to bring people together for collaborative learning (Zhang & Nunamaker, 2003). Along with its unique advantages, the improvements in technology have also facilitated the adoption and implementation of e-learning. The wide accessibility of the internet, increased bandwidth, better delivery platforms, and the growing selection of high-quality e-learning products have all added to the feasibility and attractiveness of e-learning (McCrea, Gay, & Bacon, 2000). The strategic importance of e-learning is real and many companies have been investing heavily in this education sector (Huynh, Umesh, & Valacich, 2003). In fact, 95% of the respondents of an American Society for Training and Development survey conducted in 2003 indicated that they had used some form of e-learning in their organizations (Renée, Barbara, & Eduardo, 2005) . However, many e-learning initiatives are not living up to initial expectations. According to a study done by the Silicon Valley World Internet Center on corporate e-learning (Duggan & Barich, 2001), out of 44 respondents, only 21% indicated a very high level of executive confidence in e-learning; 58% regarded top management confidence as moderate, 15% unknown and 6% as very low. Additionally, a number of studies have suggested that a large number of e-learning initiatives fail (Hamid, 2001). Such findings have spurred research which attempts to identify factors which contribute to the success of e-learning.
54
One research stream has primarily focused on the effect of technology on the success of e-learning. Researchers have indicated that text-based elearning systems tend to make learners disengaged and have proposed the use of different multimedia systems in e-learning (Merchant, Kreie, & Cronan, 2001; Zhang, 2004). Prototype systems that are multimedia integrated are also developed and tested to demonstrate the important role of technology in e-learning (Sampson, Karagiannidis, & Cardinali, 2002; Zhang, 2004). Although such studies have improved our understanding of the alignment between different technologies and e-learners, they remain hampered by one major limitation: they adopt a technological deterministic view and postulate direct links between technology and e-learning success. By its very nature, such an approach propagates technological materialism and amplifies technology specifics. Human action, interpretation, and organizational and environmental contexts play little role in this stream of research. Hence, this approach provides relatively little detail about the organizational contexts and human action that shape the observed e-learning outcome. Such a materialistic view diminishes the importance of human agency, organizational structures, and complex social environments and falls short in explaining why identical e-learning technologies succeed in some organizations but fail in others. Hence, an adequate understanding of the factors that facilitate the success of corporate e-learning requires a more balanced view which does not privilege technology over human agency and the social context (Bruckman, 2002). Another research stream has challenged the technological deterministic view and has focused on the social aspects of e-learning. A number of studies have examined the way in which organizational culture (Harreld, 1998; Nurmi, 1999), the trainers (Chute, Thompson, & Hancock, 1999; Wagner & Reddy, 1987), and differences in individual training styles (Clariana, 1997; Cohen, 1997) influence e-learn-
The Role of Organizational, Environmental and Human Factors
ing initiatives. Although this body of literature has the potential to offer a richer understanding of the role of social forces in e-learning, it tends to employ a uni-dimensional approach, isolating the influence of human agency from the influence of organizational structures on e-learning. Following the second stream of research, this study challenges the determinist view of technology and focuses on the social factors facilitating the success of e-learning. However, it differs from prior work in that it postulates that both human agency (the e-learners) and the organizational context play an important role in the outcome of corporate e-learning initiatives. Focusing on only one dimension provides only part of the story. Therefore, the goal of this study is to develop an integrated conceptual model that explains the influences of both the organizational and the human factors on the successful diffusion of e-learning in contemporary corporations. While various organizational and individual factors can be explored, we limit the scope of our conceptualization to two organizational factors and one individual factor: organizational complexity, bureaucratic control, and motivation. Our proposed conceptual model emerges at the crossroads of four areas of inquiry: e-learning, motivation, the innovation diffusion literature, and technology-related changes. Research in technology-related organizational changes provides an overview of the limitations of the technological deterministic perspective in explaining the outcome of e-learning initiatives. The e-learning, motivation, and innovation diffusion literature helps us conceptualize the relationship between the identified organizational and individual factors and the success of e-learning. The rest of the paper is organized as follows. The next section defines major terms used throughout the paper in order to eliminate ambiguity which may arise from differences in terminology. This is largely due to the vast number of definitions that have emerged from different fields of study over the years. A literature review on e-learning, innovation, and motivation is then conducted, fol-
lowed by a set of propositions. In the methodology section of the paper, we describe the manner in which a pilot study is conducted. Subsequently, we present our findings and engage in discussion of the results of the study. Finally, conclusions are drawn from the result analysis, a discussion of the limitations of this study is presented, and areas of future research are proposed.
DEFINITIONS E-Learning Over the years the term “e-learning” has been used and interpreted in many different ways in the literature. Some authors use it to refer to the use of any form of electronic learning tool such as radio, television, or computers, to deliver learning materials. Urdan and Weggen (2000) describe e-learning as “the delivery of content via all electronic media, including the internet, intranets, extranets, satellite broadcast, audio/ videotape, interactive TV and CD-ROM” .They also use the term synonymously with technology-based learning. In this paper we will use the term e-learning to refer to two forms of contemporary learning. These include Web-based and computer-based learning. Web-based learning represents learning conducted via the Internet or an Intranet or Extranets or a combination of all three. Computerbased learning on the other hand only includes learning which utilizes CD-ROM or other training technology on a stand alone personal computer.
Diffusion Rogers (1995) described the diffusion of innovations as “the process by which an innovation is communicated through certain channels over time among members of a social system”. Based on this definition, the major players in any innovation diffusion process should be the participants,
55
The Role of Organizational, Environmental and Human Factors
also referred by Roger as adopters. This has lead to many researchers and practitioners using the “number” of adopters as a measure of successful diffusion. However, research in information systems and organizational change has long suggested that adoption differs from the real usage of a technology. Using only the number of adopters as the measure of e-learning diffusion success neglects one important principle of e-learning—its goal is to enable organizations to enhance their effectiveness and competitive position (Phillips, 1997). Therefore, rather than solely considering the number of adopters as the indicator of success, we use a perceptional measure, in other words, managers’ perception of how successful the companies’ e-learning initiative is, as an indicator for the success of e-learning diffusion.
LITERATURE REVIEW The literature is examined in three distinct sections: 1.e-learning, 2. innovation theories, and 3.motivation theories. The purpose is to first investigate the research that been done in these three fields then to explore how innovation, motivation and e-learning interact in an organizational context.
E-Learning With its increasing popularity and strategic importance, e-learning has received ample attention both from practitioners and scholars. In the industry, companies have and continue to invest heavily and deploy various technologies including learning management systems, learning content management systems, and reusable learning objects. In the academia, a considerable number of studies have investigated the effects of different technologies on e-learning outcomes. For examples, some studies have examined the effects of text-based systems versus multimedia systems on e-learning success (Merchant et al.,
56
2001; Zhang, 2004). Different prototypes and architectures have also been developed in order to improve the outcome of e-learning (Sampson et al., 2002; Zhang, 2004). Although technology plays an important role in e-learning, one should not presume that if a learning system is built, learners will quickly begin to use it. Studies primarily focusing on the materialistic attributes of e-learning systems assume technology to be an objective, external, and independent force that has relatively deterministic impact on e-learning outcomes. Such studies yield seemingly universal claims as “introduction of good e-learning technology will lead to more successful diffusion of e-learning.” But as management in many organizations has discovered, the availability of sophisticated e-learning technologies does not guarantee success (Servage, 2005) and a large number of organizational elearning endeavors failed in the past (Hamid, 2001). A major concern with the deterministic view of technology is that it downplays the role of human action, interpretation, and organizational and environmental contexts in e-learning and provides little insight into how e-learning success is shaped by such social forces. Such a view is insufficient in explaining why identical e-learning technologies yield divergent outcomes in different organizations. Researchers in information technologies and organizational studies have long pointed out that an identical technology can be enacted differently in different organizational context (Boudreau & Robey, 2005; Orlikowski, 1992, 1993). They adopt a less deterministic view and propose that both human agency and the social context within which a technology operates play an important role in the outcome of the technology. Similar arguments can be made in the case of e-leaning. Adopting this less deterministic view, a number of studies have focused on the social aspects e-learning. For instance, Harreld (1998) pointed out that it is prohibitive for management to impose new technology and management process changes in
The Role of Organizational, Environmental and Human Factors
an organization where the organization culture is not ready to embrace the changes. Other researchers shift their focus from the organization aspect to the human aspect of e-learning. They address this issue by tackling the trainer’s role or individual training styles. They accentuate that to ensure the successful implementation of e-learning, trainers should assume different roles in e-training to those they typically assume when they engage in classroom-based training. Typical additional roles include instructional designer, instructional developer, materials supporter, technology supporter, facility supporter, and distance-site facilitator (Abernathy, 1998; Chute et al., 1999). Another school of researchers attempts to identify the role of individual learning styles on the effectiveness of e-learning. For example, Clariana (1997) use Kolb, Rubin, and McIntyre’s (1979) Learning Style Inventory (LSI) to study training styles in Computer Assisted Learning (CAL). He found that learning style dimensions shifted after a certain period of exposure to CAL. The degree of the shift varies with a learner’s ability and the length and extent of exposure to CAL. However, another study done by Cohen (1997) did not reveal a learning style shift after one year of exposure to CAL. Other studies also have mixed results. Gunawardena and Boverie (1992) found that there was no significant correlation between learning style and how students interact with media and methods of instruction, but a correlation did exist between learning style and students’ satisfaction. However, Larsen (1992) concluded from her study that both effectiveness and satisfaction are independent of students’ learning styles. While this stream of research has shed valuable insights into some of the social factors that may contribute to the successful implementation of e-learning in organizations, one limitation is that they only focus on a single dimension at a time, isolating the role of organizational factors from that of the human agents. This one-dimensional approach fails to reflect the complexity of reality
and may also partly explain their inconsistent results. This paper proposes an integrated model that incorporates different dimensions—organizational structure, environmental factors and human factors (motivation in particular) and provides an in-depth theoretical discuss on how those dimensions work together to influence the diffusion of e-learning.
CONCEPTUAL FOUNDATION To identify organizational factors that may contribute to the successful diffusion of e-learning, we draw on the rich stream of research on diffusion of innovation. As we limit the scope of our conceptualization to one human factor, in other words, motivation, literature on motivation will be used as the foundation for our theoretical discussion of the relationship between individual motivation and e-learning success.
Innovation Innovation literature is broad and diverse. According to Wolfe (1994), during the five years foregoing his study on organization innovation, 351 dissertations and 1299 journal articles addressing organizational innovation were written. The most basic questions at the heart of most researchers in the field of innovation adoption and diffusion are questions such as “What organizational structures and management processes facilitate or inhibit innovation?” and “Why are some organizations more innovative than others?”(Damanpour, 1991; Fichman, 2001, 2004). Researchers have also attempted to answer questions such as “Why do certain innovations diffuse successfully in certain organizations and not in others?”(Chen, 1983; Fichman, 2001, 2004). E-learning can be conceptualized as an innovation in learning and the rich innovation literature furnishes conceptual tools that aid in the identification of key organizational factors that may contribute to the success of e-learning diffusion. 57
The Role of Organizational, Environmental and Human Factors
Organizational Structural Theories Extant literature on innovation is replete with hypotheses, models, and theories that seek to identify organizational structural factors that facilitate the diffusion of innovations. The theories can all be categorized under the umbrella term Organizational Structural Theories. Structural theories have evolved over the years from uni-dimensional to middle-range theories. As explained by Damanpour and Gopalakrishan (1998), unidimensional theories use organization structural variables such as vertical differentiation (the number of levels in an organization hierarchy) to explain innovation adoption and diffusion. For instance, some researchers posit that vertical differentiation is negatively associated with innovation adoption and diffusion as it increases links in communication channels, thereby inhibiting the flow of innovative ideas (Damanpour & Gopalakrishnan, 1998; Hull & Hage, 1982). Other variables which are commonly considered in these theories are: functional differentiation (the number of different functional units in an organization), specialization (different areas of expertise in an organization), professionalism (professional knowledge including employees’ education and experience), formalization (the degree to which rules and procedures are followed in an organization), and centralization (whether decision making is centralized or distributed) (Damanpour, 1987). One limitation with uni-dimensional theories is that they neglect other dimensions and do not reflect the complexity of the real world (Damanpour & Gopalakrishnan, 1998). Researchers also criticize them due to the inconsistencies between their prediction and the results gathered from empirical studies (Downs & Mohr, 1976). This limitation in uni-dimensional theories leads to the development of Middle- range theories. Middle range structural theories incorporate other contingency factors as an explanation of the contradictory findings in prior work. Examples
58
of middle range structure theories include the dual core theory, which focuses on differentiation between types of innovation (Daft, 1978; Damanpour, 1991), the theory of innovation radicalness which examines incremental versus radical innovations (Dewar & Dutton, 1986), the ambidextrous theory of innovation which focuses on the various stages of the innovation process (Damanpour & Gopalakrishnan, 1998; Zmud, 1982). These theories postulate that the relationship between organization structural variables studied by uni-dimensional theories and diffusion of innovation is contingent on the different types of innovation, or radicalness of innovation, or stages of innovation. In the following section, we will only examine the ambidextrous theory in greater detail. This is because depending on the organization using e-learning, it could be categorized as either an administrative innovation or a technical innovation, or it could be categorized as either a radical innovation or an incremental innovation. This relativity in categorization has made the findings in the dual-core theory and radicalness theory of little value to our study. The ambidextrous theory on the other hand, is of more interest to us since there is no relativity regarding two stages of e-learning innovation. The ambidextrous theory categorizes organizational innovation into two distinct stages, the initiation stage and the implementation stage (Duncan, 1976). The initiation stage consists of all the activities pertaining to problem perception, information gathering, attitude formation and resource development leading to the decision to adopt (Rogers, 1995). This is synonymous with the term adoption which we use in this paper. The implementation stage is composed of all events and actions relating to change in both the innovation and the organization (Duncan, 1976; Rogers, 1995). This stage also incorporates the diffusion of innovation. The ambidextrous theory explores the relationship between organization structural variables and the two stages of the adoption process. Based
The Role of Organizational, Environmental and Human Factors
on its finding, high organization complexity (specialization, functional differentiation, and professionalism) and low bureaucratic control (formalization, centralization, and vertical differentiation) facilitate the adoption of an innovation, while low complexity and high bureaucratic control facilitate the diffusion of innovations (Damanpour & Gopalakrishnan, 1998). Uni-dimensional theories suggest that organizational complexity is positively related to innovation adoption and diffusion whereas the degree of bureaucratic in an organization is negatively associated with innovation adoption and diffusion (Damanpour & Gopalakrishnan, 1998). When applied to e-learning, this would mean that organizations with high organization complexity should expect e-learning to be adopted and diffused while in organizations with a high level of bureaucratic control adoption and diffusion are unlikely. The ambidextrous theory adds one more dimension (stages of innovation) to the unidimensional theory. Besides pointing out that organization complexity has a positive effect while bureaucratic control has a negative impact on both the adoption and diffusion of innovation, the ambidextrous theory also examines the correlation between structural variables and the two stages of innovation. According to the theory, the correlation between organization complexity and initiation of innovation is high while the correlation between organization complexity and diffusion is low. On the other hand, the correlation between organization bureaucracy and initiation of innovation is low while the correlation between bureaucracy and diffusion of innovation is high (Damanpour, 1991). As seen in the above discussion, uni-dimensional as well as middle-range structural theories of innovation usually aim to specify organizational structural characteristics that lead to the adoption and diffusion of innovation. However, most results from empirical studies using structural
theories tend to conflict (Damanpour, 1991). The problem with these theories is that they do not take into account other variables such as the business environment and human factors when attempting to explain how innovations diffuse into an organization.
Other Dimensions Environmental Dimension To explain the variance between the findings in empirical studies conducted using middle-range theories, innovation scholars have attempted to develop more sophisticated and more comprehensive models, which take into consideration multiple dimensions of innovation. For example, Damanpour and Gopalakrishnan (1998) added the environmental context into their framework to investigate how organizational structures and environmental factors influence the adoption of innovation. Damanpour and Gopalakrishnan focus on the dynamism of the environment and further classify environmental dynamism into two components: environmental stability and environmental predictability. These two components have resulted in four combinations of environment characteristics: stable and predictable, stable and unpredictable, unstable and predictable, unstable and unpredictable. They argue that organizations in environments which are stable and predictable tend to adopt few innovations slowly in contrast to organizations in environments which are unstable and unpredictable. Additionally, organizations in both unstable and unpredictable environments tend to have cultivated a culture of innovation. Employees in such organizations are continuously encouraged to make use of new innovations and to be creative. Moreover, the organizational structure, culture, and administrative system encourage employees to be innovative and also make use of innovations. Thus, when it comes to innovations such as e-learning one would expect that orga-
59
The Role of Organizational, Environmental and Human Factors
nizations in an environment characterized by instability and unpredictability, most members would have at some point experimented with elearning packages. However, the sustained use of this innovation would largely be dependent on the human factors which are discussed below.
Human Factors Although organization structural variables and the environment provide important insights into various factors which may influence the adoption and in some cases the diffusion of innovations, they cannot fully explain why certain innovations such as e-learning tend to diffuse in some organizations and not in others. We believe that when analyzing the diffusion of innovations that involve the transformation of human resources such as e-learning, human factors need to be taken into consideration. This argument is consistent with the recent development in information systems research and organizational change. While earlier work in these two fields tends to privilege technology over human agency, there is an increasing tendency to emphasize the importance of human agents in enacting technologies (Boudreau & Robey, 2005; Orlikowski, 1992, 1993). This human agency position suggests that individuals do not passively accept and use technology. In contrast, they actively enact technologies in different ways. They can use it minimally, maximally, or improvise in ways that are hard to anticipate. Hence, it stands to reason that human factors should be incorporated in order to have a complete understanding of e-learning success. In particular, we focus on one human factor, motivation. Motivation has been suggested as being one of the key drivers of users’ technology acceptance and subsequent usage behavior.
Motivation Motivation theories seek to identify and explain the factors that influence human behavior, particu-
60
larly the way in which individuals react to the actions of others around them and the stimuli in their environment (Wilkinson, Orth, & Benfari, 1986). Over the years several general theories on motivation have emerged, the most discussed being Maslow’s Hierarchy of Needs theory, Alderfer’s ERG Theory, Herzberg’s Two Factor Theory and McClelland’s Learned Needs Theory. Motivation theories can be categorized into three broad categories namely, Content Theories, Process Theories, and Reinforcement Theories. Content theories advocate that individuals are motivated to fulfill needs. As stated by Knoop (1994), Content Theories focus on identifying values conducive to, but not necessarily causal to, job satisfaction. Maslow’s hierarchy of Needs Theory, Alderfer’s ERG Theory and Herzberg’s Two-Factor Theory, all fall under the category of content theories. Process theories on the other hand are mainly concerned with explaining the manner in which people think and behave to get what they want. Motivation Theories included under this category include the Equity Theory and the Expectancy Theory. Reinforcement Theories are concerned with the effects of rewards upon motivated behavior. McClelland’s Acquired Needs Theory is an example of such a theory. Although these theories differ in how they define motivation, they all agree that if people are not motivated they will not engage in certain behavior.
Drivers of Motivation Drawing from Motivation Theory we were able to identify two broad drivers of motivation for human behavior, namely, intrinsic motivation and extrinsic motivation. Davis, Bagozzi, and Warshaw (1992) found intrinsic and extrinsic motivation to be key drivers of behavioral intention to use a technology. Vallerand (1997) mentions that intrinsic motivation refers to the pleasure and inherent satisfaction derived from a specific activity. Intrinsic rewards are those that come from the work itself, for instance, the feeling of
The Role of Organizational, Environmental and Human Factors
accomplishment and success one experiences from performing a task they enjoy. Both the nature of the task and the compatibility of the person with the task primarily influence intrinsic rewards. Venkatesh (1999) states that research in psychology advocates that intrinsic motivation during training leads to favorable outcomes. However, individuals tend to determine intrinsic motivation thus organizations cannot have a huge impact on various outcomes determined by intrinsic motivation. Extrinsic motivation highlights performing a specific behavior in order to achieve a specific goal (Deci & Ryan, 1987). Extrinsic rewards are those that have are not necessarily related to a task, but have strong motivational effects. Examples of these include pay, benefit, and recognition programs, and are influenced primarily by the organization. Organizations may make use of extrinsic rewards to attempt to motivate employees to achieve or perform various tasks and activities.
E-Learning and Motivation People in organizations differ from one another in their motivation to learn and participate in training programs (Ukens, 2001). Most motivational theories have not been able to agree on whether people are primarily motivated by intrinsic or extrinsic factors. Bruno and Osterloh (2002) argue that intrinsic and extrinsic motivation are interlinked and as such companies cannot opt for one or the other in isolation. It is therefore imperative that both be considered when attempting to assess whether or not employees are motivated to perform certain tasks, in this case e-learning. Research suggests that motivation in general (whether it is intrinsic or extrinsic) is an important factor driving perceptions and behavior, even in a training context (Bruno & Osterloh, 2002; Pierce & Delbecq, 1977). Compared to traditional classroom-based learning initiatives, corporate e-learning initiatives are particularly susceptible
to high dropout rates because participation is typically voluntary and often goes unsupervised. Oftentimes, companies require that e-learning courses be taken outside the work environment during one’s own time. However, studies have shown that most employees not only prefer to take e-courses during work hours but they also prefer to take them in the workplace (ASTD, 2006). If employees are not self-motivated to work outside the work environment because they cannot perceive any meaningful benefits to such work, they are likely to drop out of e-learning programs. Previous research has found that individuals tend to persist at activities that are intrinsically motivating (Rieber, 1991). Rosenberg (2001) argues that employees will only embrace learning when they perceive direct relevance and benefit (extrinsic motivation) of the learning program for themselves and when they sense support from the firm. It is therefore imperative that firms understand the factors which motivate employees (such as firm support in terms of time and sponsorship) to engage in e-learning in order to ensure its successful diffusion into the organization. When applied to the context of e-learning adoption, the theories of motivation are consistent with other technology adoption theories. The Technology Acceptance Model (TAM) proposed by Davis (1989) suggests that when users are presented with a new technology, the factors that influence their decision on whether or not to use it include the perceived usefulness of the innovation, in other words. The degree to which the innovation would enhance that employee’s job performance and perceived ease of use, or, the degree to which an employee believes that using a particular innovation would be free of effort. Perceived enjoyment, otherwise known as the extent to which an activity is perceived as being enjoyable in its own right (Davis et al., 1992), has also been found to significantly influence perceived ease of use (Hwang & Yi, 2003). Perceived ease of use of a system and perceived usefulness of the system are both extrinsic moti-
61
The Role of Organizational, Environmental and Human Factors
vators whereas the enjoyment derived from using the system is an intrinsic motivator (Igbaria & Livari, 1995). Therefore, several factors will influence an employee’s perception about a particular technology which in turn will influence actual usage of that technology. Together these factors contribute to an employee’s motivation to make use of a particular technology. To this end, we have seen from the literature that organizational structural variables (Daft, 1978; Nord & Tucker, 1987; Zmud, 1982) and the environment (Damanpour & Gopalakrishnan, 1998) in which an organization operates are key drivers of the diffusion of innovations in general. In the case of e-learning however, we believe that human factors such as motivation of people within an organization also plays a critical role in whether or not e-learning will diffuse. If the majority of the employees of an organization do not feel motivated to engage in e-learning, the e-learning initiative of that organization will not diffuse. This leads to our first proposition: P1. The diffusion of leaning innovations such as e-learning in organizations is not only a function of organizational and environmental factors, but also a function of human factors such as motivation. Furthermore, we believe that human factors such as motivation may in fact moderate the relationship between organizational and environmental variables and the diffusion of e-learning. We therefore propose that the following:
P2. Human factors such as motivation will moderate the relationship between organizational factors and e-learning diffusion. P3. Human factors such as motivation will moderate the relationship between environmental factors and e-learning diffusion.
PROPOSED FRAMEWORK Our proposed framework stems from the above discussion. According to our argument, human factors such as motivation, organizational variables, and the environment all influence e-learning diffusion. The Figure 1 summarizes the proposed relationship between motivation, organizational variables, and environment. The function in Figure 1 suggests that motivation is a key factor in e-learning diffusion. If employee motivation is low or non existent (zero), then e-learning diffusion will not occur (it will also be zero). However, if organizational structural or structural variables alone are poor/low or zero, diffusion may still occur if motivation is high. Figure 2 graphically illustrates the propositions presented above. If motivation to engage in e-learning is high among employees and organization structural variables and environmental variable are positive, then successful e-learning diffusion will be highly likely. If motivation is positive but structural and environmental variables are negative, then diffusion is still likely. However, if motivation is low and structural and environmental variables are positive diffusion is unlikely, and if motivation is low and structural
Figure 1. The relationship between e-learning, motivation, structural variables and the operating environment
organizational
E-learning Diffusion = f [motivation (Organizational Variables + Environment)]
62
The Role of Organizational, Environmental and Human Factors
Figure 2. The relationship between e-learning, motivation, structural variables and the business operating environment-the likelihood of e-learning diffusion 1
4 Likely
Employee Motivation
2
Highly likely
3
Highly Unlikely
Unlikely
Structural Variables & Environment
and environmental variables are negative then e-learning diffusion will be highly unlikely. Therefore, the desirable state for organizations engaged or wishing to engage in e-learning initiatives is to be in the top right-hand portion of the quadrant where employee motivation is high and structural and environmental variables are positive. If an organization is in the top lefthand section (1) of the diagram it needs to focus on structural variables in order to move toward 4. Activities that focus on building organization complexity (specialization, functional differentiation, and professionalism) and reducing organization bureaucratic control (formalization, centralization, and vertical differentiation) are necessary to push such organizations to 4. If an organization is in the bottom right-hand portion of the quadrant (3), it needs to focus on activities which cultivate employee motivation in order to move to 4. These and other activities are summarized in Figure 3 and include actively, providing flexible learning hours for employees, involving top management, providing relevant content, attaching financial and non financial incentives for completing e-learning modules or programs, providing easy to use technology, and making the technology readily accessible for employees. Finally, organizations in the bottom left-hand portion of the quadrant (2) need to improve both
the structural variables and the employee motivation in order to move to 4. However, trying to improve both simultaneously may in many cases prove too much of a challenge. Therefore, an organization may wish to first focus on first improving employee motivation which will move them to 1. Thereafter they may focus of techniques for improving structural elements in order to move to 4.
PILOTDY Although the purpose of the study was to develop rather than empirically test the conceptual framework presented above, we had the opportunity to conduct a pilot study to assess some of the above stated conceptual propositions. Of interest to us was the role of motivation (a human factor) on e-learning diffusion in corporate environments. Data for this study was collected from four corporations based in North East Ohio from both the employees and management.
Instrumentation The instruments used included two questionnaires. One was specifically for the management and the other was specifically for employees.
63
The Role of Organizational, Environmental and Human Factors
Figure 3. Activities that improve employee motivation to engage in e-learning Corporate Support Top management participation and involvement Flexible learning hours Relevant content Incentives Monetary o Cash Bonuses o Base salary increase o Stock options Non-monetary o Promotion to higher profile position o Recognition o Accolades Technology Easy to use Enjoyable to use Readily Accessible Technical support
The questionnaire for managers was developed by the researchers based on the literature review on e-learning, innovation theories, and motivation theories. The questionnaire for employees was based on an instrument formed by Wherry and South (1977) and the literature review. The questions in both questionnaires regarding employee motivation, organization structure, and effectiveness of e-learning were formulated to be answered using a five-point Likert scale with 1=strongly disagree, 2= disagree, 3= neutral, 4= agree, and 5=strongly agree.
The Management Questionnaire Considering that management would have better knowledge about the aggregate information of the company, we designed a questionnaire specifically for the management. The aim of the management questionnaire was to assess organizational structural variables such as Organizational Complexity (specialization, function differentiation, professionalism) and Bureaucratic Control (formaliza-
64
tion, centralization, vertical differentiation). Based on the literature review on innovation theories, 6 questions were designed to measure the organization structural variables. Although, the main thrust of the management questionnaire was to determine the various structural variables in the solicited organization, we also added a second dimension to assess the effectiveness of e-learning diffusion and management’s perceptions about the benefits of e-learning. Additionally, a question was also included to assess the environment in which the organizations are currently operating in. This question asked about the level of competition perceived by management. The final part the management questionnaire sort general personal information about the manager, such as their age, gender, and educational background.
The Employee Questionnaire The employee questionnaire aimed to address the issue of employee motivation to engage in e-learning. The questionnaire was based on an
The Role of Organizational, Environmental and Human Factors
instrument formed by Wherry and South (1977) to evaluate employee motivation. The instrument evaluates eight broad categories of intrinsic and extrinsic motivation namely: (1) responsibility, (2) challenge vs. boredom, (3)high level of activity, (4) goal orientation, (5) social recognition, (6) being appreciated, (7) being judged reliable, and (8) immediate gratification. The first four items in the above list cover intrinsic motivation while the last four cover extrinsic motivation (Wherry & South, 1977). Seventy sample questions are given in the Wherry and South (1977) instrument which, may be used to evaluate the eight broad categories of intrinsic and extrinsic motivation. We took some of these questions and used them in an e-learning context. Our questionnaire was divided into two distinct sections. A five-point Likert scale was used in the first section to assess questions about intrinsic and extrinsic motivation, while the second part of the employee questionnaire was comprised of general questions such as employee age, gender, years of working experience, and level of education.
FINDINGS The Demographic data of the respondents is summarized in Table 1. Table 2 outlines some of the findings of the pilot study, regarding the independent variables motivation, environment,
organizational structure and the dependent variable successful diffusion of e-learning. The second column in Table 2 (Motivation x1) shows the mean motivation levels of employees to engage in e-learning. For all companies on average employees seem motivated to engage in e-learning but the degree of motivation varies from company to company. The next column (Environment x 2) shows the environment in which each of the firms operates. Recall from the literature that an organization can either be in a stable and predictable environment or an unstable and predictable environment or a stable and unpredictable environment or an unstable and unpredictable environment. If an organization is in an unstable and unpredictable environment a score of +1 was assigned. If an organization is in an unstable and predictable or stable and unpredictable environment the environment is considered neutral and a score of 0 was assigned. Finally if an organization was in an environment that is stable and predictable environment a score of -1 was assigned. On average all the companies appear to be operating in an unstable and unpredictable environment. Next, organizational complexity and bureaucratic control variables were considered. Two companies, (1 and 4) were not complex in structure, while company 3 was complex and company 2 was somewhere in the middle. Previous research (Damanpour, 1991; Hull & Hage, 1982) suggests that there is a negative relationship bureaucratic
Table 1. Employee demographics Gender
Industry
Education Level
Male
Female
Service
Manufacturing
Bachelor Degree
No Bachelor Degree
65%
35%
43%
57%
81%
19%
65
The Role of Organizational, Environmental and Human Factors
Table 2. Descriptive statistics Variable Company
Motivation (x1)
Environment (x2)
Diffusion (x4)
1
0.46
1
C = -0.33 B = 0.00
0.5
2
0.54
1
C = 0.00 B = -0.67
0.75
3
0.51
1
C = 0.67 B = 0.67
0.5
4
0.54
1
C = 1.00 B = 0.33
1
Range of values for variables
-1≤ x1≤1
-1≤ x2≤1
-1≤ x3≤1 -1≤ C≤1 -1≤ B≤1
0≤ x4≤1
control and innovation, therefore a score of -1 was given to companies that are displayed high levels of bureaucratic control and a score of 1 was given to those which displayed low levels of bureaucratic control. Half the companies appear to have some level of bureaucratic control. Finally, successful diffusion of e-learning was assessed. The diffusion of e-learning could either be successful or unsuccessful. If fully successful a score of 1 was assigned while if unsuccessful a score of 0 was assigned. Companies scoring 0.5 and below were considered to be unsuccessful in their e-learning diffusion initiative while those scoring above 0.5 were considered successful in the diffusion of their e-learning initiative. Company 2 and 4 appear to be successful while company 1 and 3 appear to be unsuccessful in the diffusion of their e-learning initiative.
DISCssion The pilot study yielded some interesting results. Although a small and convenient sample was used, which subsequently prevents the research from making any probability statements, the descriptive statistics offered some interesting insights into the various variables examined in the study. Upon examination of the data it is evident that all
66
Organization Structure (x3)
companies reported that they have been operating in an unstable and unpredictable environment, so it is difficult to determine the impact of the environment on the successful diffusion of e-leaning. No pattern is evident from the relationship between organizational variables and the successful diffusion of e-learning, however due to the small sample size it is difficult to draw generalized conclusions from this. A very interesting finding from the pilot, however, is that there seem to be some differences in the motivation levels of the employees to engage in e-learning. Company 1 and 3 had the lowest employee motivation levels and also the lowest levels of e-learning diffusion, while company 2 and 4 which had the highest levels of employee motivation and the highest levels of e-learning diffusion.
CONCLUSIONS AND DIRECTIONS FOR FUTURE RESEARCH The purpose of this study was to conceptually explore the factors that influence e-learning diffusion in contemporary corporations. We treated e-learning as an innovation in learning and applied concepts from literature on organizational change, innovation diffusion, and motivation to
The Role of Organizational, Environmental and Human Factors
assess the factors that influence the diffusion of e-learning in organizations. Among these factors are organizational variables such as Organizational Complexity (specialization, function differentiation, professionalism) and Bureaucratic Control (formalization, centralization, vertical differentiation). Previous literature has also looked into the effect of the environment, so we also factored this into the analysis. However, in the case of e-learning we believe that human factors may play a crucial role in the determination of whether or not an innovation such as e-learning will successfully diffuse into an organization. We considered just one human factor (employee motivation) and suggested that motivation may not only be a factor that contributes towards the successful diffusion of e-learning, but without employee motivation e-learning initiatives in an organization could fail. Armed with this framework, future researchers could empirically explore the relationships between these three factors. We conducted a pilot study to get insights into relationships between employee motivation, organization variable, and the environment and found that in organizations which had lower levels of employee motivation the diffusion of e-learning was not as successful as organizations with higher levels of employee motivation. Clearly the pilot study has several limitations which future research could address. The first major limitation to the study was the small sample size. For the results to be generalized it’s necessary to substantially increase the sample size. A second limitation with the pilot study is the sampling procedure used. A convenient sample of companies is used. Ideally, a random sample should have been used. Without a random sample we cannot and do not make any probability statements and are obliged to only report descriptive statistics. Despite the above limitations of the pilot study, the main contribution of this paper, the conceptual framework, offers researchers the opportunity
to investigate a myriad of research questions related to the successful diffusion of e-learning, by considering e-learning as an innovation in learning and factoring in human factors. In this paper the human factor considered was motivation. Future researchers may wish to look into the role of other human factors. For example they may consider issues such as the impact of the level of technological expertise among employees on e-learning diffusion. This approach of considering organizational, environmental, and human factors together rather than in isolation as have some previous studies, has the potential to advance knowledge on factors which contribute to the successful diffusion of e-learning.
REFERENC Abernathy, D. J. (1998). The WWW of distance learning: Who does what and where? Training and Development, 52(1), 29-30. ASTD. (2006). State of the industry report (Online). Available at www.astd.org Boudreau, M.-C., & Robey, D. (2005). Enacting integrated information technology: A human agency perspective. Organization Science, 16(1), 3-18. Bruckman, A. (2002). The Future of e-learning communities. Communications of the ACM, 45(4), 60-63. Bruno, S. F., & Osterloh, M. (2002). Successful management by motivation: Balancing intrinsic and extrinsic incentives. Berlin Herdelberg: Springer-Verlag Chen, E. K. Y. (1983). Multinational corporations and technology diffusion in hong kong manufacturing. Applied Economics, 15(3), 309-312. Chute, A. G., Thompson, M. M., & Hancock, B. W. (1999). The Mcgraw-Hill handbook of distance learning. New York: McGraw-Hill.
67
The Role of Organizational, Environmental and Human Factors
Clariana, R. B. (1997). Considering learning style in computer-assisted learning. British Journal of Education Technology, 28(1), 66-68. Cohen, V. L. (1997). Learning styles in a technology-rich environment. Journal of Research on Computing in Education, 29(4), 339-350. Daft, R. L. (1978). A dual-core model of organizational innovation. Academy of Management Journal, 21(2), 193-210. Damanpour, F. (1987). The adoption of technological, administrative, and ancillary innovations: Impact of organizational factors. Journal of Management, 13(4), 675. Damanpour, F. (1991). Organizational innovation: A meta-analysis of effects of determinants and moderators. Academy of Management Journal, 34(3), 555-590.
empirical analysis. Management Science, 32(11), 1422-1433. Downs, G. W., & Mohr, L. B. (1976). Conceptual issues in the study of innovation. Administrative Science Quarterly, (21), 700-714. Duggan, S., & Barich, S. (2001). The knowledge economy and corporate e-learning. CA: The Silicon Valley World Internet Centre. Duncan, R. B. (1976). The ambidextrous organization: Designing dual structures for innovation. In R. H. Kilmann, L. R. Pondy & S. D. P. (Eds.), The Management Of Organization: Strategy And Implementation (Vol. 167-188). New York: North- Holland. Fichman, R. G. (2001). The role of aggregation in the measurement of it-related organizational innovation. MIS Quarterly, 25(4), 427-455.
Damanpour, F., & Gopalakrishnan, S. (1998). Theories of organizational structure and innovation adoption: The role of environmental change. Journal of Engineering and Technology Management, 15(1), 1-24.
Fichman, R. G. (2004). Going beyond the dominant paradigm for information technology innovation research: Emerging concepts and methods. Journal of the Association for Information Systems, 5(8), 314-355.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 318-340.
Gunawardena, C. N., & Boverie, P. E. (1992). Impact of learning styles on instructional design for distance education. Paper presented at the World Conference of the International Council of Distance Education,, Bangkok Thailand.
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1992). Extrinsic and intrinsic motivation to use computers in the workplace. Journal of Applied Social Psychology, 22(14), 1111-1132. Deci, E. L., & Ryan, R. M. (1987). The support of autonomy and the control of behavior. Journal of Personality and Social Psychology, 53(6), 1024-1037. Deloitte Research. (2002). From e-learning to enterprise learning, becoming a strategic organization. Retrieved from www.dc.com/research Dewar, R. D., & Dutton, J. E. (1986). The adoption of radical and incremental innovations: An
68
Hamid, A. A. (2001). E-Learning-Is it the’’E’’or the learning that matters? The Internet and Higher Education, 4(3-4), 311-316. Harreld, J. B. (1998). Building faster, smarter organizations. In D. Tapscott, A. Lowy & N. Klym (Eds.), Blueprint the digital economy: Creating wealth in the era of e-business. New York: McGraw Hill. Hull, F., & Hage, J. (1982). Organizing for innovation: beyond burns and stalker’s organic. Sociology, 16(4), 564-577.
The Role of Organizational, Environmental and Human Factors
Huynh, M. Q., Umesh, U. N., & Valacich, J. S. (2003). E-Learning as an emerging entrepreneurial enterprise in universities and firms. Communications of the Association for Information Systems, 12, 48-68. Hwang, Y., & Yi, M. Y. (2003). Predicting the use of web-based information systems: Self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model. International Journal of Human-Computer Studies, 59(4), 431-449. Igbaria, M., A., & Livari, J. (1995). The effects of self-efficacy on computer usage. OMEGA International Journal of Management Science., 23(6), 587-605. Knoop, R. (1994). Work values and job satisfaction. Journal of Psychology, 128(6), 683. Kolb, D. A., Rubin, I. M., & McIntyre, J. M. (1979). Organizational Psycho1ogy:An Experiential Approach (3rd ed.). Englewood Cliffs, NJ: Prentice-Hall. Larsen, R. E. (1992). Relationship of learning style to the effectiveness and acceptance of interactive video instruction. Journal of Computer-based Instruction, 19(1), 17-21. Li, F. W. B., & Lau, R. W. H. ( 2006). On-demand e-learning content delivery over the internet. International Journal of Distance Education Technologies, 4(1), 46-55. McCrea, F., K. , Gay, R., & Bacon, R. (2000). Riding the big waves: A white paper on the B2B e-learning industry. In (pp. 1-51.). San Francisco: Thomas Weisel Partners. Merchant, S., Kreie, J., & Cronan, T. (2001). Training end users: Assessing the effectiveness of multimedia CBT. Journal of Computer Information Systems, 41(3), 20-25. Nord, W. R., & Tucker, S. (1987). Implementing routine and radical innovation. Lexington, MA: Lexington Books.
Nurmi, R. (1999). Knowledge-intensive firms. In J. W. Cortada & J. A. Woods (Eds.), The knowledge management yearbook. Boston: Butterworth-Heinemann. Ong, C.-S., Lai, J., & Yishun, W. (2004). Factors affecting engineers’ acceptance of asynchronous e-learning systems in high-tech companies. Information, 41(6), 795-804. Orlikowski, W. J. (1992). The duality of technology - Rethinking the concept of technology in organizations. Organization Science, 3(3), 398-427. Orlikowski, W. J. (1993). Case tools as organizational-change - Investigating incremental and radical changes in systems-development. MIS Quarterly, 17(3), 309-340. Phillips, J. J. (1997). Handbook of training evaluation and measurement methods (3rd ed.). Houston, TX: Gulf Publishing. Pierce, J. L., & Delbecq, A. (1977). Organization structure, individual attitudes and innovation. Academy of Management Review, 2(1), 27. Renée, E. D., Barbara, A. F., & Eduardo, S. (2005). E-Learning in organizations. Journal of Management, 31(6), 920-940. Rieber, L. R. (1991). Animation, incidental learning, and continuing motivation. Journal of Educational Psychology, 83(3), 318. Rogers, E. M. (1995). Diffusion of innovations (Vol. 4th). New York: The Free Press. Rosenberg, M., 2001. . (2001). E-learning: Strategies for delivering knowledge in the digital age. Toronto: McGraw-Hill. Sampson, D., Karagiannidis, C., & Cardinali, F. (2002). An architecture for Web-based e-learning promoting re-usable adaptive educational e-content. Educational Technology and Society, 5(4), 27-36.
69
The Role of Organizational, Environmental and Human Factors
Servage, L. (2005). Strategizing for workplace elearning: Some critical considerations. The Journal of Workplace Learning, 17(5-6), 304-317.
Wherry, R. J., & South, J. C. (1977). A Worker Motivation Scale. Personnel Psychology, 30(4), 613-636.
Ukens, L. (2001). What smart trainers should know: The secrets of success from the world’s foremost experts. San Francisco: John Wiley & Sons, Inc.
Wilkinson, H. E., Orth, C. D., & Benfari, R. C. (1986). Motivation theories: An integrated operational model. SAM Advanced Management Journal, 51(4), 24.
Urdan, T. A., & Weggan, C. C. (2000). Corporate e-learning: exploring a new frontier. New York: Hambrecht & Co.
Wolfe, R. A. (1994). Organizational innovation: review, critique and suggested research. Journal of Management Studies, 31(3), 405-431.
Vallerand, R. J. (1997). Toward a hierarchical model of intrinsic and extrinsic motivation. Advances in Experimental Social Psychology, 27, 271-360.
Zhang, D. (2004). Virtual mentor and the lab system—Toward building an interactive, personalized, and intelligent e-learning environment. Journal of Computer Information Systems, 44(3), 35-43.
Venkatesh, V. (1999). Creation of favorable user perceptions: Exploring the role of intrinsic motivation. MIS Quarterly, 23(2), 239-260. Wagner, E. D., & Reddy, N. L. (1987). Design considerations in selecting teleconferencing for instruction. The American Journal of Distance Education, 1(3), 49-56.
70
Zhang, D., & Nunamaker, J. F. (2003). Powering e-learning in the new millennium: An overview of e-learning and enabling technology. Information Systems Frontiers, 5(2), 207-218. Zmud, R. W. (1982). Diffusion of modern software practices: Influence of centralization and formalization. Management Science, 28(12), 1421-1431.
71
Chapter V
Distance Education:
Satisfaction and Success Wm. Benjamin Martz, Jr. Northern Kentucky University, USA Morgan Shepherd University of Colorado at Colorado Springs, USA
AbSTRra • •
Almost 3.5 million students were taking at least 1 online course during the fall 2006 term. The 9.7 % growth rate for online enrollments far exceeds the 1.5 % growth of the overall higher education student population. (Allen and Seaman, 2007)
By 2006, the distance education industry was well beyond $33.6 billion (Merit Education, 2003). As with most markets, 1 of the keys to taking advantage of this growing market is customer satisfaction. Therefore the greater the student satisfaction in a distance program, the more likely that program will be successful. This paper identifies 5 key components of satisfaction for distance education programs through a student satisfaction questionnaire and factor analysis. A questionnaire was developed using these variables and administered to 341 distance students. The results revealed 5 constructs for student satisfaction in a distance education program (Martz and Reddy, 2005; Martz and Shepherd, 2007). Using these factors as guidance, this paper extends those findings to provide some operational and administrative implications.
Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.
Distance Education
INTRODUCTION – WHY DO WE CRE ABOUT SATISFACTION The education industry is being transformed by the ever-growing presence of distance education. The industry has many segments and ancillary components, including higher education, corporate training, and IT infrastructure. The distance education industry has several market drivers that educators, colleges, and businesses must take a serious look at to successfully implement distance education programs and courses. Howell et al. (2003) posit that those interested in distance learning education must be well informed about the trends within the industry. One overriding concern is that the interest in distance education is driving a huge investment without an understanding of what will make a distance education program successful. The first market driver is the significant market potential. International Data Corporation sets the worldwide value of the e-learning market at $23 billion for 2004. Over 90% of all colleges are expected to offer some form of online courses by 2004 (Institute of Higher Education Policy, 2000). Over 3.5 million students took at least one online course in fall 006 (Allen and Seaman, 2007). The corporate segment of this industry is growing substantially as well. Corporations envision online training warehouses that will save large amounts of training dollars. Estimates have this training warehouse market segment at $11 billion in 2003 (Kariya, 2003). At the same time, major corporations are expanding their corporate university / distance learning programs. AACSB International (1999) offers that there are more than 1600 corporate universities with their eyes on a distance learning component and further predicts there will be more corporate than traditional universities by 2010. The corporate market growth is being driven by the profit potential. Corporate managers and college administrators envision significant cost reduction (Traupel, 2004) and significantly higher
72
demand (Sausne, 2003) for training/education, both of which can create higher profits. For example, elective classes that do not have enough students enrolled in on-campus classes may pick up enough distance students to make teaching the course more feasible (Creahan and Hoge, 1998). The college or school’s mission or charter represents another driver to implement distance education programs. As most educational institutions serve a geographical region, either by charter or mission, a distance-learning program may be a practical method to help satisfy this strategic mission (Creahan and Hoge, 1998). The distance education model can be seen as a way to improve profits and improve the ability to provide education to students who may have trouble coming to a certain geographic location. A second major driver is the dramatic change in career expectations by both employers and employees. These changes are being seen in the concept of life-long learning (Howell, et al. 2003). Today, employees are not expected to stay in the same job for long periods of time. Careers today encompass jobs in multiple industries, combinations of part-time work in multiple jobs, telecommuting, leaving and re-entering the full-time work force and switching jobs more often than in the past. Today’s employee readily envisions the need to maintain a level of knowledge current with the career demands (Boyatzis and Kram, 1999). In contrast to the market drivers, the “commercialization” of education also raises concerns about the basic process of learning (Noble, 1999). By definition, distance education changes the basic paradigm of the education environment. In turn, this means that students will probably respond differently to this environment than they do to the traditional classroom. Some of the responses will be intended and some will be unintended. For example, what are some inhibitors to learning or fundamental problems caused by using a distance learning environment? One problem identified is the lower student retention found in distance programs. Students
Distance Education
seem to find it easier to “drop out” of a class. Carr (2000) reports a 50% drop out rate for online courses. The willingness and ease by which to drop online classes is a real concern for higher education segment, but may be even more problematic for the corporate segment. Corporations that schedule training and then anticipate a certain level of trained employees by the end of the distance class will court disaster if drop out rates are higher than expected. One area of explanation for the high drop out rate is the lack of social interaction found in the distance environment. Haythornthwaite et al., (2000) looked at how social cues such as text without voice, voice without body language, class attendance without seating arrangements and students signing in without attending internet class impacted students “fading back.” They found that the likelihood of students “fading back” is greater in distance learning classes than in traditional face-to-face classes. Other researchers such as Hogan and Kwiatkowski (1998) and Hearn and Scott (1998) argue that the emotional aspects of teaching methods have been ignored in the distance environment and that before adopting technology for distance teaching, education must find a way to supplement the social context of learning. Finally, Kirkman et al. (2002) suggested that trust might be a significant characteristic is the relationship between the student and the instructor in a learning environment. Their concern is that the distance environment may adversely impact this trust component. Another body of research that can be useful in understanding the distance education environment is from researchers studying how people react when introduced to technology. Poole and DeSanctis (1990) suggested a model called Adaptive Structuration Theory (AST). The fundamental premise of the model is that the technology under study is the limiting factor or the constraint for communication and that the users of the technology figure out alternative ways to communicate over the channel (technology). A good example
here is how a sender of email may use combinations of keyboard characters or emoticons (i.e. :) – sarcastic smile, ;) – wink, :o – exclamation of surprise) to communicate more about their emotion on a subject to receiver. Ultimately, whether from an educational perspective or from a corporate perspective, the key to realizing the potential of distance education is trading off the benefits with the concerns to produce a quality product. Not all distance education programs have been a success (NEA, 2002). Peters and Waterman (1982) have popularized the notion that customer satisfaction defines a quality product and a quality product, in turn, leads to a successful company. With these perspectives in mind, we suggest that customer satisfaction is an important measure of quality for distance education programs. Therefore, one of the key leading indicators to a program’s success will be the satisfaction of one of its key stakeholders – its students. It is this leading indicator that we wish to explore further.
METODOLOGY – HOW WE COLLECTED DATA The distance program used in this study is the Distance MBA offered by the College of Business at the University of Colorado in Colorado Springs (UCCS-DMBA). The UCCS-DMBA is one of the largest, online, AACSB-accredited MBA programs in the world. The majority of these students are employed full-time. The program used in this study has been in existence since Fall 1996 and has over 179 graduates. In Fall 2002, the program served 206 students from 39 states and 12 countries. The program offers an AACSB accredited MBA and its curriculum parallels the on-campus MBA curriculum. Close to 33% of the enrolled students are female. The oldest student enrolled is 60 years old and the youngest is 22. The average age of all students enrolled is 35. Over 25 Ph.D. qualified instructors participate in
73
Distance Education
developing and delivering the distance program annually. Recently, the news magazine, US News and World Report (2001) classified the program as one of the top twenty-six distance education programs. Each year, the program administration surveys the students as part of an on-going assessment process to maintain accreditation. To help gather the data we needed, a 49-question questionnaire was developed asking about the concerns identified from literature discussed earlier. Five-point
Likert questions (Strongly Disagree; Disagree; Neutral; Agree; Strongly Agree) were used to have students rate their experience with courses in the DMBA program. The questionnaire was sent to 341 students enrolled in the DMBA program. Data gathered from each subject included the subject’s grade, gender, number of courses taken, student status, amount of time expected to spend in the reference course, and the amount of time actually spent in the reference course (Martz et al., 2004).
Table 1. Questions that correlate significantly to satisfaction
ID
Question Statement
Correlation Coef.
74
Sign.
16
I was satisfied with the content of the course
.605
.000
17
The tests were fair assessments of my knowledge
.473
.000
18
I would take another distance course with this professor
.755
.000
19
I would take another distance course
.398
.000
20
The course workload was fair
.467
.000
21
The amount of interaction with the professor and other students was what I expected.
.710
.000
22
The course used groups to help with learning
23
I would like to have had more interaction with the professor.
26
The course content was valuable to me personally
28
Grading was fair
30
Often I felt “lost” in the distance class
31
.495
.000
-.508
.000
.439
.000
.735
.000
-.394
.000
The class instructions were explicit
.452
.000
33
Feedback from the instructor was timely
.592
.000
34
I received personalized feedback from the instructor
.499
.000
36
I would have learned more if I had taken this class oncampus (as opposed to online)
-.400
.000
37
This course made me think critically about the issues covered.
.423
.000
38
I think technology (email, web, discussion forums) was utilized effectively in this class
.559
.000
39
I felt that I could customize my learning more in the distance format.
.254
.001
42
The course content was valuable to me professionally
.442
.000
43
I missed the interaction of a “live,” traditional classroom
-.341
.002
46
Overall, the program is a good value (quality/cost)
.258
.017
Distance Education
RESLTS – WHAT MAKES A DIFFERENCE The results to one question, “Overall, I was satisfied with the course,” were used as the subject’s measure of general satisfaction. The data set was loaded into SPSS for analysis. Table 1 details twenty-two questions that proved significantly correlated to the reported satisfaction. These twenty-two questions identify characteristics to pay attention to in order to improve student satisfaction. With such a large number of variables impacting “satisfaction,” a more detailed analysis was needed. Kerlinger’s (1986) discussion of factor analysis as a research technique “to explore variable areas in order to identify the factors presumably underlying the variables (p. 590)” matches well with our need. A SPSS factor analysis was performed using Principal Component Analysis with a Varimax Extraction on those questions that had proven significantly correlated to satisfaction (Table 1). Using the values created from the factor analysis, each question was allocated to categorical factor. The components created from these questions clustered together around five categories that explain 66.9% of the variance in satisfaction. How well these questions fit into each category was determined and statistically, all reliability coefficients (Cronbach Alpha) are above .7000 and all eigenvalues are above 1.00 thus indicating an acceptable level for a viable factor (Kline, 1993; Nunnally, 1978). Bottom line is this; if we improve these characteristics then we improve satisfaction. In summary, twenty-two variables from the questionnaire proved significantly correlated to satisfaction. An exploratory factor analysis (Tucker and MacCallum, 1997) grouped those twenty-two variables into five constructs that we labeled: Interaction with the Professor; Fairness; Content of the Course; Classroom Interaction; and Learning Value. Each construct is summarized below.
Interaction with the Professor: The first component for study combines five questions: Q18, Q21, Q33, Q34. The ideas permeating these questions center on feedback and interaction with the professor. The results show that feedback that is timely and personalized helped raise the satisfaction ratings. Fairness: Q17, Q20, Q28 and Q31 create the second construct. These questions deal explicitly with the fairness of the tests, the course workload and grading, and the explicitness of the course instructions. All of these questions deal implicitly with setting the expectations for the course. The better the expectations are set, the higher the satisfaction. Course Content: The questions, Q16, Q26, Q39, Q42, center on course content. The questions look at content from m both a personal and professional level. As one would hope, “good” course content influences the basic satisfaction of that course. Classroom Interaction: Questions, Q23, Q30, Q36, Q43, all deal with interaction and all have negative loading values. This is interesting because the statements in these questions were testing many of the concerns expressed with the “commercialization” of distance education. This data concludes those inhibitors are not significant. A negative correlation means that the students that were more satisfied with their online experience: 1.) did not miss the interactions of a traditional classroom (Q43); 2.) did not want more interactions (Q30); 3.) did not think they would have learned more if they took the course in a traditional environment. on-campus; 4.) and, did not exhibit the feeling of being lost in the class. Learning Value: The last factor (Q19 Q22, Q37, Q38, Q46) looks at learning value. Questions ask the subject directly to rate the value of the course and whether or not the subject would
75
Distance Education
take another distance course. Other questions ask for ratings on “learning” for groups and thinking “critically.” Again, as one would anticipate, the more value a student perceives in the course the more likely they are satisfied.
MANAGING TO THE RESULTS As mentioned earlier, organizations and colleges use distance education courses not only to provide an alternative delivery method for students but also to generate revenues. As the number of distance courses and programs increase at an exponential rate, the necessity to enhance the program’s success both academically and financially also takes prominence. Competition among the various distance programs for students will grow. The results of this study can help suggest a set of operational recommendations that can impact online program success (Table 2). The results in this study indicate that timely and personalized feedback by professors results in a higher level of satisfaction by students. This suggests that distance education administrators should work closely with their faculty and offer
ways to enrich the teacher-student relationships. Paradoxically, a faculty member needs to use technologies to add a personal touch to the virtual classroom. For example, faculty should be encouraged to increase the usage of electronic discussion forums, responding to email within 24 to 48 hours, keeping students up-to-date with the latest details related to the course. Good course content and explicit instructions increase student satisfaction in the virtual classroom. So, it may well be that these characteristics set and manage the expectations for the distance student. This result suggests that faculty should have complete websites with syllabi and detailed instructions. In turn, distance education administrators should focus their attention on providing faculty with technical support such as good website design, instructional designer support, test design, user interaction techniques, etc. appropriate for distance learning. It seems that the students’ notions of the value they receive from a distance course combine learning and technology. Technology in this case not only refers to the actual software and hardware features of the delivery platform but also how well technology is adapted to the best practices
Table 2. Suggestions to help increase online program success
76
1
Have instructors use a 24-48 hour turnaround for email
2
Have instructors use a 1 week turnaround for graded assignments
3
Provide weekly “keeping in touch” communications
4
Provide clear expectation of workload
5
Provide explicit grading policies
6
Explicitly separate technical and pedagogical issues
7
Have policies in place that deal effectively with technical problems
8
Provide detailed unambiguous instructions for coursework submission
9
Provide faculty with instructional design support
10
Do not force student interaction without good pedagogical rationale
11
Do not force technological interaction without good pedagogical purpose
12
Collect regular student and faculty feedback for continuous improvement
Distance Education
of teaching. The negative correlation implies that if technology is available but not used, it lowers satisfaction. For the program administrator, this would suggest adoption of distance platforms that are customizable at the course level with respect to displaying technological options.
CONCLUSION The goal of this study was to identify potential indicators for satisfaction with distance education. Background literature surrounding the traditional versus virtual classroom debate helped develop a 49-question questionnaire. The questionnaire was administered to MBA students in a top-tier distance education program. A factor analysis extracted five basic constructs correlated to satisfaction: Professor Interaction, Fairness, Course Content, Classroom Interaction and Technology Use & Value. Using these categories, several recommendations for implementing and managing a distance program were provided. Since this study was first published, the authors have executed multiple distance courses and collected experiential data to confirm the potential for the suggestions. The communication and explicit instructions seem to have the desired impact. In fact, the instantiations of these – detailed syllabi, explicit assignment instructions, consistent (one week) turnaround on graded assignments all have been mentioned in student reviews for the courses.
REFERENC Allen, I. E., & Seaman, J. (2007). Online nation. Sloan Consortium and Babson Survey Research Group, AACSB (1999). Corporate universities emerge as pioneers in market-driven education,” Newsline, Fall 1999
Boyatzis, Richard E., & Kram, K. E. (1999). Reconstructing management education as lifelong learning,” Selections, 16(1),17-27. Carr, S. (2000). As distance education comes of age the challenge is keeping students. Chronicle of Higher Education, February 11. Creahan, T. A., & Hoge, B. (1998). Distance learning: Paradigm shift of pedagogical drift. Presentation at Fifth EDINEB Conference, September, 1998, Cleveland, Ohio. Gustafsson, A., Ekdahl, F., Falk, K., & Johnson, M. (2000). Linking customer satisfaction to product design: A key to success for volvo. Quality Management Journal, 7(1), 27-38. Haythornthwaite, C., Kazmer, M. M., Robins, J., & Showmaker, S. (2000). Community development among distance learners. Journal of ComputerMediated Communication, 6(1). Hearn, G., & Scott, D. (1998). Students staying home. Futures, 30(7), 731-737. Hogan, D., & Kwiatkowksi, R. (1998). Emotional aspects of large group teaching. Human Relations, 51(11), 1403-1417. Howell, S. L., Williams, P.B., Lindsay, N.K. (2003). Thirty-two trends affecting distance education: An informed foundation for strategic planning. Online Journal of Distance Learning Administration, 6(3). http://www.westga. edu/~distance/ojdla/fall63/howell63.html (last accessed January 26, 2008) Institute for Higher Education Policy. (2000). Quality on the line: Benchmarks for Success in Internet Distance Education. Washington, D.C. Kariya, S. (2003). Online education expands and evolves. IEEE Spectrum, 40(5), 49-51. Kerlinger, F. N. (1986). Foundations of behavioral research, 3rd. ed. Holt, Rinehart & Winston.
77
Distance Education
Kirkman, B.L., Rosen, B., Gibson, C.B., Etsluk, P.E., & McPherson, S. (2002). Five challenges to virtual team success: Lessons from Sabre, Inc. The Academy of Management Executive, 16(3). Kline, P. (1993). The handbook of psychological testing, London: Routledge. Lifelong Learning Trends: A Profile of Continuing Higher Education. 7th Edition. (2002, April) University Continuing Education Association. Martz, Wm. Benjamin, Jr., & Venkateshwar, Reddy. (2005). Five factors for operational success in distance education. Encyclopedia of Online Learning and Technology, Caroline Howard, (ed.) Martz, W. B, Reddy, V., & Sangermano, K. (2004). Assessing the impact of Internet testing: Lower perceived performance. Distance Learning and University Effectiveness: Changing Educational Paradigms for Online Learning, Caroline Howard, Karen Schenk and Richard Discenza (eds.). Hershey, PA: Idea-Group, Inc.: Martz, William B., & Shepherd, Morgan, M., (2207). Managing distance education for success. International Journal of Web-Based Learning and Teaching Technologies, 2(2), 50-59. Merit Education (2003). Six higher education mega trends what they mean for the distance learning. http://www.meriteducation.com/sixmega-trends-higher-education-continued.html (accessed January 29, 2005) NEA (2002). The promise and the reality of distance education. NEA Higher Education Research Center, 8(3). Noble, D. F. (1999). Digital diplomas mills. Retrieved November 28, 2002 , from http://www. firstmonday.dk/issues/issue3_1/noble/index. html
78
Nunnally, J. (1978). Psychometric theory. New York: McGraw-Hill Peters, T. J., & Waterman, Jr., R.H. (1982). In search of excellence.New York: Harper and Row. Poole, M.S., & DeSanctis, G. (1990). Understanding the use of group decision support systems: The theory of adaptive structuration. In Organizations and Commuincation Technology, J. Fulk and C. Steinfeld (eds.). 173-193. New Bury Park, Ca: Sage Publications. Russell, T. (2001). http://www.nosignificantdifference.org/ accessed on January 26, 2008. Sausne, Rebecca (2003). Thinking about going virtual? Better bone up on the for-profits to see what you’re up against. University Business, July http://universitybusiness.com/page.cfm?p=311 accessed January 29, 2005) Statsoft (2002). http://www.statsoftinc.com/ textbook/stfacan.html. Accessed on November 30, 2002 Svetcov, D. (2000). The virtual classroom vs. the real one. Forbes, 50-52. Traupel, L. (2004). Redefining distance to market your company. http://www.theallineed.com/adonline-business-3/online-business-028.htm, (accessed January 29, 2005) Tucker, L. F., & MacCallum, R. (1997). Unpublished manuscript available at http://quantrm2. psy.ohio-state.edu/maccallum/book/ch6.pdf Accessed on November 30, 2002 U.S. News and World Report (2001). Best online graduate programs, October.
79
Chapter VI
Group Support Systems as Collaborative Learning Technologies: A Meta-Analysis
John Lim National University of Singapore, Singapore Yin Ping Yang National University of Singapore, Singapore Yingqin Zhong National University of Singapore, Singapore
AbSTRACT Computer-based systems have been widely applied to support group-related activities such as collaborative learning and training. The various terms accorded to this research stream include virtual teams, e-collaboration, computer-supported collaborative work, distributed work, electronic meetings, and so forth. A notable and well-accepted aspect in the information system field is group support systems (GSS), the focus of this chpater. The numerous GSS studies have reported findings which may not be altogether consistent. An overall picture is much in want which attends to the synthesizing of the findings accumulated over decades. This chapter presents a meta-analysis study aimed at gaining a general understanding of GSS effects. We investigate 6 important moderators in GSS experimental research: group outcomes, namely group size, task type, anonymity, time and proximity, level of technology, and the existence of facilitation. The results point to important conclusions about the phenomenon of interest; in particular, their implications vis-à-vis computer-supported collaborative learning technologies and use are discussed and highlighted along each dimension of the studied variables. Copyright © 2009, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.
Group Support Systems as Collaborative Learning Technologies
INnion Team-based or group work and collaborations are an integral part of education and learning environments. With the advance of information communication technologies, there has been a growing potential for utilizing computerized systems to support idea generation, project assignment, and instant communication among the IT-age students and educators. The phenomenon has attracted interest from the fields of education as well as Information Systems (IS). In this connection, an emerging area in the instructional technology field called computer-supported collaborative learning (CSCL) has focused on the ways to support group learning using different forms of technologies; some examples are electronic discussion environments, distance learning systems, and intelligent agents (e.g., Koschmann, 1996; Strijbos et al., 2003; Ready et al., 2004; Bosco, 2007). Group Support Systems (GSS) research has accumulated a substantial body of knowledge on the effects of computer-based systems in supporting group work in relation to a variety of tasks such as idea generation and decision making. Based on the successful use of GSS technology to support groups in non-academic settings, researchers have begun to explore ways to apply GSS technology in classroom to support and enhance group-based learning (Tyran & Shepherd, 2001). GSS are used in a classroom setting or distance learning groups to support and structure group communication and learning activities (e.g., Leidner & Jarvenpaa, 1995; Sawyer et al., 2001; Alavi et al., 2002; Gill, 2006). While much work has been done in examining the impacts of GSS on group outcomes, the findings are not altogether consistent. Several early meta-analyses exist (e.g., Benbasat & Lim, 1993; McLeod, 1992; Shaw, 1998). Other reviews involve tabular methods which are unavoidably less rigorous (Fjermestad & Hiltz, 1999). Tyran and Shepherd (2001) presented a GSS
80
research framework for analyzing the impact of collaborative technology on group learning, by referring to an earlier framework concerning electronic meeting systems on group processes and outcomes (Pinsonneault & Kraemer, 1990). Nevertheless, as the framework is built based on face-to-face or “same time, same place” research studies (Leidner & Jarvenpaa, 1995), it is somewhat limited in its applicability to group work or learning in other forms such as distributed work or web-based distance learning. Dennis and Wixom (2002) examined five moderators (task, GSS tools, type of group, group size, and facilitation) and their potential effects on GSS use. The necessityhas been noted to look deeper than and beyond “the overall effects of GSS use” (p. 236, Dennis and Wixom, 2002). A pertinent question is under what conditions collaborative technology use would improve group performance because there are moderators that influence the specific effects of GSS (Dennis & Wixom, 2002; Beauclair, 1989). The current study looks into how key moderators individually and jointly influence important group work outcomes using a meta-analytic technique to help derive meaningful conclusions backed by quantitative analysis, as well as provide insights useful for both CSCL and GSS areas. Specifically, our primary interest concerns the use of GSS technology and research in the learning environment. Correspondingly, the paper focuses on six important moderators which are pertinent to both organizational and educational contexts; they are group size, task type, anonymity, time and proximity, level of technology, and the existence of facilitation. A research model is constructed and hypotheses are developed concerning the impacts of GSS. Next, we present a meta-analysis on thirty-three quantitative experimental studies to gain a synthesized view of the GSS effectiveness. The subsequent sections dwell on the results and discussions relating to each of the outcome variables. We conclude the paper by pointing out the relevance to, and implications for, computer-
Group Support Systems as Collaborative Learning Technologies
supported collaborative learning research. As well, future research avenues are identified.
Theear Framework Learning and GSS Technology Collaborative learning, as compared to individual learning, is helpful and important for understanding and exploring the process of learning. Embedded within the definition of cooperative learning is an enormous diversity of cooperative approaches. These may be informal as short meetings to simply discuss and share information, or formal approaches where structure is imposed with specific ways of forming teams. Students may be working together on projects or other creative activities involving specific contents. Growing interests in supporting the needs of collaborative learning, along with concurrent improvements in GSS, have led to the emergence of a research area in the instructional technology field in CSCL; research in CSCL centers on the interaction of computer-supported learning systems and GSS by integrating collaborative learning and Information Technology (IT) (O’Malley, 1995). The rising importance of teamwork and group decision-making has triggered vast research efforts since the 1960s when the GSS concept was first raised by Doug Engelbart (Wagner et al., 1993). GSS are defined as computer-based systems that support group decision making which is the result of the integration of communication, information processing and computer-based group structuring and support (DeSanctis & Gallupe, 1987; Huber, 1984; Poole & DeSanctis, 1990). The CSCL research domain encompasses benefits derived from GSS applications to support group-oriented methods of instruction, including networked discussion environments and distance learning systems. Desktop conferencing, videoconferencing, co-authoring features and applications, electronic
mail and bulletin boards, meeting support systems, voice applications, workflow systems, and group calendars are key examples of groupware (Grudin, 1991). In addition to a set of common features, different applications provide users with different tools and functions. Many IS researchers have used GSS in the classroom and experiments to enhance the learning experience (Kwok et al., 2002), whereas other works in IS and related fields have developed asynchronous learning networks (ALNs) (Coppola et al., 2002). These systems enable affective learning objectives related to interactive communication and teamwork to be achieved, on top of meeting the traditional cognitive learning objectives. GSS were originally designed to support discussion and decision making in the commercial/business sector, but in the last few years there has been a surge of interest in their usage to support collaborative learning (Khalifa & Kwok, 1999; Vogel et al., 2001). DeSanctis and Gallupe’s (1987) two-by-two framework for GSS has been applied to understanding IT usage in learning environments. Although this framework enables us to classify learning settings based on the dimensions of space and time, it does little to improve our understating of the technologies required to support the learning objectives in different settings. Sharda et al. (2004) propose extending DeSanctis and Gallupe’s framework by adding a third dimension, learning objectives achieved (cognitive and a