MMIC ICHA EL LLEN ’S’S HA ELAA LLEN e eL LE EA A RNING NN UA LL RNINGAA NN UA
20 0 88 200
About Pfeiffer Pfeiffer serves the professional development and hands-on resource needs of training and human resource practitioners and gives them products to do their jobs better. We deliver proven ideas and solutions from experts in HR development and HR management, and we offer effective and customizable tools to improve workplace performance. From novice to seasoned professional, Pfeiffer is the source you can trust to make yourself and your organization more successful.
Essential Knowledge Pfeiffer produces insightful, practical, and comprehensive materials on topics that matter the most to training and HR professionals. Our Essential Knowledge resources translate the expertise of seasoned professionals into practical, how-to guidance on critical workplace issues and problems. These resources are supported by case studies, worksheets, and job aids and are frequently supplemented with CD-ROMs, websites, and other means of making the content easier to read, understand, and use.
Essential Tools Pfeiffer’s Essential Tools resources save time and expense by offering proven, ready-to-use materials—including exercises, activities, games, instruments, and assessments—for use during a training or team-learning event. These resources are frequently offered in looseleaf or CD-ROM format to facilitate copying and customization of the material. Pfeiffer also recognizes the remarkable power of new technologies in expanding the reach and effectiveness of training. While e-hype has often created whizbang solutions in search of a problem, we are dedicated to bringing convenience and enhancements to proven training solutions. All our e-tools comply with rigorous functionality standards. The most appropriate technology wrapped around essential content yields the perfect solution for today’s on-the-go trainers and human resource professionals.
Essential resources for training and HR professionals w w w. p f e i f f e r. c o m
A bout T hi s B o ok Why is this topic important? e-Learning is here to stay, for better or worse. Let’s hope it’s for better. The founders of e-learning technology saw a means of improving the quality of individual learning experiences that no other medium offered. They worked to blend what was known about human learning and pedagogy with emerging computer and communication technologies and to study the outcomes to continuously improve effectiveness. Unfortunately, much of today’s elearning fails to build on the vision, the knowledge, and the most important achievements in learning technology. It therefore provides much less learning and performance impact than it could and should, costing organizations immensely and wasting precious learning time. What can you achieve with this book? You can improve the quality of e-learning you buy, create, and use. Contributors to this volume openly and explicitly state the lessons they have learned through decades of first-hand exploratory work. The advice and direction so bluntly shared is applicable whether you work in training or education, in corporations, non-profit organizations, military, or schools. How is this book organized? Each author was asked three questions: What intrigued you about elearning to the point of focusing so much of your career on it? After some decades of working with learning technology, what have you learned (perhaps in contrast to what’s become the general perception)? Looking to the future, what are your current visions, hopes, and expectations? Each article wrestles with these questions in a unique way, but with varying degrees of emphasis. They are organized based somewhat on this emphasis and somewhat on chronology, beginning with authors who were responsible for creating landmark systems and beginning the discussion, moving to lessons learned, and ending with those who focus on critical questions of the day.
Th e P f eif f er A n nua l Seri es The Pfeiffer Annuals present each year never-before-published materials contributed by learning professionals and academics and written for trainers, consultants, and human resource and performance-improvement practitioners. As a forum for the sharing of ideas, theories, models, instruments, experiential learning activities, and best and innovative practices, the Annuals are unique. Not least because only in the Pfeiffer Annuals will you find solutions from professionals like you who work in the field as trainers, consultants, facilitators, educators, and human resource and performance-improvement practitioners and whose contributions have been tried and perfected in real-life settings with actual participants and clients to meet real-world needs. The Pfeiffer Annual: Consulting Edited by Elaine Biech The Pfeiffer Annual: Leadership Development Edited by James Noel and David Dotlich The Pfeiffer Annual: Management Development Edited by Robert C. Preziosi The Pfeiffer Annual: Training Edited by Elaine Biech Michael Allen’s 2008 e-Learning Annual Edited by Michael Allen
Call for Papers In subsequent volumes of Michael Allen’s e-Learning Annual, we will present the most current explorations, thinking, insights, case studies, challenges, guidelines, and other helpful aids for the e-learning professional. We need your requests and your contributions. e-Learning can achieve its full potential to improve the world one learner at a time only through the sharing of best practices and through a continual questioning of ourselves and our work. I’m interested in providing a broad spectrum of articles and other contributions. If you wish to contribute or just have a need an author might address, please contact me at
[email protected] or call me at 651-203-3700.
MMIC ICHA EL LLEN ’S’S HA ELAA LLEN e eL LE EA A RNING NN UA LL RNINGAA NN UA
20 0 88 200
Copyright © 2008 by John Wiley & Sons, Inc. All Rights Reserved. Published by Pfeiffer An Imprint of Wiley 989 Market Street, San Francisco, CA 94103-1741 www.pfeiffer.com Except as specificially noted below, no part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-646-8600, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, 201-748-6011, fax 201-748-6008, or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Readers should be aware that Internet websites offered as citations and/or sources for further information may have changed or disappeared between the time this was written and when it is read. Certain pages from this book are designed for use in a group setting and may be customized and reproduced for educational/training purposes. The reproducible pages are designated by the appearance of the following copyright notice at the foot of each page: Michael Allen’s 2008 e-Learning Annual. Copyright © 2008 by John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com This notice may not be changed or deleted and it must appear on all reproductions as printed. This free permission is restricted to limited customization of the CD-ROM materials for your organization and the paper reproduction of the materials for educational/training events. It does not allow for systematic or large-scale reproduction, distribution (more than 100 copies per page, per year), transmission, electronic reproduction or inclusion in any publications offered for sale or used for commercial purposes—none of which may be done without prior written permission of the Publisher. For additional copies/bulk purchases of this book in the U.S. please contact 800-274-4434. Pfeiffer books and products are available through most bookstores. To contact Pfeiffer directly call our Customer Care Department within the U.S. at 800-274-4434, outside the U.S. at 317-572-3985, fax 317-572-4002, or visit www.pfeiffer.com. Pfeiffer also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. ISBN: 978-0-7879-8743-5 ISSN: 1046-333-X Acquiring Editor: Martin Delahoussaye Director of Development: Kathleen Dolan Davies Development Editor: Susan Rahmeler Production Editor: Dawn Kilgore Editor: Rebecca Taff Editorial Assistant: Julie Rodriquez Manufacturing Supervisor: Becky Morgan Printed in the United States of America Printing 10~NS9~NS8~NS7~NS6~NS5~NS4~NS3~NS2~NS1
Contents Preface
ix
Section 1. Visionary Pioneers Continue to Guide Us Reflections on TICCIT, C. Victor Bunderson
1
Addressing Diversity in (e-)Learning, Patrick Suppes in conversation with Michael W. Allen
31
PLATO, Plasma Screens, and Computer-Based Education, Donald L. Bitzer in conversation with Michael W. Allen
43
Section 2. Learning Technology Takes Shape Back to the Future of Educational Technology, Fred T. Hofstetter
59
It Seemed Like a Good Idea at the Time: The Development of Computer-Assisted Instruction in Schools, Peter G. Fairweather
83
Reflections on e-Learning: The Long History of a New Idea, Wallace Hannum
101
Lessons from the STARS Revisited, Thomas C. Reeves and William D. Aggen
127
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction, M. David Merrill
141
Section 3. What Does It All Mean? Pitfalls and Overhauls Dreams and Responsibilities: Learning and the New Technologies, Tom Boyle
171
Beyond Knowledge Dissemination: Learning to Solve Problems Online, David H. Jonassen
189
What’s Right and What’s Wrong with Online Learning, Greg Kearsley
207
Technology and Learning: A Proposal for Adaptive Tutorial Learning, Alfred Bork, Carole A. Bagley, and Xiwen Zhang
227
Cheap But Not Tacky: Lessons Learned During e-Learning Adventures, Sivasailam “Thiagi” Thiagarajan
249
Section 4. Are We Lost? Where Do We Go from Here? Moving e-Learning into the Workplace, Allison Rossett
265
The Business Impact of e-Learning, Nick H.M. van Dam
285
Lessons Learned . . . and Relearned, Allen Avner
307
We’re Professionals, Aren’t We? What Drives Our Performance? Will Thalheimer
325
A Final Word
339
Pfeiffer Publications Guide
345
Preface Computers needed for e-learning
useful interactive applications.
cost over $20,000 each and need a
Even with sufficient experience,
much more expensive remote support
development of interactions takes
computer as well. It costs about $1,000
considerable time and effort, perhaps
per month to connect each computer
as much as a week to develop a single
to a network. Only more expensive
interaction with feedback.
computers can display any graphics
With all these costs, e-learning
at all, and when they do, they’re not
is obviously much more expensive
very refined, and it takes extra time
than other forms of instruction,
for the graphics to plot on the screen.
yet everyone is excited about it.
The display is monochromatic or
Organizations are finding the money
can display up to eight colors (but
and time to develop courses, even
only four at a time). These expensive
when they sometimes take a year and
computers aren’t always fast, either.
a half to design and develop. They
Sometimes learners must wait up
are finding the money needed for the
to a minute for their answers to be
computers and networking services.
accepted. There’s no animation or
Serious evaluations are scheduled,
video. Sound effects are limited to
and everyone expects revisions to be
beeps.
needed. Organizations sometimes
Development of display graphics
expect preliminary evaluations to
is a very slow and arduous process,
take several months before even more
requiring specification of vector
serious evaluations are undertaken.
coordinates for each line or, more
Because of the high costs, it’s
simply but crudely, translating
important that courses be well
graphics into character graphics
designed and effective. Most
( _, ---, /, \, |, etc.). The development
organizations put their most skilled
language is difficult to learn and
instructors available on projects as
requires about two years of experience
well as their strongest subject-matter
before skills are sufficient to develop
experts. It’s not uncommon for experts
x
Michael Allenʼs 2008 e-Learning Annual
in human learning research to join
started their work. Some went about
the design team. Everyone knows
their explorations and inventions in
that the technology provides new
a carefully studied way. Others were
opportunities, and every aspect of the
pragmatic, while others were simply
instruction will need to be thought
playful. But regardless of the approach,
through carefully to avoid adherence
access to systems was expensive.
to inappropriate or overly constrictive
Those granted access were intent
paradigms. Designs are carefully
about doing something of significance.
considered before development begins,
And because each application was
not only because wasting resources
a major undertaking, it received
would have bad consequences, but
attention and required justification.
also because making changes later is
In these early days, there was
difficult and adds to the already high
perhaps a stronger appreciation than
expenses.
today of the value of better learning
Learner responses are being
experiences in both educational and
logged, and designers analyze these
training venues. Even if costs went
responses in order to improve
up significantly, people, schools,
instructions, questions, and feedback
corporations, the military, and even
and otherwise revise designs. Courses
public funding institutions felt the
are continuously if not systematically
enhanced learning opportunities
improved. While not all courses
would be worth it. Of course, everyone
are successful, even after all this
knew costs would come down if
painstaking work, some demonstrate
systems were successful and adopted
dramatic success and point the
in large numbers.
way to a new era of personalized,
Leaders were inventive. As you’ll
adaptive learning. Others advance our
note from reading the accounts
knowledge of what is ineffective.
provided in this book by some of
If you’re imaging this, then you’re
the most influential leaders of early
envisioning what the e-learning
e-learning efforts, in the face of
arena was like when I entered the
new possibilities, leaders did not
field in the late 1960s (costs roughly
dogmatically follow established
adjusted to today’s dollars). Although
principles of instruction. They did
there are some variants, many of the
not apply practices derived from the
contributors to this first
challenges of classroom instruction
e-Learning Annual encountered
because, obviously, the venue was
similar circumstances when they first
different. But neither did they follow
Preface
xi
what were the established principles of
do add something to the experience
instructional design. The era was new,
on occasion, they do not enhance
and everything was and needed to be
the underlying pedagogy or sustain
questioned.
engagement. Some of the inventiveness is just
Too Much and Too Little Invention
Today, we
too much, primarily because it misses
see too little
the point. An example is the incessant
inventiveness in
drive to create tools that make the
most e-learning.
development of learning experiences
We commonly
quick and easy. While it would clearly
see a regression to simplistic models
be nice if unskilled people could
of instruction that haven’t been very
develop learning experiences in a
engaging anywhere. The best example
blink of an eye—and today’s tools
is a tendency to present a lot of
do make much of the development
information followed by a quiz or test
work far faster and easier than it was
to see whether learners can recognize
when we had only basic programming
correct answers among multiple-
languages—the much more important
choice alternatives. This “tell-and-test”
challenge continues to be how to
model is easy to develop and falls in
make learning experiences effective.
the sweet spot of many so-called rapid
The most unaffordable instruction is
authoring tools. But as convenient
instruction that doesn’t work. Add
and inexpensive as it is to develop,
up all the time learners waste when
this instructional approach is no
they fail to develop needed skills
more attractive or effective delivered
and the opportunities lost from their
via e-learning than it has been in
inadequate performance, and you
classrooms.
almost always have a budget that
Some look to enhance the “tell-and-
far exceeds what anyone spends on
test” approach through technology.
learning interventions, including
Presentations can now have “voice-
development.
over” narration or even contain video.
From my perspective, and I think
Some try to make multiple-choice
the perspective of every contributor
questions more attractive through
to this publication, development costs
animation and novel interaction. Some
are not really the issue; achieving
feel that putting the presentations on
effectiveness is. Indeed, most
a mobile device will enhance learning.
organizations can’t afford their current
Unfortunately, while these attempts
development costs, whatever they are,
xii
Michael Allenʼs 2008 e-Learning Annual
because they are not achieving the
This series of Annuals is one
necessary impact. On the other hand,
attempt to rectify this sad situation of
they could spend many multiples of
today’s hapless e-learning and remind
their current development budgets if
us that, while too rare, some successes
only they were creating the needed
with e-learning have been spectacular
behavioral changes.
and have demonstrated the potential
The expert contributors to this
that lies in wait for all.
volume succeeded in part by breaking rules, by darting out in new directions, by continually looking at their
The e-Learning Annuals
I’m pleased that Pfeiffer
challenges from fresh perspectives.
has elected
Just breaking rules (which novices
to publish an
do without realizing it) isn’t truly
Annual for the
inventiveness or the leadership we
field of e-learning. Growth in the
need. It’s just random variation and
field continues at an impressive rate,
flailing about, which only serves
and few analysts, if any, anticipate a
at best to distract evaluation on
slowdown in the foreseeable future.
parameters that matter. Indeed, we
Many people are employed in some
have too much “rule breaking” going
aspect of e-learning production
on because the majority of e-learning
and delivery, and far more will use
designers and developers are simply
e-learning in schools, as part of
unaware of what has gone before them.
employment training, or as a personal
In contrast but equally ineffective,
elective. With the rapid growth, fueled
many of the field’s new designers
almost completely by the economic
dogmatically follow examples of the
advantages of low-cost delivery,
e-learning applications they’ve seen.
it’s no longer a select few who have
They mimic pedantic applications that
access to e-learning systems and
have become so prevalent without
the opportunity design and develop
realizing there are many alternatives
instructional applications. Everyone
and better options. They need to
has the opportunity, regardless of
think simply about what’s truly
preparation.
needed before they launch into the
On the downside of today’s position,
development of yet another painfully
a painfully small percentage of people
boring learning application.
involved in e-learning have come into
Preface
xiii
the field well prepared. Others struggle
theory. We’ll include case studies,
to use their opportunities well, not
editorials, and a well-rounded view of
having learned the lessons taught so
learning technology as it evolves.
unforgettably to our pioneers by their experiments. Lacking direction, they
A Special Issue
As the first
mimic typical classroom events or the
issue in the
few e-learning applications they’ve
series, it seems
encountered. They apply basic notions
appropriate
of content presentation, operant
to me that
conditioning, TV games (Jeopardy),
we should take a bit of an historical
and post-tests. They may or may not
perspective. The field is young enough
write an instructional objective, but
that some of the pioneers who shaped
if they do, they do so mechanically,
thinking and created groundbreaking
often oblivious of the different types
systems are still available to directly
of objectives needed for different
share their experiences and learning
purposes, sanguine about the benefits
with us. We reached out to them
that listed objectives provide learners,
and found many gracious enough to
and naïve about the situations in
recount major events in their careers
which objectives may actually diminish
and in the formation of learning
learning. This is only one, perhaps
technology.
trivial, example of the areas of ill
I wanted to create, and hope we
preparedness we find so prevalent
have succeeded in creating, a valuable
today.
and interesting book that (1) captures
My hope for these Annuals is that
the excitement and expectations
they will provide truly helpful and
of those people who witnessed the
refreshingly up-to-date information
initial availability of computation
for e-learning designers, developers,
and communication technology for
buyers, and users. Each year we will
instructional uses, experimented
attempt to assemble some of the best
with it, and shaped its beginnings, (2)
insights about teaching and learning
summarizes the major lessons these
with technology as well as thought-
pioneers learned from this work, and
provoking questions and insights.
(3) provides direction for both today’s
We’ll look for a balance between
and future work.
applicable guidance and evolving
xiv
Michael Allenʼs 2008 e-Learning Annual
I asked contributors to recall their
questions, but as you enjoy reading
early visions, hopes, and expectations
David Merrill’s contribution, you’ll
of what computing technologies
encounter the story of his listening to
brought to instruction and learning.
a presentation by B.F. Skinner. I won’t
ÿ What were the new possibilities
steal the thunder of David’s account
and why would they be valuable?
here, but as he learned, thought
ÿ What were the risks and how
leaders of merit change their beliefs
would they be mitigated?
as they continue to explore ways of
ÿ What would learning technology mean for the individual learner? ÿ What did instructional
understanding. Open-mindedness is essential, both in scientific endeavors and creative ones. Regardless of one’s
technologies mean for schools,
perspective of whether instructional
universities, corporations, the
design is an art or a science, as the
military, and government?
leaders convey in their personal odysseys, important ideas come not
For example, did we expect a new
only from focused thought but also
competitive economy of learning
from casual conversations, being in the
opportunities to emerge, leading to
right place at the right time, and being
ever more refined designs and an
receptive to new ideas—regardless of
increased understanding of human
their origins.
learning? (I did expect this, but it doesn’t seem to have happened—yet.) I asked about lessons learned after
Learning what leaders were doing when they derived their principles and perhaps when they revised or
their decades of working with learning
supplanted them gives us a depth of
technology (perhaps in contrast to
understanding we can’t gain through
what’s become the general perception).
most publications. It also encourages
And I asked about future directions.
us to participate in the exploration
Redirecting their view to the future,
of learning technology, knowing that
but cognizant of experience and
powerful insights often come just from
events, what are their current visions,
being active in the field and being
hopes, and expectations?
willing to make mistakes.
Some contributors were a bit
Too often students perceive
shy or modest about sharing their
principles learned in classes and
wisdom within the context of these
from books to have an unassailable
Preface
xv
truth that their authors would never
An Invitation In years subsequent
claim. In e-learning, we often find
to this inaugural and special edition,
student designers fervently defending
we will be bringing our readers
dreadful designs while quite accurately
state-of-the-art work and thinking
quoting principles they’ve learned and
in the field of e-learning. In my
claiming, sometimes quite faithfully,
nearly forty years now of working
adherence to frequently taught
in this field, I continue to stumble
principles. I doubt any researcher
upon amazing accomplishments and
or author would adhere to his or her
powerful insights that have gone
own principle with such veracity,
largely unrecognized and unknown.
especially in the face of the appalling
This series of Annuals represents an
designs they justify. Instead, they
opportunity to unearth those building
would use the disappointing situation
blocks to a better future. If you know
as an opportunity to reexamine the
of work that deserves such exposure,
principle. The accounts contained
whether it’s your own or another’s,
herein, in fact, demonstrate that our
please contact me directly at Annual@
thought leaders have done exactly that
alleni.com or through the Pfeiffer
throughout their careers.
Editorial Staff at 989 Market Street, 5th Floor, San Francisco, CA 941031741 (www.pfeiffer.com).
Section One
Visionary Pioneers Continue to Guide Us
C. Victor Bunderson C. Victor Bunderson’s work began in academic settings at the University of Texas and Brigham Young University (BYU), where he led an early lab and became principal investigator on the $5 million NSF-financed TICCIT computer-aided instruction project. He has been a co-founder of several companies involving advanced computer and information technologies and education and training applications. He also served as vice president for research management at the non-profit Educational Testing Service in Princeton, New Jersey, where he guided research projects designed to develop and investigate a new generation of tests more relevant to learning. He returned to academe in 1991—at BYU—to work with a cadre of talented Ph.D. students in the Department of Instructional Science (later, Instructional Psychology and Technology). His work spans many decades and many types of organizations and has led him to validity-centered design, an interdisciplinary synthesis of measurement science, the learning sciences, and change management. This work has also built the foundations for learning progress systems based on domain theories of learning and growth. He is currently leading an ambitious validation update project for Herrmann International, updating and revising the Herrmann Brain Dominance Instrument for the new millennium. This is an individual differences instrument widely used in many countries and twenty language translations.
Reflections on TICCIT
C. Victor Bunderson
This article presents the early experiences of the author and his colleagues on the courseware development team for the TICCIT computer-aided instruction system. It reviews the history of TICCIT using words of that time, and then assays to explain lessons learned during and since TICCIT times in three main areas: (1) learner control—the high goals and ideals held out for this concept and their difficulty of attainment; (2) progress maps culminating in mastery assessments—use of progress maps with navigation based on well-measured progress information; and (3) differentiated roles and technology integration (how design experiments using differentiated staffing models can serve as a replacement for premature evaluations and can lead to attainment of some of the hopedfor high expectations for TICCIT and other innovations in what CAI has become). These three areas were selected to capture some of the essential issues that drove the early TICCIT project and that remain problematic today. Some History
In the October
lot further. TICCIT and PLATO were
1971 the MITRE
the two large projects in a major $10
Corporation,
million initiative of the National Science
partnering with
Foundation’s Technology Innovations
the University
Group, under the leadership of Arthur
of Texas at Austin (UT Austin) CAI
Melmed. It was an NSF effort to jump-start
lab and Brigham Young University
CAI, after some promising beginnings at
(BYU), submitted a proposal for just
the Illinois Plato Lab and the UT Austin
under $5,000,000. TICCIT (Time-
CAI Lab, which had both received prior
Shared, Interactive, Computer-
NSF funding. With other funding these labs
Controlled Information Television)
had attracted, they were both positioned
was based on innovative concepts in
to undertake these serious efforts. Now,
hardware, software, courseware, and
thirty-six years after that first TICCIT
implementation and was designed to
proposal was funded, it is instructive to
produce a “Market Success for CAI.”
look back at those days and consider what
It is interesting to look back to those
was accomplished. Also, it is sobering to
heady times when this proposal was
remember what dreams and ideals remain,
funded, times when $5,000,000 went a
elusive, apparently just beyond our grasp,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
2
Michael Allenʼs 2008 e-Learning Annual
dancing away to the next decade and then
In that 1971 proposal, some interesting
the next, calling on another generation
claims were made, in several key areas,
to try again. One must have a certain wry
the system, the courseware, and the
humor to view the overconfidence, even
market. The bullet points that follow are
hubris, of those days, and to temper the
selected from a longer list in the proposal’s
accomplishments with the recognition of
executive summary, and are quoted
problems that remain unsolved.
exactly except for one or two minor
One way to summarize TICCIT
paraphrases (MITRE, 1971, p. i):
is to say that it was a synthesis of the contributions of three main
The System
groups—the systems engineering
ÿ Mass dissemination of CAI has
and project management expertise of
been inhibited by the high cost
MITRE Corporation, the CAI learner-
of student terminals, computer
controlled courseware technology
hardware, and courseware.
and team production know-how of
ÿ TICCIT has solved the hardware
Victor Bunderson and his colleagues
cost problem through the
at the University of Texas CAI lab,
imaginative use of commercial
and the instructional theories and
television technology and
design procedures of David Merrill and
low-cost/high performance
colleagues at BYU. The hardware itself,
minicomputers.
as well as the software and courseware,
ÿ The TICCIT terminal (at one-
embodied what became known as
tenth the cost of the nearest
Merrill’s Component Display Theory
equivalent CAI terminal) is able to
of Instruction. MITRE’s disciplines
deliver instructional movies, plus
for engineering and management of
interactive computer-delivered
innovative system solutions permitted
graphics, voice, and text.
it to use these instructional concepts,
ÿ Terminal costs will be very low,
rather than dictate system constraints to a
ranging from a low of 10 cents in
system designed for instruction should be
future home CATV applications to
built. Also, blended in was the courseware
perhaps $1.00 per hour for a profit-
team’s insistence on empiricism mixed
making, turn-keyed system leased
with theory and with a commitment to the
to a school system, including
individual learner in creating the interface
software and commercial cost of
between the learner and the system. The
loans and capital.
centerpiece was giving the learner control of that interface. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
3
The Courseware ÿ The equivalent of five full semester
ÿ Almost 20 percent of all junior college class contact hours are in
courses will be developed in
curricular areas chosen for this
English and in math.
program.
ÿ Learner control over the instructional strategy is a basic
Elsewhere in the proposal, we learn
tenet of the University of Texas’
that The MITRE Corporation had
instructional design philosophy,
invested $750,000 of its internal R&D
as a means to humanize education
funds in this program before the NSF
and minimize the inadequacies of
funding was obtained. In addition,
machinery.
the proposals detailed extensive plans
ÿ The University of Texas’ and
for coordinating with manufacturers,
Brigham Young University’s
publishers, school administrators, and
instructional design concepts
others to assure that the project would
replace courses of conventional
recruit additional investment and
instruction with completely
interest of businesses and customers
modular systems, greatly
alike in actually achieving the “Market
redefining the teacher’s role.
Success” after the NSF funding period
Dramatic increases in effectiveness
was complete. The two community
are sought through the use of a
colleges selected, Northern Virginia
systematic design approach.
Community College near MITRE and Phoenix College in Arizona, were the
The Market
sites where the evaluation occurred,
ÿ The TICCIT curriculum should
and where many lessons were learning
have a major, measurable, and nationally meaningful impact on two junior college student bodies. ÿ A new junior college is being
in subsequent years. MITRE Corporation had been developing a number of technological innovations in TICCIT in hardware
formed in the United States
and software for digital audio and
every two weeks, on average,
graphics. MITRE started with
and approximately 50 percent of
the name TICCET (Time Shared
first-time enrollees in U.S. public
Interactive Computer-Controlled
collegiate institutions in 1969 were
Educational Television) and very soon
in public junior colleges.
thereafter changed it to TICCIT. They changed “Educational Television” to “Information Television,” thus
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
4
Michael Allenʼs 2008 e-Learning Annual
broadening the scope of application of
of the following considerations. Based
the technology to include home-based
on the work of UT doctoral student
information interactions over cable.
Dan McLallen (1971) MITRE and the
Their solution, integrating as it did
UT lab jointly choose community
video, color slide projection, and print,
colleges as the implementation site
was quite advanced for its time. Now,
and the subjects of freshman English
happily (even miraculously, for many
and freshman mathematics as the
writers in this book who pushed the
target courseware projects. McLallen’s
envelope of the ancestors of today’s
dissertation showed a high penetration
technologies at great human and
of credit hours that these two subjects
financial cost), the cost concerns dealt
would encompass out of the total
with in this proposal for hardware and
of most community colleges. The
software have been revolutionized by
penetration would be much greater,
progress in the information technology
and the project would be challenged
industry. The several racks of cards
with more remedial students by
and wiring for digital audio, graphics,
extending both the math and English
and the videotape players used in the
courses downward to encompass more
TICCIT system have been replaced by
building-block lessons from high-
a few chips, with a DVD drive if you
school and remedial levels. When the
wish.
amount of courseware to be developed
The original hardware featured two
was dramatically increased, Brigham
NOVA 800 minicomputers, a terminal
Young University (BYU) was selected
processor, and a main processor. The
to develop these additional lessons.
terminal processor was designed to
As head of the courseware
handle communications with 128 Sony
development and authoring system
color TV sets adapted as terminals.
team, I evaluated the resources needed
TICCIT systems were rolled out
to pull off such a large courseware
over a couple of decades following
development project and found that
the end of NSF funding by a series of
my resources at Texas would be taxed.
corporations, first by Hazletine, then
I looked to a respected colleague,
Ford Aerospace, then Loral. It had a
Dr. M. David Merrill, with whom
relatively long life, more in military
I had developed a strong working
and industrial applications than in its
relationship during his time at the
intended target, schools.
George Peabody School at Nashville.
The courseware team was consolidated at BYU in 1972 as a result
One of my doctoral students at UT Austin, Paul Merrill, was studying a
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
5
computerized version of Dave Merrill’s
impact on the design and features of
hierarchical “imaginary science”
TICCIT. Merrill’s component display
lessons for his dissertation at the
theory of instruction was at the heart
time. When Dave Merrill moved from
of the interaction design, and he
Peabody to BYU, he was alive with
practically single-handedly worked
excitement about the unusual and
out the details of the advisor program
coordinated resources at BYU for the
we had determined was essential
design and development of innovative
in a learner-controlled system with
learning technologies. Darrell Monson
navigational maps. Combining forces
had risen to a high administrative
helped reduce costs substantially for
position at that campus and had under
travel and for duplicate equipment.
his direction an outstanding movie
Also, it reduced maintenance and
studio, learning labs, and radio and
management problems of a split team
TV station. With Merrill, Monson
(it was hard enough to manage it at
was using his influence in setting up
one location!). These were some of
a graduate program in technology
the reasons the consolidation was
in education—we later named it
favored by MITRE and NSF. I brought
Instructional Science. I felt that the
key graduate students and courseware
opportunity to apply these resources
experts from UT, including colleagues
to the TICCIT project through a team
Dr. Gerald Faust and C.J. Ross, math
united in one location would greatly
professor Dr. Ted Boessenroth,
enhance its likelihood of success.
systems programmer Steve Fine and
Instead of subcontracting the
his able wife Claire, Tom McMurchey,
remedial portions to Merrill’s team
and graduate student Fred O’Neal and
under Darrell Monson at BYU, with
his capable wife Harriet. Like Claire,
counsel from MITRE and NSF, and
Harriet became a manager within the
with understanding support from
Courseware Production Factory. Dave
Wayne Holtzman at Texas (who was
Merrill’s team included Drs. Harvey
the real founder of the Texas Lab and
Black, Ed Green, talented graduate
my mentor), I decided to move the
students, including Andy Gibbons and
project to BYU. Up to that time, only
Roland Blake, and many others. These
a planning grant had been given to the
and other people who joined the team
UT CAI Lab. This move was of great
later comprised the Courseware team,
importance, as Dave Merrill and his
which was consolidated at BYU in the
colleagues at BYU had an enormous
summer of 1972. Those joining the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
6
Michael Allenʼs 2008 e-Learning Annual
team after the initial consolidation are
WICAT company. This company later
not listed, but their contributions are
split due to business reasons, but
not forgotten.
showed that business is happy to use
This team worked out the details of
the best of either system. Another
learner-controlled courseware. Steve
instance of this: Bunderson served for
Fine and Tom McMurchey provided
a time as vice president of research
the main technical interface with
and psychometrics for TRO Learning,
the MITRE Corp engineers. With
the company that bought PLATO.
other members of the courseware
Heuston’s Waterford school and
team, they later joined Hazletine
Institute is today a strong contributor
to see the TICCIT system through
to computers in early education.
its generations, which included the
Bunderson also founded what today
migration of TICCIT to PC’s—MICRO
is called Alpine Testing Solutions
TICCIT.
and The EduMetrics Institute. Dave
In reference to such market
Merrill was instrumental in several
successes the TICCIT system
start-up companies, and his consulting
did attain, and the many spin-off
to large training companies, such
companies influenced by TICCT
as NETg, has taken ideas developed
alumni, it can be argued that team
during TICCIT times out in many
production of componentized,
directions. Other TICCIT alumni
model-based courseware built on
have been involved in many e-
the separation of content from logic
learning ventures. Of particular note
has perhaps led to more “market
is Dr. Robert Mendenhall’s career.
successes” in online interactive
He started as an undergraduate pre-
learning projects and in many
law student who helped Bunderson
companies than any particular system
manage the TICCIT lab during and
on which it was embodied. Faust,
after the evaluation period. He gave
Merrill, Ross, and Bunderson and
up law school to became general
others started Courseware Inc. Dr.
manager, then president of WICAT,
Dustin Heuston and Bunderson co-
became head of computer education
founded WICAT Systems and the
at IBM, got a Ph.D. from the BYU
Waterford school, and recruited
department of Instructional Science,
many members of the TICCIT team.
and is today the president of Western
Heuston and WICAT’s president, Bob
Governor’s University, a fairly recent
Mendenhall, later agreed to merge
“market success” as the nation’s first
WICAT and PLATO into the PLATO-
fully competency-based university. As
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
7
discussed below, it embodies several of
lessons, but also their attitudes of
the most important lessons from what
responsibility toward learning and
was right and what fell short with the
their eagerness and confidence in
early TICCIT.
taking on new learning opportunities.
Since hardware and software technology change so rapidly, the disciplines of design-science-based development and team production and the architectural features pioneered in TICCIT have provided some stability to enable predictable and productive
The Evaluation of TICCIT’s First Year in the Colleges
NSF provided an additional $1 million so that researchers at Harvard University and
courseware production. Courseware,
the Educational Testing Service could
after all, is where learning may occur.
evaluate both TICCIT and Plato. This
Many concepts and people from
they did with skill and depth, visiting
TICCIT have indeed had a broad
both before the implementation and
influence in the market for what CAI
later collecting outcome data for
has become.
a year in the two colleges. Patrick
“Learner Controlled Courseware”
Suppes provided a succinct summary
was the byline of TICCIT. It was
of the evaluation of TICCIT in an
meant to suggest that instructional
important article on the state-of-the
materials—Courseware—could be
art of CAI in1979. Suppes was the
developed by a principled process
main consultant called on by Eric
that assured a high likelihood of
McWilliams, the very effective NSF
instructional success, and that
program manager in site visits to
the elements of that courseware—
evaluate the progress of the BYU and
instructional variables—could be
MITRE team. Suppes kept our feet to
placed in the hands of the learners
what he regarded as the appropriate
through a “learner control language”
fire. He did this by drilling into one
implemented through a special
topic during each visit; nailing us to
keyboard. The TICCIT courseware
the wall on the issue during the visit.
team tried to convey with these words
After the NSF visitors left, we got
that the expertise learners could gain
down off the wall and tried to correct
through becoming proficient with
our sins of omission or commission.
the learner control language would
While his summary is terse, it is
accelerate their progress, not just
backed by his extensive personal
through the TICCIT math and English Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
8
Michael Allenʼs 2008 e-Learning Annual
knowledge of the TICCIT project (Suppes, 1979, p. 189).
“(5) The success rate of students who took the TICCIT mathematics more than once seemed to indicate that the courseware did
“The TICCIT project had the responsibility to develop two communitycollege courses, one in English and one in
not provide sufficient remedial depth to teach some of these students. “These results are not terribly surprising.
mathematics. The curriculums of the two
It seems to me important that we do not have
courses are fairly standard and will not be
some immediate evaluation of CAI on the
reviewed here. The more distinctive feature
basis of a single year’s test as in TICCIT or
of the TICCIT courses has been the effort to
PLATO. It is rather as if we had had a similar
use an explicit instructional strategy focused
test of automobiles in 1905 and concluded
on learner-controlled courseware (Bunderson,
that, given the condition of roads in the
1975; Bunderson & Faust, 1976). The
United States, the only thing to do was to stay
Educational Testing Service (ETS) evaluation
with horses and forget about the potential of
of the TICCIT courses, as summarized quite
the internal combustion engine.”
objectively in Bunderson (1977), presents the following conclusions (see also Alderman,
Suppes ends his review of the ETS evaluation appropriately in cautioning
1978). “(1) When used as an adjunct to the
against jumping to premature
classroom, TICCIT (like PLATO) did not
conclusions. Ending an evaluation
produce reliable, significant differences in
after one year is indeed a poor
comparison with classes that did/not use
evaluation strategy. It was expensive
TICCIT (or PLATO).
enough for NSF, but I now recommend
“(2) When used as an integral scheduled
that evaluation strategies should take
part of either mathematics or English classes,
on a design experiment flavor by
TICCIT students did significantly better than
spreading costs over more cycles of
non-TICCIT students.
evaluation and revision.
“(3) Characteristics of the teacher are significant in determining the performance and the attitude of students in both TICCIT
Focus of This Article
Many stories can be told of
and non-TICCIT classes, a conclusion that
these times,
matches much other research of a similar
stories generated
sort.
by the hard
“(4) There was a difference of about 20
and soft technologies, the
percent in completion rate in favor of non-
instructional psychology and research
CAI classes for the TICCIT classes.
adventures, the sobering realities of
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
9
implementation in organizations, for
statement, and what did we learn—
example, the irony that educators who
during TICCIT and afterward?
were often thrilled with the words about learner control and learner
Stories not included here:
initiative but had a need to stamp
Instructional Theory, Component
it out when it actually started to
Display Theory, and Concept
occur. Some of the most interesting
Elaboration Theory, which was born
stories involve the people who passed
during the TICCIT project at BYU.
through the labs at UT Austin, BYU,
Instructional theory stories are best
Courseware Inc., WICAT, Institute
told through the writings of M. David
for Computer Uses in Education,
Merrill, Charles Reigeluth, and others.
and other companies that built on
Other information on TICCIT and its
TICCIT concepts. Except for the few
features can be found in the excellent
side comments, however, in this short
book prepared by Dave Merrill, Ed
article I will be content to tell just
Schneider, and Kathy Fletcher (Merrill,
three stories about lessons that have
Schneider, & Fletcher, 1980). This
endured; about important problems
book provides details of the TICCIT
that remain unsolved. These three
learner control strategy, the testing
stories are about:
strategy, the materials development
1. Learner Control. How far did we
methods, and many other details not
actually go toward understanding
found in this article.
and realizing the depth of what lies beneath this term? 2. Progress Maps Culminating in
Learner
Before TICCIT,
Control
the UT CAI lab
Mastery Assessments. How well
team conducted
did we understand the importance
learner-control
and the science at the base of these
research by giving
artifacts? 3. Differentiated Roles and
the learners more and more control within the hierarchical structures
Technology Integration. We boldly
of prerequisites used at that time to
proclaimed in the proposal that
model the domain. Also, the learner
the teacher would not be replaced,
controls were constrained by the
but that teachers’ roles would be
type of tactical options we had then
greatly differentiated. How little
imagined that learners would need
did we really understand that
(see Judd, Bunderson, & Bessent,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
10
Michael Allenʼs 2008 e-Learning Annual
1970). This research was equivocal
the nature of the “controls” interaction
on comparing the effects of learner
designers give to the learner are vital.
versus program control on criteria
But unless the learner controls act
of efficiency and effectiveness.
within a system that itself is modular
Although the UT team built larger-
and model-based, how can a learner
than-usual systems,, it was doubtful
come to grasp the meaning of the
that the scope of the online learning
controls? Imagine a pilot in a cockpit.
experience was extended enough in
A good pilot has learned to understand
engagement time to determine to what
immediately the meaning of multiple
extent the ability to control one’s own
dials and gauges and instinctively to
learning may be a skill learned slowly
move the steering wheel or joystick,
and requiring substantial practice.
often reaching simultaneously with
Looking back from today’s
the other hand for other controls.
perspective, it is easy enough to ask:
Based on an interpretation of
Why not? Proficient reading itself,
information seen through the window
library searching, and now Google
and from the displays reporting
searching is a skill that manifests
measurements, the pilot can take
very wide differences in individual
just the right action required to
proficiency. In 1971, we had begun to
further a mission in the face of the
suspect that learner control might take
unexpected. The information displays
a long time to develop, but we wanted
present information that permits a
to believe that the TICCIT courseware
knowledgeable pilot to infer actual
experience would be extended enough
states of what key abstract constructs
to show an effect. At least, we thought,
refer to. These constructs include
questions about learner control
altitude, speed, wind direction, and
had a better chance of permitting
geolocation.
proficiency to be developed during
These readings interpreted by the
the TICCIT courses than in earlier
pilot all have a theory behind them
supplementary lesson systems.
and have physical consequences that
When proficiency in directing one’s
will follow lawfully and often very
own learning is seen as a domain
quickly. They are not ad-hoc points of
itself, perhaps this “learning to
decision in a learning content space
learn” proficiency is as valuable as or
that has no coherent structure or
more valuable than the outcomes in
theory behind it. What information
mathematics or English TICCIT was
and controls should be given while
designed to teach. If this is so, then
learners are engaged in a mission
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
11
in a learning space? By analogy,
non-cognitive components: volition,
information should be presented that
motivation, persistence, and voluntary
ties into a coherent set of abstract
approach versus avoidance. Other
constructs related to progress and
terms are conation, affect, attitudes,
location in a learning domain. This
and dispositions. Quoting again from
interpretive framework will reveal
the 1971 proposal to capture the words
a model of the domain. It will
and ideas of that time (MITRE, 1971,
certainly contain learning milestones
p. 73):
and goals and status information, such as TICCIT provided with the
“In summary, minimum evaluation of the
Objectives and Status display. This
success of this project is dependent only on
TICCIT map provided access to
criteria of cost, efficiency, and effectiveness
objectives describing what goals were
relative to baseline data in junior college
to be attained and colored status
math and English. . . . However, the
information depicting progress. Based
courseware design is styled and structured
on possible interpretations of this
in fundamental ways to conform to values
information, the controls the learner
and suppositions about the future which
needs should offer great utility in
be beyond stated performance objectives. .
enabling the learner to reach those
. . The goal is to encourage self-motivated,
goals safely and expeditiously, from
skillful independent learning which will lead
wherever a learner may be within the
to further self-initiated encounters with the
domain at a given time—and despite
subject matter by as many students as can be
whatever difficulties that learner may
induced to accept the challenge.”
have encountered in getting there. The nature of the domain of
How well did we do in achieving
proficiency in learner control is
the non-cognitive goals? The
certainly central in understanding
ETS evaluators discussed them as
learner control. Another issue became
background ideals (Alderman, 1978, p.
the focus of considerable attention
29):
before and during the TICCIT project. It was this: What other characteristics
“Goals, after all, tend always to exceed
besides (cognitive and psychomotor)
our grasp. . . . Since goals influence
proficiency might develop through the
decisions, it seems reasonable that such
use of learner control of instructional
statements balance an enthusiastic optimism
variables? We are dealing here
about potential accomplishments and an
primarily with variables having strong
historical realism about results of prior
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
12
Michael Allenʼs 2008 e-Learning Annual
work. Here we took the developer’s goals as
do to help the students attain these
earnest statements of intent which should
simple fundamentals.
guide, though not dictate, our attention in evaluating the TICCIT program.”
Thankfully, the ETS evaluators
Still, there was enough positive evidence to cheer us, even in that difficult first year that the evaluation
did not seriously try to evaluate our
targeted. Some of the students
attainment of these goals during that
really loved it. Some of the students
first year of learning how the two
succeeded where likely they otherwise
colleges were going to cope with what
wouldn’t have. On average, once they
we had produced. Many adjustments
had completed TICCIT, their scores
had to be made to get the best results
were higher than those in the lecture
out of TICCIT. Now the usual term is
groups.
“Technology Integration.” We went into the implementation
In the TICCIT design experiment described below in connection
thinking that the planning was
with the third TICCIT story, and as
adequate. The systems engineers
implemented in the differentiated
at MITRE put capable Ned Burr
staffing model of Western Governor’s
on the job, and he worked with us
University, we have found approaches
to produce a massive document
that will work well for technology
called an “Implementation Plan.” It
integration. “Technology Transfer” is
seemed to cover everything, training,
not an adequate term. It is not enough
documentation, dealing with people’s
to transfer tools to a group organized
attitudes. We academics were very
in a conventional manner, following
impressed with this evidence of
established roles and rules. The
MITRE’s experience in implementing
biggest role shift is in the students’
large-scale projects, but we learned
role. It is necessary to reorganize roles
how little we had really understood.
and rules substantially and to teach the
Students unfamiliar with TICCIT
students new ways of organizing their
CAI—with any CAI at that time—
time and methods of learning. These
would first have to learn how to
new patterns of learning include both
take action to work on the system
individual work and learning to work
and complete their lessons, let alone
with cohorts of students with similar
change their proficiencies to learn,
goals. These changes result not in
and their attitudes and dispositions.
technology transferred into an existing
Teachers would have to learn what to
system, but in what really should
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
13
be described as a new system. Since
and improved through riding up the
conventionally organized educational
learning curve of technology that they
institutions find it difficult or
begin to compete with the established
impossible to implement such radical
markets.
changes in the roles and rules around
Back to the Learner Control Story:
which they have been organized
Evaluating CAI, including TICCIT
for decades, even for centuries, the
as a disruptive technology does not
technology is too disruptive to use in
take TICCIT’s learner control strategy
the ways that might work best.
off the hook. It is clear that the
Disruptive Technologies: Clayton
controls provided and the information
Christensen uses the term “disruptive
displayed on the maps was not
technology” in his book Innovator’s
sufficient to achieve the broader vision
Dilemma. He explains how a
of what learner control is all about. For
technology that the current system has
one thing, it did not span an adequate
difficulty using can become a market
range of preferences as shown in
success: He points out that the way
different models of thinking and
a disruptive technology might finally
learning preferences. My colleagues
achieve a market success is to skirt
and I have conducted extensive
the existing customer base and go to
validation research with the Herrmann
market niches underserved by the
whole brain model of thinking
existing technologies. In education,
preferences using large international
organizations like University of
databases of surveys taken over the
Phoenix, Western Governor’s
last twenty-five years (Herrmann,
University, and other online
1988—see validation appendix;
universities have found such a niche—
Bunderson, Wiley, & McBride, 2007;
thirty to forty some people with family
Olsen 2007). This model interprets
and work responsibilities who cannot
four clusters of .preference choices,
travel to campuses and take classes
made under scarcity. By scarcity,
scheduled during their prime work
we mean that the respondent has
time. As these organizations achieve
limits on how much can be chosen in
market successes in a new niche that
expressing preferences. The preference
does not require the same definitions
clusters form at opposing ends of two
of roles and rules of procedures as the
bipolar factors. One bipolar factor
campuses, a point is reached when the
favors analytical, logical, mathematical
offerings of the disruptive technology
thinking at one end and interpersonal,
vendors become so well understood
emotional, thinking favoring an
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
14
Michael Allenʼs 2008 e-Learning Annual
interest in human relationships at
synthesizing tasks at a high level of
the other. Experience has shown that
generality. They needed practice going
these preference clusters are relevant
beyond fixed lists of items into larger,
to what type of learning environment
more synthesizing projects, perhaps
and instruction work best for an
even leading to capstone projects that
individual.
tie lower subdomains and objectives
TICCIT designers provided a lot for those who favored the logical left brain. There were plenty of displays TICCIT designers provided a lot for those who favored the logical left brain.
together. The TICCIT English project had its writing assignments, and our
involving logic
greatest documented successes with
and algorithms,
TICCIT came through freeing the
found in the
teacher’s time up by letting TICCIT
rules and helps.
teach the grammar, mechanics, and
Unfortunately,
organization lessons. This enabled the
what many people-oriented students
teacher to spend more time managing
needed was to work in smaller or
and mentoring writing assignments.
larger groups with others. They could
Otherwise, the TICCIT learner control
work around open tables on their own
strategy was all within the organized,
TICCIT terminals, but did not prefer
step-by-step type of thinking favored
the walled-off carrels in the TICCIT
by only one of the four quadrants of
lab. They preferred more human
thinking preference.
interaction, and we eventually learned how to provide it. The other bipolar factor in the
Can we really hope for dispositional changes in students when they are given more choice of controls that
Herrmann model features preferences
really matter and plenty of time
for order, organization, and step-by-
to practice it? The jury is out on
step methods at one end and creative
some of the goals we had, such as
synthesis, taking risks, seeing the big
enhancing a voluntary, eager approach
picture at the other. TICCIT was very
to learning. The doctoral work of
organized and orderly, with a place
Margaret Martinez (1998) poses a
for everything and everything in its
challenge to the supposition that
place. What some of the students
dispositions are really very modifiable.
needed was to see the big picture more
She has identified four “learning
completely than the TICCIT map
orientations,” the transforming,
hierarchies conveyed. They needed
performing, conforming, and resisting
better options to explore and try out
learner. These “orientations” may
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
15
shift through learning, but apparently
the behavior noted above—TICCIT
only with great difficulty. Martinez
students who over-used the advisor to
teaches instead how to maximize
find out just how many more problems
the effectiveness of learning using
they had to work in order to pass out
the orientation you have already
of that lesson.
established. Does Learner
Going way
to genetic nature or nurture and life
Control by Any
beyond the
experiences, a transforming learner
Name Have a
Whether one’s orientation is due
is going to want to see the big picture and jump around a lot, if given the
Future?
highly structured TICCIT approach to learner control,
learner controls. The conforming
do instructional psychologists/
learner wants to be told just what to
technologists today see anything
do, and what steps to follow to do
in the rapidly changing world of
it. The resistant learner is not going
technology—any evidence of the eager
to buy in at all, but when s/he does,
motivation that impels the learning of
look out, this may be a transforming
any content, logic, or skill required?
learner in disguise, one who can
Perhaps. We see the young give
come to love learning. A person in
big parts of their lives to becoming
resistant orientation may be one who
amazingly proficient in role-playing
has refused to buy into seemingly
games. We see incredible Massively
stupid demands and assignments
Multiplayer Online Role-Playing
administered in a structured
Games (MMORPG) like World of
educational system they hate. Some
Warcraft that require the development
kinds of learner control may work
of superior social and leadership skills
to effect a leap of members of this
to organize people and material for
group from resistant to transforming
successful campaigns of conquest
learning.
and acquisition. As John Seeley
In most environments, the
Brown (Brown & Thomas, 2006) has
orientation forming the biggest group
pointed out, the skills attained by a
of learners is called “performing
seventh level Guild Master are the
learners.” They know how to learn and
skills needed to organize people and
will tolerate the system, but their goal
accomplish complex and important
is to minimize effort in checking off
missions in corporate global business.
the attainments the system demands. They have attitudes that could lead to
The motivations are there in games. These are the powerful motivations
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
16
Michael Allenʼs 2008 e-Learning Annual
the TICCIT designers dreamed
It is a broad synthesis of navigational
of for learning environments, but
utility made subject to the gamer’s
could not attain through structured
growing intelligence by a growing
lessons focused directly at learning
knowledge of what the different
objectives. The funny thing is, the goal
measures of attainment mean. Games
of a game is not learning. Learning
might connect in a sort of oblique way
is an incidental by-product of deep
with academic curricula in schools and
investment in the goals of some
colleges, but that connection is hidden
imaginary game-world. These are
and incomplete at best.
larger goals that matter to the people
Perhaps most important, life in an
who choose to make the investment.
MMORPG is life in a meritocracy.
Education has broken the subject
True, you can buy your way to more
matter into neat little packages that
rapid advancement with enough
build on one another to permit a class
money, but you still have to perform to
credit that enables one to take another
stay there, and continue to perform to
class, then another, until one earns a
attain more. Thus complex multiplayer
degree.
games are meritocracies. Schools and
The role restructuring for the individual learner is there in a game world. In an MMORPG, what the
many other human institutions are hardly meritocracies. In searching for the motivational
learners control is a comprehensive
power to attain high levels of
set of functions. They choose from
proficiency and to gain strong
and also shape available goals. They
dispositions toward continued
control the methods for obtaining
learning, the designers of TICCIT
the means needed to seek the goals.
made some very small steps toward
They learn skills
the non-cognitive goals, and toward
and strategies for
the idea of high levels of learning
fighting to keep
proficiency. I am impressed with the
and add to the
online meritocracies in the open-
Gamers have multiple measures of progress at hand.
means toward the goals they value,
source world of the Internet and in the
and so on. Gamers have multiple
game worlds. An increasing number
measures of progress at hand—their
of online communities are structured
level of character attainment in each
as meritocracies, such as Slash
of their avatars, their wealth, health,
Dot, wikis, and user communities
strength, possession of weapons,
associated with online tools,
possession of defenses, and on and on.
(especially open source tools). These
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
17
communities are structured around
and attention, the broadened concepts
pure technology expertise at present,
behind TICCIT maps are likely to
even though more effective and faster
receive increased attention.
transfer of expertise might occur with
After working at WICAT for a
the application of learning design and
time, I had the opportunity to work
measurement disciplines.
at Educational Testing Service as vice
It is easy to lament their lack
president of research management
of recognition of instruction,
during a period of intensive work
measurement, and the like in the new
on the goal of developing new
meritocracies of digital expertise. It is
applications of testing integrated
easy to lament that a little technology
with learning. One ETS initiative in
expertise often trumps experience and
the research division during those
know-how in instructional technology.
days was to develop examples of
But let it grow. Maybe instructional
“Learning Progress Systems.” Such a
technologists can learn as much or
system was conceived of as having a
more from the digital natives as they
map or interpretive framework for
could (but probably won’t) learn from
valid measurements of progress along
the instructional technologists. Is this
meaningful pathways of learning
where the broader mission of learner-
progress. The concept was seen as one
controlled courseware is going to find
of several possible ways to implement
some fulfillment?
Systems which Integrate Learning and Instruction (SILAS; see Snow
Progress Maps Culminating in Mastery Assessments
Progress maps
& Mandinach, 1999). ETS continues
display learning
research with this intention under a
progress and
variety of names and descriptions.
goal information and provide
navigational controls. The TICCIT map (objectives and
I brought to TICCIT a background in measurement from Princeton and from a practical and theoretical education at ETS as a Psychometric
status display, with navigational
Fellow. I tried to assure that the
control) is a concept that has far
measurement systems underlying the
more depth than we realized at the
TICCIT practice files, status displays,
time. While the RULE EXAMPLE
and advisor status information were
PRACTICE EASY HARD HELP
respectable. Dr. Edward Schneider
tactical commands have not sustained
joined the TICCIT team at BYU and
a large amount of continued research
developed an adaptive testing system
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
18
Michael Allenʼs 2008 e-Learning Annual
for the TICCIT tests, based on the
model” that integrated performances
Wald sequential testing approach
at a higher level (Bunderson, Gibbons,
(explained in Merrill-1980). Students
Olsen, & Kearsley, 1981; Gibbons,
did not need to take all the test items
Bunderson, Olsen, & Robertson, 1995;
in the file. They could pass out or fail
and Gibbons & Fairweather, 1999,
out more quickly.
Chapter 15).
The interpretive framework feature
Charles Reigeluth struggled with
of a learning progress map, discussed
the problem of fragmentation versus
as crucial to learner control in the last
integration as a graduate student at
section, depends on having a theory
BYU during TICCIT times. Later, he
by which the measures of progress
has written on Concept Elaboration
can be judged. In TICCIT, we had
Theory (Reigeluth, 1999) in an attempt
produced a Gagne-style “learning
to deal with unfolding sequences of
hierarchy” for each lesson and unit,
increasing complexity, rather than
and indeed, for the entire course. The
fragmented learning hierarchies.
rules for producing these hierarchies
Returning to an academic
were based on analysis into the classes
environment at BYU in 1991, I worked
of learning described by Gagne in
with several colleagues and graduate
his 1985 book, The Conditions of
students in an effort to understand
Learning. These categories were
the measurement foundations for
modified somewhat to fit Merrill’s
progress maps and how to develop
Component Display Theory. Such a
them. Influenced by the work of
learning hierarchy produces one type
Samual Messick on validity, we
of theory of the domain, but I left the
developed concepts of a “Domain
TICCIT experience thinking that this
Theory of Progressive Attainments”
type of learning hierarchy could result
(Bunderson, 2006, Bunderson, Wiley,
in fragmented instructional pathways.
& McBride, 2007). A domain-specific
The TICCIT rule-example-practice
theory of progressive attainments
strategy worked well with concepts
is one candidate of replacing the
and simple rules and procedures, three
learning hierarchies used in producing
of the Gagne/Merrill types of learning,
the TICCIT maps. The definition of
but not so well with performances
validity presented by Messick (1995),
that were integrated at a higher
by the writers of the Joint Standards
level. We had noted this problem of
on Educational Tests (AERA, 1999),
fragmentation versus integration and
and by other validity theorists is the
had developed the concept of a “work
key to understanding the need for an
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
19
interpretive framework in a domain.
map provides, which is the essential
Validity resides in interpretations
meaning of modern conceptions of
and actions taken in response to an
validity.
understanding of what measurements
The idea of navigational control
mean; validity does not reside in a
throughout a courseware domain and
measurement instrument itself, or
the idea of meeting the conditions
in its scores. Therefore, if the users
of conjoint additive measurement in
cannot interpret the meanings of
the measures reported on that map
measures, they cannot respond
to guide learning progress are still
quickly and adroitly through controls
under development at the EduMetrics
whose use depends on a correct
Institute; also, it is found in the work
interpretation of the measures.
of a number of young Ph.D.s who did
Validity is lacking in such a system,
their dissertations on aspects of a
and the broader goals of learner
domain-specific theory of progressive
control discussed in the last section
attainments in areas of interest to
are unlikely to be achieved.
them (Bunderson, Wiley, & McBride,
As this work progressed, we discovered other work in a similar area that also represents a basically compatible
2007; McBride, 2005,; Strong-Krause, 2001; Xin, 2002). One of these former doctoral
and very promising approach to the
students is Dr. Thomas Zane, director
development of progress maps. This
of assessment in the NCATE-
work by a number of investigators has
accredited Teachers College of
been done under the heading Knowl-
Western Governor’s University. As
edge Spaces (see Falmagne, 1990, for
mentioned above, WGU has been able
an early account). A knowledge space
to implement a role-differentiation
is itself a rigorously defined domain-
approach that separates the roles of
specific theory of progressive attain-
teaching, assessment development
ments. It has a promising mathemati-
and administration, mentoring,
cal foundation. Knowledge spaces is a
tutoring, grading, and other matters.
candidate for the theory of the domain
The WGU approach includes social
structure needed as a foundation for
and organizational variables that help
developing the type of learning prog-
solve the problem of low completion
ress maps envisioned here.
rates manifested so sharply in the ETS
The emphasis in our work has
evaluation of TICCIT. Fundamentally,
been on assuring that learners can
WGU shifts the student’s role and
interpret and use the information the
separate a variety of teacher roles.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
20
Michael Allenʼs 2008 e-Learning Annual
WGU is a competency-based university with a method for defining
based on domain-specific theories of progressive attainments.
credible domains and developing aligned assessments. WGU starts
Differentiated
the process of domain definitions
Roles and
This last story describes
in a conventional manner, defining
Technology
how design
domains, subdomains, competencies,
Integration
experiments using
and objectives. Developing a full
differentiated
domain theory of progressive
staffing models as a replacement for
attainments in a domain would
premature evaluations can lead to the
require integration beyond the
attainment of more of the hoped-for
use of objective and competency
high expectations for TICCIT.
statements into pathways that reflect
The news of low completion
integration at higher and higher levels
rates from the junior colleges hit us
of complexity. Feasible steps toward
hard at the TICCIT lab at BYU. Now
this ideal are taken by Tom Zane,
reduced to a fraction of the staff we
whose team is encouraged by the
had during the TICCIT courseware
University officers to use integrative
development effort, we were still
performance measures, not just
burning to know how to make the
objective tests. Zane uses concepts
exciting TICCIT concepts work in
like work models, cited earlier, and
practice. It was clear that students
for assessment, makes heavy use of
didn’t already know how to use a
performance tasks, domain level
system like TICCIT to learn on their
projects, and capstone projects that
own, and that only a relatively small
integrate across domains. Zane’s staff
percentage of students really had the
uses these methods in developing,
characteristics Maggie Martinez later
evaluating, and improving new WGU
identified as those of a “Transforming
assessments. This assessment group
Learner.” These are the learners who
is able to take appropriate steps,
can learn on their own in almost any
sometimes small ones, sometimes
environment. We believed that the
larger, that fit the state of development
innovative TICCIT system was really
of the student population and the state
still in its toddler-hood and that its
of development of WGU. These are
users needed to learn many things to
steps toward realizing a fuller vision
help it grow up.
of measurement of learning progress
To do this, we found that faculty support at BYU was not strong.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
21
English faculty at BYU were polite
organizational conditions. Comparison
but disinterested. Math faculty would
from cycle to cycle depends on the
not use TICCIT in the appropriate
comparability of the measurement—
classes. TICCIT, after all, dealt with
their invariance properties—from
low-level community college material
one cycle to another. The cycle for
and did not honor math faculty
these college English classes was one
members’ vision of the importance
semester in length.
of their unified and dominant role.
During the first semester, several
Most English faculty members were
graduate student instructors taught
interested in teaching literature, and
TICCIT sections. We will examine
this left graduate students free to
the results of two of these student
teach writing with a fair amount of
instructors. The first one (Christine
freedom. The English faculty allowed
Hansen, who with advanced degrees
the TICCIT lab to recruit several
later became head of all the BYU
graduate students who were teaching
writing labs) tried hard to integrate the
sections of English in preparation for
computer. TICCIT had a set of tutorial
the university-wide required writing
lessons in grammar and mechanics.
exam.
It also had writing lessons dealing
All sections at BYU took the same
with audience, purpose, structure,
test at the end of each semester. The
and outlining. The second teacher
test consisted of two parts; a multiple-
did not believe in the computer, but
choice grammar and mechanics
was confident that she could teach
test and a written essay graded by
students to write. The baseline and the
teachers according to generally
results of the first cycle of the design
accepted standards. It is unusual for
experiment are illustrated in
a university to develop one standard
Figure 1. This figure uses the term
measurement system for all sections in a particular domain, but this was one of those unusual times. The tests
TICCIT: The J curve of Implementation in a Design Experiment
were scored by classical means and lacked desirable invariance properties, but were suitably comparable for
100% % Passing General Ed Writing Exam
the large effects noted. A design experiment takes place over repeating
65 – teacher who ignored TICCIT, Taught students to write Baseline: 55%
cycles of implement/evaluate followed 40%
by redesign, revise the social and
Cycles 0 and 1
45 – our teacher who tried hard to use TICCIT
Semester cycles of design experiment
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Figure 1. Results After One Cycle of TICCIT Design Experiment
22
Michael Allenʼs 2008 e-Learning Annual
“J-curve of implementation,” borrowed
the classes taught by all teachers in
from economics. The curve of percent
the study. TICCIT generated extensive
passing drops before starting up again.
data on student progress in every
“Cycle 0” was merely to
lesson and every segment of every
obtain baseline data from all the
lesson. From observation and data,
conventionally taught English
It was easy to see that students had
sections, so future cycles of design
no habit patterns for how to learn on
and improvement could refer back
their own with a computer. As was
to it. The results of Cycle 1 were
found with the community colleges,
not promising for the teacher who
the computer was greatly underused
sincerely tried to use TICCIT as
in the BYU lab, although the BYU
it was supposed it should be used.
college students appeared to have
On average only 45 percent of her
greater success in lesson completion
students passed the GE exam. The
that had the community college
other teacher said to her students
students. The committed teacher and
in a variety of ways: “Go use that
her support group designed better
computer we’re supposed to use in
implementation plans for the second
whatever way you wish, but here in
semester, plans designed to help the
my class I’ll teach you how to write.”
students understand their roles better,
The result of her efforts: 65 percent of
and the teacher to redefine her role
her students passed the GE exam. The
and her new rules of procedure. No
baseline average was 55 percent across
changes were made in the hardware or
the university. After one cycle, the
courseware, just in the implementation
logical conclusion would be to teach
of new roles and new rules. Students
the methods of the second teacher to
were organized into small groups
teachers in the other sections, and not
and scheduled to meet at computer
use TICCIT at all.
terminal tables, where they could see
But the English faculty had
and talk to one another. A “party” was
delegated Freshman English to
held in the computer lab to get them
graduate students, and we were spared
started and to assign tables and small
the risk of pre-judgment or premature
groups of cohorts. At the end of the
judgment.
second cycle, the TICCIT teacher’s
Christine Hansen believed she could
class did better than the second
organize and get much better results
teacher’s had the previous semester,
during the second cycle. The team of
72 percent compared to 65 percent,
researchers studied data available on
and substantially higher than baseline.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
23
Figure 2: TICCIT Individual Carrel (Above) and Open Group of Table-Mounted Terminals (Left)
More TICCIT lessons were completed
beginning of the semester, and they
in Cycle 2 than in Cycle 1.
came to the TICCIT lab at the same time, as though it were a regular class
Figure 2 illustrates one of the environmental adaptations used to create a social environment among the
period. Each group had a name, such as “Emerson” or “Thoreau.” Hansen, backed by the TICCIT
TICCIT English students. The upper
team, still believed she could do
picture features a very private, one-
better. This team also believed that
student TICCIT carrel. Designed with
the students, backed by professional
human factors expertise by Dr. Edward
developers and researchers, could
W. Schneider, this carrel featured no
do much better, so more changes in
glare on the screen, a stand for books,
roles and rules were made. At the
spaces on each side—privacy for
end of the third cycle 84 percent of
only one person. The bottom photo
that semester’s group of students
shows a long table with no partition,
in the TICCIT section passed the
designed by Scheider as well. With
GE exam. The students in these
TICCIT terminals on each side, this
subsequent semesters accepted their
arrangement enabled students at the
role to learn and practice grammar,
table to see their friends and talk
mechanics, audience, and purpose
over the table and to those seated
on the computer, and the teacher
at each side. Their regular seats and
spent more time with small groups
their group of six was set up at the
motivating and mentoring them in
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
24
Michael Allenʼs 2008 e-Learning Annual
how to write. The team made further
for working with small groups. Using
refinements and tried again. At the
the reports, those not making good
end of the fourth semester, 93 percent
progress could be identified early and
of this new group of TICCIT students
encouraged and taught individually,
passed the two-part GE exam.
while most of the class was involved
Significantly more TICCIT lessons
in practicing and studying on the
were being passed, and the scores on
computer or working on their writing.
the objective grammar and mechanics
This large effect was attributable
test were significantly higher. Even
to variables entirely from the
higher was the improvement in writing
social and organizational realm
scores—this was the key to the strong
of implementation or change
performance of the TICCIT-using
management. There were no NSF or
teacher’s students on the writing exam.
University funds left at that time to
See Figure 3 (Below).
redesign the hardware, software, or
The of implementation Implementationinin TheJJ Curve curve of TICCIT TICCIT English—Cycle English – cycle44
TICCIT English – cycle 4
100% % Passing General Ed Writing Exam
93
courseware. Bob Mendenhall was an undergraduate research assistant during these TICCIT studies. He went on to become general manager, then president of WICAT Systems. This company recruited a
Figure 3. TICCIT Design Experiment Results Over Four
good number of the TICCIT team
Baseline: 55%
members. As president of Western 40% Semester cycles of design experiment
Semesters
Governor’s University, he now uses as a standard management
The improvement in Cycle 4 has an effect size of .96 from baseline, and even higher from the low-point in the J curve. This high pass rate for Cycle 4 was attributed to the teacher beginning to use the reports generated by the computer system to identify the number of TICCIT lessons the students were completing. This implementation tactic was added to further refinements in the previous successful implementation methods
strategy the best measures he can get for all aspects of the University’s functioning, and insists on making revisions and following up to see whether the measures improve in the desired direction. It is not as tight as a design experiment, but does not require publication of results, and it is cost-effective for an organization. The WGU model of differentiated staffing is one viable model for elearning in higher education. The
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
25
model is documented at one stage of
or orientations toward learning will
its development Mendenhall’s (2001)
prove difficult or even impossible
dissertation, but due to continuing
to change. It may be better to find
data-based revisions since that time,
strategies to help learners become
the differentiated staffing model and
more successful within the constraints
other aspects of the WGU model have
of different preference profiles,
shifted a bit since that time.
learning orientations, or dispositions. Looking ahead, systems radically
Conclusions
Projects and
different from TICCIT’s highly
systems come
structured approach to learner control,
and go. People
perhaps involving games, may offer an
move on to
essential environment of meritocracy;
other challenges
also, these environments may offer an
and other accomplishments. As in
indirect approach to attaining learning
every field of human endeavor, some
outcomes, but a direct approach to
problems in succeeding with CAI are
attaining greater proficiency to learn
very difficult to solve. Three stories
what is needed to achieve other valued
were told about TICCIT and about
goals.
continued efforts to solve a class
The TICCIT story about
of problem. In the Learner Control
navigational maps with valid progress
story, a key problem is capturing
feedback shows that measurement of
enough motivation in an e-learning
learning progress with deep validity
system to attain the broad goals of
is a high goal, difficult to achieve.
increased, transferable proficiency in
Valid measurement depends on
learning, strong and eager approach
theory-connected interpretations.
toward further engagement with the
Validity of inferences drawn from
subject, and a strong sense of personal
progress measures and goals lies
responsibility for learning progress
in the interpretations and actions,
and learning attainments. It is not
not in the numbers produced by
clear that some of the non-cognitive
the online measurement process. If
traits associated with successful
constructed properly, navigational
learning are teachable and learnable,
maps can provide feedback within
even with an ideal learning language,
an interpretive framework fully
progress mapping interface, and given
understandable to the leaner. Within
long engagements in systems that use
this framework, learners can identify
them. It may be that some preferences
their location in a learning domain and
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
26
Michael Allenʼs 2008 e-Learning Annual
determine where to move next. It is
been attained. Design-experiment-
very hard to put together the tools and
like changes over cycles should
resources in hardware, software, and
replace premature evaluations of
implementation that came together
innovations. This form of data-based
for a brief moment to produce
improvement based on promising
TICCIT. Thus, whether the type of
models and theories can lead to the
measurement-based navigational maps
large effect sizes that the TICCIT
and other tools will work as hoped is
designers hoped for. The design
uncertain. Can a deeper conception
experiment presented in this article
of progress maps be implemented
did occur with TICCIT within two
with navigational controls designed
years of the ETS evaluation. The
for learning proficiency in well
principles it taught, however, can
understood learning domains? Will
be effected with a variety of types of
these tools enable learners to attain
systems that integrate technology with
proficiency and motivation sufficient
learning. Large effects are possible
to use learner controls adroitly
with well-designed and executed
enough to select the best options for
implementations of what CAI has
immediate and longer-term progress?
become. Successful implementations
Perhaps another generation will find
will involve role differentiation for
the answer.
students and teachers, and will involve
The third TICCIT story involved
integrative projects that both stretch
differentiated staffing and cycles of
students, and measure their progress
improvement in competency-based
toward high, educationally valuable,
environments. Both during TICCIT
and intrinsically motivating learning
and later, successes in implementing
challenges.
differentiated staffing models have
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
27
REFERENCES AERA, APA, NCME. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Alderman, D.L. (1978). Evaluation of the TICCIT computer-assisted instructional system in the community college. Final Report, Volume I. Princeton, NJ: Educational Testing Service. Brown, J., & Thomas, D. (2006, April). You play World of Warcraft? You’re hired! WIRED, 14(4). Bunderson, C.V. (1973, September). The TICCIT project: Design strategy for educational innovation. In S.A. Harrison & L.M. Stolurow (Eds.), Productivity in higher education. Washington, DC: National Institutes of Education. Bunderson, C.V. (1977). A rejoinder to the ETS evaluation of TICCIT (CTRC TR 22). Provo, UT: Brigham Young University Press. Bunderson, C.V. (2006). Developing a domain theory: Defining and exemplifying a learning theory of progressive attainments. In M. Garner, G. Engelhard, M. Wilson, & W. Fisher (Eds.), Advances in Rasch measurement, Volume I. Maple Grove, MN: JAM Press. Bunderson, C.V., & Faust, G. W. (1976). Programmed and computer-assisted instruction. In The psychology of teaching methods (75th Yearbook of the National Society for the Study of Education), Part I, pp. 44-90. Chicago, IL: University of Chicago Press. Bunderson, C.V., Gibbons, A.S., Olsen, J.B., & Kearsley, G.P. (1981). Work models: Beyond instructional objectives. Instructional Science, 10, 205-215. Bunderson, C.V., & Newby, V.A. (2006). The relationships among design experiments, invariant measurement scales, and domain theories. In M. Garner, G. Engelhard, M. Wilson, & W. Fisher (Eds.), Advances in Rasch measurement, Volume I. Maple Grove, MN: JAM Press. Bunderson, C.V., Newby, V.A., Olsen, J.B., and Wendt, D.C., (2007) Validity of the HBDI four-quadrant and left-right structure: A structural validation using a new millennium database, Herrmann International Validation Update Technical Report, EduMetrics Institute, Provo, Utah Bunderson, C.V., Wiley, D., & McBride, R.H. (2007). Mapping attainments to enable learner-centered education: Designing for the integration of assessment with instruction. In C.R. Reigeluth, Instructional-design theories and models, Vol. III. Mahwah, NJ: Lawrence Erlbaum Associates. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
28
Michael Allenʼs 2008 e-Learning Annual
Christensen, C.M. (1997). The innovator’s dilemma: When new technologies cause great firms to fail. Cambridge, MA: Harvard Business Press. Falmagne, J. (1990). Introduction to knowledge spaces: How to build, test, and search them. Psychological Review, 97(2), 201–224. Gagne, R. (1985). The conditions of learning (4th ed.). New York: Holt, Rinehart & Winston. Gibbons, A.S., Bunderson, C.V., Olsen, J.B., & Robertson, J. (1995). Work models: Still beyond instructional objectives. Machine-Mediated Learning, 5(3 & 4), 221-236. Gibbons, A.S., & Fairweather, P.G. (1999). Instructional strategy III: Fragmentation and integration. In A.S. Gibbons & P.G. Fairweather, Computer-based instruction: Design and development. Englewood Cliffs, NJ: Educational Technology Publications, pp. 278-296. Herrmann, N. (1988). The creative brain. Lake Lure, NC, Brain Books. Judd, W., Bunderson, V., & Bessent, W. (1970). An investigation of the effects of learner control in computer-assisted instruction prerequisite mathematics (MATHS). Technical Report No. 5, Computer-Assisted Instruction Laboratory, University of Texas at Austin, Texas. Martinez, M. (1998). An investigation into successful learning—Measuring the impact of learning orientation, a primary learner-difference variable on learning. Dissertation for Brigham Young University (UMI# 7972948). www. trainingplace.com/source/research/Martinezdissertation.pdf McBride, R.H. (2005). A domain theory of fluent oral reading. Doctoral dissertation, Brigham Young University, Provo, Utah. McLallen, D. (1974). Analysis of the need and economic feasibility for computer-assisted instruction in high volume junior college courses. Dissertation: The University of Texas at Austin. Mendenhall, R.W. (2003). A model and principles for effective internet-based distance education. Doctoral Dissertation accepted by Brigham Young University, Provo, Utah. Merrill, M.D., Schneider, E. W., & Fletcher, K.A. (1980). TICCIT, in Langdon, D.G., Series Editor, The Instructional Design Library Volume 40, Educational Technology Publications, Englewood Cliffs, New Jersey. Messick, S. (1995). Validity of psychological assessment. American Psychologist, 50(9), 741-49.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
29
MITRE. (1971). Toward a market-oriented computer-based learning system (Phase III— Implementation, Validation, Field Test, and Evaluation). K. Stetten, principal investigator. McLean, VA: MITRE Corporation. Olsen, J.B., Bunderson, C.V., Newby, V.A., and Wendt, D.C., (2007) Validity of the HBDI four-quadrant and left-right structure, A structural validation using a database to Year 2000, Herrmann International Validation Update Technical Report. Provo, UT: EduMetrics Institute, Reigeluth, C.M. (Ed.). (1999). Instructional-design theories and models: A new paradigm of instructional theory. Mahwah, NJ: Lawrence Erlbaum Associates. Snow, R., & Mandinach, E. (1999). Integrating assessment and instruction for classrooms and courses: Programs and prospects for research. Princeton, NJ: Educational Testing Service. Strong-Krause, D. (2001). English as a second language speaking ability: A study in domain theory development. Doctoral dissertation, Brigham Young University, 2001. Dissertation Abstracts International, DAI-A 61/12, 4746. Suppes, P. (1979). Current trends in computer-assisted instruction. Advances in computers, Volume 18. San Diego, CA: Academic Press. Xin, M.C. (2002). Validity centered design for the domain of engaged collaborative discourse in computer conferencing. Unpublished doctoral dissertation. Provo, UT: Brigham Young University.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Patrick Suppes Patrick Suppes has published widely on educational uses of computers and technology in education, as well as in philosophy of science and psychology. Much of his research has been focused on detailed physical and statistical models of EEG- and MEG-recorded brainwaves associated with processing of language and visual images and continued development of computer-based curriculum in mathematics, physics, and English. He is a member of the National Academy of Education (1965), the American Academy of Arts and Sciences (1968), the National Academy of Sciences (1978), and the American Philosophical Society (1991). He has received the American Psychological Association’s Distinguished Scientific Contribution Award (1972), the National Medal of Science (1990), Lakatos Award Prize, London School of Economics (2003) for his 2002 book (Representation and Invariance of Scientific Structures), and the Lauener Prize in Philosophy (2004 Switzerland). Currently, Suppes directs the Education Program for Gifted Youth (EPGY) at Stanford University, which offers online courses in mathematics, physics, and English to pre-college students of all ages, with approximately four thousand part-time students worldwide. The program recently added the creation of an online high school whose first students started in August of 2006. Suppes is also doing research on the brain, with emphasis on language and visual images.
Addressing Diversity in (e-)Learning
Patrick Suppes
In conversation with Michael W. Allen As one of the most revered names in technology-assisted learning, it was an extreme honor that Patrick Suppes took time to talk with me about his career, how the idea of using computing technology to assist learning developed, and how he came to write his seminal Scientific American paper that caught the attention of scientists, educators, and the public alike. I wonder whether it’s a coincidence that both Suppes and Bitzer are physicists, that both demonstrated leading-edge success with learning technology, and both see individualization of the learning experience as a key to the future of an enlightened society. In any case, both scholars had the knowledge, confidence, and capability to shape and harness technology, and after demonstrating phenomenal successes years ago, they’re frustrated that e-learning is so poorly designed today and so often misses the big opportunities. There are important lessons to be learned here, and I’m extremely grateful that Professor Suppes was willing once again to share his insights and profound wisdom. —Michael W. Allen Allen: Dr. Suppes, as someone who
in terms of computer learning in the
has looked seriously at learning with
future. It might be useful to use them as
the support of technology, beginning
a basis and give you my comments on
at the point when computer interface
where I feel we have failed and where
capabilities first made computers
we have succeeded. The first article
usable by learners, I especially
is “Modern Learning Theory and the
appreciate your taking time to
Elementary-School Curriculum,” first
contribute to this first edition of the
published in the American Educational
e-Learning Annual.
Research Journal. This was the paper I gave when I was invited to address the
Suppes: I have some themes to
AERA meeting in 1964. The second is
consider, and I thought we could use
“The Uses of Computers in Education,”
two well-known articles of mine from
first published in Scientific American in
the 1960s to explore them. In these
1966. This was after Dick Atkinson and
articles I talk about what we should do,
I received a grant from the Carnegie
and what we are going to be able to do,
Corporation to establish a laboratory.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
32
Michael Allenʼs 2008 e-Learning Annual
Allen: Okay, now let’s back up a bit.
As we wrote it, we realized we could
Can you tell me what led you up to
give it an educational learning theory
this point?
“twist.” We came up with the idea of computer-based instruction. That’s
Suppes: Dick and I wrote a book
how it all started, actually: out of a
of experiments on learning theory
proposal for an automated laboratory
called Markov Learning Models for
from Carnegie. John Gardner was
Multi-Person Interactions, published
president of Carnegie’s non-profit
in 1960. In this book we anticipated
foundation and very supportive of us.
much of the experimental economics
In our first experiments in late 1962
going on now. We experimented on
and 1963, we brought elementary-
various formats such as two-person
school children to the labs at Stanford.
learning games. We developed what
We soon saw that this wouldn’t work,
we considered to be an automated
as the numbers of learners would in-
laboratory to run experiments initially
evitably be small. Our first step into a
in the area of the psychology of
school was to install a teletype, con-
learning. Carnegie was looking for
nected by telephone to our computer
good projects to support, and we were
at Stanford, in an elementary school
encouraged to send in a proposal.
ten miles from Stanford. That was
Figure 1. (Above) First demonstration of a logic program to third graders in an early CRT terminal, Stanford Lab, 1963 Figure 2. (Right) Fourth grader using teletype machine, Grant School, 1965. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Addressing Diversity in (e-) Learning
33
in 1964. We had to put it in a broom
Suppes: My background was really in
closet, since enrollment then was
physics. I had an undergraduate major
bulging. Students had to go in there to
in physics and was pressured into a
work on it one at a time.
specialization in meteorology during
Out of this first trial came the
World War II, where I served as a
realization that we had to move the
meteorologist. I received my bachelor’s
terminals, because we didn’t have PCs.
degree at the University of Chicago
We actually ran terminals from some
and a U.S. Army Commission as a 2nd
central point until the early 1980s,
Lieutenant at the same time, in 1943.
when PCs became available.
I was then immediately sent overseas,
In 1972, after having taught it
as they needed meteorologists in the
twenty-two years and having written
Pacific. I went to the Solomon Islands,
a textbook, I offered a completely
one of the more remote places on
radical idea: a computerized version
earth, and was even stationed at a
of my Introduction to Logic course.
relatively remote part of those islands.
I was only the master of ceremonies
After the war, I went on to take a Ph.D.
on the first day of class. The students
in philosophy at Columbia. Since then
worked entirely on the computer, and
my philosophy work has been mainly
the results were entirely evaluated by
on the philosophy of science and logic,
the computer program. There were
so in that way I have kept my scientific
no human lectures, only an assistant
boots on. Fairly early I started working
there to help with problems. I taught
in mathematical and experimental
that course every quarter for the next
psychology. It sort of became my
twenty years—sixty times—until I
scientific side, and I have used what
retired in 1992, at the age of 72. The
I have learned from this also in my
course, too, was retired at that time.
philosophy studies.
It was a big experiment. Similarly, from about 1955 I had been teaching
Allen: So, as you devised an e-
another course, Axiomatic Set Theory.
learning course based on your book,
It was a good undergraduate logic and
did you envision it simply as a labor-
math course at a more advanced level,
saving device? What was your goal
and starting in 1974 I taught it three
with the computer-based version of
times a year as a computerized course.
your course?
Allen: Now what was your
Suppes: Several things. We could
background?
turn to my 1966 article in Scientific
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
I C
34
Michael Allenʼs 2008 e-Learning Annual
American. It was about the uses of
Now, what follows from that is an
computers in education. I would stress
idealistic program that has been only
two things. What is striking, when
partly realized. We would like to have
teaching something like logic, is the
the interaction between student and
wonderful, vast individual differences
computer be like that of a very good
in the rate of student learning. Some
dialogue between a student and a
will take to it like ducks to water,
smart tutor. We have made progress
while others have problems. The
on such dialogues, but we have a long
individual differences are huge. The
way to go.
pace of learning varies tremendously across students. The single most
Allen: Before we talk more about the
important cognitive argument for
future. What is your assessment of
using computers, from an intellectual
what is happening today? What are we
standpoint, is the detailed and
doing well and not doing well?
deep accommodation to individual differences in student learning. The second associated point is that
Suppes: Let me mention a few things from the Scientific American article,
sitting and listening to lectures is a
and talk about where we are in regard
passive activity. I have a behavioral
to them. I mentioned being very
norm for the classroom: we teachers
active. That we do well. In the courses
talk too much. I have a rule on
I have been associated with, be it
the computer in I have a behavioral norm for elementary school. the classroom: we teachers No lecture should talk too much. be longer than three minutes. One and
teaching language arts to third graders or teaching Chinese to sophomores in college, we require a good amount of active participation on the part of the student: thinking creatively and
a half minutes is much better. Then
answering questions, not just listening.
you have active exercises to complete.
We have done a pretty good job with
This is even true in the logic course
this activity component.
in the university level. Great stress is
We have also done well regarding
on an active role for the student, not
diversity in learning. I have what I call
passive listening. Individualization for
the “cone of diversity.” Along the x-axis
learning and reducing the passive role
we plot how long a student spends
of the student are the major virtues of
working on the computer in a course,
computerized courses.
and along the y-axis we plot what the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Addressing Diversity in (e-) Learning
35
student has done thus far for each time period on the x-axis. In elementary school, for example, one would talk about grade placement as a progress indicator. At the university level one would talk about how much of a course has been completed. Now the fastest student’s progress would appear on the left. But this student’s curve couldn’t be vertical, because it takes some real time to do anything. The slowest student’s progress would appear almost horizontal, but it couldn’t be completely horizontal, because that
Figure 3. Sample
would be an infinite time. Thus, the
Computer-Time
fastest student adheres closely to
Trajectory
the y-axis, while the slowest student continues close to the x-axis. Everyone falls, in principle, somewhere between the “cone” formed by these two curves. The
sixth or seventh grade. And then you have students who will take longer. Probably my best example of
differences between the fastest and
taking individual differences into
slowest student can be, cumulatively,
consideration is my work in the
very large. For example, in my current
1970s and 1980s with deaf students
activities here at Stanford with the
learning to read. Of course, it is
Education Program for Gifted Youth,
very difficult for them with their
all pre-college, my colleagues and I
auditory language deficiencies. Some
have created a large computer-based
would take hundreds of exercises to
math course for grades 1 through 7.
make progress in elementary school
For these eight years of curriculum,
reading—an amazing amount of
the best students can easily finish
time—but they would make progress.
three years early. The very best are
We are talking about students
phenomenal, taking the Advanced
genetically deaf from birth, but of
Placement Exam in calculus in the
unaffected intelligence. We are talking
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
36
Michael Allenʼs 2008 e-Learning Annual
residential schools around the country.
without any instructional design
Individual differences can be a striking
background became involved, and
phenomenon, and I believe we did a
what these people were fascinated
good job accommodating their needs.
with was the medium’s ability to present information. They would say
Allen: Do you think this is true in the
that the individualization comes into
training environment as well as in
play because you can read it at your
education?
own speed. Well, using that logic, a book is pretty individualized, too.
Suppes: Well, as we both know, there is extensive use of computers to
Suppes: Well, that’s too simple of an
teach people in industry, but I think
idea of instruction!
the answer is no, and I’ll tell you why. I think there is too much tendency
Allen: Right!
in the training industry to lecture people and not enough to check up on
Suppes: In their favor, let me say this.
what they are understanding, or to let
The biggest course and the biggest
the ones advancing
enrollment we have at Stanford in the
I think there is too much whiz ahead, and give
Education Program for Gifted Youth
tendency in the training special attention to
is the K-7 math course. I have been
industry to lecture people those who may not
working on this course since 1992.
and not enough to check be progressing as
This was preceded by the writing of
up on what they are quickly. Most training understanding,... courses, but not all, are not sophisticated
a series of textbooks for K-7 math in the 1960s, followed then by additional computer coursework during the
enough to take into consideration
whole period from 1965 to 1992.
the individual differences involved
So I am saturated in the subject. I
in learning. Perhaps the main
have been constructing the course
reason is cost. It is expensive to
in detail. It’s a huge course, with
develop instruction that is highly
more than forty thousand exercises.
individualized.
Most corporations feel they cannot put that kind of time and money
Allen: I agree. As I began work in the
into perfecting instruction trying to
field, it was individual differences that
do something much simpler to save
fascinated me. But as the Internet
money.
came around, it seemed that people Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Addressing Diversity in (e-) Learning
37
Allen: But then I ask, “What is the
best and worst students may differ,
most expensive form of instruction?”
if you have a criterion of success in
And my answer is “instruction that
learning, by an order of magnitude.
doesn’t work.”
The curriculum has to be organized, and it’s not trivial to organize it to
Suppes: I agree with you. There aren’t
accomplish that.
many educational training groups in corporations that have been producing
Allen: And the benefit of adapting to
any sophisticated learning products
individual differences?
over fifteen to twenty years. There isn’t a culture of really accepting 100
Suppes: Well, like your remark about
percent what can be done with good
the worst thing being not to teach at
instructional design, even in places
all, the worst thing is to be wasting
like IBM.
time. Let me put it this way. I have always been concerned with the top
Allen: Any other comments on your
and bottom 10 percent of students.
assessment of where we are today?
The bottom 10 percent need help and time to get up to speed so they don’t
Suppes: Connected with the notion
fail. The top 10 percent need to be able
of individualization is the question of
to move on and learn more according
how to individualize. To individualize
to their ability. We need to take care
you need sophisticated, dynamic
of their readiness and abilities. The
concepts of curriculum. I have, for
ordinary thing to do when you don’t
example, created technical specs for
have the ability to individualize is to
a program to track student progress
teach to the middle; that’s what good
in the current K-7 math course. It
textbooks do necessarily.
is a complex document describing a
Now let me review a couple
complicated, stochastic motion. To
of things from the other article I
provide for individual differences
mentioned, “The Uses of Computers
requires, on the production side, a
in Education.” These things will
much more complicated conception of
sound familiar, but the problem is
the curriculum. You just can’t provide
to implement them in a curriculum.
for individual differences in some
One point concerns taking advantage
simple way, such as giving learners
of and understanding the nature of
one set of exercises, for example.
transfer. It is a well-studied subject in
The number of exercises done by the
educational psychology and learning
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
38
Michael Allenʼs 2008 e-Learning Annual
theory in general, but not really well
the program, because it’s something
understood, even by curriculum
you really can’t do operationally in
faculty in schools of education. Out
classrooms or in groups. I consider
of these studies come other learning
a criterion of learning based on
criteria such as response latency, the
latency a very important thing—and
time between a test stimulus and
a sophisticated thing—to add, from a
the response. Can a student work
learning standpoint. It’s as important
an exercise fast enough to show real
in language learning and many other
mastery? This is hard to measure in
subjects.
the classroom, but the computer is just tailor-made for it. You can measure
Allen: I remember your talking about
learner responses to the millisecond.
“Old Church Slavonic” years and
You don’t need this, of course, as
years ago.
one-tenth of a second is good enough for ordinary work. Everyone has
Suppes: Yes, I have a 1981 edited
recognized for a long time in learning
volume of the computer-assisted
theory, particularly in education,
instruction we did at Stanford for
that response latency is a much more
the university-level courses. We
sensitive measure of mastery, in many
offered a whole range of language
kinds of learning, than just correct
courses: Armenian, Arabic, Chinese,
responses.
French, German, Russian, and Old
For example, in many basic parts
Church Slavonic. I used to joke
of mathematics, you want students
about Armenian. Stanford wouldn’t
to have answers at their fingertips,
offer Armenian, because there were
so to speak. They should be able to
never enough students. We had, at
do computations with great accuracy,
most, about one student a year for
and also with speed, so they can move
Armenian. But we actually had people
ahead with a competent base. I have
who found it useful, and we received
math races in my K-7 math course. It
a special grant to develop courses.
is assumed the student can calculate
Scholars who were studying that part
the correct answer. The exercises are
of the world or the history of the
only meant to build more competency
revision or who wanted to read more
in terms of swiftness, consolidating
documents in that language needed to
the base of learning. It’s been very
learn Armenian.
successful, and I love that part of
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Addressing Diversity in (e-) Learning
39
Allen: You usually have to have
associated language arts courses at
a large population to justify the
the elementary level, instructors
development costs, but sometimes a
do not like trudging through stacks
very small population can justify them
of individual writing exercises.
just as well.
Everywhere, teachers are backing away from doing this. We are building
Suppes: It will not be an object of
production programs whereby
extensive experimentation. It may
students are given exercises in reading
not be the best possible course, but
comprehension, for example. We give
in the vein of that great line by Oscar
them a fixed vocabulary of no more
Wilde, “If a job’s worth doing, it’s
than thirty-five to forty words to use
worth doing badly.” And here, in the
in composing a sentence. Then we
case of Armenian, the student wanting
write a big context-free grammar,
to learn the language would prefer
formally, as a computer program. The
something that did the job, even
program can do a pretty good job of
though it hadn’t been given the loving
checking for grammaticality. We are
attention, say, of a course that had an
also building a semantic analysis of a
enrollment of a thousand students per
correct paraphrasing that can be used
year.
to assess answers to a question. It’s
I should that mention we are
quite successful already and we will, of
currently working on two language
course, improve it with experience. I
courses that are meant to be
can see why teachers are shying away
completely computer-based. One is
from manual assessments, for we can
a big ESL course, and we’re starting
easily accumulate twenty thousand
from scratch. The other is a mirror
written exercises with a sizable class.
image of this: Chinese. A teaching
We are also using this new
assistant is there to help out with
technology in both the ESL and the
problems. In both of these computer-
Chinese course. We have students
based courses, we do some rather
writing and also writing down what
sophisticated things that match
they listen to in this way. We can do
some earlier predictions of mine.
accurate evaluations with a restricted
Let me amplify this remark. One
vocabulary. We use, as I said, a very
of the things that is coming to
detailed and very large context-
fruition in these language courses is
free grammar. There are some good
sophisticated recognition and analysis
computer parsing programs available
of student-constructed text. In the
for such languages.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
40
Michael Allenʼs 2008 e-Learning Annual
Allen: Are you thinking about
restricted areas of instruction like
integration of voice recognition?
mathematics are a good environment in which to do it. You put your finger
Suppes: Absolutely! We haven’t done
on something I mentioned we needed
much successfully up to now, but we
in 1968 and thought we would have
have a lot of work going on with voice
in another ten years, by the way. I
recognition. One of my objectives,
think it is unlikely there are currently
when I finish some things I am
any big production courses anywhere
working on currently, is to implement
really using voice recognition in any
voice recognition on a production
major way, but it will come. It’s not too
basis. In the K-7 math course, we
difficult to predict future functionality,
have already implemented a test basis.
but timelines are very challenging,
I think this is very important, and
indeed.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Addressing Diversity in (e-) Learning REFERENCES Suppes, P. (1964). Modern learning theory and the elementary-school curriculum. American Educational Research Journal, 1, 79–93. Reprinted in H. C. Lindgren (Ed.), Readings in educational psychology. New York: John Wiley & Sons, 207–222. Reprinted also in R. Ripple (Ed.), Readings in learning and human abilities. New York: Harper & Row, 1971. Reprinted also in H.C. Lindgren & F. Lindgren (Eds.), Current readings in educational psychology (2nd ed.). New York: John Wiley & Sons, 1971, 216-230. Reprinted also in the Bobbs-Merrill Reprint Series in Psychology, P-810, Prod. No. 69065. Japanese translation in W. H. Holtzman (Ed.), Computer-assisted instruction, testing, and guidance. New York: Harper & Row, 1970. Suppes, P. (1966). The uses of computers in education. Scientific American, 215, 206–220. Reprinted in Information: A Scientific American book. San Francisco, CA: Freeman, 1966, 157–174. German translation: Anwendungen elektronischer Rechenanlagen in Unterricht. In Information Computer und Kunstliche Intelligenz. Frankfurt am Main: Umschau, 1967, 157–172. Reprinted also in Mathematical thinking in behavioral sciences: Readings from Scientific American. San Francisco, CA, and London: Freeman, 1968, 213–222. Russian translation in Informatsia. Moscow: Mir, 1968, 165–182. Japanese translation in Information, Scientific American book, 1969. Polish translation: Zastosowania maszyn cyfrowych w nauczaniu-tlum. Tadeusz Wiewiorowski. In Dzis i jutro maszyn cyfrowych. Warsaw: 1969, 231–256. Reprinted also in Computers and computation: Readings from Scientific American. San Francisco, CA: Freeman, 1971, 249–259. Reprinted also in Contemporary psychology: Readings from Scientific American. San Francisco, CA: Freeman, 1971, 257–267. Suppes, P. (Ed.) (1981). University-level computer-assisted instruction at Stanford: 1968-1980. Palo Alto, CA: Stanford University, Institute for Mathematical Studies in the Social Sciences. Suppes, P., & Atkinson, R. (1960). Markov learning models for multi-person situations. Palo Alto, CA: Stanford University. For a full bibliography and PDFs of Patrick Suppes’ articles on e-learning, see http://suppes-corpus.stanford.edu.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
41
Donald L. Bitzer Donald L. Bitzer received his Ph.D. in electrical engineering from the University of Illinois in 1960. He was professor of electrical and computer engineering at the University of Illinois from 1960 to 1989, when he retired from the University of Illinois to become a Distinguished University Research Professor in the Computer Science Department at North Carolina State University. Bitzer’s work has involved applying signal processing and coding theory to a variety of areas, from radar signals and speech processing to the development of software and hardware required for large computer networks. The large educational computer systems “PLATO” and “NovaNet” are results of this research. His research led to the intelligent modems for telephone lines and cable systems as well as the flat plasma display panel now being used for television. He has been granted numerous patents in the computer and electronic areas. Recently his research has focused on using signal processing and coding theory to look for genomic information that controls the translation process in protein production. He is also involved in developing new methods for convolutional coding to improve communications in the presence of noise.
PLATO, Plasma Screens, and Computer-Based Education
Donald L. Bitzer
In conversation with Michael W. Allen I was so fortunate to direct research and development on advanced instructional systems for Control Data and work with the University of Illinois on the PLATO system—an invention of Donald Bitzer and his colleagues at the Control Systems Laboratory. Don’s limitless energy and enthusiasm led him to tackle challenges most people couldn’t even conceptualize, and he inspired many of us, as he continues to do today at North Carolina State University, to take on meaningful challenges, not just tasks that are obviously doable. Professor Bitzer was, as always, gracious and enthusiastic as I interviewed him about his life with PLATO, his hopes, and his evaluation of learning technology today. He continues to quickly connect multidisciplinary knowledge and focus on potential solutions to major societal needs. As a partner in such pursuits with the late William C. Norris, CEO of Control Data Corporation, and a recipient of major federal funding, Bitzer developed learning technology that was a fantasy playground for educators. Knowing that educators limited their vision to what was currently possible and therefore had difficulty imaging, let alone specifying what they needed from interactive technologies, he set out to create the most boundless system for exploration. Even today, some of the capabilities of PLATO have yet to be matched by other systems. Sagacious as he always is, you can’t talk with Professor Bitzer without his raising your sense of importance and obligation to do something significant with your life. It was such a privilege to interview him about what drew him to develop technology for learning, what he feels were the most important things he learned from the experience, and where he feels we need to turn our attention today. —Michael W. Allen Allen: With a background in electrical
without failing. That was a landmark. It
engineering and minors in math and
encouraged us to look into instructional
physics, what attracted you to using
uses. The University of Illinois,
computers as a learning tool?
under the prodding of Shal Sherwin, a professor of physics, set up a fairly
Bitzer: The real reason was that
large committee of scholars, teachers,
computers had started to become
computer engineers, and education
reliable enough to run for forty hours
experts to look into the possibilities.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
44
Michael Allenʼs 2008 e-Learning Annual
They met for several months to
us would end up supporting them in
determine whether computers could
increasingly unsustainable ratios.
be used effectively for learning. I was
Although our funding came from
never invited; I had just finished my
the military, we were free to use the
Ph.D. and was working independently
money on what we wanted as long as
on electronic components directly
we submitted annual status reports.
related to such things.
They would look at what we did the
At the end of the discussions, it
year before and determine whether we
was decided that nothing could be
were to receive the next year’s funding.
done. Daniel Alpert, the head of the
Dan Alpert said, “Let’s take a gamble.
Control Systems Laboratory (CSL),
I think we should do this no matter
was told of the findings, and he wrote
what.” I agreed. We put a little money
a letter to the dean, but then asked
into it.
me to read through it to see whether
At the next meeting of the advisory
I agreed. I read it and said, “Are you
committee, we submitted a report
kidding? Give me a week, and we’ll do
about what we were doing. They were
something useful!”
ecstatic. They said, “We are the largest
So he held the letter up and within a
educational institution in the world,
week or so, we actually had something
and this is something we need as
going. Alpert never sent the letter
part of our military systems.” That, of
and told me, “Well, maybe they can’t
course, encouraged all of us. In a year
do anything, but we can. Get going!”
or so, our PLATO (Programmed Logic
With the help of a colleague, Peter
for Automated Teaching Operations)
Bromfeld, to do some coding and a
project took over the whole lab!
technician half-time, we soon had a useful system. We were interested from the beginning in the use of technology in classroom learning because it seemed to me the majority of our large inner city schools were turning out functionally illiterate students– students who couldn’t really contribute to the workforce in any significant
Figure 1. PLATO III Classroom,
way. That was scary. Such a trend
University of Illinois at Urbana-Champaign,
would take us all down, as the rest of
Sept 1968
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
Again, what drove me
45
in possibilities to synthesize a vision.
fundamentally was trying to solve
The Illiac computer system was, as
some of the problems facing our large
you know, one of the first digital
inner city schools. Even today, the
computers—developed in the 1950s at
majority application of the PLATO
the University of Illinois. Well, as years
system is to keep students in school
went on, more, smarter people, with
and raise their abilities. That is, of
different backgrounds and opinions,
course, where the money is, too, since
were brought in, and they helped grow
schools are paid for keeping students
it into something wonderful. What was
in school at the elementary and high-
needed to get started, however, was a
school levels. One hundred to two
single person with all the right skills in
hundred thousand students each year
place and the single vision in mind.
are brought back and kept in school using this technology.
Allen: Wikipedia says, “Donald Bitzer’s success comes in large
Allen: Why do you think the
part from his rejection of modern
University’s committee said learning
educational thinking.” How do you feel
with computers wouldn’t work?
about this?
Bitzer: I have my own opinion about
Bitzer: What does that mean?
this. The committee members were a
I’ve read quite a bit about my own
mixture of people who weren’t able to
history and found most of it to be
see their common needs as educators
bogus, but benign, so I have just
and the possibilities of technology.
left it alone. True, there are a lot
Each viewed the topic in terms of
modern educational ideas I disagree
his or her own area of study. It was
with: for example, that content
a multidisciplinary meeting, not an
is unimportant, only the delivery
interdisciplinary meeting. They just
matters. That’s nonsense. You are not
never got it together, and they were
going to teach something like math
sure we wouldn’t get it together either
or physics if the teacher doesn’t know
(although they all wanted to be a part
anything about the subjects but is
of it after the fact). I was a professor
very good at presenting slides. That’s
and a computer guy, so I could
not enough. In the early days they
consider the problem from multiple
would say, “Teaching is the medium”
perspectives. I knew how to use the
(implying whatever you teach with
Illiac system we had and could factor
is the teaching, and content is not as
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
46
Michael Allenʼs 2008 e-Learning Annual
important. That is, the medium itself
would come in to see whether things
is the teaching). I never understood
we were working on could be of use to
that. I disagreed with it. I spent zero
them.
time worrying about it. I don’t worry about the things Wikipedia says
Allen: What did they do with your
about me. Well, they didn’t say I was
findings?
an idiot; that would have been more provocative.
Bitzer: Most of the accomplishments that came out of CSL were used in
Allen: You had funding from the
a significant way in the military.
Army/Navy/Air Force group. Did
They ended up having better results
they exert direction or pressure or
by letting us do our thing. A good
guidance, or did they just sit back
example of this is the Electrostatic
and watch what you were doing?
Vacuum Gyro, commonly called the “Star in a Bottle.” It was used for long-
Bitzer: They gave money, sat back,
range guidance for submarines. It was
and watched. That was part of our
a gyroscope with a year’s precession
program. Being a university lab, they
on it or more, precession being the
looked at what we did after we spent
drift of the gyroscope axis. So there
the money. Knowing they could cut
was basically no drift, no torque
the funding for things they didn’t like
suspended in a vacuum. They are still
was a kind of pressure, I suppose.
the standards in submarines. The
But as far as I know, they always
same was true for side-looking radar
came through. Now, when they had
(counting the engines on airplanes
a positive attitude for things they
from fifty miles, even though the
really liked, they would really work
antenna you were using was actually
hard to get additional money from
quite small). We solved these problems
other agencies, as they did for the
because we were interested in them.
educational projects. If they weren’t
These things are still in effect today
really into a project, they had a way
for military use. We did contribute to
of subtly discouraging you from doing
their ability to carry out their mission,
it. But they never once came in and
even though that was not our charge.
said, “Here is what we want you to do.” Basically, we were working on more
Allen: Now these weren’t part of the
general, long-range problems, and they
PLATO Project, were they?
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
47
Bitzer: Same lab, but different
They were into to psychological
projects. There was too much to do to
studies (topics Congress was a little
be working only on one project. That
disturbed with anyway, for example,
was a nice thing about the lab—there
“Does pornography
were bright, creative people working
affect the soldier?”).
They were into... topics
on a variety of things and interacting
They were furious
Congress was a
across projects. We were formal
then when this
little disturbed with.
groups, but we worked together
project came
informally, helping each other, sharing
around. But just then the Mansfield
people and resources. As long as
Amendment came along, which
we were all contributing to moving
required a disclosure on what ARPA
forward . . . that was the important
was working on and its applicability in
thing.
civilian life. Our project was the only thing they had, so we became their
Allen: How about the learning technology. Did the military find useful things there?
hero. The answer is that, all along the way, including the idea of using PLATO for military instruction was
Bitzer: Yes. The government was very
supported by them, but we didn’t write
interested in our finding the “ten most
any of their materials. They wrote
important principles of computer-
their own courseware for medics or
based education.” Many parts of
pilots, and so on. They would later
PLATO were developed by other
contract our help.
people. I was an instructor and taught
In many ways we don’t know what
in the traditional classroom. An ARPA
all they did from an educational
contract to investigate CBE was signed
standpoint. I know they did Chinook
while I was in India. I was a little
vehicle maintenance and medic
upset, but decided I would take part of
training down in Texas. I understand
the money and design a nice model for
the FAA still uses applications you
teaching by computer for the students,
developed at Control Data, Michael.
and take the rest and develop a better,
Control Data Corporation (CDC)
more interactive student display. I
actually took over a lot of service
did that. This money came from the
to the government once you got
behavioral science branch of ARPA
going there. I know more about the
(Advanced Research Projects Agency).
systems that are used in education for
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
48
Michael Allenʼs 2008 e-Learning Annual
bringing kids back. I see that every
Allen: Would these new agencies
day. I see ten thousand students, K-12,
ever give you directives or specific
signing on for basic instruction in
goals?
mathematics, English, etc. Bitzer: No, we would write proposals Allen: Later funding came from the
for what we wanted to do. We only
national Science Foundation (NSF),
fought once, and that was over
right?
evaluation techniques—formative versus summative. Sometimes I
Bitzer: Yes, eventually we received
disagreed on the statistics. Statistics
funding from several places, including
lie, you know. The average heart is
the University itself and the Tri-
in terrific shape, but everyone dies
Service (Army, Navy, and Air Force),
from heart failure. Even there it was
CSL contracts (we had to get our
no real disagreement. NSF supported
own). NSF took an interest and
involvement of the Educational Testing
supported the first real system with
Service to provide evaluation, which
many terminals (twenty to thirty)
was fine with us. But it wasn’t clear
and the cable systems that supported
that ETS was prepared to do good
the distribution. The basis of the
evaluations of this new thing. It’s really
project, I should say, was centered
hard to evaluate new projects and
around the distribution of computer
technologies, because you aren’t really
display signals by cable television to
sure what measurement techniques
a university classroom, a high school
to use or what variables to monitor.
classroom, a nursing school, and a
Measurement of any new process is
community college. More support
inherently challenging. The effects of
came from them and ARPA when we
observation, for example, such as the
started expanding the use of plasma
Hawthorne Effect discovered at GE,
display panels. Various agencies
can invalidate results.
became big supporters after a few
ETS did the job, however. The
years. That would happen often in our
typical result was that teaching on
laboratory, even with other projects.
the computer yielded equal or slightly
After Tri-Service money, particular
better results than teaching in the
agencies would take an interest and
classroom. I would contend if we can
become the primary funder.
do as well as in the classroom, that’s a good start! I mean, you can start with this success and work for sustained
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
49
improvement easier than you can with
successfully answered the question
a classroom.
of how to use learning technology
It all worked out fine. We were all
to greatest advantage, but you
friendly in the end. The government’s
couldn’t have done better to provide a
support was taken over by CDC, who
powerful system with unprecedented
provided expensive equipment, ideas,
capabilities.
software, and courseware. We really had a great relationship with CDC. As you know, we jointly developed
Bitzer: Well, that was my intent. Different people tried different things,
a system with CDC to integrate
and many turned out to contribute
simultaneously developed system
in ways that we alone could not have
software enhancements. Every week
done. People you would never have
we shipped our software changes,
thought of as contributing turned
which had been running for three
out to do so in some way. There was
weeks without error, back and forth
an English teacher for rhetoric, for
to each other. We had about eight
example, a nun named Pauline. Now
hundred terminals on our system,
I thought it would be difficult to
and CDC had about two hundred on
judge rhetoric using a computer, but
your end. It was a remarkable system
she came up with a way. “One way
that allowed rapid development of
of learning to write well,” she said,
advanced capabilities and yet remained
“is to learn to edit carefully. And we
stable for on-going instructional use.
can easily judge editing quality with a computer, even though we can’t judge
Allen: It was amazing how effective
open-ended writing. And for teaching,
the code integration system was.
that will work just as well.”
System capabilities advanced
Then a classic Latin teacher, one of
rapidly, by leaps and bounds, while
the best course writers we had, came
learners continued using courseware
up with a way to teach Latin and the
uninterrupted. Many capabilities
classics in a new way. He could handle
became more sophisticated than
four times more students, teach them
many we have on the Internet today.
more, and get them up to snuff so he
I always viewed what you were
could later teach them the fancy stuff.
doing as taking away functional
Now, I wouldn’t have thought to break
constraints so we educators could
it up the way he did, but he did it! The
explore learning possibilities with
PLATO system allowed creative people
few boundaries. I’m not sure we have
to use their own ideas in a way they
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
50
Michael Allenʼs 2008 e-Learning Annual
would have been restricted from doing
or sixty PLATO terminals in the
with a more rigid system. The whole
classroom, and the teacher could still
was much greater than a sum of the
manage the class as usual. In many
parts, and that’s what made PLATO
ways it was much better. Now when
the success it was.
any system is evaluated as being on par with traditional teaching in the classroom, I consider that a major success, but most don’t see it that way. The computer goes on and its instructional quality is consistent and also subject to instant improvement, but I would say classroom teaching quality is inconsistent. These days are showing how low we can go.
Figure 2. PLATO III system, Washington School, 1970.
e-Learning is successful. Students come back. The overall measurement shows it bringing students back. It has to or the education system can’t pay
Allen: I also read on Wikipedia that,
for it. No one is supporting or funding
“An official evaluation by an external
it now. It is walking on its own legs.
testing agency ended with [the
The whole economic aspect depends
conclusion that though] everyone
on its success rate.
enjoyed using it [PLATO] . . . it was essentially equal to an average
Allen: My hope—my expectation,
human teacher in terms of student
actually—was that a healthy
advancement.” Some saw this as a
market competition would arise.
failure, didn’t they?
It would benefit the learner by forcing organizations to improve
Bitzer: Yes. Obviously they have never
the quality of courseware or lose
done computer-based education.
out to competitors. But I’ve been
There are some things we do
discouraged by the really pathetic
electronically that exceed what a
courseware on most systems. It’s
human can do. We can run a phone
made me wonder if competitive
switching system better than humans,
economic influences will actually start
for example. The PLATO system
to engage.
reduced entropy. You could put fifty Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
51
Bitzer: I think they have in some
Bitzer: I was an optimist. I said,
ways. The problem is the judgment of
“We are going to solve problems, put
what’s good and what isn’t. You can’t
terminals in every home. Things are
expect accurate judgments, unless
really going to work out well.”
educators are very good—unless they
Well, it didn’t happen that fast, but
were outstanding in the classroom
what has happened is as good as I
to begin with. You don’t see a lot of
wanted, just not in size. We do have
advertising of vintage courseware
terminals in every home. I was right
compared to new stuff that is coming
about that, but I didn’t realize how
out. The new stuff is making the
appetizing game distractions and
same mistakes we made long ago
other applications were going to be.
and soaking up resources as it fails.
I shouldn’t complain too much, as
Fortunately, PLATO activities have
one of these distractions, namely TV
expanded in orders of magnitude.
terminals, has been one of my biggest
It all has to be paid for and
successes.
measured against the criterion of
The growth factor for our
whether it is doing the job where
instructional applications is the
the regular system is not. The
most disappointing. Part of that is
measurement has to include the
because of marketing. Remember
dropout rate. Most schools get their
you are marketing against established
funding inversely based on their
institutions of learning, where a lot of
dropout rates. The money they are
protections are built in. They protect
receiving from reduced attrition more
themselves from change in fear they
than pays for the system. In fact,
might be phased out.
it pays for it and other educational
The educational system is one of the
functions as well. I’m in different
hardest things to change. The fact that
pursuits these days, but what I see is
it has been as successful as it has, in
retention rates of 80 to 90 percent. If
light of this attitude of protectionism,
you measure it in any reasonable way,
is pretty amazing in itself. But I really
it’s a success.
wondered when technology-assisted learning took off, how could they turn
Allen: What was your early vision
it down? At some point there would
for what technology could do for
surely be a decree for schools to do
learning?
the job. Then they wouldn’t be able to get around the advantages technology offers.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
52
Michael Allenʼs 2008 e-Learning Annual
I think it will happen. If schools
problems that have to be solved, they
really get on board with “No Child
break down and do it right. We can’t
Left Behind,” they won’t be able to turn
provide effective adult education
e-learning down, and we might see
without having the flexibility to make
something happen. If schools really get on board There is, however, with “No Child Left Behind,” enough inertia in the they won’t be able to current system to turn e-learning down. keep it static. People work hard so they
adults feel comfortable and learn at their own pace and place. To get into the big marketplace, with fifty million or so students, it’s going to take a huge change in attitudes on the part of school
don’t have to worry too much if they’re
districts and everyone else involved.
doing the wrong things.
This is a difficult change. Educational
Nothing is as entrenched as the
leaders are big in their own way,
K-12 education system, and it’s
even if they’re failing. It’s a problem
failing. We are ready with solutions;
I didn’t realize we had—the kind
the educators just need to not be
generated internally. School districts
threatened and embrace what is going
are more interested in politics than
to work. Computer-based systems are
in education. It’s sad, and it shows
expandable, without expanding the
up in the students. Those students
entropy. We are ready.
who come from schools with the right attitude are scoring as well as
Allen: Adult learning opportunities
those from overseas. When they come
may be easier to sell because people
from schools that don’t have needed
are looking for flexible ways to find
values, you almost have to start over
better jobs and further their careers.
with them. I certainly underestimated
Technology-assisted learning has
how to deal with the educational
flexibility of delivery and the ability to
system, but I’m not the only one. The
accommodate individual differences.
President of the United States and the
Do you think it will work backwards
superintendents of school districts
into the schools from success with
have also underestimated how hard it
adults?
is to change the K-12 schools. We have a large number of
Bitzer: Yes, absolutely. We are
distractions in the form of people
going to have niche applications
ready to make money on learning
rather than adoption by the general
technology without actually being
school population. When places have
involved in teaching anything. I can
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
53
teach X-Y-Z. Just take your computer
fundamentals of learning, you’d be
and put this software on it. It will solve
hard pressed to say yes. Games—yes.
your problems. They are beginning
Science, math, etc.—no. They will
to sell a large number of people
not be good contributors to our
on it; it will fail and discourage
culture if they sit around spending
people and schools from employing
time on games. They are not learning
learning technologies. Schools today
what they need to
don’t really use it properly—email,
learn to be effective
They are not learning
billboards, or course outlines. There
contributors these
what they need to learn
is actually very little instruction done
days.
to be effective contributors
online. I wanted to make a system that
these days. Allen: What advice
taught rather than distracted from
would you give to those trying to
teaching. I used to tell the school
advance today’s e-learning industry?
districts that. They used to say, “We are going to get thirty terminals.
Bitzer: Well, to start, “e-learning”
Aren’t we important?” I would say,
is too broad a term. I would say we
“No. It depends on what you do with
have to concentrate on the results
them.” If you have students simply
in the important areas such as math
gathering at terminals, rather than
and science. If you can write an
learning math, English, or science, we
interaction and prove that students
have failed them. The school districts
are learning math, physics principles,
didn’t like to hear that, because they
or editing skills better, then that’s
didn’t know how to do this.
an advancement. I think everyone should be focused on where they
Allen: The younger generation is so
want to head with results. We need
computer savvy, easily using games,
to be result-oriented, have flexibility
music, video, and instant messaging
in the approach, and support the best
skills. Maybe they will create a
designers and developers. Make sure
pressure for change from the bottom
you are working on the right problem.
up.
It’s a learning literacy problem, not a computer literacy problem that we
Bitzer: You are right, but the kind
have to solve.
of pressure it creates is on having the systems available. If you ask
Allen: Any thoughts on training in
yourself, is it helping them with the
corporate environments?
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
54
Michael Allenʼs 2008 e-Learning Annual
Bitzer: In a sense, the challenges
start, if you have a way of measuring
are the same. Anything that helps
quickly and accurately to understand
employees learn the company’s
what the problem is and generating
business, such as how to use
new material to solve that problem
equipment or follow the rules of the
by the computer, then you can adapt
company, would be valuable. But if
as you go along and guarantee that
I were a company, I would think the
students will learn the material in
best investment would be to make sure
a relatively quick fashion. I think
the least of my employees could read,
that’s the kind of applications we
write, and do basic math. That is what
need. When students leave, they have
makes a good employee.
learned the material, and you know it. Tiffany Barnes was one of my
Allen: As an engineer, do you see
students who earned her Ph.D. in
any new inventions or technologies
this field. She is now a professor of
coming along that are going to open
computer science at the University
up new opportunities we should be
of North Carolina at Charlotte. She
looking at?
did a scientific study of how to apply science to fault-tolerant teaching. She
Bitzer: From the hardware perspective
has other projects underway now such
we have more than enough. We just
as how to involve learning games in an
need to get the software sorted out.
effective way. She was one of the best
We have tons of memory we never
students I had in this area.
had before, which allows us to use it
Tiffany did a beautiful analysis of
properly or misuse it. One can develop
how the computer could automatically
measurement and learning strategies
extract the major concepts being
that are scientifically based. You can
taught in the lesson. By using
gather lots of data, process it quickly,
computer-generated questions,
see what concepts are working, and
the computer could determine
then build a scientific, mathematical
which concepts the student did
model of what you want to do and
not understand. Different remedial
make measurements of how well you
strategies were possible. For instance,
are achieving it. By putting feedback
it provided help for those students
and evaluation loops into the system,
who didn’t seem to get the major
you can adapt your processes to what
concepts and different exercises for
I like to call “fault-tolerant” teaching.
those who almost seemed to get it,
Even if the material is weak at the
but not quite. It adapted to learners by
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
55
making the measurement accurately,
process. The goal is to make any
even at the concept level, going back
protein quickly, with high yield, using
and correcting misunderstandings to
E coli as the material. For instance, if
bring students along. It worked very
you have some special medication you
well. You can use the advances in
need to fight a particular disease, even
technology available to us these days
though it might be a human protein,
for improving the tools you are using
we can have E coli make that protein
in measurement and delivery of these
instantly and in large quantities. The
ideas for students.
results have been very encouraging. I keep busy, and it’s a lot of fun.
Allen: What are you working on now?
Unfortunately, there is a lag time on this new technology, and the new
Bitzer: I have been working on
medications will not be ready for me
lip-synch processing and speaker
fast enough. I’m getting old, and the
identification by extracting and
two of us won’t converge.
determining what shape the mouth was in. This work has helped with
Allen: Don, thank you so much for
animated figure generation. We have
your time. I am thrilled that you will
also looked at how to break up speech
be a part of this Annual. You have
and recognize a particular speaker,
given so many of us unprecedented
even when that speaker is trying to
opportunities to improve education
disguise his or her voice.
and training.
Currently, I’m working more intensely on genetics—applying single
Bitzer: You are too kind, Mike.
processing theory for extracting
Thank you.
hidden signals in the translation
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
S e c t i o n Tw o
Learning Te c h n o l o g y Ta k e s S h a p e
Fred T. Hofstetter Fred T. Hofstetter is professor of education at the University of Delaware. A specialist in multimedia, he developed the PODIUM multimedia application generator, the GUIDO Ear-Training Lessons, and the Serf Web-based teaching and learning environment. He has received many grants and awards and does consulting work for computer firms. He has published widely and has given lectures and workshops on multimedia and Internetworking in many locations in Europe, Africa, Australia, Canada, the United States, and the Pacific Rim. Hofstetter currently authors four IT textbooks for McGrawHill, including those on the topics of Internet Literacy, Multimedia Literacy, Advanced Web Design, and Internet Technologies at Work. His recent work has been in aligning the School of Education’s educational technology graduate programs with the AECT-SMETS and ISTE-TF standards to create a program assessment plan that is used in the University of Delaware’s NCATE accreditation process. He also chairs several doctoral committees for candidates who are using data-driven strategies to differentiate instruction and improve results in Delaware public school districts.
Back to the Future of Educational Technology
Fred T. Hofstetter
In this article, the author begins by sharing his early computer-assisted instruction experiences in the field of music. He then goes on to discuss the hopes for the larger field of CAI, the risks encountered, what has been learned about learning and the effective use of technology for education, and the implications for the further development of online learning. In the 1970s, we developed e-learning
altered our perception of how the
materials on computer terminals
network should be used in the design of
connected to an expensive mainframe
effective learning environments. This
that delivered courseware to students.
article chronicles the technological
To save costs in the 1980s, we
context in which the principles evolved,
moved these materials to standalone
reflects on lessons learned from thirty-
microcomputers and thereby lost
five years of practice, and discusses
connectivity. This was not a great loss
implications for further work.
for students, however, because most of us had been using the communication features to manage the development of
Where Did We Begin?
My work with educational
the courseware rather than to engage
technology began
students in dialogue online. The science
when I was hired as
of learning had only just begun to
a music instructor
create the design principles that would
at the University of Delaware in 1973.
guide effective use of the network. In
My assignment was to teach ear-
the 1990s the popularity of the World
training, which is a demanding two-
Wide Web created the critical mass
year course that every music student
that led to the Internet becoming the
must pass in order to remain enrolled
global information utility that powers
in a college music program. The
the 21st Century. Having regained the
problem with teaching ear-training is
connectivity we had lost, we are kind
the wide range of individual differences
of back to the future. As it always does,
among college music students. Some
however, the future changes before
students are naturally gifted at rhythm,
it gets here. In this case, the science
while others are more talented in pitch
of learning evolved principles that
perception. Singers with beautifully
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
60
Michael Allenʼs 2008 e-Learning Annual
resonant voices may stumble when
Graded Units for Interactive
trying to vocalize a notated melody.
Dictation Operations GUIDO stands
Some students who excel at melodic
for Graded Units for Interactive
intervals confuse them when the
Dictation Operations. Following
intervals combine to form chords and
Bloom’s mastery learning model,
harmonies.
GUIDO presented a graded series of musical dictation units in the
Bloom’s Mastery Learning Model
domains of intervals, melodies, chord
At the time I was facing this dilemma,
qualities, harmonies, and rhythms. In
Bloom’s (1968) learning for mastery
each domain, the student began with
(LFM) model was coming into vogue.
the first unit and practiced at that
According to this model, students
level until attaining the score needed
progress through graded units of
to advance to the next unit. Bright
instruction and are not permitted to
students progressed quickly through
go to the next level until mastering
the units, while slower students got
the current level. A meta-analysis
extra practice. Because the program
conducted by Kulik, Kulik, and
was new and I did not know exactly
Bangert-Downs (1990) found that LFM
how to sequence the units or how
increases test results by 0.59 standard
difficult to make them, I based
deviations across a broad range of
GUIDO on a table-driven design in
subjects including mathematics,
which all of the instructional variables
reading, science, and social studies.
were put in a table that the instructor
As Saettler (2004) notes, Bloom’s
could edit. Through this table, the
mastery learning model does not
instructor could make the course more
work with all forms of instruction, but
or less difficult, adjust the content and
it is highly applicable if the subject
sequencing of the units, or even create
domain is algorithmic, hierarchical,
new units of instruction.
or procedural. Because music is highly procedural, I decided to design
Analysis of Student
a mastery learning system for ear-
Response Matrices Through
training. I named the system GUIDO
controlled experiments, we found that
in honor of Guido d’Arezzo, the 11th
students who used GUIDO scored
Century monk who invented the
significantly higher on ear-training
musical staff.
examinations than students in control
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
groups using traditional methods of instruction. A series of articles
61
What Were Our Hopes?
Compared to computers today,
published in the Journal of Research
technology was
in Music Education (Hofstetter 1975,
in a primitive
1978, 1979, 1980) documented the
state in the
learning gains. These articles further
1970s. I began my work with GUIDO,
described how GUIDO kept response
for example, on Tektronix display
matrices of all the questions asked and
terminals that were not selectively
responses given. By analyzing these
erasable. If you wanted to erase
matrices, we discovered perceptual
anything onscreen, you had to erase
patterns that changed the thinking
the entire display. Imagine designing
about music theoretical concepts
e-learning modules under such a
that had been misunderstood for
constraint!
hundreds of years. We found melodic and harmonic situations in which
Acquiring a Delivery System
students were not just getting answers
We needed a better platform, so
wrong, but were also agreeing on what
we explored alternative systems.
the wrong answer was. The music
The National Science Foundation
community was highly interested in
had invested heavily in two high-
this research, and I was invited to
profile projects called PLATO and
present my work at many colleges
TICCIT. The PLATO terminals had
and universities. In 1975, Dean
plasma panels that were selectively
Helen Gouldner provided a startup
erasable and transparent. You could
grant that enabled us to begin the
rear project microfiche slides onto
National Consortium for Computer-
the plasma panel, upon which the
Based Music Instruction (Hofstetter,
computer overlaid text and computer
1976), which continues today as the
graphics. PLATO also had a computer-
Association for Technology in Music
controlled random-access audio device
Instruction (ATMI). The significance
that could play audio clips from a
of this work helped me earn tenure
15-inch magnetic disk. In order to
at UD, where I continue to be a
obtain the speed of access needed to
faculty member at one of the finest
display slides and begin audio playback
universities in the world.
promptly, the microfiche and audio devices were powered by compressed
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
62
Michael Allenʼs 2008 e-Learning Annual
air. Even though the technology was
powered PLATO. In 1976, CDC
primitive, we felt PLATO provided
acquired the rights to market PLATO.
more versatility than TICCIT, which
Our goal became achieving the critical
relied on a central bank of computer-
mass of interest needed for UD to
controlled videotape players that was
purchase a PLATO mainframe.
staff intensive and not very portable. PLATO was expensive, however.
Achieving Critical Mass My friend,
The terminals pictured in Figure 1
colleague, and mentor L. Leon
cost $6,000 each and needed to be
Campbell, who was UD’s provost
connected via leased telephone lines
throughout this period, explained
to the PLATO mainframe at the
that, in order to justify the purchase
University of Illinois, where PLATO
of the multi million-dollar mainframe,
was invented. The long-distance
I would have to show widespread
communication costs were prohibitive.
faculty interest in using PLATO
The only way for us to bring down the
across the campus. Thus began the
cost was to develop the critical mass
Delaware PLATO Project, which
of interest needed for UD to be able
lasted for fifteen years from 1974 to
to justify purchasing and installing its
1989. I became the Delaware apostle
own PLATO mainframe. Control Data
of PLATO and gave demonstrations
Corporation (CDC) manufactured
for faculty all across campus. Based
the Cyber mainframe computer that
on effective results from pilot projects in thirty-six academic departments, we were able to convince Provost Campbell to purchase a PLATO mainframe, which was installed in the UD computing center in 1978. Our work with PLATO is documented in a series of Summative Reports in the ERIC database.
Figure 1. The author’s sons Dan (standing)
Moving Beyond Programmed
and Tom (seated) use a PLATO plasma display
Instruction When I reflect on the
terminal in 1979. In the background, a student is touching the screen to place a musical note onto the staff of a GUIDO melodic dictation exercise.
broad range of applications we developed, I realize how pigeonholed the modern view of the classic PLATO system is. I say pigeonholed because
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
63
most contemporary researchers
Ramping Up In order to raise
who write about PLATO use it as an
money to help UD pay these costs,
example of programmed instruction.
we began selling PLATO services to
Lockee, Moore, and Burton (2004, p.
corporations, government agencies,
563), for example, refer to PLATO as
and school districts. Most notable
“the most prolific and long-standing
among the corporations was DuPont,
example of computer-based PI.” They
with whom we partnered to create
are correct in the sense that PLATO
a laboratory technician training
stands for Programmed Logic for
program. In government, our
Automatic Teaching Operations,
largest customer was the Federal
which was how most people used it.
Aviation Administration (FAA),
PLATO was capable of much more,
which used our PLATO system to
however, and we were using it for
teach flight inspection procedures,
more than behaviorally oriented
radar, communications equipment,
PI. We had the PLATO system, for
and electronics. In our local school
example, doing career counseling,
district, which was undergoing
demography, and psychology
desegregation, we used an Emergency
experiments. Our engineering college
School Aid Act (ESAA) grant in
even programmed a model of the
1979 to install PLATO terminals
Delaware Bay that could track the
with the goal of reducing minority
movement of oil slicks in the event of
group isolation by providing basic
a tanker spill.
skills instruction to improve student achievement, enhance student self-
What Were the Risks?
In the 1970s,
concepts, and foster more positive
attempting
interracial interactions. Under a grant
a large-scale
from Control Data in 1981, we were
computer-assisted
funded to create the Lower Division
instruction
Engineering Curriculum (LDEC).
project was filled with risks. The hardware was expensive, and we
Developing Effective Courseware
needed to raise the money to pay
One of the greatest risks in creating
for it. Budget-hungry deans wanted
a computer-aided instruction (CAI)
academic results. If we developed
project in the 1970s was that there
ineffective materials, the deans would
would not be enough quality software
argue that our budget should be spent
available to sustain the project. As
elsewhere.
Nievergelt (1980, p. 11) expressed it,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
64
Michael Allenʼs 2008 e-Learning Annual
“Today it makes no sense to start a
as reasons to reduce our funding.
CAI project unless one is willing to
Figure 2 shows the courseware
write most of the necessary software.”
development process we created
Because courseware development
(Hofstetter, 1981). It contains a variety
was expensive and time-consuming,
of feedback loops, whereby we used
we needed to make sure the
techniques of formative evaluation to
materials we produced hit the mark.
help ensure that the courseware we
Academic deans were watching us,
were developing would be effective.
and they wanted evidence that our
In a contemporary implementation of
materials were effective. We needed a
this concept, Allen (2006, p. 73) refers
mechanism for developing courseware
to this iterative process as successive
that worked from the start. Budget-
approximation.
hungry deans would use any failures
Figure 2. The Delaware Model for Courseware Development Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
65
What Would Learning Technology Mean for the Student?
Scaling Back in Recession In the 1980s, the Reagan tax cut led to an economic recession, and UD began to scrutinize every aspect of its budget. So did DuPont and the FAA, which
The loss of networking was not as devastating then as it would be today because
sought lower-cost alternatives to the
most of us had been using the
expensive mainframe connections.
communication features to manage
Reading the handwriting on the wall,
the development of the courseware
and not wanting to lead UD into a
instead of involve students in online
budget shortfall, I led the effort to
course discussions. The pioneering
convert our most successful PLATO
work of the great Russian psychologist
programs into a microcomputer
Lev Semyonovitch Vygotsky had not
format. At first, we converted
yet made it into the mainstream of
software to micro-PLATO, using
American educational technology.
microcomputers invented by Control
We did not know that the most
Data Corporation. In 1985, we began
important use of the network would
using IBM PCs programmed in
be to scaffold students and provide
TenCORE, which was a standalone
customized coaching when they
version of PLATO’s TUTOR
encountered difficulty. Vygotsky’s
programming language.
work was just being translated into
In 1989, we uninstalled the PLATO
English. Because we did not yet know
mainframe when the Philadelphia
it, we did not realize what we were
Prisons decided to discontinue its
losing when we lost connectivity. In
twenty-six terminal subscriptions and
other words, losing the network was
begin running on IBM PCs instead.
not such a great loss in 1989, because
While we saved cost and avoided
we did not yet know what to do with
a budget shortfall by moving our
it.
programs from the PLATO mainframe to less expensive microcomputers,
Skinner’s Linear Behaviorism
we lost the network. No longer could
American psychology was dominated
instructors communicate with each
by the behaviorists. Chief among them
other through PLATO notesfiles or
was Skinner (1938, 1953), who saw
press “term-TALK” to initiate an
that human behavior is powerfully
online chat.
shaped by its consequences. Moreover,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
66
Michael Allenʼs 2008 e-Learning Annual
Skinner felt that psychology was
without any connectivity. Toward the
essentially about behavior and that
end of his career, Skinner himself did
behavior was largely determined by its
not foresee the vital role computer
outcomes. Instructional designs were
networking would play in education.
based on a stimulus-response (SR)
As late as 1986, Skinner maintained
chain in which positive reinforcement
that the microcomputer is “the ideal
was provided when students answered
hardware for programmed instruction”
correctly. Such designs had no need
and proposed that when used for
for computer networking because all
computer-aided instruction it should
of the feedback was pre-programmed.
be called a “teaching machine” instead
It would be several decades before
of a computer (Skinner, 1986, p. 110).
American educational technologists would recognize the vital role of
Vygotsky’s Triangular
networking in education. As late as
Constructivism Vygotsky (1978)
1989, for example, we received an
provided the missing link by
EDUCOM/NCRIPTAL Best Tutorial
transforming the linear SR model
award for an Introduction to Statistics
of the behaviorists into a triangle
course that did not contain any
that represents how people learn
communication features, neither for
through an extrinsic process in which
instructors to coach students nor for
knowledge is mediated by student
students to help one another. The
interaction with tools, community, and
IBM PC version of GUIDO won a
understandings acquired through prior
Joe Wyatt Challenge award in 1991
learning. Figure 3 shows how Vygotsky drew this triangle by adding to the stimulus-response chain a third node called X, which stands for extrinsic. This extrinsic node is bidirectional, meaning that students can reverse the action of the SR chain by interacting
Figure 3. Vygotsky’s
with tools, instructors, experts,
Interaction
peers, knowledge bases, multimedia, datasets, and professional associations.
Triangle 1
S is the stimulus, R is the response, and X is extrinsic mediation through which students learn by interacting with objects in their environment including tools, instructors, experts, peers, knowledge bases, multimedia, datasets, and professional associations. 1
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
67
Since the learner is portrayed as
the instructor provides graduated
an active processor who explores,
prompting according to the student’s
discovers, reflects, and constructs
needs. At first, the instructor gives
knowledge, the trend to teach from
general help. If the student still
this perspective is known as the
stumbles, the instructor gives more
constructivist movement in education.
specific guidance. By making the student’s thinking visible, the dialogue
Zone of Proximal Development
provides a record of the coaching that
Vygotsky (1978) observed that when
both student and instructor can study
students are learning, inevitably
and reflect on how to improve. Today’s
they stray into a zone in which the
behaviorists consider scaffolding to be
difficulty of the problem they are
an antecedent that plays a vital role in
working on exceeds what they can
invoking the desired response; thus,
accomplish independently. Because the
through scaffolding, behaviorists have
zone is between the student’s current
discovered some common ground with
level of development and the next
Vygotzky and his followers.
level that could be achieved through
Figure 4 is an activity diagram
an expert’s guidance, Vygotsky called
that illustrates the manner in which
it the Zone of Proximal Development
I believe e-learning revolves around
(ZPD).
the zone. Following Engeström (1987),
Online learning creates the zone
the top three nodes of tools, subject,
whenever students submit assignments
and assessment represent mediated
that fail to meet the criteria for a
activity through which the student
high grade. Instead of assigning a
learns to perform the outcome of the
low grade and moving on, I believe
instructional goals. When the student
we have a responsibility to coach the
cannot accomplish this independently,
students and help them achieve the
the instructor provides coaching in
objective. To create such a learner-
the zone, which is represented by
centered environment, I invented a
the bottom inner triangle. The outer
web-based e-learning system called
triangle has a node representing
Serf (Hofstetter 1997, 1999, 2006),
community on the bottom left,
which has a feature called “give the
and professional organizations on
student another chance.” When
the bottom right. The instructor is
the instructor clicks this option, a
responsible for creating curriculum
scaffolding protocol creates a just-
that leads to certification as defined
in-time discussion forum in which
by the appropriate standards body.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
68
Michael Allenʼs 2008 e-Learning Annual
Figure 4. Activities Around the Zone in a Socially Constructed e-Learning Environment
The instructor’s role is bidirectional
three primary principles that have
in that the instructor can also provide
emerged from the science of learning.
recommendations to the professional
These principles are:
organization regarding refinements or additions that may be needed to make
1. People learn by connecting new
the standards meet community needs.
information to concepts already learned.
Have We Learned from the Science of Learning?
At the turn of
2. To learn how to reason, solve
the century,
problems, and augment knowledge
the National
in a field of inquiry, people need to
Research
understand facts and ideas in the
Council (2000)
context of a conceptual framework
published a landmark book entitled
that facilitates application to real-
How People Learn. It is a wonderfully
world problem solving.
written synthesis of developments in
3. People are motivated to learn
the science of learning. A fascinating
when they can set their own goals,
chapter on brain research (Chapter 5)
reflect on their progress, and feel
describes how learning makes physical
in control of their learning.
changes that reorganize the brain. The manner in which these connections
From these principles, it follows that
happen to rewire the brain reinforces
instructional designs should:
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
1. Take into account the learner’s
69
I believe it is important to mention
preexisting understandings and
why it cannot. Because people learn
correct any faulty preconceptions
by connecting new information
in order to prevent future
to concepts already learned, the
misunderstandings;
order in which students encounter
2. Enable students to study multiple
new material is vitally important.
examples of the concept at work
Curriculum planning matters.
in order to learn it in depth in
Students need a course of instruction
authentic contexts; and
that presents materials in a logically
3. Include metacognitive supports
appropriate order with enough real-
that make visible the learner’s
world examples to cover the concepts
reflections and enable an
in depth.
instructor to provide scaffolding and guide revisions to improve
Fitting Course Goals to
student learning and reasoning.
Student Goals In my online courses, I have evolved a process whereby
Learning From Versus
the students buy in to the course
Learning With Hill, Wiley, Miller-
content by making its goals be their
Nelson, and Han (2004) make the
own. Early in the course, I give an
distinction between learning from
assignment in which the students
versus learning with the Internet.
tell me why they enrolled and what
When learning from, the student
they hope to get out of the course.
proceeds with guidance provided
A dialogic protocol enables me to
according to protocols determined
conduct with individual students a
by the instructional design. When
little conversation in which I explain
learning with, on the other hand,
that my purpose is going to be to help
students use the Internet as a tool.
them to master their goals. Most of
Hill and his colleagues reviewed
the students provide a statement of
several studies that examined whether
competencies they hope to develop
people can learn from the Internet
as the course progresses. I respond
without designed instruction. The
to these statements by explaining
overwhelming answer from these
the manner in which the course will
studies is no. Although it amazes me
address the student goals. Later in the
to think that people would expect the
course, I use these goal statements
Internet by itself to produce results,
to help the students propose term
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
70
Michael Allenʼs 2008 e-Learning Annual
paper topics and multimedia projects
about it, normally you do not log on
through which students demonstrate
unless you are in a good mood. Thus,
the extent to which they have met the
e-learning enables faculty to teach,
course objectives. The assignment
and students to learn, when they are
protocol enables me to hold an online
in the mood for it. Negotiating goals
conversation that makes each student’s
when both student and instructor
thinking visible and records it into
are in a good mood leads to more
the course database for subsequent
ambitious projects with a higher
analysis and reflection.
quality of scaffolding
Holmberg (2003) views the didactic nature of distance education as a
Situated Cognition Situated
teaching-learning conversation,
cognition is a term coined by Brown,
in which the teacher bonds with
Collins, and Duguid (1989) to describe
a student by creating empathy.
the kind of learning that happens in
According to Holmberg, it is not only
environments designed to put students
the frequency but also the quality
in real-world contexts. Technology
of communication that is important
plays an important role in enabling
in creating this bond. In the online
learners to study concepts in authentic
environment, I believe one of the
contexts. Through interactive video,
reasons why e-learning is effective is
students can be put into real-life
because people are doing it when they
situations. A film clip of the Tacoma
are in the mood to do so. If you think
Narrows bridge collapse, for example,
Figure 5. A PODIUM Videodisc Overlay for the
Figure 6. Experimenting with Wind Speeds and
Puzzle of the Tacoma Narrows Bridge Collapse
Pulsing Actions to Find the Combination That Created the Standing Wave That Destroyed the Tacoma Narrows Bridge
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
71
has become a classic example used by physics instructors to situate the teaching of wave motion and resonance. Under a grant from the National Science Foundation, Fuller, Zollman, and Campbell (1982) created a videodisc that included the film clip along with a slide bank and full motion videos of the bridge construction, its
Figure 7. The Author Works with Interactive Videodisc in 1988
geographical setting, mathematical formulas, the influence of the wind,
point at which the focus now is on
and standing wave experiments. I
providing students with multimedia
created a program called PODIUM
computers to discover knowledge in
that let the faculty show instantly any
real-world contexts. At the Internet
slide, motion sequence, or experiment
Plasma Physics Education Experience
on the Tacoma Narrows videodisc. By
(IPPEX) Online, for example, students
simply touching the screen of the IBM
can run the same nuclear reactor
Infowindow display shown in Figure
simulation that Princeton University
5, the faculty could show students the
scientists use to determine optimal
historical background and illustrate
settings for tokamak fusion reactors.
the geophysical conditions that led to
By manipulating the sliders that
the catastrophe. Figure 6 shows how
control the plasma density, heating
the faculty could experiment with
power, and magnetic field, students
different wind speeds and pulsing
can explore how these parameters
actions to recreate the standing waves
interact and develop an intuitive feel
that led to the bridge collapse.
for the process scientists go through
Figure 7 shows the assortment
in designing tokamak reactors. The
of equipment used to make
IPPEX simulation is online at ippex.
presentations back then. Multimedia
pppl.gov/tokamak/tokamak.htm.
was so expensive that you could not realistically think about giving each
Learning and Transfer As Bruning,
student a computer. Instead, the focus
Schraw, & Ronning (1995, p. 216)
was on equipping each classroom
explain, “The aim of teaching, from
with a workstation that the teacher
a constructivist perspective, is not
could use in making presentations.
so much to transmit information,
Happily, costs have dropped to the
but rather to encourage knowledge
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
72
Michael Allenʼs 2008 e-Learning Annual
formation and development of
(1998) found that, in eighth grade
metacognitive processes for judging,
mathematics, using the computer
organizing, and acquiring new
for drill and practice is negatively
information.” Rumelhart (1981),
related to student results (-.59 grade
following Piaget, introduced the
levels), while using the computer for
notion of schemata, which are mental
simulations and applications increases
frameworks for comprehension that
results (+.42 grade levels). Too much
function as scaffolding for organizing
drill results in overly contextualized
experience. At first, the instructor
learning and reduces time available for
provides instructional scaffolding that
problem solving and application.
helps the student construct knowledge. Gradually, the instructor provides less
Increasing Motivation According to
scaffolding until the student is able to
Allen (2003), motivation is the most
construct knowledge independently.
important factor in achieving success
In How People Learn, the National
from e-learning. Allen’s law states
Research Council (2000, p. 53)
that e = m 2ci, where m is motivation,
identifies four key transfer principles
c is content, i is interaction, and e is
that govern this process:
e-learning outcomes. If motivation
1. Initial learning is necessary for
is lacking, the m in Allen’s law will
transfer. 2. Knowledge that is overly contextualized can reduce transfer. 3. Transfer is best viewed as an active, dynamic process. 4. All new learning involves transfer based on previous learning.
be zero, and therefore e will be zero, meaning that no learning will occur, regardless of the quality of the content and its interactivity. Social opportunity is important in motivating people to learn (National Research Council, 2000). To create social motivation, I have created a
Instructional designers need to take
gallery feature, which is an option
these principles into account when
the instructor can configure for any
choosing among behavioral, cognitive,
Serf assignment. If the instructor
and social constructivist approaches.
clicks to activate the gallery, students
Too much drill, for example, can cause
can see each other’s submissions.
negative results. In an Educational
Students learn a lot by viewing each
Testing Service study of the 1996
other’s work, which they discuss
National Assessment of Educational
in the online forum. Depending on
Progress (NAEP) database, Wenglinsky
the nature of the assignment, the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
73
instructor can configure the gallery
substantial progress in making e-
to reveal the student names or keep
learning more effective. These three
them anonymous, and display the
areas are:
instructor’s feedback or keep it
1. Reducing transactional distance,
private. In my web design courses, I
2. Making assessment metacognitive,
have a “cool tool” assignment in which I have the students (1) identify the
and 3. Creating communities of learners.
tool they consider most useful and (2) write an essay explaining why they
Reducing Transactional
think it’s cool. Through the gallery,
Distance As Vygotsky (1978) notes,
students explore each other’s tools and
learning inevitably creates a zone in
make discoveries richer than anything
which students encounter problems
I could design on my own.
they cannot solve on their own. At this
The innate human desire to
point, the student needs help from
develop competence is another factor
the instructor. In Serf, I have created
that is important in motivating
a scaffolding protocol that enables the
people to learn (National Research
instructor to identify the zone, provide
Council, 2000). To create competence
just-in-time coaching, and give the
motivation, I let students choose from
student another try. By repeating this
a selection of labs that fit a variety of
process until the student masters the
school or workplace settings. Students
assignment, educators can produce
invariably choose to do the labs that
better prepared students who have
are perceived as aligning with their
truly learned the material.
career paths or helping accomplish tasks in their workplaces.
The problem I have noticed in my current application of the zone, however, is that students do not
What Are Implications for
know that I have provided them
Further Work? In setting future
with feedback until the next time
directions, educational technologists
they log on. This creates the kind of
must consider the extent to which
gap that Moore (1993) refers to as
technology has emerged for addressing
transactional distance. The more time
the problems we see in our current
it takes the student to receive help
work. I believe there are three areas
needed from the instructor, the larger
in which technology has evolved
is the gap of transactional distance.
to the point at which we can make
It is a psychological gap created
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
74
Michael Allenʼs 2008 e-Learning Annual
by communication latency, not a
I plan to create a short messaging
geographical gap caused by physical
service (SMS) to which students can
distance.
subscribe if they would like to be
When I am grading final projects,
notified of course events via TXT
for example, I often send e-mail to
messages that Serf will send to their
notify students that I have found
cell phones.
problems in their submissions and
Eventually, the cell phone interface
have provided feedback along with a
could be extended to TXT a message
chance to resubmit their assignments
whenever someone writes a response
for a higher grade. Needless to say,
in a discussion to which the user has
students place a high value upon being
subscribed. I believe such a cell phone
given a chance to make revisions
interface could substantially increase
and earn a higher grade. It is time-
the amount of interaction in a course,
consuming, however, for the instructor
especially if the student could TXT
to send these e-mails. I plan to
a reply back to the online discussion
automate this process by creating an
forum.
option whereby students can subscribe to a notification service that will
Making Assessment
send them an e-mail automatically
Metacognitive One of the most
each time the instructor leaves them
important things we do in education
feedback on an assignment. This
is help students learn to reflect
e-mail will include the name of the
on whether their current level
assignment, the instructor’s message,
of understanding is adequate. By
and a link to click to go to that point
making the student’s thinking visible,
in the online course. In like manner,
metacognitive tools enable students
I plan to create an e-mail service
to reflect on their progress toward
whereby the instructor can choose
learning to think like an expert.
to be notified (or not) each time a
Through conversational assignment
student responds to the scaffolding.
protocols that record the dialogue
Electronic mail may not suffice,
between students and instructors,
however, to reduce transactional
an e-learning environment can make
distance for the younger generation
student thinking visible. By posing
of students who prefer to use TXT
questions that make students reflect
messaging on cell phones instead of e-
on whether their current level of
mail on personal computers. To reach
understanding is adequate, the
students who live on their cell phones,
instructor can help students learn
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
75
to be aware of the progress they are
to which the student has met each
making toward understanding.
standard. By requiring the students to
In my college-level courses, for
have an expert review their artifacts,
example, students propose and
include the expert’s comments in the
negotiate the topics of their projects
portfolio, and respond to the expert
and term papers. Through an online
by explaining what they plan to do
consultation protocol, I help students
to follow up on the expert’s advice,
create a project that not only satisfies
I believe we can make the student’s
their interests, but also meets national
thinking visible in the context of
standards in their chosen career fields.
professional standards. By making
The dialogue I have with my students
assessment metacognitive in this
is recorded in the course database
context, we can prepare students to
and can be viewed at any time on the
maintain this awareness throughout
course assignment page. I encourage
their careers
students to reflect on this dialogue, think about their progress toward
Building Communities of
meeting the standards, and become
Learners Reflecting on his invention
actively involved in setting their
of the virtual high school (VHS)
learning priorities.
concept, Tinker concludes that
The next assignment dialogue I plan
“Without collaboration, the social
to create is an ePortfolio protocol in
value of networking is lost and online
which students will document and
courses become simply extensions of
reflect on the extent to which they
existing course formats” (2005, p. 413).
meet professional standards in their
Riel (2005) identifies three overlapping
field. Instead of having students use
ways in which online education should
canned software that creates the
be community based: (1) collaborative
portfolio for them, I believe we should
learning in the context of a student
have students learn to use industry
cadre, (2) theoretical learning through
standard web authoring software
community experiences, and (3)
to create their own portfolios and
transformational learning in one’s
thereby acquire strategic 21st Century
community of practice. In Serf, I
multimedia authoring skills. The
have created a community-building
portfolio assignment protocol I plan
component called the affinity cluster.
to create will prompt the students
Instructors who own an affinity
to provide a link to the spot in their
cluster have the power to create one
portfolio that demonstrates the extent
or more affinity groups, each of which
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
76
Michael Allenʼs 2008 e-Learning Annual
can consist of one or more threaded
Conclusion
In the book How
discussion forums, document sharing
People Learn,
libraries, and electronic magazines
the chapter
(a.k.a. newsletters). A role-based
on technology
permissions model enables the
concludes by
instructor to assign privileges to end-
stating that “Much remains to be
users, who can be given read, write,
learned about using technology’s
moderator, or administrator access to
potential: to make this happen,
one or more affinity groups.
learning research will need to become
Affinity groups can have different
the constant companion of software
kinds of purposes and targeted
development” (National Research
audiences. Some groups may be
Council, 2000, p. 230). I personally
intended for experienced users
believe that e-learning will evolve
to participate in multithreaded
to the point at which the computer
discussions, for example, while other
becomes transparent. When that
groups may be intended for novices
happens, we should remove the
who need a simple menu-driven
hyphen from the term e-learning. As
discussion forum. To categorize
Chute (2003) reminds us, Webster’s
affinity groups according to different
unabridged dictionary defines the
purposes, the instructor creates
prefix e as meaning thoroughly, as in
an organizational entity called a
the word evaporize, which means to
“community” and assigns to it affinity
vaporize thoroughly. Thus, we may
groups that share the community’s
define the term elearning as meaning
goals and objectives.
to learn thoroughly, which we will
I believe a logical next step in the
achieve by applying design principles
development of the communities
from the science of learning to create
would be to create a wiki-style
effective multimedia teaching and
knowledge building environment
learning environments.
(KBE). This would help answer
This is what I have been attempting
Romiszowski’s (2005) criticism that, in
since I began working with educational
spite of what is known about creating
technology thirty-five years ago. It
KBEs, IMS vendors have done little
is hard to predict how many decades
to build these kinds of cooperative
remain for me personally to work
learning protocols into their products.
in this exciting field. In 2004, I was
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
77
honored to receive a pioneering award
One of the bodies objects that it is not
for creating the first web-based course
dead yet. Neither am I done yet. God
at the University of Delaware back
willing, I hope to spend a few more
in 1997. In my acceptance speech,
decades inventing, teaching, learning,
I recounted a scene from the movie
mentoring, contributing, and sharing
Monty Python and the Holy Grail in
as we work to get to the point at which
which the Dead Collector carts off to
our field can remove the hyphen from
burial bodies dead from the plague.
e-learning.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
78
Michael Allenʼs 2008 e-Learning Annual REFERENCES Allen, M.W. (2003). Michael Allen’s guide to e-learning. Hoboken, NJ: John Wiley & Sons. Allen, M.W. (2006). Creating successful e-learning: A rapid system for getting it right first time, every time. San Francisco, CA: Pfeiffer. Bloom, B.S. (1968, May). Mastery learning. In Evaluation Comment, 1(1). Los Angeles: University of California at Los Angeles, Center for the Study of Evaluation of Instructional Programs. Brown, J.S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 28, 32-42. Bruning, R.H., Schraw, G.J., & Ronning, R.R. (1995). Cognitive psychology and instruction. Englewood Cliffs, NJ: Merrill/Prentice-Hall. Chute, A.G. (2003). From teletraining to e-learning and knowledge management. In M.G. Moore & W.G. Anderson (Eds.), Handbook of distance education (pp. 297-313). Mahwah, NJ: Lawrence Erlbaum Associates. Engeström, Y. (1987). Learning by expanding: An activity-theoretical approach to developmental research. Helsinki, Finland: Orienta-Konultit. Fuller, R., Zollman, D., & Campbell, T. (1982). The puzzle of the Tacoma Narrows bridge collapse (Videodisc). Hoboken, NJ: Wiley Educational Software. Hill, J.R., Wiley, D., Miller-Nelson, L., & Han, S. (2004). Exploring research on internet-based learning: From infrastructure to interactions. In D.H. Jonassen (Ed.), Handbook of research on educational communications and technology (2nd ed.) (pp. 433-460). Mahwah, NJ: Lawrence Erlbaum Associates. Hofstetter, F.T. (1975, May). GUIDO: An interactive computer-based system for improvement of instruction and research in ear-training. Journal of Computer-Based Instruction, III, 100-106. Hofstetter, F.T. (1976). Foundation, organization, and purpose of the National Consortium for Computer-Based Musical Instruction. Journal of ComputerBased Instruction, 3, 21-33. Hofstetter, F.T. (1978). Computer-based recognition of perceptual patterns in harmonic dictation operations. Journal of Research in Music Education, 26, 111-119. Hofstetter, F.T. (1979). Controlled evaluation of a competency-based approach to teaching aural interval identification. Journal of Research in Music Education, 27, 201¬213.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
Hofstetter, F.T. (1980). Computer-based recognition of perceptual patterns in chord quality dictation operations. Journal of Research in Music Education, 28(2), 83-91. Hofstetter, F.T. (1981). A model for administering computer-based education. In B. Lewis and D. Tagg (Eds.), Computers and education. Amsterdam: NorthHolland Publishing Company. Hofstetter, F.T. (1997). Serf ’s up! Teaching and learning with Serf: Your servant on the internet. In Virtual learning environments: AACRAO Policy Summit (pp. 123-135). Washington, DC: American Association of Collegiate Registrars and Admissions Officers. Hofstetter, F.T. (1999, Fall). Three waves of Serf, a web-based teaching and learning environment. College & university media review, pp. 99-123. Available: http://www.udel.edu/fth/serf/serf1-3.html. Hofstetter, F.T. (2006). Serf 4.0. Newark, DE: Serfsoft Corporation. Available: http://serfsoft.com. Holmberg, B. (2003). A theory of distance education based on empathy. In M.G. Moore & W.G. Anderson (Eds.), Handbook of distance education (pp. 79-86). Mahwah, NJ: Lawrence Erlbaum Associates. Kulik, C.C., Kulik, J.A., & Bangert-Downs, R.L. (1990, Summer). Effectiveness of mastery learning programs: A meta-analysis. Review of Educational Research, 60(2), 265–299. Lockee, B., Moore, D., & Burton, J. (2004). Foundations of programmed instruction. In D.H. Jonassen (Ed.), Handbook of research on educational communications and technology (2nd ed.) (pp. 545-569). Mahwah, NJ: Lawrence Erlbaum Associates. Moore, M.G. (1993). Theory of transactional distance. In D. Keegan (Ed.), Theoretical principles of distance education (pp. 22-38). London: Routledge. National Research Council. (2000). How people learn (expanded edition edited by J.D. Bransford, A.L. Brown, & R.R. Cocking). Washington, DC: National Academy Press. Nievergelt, J. (1980, September). A pragmatic introduction to courseware design. IEEE Computer, pp. 7-21. Riel, M. (2005). Building communities of learners online. In G. Kearsley (Ed.), Online learning: Personal reflections on the transformation of education (pp. 309-320). Englewood Cliffs, NJ: Educational Technology Publications.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
79
80
Michael Allenʼs 2008 e-Learning Annual
Romiszowski, A.J. (2005). Online learning: Are we on the right track? In G. Kearsley (Ed.), Online learning: Personal reflections on the transformation of education (pp. 321-349). Englewood Cliffs, NJ: Educational Technology Publications. Rumelhart, D.E. (1981). Schemata: The building blocks of cognition. In J.T. Guthrie (Ed.), Comprehension and teaching: Research reviews (pp. 3-26). Newark, DE: International Reading Association. Saettler, P. (2004). The evolution of American educational technology. Greenwich, CT: Information Age Publishing. Skinner, B.F. (1938). The behavior of organisms. New York: Appleton-CenturyCrofts. Skinner, B.F. (1953). Science and human behavior. New York: Macmillan. Skinner, B.F (1986, October). Programmed instruction revisited. Phi Delta Kappan, 68(2), 103-110. Tinker, R. (2005). Learning through online collaboration. In G. Kearsley, (Ed.), Online learning: Personal reflections on the transformation of education (pp. 402-414). Englewood Cliffs, NJ: Educational Technology Publications. Vygotsky, L.S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. Wenglinsky, H. (1998, September). Does it compute? The relationship between educational technology and student achievement in mathematics. Princeton, NJ: Educational Testing Service (ETS) Policy Information Center. Retrieved 22 December 2006 from http://www.ets.org/Media/Research/pdf/ PICTECHNOLOG.pdf.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Peter G. Fairweather An elementary school teacher beset with questions, Peter G. Fairweather earned a doctorate in reading and language (psycholinguistics) while puzzling about the processes people use to understand text and what happens as these processes evolve or break down. While exploring models of text comprehension, he served on the faculties of Northwestern University and the University of Texas, extending this thread of research to include cognitive simulation. Seduced by more available resources in industry, he held a number of posts in learning technologies companies, working on device and process simulations with both commercial and academic application. He has also designed interactive elementary and middle school material offered by several publishers. He is the co-author of Computer-Based Instruction: Design and Development, a popular instructional design text that by now both he and his co-author have vigorously repudiated. He has published over fifty papers in the areas of learning technologies, assistive technologies, and accessibility and has been awarded several patents in these areas. Currently, he works with a team on the problem of how to affordably and efficiently develop simulations that infer individuals’ learning status and then adapt appropriately.
It Seemed Like a Good Idea at the Time
Peter G. Fairweather
The Development of Computer-Assisted Instruction in Schools In this article, the author explores the development of computer-aided instruction in schools. Although educators and instructional designers had high hopes for instructional technology, they’ve been disappointed thus far. Various factors converged to force CAI to focus narrowly on particular items or objectives, even though research exists to show this is not the most effective way to teach. The author closes by reviewing a few recent, more successful programs. The midwives of the technology
were first made using the same
revolution in education that began in
vocabulary used for what teachers did.
the late 1960s saw a future of computers
These included:
working as tireless classroom teaching
ÿ Optimal calibration of practice
assistants. For their nascent field,
items to a student’s level of skill,
they adopted the label “computer-
ÿ Determination of the number of
assisted instruction” (e.g., Suppes &
items to be assigned based on the
Morningstar, 1969, 1970), emphasizing
learner’s performance history,
their goal to support the pedagogy and curriculum of the school*. Almost
ÿ Management of the student’s progress through the curriculum,
immediately, however, instructional
ÿ Provision of feedback to the student,
technology and the nurturing context
ÿ Generation of reports about each
into which it was born began to evolve on a path toward failure.
student’s performance, ÿ Determination of optimal limits on the time allowed for practice, and
What We Thought Instructional Technology Could Do
Because
ÿ Maximization of time-on-task.
classroom activities have
Indeed, if a teacher were able to do
always been
these things regularly, at that point in
directed by
history we would have thought that
teachers, it was not surprising that the
teacher to be creditably skilled and
promises of educational technologies
likely to be welcomed into any school
*The focus of this chapter on technology in schools should not be interpreted to mean that schools alone nurtured its development. Other organizations, particularly those involved with training, developed forwardlooking applications in areas ranging from medicine to language learning.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
84
Michael Allenʼs 2008 e-Learning Annual
district. However, the evolution of then-current models of instruction, learning, and even education
ÿ Means for learners to manage their instruction, and ÿ Opportunities for groups of
technology itself combined to
learners to build reflective and
undermine the vision of the pioneers
critical collaborations around
of computer-based instruction.
problem-solving activities.
Early Curriculum: Problems Give Way to Items
The earliest
In contrast, around the same time,
developers of
Patrick Suppes and his co-workers
computer-based
(Suppes & Morningstar, 1969, 1970;
educational
Suppes, Fletcher, & Zanotti, 1976) at
materials
Stanford’s Institute for Mathematical
organized instruction in a variety of
Studies in the Social Sciences
ways. For example, Ludwig Braun and
conceived a different structure
his colleagues used the computer to
altogether for computer-based
interactively manipulate quantitative
instruction—one that dominated the
representations of such phenomena
field for nearly twenty years. Rather
as the near extinction of the buffalo,
than using the computer’s ability to
fluctuations of a population of gypsy
simulate physical, natural, or social
moths, or efforts to manage pollutants
phenomena and providing a means
in a body of water (Braun, 1971a,
for learners to interact with them, the
1971b). This approach to instruction
Stanford team sought to represent the
based on interactive simulations
learner as a traveler on a path through
extended the model-based batch mode
a measured curricular space. Suppes
games used in business schools at the
and his colleagues represented the
time. Indeed, this method anticipated
content and processes of computer-
some of the advantages claimed today
based learning as:
by developers of problem-based
ÿ A set of task items arranged
curricula, including:
in order of difficulty within an
ÿ Authentic problem contexts,
academic subject, sort of a Russian
ÿ Rich, high-fidelity computational
egg that might comprise “division
models supporting instruction, ÿ Reactive problem surfaces allowing learners to deal directly with the
of fractions” within “numerical computation” within “arithmetic” within “mathematics,” for example;
consequences of their decisions,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
ÿ A trajectory through a curriculum
85
was able to provide enough correct
that is composed of these sets of
answers to a set of problems, each of
tasks;
which was described by a horizontally
ÿ An optimization function that
formatted subtraction problem where
advanced learners along their
the minuend is composed of two
prescribed trajectories according
digits each greater than zero and the
to their error rate and, sometimes,
subtrahend is composed of a single
time-on-task;
digit whose value is greater than the
ÿ A set of tasks arranged to
rightmost digit of the subtrahend,
minimize the differences between
he or she was deemed to have a skill
items at one level within a
that might be labeled “subtraction
sequence and those at the next;
with borrowing involving a two-
ÿ A curriculum based on detailed
digit minuend and a single digit
descriptions of the items to be
subtrahend.” While we might admire
presented to learners;
the emphasis on performance-based
ÿ A set of responses associated with
assessment, the definition of particular
each item that the computer could
skills made by identifying them with
judge correct or incorrect;
item descriptions brought about a
ÿ A variably controllable feedback
reductionism that would contribute
mechanism to reinforce or
to the rejection of these curricula by
punish the performance of the
schools.
learners and to provide them with assistance, if needed; and ÿ Different modes of presentation
Later investigators would show that even the simplest subtraction items could evoke any of several different
for items including assessment,
cognitive solution strategies in
practice, tutorial explanation, or
learners (Resnick, Carpenter, Moser, &
demonstration.
Romberg, 1982). Careful chronometric analyses of learners’ solutions showed
Detailed item descriptions
that the skills they applied to problems
identified the corpus of skills to be
sharing a common description could
learned in any segment of curriculum.
differ strongly from one another.
In effect, to have learned a skill was
(Instruction delivered by intelligent
defined as the ability to respond
tutors, such as the PUMP Algebra
correctly at some criterion level
curriculum [Koedinger, Anderson,
to a type of item with a specific
Hadley, & Mark, 1997] try to model
description. For example, if a learner
the mechanisms of problem solving
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
86
Michael Allenʼs 2008 e-Learning Annual
rather than organizing themselves
and Jostens Learning sought ways to
around problem descriptions. It should
span entire “subject areas.” All of them
not be surprising then, that such
developed efficient reusable driver
tutors are the only forms of computer-
programs that operated on sets of data
based instruction in mathematics that
representing the subject being taught.
have proved themselves consistently
While this allowed for the delivery
and significantly effective without
of a huge collection of “items,” each
strong teacher direction).
of them exhibited an unfortunate sameness that, if nothing else, led
That Nasty Problem of “Coverage”
Early computer-
students to expect problem formats,
based
procedures, and types of responses.
instruction
More insidious was the reduction
in schools
of the curriculum into a set of items
concentrated
that could be answered with a single
on arithmetic and language usage
simple response. While multi-stage
(Suppes & Morningstar, 1970). While
problems requiring the evaluation
later developers produced media-
of complex solution strategies
rich thought-provoking examples
can be presented interactively by
in biology, physics, chemistry, and
using only simple easily evaluated
social studies, these tended to address
responses, commercial developers
such narrow curricular segments
usually failed to design to that goal.
that schools could not justify their
Moreover, during the early 1970s, the
purchase. A teacher friend of mine
curriculum recommendations made
expressed his pet criticism as follows:
by organizations such as the National
I can spend $150 on that wonderful
Council of Teachers of Mathematics
program about Brownian motion.
or the National Council of Teachers
The kids love it and learn a lot
of English did not emphasize such
running the simulation over and over.
skills as problem finding and solving
Their passion and its effectiveness
or critical thinking. However,
notwithstanding, I’ve shot my whole
even these organizations seemed
budget on something that doesn’t even
progressive compared to state or
rate a whole class period.
district curriculum departments, so
Conversely, developers like
there was little pressure to go beyond
Computer Curriculum Corporation
simple items requiring equally simple
(CCC), Milliken Publishing, Science
response formats.
Research Associates (SRA), WICAT, Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
87
The requirement of simple
from their influence for another
responses, combined with the
twenty years. While early developers
provision of remedial explanatory
showed their disdain for conditioning
sequences or “helps” that failed
theories of learning (see Skinner, 1953)
to adapt to students’ difficulties,
by rejecting the claim that computer-
further divorced the computer-based
based instruction was simply
curriculum from either classroom or
“programmed learning by computer,”
real-world application. Not only did
nevertheless, they seemed to embrace
the computer have to be able to judge
some of principles of operant
a response, it had to be able to trace
conditioning by designing programs so
for the student a path to a correct
that:
solution. Because artificial intelligence
ÿ Practice involved relatively small
was an expensive and impractical
pieces of behavior with which to
dream at the time, computer-based
gain stimulus control;
instructional applications could not adapt to particular learner skill deficits and generate custom explanations.
ÿ Reinforcement was applied immediately to a correct response; ÿ Small pieces of behavior were
Instead, in most cases, remediation
chained together into more
and explanation sequences only
complex organizations;
recapitulated static solution sequences
ÿ Practice items were reviewed
for simple problems, illustrating them
periodically to prevent extinction
with preconstructed steps. If a student
of acquired skills or to capitalize
did not grasp the approach after one
on the facilitative effect on the
of these static non-generative help
reacquisition of skills by those
sequences, she had no place to turn.
previously learned but forgotten; and
Before Practice Became a Dirty Word
Although signal events marking
ÿ Reinforcers and their schedules could be varied.
the rebirth of cognitive psychology (e.g.,
Rejecting behaviorism, these same developers embraced the newly minted
Miller, Galanter, & Pribram, 1960)
information processing psychology
happened on the same campus where
(e.g., Atkinson & Shiffrin, 1968). It
Suppes and his colleagues worked,
hypothesized a staged pathway for
the central vein of computer-based
pieces of information that ran from
instruction would remain immune
sensory acquisition through a sensory
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
88
Michael Allenʼs 2008 e-Learning Annual
store, through short-term memory,
the language of objectives would
through a working memory where it
provide a common set of descriptors
combined with and was organized
permitting them to be compared.
by material retrieved from long-
Moreover, educators and instructional
term memory, to long-term memory.
designers hoped that objectives, at
It defined information as small
least in their earliest forms, would
packets that could be experimentally
provide statements of the performance
tracked through these stages of
expected of learners.
memory. Computer-based instruction
However well-intentioned this
developers designed with simple
transformation into objectives may
items asking for simple responses,
have been, it helped grind curricula
fitting the information processing
into tiny pieces. Commercial
model nicely. This model supported
curriculum developers embraced
developers’ efforts to explore complex
the “objectives movement” because
learning management strategies while
it enabled them to compare their
maintaining fragmentary instructional
offerings with goals and objectives
items and simple learner responses.
pursued by states. Each vendor
Educational practitioners and
developed technology to enable them
curriculum developers were exposed
to map their objectives to those
to the so-called “behavioral objectives”
of a district to determine degrees
methodology (Mager, 1975) that
of similarity and to identify where
claimed kinship with information
computer-based instruction products
processing theories of learning and
had “holes.” Objectives became
memory. Although learning objectives
marketing weapons for vendors so
came to be defined in different ways,
their reductionism became a continual
early forms generally included only:
feature of and influence on vendors’
ÿ A description of the behavior to be
materials.
demonstrated, ÿ The conditions under which the behavior would be performed, and ÿ A statement of the criterion used to discriminate acceptable from unacceptable performance.
Johnny Learned This Because the Computer Says So
Dicing curricula into smallgrained chunks smoothed learners’ trajectories
Educators hoped that the transformation of curricula into
through the space defined by the items presented. Aficionados of computer-
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
89
based instruction claimed that such
interesting and well-crafted pieces of
atomization enabled precise tracking
instruction.
of what learners had mastered.
Their bloated costs prevented
However, rather than pinpointing
help sequences from modeling
where learners were on their road to
or explaining much. Most of
mastery, learners’ positions within
them involved predetermined
item sequences identified that which
demonstrations of a process or
the learner had to “get past” but not
procedure with which the learner had
necessarily learn.
trouble. Others did even less, resorting
Designers and developers believed
to verbal explanations of the rule,
that if learners successfully performed
concept, or procedure being taught.
on enough items of a particular type,
Most such sequences would cap their
then they must have learned the
explanations or demonstrations by
objective associated with those items.
simply revealing to the learner the
In making this assumption, educators
correct response that she should
became confused and disappointed
have made. While practicing with
because reported results often
previously worked out problems has
exceeded what students could actually
been shown to be an effective means
do.
of reducing cognitive load so that
The brutal economics of
learning can take place (Sweller &
development drove this inconsistency.
Cooper, 1985), that was hardly the
The preparation of the data to present
reason for telling the student the
a single item, to capture a learner’s
answer. Instead, by revealing the
response, and to judge that response
answer, the developer marked the end
cost relatively little in the context of
of the sequence beyond which further
thousands of items in a curriculum.
attempts to model or explain the
In contrast, the machinery to present
objective would cost too much.
a remedial or explanatory sequence,
Because of these static sequences
usually termed a “help,” cost much
that concluded by revealing the
more. Developers averred that their
correct response, students could make
best work lay within such help
progress without actually learning
sequences and expressed frustration
what was required. For example,
that learners who performed well
recently an exquisitely crafted
enough to be judged not in need of
computer-based multimedia early-
help never experienced the most
reading curriculum was evaluated in
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
90
Michael Allenʼs 2008 e-Learning Annual
a large urban district. Observers were
words. Desperately wanting to avoid
stunned with how tightly the children
false negative judgments where the
were riveted to the arresting visual
application told a student that she had
displays and rhythmic audio. Indeed,
erred when she had not, we tailored
when the children were called on to
the feedback to a possibly incorrect
respond to assessment or practice
response by having an animated
items, they sometimes missed items
character, a panda, ask the student to
intentionally in order to re-experience
repeat the response as if it had not
entertaining sequences presented
heard it. After a variable number of
earlier. Help sequences rather briefly
responses judged incorrect, the panda
led them to a point where the correct
would pronounce the word or phrase
response was revealed. By simply
and move on. Although the system
remembering how to respond, learners
would report these problems to the
could work through the curriculum
teacher, the student could continue
without actually learning the
through a series of conversationally
phonological awareness and sound-
entertaining but pedagogically
symbol correspondence skills the
questionable sequences. Indeed, in
developers intended for them to know.
one instance, we observed a student
As another example, my colleagues
happily conversing with the panda
at the IBM T.J. Watson Research
at the back of a classroom. Moving
Center and I developed a practice
over to get a closer look, we found
environment for reading skills
him reading words and phrases to
that used speech recognition (Nix,
the panda, who asked for every single
Fairweather, & Adams, 1998; Williams,
response to be repeated as if the
Nix, & Fairweather, 2000). Recognizing
student were responding incorrectly all
words spoken by children proved
the time. Further examination revealed
challenging, but
that, although his earphones were
Recognizing words spoken by a large speech
properly plugged in, his microphone
children proved challenging. data collection
was unplugged. He “finished” the book
effort helped
without the application detecting even
work the error rate down to just over
one correct response, all because it
5 percent. While that may sound good,
responded so neutrally and because it
consider a situation in which a teacher
wound up pronouncing each word or
working on oral reading skills accepts
phrase for him anyway.
a wrong utterance or, worse, punishes a correct one once every twenty
Although the vendors of the late 1970s and 1980s insistently claimed
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
91
that their materials helped students
effectively remediated the most basic
learn, with equal determination they
skills in those lacking them (see Aeby,
refused to guarantee results. Vendors
Powell, & Carpenter-Aeby, 1999, for
argued that because they had no
use with low-achieving learners).
control of the instructional delivery
Because this effect could be detected
environment, they could not ensure
in schools with a high proportion
that the instructional systems and
of low-achieving students, many
materials were properly used by their
vendors of computer-based instruction
customers. If the materials were not
materials shifted their marketing
properly used, they argued, it was
focus to “Title I” schools, so-called
unreasonable to be held responsible
because of the funds made available
for student learning gains. Although
for disadvantaged students by that
this seemed to be a tacit admission of
part of the Elementary and Secondary
the deficiency of their instructional
Education Act.
strategies to independently enhance
Hersh Waxman and his colleagues
learning, vendors continued to
(2003), in a thoughtful and fairly
promote this specious argument
recent meta-analysis of the effects of
without resistance until the traditional
computer-based instruction, found a
computer-based instruction for
moderately strong effect. However,
schools industry collapsed.
they offer the following trenchant qualification:
Of Course It’s Effective, Right?
From 1970 to
One of the most important issues related
2000, it was
to teaching and learning with technology
easy to find
that needs to be addressed is the soundness
reports of the
of the research for the implementation and
salutary effects
improvement of technology programs. First,
of computer-based instruction (e.g.,
there were few quantitative studies published in
Fletcher, Hawley, & Piele, 1990;
the last five years that included relevant data to
Kulik, Kulik, & Bangert-Drowns,
permit a meta-analysis and calculation of effect
1985; Waxman, Lin, & Michko,
sizes. Scientific journals that use independent
2003). For most of that period, their
peer review in deciding what research merits
combined weight crushed the few
publication are generally considered to be
who questioned whether this emperor
the highest standard of research, yet much of
had clothes (e.g., Becker, 1994). There
the work in the field of teaching and learning
had never been much question that
with technology does not meet that standard
traditional computer-based instruction
[emphasis added]. The lack of quality, refereed
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
92
Michael Allenʼs 2008 e-Learning Annual
quantitative studies points to a serious problem
small-group studies cobbled together.
of research in the field. (p. 16)
Wenglinsky found that it was how computers were used more than how
It seemed that the effectiveness of
much they were used that influenced
computer-based instruction would
their achievement in mathematics. In
forever be sketched by meta-analytic
particular, the time spent by students
studies summarizing effects found
who used the computer to practice
across studies of small groups of
low-level skills did not correlate
students in widely varying learning
positively with better test performance
environments. The effects of computers on The use of student achievement measured at small groups a national level did not warrant of learners the considerable amount of continually money spent to acquire them. weakened studies and forced
and, for some segments of the population, correlated negatively. Economists addressed the issue by examining the return on investment of lottery funds by the government of Israel in educational technology (Angrist & Lavy, 2002).
the reliance on meta-analysis for an
They concluded that the effects of
obvious reason: cost. No one could
computers on student achievement
afford to equip hundreds or thousands
measured at a national level did not
of learners to credibly prosecute this
warrant the considerable amount of
question.
money spent to acquire them and train
Computer-assisted instruction finally had its day in court at a national
teachers to use them. What was the result of these
level. Harold Wenglinsky (1998), then
studies? Hardly anything. At their
of the Educational Testing Service,
publication, the industry built
examined the relationship of the
around directed instruction by
amount of time students in different
computer was on the verge of sudden
grades spent on computers at home
collapse. The low cost of shrink-
and at school with their aggregated
wrapped educational software had
performance on the most recent
already mortally wounded almost
National Assessment of Education
all of the purveyors of large-scale,
Progress (NAEP) available at the time.
broad curriculum computer-based
This relationship, based on thousands
instruction. Moreover, even the
of students assessed with a common
market for shrink-wrapped packages
standardized instrument, contravened
evaporated shortly thereafter,
the conventional wisdom built on
plummeting from $458 million in 2000
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
93
to $152 million four years later, a 70 percent decline (Richtel, 2005). The problem of financial
Frustration with computer-based instructional tutorials in schools contributed to vigorous development
weakness, compounded by increasing
of computer tools for learning. These
pressure from school and teacher
were pressed into the service of the
accountability, reshaped the ways
constructivist perspective on learning.
teachers and principals applied their
Rejecting the “transmission” model
time and budgets. Unless computer-
of learning that depicted knowledge
based instructional materials could
moving from a knowledge source
promise to do at least as good a
into the student, constructivists
job as the teacher, that teacher
believed that effective learning
would be reluctant to turn over
required the student to build his or
a significant amount of time to a
her own knowledge, subjecting it
machine. Moreover, as vendors began
to review, critique, demonstration,
to insist that proper use of their
and reorganization. Computer tools
materials required close coupling
included “productivity” software such
with classroom activities, teachers
as word processors or spreadsheets,
saw themselves being forced to give
visualization tools such as concept
up even more time. For increasing
mappers or function graphing
numberings of them, it simply was not
programs, or even intelligent
worth it.
composition support environments, to name a few. These tools seemed well
Was It a Complete Waste of Time and Money?
Over the first
suited to the self-directed, hypothesis-
thirty years or
driven, problem-oriented learning of
so, traditional
the constructivist enterprise.
computer-based instruction
For example, Geometer’s Sketchpad departed from the traditional tutorial
in schools lost its way. Contrasting
format. Instead of explaining concepts
approaches, such as the intelligent
by choosing words and illustrations
tutoring materials already mentioned,
aimed at the mythical “average
provided rare exceptions that
student,” this program offered learners
delivered on their promises. However,
opportunities to make constructions
their extraordinary development costs
and generate proofs, recapitulating the
keep most of them from successfully
explorations of Euclid, Archimedes,
making the trip from the laboratory to
and Descartes.
the classroom. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
94
Michael Allenʼs 2008 e-Learning Annual
range of domains, but proof eluded them. Geometer’s Sketchpad, on the Figure 1. The
other hand, is almost always used
Geometer’s
by students supported and guided
Sketchpad® is
by a skilled geometry teacher. In a
published by
sense, its effectiveness is an extension
Key Curriculum
or magnification of that of the
Press. For more
teacher. Ironically, to be effective,
information, visit
constructivist methods require a
www.keypress.
bit of the “directiveness” that, in
com/gsp.
overwhelming doses, crippled earlier forms of computer-based instruction. Looking back, even computer-based The Logo computer language,
developed by Wally Feurzeg and others at Bolt, Baranek, and Newman, and famously championed by Seymour Papert of MIT, also contrasted sharply with traditional computerbased instruction by providing learners a means to pose and solve problems through programming. Teachers using Logo believed that students could productively explore mathematical, symbolic, and graphical representations of phenomena ranging from biology to linguistics to physics. However exciting the promised intellectual journey of such programs might seem, it has become clear that effective use of these tools requires considerable guidance by a skilled teacher (Kirschner, Sweller, & Clark, 2006). Adherents of Logo had long claimed that it enhanced learners’ problem-solving abilities in a wide
instruction did not follow the path its creators envisioned: its failures bore fruit. It threw a painfully bright light on what not to do and why and, in so doing, helps illuminate where we might explore next. For example, the structure of problems, their presentation, their adequacy as models of processes and objects in the real world, and their ability to elicit a problemsolver’s understanding of the need to learn something have suffered a stunted evolution in the context of computer-based instruction. How might these problems be transformed in a collaborative virtual world such as Second Life? Consider Donna Dillenberger’s (personal communication, 2007) representation of a data center. Because it feeds on data from real data centers, it presents an authentic model of the problems
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
95
faced by managers. As one moves
acquired each of those skills. As
about the 3-D representation, one
fine-grained, evidence-based, and
goes through a “cognitive registration”
revolutionary as such models might
process (LeDoux, 1996) that orients
be, their narrow scope and fleeting
the learner and exposes the artifacts of
duration imprisons their promise.
learning within an authentic context.
Imagine if, instead of a model made
If the learner hasn’t mastered the
up of discrete context-free skills,
anatomy of a router, its importance is
one made up of estimates context-
impressed upon the learner caught up
sensitive, interacting abilities and
in the problem of a breach of security.
disabilities guided tool use. What
Moreover, the environment offers
might be inferred if a learner were
itself for reorganization, enabling
able to effectively summarize the
questions such as, “Can we afford to
methods by which an agreement
create a lower bandwidth channel
was negotiated, even if that learner
between these two servers?” How does
did nothing more than observe the
a learner formulate a question to the
process? Is that inference consistent
expert summoned to the scene? Often
with estimates of related learner
the degree to which a question or a
characteristics that have evolved over
statement can be clearly expressed
years of technology-based observation,
reflects the speaker’s understanding.
hypothesis generation, experiment,
While the field has advanced from
and re-estimation? It may be, for
simple problem descriptions to finely
example, that our learner withdraws
articulated models of their solution,
in social situations, real or virtual,
user modeling has only crept along.
but that abstraction, summarization,
This may seem to make little sense,
and communication historically
given that it has advanced so far
have remained exceptional. Should
from the trajectory models through a
the technology aim to ameliorate
curricular space composed of problem
weaknesses or engineer experiences
descriptions. The most effective
that address learner strengths?
cognitive tutors, such of those of
Can we approach the perennial
John Anderson and his colleagues
goal of continuous authentic learning
(Anderson, Corbett, Koedinger, &
by insinuating software interfaces
Pelletier, 1995), model learners as a
between learning experiences and
collection of skills. Problem-solving
their real-world consequences?
performance enables the inference
Reconsider Donna Dillenberger’s
of the degree to which a learner has
(personal communication, 2007)
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
96
Michael Allenʼs 2008 e-Learning Annual
virtual data center and its possible
intervene and “change history” to
evolution. A first phase involved
evaluate such actions against what
user activities such as exploring the
“really happened.” But why stop there?
virtual data center, manipulating the
Why not blend the training experience
environment, sharing observations,
with the actual operational one by
seeking information from one another,
recapitulating the evolution from
or critiquing one another’s ideas.
“canned data,” to “real data,” then
Even operating on “canned data,”
extending it to “the real thing?” With
this virtual data center was able to
the learning interface unified with
unfold a rich manifold of experience
the operational one, members of the
within which a guiding instructional
staff running the center could move
executive could operate. (Even without
back and forth across these stages,
it, the experience affects everyone
using technology, each other, and the
undergoing it). She has implemented
data center itself to fine-tune their
a next phase operating on a stream
performance.
of actual data flowing from operating
So, even though the history of
data centers that enable an authentic
computer-based instruction yielded
mirror to events, processes,
only the faintest image of our initial
procedures, and people. With it,
dreams for it, as with other fields of
one could create training scenarios
human endeavor, a new, sometimes
comparing learner group performance
difficult to recognize set of dreams
to data center staff or the ability to
will guide us now.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time REFERENCES Aeby, V.G., Powell, J.V., & Carpenter-Aeby, T. (1999). Effects of SuccessMaker Computerized Curriculum on the behavior of disruptive students. Journal of Educational Technology Systems,28(4), 335-347. Anderson, R., Corbett, A.T., Koedinger, K.R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), pp. 167-207. Angrist, J., & Lavy, V. (2002). New evidence on classroom computers and pupil learning. The Economic Journal, 112, 735-765. Atkinson, R.C., & Shiffrin, R.M. (1968). Human memory: A proposed system and its control processes. In K.W. Spence and J.T. Spence (Eds.), The psychology of learning and motivation: Advances in research and theory, Vol. 2. New York: Academic Press. Becker, J. (1994). How exemplary computer-using teachers differ from other teachers: Implications for realizing the potential of computers in schools. Journal of Research on Computing in Education, 26(3), 291-321. Braun, L. (1971a). Huntington II Simulation Program-POLUT. Teachers guide. Maynard, MA: Program Library, Digital Equipment Corporation. Available from Educational Resources Clearinghouse Center, ERIC number: ED093644. Braun, L. (1971b). Huntington II Simulation Program-POP. Maynard, MA: Program Library, Digital Equipment Corporation. Available from Educational Resources Clearinghouse Center, ERIC number: ED179414. Dillenberger, D. (2007). Personal communication on the evolution of shared virtual worlds as learning tools. Yorktown Heights, NY: IBM Research. Fletcher, J.D., Hawley, D.E., & Piele, P.K. (1990). Costs, effects, and utility of microcomputer-assisted instruction in the classroom. American Educational Research Journal, 27(4), 783-806. Kirschner, P.A., Sweller, J., & Clark, R. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based learning. Journal of Educational Psychology, 41(2), 75-86. Koedinger, K.R., Anderson, J.R., Hadley, W.H., & Mark, M.A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43. Kulik, J.A., Kulik, C.-L. C., & Bangert-Drowns, R.L. (1985). Effectiveness of computer-based education in elementary schools. Computers in Human Behavior, 1, 59-74.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
97
98
Michael Allenʼs 2008 e-Learning Annual
LeDoux, J. (1996). The Emotional Brain: The Mysterious Underpinnings of Emotional Life. New York: Simon & Schuster. Mager, R. (1975). Preparing instructional objectives. Belmont, CA: Fearon. Miller, G.A., Galanter, E., & Pribram, K.H. (1969). Plans and the structure of behavior. New York: Henry Holt. Nix, D., Fairweather, P., & Adams, W.H. (1998). Speech recognition, children and reading. Conference on human factors in computing systems. New York: Association for Computing Machinery. Resnick, L.B., Carpenter, T.P., Moser, J.A ., & Romberg, T.A. (1982). Addition and subtraction: A cognitive perspective, Mahwah, NJ: Lawrence Erlbaum Associates. Richtel, M. (2005, August 22). Once a booming market, educational software for the PC takes a nose dive. New York Times. Skinner, B.F. (1953). Science and human behavior. New York: Macmillan Suppes, P., Fletcher, J.D., & Zanotti, M. (1976). Models of individual trajectories in computer-assisted instruction for deaf students. Journal of Educational Psychology. 68(2), 117-127 Suppes, P., & Morningstar, M. (1969). Computer-assisted instruction, Science, 166, 343-350. Suppes, P., & Morningstar, M. (1970). Four programs in computer-assisted instruction. In W.H. Holzman (Ed.), Computer-assisted instruction, testing, and guidance (pp. 233-265). New York: Harper & Row. Sweller, J., & Cooper, G.A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2, 59–89. Waxman, H.C., Lin, M., & Michko, G.M. (2003). A meta-analysis of the effectiveness of teaching and learning with technology on student outcomes. Naperville, IL: Learning Point Associates. Available: www.ncrel.org/tech/ effects2/ Williams, S.M., Nix, D., & Fairweather, P. (2000). Using speech recognition technology to enhance literacy instruction for emerging readers. In B. Fishman and S. O’Connor-Divelbiss (Eds.), Fourth International Conference of the Learning Sciences (pp. 115-120). Mahwah, NJ: Lawrence Erlbaum Associates. Wenglinsky, H. (1998). Does it compute? The relationship between educational technology and student achievement in mathematics. Princeton, NJ: Educational Testing Service. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Wallace Hannum Dr. Wallace Hannum is a faculty member in the School of Education and coordinator of the educational psychology program at the University of North Carolina at Chapel Hill and associate director for technology of the National Research Center on Rural Education Support. Dr. Hannum teaches graduate-level courses on learning theories, instructional design, and the use of technology in education. Dr. Hannum’s research focuses on instructional uses of technology, especially distance education. Dr. Hannum has consulted on instructional uses of technology with many organizations, both public and private. He has participated in the design and implementation of numerous technology-based programs and projects. He has worked extensively on education projects in Africa, Asia, and Latin America. Dr. Hannum is author of five books and numerous articles on topics related to technology and instructional design.
Reflections on e-Learning
Wallace Hannum
The Long History of a New Idea The revolution in educational technology that was to follow the invention and use of personal computers never happened. But why? The short answer that I learned the hard way is that there is no magic in the machine. Simply placing instructional content on a computer and delivering it to students does not enhance instruction nor guarantee learning. Rather, to achieve effective instruction with technology, educators and designers must pay attention to both pedagogy and past lessons while endeavoring to extend technology’s strengths, rather than simply duplicate what’s already been done in the past. While the specific origins of the term
term in Google), you would think we
“e-learning” are not entirely clear, Jay
all understood the term in a similar
Cross used the term about seven years
fashion. Yet after even a brief analysis
ago to refer to the convergence of
of articles or a discussion with people
learning and networks. He stated that
who use or promote e-learning, it is
e-learning was to traditional training
clear that we do not understand e-
what e-business was to business as
learning the same way.
usual. Computer and communications
What qualifies as e-learning
networks were changing the very nature
according to one person’s concept of
of how we conduct business and how
e-learning fails to qualify according
we conduct training. On the surface,
to another’s. In short, we understand
e-learning seems a recent phenomenon
e-learning in different ways. When
following from the proliferation of the
someone completes a computer-based
Internet, yet writing several years ago
training (CBT) lesson on a CD, is that
Mühlhäuser (2004) entitled his paper
e-learning? Some say yes; others say no.
“e-Learning After Four Decades: What
Some refer to most distance learning
About Sustainability?” It may seem a bit
today as e-learning if it involves the
odd that e-learning can have a history
Internet in any fashion, even if only
of almost five decades, despite seeming
to download class materials. Others
to be new and identified by name only
challenge this being e-learning.
a few years ago. The reason for this
Some use e-learning to refer to any
incongruence follows from the meaning
instruction delivered via computers,
of e-learning. Given how often the term
whether or not they are networked or
is used today (70,300,000 hits for the
use the Internet. Still others include
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
102
Michael Allenʼs 2008 e-Learning Annual
the use of remote exchange and
from the communications metaphor
collaboration in their definition of
in which the Internet is viewed as a
e-learning, whether or not it includes
means for instructors and students
delivering of instructional materials.
to communicate for instructional
If an instructor sends an email with an
purposes (Hannum, 2001). This
attachment for students to download,
definition of e-learning includes the
print, and read, some refer to this as
student working alone through an
e-learning. In essence, this example
interactive tutorial on calculus as well
describes the old vintage WWII
as the student working collaboratively
correspondence study in which the
with others at remote sites on an
instructor sent students materials to
exercise in calculus, perhaps under
read and exercises to complete, and
their instructor’s tutelage. Under the
which they returned to be graded. In
auspices of an educational agency
its more recent version, the Internet
or department, both are organized,
has replaced the post office as the
planned instructional experiences with
means for instructors and students
expected learning outcomes.
to send exercises and exams back
My definition of e-learning does
and forth. While some see this as
not discount the value of an individual
e-learning, others insist that e-
working alone on the Internet
learning must involve the use of “new
searching out something of interest
multimedia technologies,” and not
and possibly learning from it as a
simply the sending of print materials
result. I use Google daily when I am
over the Internet. Those who speak
curious about something. Last week
and write about e-learning know what
I wanted to confirm the nationality
they mean by the term, yet they know
of Archduke Ferdinand, whose
it in different ways. Hence, there is
assassination provoked WWI, and
confusion over exactly what e-learning
I also wanted to find out why diesel
is and when it began.
automobiles made that knocking noise
For purposes of this chapter,
at idle. In both cases I easily found and
I am using e-learning to refer to
learned the answer using the Internet,
instruction delivered at least in part
but was I engaged in e-learning? No,
over the Internet. This instructional
not as I see it. e-Learning involves a
use of the Internet may follow from
planned, coordinated effort involving
the publishing metaphor in which
a sponsoring organization to produce
the Internet is viewed as a means to
specific learning outcomes for a
distribute instructional materials or
specific group of learners. I was using
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
103
the same technology and perhaps even
requires that we situate it historically,
a website or reference they might use,
describe what we know about it, and
but my use of the Internet to research
speculate on what will happen next,
a specific question is not e-learning
despite the obvious risks of such recall
any more than someone running to the
and speculation.
library years ago to look up these same questions in Encyclopedia Britannica
My Journey
My first use
would have been considered to be
of a computer
instruction.
in schools was
The recent furor over e-learning
1967 or 1968
ranges from estimates that it will
(my recall is not
replace all other forms of instruction
more precise about the exact year).
and greatly improve learning
The computer was an IBM 1401,
outcomes, to those who see e-learning
an early transistor-based computer
as a dangerous distraction that is likely
with all of 4K of memory running
to fail. Any reasoned consideration
FORTRAN.
of e-learning, especially its potential impact and future, should first look at antecedents and see what can be learned from them before speculating about what might happen in the future with e-learning. This chapter attempts to explore the rather long history of e-learning, identify some lessons learned from the
Figure 1. IBM 1401
past, and speculate on some possible futures for e-learning. Speculating
The first application the team I
about the future is always risky,
was working with developed was
especially when that speculation
an automated payroll system. As a
centers on technology. Recounting the
result of our work, teachers were
past is also risky, as those who were
able to get their checks two days
there and remember certain events
sooner. Although we were much
don’t agree on what happened, much
more interested in instructional
less those who did not witness the
applications, the decision to do the
recalled events directly. However,
payroll application first was made by
putting e-learning into context
the person who directed research for
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
104
Michael Allenʼs 2008 e-Learning Annual
the school system. He was a crusty,
learning objects, although much of
old, retired Army sergeant who had
the actual content was in an analog
learned how to get things done with
medium such as filmstrips, video,
people through many battles, both
or audio recordings. It was modular
literal and figurative. As a result of
at fine levels of granularity, down to
our automating payroll, many teachers
parts of an individual lesson linked
and administrators said something
to individual objectives, searchable
to the effect of, “I don’t know what
using various ways of matching
computers are, but if they help me get
content to instructional needs, and
paid sooner, I like them.” Thus was our
it was reusable. Teachers appreciated
introduction of computers into a small
the ease with which they could locate
school system in a southern state forty
appropriate instructional resources
years ago.
for specific objectives and for specific students. Had the content itself been
A Foray into Learning Objects
in digital format, I would argue we
Following that, we turned our
were working with learning objects
attention to instructional applications
back then.
and created a classification system
Following this we began
for instructional materials matched
what interested us most—direct
to specific instructional objectives,
instructional applications of
teaching approaches, and student
computers in the form of computer-
characteristics. Essentially this was
assisted instruction (CAI). Our
a large database, although it was
efforts were very elementary—not
hard coded years before any database
much more than automating flash
program like Access or Oracle was
cards—but students were able to
available. A teacher could query
interact directly with a computer that
the database we created to locate
asked them a question, accepted their
instructional materials on a specified
answers, judged the adequacy of their
topic that used a selected teaching
answers, and immediately provided
approach for a student or group of
the appropriate feedback. This allowed
students with certain characteristics.
students to advance at their own pace
We saved teachers considerable time
and provided them with many more
they would have otherwise had to
practice opportunities than a teacher
spend looking for, or creating, such
in a classroom with twenty-five to
instructional content themselves. In
thirty other students could provide.
many ways, this was a forerunner of
Plus, we were able to match the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
105
Figure 2. (Left and Above) IBM 1500
students to individual lessons rather
most anybody’s definition. Here’s how
than the class as a whole. The greatest
it happened. We had an IBM 1500
advantages seemed to accrue to those
system that was specifically designed
students who were either considerably
to deliver CAI to students through
ahead or behind their peers. I suspect
sixteen CRT terminals connected
this is still true today.
locally to the 1500 system by cables. Content was developed, laboriously
Early e-Learning With my interests
I might add, by using a programming
in computer-assisted learning (CAI)
language called Coursewriter that
blooming, I relocated to Florida State
was designed for this purpose. In
University the next year to work in
order to allow remote access to the
a CAI lab that had been established
system and thus the CAI lessons,
there. It was over thirty years later
a Digital Equipment PDP-8 was
before anyone used the term e-
interfaced with the IBM 1500 system
learning and before the Internet was
to send signals over telephone lines
invented, but we were delivering
to terminals at remote locations.
CAI in reading and mathematics
These terminals were teletypes that
to underprivileged elementary age
would print character-by-character
students who were located in another
what was sent by the computer. They
county some twenty-five miles away.
would also accept keyboard input
Reflecting back on that experience
from the students and transmit it
now, I would call that e-learning by
back to the IBM 1500 system as if the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
106
Michael Allenʼs 2008 e-Learning Annual
student were sitting in the adjacent
now. Figuring in all the factors we
room at a terminal hard-wired into
could consider and pricing them
the computer. There were no PCs, no
appropriately, the actual costs of
Internet, no multimedia, no email,
providing CAI once we did a project
and no discussion forums, but we did
would always outstrip our estimates of
have students at a remote location
the cost by a factor of ten. We got so
completing CAI lessons. As we say
we would chuckle after doing a careful
today about e-learning, they were
estimate of the cost of providing CAI
learning at their place, at their pace,
in some situation when somebody
and at the time of their choosing. It
would mention something to effect of,
was any place, any pace, and any time
“Now that you have the estimate done,
learning. Isn’t this how we describe e-
move the decimal over one place.”
learning today?
Invariably this would wind up being a
The economics of early e-learning
better predictor of the real costs as $1
were very different than the economics
at the time of estimate became $10 or
of today’s e-learning. There were few
more when the project was done.
computers capable of supporting CAI back then. Florida State had one of
Blended Learning Largely because
only seven prototype 1500 systems.
of the expense of developing
The cost of computer time was high
and delivering CAI, we began to
in those days of mainframe-only
experiment with another form of
computing, and even simple CAI
instruction in which all the content
required considerable computer time.
was not delivered via the computer.
The time and costs of developing
Rather we gave students packages
a CAI lesson were also quite high.
of printed materials to read and
Estimates ranged from 150 to 400
had them use the computer only
hours of development to create
for testing, obtaining feedback, and
one hour of CAI. The long distance
receiving their next assignment based
charges of a dedicated telephone line
on their test results. If students
were also high back then, making the
mastered the content, as shown by
overall costs of remotely delivering an
their responses to criterion-referenced
hour’s worth of CAI to one student
test items, they were directed to the
prohibitively expensive regardless of
next lesson. If they failed to master
the effectiveness.
the content, the specific objectives
I still recall our estimates of costs to deliver CAI with some humor
they failed were noted and they were directed to additional readings
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
107
and reference materials associated
As before, a student who was not
with those objectives. When they
successful could meet with his or her
completed these assignments, students
instructor. Pick up any journal three
would sign back on the computer
decades later and you will read about
to take an alternate form of the
the advantages of blended learning
assessment for their content before
in which all the instruction does
being allowed to advance. Students
not happen on computers—some is
failing to reach mastery on this second
online, some is face-to-face. As with e-
attempt were directed to schedule an
learning, blended learning is a recent
appointment with the instructor or a
term with a long past.
teaching assistant for some coaching and mentoring before they could
Incorporating Graphics From
continue in the course. We called
the work with computer-managed
this computer-managed instruction
instruction, “blended learning” if
(CMI) to distinguish it from CAI in
you like, we turned our attention to
which all the instruction happened
creating computer-based learning
online. In CMI most of the learning
environments that extended beyond
happened offline when students read
text on screens that had typified
instructional materials or met with
most of the CAI efforts at that
their instructors. The computer was
date. Computer graphics were
used to manage learner progress
possible, but only by using ASCII
through the course. In some CMI
characters or laboriously going in
courses, we used those students who
and turning each individual pixel off
had successfully passed certain lessons
or on to form an image. In general,
as peer-tutors for those who had yet to
it was too time-consuming and too
pass those lessons. Then, if a student
expensive to use much in the way of
was not successful on the exam for a
computer-generated graphics with
lesson, in addition to being directed
the technology
to additional resources, he or she
of the day. What
It was too time-consuming
would also be directed to classmates
was possible, and
and too expensive to use
who had passed that lesson. It was not
what we did, was
much in the way of
mandatory that a student meet with
to interface other
computer-generated graphics.
one of the classmates who had been
display devices and
successful, but we tried to facilitate
bring them under computer control.
their getting in touch with each other
We created hardware and software
for discussion about the content.
that enabled the computer to control
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
108
Michael Allenʼs 2008 e-Learning Annual
a slide projector. Thus, when a lesson
processes have strong antecedents.
called for a color image of a giraffe,
I suspect the next “new thing” in
the computer would turn the slide
educational technology will also have
projector on and advance it to the
strong ties to something that was
next slide to display a slide image of
going on several decades ago. Ignoring
a giraffe on a separate screen that
the lessons of past projects and past
we located next to the computer’s
research holds us back and has us
CRT display. We also rigged filmstrip
making unnecessary missteps when
projectors and audio recorders the
we should be advancing the field, not
same way so that they could be
repeating past errors. To this end I
brought under computer control to
offer some observations about lessons
display images and sounds that were
learned.
essential to the CAI lesson. It was
I start by acknowledging I have a
crude and complex to program, but it
bucketful of seemingly good ideas
was early multimedia.
that didn’t work then and likely won’t work now. I cringe when I see these
The Importance of Antecedents
playing out now in ways that are
The central point in each of these
quite predictable. In the early days
examples is to trace one person’s
of CAI, we thought the ideal was
earlier experiences using technology
one student working alone on one
for instruction to demonstrate that
computer, going through a lesson
most, if not all, of what we think is
that was designed to progress in
contemporary in terms of technology
small steps and through a series of
applications such as e-learning and
questions he or she had to answer
interactive multimedia have been
to receive immediate feedback. This
in existence for several decades. If
was the basic model, whether it was
we expect to use e-learning today
exclusively question/answer sets (as in
to produce significant learning
drill-and-practice CAI) or whether it
improvements, we would do well
contained some initial instruction in
to start by looking back at work
the form of passages to be read before
done four decades ago, not just at
completing the question/answer sets
what people are doing this year.
(as in tutorial CAI). Individualized
Without question, the technologies
instruction reigned supreme, and it
involved have changed greatly—more
was individualized instruction we
than anyone had imagined. Yet the
sought to provide through computer
underlying instructional concepts and
applications to instruction.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
109
We had some interesting findings
were not studying that. I also recall
emerge from our research studies that
some students coming into the CAI
questioned our viewpoint on how to
lab to take part in an experiment
use technology to deliver instruction.
and remarking something to the
We found unexpectedly that students
effect that, “I’m not very good in
could and would learn, even if they
mathematics” or whatever the subject
didn’t have immediate feedback. We
matter involved in the experiment was.
found that repeated practice did not
Usually one of us would try to “pump
lead to better retention. We found that
them up,” saying that “A lot of students
students could learn equally well from
who have said this have done well in
a CAI lesson whose sequence was
the study. Just pay attention and you
randomly scrambled when compared
will learn; everybody has.” At times
with a lesson that was carefully
we would reassure them by saying we
sequenced to promote learning.
weren’t too good at
We found that students who only
mathematics either,
Students could and would
read through a CAI lesson without
but “this lesson is
learn, even if they didn’t have
responding to questions and receiving
very clear and you
immediate feedback.
feedback learned as well as those who
will get it.” Bandura
used the CAI lesson as planned. We
(1997) had not yet described the
found that students who missed many
impact of self-efficacy on learning. In
parts of the lesson by skipping would
hindsight, I think we were attempting
learn as well as those who carefully
to influence self-efficacy, and that is
completed each step in the lesson.
what might have been producing any
I began to appreciate the ability
learning gains we observed, not the
to learn that most students possess.
specific variables under investigation.
They learn even when they shouldn’t!
Causality was and remains a difficult
I remember joking with other
thing to establish, even in controlled
researchers that if we would pin a
experiments.
$10 bill to the top of the computer
Moving forward in time, Apple
terminal and say to the student, “If
and later IBM introduced personal
you pass the test that follows this
computers, and I was excited about
lesson, you can keep the $10,” none of
being able to study computer
the variables we were studying would
applications to instruction without
matter! In hindsight, I think what
all those large costs associated with
I was learning was the importance
mainframes. Finally, CAI would
of motivation in learning, but we
become affordable. Computers
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
110
Michael Allenʼs 2008 e-Learning Annual
were so much less expensive and
chores to create many CAI lessons
so much more widespread. .Newer
across the curriculum. I still recall the
programming languages, such as
day when a faculty member showed up
BASIC and PILOT, allowed for easy
for the initial meeting with the team
content creation. Two decades ago,
assembled to assist him in creating
I worked with several university
CAI and placed a large three-ring
faculty members to create CAI lessons
binder on the table. He said, “Here’s
for their courses. With stronger
my course, call me when you have
programming tools and infinitely less
the computer lessons ready.” Other
expensive computers proliferating,
faculty members were less blunt and
I thought CAI nirvana was at hand!
offered more of their time to the
I knew that soon all students would
effort, yet they continued in a similar
receive most of their instruction in a
manner to take what had been a
very effective and efficient manner by
traditional course and simply place it
having their own personal tutor that
on a computer. The result was seen
patiently would lead them through the
by many as new technology-based
content providing tutoring, support,
instruction—a real advance, several
and reinforcement where needed,
would say.
how needed, and when needed. It
In reality, it was the same old stuff
was all now in reach, but something
ported to a new delivery medium.
happened.
When students went through this
Computers were cheap compared
“new” instruction learning at roughly
with the mainframe computers
the same levels as before, it became
I first used for CAI, and we had
clear that technology alone adds little
many of them. Programming was so
(Clark, 1994). Remember that multiple
much easier and faster as a result
meta-analyses of considerable CAI
of developments in programming
research indicate modest, even trivial
languages. PCs had graphics
differences (effect sizes averaging
capabilities, and you could even get
two to three) when compared with
sound out of them. We had all we
traditional instruction (Jenks &
needed, but the revolution never
Springer, 2002: Kulik & Kulik, 1986;
happened.
Kulik, Kulik & Cohen, 1980; Schacter,
I worked for several years with
1999). Such is also the case with our
different faculty members enlisting
rush into e-learning. Lessons that
the assistance of computer science
are repurposed or ported over will
students to help with the programming
not be better than before. This will
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
111
cause some to question the value of
probably better than most teachers—
e-learning just as some, actually many,
because it could give students more
questioned the value of CAI. When
individualized practice and feedback
was the last time you heard someone
as well as going at each student’s own
talking positively about CAI? Such will
pace. We also recognized that the
be e-learning’s future if it progresses
computer didn’t frown at learners or
along a similar path.
castigate them. It had no knowledge of whether the student sitting at a
Early Hopes
The hopes
terminal was male or female, black or
we had for
white, the mayor’s son or the janitor’s
the earlier
daughter, the fastest learner in the
applications
class or the slowest. The computer
to instruction
worked tirelessly and consistently in
centered around providing each
presenting lessons. We thought we
student with some time each day
could embody the characteristics of
during which she or he would have his
the best teaching in the CAI software
or her own infinitely patient personal
to benefit every learner. We saw the
tutor in the form of CAI that would
weakest students in the class achieving
lead them to master some subject-
well for a change.
matter content. While we were hoping to give the students the best teaching possible, frankly and in retrospect,
Risks and Their Mitigation
We saw our adversaries as
we were working to “teacher-proof ”
the expense
the curriculum as well. We wanted
of mainframe
all students, even those with less
computer
than stellar teachers, to be learning
time and educators who thought
at high rates of mastery. Rather than
using computers would dehumanize
the older model of teacher-to-student
education. We tried to mitigate the
transfer of knowledge, we recognized
costs of computer time by evolving
a new possibility that would bypass
the CMI model in which learners
the teacher, at least for part of the
were online for only a brief time for
day, by transferring knowledge
each lesson, in contrast with CAI in
directly from computer to student.
which learners were online for the
We recognized that the computer per
entire lesson. Later, as PCs emerged,
se did not teach, but we thought the
we realized that computer time for
software we developed would teach—
instruction would become low cost
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
112
Michael Allenʼs 2008 e-Learning Annual
and not a barrier. As for those who
could and should be replaced, at
thought computers would dehumanize
least for that class. If a teacher
education, we tried to explain the
recognizes that all she or he is doing
limitations of having a student sit
is reading notes to the class in the
passively through a lecture that might
form of a lecture, that teacher should
be considerably over his head and
fear replacement by technology. In
frustrate him or considerably below
fact, that teacher should have been
her current level of understanding
replaced by the invention of the
and bore her. Some, perhaps many,
printing press long before computers
teachers were challenged by the
became available. There remain many
technology and had concerns about
uniquely human interactions between
being replaced just as university
faculty and students that even today’s
faculty feared being replaced by the
technology can’t duplicate. Wonderful
printing press centuries earlier and
faculty members continue to inspire,
by the Internet and e-learning today.
challenge, support, and motivate
More than a few faculty today object
students in ways that make technology
to having to put their syllabus online,
look pale. However, too many faculty
much less all their course content.
members continue to do little more
Just as they had previously objected
than present or “cover” content.
to having lectures taped, whether by
Technology does this quite well.
audio of video, today they object to
Our approach to dealing with this
the university “getting” their course
concern early on was to mention that
captured into a digital medium for fear
schools could not afford technology
they will no longer be necessary. That
for all the instruction or even most
is, some faculty members fear being
of it. Faculty were protected by the
replaced by technology.
high costs of providing CAI. We also
My response to fears of faculty
mentioned they could use CAI as a
replacement by technology was then
supplement to help those learners
and is now—those faculty members
needing extra practice they could not
who could be replaced by technology
provide during class. This was what
should be replaced by technology.
has become the blended learning
If a faculty member conducts class
argument—technology should be
by reading his or her notes to the
combined, or blended, with traditional
students or, a more recent trend,
approaches to teaching rather
reading his/her PowerPoint slides to
than delivering all instruction via
the students, that faculty member
technology. We also used the drudgery
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
113
argument that goes something like
In hindsight, I think the endorsement
this: “Find what the teachers don’t
was really for anything but “regular”
like to do and automate it; find what
instruction, not necessarily for the
they enjoy doing and leave it alone.”
CAI lessons themselves. This is like
That is, take the drudgery away from
many a political race in which one
them. Develop CAI for the lessons
candidate has many negatives; votes
they do not like to teach but not for
aren’t so much for the one candidate
the lessons they enjoy teaching the
as they are votes against the other. CAI
most. Then teachers will welcome
had the advantage of not being regular
technology. Such will likely be the case
classroom instruction. e-learning is
with e-learning.
enjoying this advantage today. Perhaps students of today are
Individual Learner Reaction
From the
different, as many
beginning we
have suggested.
Perhaps students of today
tried to measure
They have iPods™
are different, as many
the learners’
and cell phones.
have suggested.
reactions to using
They are used to
computers for instruction because
instant messaging their friends and
we were curious about how they
going online to find information
viewed learning through a computer.
immediately. They have become
One consistent finding was that they
used to immediate gratification,
usually liked it! Their attitudes about
24/7 support, and a fast pace. They
using computers for instruction
hear about a new song at 3:14 one
were positive, as we had hoped.
afternoon and by 3:16 that afternoon
Furthermore, their attitudes toward
they have downloaded that song and
the subject matter itself were more
are playing it on their MP3 players.
positive than before and more positive
Information comes to them through all
than students learning in a traditional
senses using rich, mediated products.
manner. They indicated that they
And it comes quickly. School must
would like to take more lessons and
seem odd and slow to them. First, we
more classes through CAI, and if given
break the content down into separate
a choice between CAI and “regular”
disciplines with sharp boundaries
instruction, they would rather take the
between. Second, we control the pace
class via CAI. We thought this was a
at which they go through lessons,
ringing endorsement from students
even segmenting classes into fixed
for using technology for instruction.
time periods. Third, we start teaching
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
114
Michael Allenʼs 2008 e-Learning Annual
by telling them things, not by asking
organization. A key factor influencing
them things or by allowing them to
instructional use of technology in
control what happens. Fourth, we
organizations is that of opportunity
insist that they pay attention to us
costs. Organizations that value
and only us during a lesson. Fifth,
learners’ time tend to use technology
we allow them no input into what to
for learning when it reduces the time
study, how to study, when to study,
learners spend on a lesson. When
or where to study. How alien all this
corporate employees are taking a
must seem to today’s technology-
class, they are on the payroll and being
using students. They multi-task, move
paid for these hours. Further, when
in many directions at once, initiate
in class, they are not available to the
their own direction for what they
corporation to produce a product
want to know, work in a non-linear
or provide a service. In short, they
manner, bouncing from one topic to
are not bringing a nickel into the
the disconnected next topic, learn by
corporation while they are in classes.
negotiating meaning with peers not by
Thus, anything that can reduce the
one-way communications, consume
time spent in learning and have them
copious amounts of multimedia
back performing their jobs sooner
simultaneously, and expect immediate
is attractive to corporations. Such is
responses to queries. We can’t
also the case in the military services.
compete with this if we stick with
The sooner they can get their men
traditional instruction, but by using
and women through the necessary
technology we can begin to provide
education and training programs,
them with a more comfortable and
the quicker they can be helping
familiar environment in which to
the military accomplish the work it
learn.
does. Technology typically produces just such results by reducing the
Organizational Views of Technology
Organizations,
time required for learning. These
whether school
organizations have a history of
systems,
adopting technology for that reason.
universities, corporations,
On the other hand, educational institutions derive their revenues
governmental agencies, or military
while students are in classes, not
services, adopt, adapt, or reject
when they are out of classes. While
technology in different ways,
corporations and governmental
depending on the nature of the
agencies pay employees while they
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
115
are attending classes, educational
instructor in front of the room talking
organizations do not. In fact, students
and a group of students listening and
pay educational organizations based
taking notes. The role of a learner in
at least to some extent on the amount
schools is being a student. The role
of time they spend in class. When
of a learner in business and industry
we talk about the costs of a public
is being a productive employee. The
education, we rarely figure in the
employees’ job is to produce some
opportunity costs. That is, we assume
product or provide some service.
there is no cost associated with the
If they are in a learning mode, it is
loss of opportunity for students to be
to help them achieve higher quality
doing something other than attending
or efficiency as quickly as possible.
school. Essentially, in K-12 public
Education in a corporate environment
education, we set the opportunity
has a focus and direction grounded
cost at zero and place no value on
in their daily realities as employees of
student time. There is little, if any,
their organizations. What an employee
incentive for schools to reduce class
learns today, he or she is expected
time. In addition, most public schools
to apply right away. Governmental
are required to meet for a certain
agencies and the military services are
number of days each year. University
similar in this regard. The education
classes are required to meet for a
they provide employees is focused
fixed number of hours each semester.
and immediately applied. This allows
Even if technology would enhance the
corporations and governmental
learning process and have students
agencies to try different approaches to
learning and achieving at faster rates,
instruction and quickly see the results
they would still have to meet the
in job performance and even their
required number of hours and days
“bottom line.” This ability to try some
for that class or that semester. When
innovation and see the effects is key
institutions regulate the time required
to accepting and using an innovation
for instruction, they don’t go looking
(Rogers, 2003). Corporations and
for efficiencies. Thus, there is not the
governmental agencies are better
incentive to use technology when its
positioned to try an innovation like
principle advantage is to reduce the
e-learning than is public education.
time required for learning.
Rogers notes that, in addition to
Tradition plays a role in technology
trialability, if an innovation provides
use as well. In educational institutions,
a relative advantage and is compatible
the longstanding tradition is an
with an organization, the innovation
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
116
Michael Allenʼs 2008 e-Learning Annual
is more likely to be adopted. By
there is no magic in the machine.
shortening learning times and
Simply placing instructional content
providing a flexible way to enhance
on a computer and delivering it to
skills of employees, e-learning can be
students does not enhance instruction
a relative advantage to a corporation.
nor guarantee learning. It does
In today’s world of knowledge workers
allow for any time, any place, and
and the creative class, having your
any pace learning, but these are just
employees know more and be better
logistics, not pedagogy. What matters
skilled is what gives any corporation
is pedagogy. A bad lesson placed on
its competitive advantage. e-Learning
a computer is still a bad lesson. It
also fits nicely into many organizations
may be more flexible, but it will not
where time and location have become
produce greater learning. Attention
more flexible, and everybody uses
to the design of instruction remains
technology as an inherent part of
paramount—despite what many
their work. Since e-learning is more
observers have been saying in this
trialable, gives a relative advantage to
high-tech age. It still comes down,
users, and is more compatible with
fundamentally, to a learner trying to
the daily way of life in corporations
make sense out of some new content,
and governmental agencies than it
whether it comes out of the mouth
is in public education, the work of
of a teacher, from the printed words
Rogers would suggest it is more likely
on a sheet of paper, off the screen of
to be adopted in these environments.
a computer, out of the speakers of
I do not expect e-learning to have a
an iPod, as images off the Internet,
significant impact on public schools,
or as video on a DVD. e-Learning
but I do expect e-learning to change
should be about learning, not about
how people learn in non-school
technology. As numerous studies going
situations.
back several decades have shown, technology itself adds very little. I
What Happened?
The revolution
recall a keynote presentation at a
in educational
conference years ago just as interactive
technology that
multimedia was emerging. The
was to follow
presenter, a key figure in the field—
the invention
maybe the key figure—mentioned a
and use of personal computers never
specific interactive multimedia project
happened. Why? The short answer
as setting the direction for the future.
that I learned the hard way is that
Later that afternoon I saw the person
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
117
who was directing the project, an
With considerable effort we
old friend and former colleague, and
developed what seemed to be a
mentioned that she must be thrilled to
pretty good simulation of how an
have such a wonderful endorsement.
anthropologist does fieldwork in a
I asked to see her work so I could see
different culture. When it was time to
the future of interactive multimedia.
test the lessons, we found we had too
It was then that she embarrassingly
many students (it was a large class)
told me they were not funded as well
and too few PCs. Without a blink,
as they would like, so what they had
the anthropology professor placed
done was take some old filmstrips and
three students at each computer and
place them onto an interactive DVDs.
said, “One of you work the keyboard
She said there was no reason to see the
while all of you discuss the simulation
project. We have a long history of the
and decide what to do next at every
latest and greatest being little more
step.” She and her graduate assistants
than old content and old pedagogy
worked in typical anthropologist
delivered by a new technology. Years
fashion by observing the students
of results show the learning outcomes
and taking field notes on what was
are about the same. e-Learning is no
happening. This is certainly not how
different.
I had evaluated CAI lessons; we
I mentioned the high regard we all
always used pretests and posttests
held for individualized instruction. In
to measure knowledge gains. Our
many ways everything we were doing
evaluation would have missed what
and had been doing with technology
was happening as multiple students
for several decades centered on trying
completed each CAI simulation
to deliver individualized instruction to
together. They read the situation
learners. I don’t remember any of us
displayed on the screen and looked at
questioning this direction during the
options that were available for their
early years, but something happened
next move. But before deciding what
twenty plus years ago when I was
to do, they had rich discussions and
working to help an anthropology
debates. One student would describe
professor create CAI in the form
what she thought was happening and
of a simulation for her class of
what the best option was, while others
undergraduates. The experience left
would question her. Then another
me wondering about individualized
student would describe how he saw
instruction.
the situation and the available options and so on. They would talk through
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
118
Michael Allenʼs 2008 e-Learning Annual
their perceptions and thoughts as they
2. Do more, much more than having
negotiated meaning and discussed the
learners read text or view graphics
best option to select next. Once they
in lessons; don’t turn technology
agreed, the designated keyboarder
into a textbook, workbook,
would enter their collective response
videotape, or filmstrip.
to see what would happen, and
3. Design lessons that require human
they would all begin to process
communication, not individualized
the feedback. I saw individualized
instruction; allow and support
instruction destroyed before my very
learners working collaboratively.
eyes, and it was a beautiful thing to
4. Provide flexibility and encourage
see! The sacred goal of providing
learners to use metacognitive
individualized instruction for every
strategies to monitor their thinking
student was replaced by the goal of
processes while completing
providing a collaborative learning
lessons.
environment for all students, at least
5. Avoid fascination with what
in my mind. If we treat e-learning
technology can do, and keep
as a way to deliver individualized
your focus on what is necessary
instruction, I think we have missed the
for learning to occur; too much
real opportunity. People learn quite
multimedia gets in the way of
well from and with other people—
learning as the gee-whiz factor
maybe better.
takes over and learners pay attention to the technology and
Lessons Learned
If I compiled my
not the instructional content.
experience from
6. Provide structure to lessons and
working with
provide guidance to learners as
technology into
they complete lessons, but set
a set of lessons
learners free to do as they wish
learned about e-learning, it would include the following: 1. Engage and involve the learners
during a lesson. 7. Understand as much as you can about the learners, especially their
in active experiences, rather than
prior knowledge of the content,
leading them through carefully
before you begin to design lessons
prescribed lesson sequences.
for them.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
119
8. Provide informative feedback
14. Think minimalist; rather than
to learners that helps them
designing lessons that include all
understand and correct any
the content you can and doing
misconceptualization or erroneous
everything that can be done with
information they may have about
the technology, provide only the
the content; don’t be concerned
content essential to mastering the
about rewarding or reinforcing
objectives; provide technology
them.
effects such as animations only
9. Design interactions that require
when absolutely necessary
more than clicking to continue to
to support specific learning
the next screen.
outcomes. Otherwise, it’s just
10. Design lessons that adapt to learners to keep the level of
distraction. 15. Watch learners, talk with learners,
challenge appropriate by allowing
listen to learners, see how they use
them to skip what they already
technology, see how they go about
know and not encounter that for
learning, see what bothers them,
which they are not yet prepared.
observe them working through
11. Give learners choices, but give
the lessons, pay attention to them,
them advice on selecting from
and involve them in the design of
among choices.
e-learning.
12. Design lessons that include a
16. Design e-learning that encourages,
digital mentor who functions as a
supports, and enables play on part
student’s grandmother would, by
of the learners during lessons; pay
providing support and advice as
attention to the engaging features
needed to guide learners through
of computer games to see what
the e-learning lessons.
works.
13. Remember that a content dump is
17. Elicit cooperation from learners
not a lesson; build in pedagogical
by being clear about why they
functions that have been shown to
are doing this lesson and discuss
enhance learning.
expectations regarding what they will get out of it.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
120
Michael Allenʼs 2008 e-Learning Annual
18. Start your lesson design based on
22. Remember the science of designing
empirically validated principles
lessons for e-learning, but don’t
of learning; then take suggestions
forget the art. Remember the art
from intended learners on how
of designing lessons for e-learning,
to improve the lessons. When in
but don’t forget the science.
doubt, listen to the learners.
23. Expect revisions, plan for
19. Don’t attempt e-learning unless
revisions, allow time for revisions,
you have someone who is a
seek input for revisions, and make
virtuoso with the content, someone
revisions even to your best e-
who is an extremely skilled
learning lessons.
instructional designer with both
24. Interactivity is vital—learners
academic and practical credentials,
need to interact with the content;
someone who has successfully
learners need to interact with
taught the content more than once,
the instructor; learners need
and someone who is skilled in
to interact with other students.
developing for the technology but
Forget any of these and e-learning
understands that pedagogy should
will be less successful.
drive the technology. 20. Do whatever is necessary to keep
25. First figure out what will help someone learn the content, then
from repackaging old content into
figure out how to do that with
e-learning and from making e-
technology—never work the other
learning work just like some older
way around.
technology. 21. Remember that most studies
26. Respect copyrights and protected materials, but use whatever you
comparing technology-based
can find that fits what you are
instruction with traditional
trying to do in an e-learning
instruction find no significant
lesson. Develop new content only
differences. To be worthwhile,
as necessary and be sure that your
e-learning has to be carefully
design, not what you find, drives
planned and crafted, not thrown
what you use.
together, as the technology alone will add no value to the learning.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
121
27. Pay attention to how something
another course that had full online
you are doing with e-learning
facilitation, high quality instructors,
will impact what is going on in
streaming video, and people to
the learners’ heads. If something
respond to any question from
in your e-learning lesson is not
students. Over 80 percent dropped
facilitating an internal mental
out of this e-learning program. By
process associated with learning,
contrast the completion rate for
then don’t do it.
their instructor-led programs was nearly 100 percent. It is not unusual
What Matters
My fascination
to find that, of those who sign up for
with technology
an online course, between 50 and 80
in the form
percent do not complete the course.
of personal
While I can’t know their motivation
computers
to withdraw, I do know that many
remains, but I am more cognizant
report that technology-based courses,
of what matters now. Immediate
from the early CAI courses to the
feedback is not essential. In fact, when
latest e-learning courses, are neither
the learning outcomes are complex,
particularly interesting nor engaging.
immediate feedback is not desirable. It
Many courses are little more than
is not necessary that learners move in
a boring “content dump”—like the
linear sequences through small steps.
professor who brought his three-ring
This is likely to slow down and even
binder and told us to “put this on the
impede learning, possibly contributing
computer.”
to the large numbers of students who
Repurposing
fail to complete lessons delivered via
instruction from
from one medium to another
technology. Technology may allow any
one medium to
has never enhanced the
time and any place learning, but this is
another has never
instruction.
not enough. Such is becoming the case
enhanced the
with e-learning.
instruction. Witness the early
A national provider of IT training
Repurposing instruction
videotaped lectures obtained by
reported that only 30 percent of the
placing a video camera in one of the
people who started an e-learning class
seats when a professor lectured. The
completed it, despite numerous emails
video lecture will certainly not be
and telephone calls from the instructor
better than seeing it live. Such was the
and course manager (Van Liew, 2006).
case with early film in which a filmed
The same provider followed this with
play became a movie. It may have
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
122
Michael Allenʼs 2008 e-Learning Annual
been a decent play, but it certainly
to make optimistic predictions about
made a poor movie. A DVD player
e-learning—and even more optimistic
does not improve a filmstrip, nor does
predictions about m-learning, or
a computer improve on a workbook.
mobile learning. However, I also made
Incidentally, much tutorial CAI that
optimistic predictions about CAI and
was thought to be new and wonderful
even more optimistic predictions
was the same instruction students had
about CMI. Really, how many have
received in schools a half-century ago
even heard of CMI? e-Learning can
as workbooks—simply repackaged.
easily have the same future. The
Of course, the research continues to
path to oblivion for any technology
demonstrate no significant differences:
in education is clear: Ignore the
It’s often simply the same instruction
lessons that could and should have
delivered two different ways.
been learned from working with the
So now we add e-learning to
previous technology and let the focus
the mix. If we deliver the same
shift from learning to the technology.
instructional content through e-
Then it is doomed.
learning, we will get the same results. The future of e-learning does not have to be rosy; it may
Future Directions
The optimistic future of e-
follow a long tradition of technology
learning would
in which early optimism waned as
realize the goals
results began to accumulate until
put forward
finally it was dismissed. Then the
for technology in education several
next technology came and the tune
decades ago: Learners would have a
started again. I know no one excited
rich curriculum delivered to them
about instructional radio, yet many
in a form from which they could
fervently embrace pod-casting. I
readily learn. Technology would
know no one excited about filmstrips,
be there whenever they wanted to
yet many praise multimedia. I know
learn, however they wanted to learn,
no one excited about CAI, yet many
whatever they wanted to learn, at
are rushing to e-learning. Pod-
the pace they wanted to learn, and
casting, interactive multimedia, and
it would treat each student fairly,
e-learning can easily go the way of
providing what they needed without
instructional radio, filmstrips, and
prejudice. So what would that future
CAI if we don’t heed the lessons that
look like?
are there for us to see. I am inclined Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
123
My hope for e-learning is that
content, not just distribute the same
the young person in rural Montana
old content to learners as has been
or rural Mississippi would have
the history of repackaging the same
the same access to superb learning
old content for a new technology. I
environments as the young person
would like to see more conversation
in the well-to-do suburban schools
among people applying e-learning that
of Cambridge or Alexandria has
focuses on teaching and learning and
today. I want the worst schools in the
fewer conversations focused on the
country to be almost as good as the
technology. Rather than seeing many
best schools. More broadly, I hope the
e-learning developers taking the same
young person born in West Africa will
first step, I would like to see some e-
have access to fine instruction, not
learning developers who have watched
just the young person born in Finland.
others take the first step and now take
To some extent technology can be a
the second step to move e-learning
much-needed equalizer in educational
forward. I would like to see the move
opportunity.
from e-learning to m-learning, making
I would like to see e-learning take
access affordable and more portable.
the theory and research on human
I would like to see richer lessons in
learning seriously and design lessons
terms of depth of content and quality
accordingly instead of slapping
of pedagogy. I would like to see people
something up on a webpage and
learning with the enthusiasm for
calling it e-learning. I would like to
learning that a four-year old has. I
see our content management systems
want to see e-learning thrive where
become something more than shovels
CAI and CMI did not. I want to see
to move content around. I would like
learners smile.
to see them add some value to the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
124
Michael Allenʼs 2008 e-Learning Annual REFERENCES Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman. Clark, R.E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53, 445-459. Clark, R.E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21-29. Hannum, W.H. (2001). Design and development issues in web-based training. In B. Khan (Ed.) Web-based training. Englewood Cliffs, NJ: Educational Technology Publications. Jenks, M.S., & Springer, J.M. (2002). A view of the research on the efficacy of CAI. Electronic Journal for the Integration of Technology in Education, 1(2), 43-58. Kulik, C., & Kulik, J. (1986). Effectiveness of computer-based education in colleges. AEDS Journal, 19, 81-108. Kulik, J., Kulik, C., & Cohen, P. (1980). Effectiveness of computer-based college teaching: A meta-analysis of findings. Review of Educational Research, 50(4), 525-544. Mühlhäuser, M. (2004). eLearning after four decades: What about sustainability? In Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2004 (pp. 3694-3700). Chesapeake, VA: AACE. Rogers, E.M. (2003). Diffusion of innovation (5th ed.). New York: The Free Press. Schacter, J. (1999). The impact of education technology on student achievement: What the most current research has to say. Milken Family Foundation. Retrieved July 2, 2007, from: www.mff.org/publications/publications. taf?page=161. Van Liew, R. (2006). e-Learning’s “dirty little secret.” Retrieved July 2, 2007, from: http://traininghott.com/E-learning-Secrets.htm.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Thomas C. Reeves Thomas C. Reeves is a professor of instructional technology at The University of Georgia. He has developed and evaluated numerous interactive multimedia programs for both education and training, and given numerous presentations and workshops in the United States and twenty-three other countries. His research interests include evaluation of instructional technology, socially responsible educational research, authentic learning tasks, and instructional technology in developing countries. In 2003, he was the first person to receive the AACE Fellowship Award from the Association for the Advancement of Computing in Education. His Interactive Learning Systems Evaluation book (with John Hedberg) was published in 2003.
William D. Aggen William D. (Bill) Aggen has over twenty-five years of experience producing various forms of e-learning and computer-based training programs in academic, corporate, and entrepreneurial work environments. He is currently an independent consultant performing project management and instructional design activities for national and international clients. Most recently, Bill was founder of Learnwright, Inc., a training-focused technology leader that creates and delivers web-based, multimedia e-learning courses to pharmaceutical and biotech drug and medical device companies operating globally in FDA-regulated industries following Good Manufacturing Practice (GMP) and Quality System Regulations (QSR). He has developed numerous award-winning interactive multimedia programs in a variety of sectors, including military education, science education, continuing medical education, and compliance training.
Lessons from the STARS Revisited
Thomas C. Reeves William D. Aggen
Using their experience with STARS, a literacy training program developed for the U.S. military, the authors extrapolate eight lessons for teams working on e-learning projects. The lessons—dream, work, plan, communicate, evaluate, reward, manage, and enjoy—are critical to the successful development of e-learning projects as illustrated by the lessons learned by the authors more than a quarter century ago. Introduction
Most young
from the interactive CD-ROMs they
adults born
began to play with as children twenty
after 1980 are
years ago.
considered to be members
As members of the Baby Boomer generation, we can remember the
of the generation known variously as
earliest days of interactive video when
Millennials (Howe & Strauss, 2000),
the “Winky Dink and You” television
Generation Y (Martin & Tulgan, 2001),
show premiered on a Saturday morning
or the Net Generation (Oblinger &
in 1953 (www.tvparty.com/requested2.
Oblinger, 2005). Regardless of what they
html). Special pens and a magic plastic
are called, some of the oldest among
screen placed over the TV screen
them have begun to play significant
allowed children to “interact” with host
roles in the design, development,
Jack Barry’s instructions by drawing
and/or implementation of e-learning
a car or decoding a secret message.
programs, either having completed
The interactions were quite crude by
a master’s degree in instructional
today’s videogame standards, but they
technology or a related field, or perhaps
were creatively interactive, especially
entering directly from an undergraduate
considering that the show was
program in computer science or a
broadcast live.
related field. It is doubtful that many
Our experience with the pioneering
of these younger e-learning designers
days of interactive video may have
are aware that interactive video has
planted seeds that came to fruition
been around for more than fifty years.
in 1980 when we served as members
Instead, we suspect that most of
of the University of Maryland
them conceive of the earliest days of
University College (UMUC) team
interactive multimedia as stemming
that collaborated on the design,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
128
Michael Allenʼs 2008 e-Learning Annual
development, and implementation
(about $60,000 today). The STARS
of the STARS (Space Time Army
delivery system featured a Cromemco
Reconnaissance System), one of
microcomputer and Pioneer videodisc
the earliest large-scale interactive
player integrated with state-of-the-art
videodisc programs. Funded by the
graphics cards to enable single screen
U.S. Army, STARS was an interactive
display of video from the videodisc
multimedia simulation designed to
with computer graphics and text from
help soldiers develop better functional
the computer. Soldiers interacted
literacy skills. Our own needs
with the program primarily using
assessment revealed that the reading
a touch screen, but a keyboard and
levels of the technical manuals used by
mouse were also available. The system
the military exceeded the functional
also integrated digitized audio with
literacy skills of as much as 40 percent
sound recorded on the videodiscs.
of the junior enlisted personnel in the
All programs were stored on the ten
Army at that time.
megabyte hard drive (which seemed
Working most of the time at
huge at the time) that had to be
the Heidelberg, Germany, campus
updated using eight-inch diskettes
of UMUC and other times at the
that were truly floppy. The operating
University of Maryland main campus
system was CPM (Control Program for
in College Park, Maryland, we were
Microcomputers) and the authoring
challenged to develop an interactive
language used to create the interactive
learning environment that could be
program was a prototype version of
deployed at remote sites throughout
PILOT (Programmed Instruction,
Europe where U.S Army units were
Learning, or Teaching).
guarding the frontiers of the Cold
While considerable stress was
War. This functional literacy training
experienced by all of the UMUC
program had to be accessible twenty-
team members at the time of the
four hours a day, highly motivating,
development of STARS, largely
efficient, and, most importantly,
stemming from equipment failures,
effective in helping soldiers learn
programming bugs, budget
to read and apply the contents of
constraints, and unrealistic timelines,
sophisticated technical manuals.
both of us, and we suspect all of the
The STARS program was developed
other members of the STARS team,
for delivery via an early prototype
look back on the twenty-four-month
interactive videodisc system that cost
design, development, implementation,
approximately $25,000 per unit in 1980
and evaluation effort as one of the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
129
most exciting professional activities
At least part of this problem may
we have ever experienced. Inevitably,
stem from the failure to do what the
we learned many lessons in the
Disney people call “imagineering,” that
process, and thus the purpose of this
is, coming up with original solutions
article is to look back and review those
to design challenges, solutions that
lessons, partly as a reminiscence for
combine the best of what we know
others like us who participated in
with the best we can envisage. To
those trailblazing days of e-learning,
allow us to maximize our creativity in
but more so for members of the Net
the STARS project, we held a week-
Generation who have been challenged
long brainstorming session involving
to create the e-learning counterparts
all design team members and a
to STARS at this time.
professional facilitator. During this session, no ideas were too outlandish
Lesson One— Dream
The first lesson
as we covered scores of large sheets
we learned
of paper with drawings and words
from the STARS
to express our “thinking outside the
project is that
box” design concepts. Of course, this
designing
“imagineering” session was only held
an engaging interactive simulation
after several
requires imagination and creativity.
weeks of needs
Designing an engaging
Ever since the dawn of computer-
assessment,
interactive simulation requires
aided instruction (CAI) on mainframe
learner analysis,
imagination and creativity.
computers in the 1960s, a very
and other front-
frequent complaint has been that
end work (Rossett, 1987, 1999).
interactive learning programs all too
The research literature provides
often simply duplicate what could have
numerous tools and strategies for
been done with less expensive media.
encouraging creativity (Adams, 1987;
Even today, many web-based learning
de Bono, 1992), but the sine qua non is
environments resemble page-turning
freedom from judgment. An important
print materials or at best interactive
aspect of the STARS brainstorming
PowerPoint presentations. Despite the
week was that we designed the ideal
availability of valid design principles
program before considering the real.
(Aldridge, 2006; Allen, 2003; Clark &
We encouraged all of those present
Mayer, 2003; Quinn, 2005; Rosenberg,
during the week to ask themselves
2006), most e-learning programs still
how they would help soldiers learn
fail to be engaging.
functional literacy skills if they were
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
130
Michael Allenʼs 2008 e-Learning Annual
completely unfettered by spatial,
computer graphics were state-of-
temporal, or financial constraints.
the-art at the time. For the World
So what was creative about STARS?
War II scenario, actual World War II
First, the essence of the STARS
footage was edited together with a
program was a time-travel simulation
superimposed robotic assistant, and
that involved soldiers in a series of
for the future scenario, video was
elaborate interactive adventures in the
recorded in buildings that were as
past and the future. In the first part
“futuristic” as we could find in and
of STARS, soldiers traveled back in
around Washington, D.C., in 1980.
time to the Battle of the Bulge during
Our budget only allowed us to hire one
World War II, and in the second part,
professional actor, but we were able to
soldiers leaped three “xergons” into
recruit several volunteer actors from
the future to avert a catastrophic
the University of Maryland campus in
conflict. In both adventures, soldiers
College Park, Maryland, where most of
confronted realistic problem-solving
the actual production was completed.
situations that required application
Additional scenes for the second part
of a range of functional literacy skills
of STARS were filmed on a military
to solve. If a soldier demonstrated
base in Colorado using real soldiers
the required skill and thereby solved
operating their actual equipment.
the problem, he/she advanced in
Interestingly, although the
the simulation, earning “time-warp
production values would seem
energy units” that would be needed
unpolished today, we will make
eventually to return to the present. If
the argument that the pedagogical
the soldier’s performance indicated a
(or instructional) design of STARS
lack of a certain literacy skills, he/she
was advanced and would even be
would be branched into an interactive
seen that way today. The major
lesson keyed to that specific skill
instructional model incorporated
taught within a military context. After
into STARS was learning through
successfully completing a mini-lesson,
authentic tasks, a line of research that
the soldier would return to the STARS
we are still pursuing (Herrington,
adventure for another try at solving
Reeves, & Oliver, 2006). Although
that particular problem.
they were presented in an overall
Although the actual program as
“fantasy” scenario, the authentic tasks
displayed on the screen would seem
incorporated into STARS had real-
somewhat crude if judged by today’s
world relevance for the soldiers who
production values, the video and
used the program. Assessment of their
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
131
literacy skills was seamless with the
Kidder’s award-winning 1981 book,
authentic tasks. Even the lessons to
The Soul of a New Machine, about the
which poor performance in the STARS
development of a new computer at the
simulation would link soldiers were
Data General Corporation, we readily
designed using design principles that
identified with the passion and effort
are still advanced in comparison to
with which the Data General engineers
many of today’s e-learning products.
pursued their ambitious goals. For
At the time of the STARS creation,
example, during the video production
we felt that we were barely scratching
phase of STARS, key staff members,
the surface of what could be done with
including both of us, worked six
interactive multimedia to support
straight weeks without a day off,
learning. Perhaps our proudest
averaging ninety or more hours
achievement is the fact that several
per week. Given the temporal and
award-winning multimedia producers,
financial constraints and the novelty
who came into their own in the mid-
of what we were trying to achieve,
to late-1980s, told us that their work
no amount of planning or time
was partially inspired by what they had
management could have prevented
seen in the STARS program.
these extreme demands from arising. Can effective e-learning programs
Lesson Two— Work
The second
be created within the boundaries of
lesson from
a normal work week? We think they
STARS is
can, especially if time and money are
closely related
not major constraints. However, we
to the first. To
have very rarely
paraphrase Thomas Edison, creating
experienced
Can effective e-learning
an effective e-learning environment
normal work
programs be created within
is 1 percent inspiration and 99
weeks during
the boundaries of a normal
percent perspiration. Throughout
e-learning
work week?
the STARS project, we abandoned
development
the normal forty-hour work week
projects, at least during the most
and often worked far into the night
creative phases of a significant
and throughout our weekends. Most
development initiative. Clearly,
of the key members of the STARS
obtaining individual and institutional
team “signed on” to this project with
support for the kind of commitment
enormous commitment and unflagging
we and our colleagues made to STARS
energy. When we later read Tracy
is more difficult or even impossible
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
132
Michael Allenʼs 2008 e-Learning Annual
in certain types of contexts. For
The PERT chart provided all team
example, in a military training
members with a reference point with
setting, both military and civilian
which to coordinate activities and
personnel are usually responsible for
gauge project progress and, on several
many duties outside the boundaries
occasions, helped us to avert major
of an e-learning development effort
scheduling conflicts that would have
that may prevent their participation
caused significant delays.
in overtime hours. This is why
Today’s e-learning developers have
institutions of higher education and
the advantage of powerful computer-
innovative start¬up companies often
based planning software available
are ideal partners in the design and
from a large range of large and small
development of cutting-edge “proof of
vendors. We have found that the
concept” design projects.
purchase of project management and planning software is only worthwhile
Lesson Three— Plan
The third lesson
if it is matched by an investment
learned from
in learning the ins and outs of
STARS is that a
the software. We are reluctant to
little planning
recommend any specific products
goes a long way
because the market is changing so fast
and more planning goes further. As
and also because frankly we have yet
noted above, no amount of planning
to find the ideal software ourselves.
can avoid unusual demands for time
Adding someone with management
and effort during a truly creative
and planning expertise to an e-
initiative, but inadequate or poor
learning development team, perhaps
planning can foster situations that no
a young M.B.A. graduate willing to
amount of overwork can overcome.
cut his/her teeth on an ambitious
One of the most valuable activities
design initiative, may also be a good
early in STARS was the development
investment. At the very least, hiring an
of an exhaustive PERT (Program
honest and competent consultant for a
Evaluation and Review Technique)
few days or weeks of up-front planning
chart for the project (Cook, 1971).
assistance may also be helpful early
Producing this plan consumed several
in a creative development initiative.
days of key staff time and the services
This was certainly our experience with
of an excellent planning consultant,
STARS.
but it was time and money well spent.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
Lesson Four— Communicate
133
Communication is one of those
“signed on” to the project. As described in Kidder’s (1981)
“buzz words”
book, The Soul of a New Machine,
that finds
“signing-on” is the process by which
its way into
people establish a personal stake in
virtually every human activity, and
a project. The nature of e-learning
thus it seems somewhat trite to
development is that various personnel
state that the STARS team members
will enter and leave the project at
worked hard to maintain open lines
different intervals, while others remain
of communication. And yet, we must
engaged throughout an initiative with
admit that of all the problems we faced
a much greater opportunity to develop
during STARS, the most frustrating
“ownership” of a project. It is no easy
was poor communication between and
task to manage the efforts of ever-
within subgroups of the project team.
changing personnel from an extremely
Although job descriptions were
wide variety of
flexible and several of us fulfilled
backgrounds.
There must be opportunities
several roles simultaneously, the
There must be
for project personnel
STARS team was roughly divided
opportunities for
to meet together to air
into four functional subgroups: (1)
project personnel
misunderstandings.
instructional design, (2) videodisc
to meet together
production, (3) programming, and
to air misunderstandings and vent
(4) implementation. (Management
frustrations. We found that an outside
and evaluation functions were carried
facilitator and a change of location
out by personnel who belonged to all
to a forest retreat encouraged the
the teams.) Numerous efforts were
level of frank communication needed.
made to improve communication,
Exchanging roles from time to
including the establishment of an
time in an effort to walk in another
early form of a listserv, but serious
person’s shoes for a while also helped.
misunderstandings arose from time to
We encouraged an atmosphere of
time. Some of the misunderstandings
mutual teaching and learning on
stemmed from project personnel being
the part of all personnel engaged
located on both sides of the Atlantic
in the STARS initiative, a strategy
Ocean in Heidelberg, Germany, and
that helped to resolve some of the
College Park, Maryland, but the major
communication problems. But again,
contributors were inequities in the
variance in “signing on” led to most
degree to which various personnel had
of our communication problems, and
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
134
Michael Allenʼs 2008 e-Learning Annual
working those differences out is rarely
the development of a creative and
easy.
effective interactive videodisc learning
In today’s increasing “flat world”
system. To be sure, we were aware of
(Friedman, 2005), e-learning
the design principles derived from
projects often involve international
learning, communications, and media
collaborators wherein subject-matter
theories at the time, but they did not
expertise may reside in one country,
add up to instructional formulas or
instructional design in another, and
prescriptions that could be blindly
production and programming in yet
followed. Instead, numerous types
a third country. Although we now
of review, needs assessment, and
have access to sophisticated Internet
formative evaluation were required to
meeting tools such as WebEX (www.
drive the instructional design process
webex.com) and its competitors,
(Reeves & Hedberg, 2003). No amount
communication may be more difficult
of creative effort can yield e-learning
than ever, especially when different
programs immune to the refinement
cultures are involved. We have
process provided by rigorous
reviewed several e-learning programs
formative evaluation.
in recent years developed for Fortune
Most importantly, formative
500 companies by international
evaluation should be focused on
development teams and found that
products, projects, and programs,
the resultant products were seriously
not directly on people. A formative
flawed. We suspect that more than
evaluation focused on products and
a few of the flaws stemmed from
processes rather than personnel was
miscommunications.
the strategy we used in STARS and it is one we have advocated for the past
Lesson Five— Evaluate
Another
twenty-five years and more. Another
important
important principle related to the
lesson from
“evaluate” lesson is that evaluation
STARS is that
methods should be aligned carefully
instructional
with evaluation questions that are
design without continuous evaluation
keyed to specific design decisions
is a waste of time. In 1980, we knew
that the development team faces.
precious little about the science of
Evaluation is not an end in itself, but a
instruction or the art of design that
means to better decision making.
would dictate or even adequately guide
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
Lesson Six— Reward
135
The next STARS
important for us than the financial
lesson concerns
benefits.
how to reward
Long hours and enormous effort
members of
inevitably take their toll, and even the
a creative e-
most dedicated team members require
learning development team for the
respites from time to time. In addition
extraordinary commitments that
to vacations, we have found that
they may be called on to make to a
attending professional conferences
project. Although the first reward
to demonstrate the products of our
that may come to mind is financial
labors has been a great reward for
remuneration, we found this to be
ourselves and others. e-Learning
a relatively insignificant source of
project management personnel should
reward during the STARS project. We
share such professional presentation
were all working for the University
opportunities with as many team
of Maryland University College,
members as possible. In academic
an institution that paid decent, but
contexts, publications are usually
certainly not high, salaries. Of course,
important, and thus opportunities to
psychologists have reported that
publish articles about various aspects
job satisfaction is usually derived by
of an e-learning design initiative
several factors of which money is only
should also be encouraged.
one (Spector, 1997). We found that one of the most
Although it may sound naïve in the 21st Century, we have always
rewarding aspects of the STARS
found that intrinsic rewards are much
project was the freedom to see
more powerful
things in new ways and express our
than extrinsic
Personal satisfaction with a job
creativity (see Lesson One above).
ones. Personal
well done and the simple joy of
The creative experience of designing
satisfaction with a
collaborating with others in a
STARS and numerous e-learning
job well done and
creative design process are the
projects since then has been a major
the simple joy of
best rewards.
factor that has kept us and many of
collaborating with
our STARS teammates in the field
others in a creative design process
of instructional technology for the
are the best rewards. Twenge (2006)
quarter century plus since STARS.
has conducted extensive research
The psychic rewards of collaborating
that supports the conclusion that the
on cutting-edge projects with other
workers in the current Net Generation
creative people have been much more
may not be as turned on by intrinsic
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
136
Michael Allenʼs 2008 e-Learning Annual
rewards as the Baby Boomer
as the failure resulted from sincere
generation was, a trend that may
efforts to achieve something new and
prove to be detrimental to the types of
more effective.
advances needed in the design and use of e-learning.
On STARS, we found that another essential function of management was managing relationships with
Lesson Seven— Manage
One of the
people and organizations outside the
STARS team
development team. Frankly, to be
members said,
creative in an academic environment
“Finding people
sometimes demands stretching, or
who can work
even breaking, “the rules” from time
together as a team is probably the
to time. For example, occasionally
most important thing a manager
purchases must be made in an
can do.” We learned in STARS that
expedient manner that may not
the essence of good management is
conform to state or university
team building. Creative individuals
purchasing procedures. Good planning
inevitably have a wide variety
(see Lesson Three) can eliminate
of personalities, attitudes, work
some of these difficulties, but a truly
styles, and communication skills.
creative project will usually involve
Instructional design, programming,
some unusual “tests” of the system.
video production, and other e-learning
Managers may also have to buffer
development roles often attract widely
the development team from problems
diverse types of people, and there is
involving subcontractors and
also considerable variance within each
commercial vendors. It would be ideal
of these subgroups. Project managers
if e-learning designers could always
must continually strike a balance
count on hardware and software
between individual rewards and team
systems to perform as advertised,
recognition. It is also important that a
but this has not been our experience
manager of any project that demands
in the real world of rapidly evolving
creativity not over-manage. e-Learning
interactive technologies. In some
project managers must be prepared to
ways, STARS was like creating a
take risks and give staff members the
motion picture using film that may
freedom to do the same. When things
not develop that is intended to be
go wrong, as they inevitably will at
shown on a projector that hasn’t been
some point, managers must strive to
invented yet. For example, getting a
support their team members as long
videodisc pressed in those days took
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
137
months! The authoring tools we were
(Friedman, 2005). e-Learning has
using were rife with bugs, many of
the potential to engage learners in
which we were the first to discover.
ways that have barely been imagined
All in all, these and other problems
(Allen, 2003). All that is needed to
added to the excitement of the STARS
help e-learning reach its full, and
project. (Or at least that is what we
as yet unrealized, potential is the
are able to tell ourselves after so many
creative genius and committed effort
years.)
of individuals and teams turned on to the excitement and rewards of meeting
Lesson Eight— Enjoy
The final lesson
this challenge.
from the STARS project is that,
Summary
Dream.
if designing
Work. Plan.
e-learning
Communicate.
ceases to be fun, it is probably time
Evaluate.
to move on to something else. There
Reward.
are few kinds of work that are always
Manage. Enjoy. The STARS team was
enjoyable, but if designers lose the joy
surely not the first group to come
of designing, they become stale and
up with lessons such as the ones
their work shows it. The enjoyment
summarized above. These lessons go
of working in e-learning design flows
back centuries. However, it seems
from several aspects, including the
clear that they must be relearned
camaraderie of teamwork, the sense
to some extent in each new context
of achievement that stems from
and time. Perhaps some wisdom can
meaningful effort, and mutual respect
be passed on from those of us with
for creative ideas. As the Disney
“experience.” We hope that this article
people would have it, there is joy in
serves to stimulate the new Net
“imagineering” (Kurtti, 2006).
Generation of e-learning designers to
The need for more effective education, training, and performance
reach for the stars and go much, much further than we did.
support around the globe is enormous
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
138
Michael Allenʼs 2008 e-Learning Annual REFERENCES Adams, J.L. (1986). The care and feeding of ideas: A guide to encouraging creativity. Reading, MA: Addison-Wesley. Aldridge, C. (2006). Simulations and the future of learning: An innovative (and perhaps) revolutionary approach to e-learning. San Francisco, CA: Pfeiffer. Allen. M. (2003). Michael Allen’s guide to e-learning: Building interactive, fun, and effective learning programs for any company. Hoboken, NJ: John Wiley & Sons. Clark, R.C., & Mayer, R.E. (2003). e-Learning and the science of instruction. San Francisco, CA: Jossey-Bass. Cook, D. L. (1971). Educational project management. Columbus, OH: Charles E. Merrill. de Bono, E. (1992). Serious creativity. New York: HarperCollins. Friedman, T.L. (2005). The world is flat: A brief history of the twenty-first century. New York: Farrar, Straus and Giroux. Herrington, J., Reeves, T.C., & Oliver, R. (2006). Authentic tasks online: A synergy among learner, task, and technology. Distance Education, 27(2), 233 -247. Howe, N., & Strauss, W. (2000). Millennials rising: The next great generation. New York: Vintage Books. Kidder, T. (1981). The soul of a new machine. New York: Little, Brown. Kurtti, J. (2006). Walt Disney’s legends of imagineering and the genesis of the Disney theme park. New York: Disney Editions. Martin, C.A., & Tulgan, B. (2001). Managing generation Y. Amherst, MA: HRD Press. Oblinger, D., & Oblinger, J. (Eds.). (2005). Educating the Net Gen. Washington, DC: EDUCAUSE. Quinn, C. (2005). Engaging learning: Designing e-learning simulation games. San Francisco, CA: Pfeiffer. Reeves, T.C., & Hedberg, J.G. (2003). Interactive learning systems evaluation. Englewood Cliffs, NJ: Educational Technology Publications. Rosenberg, M.J. (2006). Beyond e-learning. San Francisco, CA: Pfeiffer. Rossett, A. (1987). Training needs assessment. Englewood Cliffs, NJ: Educational Technology Publications Rossett, A. (1999). First things fast: A handbook for performance analysis. San Francisco, CA: Pfeiffer.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
Spector, P.E. (1997). Job satisfaction: Applications, assessment, causes and consequences. Thousand Oaks, CA: Sage. Twenge, J.M. (2006). Generation me: Why today’s young Americans are more confident, assertive, entitled—and more miserable than ever before. New York: The Free Press.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
139
M. David Merrill M. David Merrill is an instructional effectiveness consultant, a visiting professor at Florida State University, a visiting professor at Brigham Young University–Hawaii, and professor emeritus at Utah State University. Since receiving his Ph.D. from the University of Illinois in 1964, he has served on the faculty of George Peabody College, Brigham Young University-Provo, Stanford University, the University of Southern California, and Utah State University. He is internationally recognized as a major contributor to the field of instructional technology, has published many books and articles in the field, and has lectured internationally. Among his principle contributions: TICCIT Authoring System (1970s), Component Display Theory and Elaboration Theory (1980s), Instructional Transaction Theory, automated instructional design and ID based on Knowledge Objects (1990s), and currently First Principles of Instruction. He was honored to receive the AECT Life Time Achievement Award. He and his wife Kate together have nine children and thirty-seven plus two (by marriage) grandchildren, which he claims as his most important accomplishment.
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
M. David Merrill
In this article, the author chronicles his career, from his interest in making education more effective to an epiphany about theories. His extensive experience with computer-assisted learning spans from early efforts to teach concepts to more recent work with automated authoring systems. His recent work attempts to identify underlying principles common to most theories of instruction. The author suggests that there should be less emphasis on merely convey information using technology and more emphasis on how to make instruction effective, efficient, and engaging. The Beginning
This article
change my major from engineering to
is a very
education so that I could learn how to
autobiographical
be an effective teacher.
account of my
My major in secondary education
career. Perhaps
was a frustrating experience. While
my personal quest for understanding
I learned how to minimize liability,
the nature of instruction will be
how to create a resource file, how to
instructive to those of you who are
pass out papers with a minimum of
also pursuing this important question.
confusion, and other necessary skills
While I have learned much, I have also
for a classroom teacher, I can only
realized that there is much more to
recall one or two lectures that dealt
learn. If I had a wish, it would be to
with the topic of effective instruction.
be able to begin my career again while
I had a wonderful experience as an
retaining what I have learned.
unconventional student teacher, but
From July of 1957 until July of
caused considerable consternation
1959 I found myself in Indiana,
for the principal of the school where
Ohio, and Michigan, where I served
I was assigned. I was advised that I
as a missionary for the Church of
probably would not survive the public
Jesus Christ of Latter-Day Saints.
schools. It occurred to me that perhaps
Some of our missionaries were very
my decision to major in education
effective teachers; for others, their
was a mistake and that I should seek
attempts to share our message was an
another career path. As I expressed my
embarrassment. I determined that when
frustration to one of my professors,
I returned from this experience, I would
he gave me a challenge that put me
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
142
Michael Allenʼs 2008 e-Learning Annual
on the career path that I have since
scene. Binary arithmetic, base 8,
pursued. In response to my suggestion
base 16, and other representations of
that I abandon education, he simply
numbers were not in the repertoire of
stated, “You could pursue another
a small-town undergraduate student
career, or you could realize that there
scrambling to complete his bachelor’s
is much that needs to be done. Perhaps
degree.
you could make
This particular class in number
Perhaps you could make
positive change
theory was, for this student, a unique
positive change in the field.
in the field.” He
math class: no problems to work, no
suggested that
homework, a very small textbook. At
funding was available to pursue a
the end of each lecture the professor
Ph.D. Not being one to back away
merely said, “Think about it!” Think
from a challenge, I decided to take up
about what? How do you think about
the gauntlet. Applications to several
mathematics? In desperation, and
major universities resulted in a full-
as a substitute for thinking, I read
ride NDEA three-year fellowship to
the textbook every week. It wasn’t
the University of Illinois.
difficult; it had only ninety-seven
In my final semester of
pages and a bright yellow cover.
undergraduate work, a check with
However, the concepts presented
the registrar showed that I was short
floated over my head like clouds in
one hour of credit for my minor in
the sky. I had no idea what the course
mathematics. Naturally there was
was about or what the text was about.
no one-hour math classes, so it was
Each week a lecture, the injunction
necessary to enroll in a three-hour
to “Think about it!” and another read
class. The University of Illinois had
through the text.
already awarded me a fellowship
The midterm exam was a disaster. It
for my Ph.D. study; therefore
had no problems to work, only a single
the completion of the additional
question: “Invent a number system.”
math class assumed considerable
Invent a number system? What in the
significance for my future. A class
world does that mean? In true survival
in number theory appeared, on
mode I wrote for the whole two hours.
the surface, to be the easiest path
However, it didn’t fool the professor.
to the necessary credit. The year
There were seven students in the
was 1961. New mathematics in the
class; there were seven F’s on the
public schools was still in the future.
midterms. When we objected, the only
Computers were just coming on the
explanation from the professor was,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
“Think about it!” My anxiety was at an all-time high.
proceeded to define a binary number system with two elements, an oar and
My graduate career was about to be
a rubber boot. I was in the professor’s
terminated before it began by the
office the next day to see if I was going
unnerving command, “Think about
to graduate school or not. He handed
it!” I tried every avenue of escape:
me my paper with a large red A
Another class? Getting the registrar to
written across the top. I thanked him,
waive the credit? Home study? There
breathed a sigh of relief, and vowed to
were no other options. My bachelor’s
never take another math class as long
degree, and hence my entrance to
as I lived.
graduate school, were both riding on a
My first year of graduate school
class in which I had received a failing
was very difficult. Not only was there
grade on the midterm and, worse,
a tremendous amount of work, but
a class that was to me completely
there also seemed to be too many
incomprehensible.
contradictions. The content of
Somewhere in the thirteenth week,
learning psychology challenged many
the light came on. Number systems
of my fundamental beliefs. There
are inventions. They are not natural
were numerous contending systems,
phenomena. Number systems are like
each claiming to explain learning. I
any other invention: an assembly line,
struggled for days trying to explain
an organization. A number system is
learning of the concept green using
merely a system of logic consisting
only S-R (stimulus-response) bonds.
of premises and conclusions. Base 10
I found myself in the basement of the
is only one of many possible number
psychology building feeding rats that
systems. Base 10 numbers are useful
were on a deprivation schedule. Why
for many everyday things, but other
was I feeding rats when I wanted to
systems might be equally useful.
know how to teach children? I was
The day of the final arrived. My anxiety was still high, but at least I thought I understood. You guessed it,
about ready to give up and look for a real job. About this time B.F. Skinner visited
only one question, “Invent a number
the campus. Like my fellow classmates,
system.” Either I understood or it was
I went to hear the great man. I don’t
too late. My future graduate studies
remember any details of his lecture,
depended on my ability to invent
but his response in the question-
a number system. So I wrote, “Let
and-answer period changed my life.
there be an oar and a rubber boot.” I
A member of the audience said, “Dr.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
143
144
Michael Allenʼs 2008 e-Learning Annual
Skinner, in your book (which he
how closely it fits, but none are reality,
named) you said such and such (some
merely inventions.”
detail of Skinner’s theory); but tonight
I returned to my studies with
you seemed to contradict yourself by
renewed enthusiasm. I looked upon
saying such and such”—he quoted a
all theories as artificial systems and
part of Skinner’s speech.
found them fascinating. I stopped
“Hell,” said Skinner, “do you think I believe everything I ever wrote?” This was a great “Do you think I believe everything I ever wrote?”
trying to make all theories agree and force them to form one great truth. It became a game to see if I could
insight for me.
identify the theorist’s1 assumptions
Here was a great
and conclusions. It was fascinating
author saying he
to observe that some systems were
changed his mind and now disagrees
carefully constructed and logical,
with his earlier self. However, what he
while other systems were very loosely
said next changed my life.
constructed and often violated the
“What I’ve tried to do,” continued
canons of logic. I realized that theory
Skinner, “is to make only a few
building is our puny attempt to
assumptions and then see how much
understand our world by inventing
of human learning we can explain with
artificial systems and trying them out
only these assumptions.” He went on
against the world.
to defend his theory and the point he
Later in my graduate career I
made in his speech. I stopped listening
had one additional insight. We
before he ended his explanation.
were studying learning and some
“Good grief,” I thought, “psychology
instructional theories. It was apparent
is just an oar and a rubber boot as
that learning theories tended to
well.
explain how persons acquire and store
Psychological systems are not reality
knowledge, but they have very little
either, but merely logical systems that
to say about how an instructor should
try to explain what we observe in the
structure and sequence knowledge
real world. Behavior is merely one
to promote efficient and effective
logical system that is tested against
learning. It occurred to me that one
reality to see how good a match can
could build a logical system, a theory,
be found. Just like there can be many
about instruction. So I said, “Let there
different number systems, there can be
by an instructional oar and a rubber
many different psychological systems.
boot” (from the preface of Merrill,
Each is tested against reality to see
1994).
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
In Pursuit of e 3 (Effective, Efficient, Engaging) Instruction
Early in
I had begun to prepare a proposal
my career I
for doing research on concept
realized that
teaching. Bob Tennyson, whom I had
instructional
not previously met, walked into my
design consisted
office and announced, “I’ve come to
of two primary
work with you.” I indicated that I had
activities: what to teach and how to
no research funds and would be unable
teach. The goal became to build a
to pay him for his work. He offered to
system of instructional design that
work merely for the experience. “Have
followed Skinner’s strategy, that is,
you ever written a proposal?” I asked.
make as few assumptions as possible,
He had not, but was sure that he
form some prescriptive statements
could. I handed him my notes, the call
about instructional design, and see
for a proposal from NSF, and sent him
whether we can prescribe a wide
away. Two days later he put a draft
variety of instructional strategies from
proposal on my desk. It was terrible,
these fundamental components.
but it was a draft. After considerable discussion and several drafts, we
Teaching Concepts
In 1968, I
submitted the proposal. It funded
returned to
Bob’s graduate education for the next
Brigham Young
three years.
University
Our first study (Tennyson,
from a year
Wooley, & Merrill, 1972) found
as a visiting assistant professor at
exceptional results. The best strategy
Stanford University. I was working on
for concept teaching consisted of
an instructional strategy for teaching
presenting a definition, presenting
concepts. My studies of formal
matched examples and non-
concept learning in graduate school
examples, presenting a divergent
suggested that learning a concept
set of examples, and using an easy-
involved presenting examples and
to-hard sequence of examples. Our
non-examples (Bruner, Goodnow,
measure was correct classification
& Austin, 1956). Most of these
of subsequent randomly sequenced
laboratory studies had used formal
examples and non-examples. We
concepts—red circles, green triangles,
also measured classification errors
and so forth. Very little had been done
of over¬generalization, under-
to investigate teaching concepts using
generalization, and misconception.
real-world tasks.
We compared four treatment groups.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
145
146
Michael Allenʼs 2008 e-Learning Annual
We hypothesized that the group that
first volume of AERA Review of
did not have matched examples and
Research in Education (Merrill &
non-examples would over-generalize—
Boutwell, 1973). As we examined
they did; the group that did not have
the literature, it became evident
a divergent set of examples would
that different investigators often
under-generalize—they did; and the
used the same words in reference to
group that had examples that shared
completely different strategies. In
an irrelevant attribute would form a
order to make prescriptive statements
misconception—they did.
about objectives (what to teach) and
We subsequently conducted
instructional activities (how to teach),
additional studies refining our
it was evident to us that there needed
strategies using different age
to be a descriptive language that
subjects and different concepts
allowed precise description of these
(Merrill & Tennyson, 1977a). Our
two aspects of instructional design.
research continued to support our
A way to precisely describe what was
winning strategy. Tennyson spent
taught and how it was taught was
the next several years continuing the
needed.
investigation of concept teaching
While in graduate school I had the
(Tennyson & Cocchiarella, 1986;
opportunity to read The Conditions
Tennyson & Park, 1980). We described
of Learning in manuscript before it
and illustrated our instructional
was published (Gagne, 1965). My
strategy in a book that has since
search for prescriptive guidelines
guided the effective, efficient, and
for effective, efficient, and engaging
engaging design of concept lessons for
instruction was rewarded. I
many designers (Merrill & Tennyson,
immediately resonated with Gagne’s
1977b; Merrill, Tennyson, & Posey,
categories of learning and his
1992). It is still a very effective
assumption that different kinds of
instructional strategy.
learning required different kinds of strategy for effective presentation
The Task/ Content Matrix
In 1972, it was
and assessment. Most of my previous
a surprise and
explorations in learning theory
an honor to
attempted to explain all learning with
be invited to
a single set of principles. It was also
contribute a
difficult to translate these theories
chapter, Instructional Development:
of learning into prescriptions for
Methodology and Research, for the
instruction.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
Emotional
Emotional (Signal Learning)
Psychomotor
Memorization
Complex Cognitive
147
Topographic
Chaining
Complex Skill
(Stimulus Response)
(Chaining)
(new)
Naming
Serial Memory
Discrete Memory
(new)
(Verbal Association)
(Multiple Discrimination)
Classification
Analysis
Problem Solving
(Concept Learning)
(Principle Learning)
(Problem Solving)
Table 1. Ten Categories of Learned Behavior
As we attempted to apply Gagne’s
more logical to us to separate the
1965 categories, we felt that they were
content to be learned from the
incomplete. I had previously attempted
performance of the student with
to clarify his categories by proposing
regard to the content. We suggested
a two-dimensional scheme (Merrill,
three levels of performance: remember
1971) (see Table 1). On one dimension
content, use content, and find new
I put a unit, a chain, and multiple
content. Following Gagne’s lead, we
chains. On the other dimension I
suggested four kinds of content: facts,
included emotional, psychomotor,
concepts, procedures, and principles.
memorization, and complex cognitive
The result of our reflection on
learning. Gagne’s categories are in
how to describe what to teach was
parentheses.
a performance/content matrix for
As we attempted to describe the
classifying instructional outcomes,
content involved in the studies we
at least in the cognitive domain (See
were reviewing, we felt that there were
Table 2 on next page).
some categories missing in the Gagne
Gagne’s category of concept
scheme. His multiple-discrimination
classification corresponded to our
accounted for remembering facts but
use-concept cell; his rule-using
did not obviously include categories
category corresponded to our use-
for remembering the definition of
procedure cell; and his higher order
a concept, remembering the steps
rule category corresponded to our use-
in a procedure, or remembering the
principle cell. Gagne’s problem solving
statement of a principle. It seemed
or higher order rules did not seem
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
148
Michael Allenʼs 2008 e-Learning Annual
Table 2. The Performance – Content Matrix
Find
Use
Remember
FACT
CONCEPT
PRINCIPLE
PRINCIPLE
sufficient for creative behavior with
analysis approach. In a lecture to my
regard to concepts, procedures, and
instructional design class, I proposed
principles. The learned performance
a whole-task approach in which the
of defining a new concept (an activity
analysis started with the simplest
essential to science), of inventing a
version of a whole task and then
new procedure (an activity common to
with successive cycles of instruction
engineering and especially computer
gradually elaborated this task by
programming), and discovering a new
adding complexity. Charlie Reigeluth
principle (the primary goal of science)
was in the class and volunteered
did not appear to be adequately
to write a version of my lecture for
included in Gagne’s scheme. We
publication. This started our work on
added a find performance category to
elaboration theory (Reigeluth, Merrill,
accommodate these types of learning.
Wilson, & Spiller, 1980). Reigeluth went on to investigate this approach
Elaboration Theory
Gagne’s
in considerable detail (Reigeluth, 1983,
hierarchy
1999a). Elaboration theory started
analysis is a
our thinking about whole tasks that
bottom up
led to the principle of task-centered
prerequisite
instruction in our more recent work.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
My father was an artist—a landscape
teacher stresses this observation in
painter. As a small boy I wanted to
class—but I didn’t realize this until
be an artist like my father. I thought
some years later.) I also observed
that if I could just have a large box
that there were really only two things
of Crayola Crayons, I would be able
that an instructor could do with
to paint beautiful landscapes like my
content: present it to the student
father. For every birthday, I would
or ask the student to remember or
request a large box of crayons. I was
use the content. Combining these
unaware of the precarious financial
two dimensions led to the primary
affairs of my parents. My dad never
presentation form matrix. The
said they couldn’t afford a large box
content dimension had two values:
of crayons. He merely sat down with
generality (for the general case) and
four crayons—red, yellow, blue, and
instance (for the specific case). On
black—and a piece of paper and
the instructional dimension we called
colored a beautiful landscape. By
presenting expository and application
example, he taught me the primary
inquisitory. This led to four primary
colors and mixing pigments. For my
presentation forms (PPF): expository
birthday I would get a box of eight
generality (EG), expository instance
Crayola Crayons. My dad would point
(Eeg), inquisitory generality (IG) and
out that I had four more crayons than
inquisitory instance (Ieg). We used
I needed to paint beautiful landscapes.
the symbols in the parentheses as
I didn’t become an artist, but I never
shorthand for describing instructional
forgot my primary colors and how to
strategies. This allowed us to
combine them to get all the colors of
unambiguously describe almost any
the rainbow.
instructional strategy in terms of the
As I attempted to find a way to
primary presentation forms involved.
describe instructional strategies, it
For example, EG1, EG2, EG3, etc.,
occurred to me that there must be
describes a lecture presenting a series
some primary presentation forms from
of generalities without examples.
which all instructional strategies could
The numbers indicate different
be constructed and thus described. As
generalities, as contrasted with a
I studied the instructional strategies
repeat of the same generality. The
in the literature we had collected,
strategy EG1 Eeg1, EG2 Eeg2, etc.,
it occurred to us that there are two
describes a presentation of a series
levels of content: a general level and
of generalities, each followed by a
a specific level. (Every composition
corresponding example. A discovery
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
149
150
Michael Allenʼs 2008 e-Learning Annual
strategy might be represented by Ieg1,
presentation form comprised a display
Ieg2, Ieg3, IG from which the learner is
to the student. These displays are
to discover the generality from a series
the components of an instructional
of examples that he or she must figure
strategy, hence, component display
out.
theory.
We also realized that there were
CDT, as it came to be called,
secondary presentation forms (SPF)—
consists of three parts: (1) a scheme
teaching activities that enhanced
for describing the content to be
learning but were not directly related
taught–the task/content matrix; (2)
to presenting or applying the content.
a scheme for describing instructional
Our attempts to unambiguously
strategies—primary presentation
describe instructional strategies also
forms, secondary presentation forms,
caused us to recognize a number
and interdisplay relationships; and (3)
of relationships among the primary
a set of rules that relate the two. CDT
presentation forms that we called
identifies which combinations of PPF,
interdisplay relationships (IDR).
SPF, and IDR are most appropriate for teaching each cell in the task/content
Component Display Theory (CDT)
During the 1970s, I was
matrix. Selecting Instructional Strategies
director of the
and Media (Merrill & Goodman, 1972)
Brigham Young
presented an early version of CDT.
University
The Instructional Strategy Diagnostic
Division of Instructional Research,
Profile (Merrill, Reigeluth, & Faust,
Development, and Evaluation. Our
1979; Merrill, Richards, Schmidt,
students during this period conducted
& Wood, 1977), later called The
scores of research studies exploring
Instructional Quality Inventory (Ellis,
many aspects of the task/content
Wulfeck, Merrill, Richards, Schmidt, &
matrix, primary presentation forms,
Woods, 1978), also presented a manual
secondary presentation forms, and
to train people in developing effective
interdisplay relationships. As a result
instructional strategies based on our
of this work, I formalized our content
early work on CDT. The presentation
classification scheme and strategy
of CDT in the Reigeluth Green Books
description into what came to be
(Merrill, 1983, 1987a, 1988) provided
called Component Display Theory
the wide spread dissemination of this
(CDT). The name deserves some
work. The most complete presentation
explanation. We felt that each primary
of CDT is found in Merrill (1994).
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
TICCIT
In 1966, I visited
The challenge of the project was
Vic Bunderson
to design an authoring system. We
at the University
struggled with many ideas. Bunderson
of Texas. He had
proposed a system that had a variety
established a
of different instructional approaches
computer-based instruction laboratory
built into the system. The learner
that involved an IBM 1500 system,
could then select the instructional
one of the early main¬frame CBI
approach they felt was best for them.
systems. This was the beginning of
Harvey Black, a colleague at Brigham
our collaboration. We vowed at this
Young University and a collaborator
time that we would one day work
on the TICCIT project, felt that
together in the same institution. In
students would not have sufficient
the early 1970s, I had arranged an
information to make a wise selection
appointment for Vic at Brigham Young
before the fact. He suggested that
University to work with our laboratory
learners could only make a decision
there. During the interview with the
about what they needed next when
administration, he received a phone
they were involved with the learning.
call from NSF indicating that he had
We came up with the notion of a
received major funding to build a
“momentary comprehension index,”
new CBI system. He turned down
that is, what the learner understands
the appointment at BYU because
at a given moment in time.
he needed his lab at Texas to do the project. Later we shared a room at a
About this time, NSF decided to consolidate the project at a single university and BYU was chosen for
convention, and he indicated that the
the site. Bunderson and many of his
project was bigger than he anticipated.
personnel from the Texas lab moved to
I offered to help. We visited NSF
Provo, Utah, to continue the project.
together, and I returned to BYU with
In the meantime, we were struggling
a contract to work on the TICCIT 1
with how to author for the system. I
project.
was working on Component Display
1
TICCIT is an acronym for Time-Shared Interactive Computer-Controlled Instructional Television. The name was given by MITRE Corporation, who did the hardware development on the project. The original idea was to deliver instructional television into all the homes in Reston, Virginia. The nature of the project changed dramatically, but the name was never changed to more accurately reflect the true nature of the system. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
151
152
Michael Allenʼs 2008 e-Learning Annual
Theory, and we had already identified
lecture, he suggested that the teaching
primary presentation forms. Harvey
machines of the future would contain
Black suggested that only when
a teacher function that would interact
given a rule (expository generality)
with the student and a professor
could students determine whether
function that would monitor the
they needed an example. Only after
teacher’s strategy, and when a given
having studied several examples could
strategy did not seem to be working
students determine whether they
would provide advice as to what to try
were ready for practice. So we put the
next. I remembered this lecture and
primary presentation forms as button
answered this concern by suggesting
commands on a special keyboard.
that we would have an advice key that
Thus, given an objective for a segment
would access an advice function that
of instruction, students could select
would help the student decide which
rule, example, or practice by the
learner control key to select next.
touch of a button. We also added an easy and hard button, which allowed It allowed learner control, not just of content, but of the strategy to be used.
After the meeting, our team strongly suggested that I had gotten
the student to
carried away, that we had no idea
find an easier
how to build an advice system. After
or harder rule,
several members of our team had
example, or
struggled with this problem, the
practice item.
task to design the system fell on my
By means of these buttons, TICCIT
shoulders. Expert systems were not yet
was unique in that it allowed learner
widely available, so I decided to build
control, not just of content, but of the
a decision tree that would provide
strategy to be used.
the required advice to the student.
During a site visit from NSF,
To make this work, we determined
someone raised the concern that
an ideal strategy, “the all-American
students may not know which button
strategy.” This strategy represented
to select next. The help key told
what we thought would be the best
them what was available but did not
use of the learner control keys by
give them any advice about which
a student. We then compared the
presentation form would be most
student’s path through the learner
helpful. I remembered a lecture on
control keys with our ideal strategy.
the future of teaching machines
The adviser then gave “local” advice;
(circa 1963) by my major professor,
that is, it recommended the key that
Lawrence Stolurow (1961). In this
would be the best for the student
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
to try at the time the advice was
an expert system advisor had not yet
requested. It was a very sophisticated
been built.
early version of an overlay expert system. Our team developed a complete
The TICCIT system pioneered a number of developments that are now common on all computer systems.
math zero (basic algebra) curriculum
Although multimedia was still in the
and an English writing program.
future, the system was one of the first
The TICCIT system and curriculum
to have colored text and graphics
were tested at Maricopa Community
(they were pretty primitive by today’s
College in Phoenix and at BYU, and it
standards). The system used an early
continued to run at both locations into
version of Windows, also pretty
the 1990s. At Maricopa Community
limited by today’s standards. The
College, TICCIT was used as mainline
system constructed displays for the
instruction. An evaluation of the
student “on-the-fly” from resource
system showed that English students
files that were combined with text
scored higher on tests of writing
templates to create either presentation
skills and essay tests than did the
or practice displays from the same
comparison group in regular lecture
content files. The design of the
classes. In the math classes students
TICCIT system is described in Merrill,
scored higher on their posttests than
Schneider, and Fletcher (1980).
did the comparison groups in regular classes (Alderman, 1979). The English
Learner Control As far as we
program has been transported to a
were able to determine, no one had
more current programming language
previously provided for learner
and is still available online (http://
control of instructional strategy. The
webclips.byu.edu). I suspect that this
TICCIT system allowed for both
program holds the record for the
content control, selecting the next
longest-running CBI program.
segment of instruction from a menu,
There was an attempt to create a
and strategy control, provided by the
commercial version of the TICCIT
rule, example, practice, easy, hard,
system. It was used for a time by the
and advice learner-control keys. We
military. The advisor system was not
were anxious to learn more about the
included in the commercial version.
effectiveness of this version of learner
To our knowledge, another system that
control. With the TICCIT system, we
allows learner control of strategy with
had a very good laboratory instrument
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
153
154
Michael Allenʼs 2008 e-Learning Annual
for conducting research. Our students
out above all the rest. The syntax
conducted dozens of studies. The
revealed that it was obviously written
results showed (Merrill, 1975, 1980,
by an international student, but the
1984):
content was very insightful. I asked
1. Students can decide how many
Zhongmin Li 2 to stop and see me
examples and practice items they
after class. Thus began a very fruitful
need for mastery;
collaboration.
2. Allowing students to determine
During this period of time, one
the sequence of rule, example, and
of my consulting opportunities
practice forms does not improve
was to review some computer-
performance over those who have
based instruction developed by a
a fixed rule, example, practice
major Air Force training company.
sequence;
The instruction was early CBT
3. Students who have control over
and involved a kluge of equipment
generality help performed more
including two monitors and an
efficiently and effectively than
analog audio system. The instruction
other groups; and
consisted of wall-to-wall text on one
4. Students who have control over
monitor and a very nice graphic on the
example and practice help perform
other monitor. The audio read the text
less efficiently and less effectively
to the student. I suggested that this
than other groups.
was not a very effective instructional strategy, and Mayer (2001) has since
Automating Instructional Design
During the
demonstrated the ineffectiveness
1980s, I found
of this instructional approach. The
myself at the
company agreed to let us try to build
University
some instructional shells that would
of Southern
enable their “designers-by-assignment”
California. I taught a seminar on
to build more effective instruction for
authoring systems. We explored many
different kinds of learning outcomes. I
of the very early authoring systems.
approached Zhongmin about building
One of the assignments of the class
such a shell for naming parts. He
was to design an authoring system. As
agreed. A week later he demonstrated
I read the student projects toward the
for me not only a shell into which
end of the semester, one paper stood
any content for naming the parts of
2
Dr. Li eventually immigrated to the United States, changed his name to James Z. Li, founded LeadingWay Corporation, and has built a very successful business in knowledge management. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
something could be imported, but he
campus and subsequently provided
also demonstrated an authoring system
a very large contract to build ID
that could be used by designers-by-
Expert, a commercial version of our
assignment for importing the content
instructional design expert system
into the shell. Subsequently, we also
(Li & Merrill, 1991; Merrill, 1987b;
designed and programmed shells
Merrill & Group, 1998; Merrill & Li,
for teaching concepts and teaching
1989, 1990). Version 1 of the system
procedures (Li & Merrill, 1990).
was completed when the company
In a contract with the Army
suffered serious financial problems
Research Institute, we proposed to
and discontinued its business. We also
build an instructional design expert
lost another very large contract at this
system. This system would ask a
same time and the work on ID Expert
series of questions of the author and
was discontinued.
then prescribe the most effective
ID Expert was a very ambitious
instructional strategy to use (Merrill
project. The goal was to reduce the
& Li, 1989). It then occurred to us to
labor involved in authoring computer-
combine our authoring shells with
based instruction by an order of
the prescriptive system so that the
magnitude. The system consisted of
system would not only recommend an
several subsystems working together:
appropriate strategy but would also
(1) a set of reusable instructional
select an appropriate authoring shell
strategy algorithms (transaction shells)
and prompt the user to import their
for different kinds of instructional
content into this shell for delivery to
outcomes based on instructional
their students.
principles (CDT); (2) a decoupled knowledge base that allowed the
ID Expert At this time both
designer or system to attach a given
Zhongmin and I moved to Utah
knowledge object to any of the
State University and, with funding
instructional strategy algorithms,
from various government sources
thus allowing for reuse of knowledge
and a major contract with IBM,
objects within the system; and (3) a
founded the ID2 Research Group.
set of instructional parameters that
We were joined by Mark Jones and
allowed the instructional strategies
a number of graduate students and
to be modified by merely selecting a
continued our work on designing an
different parameter value (Merrill,
instructional design expert system.
Jones, & Li, 1992; Merrill, Li, & Jones,
A group from Germany visited our
1992a).
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
155
156
Michael Allenʼs 2008 e-Learning Annual
Initial authoring consisted of
could describe content that could be
identifying the content to be taught
manipulated in a computer system to
and entering it into the knowledge
automatically create instruction from
base. The system contained default
only the content. It was also necessary
strategy parameter values that
to develop a more detailed description
would then allow it to automatically
of instructional strategies on which
format the instructional strategies
our instructional design shells could
for interaction with the student. A
be based.
second level of authoring allowed the
The what-to-teach side of the
author to “open the hood” and adjust
equation was extended to describe
strategy parameter values to modify
knowledge objects (Jones, Li, &
the instructional strategies. The
Merrill, 1990; Merrill, 1998, 2001b,
system allowed for multiple strategies
2001c, 2002b). We identified three
to be assigned to a given lesson or
components of knowledge: entities,
different strategies to be assigned
activities, and processes. We also
to different students. The ultimate
identified properties and defined
goal was to include a “professor”
each of these components in terms
function that would monitor student
of properties. An entity is defined by
performance and automatically adjust
a set of properties each of which has
strategy parameters as necessary to
a set of legal values. Each legal value
accommodate the learning needs
can be represented by a portrayal.
of a given student. This adaptive
Changing the portrayal changes the
instruction feature was planned for
appearance of the entity. An activity
Phase 3 of the project but was never
is defined as some action on the part
completed because of the demise of
of the learner that triggers a process.
the funding company.
When executed, a process results in some consequence that is defined
Instructional Transaction
as a change in the value of one or
Theory In order to develop our tools
more properties of some entity. A
for automating instructional design,
process also has a set of conditions
we found that it was necessary to
that are defined as values on a set of
develop more precise instructional
properties. A process only executes
design theory (Merrill, Li, & Jones,
when the conditions equal the
1990a, 1990b, 1991). My contribution
appropriate property values.
to the work was to try to extend CDT to a more precise level, so that we
We also simplified the Performance/ Content Matrix to identify five kinds
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
of learning outcomes: information-
157
We also identified PEA-nets
about, parts-of, kinds-of, how-to, and
(process-entity-activity –nets) that
what-happens. Information-about and
define the dynamic relationship
parts-of correspond roughly to facts;
among these three components of a
kinds-of corresponds to concepts;
knowledge object. An activity (action
how-to corresponds to procedures;
by a learner) triggers a process. If the
and what-happens corresponds to
conditions are true (the condition
principles in the former scheme.
properties have the appropriate
We also identified an information-
values), then the process executes—
level (generality) and a portrayal-
meaning that it changes the values of
level (instance) for each of the five
the consequence properties.
categories of learned outcomes. In this knowledge representation,
The how-to-teach side of the equation was extended to more
a definition of a concept (kind-of )
precisely identify the relationship
is a list of defining properties and
between our knowledge objects and
the value each must assume for a
the demonstration and application of
portrayal to be a member of the class.
this knowledge (Li & Merrill, 1990;
A step in a procedure (how to) is an
Merrill & ID2 Research Team, 1993;
action performed by the student. A
Merrill, Jones, & Li, 1992; Merrill,
condition is a value on a property. A
Li, & Jones, 1991, 1992a, 1992b). An
consequence is a change in the value
instructional strategy is defined as
of a property.
a primary presentation form, which
INFORMATION PRESENT (TELL)
RECALL (ASK)
DEMONSTRATE (SHOW)
APPLY (DO)
Tell the definition.
Recall the definition.
Show several specific examples.
Classify new examples.
Tell the steps and their sequence.
Recall the steps and their sequence.
Show the procedure in several different situations.
Carry out the procedure in new situations.
Show the process in several different situations.
Predict a consequence or find faulted conditions in new situations
Kinds-of
How-to
What-happens
PORTRAYAL
Tell the conditions and consequence involved in the process.
Recall the conditions and consequence involved in the process.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Table 3. Consistent Information and Portrayal for Categories of Learning
158
Michael Allenʼs 2008 e-Learning Annual
we simplified to TELL, ASK, SHOW,
Instructional Simulator Leston
DO operating on a component of a
was the primary developer of the
knowledge object. Table 3 (previous
Instructional Simulator, a very
page) indicates the consistent
easy-to-use simulation authoring
strategies for kinds-of, how-to, and
and delivery system that combined
what-happens.
instructional strategies with simulation (Merrill, 1999). This tool
Electronic Textbook Mark Lacy
was available for a short time as a
and Leston Drake joined our ID2
commercial product under the name
Research Group shortly before the
IDVisualizer.
demise of the ID Expert project. Using
The Instructional Simulator
largely internal funds, we were able to
implemented our work on knowledge
continue our work to develop systems
objects. It is based on PEA-nets,
for automating instructional design.
as described earlier in this article.
Mark was the primary developer of
Authoring the simulator consists of
the Electronic Textbook, a very easy-
first defining a set of dynamic entities
to-use authoring shell with built-in
that may be parts of a larger system.
instructional design for teaching
For each entity, a set of properties is
naming, concepts, and procedures
identified along with the legal values
(Merrill & Thompson, 1999). This
that this property can assume. For
tool was available for a short time as a
each legal value, a portrayal (usually a
commercial product under the name
graphic representation) is developed
IDXelerator.
and stored in a knowledge-base.
The Electronic Textbook consisted
Attached to each entity is a set of
of three built-in instructional
activities (actions defined by clicking
algorithms, each implementing the
on some part of the portrayal of
rules of Component Display Theory:
the entity), one or more processes
an information-about (facts) lesson,
triggered by each activity, and the
kinds-of (concept) lesson, and a how-to
consequence of the process defined as
(procedure) lesson. Authoring consists
a change in the value of one or more
of putting the appropriate content
properties. The underlying system
into a knowledge base for each kind
is an expert system that executes
of lesson. The system then generated
the PEA-net. Thus, the portrayals
presentation and practice for the
on the display can be manipulated
student appropriate for each type of
by the student. (See Figure 1, which
lesson.
illustrates the interface.) Students
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
159
Figure 1. Interface for Instructional Simulator
interact by clicking on various parts
went on to form their own company,
of the pump to connect or disconnect
LetterPress Software. The ID 2 research
them. The authoring system prompts
group was unable to find additional
the input of the activity-process-
funding to continue this work, and
consequence rules in a very easy-to-
the company that bought the rights
use format. Once the portrayals for
to this produce were unsuccessful in
the different values of the properties
commercializing this work.
of the entities have been developed, the actual simulation can be developed in a matter of a few minutes to a few
First Principles of Instruction
When I developed CDT,
hours, depending on the complexity
I was not trying
of the system. Authoring simulations
to develop an
in the Instructional Simulator was
alternative
considerably more efficient than
method for designing instruction.
programming. The instructional
The primary purpose was to develop
simulator was our most sophisticated
a more precise terminology for
implementation of automated
describing instructional outcomes and
instructional design. Mark and Leston
instructional strategies. The second
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
160
Michael Allenʼs 2008 e-Learning Annual
purpose was to identify relationships
to elaborate, clarify, and explain
between the outcomes of instruction
these principles and to demonstrate
and the strategies thought appropriate
their presence in other instructional
to produce these outcomes. My
design theories and models (Merrill,
intent was to identify underlying
2001a, 2002a, 2006a, 2006b, 2007, In
principles that were common to all
Press-a, In Press-b). I also proposed
models of instructional design. In
a content-first alternative to the
spite of my intentions, CDT was
classic ADDIE instructional design
often characterized as an alternative
model called A Pebble-in-the-Pond
approach to instructional design
approach to instructional development
rather than a more precise way to
(Mendenhall, Buhanan, Suhaka,
characterize existing approaches to
Mills, Gibson, & Merrill, 2006;
instructional design.
Merrill, 2002c, in press-c). Recent
When the second green book was
work has attempted to elaborate the
published (Reigeluth, 1999b), Charlie
content analysis required for task-
indicated in the preface that there
centered instruction and to define an
were many different instructional
effective task-centered instructional
design theories and models, and that
strategy for teaching complex content
designers should learn many of these
(Mendenhall, Buhanan, Suhaka, Mills,
different approaches and use the
Gibson, & Merrill, 2006; Merrill, in
approach that was most appropriate
preparation).
for a given situation. I thought that
TRADOC (the army training and
most of these different approaches
doctrine command) has recently
were all based on the same underlying
adopted a new training methodology
principles and that they differed
called GEL (Guided Experiential
mostly in implementation details.
Learning). GEL is based on First
Charlie challenged me to demonstrate
Principles of Instruction. A task force
my assumption. I set out to determine
of the audiovisual department of the
the fundamental principles that were
Church of Jesus Christ of Latter-
common to many of these different
Day Saints has recently adopted
approaches.
First Principles of Instruction as
The result of this effort was a
the instructional methodology for
set of principles that I called “First
online training materials offered by
Principles of Instruction.” These
the church. The author is humbled
principles are summarized in Table
by these developments and anxiously
4. In the past few years I have tried
awaits data from these two large
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
Task-centered approach
Application principle
ÿ Learning is promoted when learners are
ÿ Learning is promoted when learners engage in
engaged in a task-centered approach, which
application of their newly acquired knowledge
includes demonstration and application of
or skill that is consistent with the type of
component skills.
content being taught.
ÿ A task-centered approach is enhanced when learners undertake a progression of whole
ÿ Application is effective only when learners receive intrinsic or corrective feedback. ÿ Application is enhanced when learners are
tasks.
coached and when this coaching is gradually Activation principle
withdrawn for each subsequent task.
ÿ Learning is promoted when learners activate relevant cognitive structures by being directed
Integration principle
to recall, describe, or demonstrate relevant
ÿ Learning is promoted when learners integrate
prior knowledge or experience. ÿ Activation is enhanced when learners recall or acquire a structure for organizing the new
their new knowledge into their everyday life by being directed to reflect on, discuss, or defend their new knowledge or skill. ÿ Integration is enhanced when learners create,
knowledge.
invent, or extrapolate personal ways to use their Demonstration principle
new knowledge or skill in situations in their
ÿ Learning is promoted when learners observe a
world.
demonstration of the skills to be learned that is consistent with the type of content being
ÿ Integration is enhanced when learners publicly demonstrate their new knowledge or skill.
taught. ÿ Demonstrations are enhanced when learners receive guidance that relates instances to generalities. ÿ Demonstrations are enhanced when learners observe media that is relevant to the content.
Table 4. First Principles of Instruction
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
161
162
Michael Allenʼs 2008 e-Learning Annual
organizations as to the effectiveness
have just scratched the surface of how
of their training based on these
to design what to teach and how to
principles.
teach. What is my greatest concern? I’m
Summary
I hope that
concerned that there is not more
this nostalgic
effort being devoted to this question.
journey through
The current zeitgeist seems to
a few highlights
emphasize communities of learners,
of my career
repositories of content, and electronic
has provided a bit of insight into
communication. There seems to be
how ideas develop and evolve. What
an assumption that information is
have I learned about how to make
sufficient and that e 3 direct instruction
instruction more effective, efficient,
is no longer necessary. The Internet
and engaging (e 3)? Considerably more
is swollen with information, and
than I knew at the beginning of my
amidst this flood, there are only
career. I do believe that we know a bit
isolated islands of e 3 instruction.
more about how to design instruction
I am gratified by the adoption of
that works. I have been gratified by
our work by the Army and the LDS
the many designers who have found
church. Nevertheless, my hope is that
our work helpful;. however, far too
our work may continue to provide a
much instruction is still not effective,
catalyst for further efforts to find ways
not efficient, and not engaging. Do we
to make instruction more effective,
have more to learn? I believe that we
more efficient, and more engaging.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction REFERENCES Alderman, D.L. (1979). Evaluation of the TICCIT computer-assisted instructional system in the community college. ACM SIGCUE Outlook, 13(3), 5-17. Bruner, J.S., Goodnow, J.J., & Austin, G.A. (1956). A study of thinking. Hoboken, NJ: John Wiley & Sons. Ellis, J.A., Wulfeck, W.H., Merrill, M.D., Richards, R.E., Schmidt, R.V., & Wood, N.D. (1978). Interim training manual for the Instructional Quality Inventory SR 78-5. San Diego, CA: NPRDC. Gagne, R.M. (1965). The conditions of learning. New York: Holt, Rinehart & Winston. Jones, M.K., Li, Z., & Merrill, M.D. (1990). Domain knowledge representation for instructional analysis. Educational Technology, 30(10), 23-30. Li, Z., & Merrill, M.D. (1990). Transaction shells: a new approach to courseware authoring. Journal of Research on Computing in Education, 23(1), 72-86. Li, Z., & Merrill, M.D. (1991). ID Expert 2.0: Design theory and process. Educational Technology Research and Development, 39(2), 53-69. Mayer, R.E. (2001). Multimedia learning. Cambridge, UK: Cambridge University Press. Mendenhall, A., Buhanan, C.W., Suhaka, M., Mills, G., Gibson, G.V., & Merrill, M.D. (2006). A task-centered approach to entrepreneurship. TechTrends, 50(4), 84-89. Merrill, M.D. (1971). Necessary psychological conditions for defining instructional outcomes. Educational Technology, 11(8), 34-39. Merrill, M.D. (1975). Learner control: beyond aptitude-treatment interactions. A V Communications Review, 23, 217-226. Merrill, M.D. (1980). Learner control in computer-based learning. Computers and Education, 4, 77-95. Merrill, M.D. (1983). Component display theory. In C.M. Reigeluth (Ed.), Instructional design theories and models: An overview of their current status. Mahwah, NJ: Lawrence Erlbaum Associates. Merrill, M.D. (1984). What is learner control? In R. Bass & C.R. Dills (Eds.), Instructional development: The state of the art II. Dubuque, IA: Kendall/Hunt. Merrill, M.D. (1987a). A lesson based on component display theory. In C.M. Reigeluth (Ed.), Instructional design theories in action (pp. 201-244). Mahwah, NJ: Lawrence Erlbaum Associates.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
163
164
Michael Allenʼs 2008 e-Learning Annual
Merrill, M.D. (1987b). An expert system for instructional design. IEEE Expert, 2(2), 25-40. Merrill, M.D. (1988). Applying component display theory to the design of courseware. In D.H. Jonassen (Ed.), Instructional designs for microcomputer courseware (pp. 61-95). Mahwah, NJ: Lawrence Erlbaum Associates. Merrill, M.D. (1994). Instructional design theory. Englewood Cliffs, NJ: Educational Technology Publications. Merrill, M.D., & ID2 Research Group. (1998). ID expert: A second generation instructional development system. Instructional Science, 26, 243-262. Merrill, M.D. (1998, March/April). Knowledge objects. CBT Solutions, pp. 1-11. Merrill, M.D. (1999). Instructional transaction theory (ITT): Instructional design based on knowledge objects. In C.M. Reigeluth (Ed.), Instructional design theory and models: A new paradigm of instructional theory vol. II. Mahwah, NJ: Lawrence Erlbaum Associates. Merrill, M.D. (2001a). First principles of instruction. Journal of Structural Learning and Intelligent Systems, 14(4), 459-466. Merrill, M.D. (2001b). A knowledge object and mental model approach to a physics lesson. Educational Technology, 41(1), 36-47. Merrill, M.D. (2001c). Toward a theoretical tool for instructional design. Instructional Science, 29(4-5), 291-310. Merrill, M.D. (2002a). First principles of instruction. Educational Technology Research and Development, 50(3), 43-59. Merrill, M.D. (2002b). Knowledge objects and mental models. In D.A. Wiley (Ed.), The instructional use of learning objects (pp. 261-280). Washington, DC: Agency for Instructional Technology & Association for Educational Communications and Technology. Merrill, M.D. (2002c). A pebble-in-the-pond model for instructional design. Performance Improvement, 41(7), 39-44. Merrill, M.D. (2006a). Hypothesized performance on complex tasks as a function of scaled instructional strategies. In J. Elen & R.E. Clark (Eds.), Handling complexity in learning environments: Theory and research (pp. 265281). Amsterdam: Elsevier. Merrill, M.D. (2006b). Levels of instructional strategy. Educational Technology, 46(4), 5-10.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
Merrill, M.D. (2007). First principles of instruction: A synthesis. In R.A. Reiser & J.V. Dempsey (Eds.), Trends and issues in instructional design and technology (2nd ed., pp. 62-71). Upper Saddle River, NJ: Merrill Prentice Hall. Merrill, M.D. (In press-a). Converting e 3 learning to e 3 power learning: An alternative instructional design method for e-learning. In S. Carliner & P. Shank (Eds.), The e-learning handbook: Past promises, present challenges. San Francisco, CA: Pfeiffer. Merrill, M.D. (In press-b). First principles of instruction. In C.M. Reigeluth & A. Carr-Chellman (Eds.), Instructional design theories and models III. Mahwah, NJ: Lawrence Erlbaum Associates. Merrill, M.D. (in press-c). A task-centered instructional strategy. Journal of Research on Technology in Education. Merrill, M.D., & Boutwell, R.C. (1973). Instructional development: Methodology and research. In F. Kerlinger (Ed.), Review of research in education, vol. 1. Ithaca, NY: Peacock Publishers. Merrill, M.D., & Goodman, I.R. (1972). Selecting instructional strategies and media: A place to begin. Syracuse, NY: National Special Media Institutes. Merrill, M.D., & ID2 Research Group. (1998). ID Expert: a second generation instructional development system. Instructional Science, 26, 243-262. Merrill, M. D., & ID2 Research Team. (1993). Instructional transaction theory: knowledge relationships among processes, entities and activities. Educational Technology, 33(4), 5-16. Merrill, M.D., Jones, M.K., & Li, Z. (1992). Instructional transaction theory: classes of transactions. Educational Technology, 32(6), 12-26. Merrill, M.D., & Li, Z. (1989). An instructional design expert system. Journal of Computer-Based Instruction, 16(3), 95-101. Merrill, M.D., & Li, Z. (1990). An instructional design expert system. In S. Dijkstra, B.H.A.M. Van Hout Wolters & P.C.V. Sijde (Eds.), Research in instruction: Design & effects. Englewood Cliffs, NJ: Educational Technology Publications. Merrill, M.D., Li, Z., & Jones, M.K. (1990a). Limitations of first generation instructional design (ID1). Educational Technology, 30(1), 7-11. Merrill, M.D., Li, Z., & Jones, M.K. (1990b). Second generation instructional design (ID2). Educational Technology, 30(2), 7-14. Merrill, M.D., Li, Z., & Jones, M.K. (1991). Instructional transaction theory: An introduction. Educational Technology, 31(6), 7-12. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
165
166
Michael Allenʼs 2008 e-Learning Annual
Merrill, M.D., Li, Z., & Jones, M.K. (1992a). Instructional transaction shells: Responsibilities, methods, and parameters. Educational Technology, 32(6), 12-26. Merrill, M.D., Li, Z., & Jones, M.K. (1992b). Instructional transaction theory: an introduction. In S. Dijkstra, Z. Li, & M.D. Merrill (Eds.), Instructional models in computer-based learning environments. New York: Springer Verlag. Merrill, M.D., Reigeluth, C.M., & Faust, G.W. (1979). The instructional quality profile: A curriculum evaluation and design tool. In H. O’Neil (Ed.), Procedures for instructional systems development. New York: Academic Press. Merrill, M.D., Richards, R.E., Schmidt, R.V., & Wood, N.D. (1977). The instructional strategy diagnostic Profile training manual. Orem, UT: Courseware Inc. Merrill, M.D., Schneider, E.W., & Fletcher, K.A. (1980). TICCIT. Englewood Cliffs, NJ: Educational Technology Publications. Merrill, M.D., & Tennyson, R.D. (1977a). Concept classification and classification errors as a function of relationship between examples and non examples. Improving Human Performance. Merrill, M.D., & Tennyson, R.D. (1977b). Teaching concepts: An instructional design guide. Englewood Cliffs, NJ: Educational Technology Publications. Merrill, M.D., Tennyson, R.D., & Posey, L.O. (1992). Teaching concepts: An instructional design guide (2nd ed.). Englewood Cliffs, NJ: Educational Technology Publications. Merrill, M.D., & Thompson, B. (1999). IDXelerator: Learning-centered instructional design. In J.v.d. Akker, R.M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design methodology and development research in education and training (pp. 265-277). Dordrecht/Boston/London: Kluwer Academic Publishers. Reigeluth, C.M. (1983). The elaboration theory of instruction. In C.M. Reigeluth (Ed.), Instructional design theories and models: An overview of their current status (pp. 335-381). Mahwah, NJ: Lawrence Erlbaum Associates. Reigeluth, C.M. (1999a). The elaboration theory: guidance for scope and sequence decisions. In C.M. Reigeluth (Ed.), Instructional design theories and models: A new paradigm of instructional theory, volume II (pp. 425-453). Mahwah, NJ: Lawrence Erlbaum Associates.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
Reigeluth, C.M. (Ed.). (1999b). Instructional-design theories and models: A new paradigm of instructional theory (Vol. II). Mahwah, NJ: Lawrence Erlbaum Associates. Reigeluth, C.M., Merrill, M.D., Wilson, B., & Spiller, R.T. (1980). The elaboration theory of instruction: a model for structuring instruction. Instructional Science, 3, 30-31. Stolurow, L.M. (1961). Teaching by machine. Washington DC: U.S. Department of Health, Education, and Welfare. Tennyson, R.D., & Cocchiarella, M.J. (1986). An empirically based instructional design theory for teaching concepts. Review of Educational Research, 56(1), 40-71. Tennyson, R.D., & Park, O.-C. (1980). The teaching of concepts: A review of instructional design research literature. Review of Educational Research, 50(1), 55-70. Tennyson, R.D., Wooley, R.R., & Merrill, M.D. (1972). Exemplar and non exemplar variables which produce correct classification errors. Journal of Educational Psychology, 63, 144-152.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
167
Section Three
What Does It All Mean? Pitfalls and Overhauls
Tom Boyle Professor Tom Boyle is director of the Learning Technology Research Institute (LTRI) at London Metropolitan University. He has a long history of developing and evaluating innovative multimedia learning technology. Tom led a major project in the development, use, and evaluation of learning objects that won an EASA (European Academic Software Award) in 2004. Currently, Boyle’s work concerns the Centre for Excellence in Teaching and Learning (CETL) in Reusable Learning Objects, which he directs. This multi-million-dollar project is funded by the Higher Education Funding Council for England. It is a collaboration among three universities—London Metropolitan University, the University of Cambridge, and the University of Nottingham—to develop and evaluate high-quality learning resources across a range of subject areas.
Dreams and Responsibilities
Tom Boyle
Learning and the New Technologies This article reviews and comments on major developments in technologyenhanced learning over the past two decades, as the author reflects on how his own work has fit into the landscape of these momentous changes. The central theme of the article is the need for a profound and robust pedagogy to enable us to exploit effectively the opportunities offered by new technologies. “In dreams begins responsibility.”
session I would print a ticker-tape roll
—W.B. Yeats
of paper with punched holes that held my data (there was a neat little printer
The leitmotiv of this article is to
attached to the terminal). You could
outline the dreams of what the
put an elastic band around the roll,
new technologies can deliver,
drop in your pocket, and walk away
counterbalanced by our responsibilities
feeling that your data were secure. This
to learners. A major organizing theme
was mobile computing in the mid-
is the concept of “technological
1970s. We are only talking about thirty
amplifiers” of human ability described
years ago. Never has a technological
by Jerome Bruner (in turn influenced
storm come with so much force, over
by the ideas of Vygotsky).. This idea is
so short a time, and with such impact.
used to establish the conceptual basis
The research group I lead, among
on which the relationship between
other things, now designs multimedia
learning and new technologies is
learning objects for mobile phones.
explored. What manifests is the need
Mobile computing for education has
for pedagogical design principles that
moved a long way in thirty years.
will enable us to exploit the possibilities
Mobile computing is only one of
of the new technologies enabling us to
a huge range of developments: the
creatively utilize the new technology,
micro-computer revolution of the
rather than engaging in surface
1980s, the multimedia revolution of
adaptation to the latest innovation or
the early 1990s, the World Wide Web
fashion.
from the mid-1990s, mobile ubiquitous
My first use of computers was in
computing, and so on. How do we
the mid-1970s doing statistical tests
develop a pedagogy that is other than
for my Ph.D. When I had finished a
in a state of constant reaction to
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
172
Michael Allenʼs 2008 e-Learning Annual
these new developments? How do we
are more profound and have wider
develop a pedagogy that can shape the
implications than others. These
technology to enhance learning in a
deeper questions are often difficult
profound and principled way?
to formulate—let alone resolve. So our discourse in e-learning is often
Questions and Perspectives
Through the
dominated by short-term tactical
example of
considerations, rather than by the
the modern
more profound strategic impact of the
book, Alan
new technology.
Kay points to two broad classes of questions we
Technology and Learning The
can ask about the impact of new
most basic question is not how
technologies: How can they make
technology will impact learning; but
established teaching and learning
rather how the new information and
approaches more effective? and What
communication (ICT) technologies will
is truly new? Kay argues that the
impact it. We have had technology-
modern book did not arrive with the
enhanced learning for several
advent of the printing press, as many
hundred, if not thousands, of years.
assume. In an entertaining story,
The greatest technological invention
Kay recounts how Aldus Manutius
to impact on learning and education
helped to invent the modern book a
has been that of literacy. Literacy
considerable period after the invention
created a new form of permanent
of the printing press by, among other
memory. This transformed how we
things, measuring saddlebags to work
solve problems and how we learn. This
out the optimal size. In this amusing
transition was not without problems.
anecdote, Kay makes a salient point.
Literacy as a technology may have
The first use of new technologies is to
enhanced certain processes, but it
automate the old way of doing things.
seemed to undermine certain other
But the really important breakthrough
abilities. In Plato’s Phaedrus, Socrates
is to find the new ways that are made
produces several strong arguments
possible by the technology.
in defending the oral tradition. He
In tackling any major endeavor, the
criticizes the indirect and removed
most important thing is to ask the
nature of written texts. He contrasts
right questions. In e-learning, different
the “inert” words on a page with the
people ask different questions for
living words of the speaker, which can
different purposes. Some questions
be interrogated and defended. Socrates
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Dreams and Responsibilities
173
further argues that “this discovery
education. The fundamental challenge
. . . will create forgetfulness in the
is to creatively rethink pedagogy to
learners’ souls because they will not
take advantage of the opportunities
use their memories; they will trust to
offered by the new technology.
the external written characters and
One of the most profound insights
not remember of themselves” (Plato,
on the relationship of technology and
1999).
pedagogy is given by Jerome Bruner
In quite an important way, human
(1964) in his paper, The Course of
memory (both individual and social)
Cognitive Growth. Bruner argues that
changed with the advent of literacy.
human evolution has been shaped in
It enabled us to do things that we
a powerful way by the cultural tools
could not do before; yet, in that
we have developed. A long childhood
transition, feats of memory valued in
enables powerful cultural influences
the oral tradition become devalued
on cognitive
and withered. Technologies do not
development.
Technologies do not just
just change the means of learning;
Bruner discusses,
change the means of learning;
they also change the goals. The status
in particular, a set
they also change the goals.
of literacy is now unchallenged,
of technological
but it is refreshing to revisit this
amplifiers of human abilities.
trenchant critique. In the transition
Although our physical abilities, both in
to new technologies, we should think
terms of strength and speed, are quite
freshly about both the established
limited, we have invented such cultural
technology—its strengths and its
implements as cars and planes that
weaknesses—and about the possible
enable us to transcend our biological
impact of new technologies on our
limitations. Equally, our perceptual
goals and values.
capabilities are quite limited. However, we can transcend these biological
Rethinking Pedagogy New
limitations through the inventions of
technologies can be disruptive. They
microscopes, telescopes, and other
threaten to render obsolete many of
more specialized imaging equipment.
the old, well-established ways of doing
Crucially, in addition to physical
things. This disruption encourages
and perceptual amplifiers, Bruner
us to rethink, to re-evaluate how
posited a third type of amplifier—the
we do things. The revolution in
cultural inventions that enhance our
ICT stimulates us to confront our
cognitive abilities. Bruner called
ideas about learning, pedagogy, and
these “ratiocinative” amplifiers.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
174
Michael Allenʼs 2008 e-Learning Annual
He quoted as a prime example the
computing, and in 1986 I began to
invention of mathematics. The
teach computing to undergraduate
technology of mathematics enables
and postgraduate students. For the
us to solve problems that would
first time, I had access to sufficiently
otherwise be impossible to solve.
rich computer resources to think of
There is an extension to these classes
innovative teaching and learning using
of amplifier—the computer as an
computers.
amplifier of our ability to learn—an
The primary impulse to my first
entrancing vision. This is the central
large-scale development of a CAL
dream that has shaped my involvement
system was to address the way in
with “technology enhanced” learning.
which computing, and in particular
How might we approach such a
programming, was taught. Many
challenge? To echo Papert (1980)—
students struggled with learning
who was strongly influenced by Piaget
to program (a problem that has
(1970)—we need “powerful ideas” and
unfortunately become worse over
a solid foundation on which to build
the past decades). I initially found
and extend these ideas. We need a
this surprising. Why should students
pedagogy that has some of the power
find it so difficult? When I talked to
of mathematics to guide and shape our
the lecturers, it was clear that they
design of contexts for learning. This
had strong ideas about how to teach
pedagogy needs to be powerful enough
programming. Their approach placed
to shape and use the opportunities
a strong emphasis on formalism
offered by ICT technology, rather than
and abstraction. A basic perusal of
being shaped and distorted by it.
the psychology of learning showed this teaching approach to be deeply
Early Involvement and Reflection
I became
problematic. This is neatly summed
actively involved
up by Margaret Donaldson (1978) in
in what was then
her book, Children’s Minds. She points
called computer-
to a basic misunderstanding that,
assisted
because language is a formal system,
learning (CAL) in the late 1980s. At
many people believe that it is learned
that stage I had already completed
in a formal way. Because computer
a Ph.D. and taught psychology for
programs are formal systems, the
ten years, focusing on cognitive
tutors believed that they should be
development. I then took a
taught in a formal way.
postgraduate conversion degree in Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Dreams and Responsibilities
The formal, abstract approach
175
children learn natural language. This
caused considerable problems for
is highly creative as children make
students. The tutors’ most frequent
“productive” errors and shape their
explanation was that students could
understanding based on feedback
not abstract. As human language is
from mature speakers. The CORE
based on abstraction, and we are all
(Context, Objects, Refinement,
rather good at learning it, this seemed
Expression) approach aimed to
an unlikely explanation. It seemed
emulate processes in natural language
more likely that the students were
learning and apply these principles to
failing because of flaws in the teaching
learning programming skills. It is a
methods. The aim of a CAL system,
“guided discovery” approach whereby
I believed, should then be to tap into
students learn from examples, rather
the students’ natural intelligence
than abstract descriptions. They have
and enable then to learn effectively.
to infer ideas about programming
Artificial intelligence approaches were
structures from examples and
popular at the time. I was fond of the
contingent feedback given on their
aphorism at that time that “the central
attempts to construct understand
problem to be tackled is not artificial
(Boyle, Gray, Wendl, & Davies, 1994).
intelligence, but artificial stupidity.” In her beautiful little book,
This approach was rather radical. I was unsure how the students would
Donaldson demonstrates that
react, so it was tried initially in a
whether children succeed or fail
pencil-and-paper form. This test
depends crucially on the learning
seemed to work well in terms of both
situation. This reflects a broad trend
student acceptance and effectiveness
in psychology on the importance of
in acquiring the target skills. If the
the design of learning and problem-
empirical evidence had indicated
solving contexts. The design of
otherwise, I would have tried a
contexts for learning, including virtual
different approach.
multimedia contexts, became the pivot
We first produced a DOS-based
at which learning theories could be
version for learning Pascal, which was
translated into effective pedagogy.
published as a booklet and a floppy
This concern has been central to much
disk (Boyle & Margetts, 1992). An
of my work.
extended team then produced CLEM
I began to develop a method for learning programming based on how
(Core Learning Environment for Modula-2).
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
176
Michael Allenʼs 2008 e-Learning Annual
Figure 1. CLEM Table of Contents
This was introduced in 1990, just
new lecturer for the course (now an
as Windows 3 was introduced into the
eminent computer scientist) promptly
teaching labs in the department. This
reinstated the lectures, but kept
CAL system covered the whole first-
CLEM as “a valuable resource.” This
year syllabus. We got rid of lectures
action reflected a theme that runs
and reinvested the time in seminars.
through the history of computer-
At the end of the year we had obtained
assisted learning: the resistance to
substantial improvements in pass
and neutering of change by faculty.
rates (Boyle, Gray, Wendl, & Davies,
Tradition and inertia are powerful.
1994). The package was sold to other
CLEM and the replacement of lectures
universities and used for most of the
challenged the traditional way of
decade until Modula-2 was replaced as
teaching and learning. To properly
the teaching language.
resolve the issue, it would have been
After Year 1 the whole team was
necessary to examine the evidence
moved off the course to develop
on the effectiveness of the different
CAL systems for the faculty. The
approaches. However, evidence-based
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Dreams and Responsibilities
177
teaching is rare in higher education,
technologists” (the term came later).
and as long as this continues, then
In the UK, the Teaching and Learning
unnecessary failure to learn will be
Technology Programme (TLTP)
a major outcome of our education
fostered over seventy projects in
system.
computer-assisted learning. In many
At this time I wanted to pursue
of these projects, people were learning
technology-supported innovative
about design and development of
pedagogy on a wider scale, and the
multimedia educational applications
early 1990s were a very dynamic and
from scratch. The main outcome
interesting era of change. I developed
of this program was not many
a number of multimedia systems, both
widely used applications; it was the
educational and commercial. Many of
development of a new community.
these are reported in my book, Design
This was fostered in the UK through
for Multimedia Learning (Boyle, 1997).
the creation of the Association for Learning Technology in 1993.
The Early 1990s: The Multimedia Revolution
The early
International conferences, such as
1990s were
Ed Media, became a major way that
dominated by
the various national communities
the multimedia
communicated with each other.
revolution.
There was a second very significant
Windows 3 had, at last, brought
strand of development at this time. A
the graphical user interface to PCs.
series of papers on “constructivism”
There was a tremendous burst of
was published in the early 1990s (e.g.,
development in moving toward the
Duffy & Jonassen, 1991; Jonassen,
multimedia desktop computer. Video,
Mayes, & McAleese, 1993). These
which was at first almost impossible
papers rejected the systematic,
to employ on a desktop machine,
didactic approach of traditional
became readily available within a
instructional systems design (Gagné
matter of a few years. The integration
& Briggs, 1979) and argued for
of text, graphics, animation, and
authentic learning experiences and
video to create rich CD-ROM-based
rich interactivity. Within a decade,
learning environments became a major
constructivism became the dominant
endeavor.
approach to pedagogical design for
In parallel, governments began to
technology enhanced learning. There
actively foster the development of new
is much that is rich and productive in
communities of tutors and “learning
constructivism. I strongly advocated
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
178
Michael Allenʼs 2008 e-Learning Annual
a constructivist approach (Boyle,
machines dropped dramatically. The
1997), but a major reason for this
Web increasingly presented a unified
popularity also seems to be the
interface to what were previously a
freedom of interpretation over what
disparate range of tools and resources.
constructivism means. This fuzziness
The term “computer-assisted learning”
has made constructivism very popular,
became increasingly dated as the
but a radical critique is overdue.
role of communication technologies
In the early 1990s there were
became more ubiquitous and salient.
thus immense advances on three
We moved fully into the era of ICT-
fronts: multimedia technology,
supported learning, and the broad
new pedagogies that pointed to
but vague term of e-learning was
more effective ways to exploit this
increasingly used to characterize the
technology, and the development of
field.
new communities. Collaborative Learning Network
The Mid–1990s On: The World of the Web
The great
mediated cooperation as a basis for
change in
learning became a central concern.
the mid-
The ideas of collaborative learning,
1990s was the
“communities of practice,” and
emergence and
learning as the formation of identity
phenomenal growth of the World
in such communities became central
Wide Web. I remember at the 1996 Ed
strands of interest. Emphasis switched
Media conference in Boston how the
from individually focused learning
conference was alive with excitement
applications to collaborative learning
over a new phenomenon. The
and group-based activities. Socially
hypertext (Memex) dream of Vannevar
oriented approaches became very
Bush in the 1940s was coming true
popular, supported theoretically by
(Bush, 1945). Over the next few
ideas of situated learning and social
years the impact of hypertext on
constructivism (Lave & Wenger, 1991;
pedagogy and learning was great but,
Wenger, 1998). The Web, together
as with so many other technologies
with the influx of contributors from
useful in learning, it was a spin-off
many disciplines, led to a plethora of
from other applications. The Web
views, sometimes based on conflicting
revolutionized access to information.
epistemological positions derived
Problems of interoperability and
from different originating disciplines.
access to applications across different
There was thus a great diversification
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Dreams and Responsibilities
179
of pedagogical approaches, which
systems echoes Kay’s point that new
enriched the study of ICT supported
technologies are often used initially
learning and provided new foci of
to automate the old rather than
interest as well as fresh theoretical
exploiting opportunities for new,
perspectives.
more effective ways of doing things. Managed learning environments
Multimedia At the same time there
sit strangely with the Web. On the
was a relative retrenchment in the
one hand, we have an exhilarating
use of rich, interactive multimedia.
expansion of user control. In the other,
The early Web could not support
we have organizational management,
rich multimedia. This encouraged
tracking, and a one-stop shop for
the proliferation of flat, text-based
reluctant tutors to move to e-learning.
learning resources linked through hypertext. The exploration of rich
Learning Objects The effects of
interactive multimedia was sidelined
another major development in this
for a while. As the Web has matured,
period have become increasingly
however, the media-rich techniques
important as we move forward into
available in the CD-ROM systems
the new millennium: the development
of the mid-1990s have become
of international specifications and
increasingly available.
standards for learning delivered through ICT. A central focus in
Managed Learning
this period is on “learning objects.”
Environments The largely text-based
Standards have been developed to
instructional material fit easily into
support the capture, storage, and
another significant development over
retrieval of reusable learning objects
the past decade: managed (or virtual)
and, increasingly, learning designs.
learning environments (MLEs or
The work on standards was driven by
VLEs). These have probably made the
a knowledge engineering approach
largest practical impact on e-learning
that is “pedagogical neutral.” Standards
in universities. The resources loaded
have been produced for “packaging”
into these management systems,
learning resources in software
however, often represent the lowest
structures that enable interoperability
level of instructional sophistication
across different software systems,
in extending traditional approaches
especially learning management
(loading course information and
systems (IMS, 2006). There is an IEEE
lectures online).The success of these
international metadata standard on
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
180
Michael Allenʼs 2008 e-Learning Annual
how to describe these resources (IEEE
the idea of identifying basic units
LOM, 2002), but the central issue of
of learning (learning objects)—and
pedagogical design was ignored for a
the means by which they can be
long time. One result is the creation
combined to form more complex
of repositories of learning objects that
learning experiences—struck me
are frequently not used (Koppi, Bogle,
as a foundational challenge. At a
& Lavitt, 2004). There is a need for a
practical level, learning objects offer
much greater emphasis on pedagogical
great promise for the exchange and
quality, as opposed to technical
reuse of quality learning resources.
quality, in adherence to standards.
Since they were small, rich interactive
Our work on learning objects has been
learning objects could be delivered
driven by this perspective.
over the Web. They also seemed to
My own work in this period
offer an opportunity for tutors to
continued to be driven by the twin
become involved. As these resources
concerns of advancing conceptual
were comparatively small, their
development and practical application.
development could be fit into the busy
Work in the late 1990s focused on
routine of tutors and lecturers.
developing enriched multimedia
Our work at the Learning
applications on the Web. My ideas
Technology Research Institute (LTRI)
on theory were expounded most
focused on effective pedagogical
systematically in the paper, Toward
design and structural design principles
a Theoretical Base for Educational
that would facilitate reuse (Boyle,
Multimedia Design (Boyle, 2002).
2003). These principles underpinned
This paper argued for a systematic
a major project that used learning
basis for pedagogical design, based on
objects within a blended learning
a synthesis of ideas from generative
environment to improve the learning
linguistics, psychology, and other areas
of computer programming (Bradley
of cognitive science. The convergence
& Boyle, 2004). We returned to this
of theory and practice in my recent
area because there was a major crisis
work has focused on the initially
in learning to program (Jenkins &
unlikely candidate of reusable learning
Davy, 2001). Each cohort in the project
objects and then on reusable learning
consisted of several hundred students
designs.
studying one of four programming
The area of learning objects
modules (courses) across two
has both theoretical and practical
universities. The result of the project
implications. At a theoretical level,
was a major improvement in pass rates
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Dreams and Responsibilities
181
of between 12 to 23 percentage points
widely available. A tool is currently
in year one, and 12 to 27 percentage
being developed that will enable
points in year two (Bradley & Boyle,
tutors to author and repurpose (adapt)
2004). The multimedia learning
learning designs developed using
objects we developed for programming
these generative patterns. This, in
subsequently won the European
turn, contributes to the development
Academics Software Award in
of “layered learning design”—where
September 2004 (EASA RLOs, 2004).
learning designs at different levels of
In 2005, I led a successful
granularity or scope are related to each
collaborative bid for establishing a
other systematically. In this way, the
Centre for Excellence in Teaching
work on GLO designs should integrate
and Learning (CETL) in reusable
with the considerable work on learning
learning objects (CETL-RLO, 2007).
designs at the broader lesson plan
The Higher Education Funding
level fostered by the IMS LD (2006)
Council for England has provided
specification.
funding of £3.3 million to support the CETL for the period 2005 to 2010. The partners in the CETL are London Metropolitan University, the
And What About the Future?
When I was younger, I was a keen
University of Cambridge, and the
fan of science
University of Nottingham. The CETL
fiction. I will
is fostering the development and use
not attempt to compete with science
of learning objects across the three
fiction writers’ visions of the future.
institutions and more widely. At the
They are more imaginative and foretell
theoretical level, the CETL is pursuing
more entertaining visions than any
the development of second-generation
I could provide. But it is clear that
learning objects based on reusable
learning technology will become
learning designs—“generative learning
ubiquitous and all-pervasive, involving
objects” (Boyle, 2006).
us all in major adaptive changes in
Generative learning objects (GLOs)
how we relate to our environment.
invert the established basis for reuse in
We will have to learn to make these
learning objects. It is the pedagogical
adaptations. At the same time the
design rather than the content that
technology has the potential to
provides the basis for reuse. The goal
greatly enhance the efficiency and
is to make quality pedagogical designs,
effectiveness of learning.
at the level of learning objects, more
The great challenge is to move to
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
182
Michael Allenʼs 2008 e-Learning Annual
the next stage of Bruner’s cultural-
Department of Education on a
technological amplifiers. This requires
strategy for a major new initiative in
creativity and great design insight. It
technology enhanced learning. The
requires a discipline with a deep and
conversation was envisioning where
rigorous basis, where learning is at the
we would be in five years’ time. One
center, and the tools and resources of
speaker eloquently argued that there
the new technologies are shaped by
was no point in trying to predict this
and supportive of this creative vision.
because technology changes so quickly.
The new learning designers should
There was much murmured support
be, in many ways, like architects.
for this argument from the group. I felt
The architect has to understand the
the need to point out that we could
tools and materials available. A great
probably predict all too clearly where
architect will master the potential
we would be in five years’ time. We
of new materials to shape them into
have a crisis in learning mathematics
a vision of his/her making, as must
that has endured for decades. It was
great instructional designers. But
depressingly easy to predict that
there is a crucial difference—the
this crisis in learning would still
architecture created by the learning
be there in five years, despite all
designer is virtual and replicable. The
the new technology. In fact, as new
immense challenge in creating these
technology has swept in over us in the
new learning architectures is matched
past three decades—microcomputers,
by the huge scope of their potential
multimedia, the Web—the crisis in
application with learners across the
mathematics education has become
world.
worse. This sobering thought balances
This may seem removed and airyfairy for some readers. And I have some sympathy with this. Should
the exciting vision of the potential for the future. One major question is: Will the new
we be proposing great visions of the
technology help us to deal effectively
future when we are failing to deal
with learning and educational failure?
with the challenges of the present?
If we predict forward from where
There are more mundane but vitally
we are, the prognosis is not very
important ways to look at the
good. We need to stop chasing the
challenge and potential of learning
technology, seeing the latest fashion
technologies.
(social networking, learner generated
I was involved recently in a consultation group for the UK
content, etc.) as the silver bullet that solves the problem. We need to go
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Dreams and Responsibilities
183
beyond the shallow adaptation to the
fiction dreams. The ability directly to
affordances of the new technology. We
record neural patterns and understand
need powerful design—we need highly
their significance could enhance
creative architects of technology
learning in powerful ways. Bio-
supported learning environments.
feedback systems could be provided
Some researchers have argued that
to enable learners to most efficiently
the central problem to be solved for
and effectively guide their learning.
the future is how to provide artificial
The ultimate issue that then arises
intelligence support for learning. But
from these speculations is: Could we
the central problem is not, and never
directly shape the neural state?
has been, artificial intelligence. It is,
This resurrects the image of the
first and foremost, how do we prevent
Nurnberg funnel. On the cover of his
artificial stupidity? Our education
book on minimalism, Carroll (1990)
system fails learners not because the
presented a picture of the Nurnberg
learners are stupid, but because our
funnel.
education systems and processes are deeply at fault. Creative use of the technology has to play a major role in providing more effective learning experiences for pupils and students. It must evoke, support, and enhance our powerful natural intelligence and ability to learn. Pedagogy is concerned with structuring the interactions of the learner to optimize learning. It is an interaction-based approach to enhancing learning. We are still a long way from producing optimal interaction contexts for learning. Is there a future science that will enhance learning by focusing directly at the neuro-physiological layer? Advances in computing and neural
Figure 2. The Nurnberg Funnel, by John M.
science are pointing to the realizability
Carroll, published by The MIT Press.
of what were, until recently, science
http://mitpress.mit.edu/9780262031639/
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
184
Michael Allenʼs 2008 e-Learning Annual
It represents learning as pouring
contribution, I have passed over what
knowledge through a funnel into
others may regard as major influences,
the head of the recipient. Carroll
such as artificial intelligence and
uses this image as a false view of
intelligent tutoring systems. This
learning, which stands in contrast to
is deliberate. I believe that this is
the constructivist view of learning
not where the central problem lies.
advocated in the book. Will future
The key question that we need to
technologies permit a partial return
ask, to make these technologies
of the Nurnberg funnel? This is a
work, lies elsewhere. It concerns
problematic vision. The biological
the fundamental relationship of
computer of the brain has actually
technology and the development of
evolved to resist such instant
human adaptive ability. This issue is
programming (because it can lead to
expressed most clearly in Bruner’s
inadequate, premature adaptations).
concept of cultural technological
Technological enhancement of natural
amplifiers. The central dream is
adaptive processes seems to represent
to shape the new technology as an
the most powerful way forward toward
amplifier of human learning. To
cultural-technological amplifiers of
pursue this dream we need creative
learning.
insights tempered by the evidence of experience. We need “beautiful ideas,”
Conclusion
Michael Allen asked
but we also need to be rigorous in
the contributors
assessing those ideas that work and
to this book to
those that do not. We need creativity
each produce a
and responsibility in pursuit of the
personal view. In my
realization of dreams.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Dreams and Responsibilities REFERENCES Boyle, T. (1997). Design for multimedia learning. Englewood Cliffs, NJ: Prentice Hall. Boyle, T. (2002). Toward a theoretical base for educational multimedia design. Journal of Interactive Media in Education: Special issue on Theory for Learning Technologies, 2. Available at www-jime.open.ac.uk/2002/2 Boyle, T. (2003). Design principles for authoring dynamic, reusable learning objects. Australian Journal of Educational Technology, 19(1), 46-58. Available: www.ascilite.org.au/ajet/ajet19/boyle.html Boyle, T. (2006). The design and development of second generation learning objects. Invited talk at Ed Media. In E. Pearson & P. Bohman (Eds.), Proceedings of Ed-Media 2006, World Conference on Educational Multimedia, Hypermedia & Telecommunications, June 26-30, 2006, Orlando, Florida. Boyle, T., Gray, J., Wendl, B., & Davies, M. (1994). Taking the plunge with CLEM: The design and evaluation of a large scale CAL system. Computers and Education, 22(1/ 2), 19-26. Boyle, T., & Margetts, S. (1992). The CORE guided discovery approach to acquiring programming skills. Computers in Education, 18(1-3), 127-133. Bradley, C., & Boyle, T. (2004). The design, development and use of multimedia learning objects. Journal of Educational Multimedia and Hypermedia: Special Edition on Learning Objects, 13(4), 371-389. Bruner, J. (1964) The course of cognitive growth. American Psychologist, 19, 115. Bush, V. (1945). As we may think. Atlantic Monthly, 176(1), 641-64. Carroll, J.M .(1990). The Nurnberg funnel: Designing minimalist instruction for practical computer skill. Cambridge, MA: MIT Press. CETL_RLO. (2007). www.rlo-cetl.ac.uk Donaldson, M. (1978). Children’s’ minds. London: Fontana. Duffy, T.M., & Jonassen, D.H. (1991). Constructivism: New implications for educational technology? Educational Technology, 31(5), 7-12. Gagné, R.M., & Briggs, L.J. (1979). Principles of instructional design. New York: Holt Rinehart and Winston. EASA RLOs. (2004). www.londonmet.ac.uk/ltri/learningobjects/list.htm IEEE LOM. (2002). Draft Standard for Learning Object Metadata. Available: http://ltsc.ieee.org/wg12/files/LOM_1484_12_1_v1_Final_Draft.pdf IMS. (2006). IMS Global Learning Consortium: www.imsglobal.org/
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
185
186
Michael Allenʼs 2008 e-Learning Annual
IMS LD. (2006). IMS Learning design specification. Available: www.imsglobal. org/learningdesign/index.html Jenkins, T., & Davy, J. (2001). Diversity and motivation in introductory programming. Italics, 1(1). Available at: www.ics.heacademy.ac.uk/italics/ issue1/tjenkins/003.PDF. Jonassen, D., Mayes, T., & McAleese, R. (1993). A manifesto for a constructivist approach to uses of technology in higher education. In T.M. Duffy, J. Lowyck, D.H. Jonassen, and T.M. Welsh (Eds.), Designing environments for constructive learning. Berlin: Springer-Verlag. Kay A. (undated) The Dynabook revisited. Available: www.squeakland.org/ school/HTML/essays/dynabook_revisited.htm Koppi, T., Bogle, L., & Lavitt, N. (2004) Institutional use of learning objects: Lessons learned and future directions. Journal of Educational Multimedia and Hypermedia, 13(4), 449-463. Lave, J., & Wenger, E. (1991) Situated learning: Legitimate peripheral participation. New York: Cambridge University Press. Papert, S. (1980). Mindstorms: Children, computers and powerful Ideas. New York: Basic Books Piaget, J. (1970). Piaget’s theory. In P.H. Mussen (Ed.), Carmichael’s manual of child’s psychology (3rd ed.). Hoboken, NJ: John Wiley and Sons. Plato. (1999). Phaedrus. (Jowett, trans.). The Project Gutenberg Etext of Phaedrus. Available: www.gutenberg.org/dirs/etext99/phdrs10.txt Vygotsky, L.S. (1962). Thought and language. Cambridge, MA: MIT Press. Wenger, E. (1998) Communities of practice: Learning, meaning, and identity. New York: Cambridge University Press. Yeats, W.B. (1914). Responsibilities. From The collected works of W.B Yeats (p. 112). New York: Macmillan (published 1967). All web references checked on 17 May 2007.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
David H. Jonassen Dr. David H. Jonassen is Distinguished Professor of Education at the University of Missouri, where he teaches in the areas of learning technologies and educational psychology. Since earning his doctorate in educational media and experimental educational psychology from Temple University, Dr. Jonassen has taught at the Pennsylvania State University, University of Colorado, the University of Twente in the Netherlands, the University of North Carolina at Greensboro, and Syracuse University. He has published thirty books and numerous articles, papers, and reports on text design, task analysis, instructional design, computer-based learning, hypermedia, constructivist learning, cognitive tools, and technology in learning. He has consulted with businesses, universities, public schools, and other institutions around the world. His current research focuses on the cognitive processes engaged by problem solving and models and methods for supporting those processes during learning.
Beyond Knowledge Dissemination
David H. Jonassen
Learning to Solve Problems Online In everyday and professional settings, people expend the greatest intellectual effort solving problems. Despite that, the corporate training and e-learning fields have ignored the need to help people learn to solve problems. In this article, the author argues that, if e-learning is to affect meaningful learning, it must constitute environments that engage and support problem solving. The author goes on to show how problem-solving skills can be taught online, illustrating his argument with two types of problems—troubleshooting and policy problems. Why Learn to Solve Problems?
Throughout the
retained more information, that is,
forty-year history
they learned more. The history of
of online learning
education technology and online
(beginning with
learning have paralleled the history of
computer-assisted
formal instruction, all of which focus
instruction through current interactive
on presenting students with content.
web sites), the primary focus of those
Students in sociology courses learn
efforts has been to support knowledge
about sociology. Students in history
acquisition. Until the constructivist
courses learn about history, and so on.
and situated learning revolution in
e-Learning has followed this trend. If
the field of educational technology
you present it, they will learn.
that began in the early 1990s, design
Beginning in the early 1990s,
and development effort focused
learning theory and (to a lesser extent)
on designing increasingly effective
educational practice were influenced
messages that were transmitted to
by constructivist and situated learning
students through the technology. The
beliefs. The important epistemological
belief, supported in many cases with
assumption of constructivism and
empirical research, was that better
situated learning is that meaning is a
designed instructional messages more
function of how the individual creates
effectively communicated ideas to
meaning from his/her experiences and
students. When student learning,
how we socially co-construct our belief
typically in the form of recall, was
systems and understandings (Jonassen,
measured, students exposed to more
1991). We all conceive of the external
effective instructional messages
reality somewhat differently, based on
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
190
Michael Allenʼs 2008 e-Learning Annual
our unique set of experiences with
and experiential knowledge)
the world and our beliefs about them.
(Jonassen, 2006). Epistemological
Despite how we design messages
knowledge types result from applying
or educational
knowledge in long-term memory,
Despite how we design experiences,
while phenomenological knowledge
messages or educational learners will
types result from the experiences of
experiences, learners will necessarily interpret necessarily interpret them. them somewhat differently. Constructivism has been
applying ontological knowledge in long-term memory. Those types of knowledge most frequently result from solving problems, especially
consistently misinterpreted as an
ill-structured problems, so they
instructional methodology normally
cannot be directly taught. Given that
associated with discovery and
phenomenological (experiential)
experiential learning methods.
knowledge representations, in the
Kirschner, Sweller, and Clark (2006)
form of stories, are the most natural,
recently launched an attack on
comprehensible, and memorable
“constructivist, discovery, problem-
representations of knowledge, the case
based, experiential, and inquiry-based
for direct, guided instruction (online
teaching,” claiming that direct, guided
or not) is not as strong. To summarize
instruction has always been more
these arguments, unless content is
effective for supporting learning.
applied (content-in-use), it is not as
But what are the students learning?
useful. Given that, online learning
The knowledge students recall is
should focus less on delivering content
ontological (knowing that) and too
and more on engaging learners in
often becomes inert because it is
meaningful tasks that require them
disembedded from its meaningful
to apply what they are learning, and
context and is not applied. When
in doing so, to construct other more
knowledge is constructed by learners
meaningful (epistemological and
and is applied to some intentional
phenomenological) knowledge. The
and meaningful task, that knowledge
most meaningful (purposeful and
includes epistemological knowledge
authentic) learning task is problem
types (procedural, situational,
solving.
and strategic knowledge) that
In my work, I argue that the
are based on experiences, that is,
only legitimate goal of education
phenomenological knowledge types
(formal, informal, or other) in every
(tacit, compiled, socio-cultural,
educational context—public schools,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Beyond Knowledge Dissemination
191
universities, and (especially) corporate
is too quickly forgotten, cannot
training—is problem solving. I support
be effectively applied, and in most
this claim with five warrants. First,
disciplines becomes obsolete in a short
problem solving is the most authentic
time. Therefore, the primary purpose
and therefore the most relevant
of online learning should be to
learning activity that students can
engage and support learning to solve
engage in. Karl Popper (1999) wrote
problems.
a book of essays in which he claimed that all life is problem solving. In everyday contexts, including work and personal lives, people solve problems. No one in personal and professional contexts is rewarded
Constraints in ProblemSolving Instruction
Having justified the importance of learning to solve problems, it
solely for memorizing information
is also important to address some of
and completing examinations. In
the constraints of problem-solving
professional and personal contexts,
instruction. First, the nature of
people solve problems constantly.
problems varies. That is, there are
Second, research has shown that
different kinds of problems that
knowledge constructed in the
require different cognitive processes in
context of solving problems is better
order to solve. How do problems vary?
comprehended, retained, and therefore
Problems and problem solving
more transferable (Sawyer, 2006).
vary in several ways. Several authors
Third, problem solving requires
(Jonassen, 1997; Simon, 1971; Voss
intentional learning. Learners must
& Post, 1989) have distinguished
manifest an intention to understand
well-structured from ill-structured
the system or context in which
problems and recommended different
problems occur in order to solve
learning approaches for each. Most
problems effectively. Meaningful
problems encountered in schools
learning cannot occur until and unless
and universities are well-structured
learners manifest an intention to learn.
problems. Well-structured problems
Fourth, life is short. Time allocated
typically present all elements of the
to learning in every context is always
problem; engage a limited number of
limited. So why not make the most
rules and principles that are organized
effective use of the time available?
in a predictive and prescriptive
Fifth, knowledge that is recalled and
arrangement; possess correct,
not used in some authentic tasks
convergent answers; and have a
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
192
Michael Allenʼs 2008 e-Learning Annual
preferred, prescribed solution process.
A final dimension of problems and
Ill-structured problems, on the other
problem solving is domain specificity.
hand, are the kinds of problems that
In contemporary psychology, there
are encountered in everyday practice.
is a common belief that problems
Ill-structured problems have many
within a domain rely on cognitive
alternative solutions, vaguely defined
strategies that are specific to that
or unclear goals and constraints,
domain (Mayer, 1992; Smith, 1991;
multiple solution paths, and multiple
Sternberg & Frensch, 1991). These are
criteria for evaluating solutions, so
often referred to as strong methods, as
they are more difficult to solve.
opposed to domain-general strategies
Problems also vary in complexity.
(weak methods). For example,
The complexity of a problem is a
Lehman, Lempert, and Nisbett (1988)
function of the number of issues,
conclude that different forms of
functions, or variables involved in the
reasoning are learned in different
problem; the number of interactions
graduate disciplines. Graduate
among those issues, functions, or
students in the probabilistic sciences
variables; and the predictability of the
of psychology and medicine perform
behavior of those issues, functions, or
better on statistical, methodological,
variables. Ill-structured problems tend
and conditional reasoning problems
to be more complex; however, there
than do graduate students in law and
are a number of highly complex well-
chemistry, who do not learn such
structured problems, such as chess.
forms of reasoning. The cognitive
Another dimension of problem
operations are learned through the
complexity is dynamicity. In static
development of pragmatic reasoning
problems, the elements of the problem
schemas, rather than exercises in
space do not change. In dynamic
formal logic. Graduates in different
problems, the relationships among
domains develop reasoning skills
variables or factors change over time.
through solving situated, ill-structured
Changes in one
problems that require forms of logic
In dynamic problems, the
factor may cause
that are domain-specific.
relationships among variables
variable changes
How do problems vary within
or factors change over time.
in other factors.
these dimensions? Jonassen (2000)
The more intricate
describes a typology of problems that
these interactions, the more difficult
vary primarily along a continuum
any solution is. Ill-structured problems
from well-structured to ill-structured,
tend to be more dynamic.
including puzzles, algorithms, story
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Beyond Knowledge Dissemination
193
problems, rule-using problems,
and test those solutions. While
decision making, troubleshooting,
problem-solvers do engage in those
diagnosis-solution problems, strategic
activities while solving problems,
performance, systems analysis,
they perform them in different ways
design problems, and dilemmas.
in different contexts with different
This typology assumes that there are
problems. Traditional models do not
similarities in the cognitive processes
adequately represent the complexity of
for solving problems within classes.
problem solving.
Within each category of problems that
Another constraint of problem-
is described, problems can vary with
solving instruction, especially in online
regard to abstractness, complexity,
instructional contexts, is the support
and dynamicity. Because of space
systems for delivering instruction.
limitations, I will describe only some
Current course management systems,
of these problem types.
such as Blackboard or Moodle, were
The important design principle
designed to replicate the activities in
is that, because problems and the
traditional, face-to-face instruction.
cognitive processes for solving them
These systems lack pedagogical
differ, the strategies for instructing
affordances. They do not support the
different kinds of problem solving
use of alternative forms of knowledge
must also differ. Traditional models
representation by learners, authentic
of transmissive instruction cannot
forms of assessment, or the use of
support problem solving. Telling
distributed tools to scaffold different
students how to solve problems
forms of reasoning required to solve
and expecting them to apply those
problems, limiting the range of student
instructions cannot enable learners
learning outcomes to reproductive
to solve the range of problems they
learning—unless faculty members and
will encounter in their everyday lives.
students work around these limitations
Additionally, traditional approaches
(Marra & Jonassen, 2001). What
to problem solving instruction (e.g.,
online learning designers need are
Bransford & Stein, 1984) are not
architectures for authoring learning
adequate for enabling learners to
environments in order to support
solve the range of problems they will
different kinds of problem solving. In
encounter in their everyday lives.
the remainder of this article, I briefly
Traditional approaches help learners
describe architectures for designing
to construct a problem representation,
two different kinds of problems,
search for solutions, and implement
troubleshooting and policy problems.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
194
Michael Allenʼs 2008 e-Learning Annual
I have designed models for other kinds
troubleshot many years before. The
of problems, such as story problems
problems that they most completely
(Jonassen, 2003) and am working
and accurately recall are those that
on design models for other kinds of
were most difficult to solve, because
problems as well, such as design and
they were more conceptually engaged
decision-making.
by the process. So teaching novices to troubleshoot requires that they
Learning to Solve Troubleshooting Problems
Effective
troubleshoot as many problems
troubleshooting
as possible in order to gain the
requires
experiential knowledge that will
conceptual
integrate the conceptual, procedural,
understanding
and strategic knowledge that is
of the functions of the malfunctioning system, including knowledge of
required to troubleshoot. Figure 1 (right) illustrates
the topography of the system, the
an architecture for building
functional relationships among system
troubleshooting learning
components, and the normal states of
environments (TLEs). The model
those components; knowledge of the
assumes that the most effective
procedures for testing and preparing
way to learn to troubleshoot is by
system components; and strategic
troubleshooting problems. Learning
knowledge about when, where, and
to troubleshoot problems requires
why to apply various procedures
presenting learners with the symptoms
(Jonassen & Hung, 2006). Those
of novel problems and requiring them
kinds of knowledge are anchored to
to solve them. The major components
and indexed by the troubleshooter’s
of the TLE include a case library
experiences. The most fundamental
of previously solved problems, a
difference between expert and novice
diagnoser that enables the learner to
troubleshooters is their level of
practice troubleshooting, and a rich
experience. Troubleshooting relies
conceptual model of the system being
on experiential knowledge, which is
troubleshot. The conceptual model
exactly what novices lack. Experienced
supports the construction of systems
technicians through physicians
knowledge; the diagnoser supports
index their knowledge around their
the construction of procedural and
troubleshooting experiences. Often,
strategic knowledge; and the case
they can recall with extraordinary
library supports the construction
accuracy problems that they have
of the experiential knowledge that
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Beyond Knowledge Dissemination
195
Problem Set
Context Symptoms
Previously Solved Problems
Fault Symptoms Frequency Actions Hypothesis Results Topography Function Strategy Solution
Diagnoser
Action Hypothesis Probability Result Interpretation
Conceptual Model Layers Pictoral Topographic State Functional Strategic Action
Figure 1. Model for Designing Troubleshooting Learning Environment
integrates all of the other kinds of
car that will not start, the mechanic
knowledge. I briefly describe each
will not resort to trial-and-error,
below.
serial elimination, or other weak troubleshooting strategies, because
Conceptual Model The
s/he knows how the automotive
troubleshooting environment is
electrical system works, that is, the
oriented by the conceptual model
function of each of the components
of the system being troubleshot.
and how they influence each other.
The conceptual model illustrates
The novice may not understand all of
the interconnectedness of systems
the components or their interactions
components; that is, all of the
and so may access instruction
components and subsystems of the
about the components and their
system in which the problem occurs
interrelationships. Conceptual
and how they influence each other.
understanding of the system is
For example, when troubleshooting a
essential for troubleshooting.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
196
Michael Allenʼs 2008 e-Learning Annual
Diagnoser The heart of the TLE is
an implicit form of argumentation,
the diagnoser, a simulation (see Figure
requiring the learner to justify the
2) whereby the learner functions
action taken. If the hypothesis is
like an experienced troubleshooter
inconsistent with the action, then
by troubleshooting new cases.
feedback can be immediately provided
After listening to a story about the
about the rationale for taking such an
automotive problem that describes
action. Next, the learner must also
the symptoms just before the car
identify the sub¬system in which
ceased to work, the learner (like an
the fault occurs. If the subsystem
experienced troubleshooter) first
is inconsistent with the action, the
selects an action using the pull-
learner is immediately sent to the
down menu at the left of the screen,
conceptual model to better understand
such as ordering a test, checking a
the workings of the subsystem that
connection, or trying a repair strategy.
leads to the action or hypothesis.
The novice may be coached about
The learner then receives the result
what action to take first based on
of action (e.g. test results, system
the symptoms, or s/he may be free to
information, etc.) to the right of the
select any action. Each action taken
subsystem and must interpret those
by the troubleshooter shows up in
results using the pull-down menu at
the systems model. For each action
the right of the troubleshooter. If the
the learner takes, the troubleshooter
interpretation is inconsistent with
next requires the learner to state or
the action, hypothesis, or subsystem,
select a fault hypothesis that s/he is
then an error message is triggered.
testing using the pull-down menu to
The error checking uses a very simple
the right of the action menu. This is
evaluation system.
Figure 2. Diagnoser Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Beyond Knowledge Dissemination
The diagnoser requires the learner
197
kind of community. Stories about
to think and act like an experienced
how experienced troubleshooters
troubleshooter. The environment
have solved similar troubleshooting
integrates the troubleshooting
problems are contained in, indexed
actions, knowledge types (conceptual,
by, and made available to learners in
strategic, and procedural), and
a case library (also known as a fault
conceptual systems model with a
database).
database of faults that have occurred
The case library or fault database
with the system that the learner and
contains stories of as many
others have solved. Initial instruction
troubleshooting experiences as
in how to use the system is provided
possible. Each case represents a story
by worked examples. As learners
of a domain-specific troubleshooting
solve troubleshooting problems, the
instance. Case libraries, based on
results of their practice cases can be
principles of case-based reasoning,
added to the learner’s case library of
represent the most powerful form of
fault situations, so that the learner
instructional support for ill-structured
can learn from his/her own personal
problems such as troubleshooting
experience.
(Jonassen & Hernandez-Serrano, 2002). The case library indexes each
Case Library If the diagnoser is the
case or story according to its system
heart of the TLE, the case library is the
fault, the system or subsystem in
head (memory) of the TLE. Discourse
which the fault occurred, and the
is essential to socially negotiating
symptoms of the fault, similar to the
problems, solutions, or meaning. In
troubleshooter. The failure mode,
troubleshooting situations in everyday
hypotheses or strategies that were
contexts, the primary medium of
tested, the results of those tests, and
negotiation is stories. That is, when a
what lessons were learned from the
troubleshooter experiences a problem,
experience are also contained in the
s/he most often describes the problem
case library.
to someone else, who recalls from
The case library represents
memory a similar problem, telling
the experiential knowledge of
the troubleshooter about the recalled
potentially hundreds of experienced
experience. These stories provide
troubleshooters. Why? Because
contextual information, work as a
troubleshooters almost invariably
format for diagnosis, and also express
store their knowledge of problems and
an identity among participants in any
solutions in terms of their experiences.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
198
Michael Allenʼs 2008 e-Learning Annual
The best troubleshooters are the most
a case library during learning how
experienced troubleshooters, and their
to solve problems improves complex
experiential knowledge is precisely
problem-solving performance on an
what learners do not possess. So when
examination.
a learner encounters any difficulty or is uncertain about how to proceed, the
Integrating TLE Components Figure
learner may access the case library to
3 illustrates an evidence screen from a
learn about similar cases, what was
troubleshooting environment designed
done, and what the results were.
to support a course on entomology.
The TLE can also be programmed
Rather than memorizing the names
to automatically access a relevant
and attributes of several hundred
story when a learner commits an
insects, students in this introductory
error, orders an inappropriate test, or
entomology course were presented
takes some other action that indicates
with insect problems, including an
a lack of understanding. Stories are
infestation problem, an insect bite
easily collected from experienced
problem, and several others. The
troubleshooters by presenting them
illustration in Figure 3 is part of
with a problem and asking them if
the evidence that is presented to
they are reminded of a similar problem
support solution of a case in which
that they have solved. Invariably
someone returning from a trip to Asia
they are. Hernandez-Serrano and
was covered with insect bites. The
Jonassen (2003) show that access to
environment includes a conceptual
Figure 3. Insect Troubleshooting Environment Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Beyond Knowledge Dissemination
199
model in the form of the taxonomy,
are usually many perspectives and
pest sheets, and the log book in
themes (personal, economic, political,
which students take actions while
social, historical, and so forth) that
troubleshooting the problem.
impinge on the policy. As such, they represent the antithesis of most
Policy Problems
Pick up any
formal education, which focuses on
newspaper,
correct answers and finding “truth.”
and it is likely
Analyzing and trying to solve these
that the front
problems is challenging for most
page includes
learners. Policy analysis problems
at least one article that describes
often present unknowable phenomena
a policy problem. Governments,
that must be socially negotiated and
agencies, and organizations of all types
co-constructed. There never is a single
confront policy problems regularly.
perspective that represents the truth.
Policy problems generally describe
Solving these problems requires that
the efforts of organizations to define
students accommodate ambiguity.
and adopt policies that constrain their
I briefly describe the components
operations. Policy analysis problems
of policy analysis problem-solving
tend to be complex, ambiguous, and
environments below, the model of
very ill-structured because there
which is illustrated in Figure 4.
Figure 4. Architecture for Policy Analysis Problem-Solving Environments Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
200
Michael Allenʼs 2008 e-Learning Annual
Problem Representation Because
and, equally important, what are the
policy analysis problems (as all ill-
contextual constraints that affect the
structured problems) tend to be
problem?
more context-dependent than well-
Policy analysis problems are often
structured problems, it is necessary to
represented by stories or reports. As
develop a more authentic and situated
described earlier, stories are better
task environment. If policy analysis
understood and remembered than
is constrained largely by the context
didactic representations of problems.
and the domain that it represents, it is
A few years back, we created a policy
important to adequately describe the
analysis environment related to the
social, political, and organizational
Kosovo crisis in 1999. The students’
context of the problem. Therefore,
task was to advise then-President
a context analysis needs to be
Clinton on how the U.S. should
conducted. What is the nature of the
proceed. It was a complex policy
domain? What are the constraints
analysis problem that raised a host
imposed by the context? What kinds
of issues and perspectives. Figure 5
of problems are solved in this domain,
shows the first part of the original
Figure 5. Initial Report in a Policy Analysis Environment Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Beyond Knowledge Dissemination
201
presidential briefing. The task that
concept mapping tools to generate
orients a policy analysis must be
semantic representations of problems.
fairly specific, with an identifiable,
Other tools such as expert systems and
assessable outcome. That is, there
systems dynamics modeling tools such
should be a specific kind of outcome
as Strella, are more difficult to learn
(advice) associated with the task: not
but produce richer models of problems
just a report but a report with specific
situations. See Jonassen, 2006, for a
action items. This does not mean that
description of these tools.
a particular kind of advice should be given, just the form of advice.
Generate Solution Options:
The more purposeful the task, the
Accommodating Multiple
more engaging it will be. The same
Perspectives In order to generate
environment with all of its support
viable solutions to policy analysis
systems may be altered by redefining
problems, students must learn to
the task. For example, students using
accommodate multiple perspectives.
the Kosovo environment might
These perspectives may be personal
construct funding recommendations
or they may be thematic. I have
for reconstruction. Except for the task,
found one of the best models for
the remainder of the environment may
facilitating the consideration of
be the same or very similar.
multiple perspectives and becoming an epistemically mature leaner is
Problem Representation Tools The
cognitive flexibility theory (Spiro &
goal is to get learners to construct a
Jehng, 1990). Cognitive flexibility
meaningful, conceptual model of the
theory stresses the conceptual
problems they are trying to solve.
interrelatedness of ideas and their
The ways that we represent problems
interconnectedness. Cognitive
to learners in the problem statement
flexibility environments intentionally
affects how they mentally represent
represent multiple perspectives or
the problems they are trying to solve.
interpretations of the content in the
Problem representation is only one
cases that are used to illustrate the
source of influence, however. The
content domain. The ill-structuredness
model for engaging learners in case
of any knowledge domain is best
analysis problems calls for the use of
illustrated by multiple perspectives
tools by learners to construct their
or themes that are inherent in the
own external representation of the
problems that are represented. In the
problems. Learners regularly use
Kosovo environment, the President
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
202
Michael Allenʼs 2008 e-Learning Annual
Figure 6. One of Several Perspective Provided in a Presidential Briefing
receives a complete briefing, including
used to support different arguments
the perspectives of numerous federal
with opposing assumptions underlying
agencies (see left column in Figure 6).
them (Churchman, 1971). So it is
In order to render a judgment on the
important that learners be able to
continuation of the war, students had
articulate the differing assumptions
to understand and accommodate these
in support of arguments for whatever
viewpoints. It is essential that students
solution that they recommend. The
understand the different perspectives
argument provides the best evidence
that make case analysis problems
for domain knowledge that they
complex in order to generate and
have acquired. Developing cogent
evaluate different solutions.
arguments to support divergent thinking (reflective judgment
Argumentation Ill-structured
[Kitchner & King, 1981]) engages not
problems such as policy analysis
only cognition and metacognition
problems are dialectical in nature,
of the processes used to solve the
in which two or more opposing
problem, but also awareness of the
conceptualizations of the problem are
epistemic nature of the process
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Beyond Knowledge Dissemination
203
and the truth or value of different
organizing and presenting information
solutions (Kitchner, 1983). In the
to be learned. The constructivist
Kosovo environment, we provided
and situated learning revolutions in
students with a scaffold for
education have resulted in learning
constructing a coherent argument in
environments in which learners
support of their recommendations to
must learn how to do something. In
the president. Coaching or prompting
doing so, they construct different
can be provided in the form of a series
kinds of more meaningful knowledge
of reflective judgment prompts or
than would result from declarative
questions (Kitchner & King, 1981),
presentations.
such as:
Further, I argued that the most
ÿ Can you ever know for sure that
meaningful kind of doing is problem
your position is correct? Will we
solving, because that is what people do
ever know which is the correct
most in their everyday lives. In order
position?
to make problem-solving instruction
ÿ How did you come to hold that
viable (scalable) for online learning,
point of view? On what do you
we need to develop templates or
base it?
architectures for orienting the design
ÿ When people differ about matters
and development of problem-solving
such as this, is it ever the case that
instruction. I briefly demonstrated
one is right and the other wrong?
architectures and examples for
One opinion worse and the other
environments that engage and
better?
support troubleshooting and policy
ÿ How is it possible that people can have such different points of view? ÿ What does it mean to you when the experts disagree on this issue?
analysis problems. I am working on similar architectures for other kinds of problem-solving environments that should become available in the future. If we are to maintain our
Summary
In this paper,
economic leadership and remain a
I have argued
productive society and economy, our
that the brief
graduates and employees of the future
history of
must be better prepared to solve the
online learning
increasingly complex and structured
has been oriented by efforts to
problems of the 21st Century.
tell students about the world by
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
204
Michael Allenʼs 2008 e-Learning Annual REFERENCES Bransford, J.D., & Stein, B.S. (1984). The ideal problem solver: A guide for improving thinking, learning, and creativity. New York: W.H. Freeman. Churchman, C.W. (1971). The design of inquiring systems: Basic concepts of systems and organizations. New York: Basic Books. Hernandez-Serrano, J., & Jonassen, D.H. (2003). The effects of case libraries on problem solving. Journal of Computer-Assisted Learning, 19, 103-114. Jonassen, D.H. (1991). Objectivism vs. constructivism: Do we need a new paradigm? Educational Technology: Research and Development, 39(3), 5-14. Jonassen, D.H. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology: Research and Development, 45(1), 65¬-95. Jonassen, D.H. (2000). Toward a design theory of problem solving. Educational Technology: Research & Development, 48(4), 63-85. Jonassen, D.H. (2003). Designing research-based instruction for story problems. Educational Psychology Review, 15(3), 267-296. Jonassen, D.H. (2006a). Accommodating ways of human knowing in the design of information and instruction. International Journal of Knowledge and Learning, 2(3/4), 181-190. Jonassen, D.H. (2006b). Modeling with technology: Mind tools for conceptual change. Columbus, OH: Merrill/Prentice-Hall. Jonassen, D.H., & Hernandez-Serrano, J. (2002). Case-based reasoning and instructional design: Using stories to support problem solving. Educational Technology: Research and Development, 50(2), 65-77. Jonassen, D.H., & Hung, W. (2006). Learning to troubleshoot: A new theorybased design architecture. Educational Psychology Review, 18, 77-114. Kirschner, P.A., Sweller, J., & Clark, R.E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75-86. Kitchner, K.S. (1983). Cognition, metacognition, and epistemic cognition: A three-level model of cognitive processing. Human Development, 26, 222-232. Kitchner, K.S., & King, P.M. (1981). Reflective judgment: Concepts of justification and their relationship to age and education. Journal of Applied Developmental Psychology, 2, 89-116.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Beyond Knowledge Dissemination
Lehman, D., Lempert, R., & Nisbett, R.E. (1988). The effects of graduate training on reasoning: Formal discipline and thinking about everyday-life events. Educational Psychologist, 43, 431-42. Marra, R.M., & Jonassen, D.H. (2001). Limitations of online courses for supporting constructive learning. Quarterly Review of Distance Education, 2(4), 303-317. Mayer, R.E. (1992). Thinking, problem solving, cognition (2nd ed.). New York: Freeman. Popper, K. (1999). All life is problem solving. London: Routledge. Sawyer, K. (2006). The Cambridge handbook of the learning sciences. Cambridge, UK: Cambridge University Press. Simon, H. (1971). The sciences of the artificial. Cambridge, MA: MIT Press. Smith, M.U. (1991). A view from biology. In M.U. Smith (Ed.), Toward a unified theory of problem solving. Mahwah, NJ: Lawrence Erlbaum Associates. Spiro, R.J., & Jehng, J.C. (1990). Cognitive flexibility and hypertext: Theory and technology for the non-linear and multi-dimensional traversal of complex subject matter. In D. Nix & R.J. Spiro (Eds.), Cognition, education, and multimedia: Explorations in high technology. Mahwah, NJ: Lawrence Erlbaum Associates. Sternberg, R.J., & Frensch, P.A. (1991). Complex problem solving: Principles and mechanisms. Mahwah, NJ: Lawrence Erlbaum Associates. Voss, J.F., & Post, T.A. (1989). On the solving of ill-structured problems. In M.T.H. Chi, R. Glaser, & M.J. Farr (Eds.), The nature of expertise. Mahwah, NJ: Lawrence Erlbaum Associates.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
205
Greg Kearsley Greg Kearsley is an independent consultant who specializes in the design, development, and evaluation of online courses and programs. He has taught or been involved in the development of many online graduate programs, including the George Washington University, the University of Maryland College Park, Nova Southeastern University, the University of Wisconsin, and the U.S. Merchant Marine Academy. He also designed and taught in online teacher training programs for the North Central Educational Lab and Walden Institute. He has written over twenty-five books about technologybased learning. Currently, Kearsley is e-learning consultant to The Human Equation (www.thehumanequation.com), a provider of online courses for HR training and risk management technology, and also to the U.S. Merchant Marine Academy, where he assists in the development of an online master’s degree in marine engineering (www.usmma.edu/gradcourse).
What’s Right and What’s Wrong with Online Learning
Greg Kearsley
In this article, the author explores the evolution of online learning using his own experiences with both teaching online and teaching others to teach online. Beginning with individual courses, then moving on to whole programs and eventually learning communities, the author summarizes what works and what does not and presents some of the obstacles that have to be overcome to make online learning more effective. In this article I discuss the evolution
(e.g., television, correspondence,
of online learning from my personal
telephone/fax). 1 Also, computers
perspective, which is based on more
are used for learning in classroom
than twenty-five years as a teacher,
settings that do not involve remote
course developer, researcher, and
networking (e.g., use of CD-ROMs or
evaluator. I include case studies from
slideshow presentations); I would not
projects I have been involved with
consider this to be a form of online
as illustrations of specific issues and
learning. It is the virtual nature (that
topics. My work has spanned business
is, lack of physical presence of others)
training, graduate education, and public
of online learning, accomplished via
schools (see Kearsley, 1985, 2000, 2005).
networks, that produces its distinctive
A clarification about terminology: I
characteristics. 2
am using online learning to refer to
According to this definition of online
any form of learning using a computer
learning, many of the earliest forms
that takes place at a distance. However,
of computer-based instruction (dating
online learning is not synonymous
back to the 1960s) would be included.
with distance learning, since there
To the extent that students worked by
are other forms of delivering distance
themselves at a networked computer,
learning that don’t involve computers
we can consider them engaged in
1
On the other hand, most forms of distance learning do involve computers to some extent, so the terms are close to being synonymous (at least in developed countries). As all media becomes digital in format (e.g., video, voice), this distinction will become even more slight. (For more discussion about the nature of distance education, see Moore & Kearsley, 2005).
2
The question of whether learning delivery via local area networks (LANs) within an institution counts as online learning is a bit tricky since it can involve remote presence. The key consideration is whether the instructor and students are physically together in a classroom. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
208
Michael Allenʼs 2008 e-Learning Annual
online learning. So, in this review,
e-mail. Digital communications made
we will take into account some of the
a major improvement over the phone
historical background that began with
conferences because most interaction
computer-based instruction.
took place asynchronously, eliminating the need for scheduling class times,
Online Learning by Necessity
The rationale
and it was easy to exchange written
for engaging
assignments without having to use the
in online
fax or go to campus.
learning is a critical factor
While I originally adopted the use of media to teach to make it easier
in whether it will be successful.
for me, it quickly became clear that
Because of its importance, it is one of
it made things a lot easier for my
the issues that I am going to probe a
students, most of whom were working
number of times in this article. I’d like
adults and also traveled extensively.
to begin examining the rationale for
Without any intention of doing so,
online learning by describing how I
I found myself involved in distance
became involved with it personally.
education. In fact, once I became used
Soon after I completed my Ph.D. in
to teaching at a distance, I developed
1978, I began to teach as an adjunct
a dislike for conventional classroom
professor in graduate programs
teaching. Not only was it inconvenient
at local (San Diego) universities.
to drive to campus to teach a class
However, I was traveling extensively,
(assuming I was in town), but I found
and it was problematic to be present to
the limited interaction with students
teach classes. To solve this dilemma,
possible during a sixty- or ninety-
I discovered that I could have a
minute class session to be much less
speakerphone set up at the front
satisfying compared to the extensive
of the classroom and then conduct
interaction that occurred in an online
my class via phone from wherever
course. In online courses, I could
I was. Although interacting with
spend as much time as I wanted
students via phone wasn’t as good
with each student, reviewing his/her
as being there, I was surprised that I
work and coaching him/her. And this
could accomplish quite a bit. A few
level of student interaction with the
years later, electronic bulletin boards
instructor, as well as among students,
appeared, and that made it possible
was much more satisfying to students
for me to interact with the students
as well.
via discussion postings, chats, and Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Whatʼs Right and Whatʼs Wrong with Online Learning
Indeed, as time went on, I realized
209
it is based on a fascination with
that peer interaction was the real
technology. What does not work is
strength of online learning and also a
simply being told to take or teach a
solution to a major dilemma of online
course online when there is no strong
teaching. It became clear that, given
personal desire to do so. The level of
the right stimulus, students could
commitment and discipline needed to
learn a lot more from each other than
do well in online learning or teaching
they could from me. The necessary
is much too high without such a
stimulus was a problem, case study, or
rationale.
project that required them to interact
Much of the initial push for
in a structured and meaningful
computer-based learning in schools
fashion, and I began to design my
and companies was based on the idea
course activities to provide such
that it would somehow save money.
learning events. And this helped to
However, organizations that create
solve the dilemma of having to spend
online learning programs based on
so much time providing individual
cost/benefit or revenue-generation
feedback on assignments; I had them
justifications without a genuine desire
critique and evaluate each other’s
to learn/teach online on the part of
work. 3
the staff or faculty are very unlikely to
What I want to emphasize about
succeed.
my initial involvement with online learning was that it came about for pragmatic reasons—as a solution to teaching while traveling. But it
From Individual Courses to Programs
By the end of my first ten years of teaching, I was a
was sustained by my belief that it
major advocate
resulted in a much more effective
of distance
learning environment. When you
education and began to examine
analyze successful examples of online
related research and practice. This led
learning, you will find a compelling
to my involvement in the Master of
rationale based on the convenience of
Educational Technology Leadership
teaching/learning online or increased
(ETL) program at The George
learning effectiveness. In some cases,
Washington University, which was one
3
Many commercially offered online courses (e.g., business training) do not involve any form of peer or instructor interaction but are strictly self-study. In my opinion, these courses are unlikely to result in very effective learning because they are missing the most important elements of online learning. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
210
Michael Allenʼs 2008 e-Learning Annual
of the first online graduate programs
web pages for course information. The
in the United States (see www.gwu.
latter were dependent on the HTML
edu/etl). Up to this point, I had simply
prowess of the faculty member and
taught individual courses and wasn’t
ranged from simple text documents to
concerned with the issues associated
pages with multimedia. 4
with an entire program. A program consists of multiple
Needless to say, there was a lot of variability in the effectiveness of the
courses, multiple faculty and support
online environment across courses
staff, and many students, all of which
as some faculty made better or worse
has to be managed and coordinated.
use of each component. Of course,
If each faculty member adopts his
comprehensive training and support
or her own approach to teaching
can made a big difference in faculty
online, the resulting variations and
skills with technology, but we didn’t
inconsistencies can be very frustrating
have this for the ETL program.
for students—while also making
One of the interesting aspects
things complicated for technical and
of the ETL program was that the
administrative support. Furthermore,
course enrollments were often quite
faculty are often asked to teach online
large (fifty to one hundred students)
courses because of their areas of
because the program was based
expertise, not because they have any
on a broadcast model for distance
interest or inclination to teach online.
education (where large classes
This can be a serious problem, because
with little interaction are typical).
online teaching is very different from
Such large online classes cannot
classroom teaching, and someone who
be reasonably handled by a single
is good at one may not be good at (or
faculty member, so we resorted to
enjoy) the other. The result is that the
the same methodology that is used
quality of the program varies across
in undergraduate courses: teaching
courses as a function of each faculty
assistants (TAs). We hired our own
member’s approach, aptitude, and
graduate students, who had completed
attitude toward online teaching.
courses and done well, to be online
The ETL online delivery system
TAs. Courses with large enrollments
consisted of e-mail distribution lists,
were divided into sections of twenty
a discussion board, and eventually
to twenty-five students, and a TA was
4
The original ETL program also involved weekly cable television broadcasts, which added another element of complexity to the teaching and administrative environment. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Whatʼs Right and Whatʼs Wrong with Online Learning
211
Figure 1. Blackboard & WebCT
assigned to each section. Their role
(see http://mepp.engr.wisc.edu). By
was to grade assignments and provide
this time, two major developments had
feedback to students. The main job
occurred in distance learning software
of the faculty member was to prepare
tools: course management systems
and maintain the course materials and
(WebCT, Blackboard, DesiretoLearn)
supervise the TAs. This was a great
and web conferencing tools
boon to the students who served as
(Placeware, HorizonLive, Centra).
TAs because they gained first-hand
The former provide an integrated
experience with online teaching. It
set of tools that include a discussion
also introduced another level of peer
board, e-mail, test creation and
learning into the program.
scoring, a gradebook, file uploading/
The ETL program was quite
downloading, and content editing,
successful, despite a somewhat
including multimedia information.
fragmented and chaotic online
Course management systems represent
delivery system and minimal
the latest incarnation of authoring
technical/administrative support. 5
tools for computer-based instruction,
This success was probably due to the
which have been evolving for many
enthusiasm of most of the faculty
years (Authorware, Toolbook,
(who were educational technologists)
CourseBuilder). Similarly, web
and the students (who were teachers
conferencing tools, which provide a
and trainers partial to educational
shared whiteboard and audio, are more
technology). In other words, we
advanced forms of simple text-based
had a very favorable milieu for
chat systems that have been a part of
online learning, despite the lack of
the Internet landscape for a long time.
institutional resources. The next program I was involved
The MEPP program used both classes of tools, which provided
in was the Master’s in Engineering
a sophisticated and consistent
Professional Practice (MEPP) at the
delivery system. They were easy
University of Wisconsin, Madison
for students/faculty to learn to use,
5
While it is difficult to define the success of a program, it can be done quantitatively—in terms of the number of graduates and the completion rates, as well as qualitatively—namely how the graduates rate the effectiveness and utility of what they learned in the program. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
212
Michael Allenʼs 2008 e-Learning Annual
very reliable, and supported a broad
responsibility was critical. Faculty
range of learning strategies. They
who taught in the program included
were supplemented by two additional
full-time UW professors as well as
tools: a screen recording program
individuals from other institutions and
(Camtasia) and a document-sharing
consultants. All MEPP faculty elected
system (Xerox Docushare).
to teach in the program and were interested in distance teaching. They received both formal and informal preparation for online teaching over a period of several months while developing their courses.
Figure 2. Camtasia
One other distinguishing
Studio
characteristic of the MEPP program was the instructional model, which The first allowed faculty to prepare
short tutorials and demonstrations by dynamically capturing screens and narrating them. The second provided students and faculty with greater capability to edit and manage files than was provided in course management systems. In addition to the delivery system, the MEPP program featured a dedicated team, including a program director, administrative assistant, instructional designer, editor/webmaster, counselor, and IT specialist. This team was able to draw on the resources of the College of Engineering and university for assistance with marketing, administrative, and technical support. All of the team members had other duties and responsibilities besides the MEPP program. But the fact that the program was their primary
was based on engagement theory (see http://home.sprynet.com/~gkearsley/ engage.htm). Engagement theory emphasizes that learning activities should be meaningful and authentic. So instead of typical abstract textbook problems for homework or quizzes/ exams to evaluate learning, students were asked to explain how what they were learning applied to their own work settings and to complete projects. In most cases, projects were done in teams, so collaborative/peer learning was involved. The net result of a powerful delivery system, strong program team, enthusiastic faculty, and engaging curriculum was a very successful program. Evaluation data collected from MEPP graduates indicates a very high level of satisfaction with the program in terms of improving their
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Whatʼs Right and Whatʼs Wrong with Online Learning
213
job skills and career opportunities
Faculty who are comfortable with the
as well as personal/professional
simple, structured environment of
development. Furthermore, the
the classroom will find the complex
level of completion was very high
and dynamic environment of online
(approximately 95 percent), which
learning unsettling and unpleasant.
is unusual for distance learning. It is
But even for the eager and willing,
typical for distance learning courses
learning how to teach online takes
to have a dropout rate of 25 to 40
considerable time to master. In 1997,
percent. In fact, the completion rate
I joined Walden Institute (a now
can be used as a measure of the
defunct element of Walden University)
effectiveness of a distance learning
to develop online teacher training
program, since the reasons why most
courses for higher education faculty
students drop out have to do with
and corporate trainers. These were
technology problems, insufficient
two- or three-month online courses
feedback on progress, or lack of
using Blackboard and/or the Centra
engagement in the courses.
web conferencing tool that attempted to prepare online teachers or learning
Learning to Teach Online
As I gained
facilitators. The facilitator course
more experience
addressed the following competencies:
with online
ÿ Understanding online tools and
teaching and developing online learning programs, I came to the conclusion that the Achilles’ heel of all distance education was teacher
their purposes, ÿ Strategies for effective online teaching, ÿ Evaluation and quality processes for online learning environments,
selection and training. First, I don’t
ÿ Administering and implementing
believe that all teachers (even those
online courses and programs, and
who are judged to be excellent in the classroom) have the aptitude to be
ÿ Ethical and legal issues associated with online courses.
good online teachers. Being a good online teacher requires a willingness
The primary course activity and
to interact extensively and intensively
outcome was the preparation of a
with students, the propensity to spend
proposal for an online course or other
a lot of time sitting at a computer,
online educational program relevant
and a lot of flexibility to deal with
to the facilitator’s own interests or
technology and logistical problems.
work setting. Each week’s activities
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
214
Michael Allenʼs 2008 e-Learning Annual
involved development of a portion of
listing all learning objects and peruse
this proposal, such as a description of
them in any order. This allowed the
the intended audience, instructional
course to be used as a structured
strategies, assessment methods,
learning experience or as a random-
ethical/legal considerations, and
access reference database.
implementation issues. Facilitators
The Walden online teaching
were required to read and critique
certification courses were taken
each other’s work as well as analyze
by instructors at many institutions
case studies.
and organizations who wanted to
An interesting aspect of the
ensure they would be effective online
Walden courses was that they were
teachers. We did follow-up surveys
intended to certify online instructors
with course graduates, and they
by establishing some well-defined
reported that the course had been
competencies required for online
valuable in preparing them to teach
teaching. Participants who completed
online.
the courses did indeed have the
Subsequent to the work at Walden
knowledge and skills needed to teach
Institute, I had the opportunity to
online; however, the course did
work with the North Central Regional
not provide much practice actually
Educational Lab (NCREL) and develop
teaching, so there was an inherent flaw
online teacher training programs for
in the certification process. What was
K-12 teachers. To prepare teachers
needed was some form of practicum
to teach online, we created a six-
in which participants taught an online
week course called Online Teacher
course, were coached during the
Facilitator Course (OTFC). (For more
process, and were then evaluated on
details about the OTFC course, see
their performance.
Kearsley & Blomeyer, 2004.) The
Another interesting aspect of
OTFC course used Blackboard for
the Walden courses was that later
asynchronous (discussion boards)
versions were designed as a collection
and synchronous (chats) interaction.
of learning objects. A learning object
OTFC addressed the following
was one or more web pages about
competencies based on the ITSE NETS
a specific topic that included self-
standards (see cnets.iste.org):
evaluation (quiz) items. Course
ÿ Explain and use the basic
units required students to examine a
elements of online courses: e-mail,
specific set of learning objects, but
threaded discussions, real-time
students could also go to a menu
conferencing;
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Whatʼs Right and Whatʼs Wrong with Online Learning
ÿ Facilitate asynchronous and synchronous online discussions; ÿ Create collaborative learning activities for students; ÿ Identify issues associated with the management of online classes;
215
ÿ Describe ethical and legal issues associated with online learning; ÿ Describe evaluation and assessment methods for online learning; and ÿ Prepare a plan for online learning.
ÿ Discuss the ethical and legal issues associated with online education; ÿ Explain the accessibility issues
A discussion of the initial results of the MOL course is provided at
associated with online education;
http://home.sprynet.com/~gkearsley/
and
MOLarticle_Oct04.htm.
ÿ Discuss strategies for integrating online and classroom instruction.
In both OFTC and MOL, we used the succession model pioneered many years earlier in the ETL program:
To prepare administrators to
graduates of the initial courses taught
manage online learning in their school
subsequent courses, becoming master
systems, we developed a six-week
online teachers in the process.
Management of Online Learning
There were many important
(MOL) class. MOL used Blackboard
outcomes from the OFTC and MOL
as well as the HorizonLive web
courses. First, since the course also
conferencing system for weekly live
provided first-hand experience with
sessions. The primary objectives of
online learning, completion of the
MOL, which were based on the ISTE
course ensured that participants
TSSA (NETS-A) standards, were:
possessed the necessary skills to teach
ÿ Explain the benefits and
or manage online courses. The course
limitations of online learning and
also provided first-hand experience
how they apply to their schools;
with online learning. Since the courses
ÿ Identify the technology and human
were taught by experienced online
resources required to implement
facilitators, the courses provided
online learning in a school;
good models of online learning.
ÿ Describe the cost, funding, and
Furthermore, the opportunity to
budget implications of online
share experience and knowledge with
learning;
other teachers or administrators was
ÿ Select and prepare teachers for online learning;
a valuable professional development activity. After completing these courses, teachers and administrators
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
216
Michael Allenʼs 2008 e-Learning Annual
had a realistic understanding of what
high school teachers based on the
online learning involves and the
STRP (Strategic Training for Reading
problems to be addressed.
Program) developed by NCREL. The
The OFTC and MOL courses were
online STRP course was a fourteen-
completed by hundreds of teachers and
week course built around the six
administrators in the Midwest area
reading strategies covered in the
served by NCREL and demonstrated
program. It involved the use of digital
that this was an effective approach
audio recording to allow teachers to
to preparing school staff for online
practice “read-aloud” strategies as
learning. However, the hundreds
well as discussion forums and chat
taught were just a tiny fraction of the
sessions to interact with each other.
staff in these school systems. These
Assignments in this course required
courses would have to be completed by
the teachers to try out reading
a relatively large percentage of faculty
interventions with their current
and administrators in order to have
students and share the results with
any significant impact. Furthermore,
the other teachers in the course,
the teachers and administrators
producing immediate benefits to
completing these courses did not
their teaching. One could envision a
come from the same school systems,
series of curriculum-specific online
so it was not a systematic approach to
training programs in areas such as
catalyzing online education.
science and math, foreign language or
Successful implementation
social sciences teaching. Perhaps such
of distance learning has to deal
programs will be provided by schools
with all the relevant components
of education (or schools systems
simultaneously. A school with
themselves) someday.
properly trained teachers but unprepared administrators, or viceversa, is unlikely to succeed with online learning. To date, I have not seen a school system or educational institution approach online learning using a systems approach. While the OTFC was a generic
From Courses to Learning Communities and Events
Up to this point, all the online learning discussed had been in
the context of structured courses. However, I don’t think this is a very
online teaching course, we also
good model for the future of online
explored the idea of developing
learning. Instead of being forced to
curriculum-specific training for
take a pre-defined sequence of lessons
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Whatʼs Right and Whatʼs Wrong with Online Learning
217
that stretch over a period of weeks
MedConnect (www.medconnect.
or months, most individuals prefer
com) is an online medical community
to study a single topic for a short
based around a set of e-journals. The
duration until they have learned as
Elearnspace website (www.elearnspace.
much as they desire to know for their
org) provides access to blogs about
immediate needs. In fact, the most
learning and technology. There are
common way that people (either kids
also commercial systems available
or adults) use the Internet to learn
that foster the creation of online
things informally is to perform a
communities, such as Knowledge
search and then examine as many of
Forum (www.knowledgeforum.com).
the resulting sources as they want.
Instead of developing and offering
The other usual mode of informal
courses, it makes more sense to
learning via the Internet is to seek
create an online community for a
out discussion groups (or blogs) on a
given subject domain that includes
topic of interest and read postings, as
discussion threads/blogs on major
well as interact with the participants
topics, along with regularly scheduled
of that group, until they have satisfied
real-time
their curiosity. And of course, people
“events” in the
It makes more sense to create
will e-mail friends and knowledgeable
form of chat or
an online community for a given
sources to tap into the experience and
web conference
subject domain that includes
ideas of others.
sessions on
discussion threads/blogs on
specific issues.
major topics.
A more enlightened model for online learning would build on these
E-mail can be
natural strategies. In fact, this model
used to distribute announcements
has been employed for some time in
about upcoming events and highlights
the form of online communities (e.g.,
of current discussions. Participants
Renninger, 2002; Rhinegold, 2000).
are free to choose the extent and
Online communities are basically
nature of their participation,
ongoing discussion groups formed
although in the context of formal
around a given interest area—they
educational programs, there may well
can be political, social, or scholarly
be some required participation and
in nature. For example, SRI hosts
contributions. Such a community
an online community of teachers
may offer book or product reviews,
called TappedIn (www.tappedin.org).
newsletters, reports on conferences
Seniornet (www.seniornet.org) is an
or meetings, consulting and tutoring
online community for older adults.
services (free or otherwise),
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
218
Michael Allenʼs 2008 e-Learning Annual
reporters and columnists, and
idea that developing online learning
games. Historically, access to online
environments will be a collaborative
communities has been free, but it’s
effort involving individuals from many
quite possible that some portions
different groups and organizations.
require paid membership (tuition or
Interestingly, professional/trade
registration fees).
associations are well-suited to the
Online communities create a much
creation of such learning communities,
more dynamic and “on-demand”
since they typically have some of
learning environment than traditional
the key ingredients needed, such
online courses. They are more
as newsletters, reference materials,
likely to satisfy a broader range of
meeting/conference events, and a
learning interests than a structured
membership. However, converting
series of lessons. Furthermore, the
to online delivery will require extra
information they contain is an open-
resources and specialized expertise,
ended database rather than the closed
which may be a problem.
nature of course curriculum, which is usually not available after a course is completed and can only be updated by
Organizational Factors
In reviewing the case histories
the course developers. The degree of
provided
participation can be determined by the
in Kearsley
individual based on intrinsic interest
(2005), as well
in a subject and external motivations
as Berge (2000) and Khan (2001),
(for example, credit or job training
it becomes clear that the biggest
requirements).
category of obstacles to the wider use
A practical problem with online
of online learning are organizational
communities, however, is that, unlike
factors. The tendency to maintain the
courses, they cannot be created
status quo (classroom instruction)
by a single person. It takes many
in education is very strong, and
individuals to create and maintain
establishing a new approach to
an online community. At the very
learning in any organization or
least, it takes a small team to
institution is always very difficult.
manage discussions and events and
Furthermore, many online learning
update information. So educational
projects and programs have been
institutions have to move beyond
implemented in organizations, but
the lone author/instructor model
have not been sustained over time.
for online courses and embrace the
What can be done to increase the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Whatʼs Right and Whatʼs Wrong with Online Learning
long-term success of online learning
219
The most effective
What can be done to increase
solution to these
the long-term success of online
extensively in the business world,
problems with
learning programs?
but has not become a mainstream
online learning
form of training in any organization.
in organizations has been to rely
The business case for online training
heavily on synchronous delivery in the
is easy to make in terms of reduced
form of live web conference sessions.
travel and on-demand availability of
Live sessions require participation at a
training. Technology is usually not an
given time (like conventional classes),
issue, since most organizations have
which can be scheduled as work
ample capabilities and almost all office
tasks. Furthermore, web conferences
workers have a networked computer
are similar in nature to meetings,
in front of them. Alas, problems arise
and instructors can rely on familiar
in terms of instructors understanding
presentation strategies (e.g., slide
how to teach online and develop
presentations and group discussions),
online materials. In organizations that
reducing the need for new teaching
have training departments (fewer and
skills or online content development.
fewer do), this issue can be addressed
While this form of online learning
through instructor training and course
is not particularly powerful, it
developers. A more subtle issue has
does overcome some of the major
to do with how to integrate online
organization obstacles present in the
training into the work environment.
business domain and is therefore likely
With conventional training, people
to be sustainable.
programs? Online learning has been employed
leave their workplace to attend
In the higher education realm,
workshops or classes. But online
online learning faces different
learning is done in the workplace,
obstacles. Most universities have
mixed in with usual work tasks.
adopted online learning to some
However, taking time to learn has a
extent, and it is seen as a competitive
lower priority than getting work done
advantage to increase enrollments
(especially in the eyes of your boss)
(hence revenue) by administrators.
and tends not to happen. For these
Furthermore, it is viewed positively
reasons, online learning programs
by many faculty and students in
that seem to offer much promise often
terms of making teaching or learning
yield poor results in terms of learning
more convenient. Universities also
outcomes or training impact.
have ample technology available and
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
220
Michael Allenʼs 2008 e-Learning Annual
good support structures—plus, most
and compete with existing institutions
students own laptops.
for students, they do not have to
Given this set of favorable
overcome the organizational obstacles
circumstances, one would expect
that limit the success of online
online learning to become the
learning elsewhere. Recognizing this
dominant form of instructional
fact, many traditional universities have
delivery in higher education. But it
spun off autonomous entities to run
hasn’t done so. Existing universities
their online programs, hoping they
have too much vested interest in
will succeed if they are independent of
their campuses—research labs,
the campus-based juggernaut.
residences, lecture halls, sports
In the K-12 domain, we see the
facilities, cafeterias, parking garages,
strongest organizational impediments
etc.—and the physical presence of
to the success of online learning.
students to adopt online learning on
School systems are locked into very
a large scale. Many of these facilities
rigid traditions and policies that are
generate revenue, and all have their
often dictated by state and federal
own staff and advocates. There is no
regulations. Like institutions of
way that a traditional university can
higher education, school systems
dispense with these facilities, which
have considerable vested interest in
are superfluous for online learning.
facilities and the physical presence
This is not an issue for new
of students. Funding protocols are
institutions of higher education that
based on student attendance, which
have been created around online
is difficult to translate into online
learning (Phoenix, Walden, WGU,
participation. Teaching duties are
etc.). They are not encumbered by the
based on classroom assignments
facilities of a traditional university
rather than learning accomplishments.
that have no significance for online
And technology infrastructure and
learners. Furthermore, they are free
support is often very weak.
to develop policies and procedures
Earlier in this article, emphasis was
for registration, grading, teaching,
placed on the training of teachers
and participation that are congruent
and administrators to enable online
with online learning but that would
learning in the K-12 setting. And while
conflict with those intended for
this is certainly an essential step, it
traditional campus-based programs.
ignores the changes needed in the rest
While these “virtual universities”
of the system to make online learning
struggle to establish their reputations
a viable approach. Parents and school
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Whatʼs Right and Whatʼs Wrong with Online Learning
board members must demand that
221
ÿ Course activities involving
school systems offer online learning.
meaningful and relevant tasks
At the state and district levels,
and emphasize peer learning
senior administrators must put in
through collaborative projects and
place policies and procedures that
discussions.
are conducive to online learning
ÿ Modern learning management
programs. Such advocacy assumes
systems and web conferencing
that people understand the nature
tools result in reliable and
and potential of online learning—
consistent online learning
something that is generally not the
programs.
case. So, for some time to come, organizational factors will mitigate
What’s Wrong
against the success of online learning
ÿ Requiring instructors to teach in
in the K-12 world.
an online program regardless of whether they want to or are suited
Summing Up
Throughout this rambling tour of
to do so. ÿ Trying to operate an online
past projects, I
learning program without adequate
have identified
technical and administrative
aspects of online
resources.
learning that I believe are right or wrong. Here is a summary.
ÿ Placing emphasis on creating online courses instead of learning communities and events.
What’s Right ÿ Involvement in online learning is based on a desire to make learning more convenient or effective
ÿ Expecting existing educational institutions to nurture online learning. ÿ Implementing online courses and
or because it fits the learning/
programs in an organization/
teaching style of the participant.
institution without a systems
ÿ The creation of a program in which graduates of an online
perspective on the changes involved.
course subsequently teach the course (succession model). ÿ Standards for evaluating the
Overall, online learning has evolved considerably over the past twenty-
effectiveness of online instructors
five years in terms of the technology
and courses.
available and its acceptance as a
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
222
Michael Allenʼs 2008 e-Learning Annual
legitimate form of learning. Certainly,
that we have not yet discovered
the Web has made online learning
appropriate strategies and methods
courses proliferate and easily available.
that will allow us to create highly
On the other hand, the instructional
effective online learning experiences.
quality of most online learning courses
I do think the day will come when
has not improved significantly over
most learning occurs online, but I’m
the years, even though we have much
not sure that it will be the dramatic
more powerful authoring tools and
improvement over classroom
many experienced designers. I suspect
instruction that everyone predicted.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Whatʼs Right and Whatʼs Wrong with Online Learning REFERENCES Berge, Z. (2000). Sustaining distance training: Integrating learning technologies into the fabric of the enterprise. San Francisco, CA: Jossey-Bass. Kearsley, G. (1985). Training for tomorrow: Distributed learning through computer and communications technology. Reading, MA: Addison-Wesley. Kearsley, G. (2000). Online education: Learning and teaching in cyberspace. Belmont, CA: Wadsworth. Kearsley, G. (2005). Online learning: Personal reflections on the transformation of education. Englewood Cliffs, NJ: Educational Technology Publications. Kearsley, G., & Blomeyer, R. (2004, January/February). Preparing K-12 teachers to teach online. Educational Technology, 44(1), 49-52. Khan, B. (2001). Web-based training. Englewood Cliffs, NJ: Educational Technology Publications. Moore, M., & Kearsley, G. (2005). Distance education: A systems perspective (2nd ed.). Belmont, CA: Wadsworth. Renninger, K. (2002). Building virtual communities. New York: Cambridge University Press. Rhinegold, H. (2000). The virtual community: Homesteading on the electronic frontier. Cambridge, MA: MIT Press.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
223
Alfred Bork Alfred Bork has worked in the field for forty-five years at such institutions as the Dublin Institute of Advanced Studies, the University of Alaska, at Reed College, and at Harvard University, and starting in 1968, at UCI. He was co-director and keynote speaker at the NATO Advanced Study Institutes on Computers in Science Education, at the Catholic University of Louvain in Belgium (1976), and at the San Miniato Conference Center (1985). He was the Millikan Award Lecturer for the American Association of Physics Teachers in summer 1978 and consultant to United Kingdom National Development Programme in Computer Aided Learning in 1975. He served four years as chair of the Special Interest Group on Computer Uses in Education of the Association for Computing Machinery. Dr. Bork is interested in effective use of highly interactive multimedia technology to make order of magnitude improvements in learning at all levels and has published hundreds of papers and books about these issues, including his most recent book, with Sigrun Gunnarsdottir, Tutorial Distance Learning. Alfred Bork is Professor Emeritus of Information and Computer Science and Physics, at the University of California, Irvine. He is founder and director of the Educational Technology Center, for research and development of technology-based learning material. He is also vice president of A Bork Endeavors.
Carole A. Bagley With more than thirty years in the field of instructional technology, Dr. Carole A. Bagley has made significant contributions to the field through her current and prior associations with the Minnesota Educational Computing Consortium (MECC), Control Data Corporation–Plato project, MN Department of Corrections Schools, Perfect Access Speer, Future Kids, ISPI, ASCD, the Digital Learning Forum (DLF), The Technology Group, U.S. Government. Schools, numerous public, private and charter schools, corporations and law firms. She has been recognized for her outstanding work by ISPI (International Society for Performance Improvement) and ASTD. Dr. Bagley has provided e-learning leadership, design expertise, technology integration expertise, strategic planning expertise, workshops, seminars, publications, and keynote presentations for numerous corporations and school districts as well as for professional organizations throughout Minnesota, Wisconsin, and in several other states and internationally in Australia, Tanzania, Scotland and Russia. Dr. Bagley is a Distinguished Professor for the University of St Thomas, St. Paul, Minnesota, and provides team leadership and consulting for the Technology Group, Inc.
Xiwen Zhang Xiwen Zhang is a faculty member in the Pfau Library and the US-China 1+2+1 dual degree coordinator at the California State University, San Bernardino. She has a BA in French literature from Beijing University, an MA in French literature from the University of California, Santa Barbara, and an MA in information studies from the University of California, Los Angeles.
Technology and Learning A Proposal for Adaptive Tutorial Learning
Alfred Bork, Carole A. Bagley, and Xiwen Zhang
In this article, the authors review some of their own experiences and frustrations with the early development of computer-aided instruction. Then they go on to discuss the lack of effectiveness of much online learning and to present one possible solution—an adaptive tutorial process. The latter part of the article is devoted to discussing the various elements and considerations that instructional designers should be aware of when creating adaptive tutorial learning.
The potential of technology, particularly the computer, to aid learning has been frequently touted by trainers, authors, and many others. So why has technology not played
A Major Curriculum Effort Before Computers
A major effort in curriculum development in the United States occurred after the
a major role in assisting learning?
USSR’s Sputnik was launched. The
Somehow the potential of technology
success of this Soviet mission before
to greatly improve learning has never
the United States had launched an
been realized. This article investigates
earth-orbiting satellite convinced
why this is so, going on to introduce
many people in the United States that
a form of computer-based learning
learning needed to be improved for the
technology—adaptive tutorial
United States to be competitive.
learning—that has the potential to lead
Sputnik was a turning point in the
to an order of magnitude improvement
world of science and education,
in learning globally. We begin with
spurring advances in curriculum, as
some history of learning development
well as sparking an era of research in
in the United States then discuss the
education (DeBoer, 1991). The funding
work of using computers in learning
for this post-Sputnik curriculum
environments at the University of
development was large, with many
California, Irvine.
courses receiving millions. Book-
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
228
Michael Allenʼs 2008 e-Learning Annual
based courses were developed at all
professional ideals and successful
levels, from elementary to university,
teaching practices.
with a concentration on science and
The post-Sputnik movement died
mathematics. All of this development,
quickly, due heavily to a particular
given its time, involved pre-computer
course that had major political
technology. Toward the end of this
problems. In an attempt to teach
effort, however, several unsuccessful
anthropology to ten-year-olds, Jerome
attempts were made to obtain funding
Bruner and his colleagues designed
for developing related computer-based
Man: A Course of Study (MACOS),
learning units.
an elementary school course that
The problem of the professional
combined pioneering fieldwork on
development efforts of the 1960s
the social behavior of baboons, a
and 1970s, then, was not based on
film-based ethnographic study of an
the number of teachers needing
Eskimo tribe, and novel “hands on”
professional development, but in
classroom materials. MACOS was
ignoring teachers’ views of effective
hailed as an original and exciting way
teaching and learning, in turn making
to promote science literacy and to
classroom implementation from
teach young people how to think like
these experiences unpredictable.
social scientists. Teachers and students
Many teachers participated in these
alike expressed enthusiasm for the
programs, and many returned to
dynamic nature of the course, and it
their classrooms more knowledgeable
achieved nationwide distribution and
teachers. The measure of success,
widespread recognition as one of the
however, was not what teachers
outstanding social science curriculum
knew and were able to apply to the
projects of the period. Yet by 1975,
classroom, but whether the teachers
MACOS had been driven out from the
could take the curriculum projects
schools, a casualty of a small but vocal
to which they were introduced and
group of conservatives critical of its
faithfully reproduce them in their
content and methodology.
classrooms (DeBoer, 1991). Using that
Other courses receiving federal
measure, the post-Sputnik reforms
funding that were developed at this
were a failure. Why? The developers
time included Science Curriculum
forgot that teachers were active, social,
Improvement Project (often called
creative, thinking human beings
SCIS) at the University of California,
who possessed previously formed
Berkeley, directed by Robert Karplus,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Technology and Learning
229
and the Elementary Science Study
since 1958. Since that time, many
at MIT. These courses were very
directions using computers have
different from existing practice at
been taken to enhance the learning
the elementary level, with much
experience.
more emphasis on the processes of
Major projects such as PLATO
science, and much less emphasis on
(Programmed Logic for Automated
memory. However, they demanded
Teaching Operations) and TICCIT
more individualized attention than
(Time-Shared, Interactive Computer-
was usually possible in the classroom
Controlled Information Television) in
environment.
the United States had large funding
Few of the courses from the post-
from the government and the National
Sputnik era survived, in spite of this
Science Foundation (NSF). In the early
enormously expensive development
1970s, PLATO was one of the first
effort. A major problem was that
computer-assisted instruction systems.
teachers were not prepared for new
Originally built by the University of
learning approaches, and the vast
Illinois and later taken over by Control
sums spent for training and retraining
Data Corporation, PLATO pioneered
teachers, which were actually far more
concepts such as online forums and
than what was spent on development,
message boards, learning management,
did not do the job. Training of
online testing, email, chat rooms,
teachers has frequently been the
instant messaging, remote screen
Achilles’ heel of improving learning.
sharing, and online games.
Only the biology courses, produced
In the mid 1970s, the MITRE
by the Biological Sciences Curriculum
Corporation developed and
Study group, are in common use today.
implemented the TICCIT system as a computer-assisted instruction
Learning and Technology
Technology,
(CAI) system for community colleges.
particularly
MITRE subcontracted with the
the computer,
CAI Laboratory at the University
provided new
of Texas at Austin and also with
possibilities
the Instructional Research and
for learning. The authors have been
Development Department of Brigham
personally involved with technology
Young University to refine the user
and learning for many years in
interface and create the massive
multiple states and internationally
amounts of courseware needed for
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
230
Michael Allenʼs 2008 e-Learning Annual
complete college-level English and
Work in Irvine
We review
algebra courses. The courses were self-
briefly in this
paced.
section work at
Also, in the 1970s, the Computer
the University
Curriculum Corporation developed
of California,
and sold computer-assisted courses
Irvine. With NSF support beginning
in mathematics, reading, language
in 1968, we developed some rewarding
skills, and other subjects. The courses
early projects.
provided pre-testing and post-testing, with an attempt to ensure mastery
Introductory University Physics
of content. These courses were used
Course In 1970, our first extended
in K-12 schools and correctional
project using computers to assist
institutions for remedial as well as
learning was one quarter of a
main instruction.
beginning calculus-based university
States and school districts also
physics course. This course was based
spent very large sums on acquiring
on the Keller Plan, the Personalized
computer technology, both hardware
System of Instruction, which required
and software, and on training teachers
students to fully master each point of
to use this technology. The Minnesota
the course before moving forward. The
Educational Computing Consortium,
course we developed was based on a
a state-funded company founded
set of online exams, with the questions
in 1973, was a national leader in
coming from problem generators.
providing educational computing
Each exam covered about one week’s
hardware, software, and courseware
work, but there was no fixed pace;
for schools and provided training for
students took an exam when they
teachers in how to use the technology.
wanted. Students sometimes needed
Courseware developed by MECC
to take an exam several times until
included Oregon Trail, Amazon Trail,
they completed it perfectly. We never
Lemonade Stand, Odell Down Under,
gave the same problem twice because
Number Munchers, and Dinopark
all problems came from the problem
Tycoon. The state of Minnesota
generators. No multiple choice was
spun off the company as a private
used.
corporation (The Learning Company) in the late 1980s.
Help was available to students in trouble in several ways. The most important help was available directly online as the student was taking the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Technology and Learning
231
exam. This assistance was based on
One important direction in the
incorrect student responses to the
Scientific Reasoning Series was to
problems, often revealing to students
allow students to discover their own
the nature of the problems they were
knowledge. In one of the ten modules,
having. In this way the computer
for example, students discover the
system moved smoothly, back and
laws of genetics. In another, students
forth, between exams and learning
discover the laws of simple electrical
assistance.
circuits. In a third program in the
This course ran on a time-sharing
series, students invent the concept
machine, a Sigma 7. Emphasis on
of heat. We followed this strategy of
mastery continued to guide our later
student discovery in later work. In
projects at Irvine, even though we did
spite of the success of these units,
not use the Keller Plan. It received
we could not find support to update
some interesting national publicity,
these modules and to develop further
but due to timesharing constraints at
modules in a similar direction.
the time, it could not be made more widely available.
Understanding Spoken Japanese A later project,
Scientific Reasoning
Understanding Spoken Japanese, was
Series With the coming of personal
developed with funding and support
computers, we moved in new
from Fujitsu and Nippon Television
directions. The Scientific Reasoning
Network. Each module was based
Series, about twenty hours of adaptive
on a video sequence, made in Japan,
learning units, was the first effort.
but for reasons that will be discussed
It was marketed by IBM through
later we only used a few seconds of
their K-12 division. Funding for
video at any one time. Many features
development came from NSF and the
of this project were similar to those in
Fund for the Improvement of Post-
our earlier work, including individual
Secondary Education (FIPSE). The
pacing for each student. We were at
aim of these units was to help young
work on the second set of ten modules
students to think like scientists. The
when troubles with the Japanese
typical audience was students of about
economic system ended the project.
eleven years old. The pace through
In a typical sequence, a bit of video
the material varied from student to
is played, and then the student is asked
student. The ten programs were highly
what the people in the video were
interactive and conversational.
discussing. If the student knows, we
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
232
Michael Allenʼs 2008 e-Learning Annual
proceed to another question. If not,
in learning environments, there is
we backtrack in student learning as
little improvement in learning, with
needed, even to the point of helping
or without technology. Although the
the student recognize individual
use of technology in learning shows
Japanese words. With this approach we
no significant difference, that is,
never “teach” something to someone
computer learning is no better than
who already knows it! This has been
traditional instruction (Clark, 1994),
important in our later work.
learners have been provided with the
Another strategy used was to allow
convenience of any time, any place
students access to a small piece of
learning. Hence, we have witnessed
video. Using controls similar to those
a dramatic increase in distance
on a video player, students could move
learning/e-learning courses and the
back and forward. Students were asked
rise of virtual universities and K-12
to find locations on a map using the
virtual schools. International Data
words from the people in the video.
Corporation (www.idc.com) and the
Again, help was given when necessary.
Gartner Group (www.gartner.com)
The tactics used in these programs
have predicted 5 to 10 percent growth
could be used in other language
of e-learning each year since 2001.
learning situations, such as in ESL
However, dropout rates have been
activities. For example, they would
high due to a lack of discipline and
be very useful in Japan now, since the
motivation and the increase of cheap-
Japanese system now requires that
to-produce courseware that lacks
all students learn English in primary
instructional motivation for many
school.
learners. Research shows that student
Technology So Far Has Not Helped Learning
The results
understanding and retention improves
of using
when students learn by experience.
technology,
Technologies such as collaboration,
particularly
interactivity, modeling, simulations,
computers,
virtual reality interfaces, and gaming
in learning have so far not been
will help students experience the
impressive. A variety of studies and
skill being taught. For example,
opinions have questioned the use
collaboration technologies are already
of technology to improve learning.
available that can simulate real-time
Although it has been many years since
conversations with techniques such as
computers have begun to be used
instant messaging, chat rooms, shared
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Technology and Learning
233
geographical space, and community
school administrations and governing
software (Bagley & Chou, 2007; Clark,
boards doubt that these programs
2003; O’Neill, 2003). Much more
are worth the money they cost. Is it
research is necessary to conclude that
because these learning programs are
certain strategies do indeed increase
increasingly available through the
learning in all subject areas.
Internet and/or because the audience
A number of recent articles have commented that science education
itself has moved to e-learning? The recent
is no better today than it was fifty
domination of the
Science education is no
years ago. The National Assessment
No Child Left Behind
better today than it was
of Educational Progress (NAEP)
(NCLB) program in
fifty years ago.
shows that in most areas today’s
the United States may
students are achieving at about the
also be a factor. Has the importance
same levels as students tested in 1971
of NCLB and the testing regime
(Robinson, 1997). One could argue
decreased the use of computer-based
that learning in all subjects is inferior
learning? Is this because it is viewed
now compared with fifty years ago,
as requiring “EXTRA” time that now
demonstrated by so much of the high-
must be spent on remedial, multiple-
stakes testing of students now—and
choice, low-level learning so students
with poorer methods such as multiple
can pass the NCLB tests? Since the
choice used in most federal No
implementation of NCLB, reading
Child Left Behind testing (Campbell,
scores among fourth- and eighth-
Hombo, & Mazzeo, 2000; Clarke, 2003;
graders showed little improvement
Darling-Hammond, 2003; Linn, 2003).
over the past two years, and math
This is in spite of the hundreds of
gains were slower than in previous
billions of dollars that have been spent
years. Most troubling for educators
“improving” learning, including vast
are the sluggish reading skills among
sums spent on computers.
middle-school students, which have
The market for computer-based
remained virtually unchanged for
software in learning has decreased
fifteen years, according to the National
continuously beginning in 1998
Assessment of Educational Progress,
(Ewalt, 2001; IDC Learning Systems,
which administers the federal test and
1998; Minneapolis/St. Paul Business
bills itself as the “nation’s report card”
Journal, 2001; Quality Education
(National Assessment of Educational
Data, 2001). This may indicate that
Progress, 2007).
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
234
Michael Allenʼs 2008 e-Learning Annual
We now consider some of the
the key to improving learning. Among
reasons why technology has not led to
the many examples of this are Basic (a
improvements in learning, globally.
programming language), time sharing, the personal computer, the mouse,
Technology and Improving
computer games, email, artificial
Learning Emerging technologies have
intelligence, the Internet, higher
the potential to support and motivate
speed Internet connections, reusable
learning, creativity, and problem
learning objects, open source software,
solving. Inventively infused into
Linux, blogs, faster processors,
active learning, they can open up the
learning management systems, games,
world for learners of all ages, in every
Wikis, instant messaging, the semantic
setting; but when new technologies
web, and others.
are bolted onto uninspiring curricula
As one example of the ways new
or mind-numbing drill, they can
technologies grab attention but fail to
deaden educational experience. In
deliver improvements, consider faster
addition, we need research that points
Internet access. Many people state
toward ways to increase learners’
that broadband access for students
access to educational technologies
and teachers will lead to better
and to narrow the divide between
learning. But no convincing argument
technology “haves” and “have-nots.”
for this additional cost is offered.
We also need to reconsider many
Another example is reusable learning
facets of educational practice across
objects, used in ways that have little
all of the priorities, with a view toward
connection with the increase in
enhancing the learning potential and
learning, but instead with the lowering
minimizing the risks of educational
of production costs. Although learning
technologies (Robinson, 1997).
objects have potential, the important question is whether they will play a
Grabbing Onto Each New
genuine role in increasing learning.
Technology Technology enhancements have brought a constant
Failure to Continue Successful
stream of new approaches and
Developments Funders often prefer
products, both hardware and software.
to look for something new rather than
These products receive considerable
follow up on successful approaches
marketing publicity. More often than
because they want to make a mark by
not, teachers and developers appear
being in the forefront. Funders want to
to believe that each new technology is
make a statement, and following up on
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Technology and Learning
235
someone else’s work does not provide
should be attaining with computers.
them with the credit or “name” they
Further, this type of evaluation often
desire. We have had that problem
shows only the Hawthorne effect of
with follow-ups to the Scientific
something different, that is, learning
Reasoning Series, for example. In some
seems to increase, however, it is only
cases, such as with PLATO or MECC
temporary due to an excitement
courseware, the strategies are picked
about something new. Once the
up by commercial sources, but this is
“newness” disappears, the learning
still rare.
decreases. The number of learners in good learning evaluations should be
Poor Evaluations Many education
in the thousands, in many different
innovations have inadequate
environments; an example of an
evaluations or none at all. Sometimes
evaluation with adequate numbers
only student opinion polls are
was that of the Writing to Read
presented as evidence, a useless
program, conducted by Educational
approach, as it tells us little about the
Testing Service in 1984. Ten thousand
learning effects of the innovations.
kindergarten and first-grade students
Even typical university-based
were tested in twenty-one sites, and
evaluations by faculty in education
in the second year concentrated on a
and related areas are, for the most
“core sample” of 3,210 students using
part, useless. The usual education
Writing to Read and 2,379 comparison
experiment involves approximately
students in classes not using Writing
one hundred students with a single
to Read. The evaluation study
treatment, and a similar number with
used rigorous techniques of group
another, usually in only one or two
comparison, “before-and-after” testing,
locations. The model is a drug-testing
and statistical analysis to sort out the
model, but with far fewer people.
effects of the program as compared
Given the wide variety of students,
to pre-existing differences in ability
with different learning approaches
level. The students were able to use
and capabilities, in a country or in the
the technology, and on standardized
world, these numbers (hundreds of
reading tests, kindergarten and first-
students) are entirely inadequate. At
grade Writing to Read students, on
best these evaluations purport to show
the average, progressed faster than the
very small improvements in learning,
national norm samples (Educational
not the major improvements we
Testing Service Princeton, 1984).
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
236
Michael Allenʼs 2008 e-Learning Annual
Failure to Relate Technology to
many students in a given classroom. In
Learning One failure of many “new”
corporations, the costs associated with
approaches is that they are driven by
delivering different instruction for
the technology rather than by learning.
varied learning styles is prohibitive. In
They do not start with the problems
the United States and elsewhere, this
of learning, but only with recent
problem is further complicated by the
developments in technology. That is
emphasis on multiple-choice tests,
a backwards approach. If we are to
which fail to assess learner capabilities
see major improvements in learning,
and needs in an accurate, meaningful
we need to begin with the learning
way. This suggests the need for both
problems that we face, not with new
better learning methods and better
developments in technology.
learning environments. A related serious problem raised in
What Is Missing Today in Learning?
In workshops
these discussions is that many students
conducted
do not learn, or learn only partially,
worldwide, we
in current learning situations. Our
often begin
grading systems reflect this, as all
the session by
students do not receive excellent
asking the participants about what is
marks. Oddly, many in universities
wrong with learning today. In these
see the giving of many low grades
brainstorming discussions computers
as showing that the learning system
are not mentioned, and the results
is of high quality. This is because
everywhere are surprisingly similar.
of the belief that only the most
One factor that always comes up
intelligent, hard-working students
is that we do not adapt the learning
would be able to learn everything in
process to the wide variety of students,
the course and exhibit that learning
with many different backgrounds,
in an exam. This is a peculiar way
learning styles, and interests.
to measure the quality of learning.
Educational psychologists by and large
Indeed, if the course could be altered
agree that students differ greatly in the
for each individual’s learning styles,
ways they learn. Very few teachers or
it is highly likely that more students
professors can adapt learning to each
would receive As. A related problem is
student in the typical large classes
that this emphasis on grades leads to
we have today in either schools or
widespread cheating.
universities, because there are so
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Technology and Learning
237
Benjamin Bloom (1984) showed
come to learning with different
twenty-five years ago, as reported
experiences and therefore leave the
in his 2 sigma paper, that almost all
learning with difference knowledge,
students can learn to the mastery
and Constructivism preaches active
level, given the right learning
learning wherein the learners
environment. But this important work
construct their own learning given the
is unfortunately unknown to most
tools and information to manipulate.
teachers and university professors. In
Constructivism works well when the
his experiments the most successful
fundamentals are in place.
learning strategy was tutoring (to be discussed further later in this article). Another factor is that many students do not enjoy learning. Students often say that they hate schools, thus lacking motivation to
How Can We Overcome These Learning Problems with Technology?
spend the time to learn. This failure to
The problem in most attempts at improving learning is not with the technology, but
enjoy learning has devastating effects
with how it has often been used. The
in the future on lifelong learning.
question is: How can the reasonable
Finally, we mention that we do not
use of computers in learning lead to
have an adequate theory of learning
an order of magnitude improvement?
today, in the sense of the powerful
In our opinion, this is not only
predictive theories we have in some
a possibility, but one with good
areas such as physics. Indeed, we do
prospects. First we consider adaptive
not even seem to be close to such a
learning and then tutorial learning as a
theory. Education schools discuss
way to achieve adaptive learning.
many theories, but none of them are proving adequate. Behaviorist, Cognitive, and Constructivist theories
Adaptive Learning
As mentioned, a critical factor
of learning have their place; however,
missing in most
none of them is a panacea. Behaviorist
learning today is
learning tends to be useful at the
the capability to
lower levels of knowledge in that the
adapt very frequently to each student.
expectation is that every learner will
At any point in learning, each student
achieve the same outcome. Cognitive
is a unique individual in many ways.
approaches realize that learners
Several factors are important for adapting to student needs and desires.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
238
Michael Allenʼs 2008 e-Learning Annual
Each Student Should Move at a
Each Student Should Be
Unique Pace Given all the variations
Successful in Learning A major
between student backgrounds,
advantage of adaptive variable pacing
interests, and abilities, it is highly
is that the students can continue
desirable to allow each student move
to learn in a given area until they
at a unique pace in the learning units.
have learned the material. We know
Although there are some K-12 schools
from Bloom’s research that almost
that engage in this practice whereby
all learners can succeed and achieve
students move along at their own pace,
mastery, but some learners need more
it is nearly impossible in traditional,
time and more practice than others.
teacher-led environments. However, we can allow this in the computer
When Something Is Successfully
learning environment because
Learned, the Learner Should
computers have branching capabilities.
Move On Often in classroom learning,
For reasons that we will soon
after a student has learned something,
mention, we expect many students
the class continues working on the
in an adaptive learning environment
topic, boring the student. This will
to learn faster than at present, for
not occur in a fully adaptive learning
reasons that we will discuss later. This
environment.
could be a major change in learning for many students. It will also lead
No One Should Be Taught
to more economical learning, and it
Something He/She
will allow individuals to become more
Already Knows Similarly, in
productive members of society at an
conventional learning systems today,
earlier age.
we teach things that are already known to the students. As an example,
Adaptation Should Be Very
consider students at an early stage
Frequent Changes based on
in mathematics. Such students may
occasional exams are inadequate.
or may not know how to count
Learning activities should adapt to
objects. If they cannot, they need
each student on a moment-by-moment
to learn this. If they can, learning
basis. Students should feel that the
can proceed to another topic. But
adaptive program is responding to
this is very difficult to achieve in the
them as individuals.
learning environments now available. By assuring learner competencies, avoiding unneeded instruction, and
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Technology and Learning
239
moving each student forward when
students, as described by Plato, but
ready, we expect to achieve a major
only asked questions. Student answers
reduction of learning time, but this
prompted new questions that led
cannot be verified empirically until we
students to reason out answers to
have a full range of computer-based
the original questions. Very difficult
adaptive learning units.
subjects could be learned with a skilled tutor like Socrates.
Adaptive Tutorial Learning for Tomorrow
How can we attain adaptive
Very Frequent Questions The
learning for
questioning in the tutorial learning
everyone in the
environment is frequent. In
world? The key
experiments done many years ago at
factor is a learning method over 2,500
the University of California, Irvine, we
years old—tutoring–the approach
found that to hold student attention
used in the experiments of Bloom,
and to react frequently to student
mentioned earlier. We cannot afford
needs, the time between questions
an excellent human tutor for every
typically should not exceed twenty
student, but the computer can now
seconds. This means that students
be a tutor, making tutorial learning
are always active, interacting with
possible for everyone. No tactics from
the computer. The situation is like a
artificial intelligence are required for
conversation or a dialogue. Students
the system proposed here. But we
are not usually given long passages
need learning modules that follow
to read. And they do not watch large
an adaptive, tutorial approach in all
amounts of video, as noted in our
levels of learning. Together with some
discussion of the Understanding
collaboration and Socratic questioning
Spoken Japanese work. Such activities
with the teacher and other students
would violate the twenty-second rule.
to clarify and provide another view of the learning, adaptive tutorials would
Storing and Using Information
provide the majority of the teaching.
About the Student Designers may decide at various points in the
Socratic Tutoring The model
program to save information about
for this proposed “new” approach
student performance. This information
to learning is the tutorial process
can be used later in the program, as
used by Socrates. He did not supply
decided by the designers. No attempt
information to his small group of
is made to build a model for each
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
240
Michael Allenʼs 2008 e-Learning Annual
learner; we do not now know enough
working in pairs and with mentors
about learning to create such a model.
received the highest course grades,
Such stored information is also useful
however, those in groups showed
in evaluation and in research on better
higher cognitive and metacognitive
understanding the learning process.
learning. Within a group, students may
The Next Question A critical role of
progress at different rates, so
instructional designers is to analyze
occasionally the computer will ask
student responses, looking for various
students to work alone in order to
likely answers for their intended
monitor their individual progress.
student population and devising
Based on this information and on
answers that can immediately give
stored information about students, the
assistance to the student. Student
computer will occasionally rearrange
messages should be friendly help, not
the groups, so that each student will
viewed as critical by the students.
be in the best learning environment.
Based on this analysis and on previous
Working in groups of this size has
student input, the designers decide
important social advantages. There
what question to ask next. No use of
are very frequent student interactions
artificial intelligence is needed in this
within the group, focused on the
analysis.
learning activity. As groups change, students will work with many different
Peer Learning Our studies at the
students. Social interaction is superior
University of California, Irvine—based
to that typically found in classes.
on research in which we videotaped groups of various sizes—have shown
Invisible Tests Neither students nor
that it is best to have two to four
teachers like tests. In the tutorial
people working together in adaptive
environment, learning and testing are
tutorial learning, to gain the important
parts of the same process, so students
advantages of peer learning and social
are not conscious of taking tests. The
interaction. Current research by one of
questions from the computer generate
the authors confirms the importance
both learning and testing. This means
of collaboration in learning (Bagley
also that cheating in the usual ways
& Chou, 2007) and the importance
is impossible, eliminating another
of pairs, mentors and small groups.
common problem in learning today.
Findings indicated those students
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Technology and Learning
241
Enjoying Learning Lack of testing
Creativity Discovery of knowledge
is one factor in enjoying learning, as
is one factor leading to more creative
students do not like tests. Success in
individuals. Instructional designers
learning is another factor; students
should also seek ways to promote
enjoy success. Friendly language
creativity, important for the future
also helps and is a requirement of
of our global society. Creativity is
the design process. Being sure that
increasingly disappearing in learning
students like to learn is another task
today as teachers fine-tune the lessons
of the designers. Although it is not
so that all students will be able to
possible to ensure that each student
pass the NCLB multiple-choice tests.
enjoys the learning, it is possible to
This eliminates student choices and
provide learning experiences that are
lowers motivation and interest for
aligned to a learner’s preferences.
the student. Through adaptation in
Some students will be more interested
learning, choices will be available and
if a game format is built into the
creativity will increase.
tutorial is used, others want to solve problems or case studies, and others are fine with the tutorial and collaborative engagement. Adaptation for each learner is the critical element.
Production of Adaptive Tutorial Learning Units
A complete system is available for developing adaptive tutorial
Discovery Learning One great
modules (see Bork & Gunnarsdottir,
advantage of adaptive tutorial learning
2004; Bork, Ibrahim, Milne, & Yoshii,
is that students can discover much
1992).
of their own knowledge. This leads to better understanding of what is
Design The most important stage is
learned and better retention. In this
design, done by groups of excellent
discovery situation, students behave
teachers and researchers in the area
like scientists. There are several
under development. In this stage,
examples of such discovery in the
the overall form of the material is
Scientific Reasoning Series developed
specified in outline form, with a brief
about twenty years ago and marketed
description of each module to be
by IBM. Virtual worlds with games are
developed. These descriptions are the
beginning to be developed and should
basis of the next step, detailed design.
be included in the adaptations for some learners. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
242
Michael Allenʼs 2008 e-Learning Annual
The role of designers in this stage,
Implementation In this stage we
briefly:
move from the script to a running
ÿ Formulation of messages and
program ready for evaluation.
questions asked by the computer, ÿ Specification of the analysis of student responses to questions, ÿ Decision of what message or question is to come next, ÿ Decision of what student
ÿ Some human programming will generally be needed. ÿ Media of all types is prepared by media specialists. ÿ Beta testing finds errors in the program and these are corrected.
information is to be stored and how this information is to be used
Full Evaluation and Improvement
later in the program,
Before Use The final stage in
ÿ Criteria for determining when a
producing adaptive tutorial learning
student has achieved mastery in a
units is evaluation. As designers are
given area,
people, they may not take all factors
ÿ Keeping the student happy,
into account. So a critical stage in
ÿ If teachers or parents are available,
development is evaluation of the units.
planning for information to be sent
This is followed by improvements
to them, and
based on the information gained from
ÿ Designing the strategy for the formation of learner groups to
evaluation. Evaluation involves thousands
keep peer learning groups at the
of students. As they use the units,
same level.
information is recorded by the program. This extensive information
It should be noted again that no use
may show factors overlooked by the
of artificial intelligence occurs. All
designers. For example:
decisions are made by the designers.
ÿ There may be reasonable answers
Designs are recorded in a form called
to questions not thought of by the
a script, stored in the computer.
designers.
Alternate scripts are created that will be used in the adaptation.
ÿ There may be unanticipated responses that should lead to immediate assistance.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Technology and Learning
243
ÿ There may be sections in which the
(reusable learning objects) that will
time between questions is too long.
be created for different adaptations.
ÿ There may be sections in which
Experience with adaptive tutorial units
students do not reach mastery in a
is needed to determine all the relevant
reasonable time.
factors. The new system proposed
ÿ We need information about how long it takes students to learn in
will cost less when large numbers of students are involved.
this environment, as compared to students in conventional classroom environments. ÿ Students may not enjoy learning.
Adaptive Tutorial Units and Global Learning
The new learning strategy proposed here has great promise for
These and other data may suggest improvements. We recommend two
lifelong global learning in the future.
cycles of evaluation and improvement,
Two steps are proposed to realize this
with the second focusing primarily on
promise.
the effectiveness of the learning units.
1. An initial extensive experiment is needed to demonstrate the
Financial Considerations For such
effectiveness of adaptive tutorial
a new learning system to be widely
learning units. Several areas should
adaptable, the learning cost for a
be explored in this experiment.
student hour of learning must be
We favor an experiment with very
reasonable. In looking at costs, it is
young children, at the beginning
important to consider all factors; many
of “school” learning, including
considerations of cost miss some.
reading and writing, mathematics,
Instructional design and development
and science.
ranges in terms of cost from low-
2. Another possibility, perhaps a
cost performance support to high-
simultaneous experiment, would
cost simulations. The cost of design,
be an experiment covering several
programming, graphic design, and
courses at the introductory college
management are typically built in.
level. As this is an experiment, and
Additional costs that will be necessary
so might fail to meet expectations,
for this system will include the
we would not proceed further
adaptive technology software itself,
unless the experiment were
as well as multiple course segments
successful.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
244
Michael Allenʼs 2008 e-Learning Annual
ÿ If the experiment suggested further
ÿ A new distribution system must
activity, we would need a full plan
be developed to reach everyone on
to proceed.
earth. Satellites seem to be a good
ÿ Further units would need to be developed to cover the range of learning activities from birth to old age. ÿ Units would need to be translated to many languages and localized
possibility. ÿ Financing should be sought for these developments, and new learning organizations for managing the new system will be needed for the future.
for cultural differences if this system were to support global
Conclusion
So far technology
learning. ÿ The learning units would need to
has not led
work in schools and in areas with
to the major
no schools or where very poor
improvements
schools exist.
in learning that were expected. But
ÿ New inexpensive computers would
the use of adaptive tutorial learning
be needed, particularly for the very
with computers may lead globally to
poor parts of the world. We would
orders of magnitude improvements in
need to consider areas where no
learning at all levels.
electricity is available.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Technology and Learning
245
REFERENCES Bagley, C., & Chou, C. (2007). Collaboration as a pedagogy and its impact on cognitive and affective strategies and problem solving by novices learning java computer programming. Dundee, Scotland: ITiCSE (Innovation and Technology in Computer Science Education). Bloom, B.S. (1984, July). The 2 sigma problem: The search for methods as effective as one-to-one tutoring. Educational Research. Bork, A. (1999, July/August). Interview. EDUCOM Review. Bork, A. (2000). Four fictional views of the future of learning. The Internet and Higher Education, 3. Bork, A. (2001, March). Tutorial learning for the new century. Journal of Science Education and Technology, 10(1). Bork, A. (2001). What is needed for effective learning on the internet. Educational Technology and Society, 4(3). Bork, A. (2005). Distance learning, today and tomorrow. In G. Kearsley (Ed.), Online learning. Englewood, NJ: Educational Technology Publications. Bork, A. (2006). Adaptive lifelong global learning: Vision and plan (unpublished manuscript). Bork, A., & Gunnarsdottir, S. (2004). Tutorial distance learning: Rebuilding our educational system. New York: Kluwer Academic Publishers. Bork, A., Ibrahim, B., Milne, A., & Yoshii, R. (1992). The Irvine-Geneva course development system. In R. Aiken (Ed.), Education and society, Information processing 2. Holland: Elsevier, Holland. Campbell, J.R., Hombo, C.M. & Mazzeo, J. (2000). NAEP 1999 trends in academic progress. Washington, DC: National Center for Education Statistics. Clark, R. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445-459. Clark, R. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21-29. Clark, R. (2003). What works in distance learning: Instructional strategies. In H.F. O-Neil (Ed.), What works in distance learning. Los Angeles: Harold F., University of Southern California/CRESST, Center for the Study of Evaluation National Center for Research on Evaluation, Standards, and Student Testing, UCLA.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
246
Michael Allenʼs 2008 e-Learning Annual
U.S. Department of Education (1996). National Center for Education Statistics, Report in brief: NAEP 1994 trends in academic progress. Washington, DC: U.S. Government Printing Office, 4-5. Clarke, M. et al. (2003, March 13). Perceived effects of state-mandated testing programs on teaching and learning: Findings from interviews with educators in low-, medium-, and high-stakes states. National Board on Educational Testing and Public Policy. www.bc.edu/research/nbetpp/statements/nbr1. pdf> Darling-Hammond, L. (2003, March 13). Standards and assessment: Where we are and what we need. Teachers College Record. www.tcrecord.org/Content. asp?ContentID=11109>. DeBoer, G.E. (1991). A history of ideas in science education: Implications for practice. New York: Teachers College Press. Educational Testing Service Princeton. (1984). ETS evaluation of writing to read. Princeton, NJ: Educational Testing Service. Ewalt, D.M. (2001). PC industry’s first decline predicted. Information Week. www. informationweek.com/story/IWK20010608S0003. Accessed June 14, 2001. IDC Learning Systems. (1998). http://sec.edgar-online.com/1998/11/12/13/0000 950133-98-003778/Section8.asp Linn, R. (2003, March 23). CRESST policy brief on standards-based accountability: Ten suggestions, 1999. National Center for Research on Evaluation, Standards, and Student Testing. www.cresst.org/products/ newsletters/policypaper.pdf. O’Neill, H. (2003). What works in distance learning. Washington, DC: CRESST. Office of Naval Research Award Number N00014-02-1-0179. Minneapolis/St. Paul Business Journal. (2005). Plato expects Q3 loss, as sales fall. http://www.bizjournals.com/twincities/stories/2005/08/01/daily57.html. Quality Education Data. (2001). QED’s technology purchasing forecast, 20002001 (6th ed.). Denver, CO: Quality Education Data. Robinson, S.P. (1997). Building knowledge for a nation of learners: A framework for education research. www.ed.gov/offices/OERI/RschPriority/plan/index. html National Assessment of Educational Progress, 2007. http://nces.ed.gov/ pubsearch/getpubcats.asp?sid=031#016
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Sivasailam Thiagarajan Thiagi (Dr. Sivasailam Thiagarajan) is currently the Resident Mad Scientist at the Thiagi Group, Inc. For the past twenty years, he has been making a fairly decent living by playing games and helping others to play. Since March 21, 1999, he has been conducting a longitudinal research study that involves his designing a new interactive training activity every day (including weekends and holidays). As a result of this neurotic behavior, he has published more than two hundred games and fifty books related to interactive experiential strategies. A four-time president of the North American Simulation and Gaming Association (NASAGA), and two-time president of the International Society for Performance Improvement (ISPI), Thiagi has lived in three different countries and has worked and played in twenty-five other countries.
Cheap But Not Tacky Lessons Learned During e-Learning Adventures
Sivasailam “Thiagi” Thiagarajan
In this article, game-guru Thiagi opens by discussing his very early forays into e-learning and instructional design, providing a set of lessons learned for each. He then goes on to explain several e-mail games and his fourdoor approach for online learning, using specific examples so the reader can appreciate the effectiveness and flexibility of this model. He closes by presenting the philosophy and principles that continue to guide his work. Mine is the story of an ordinary person
Library about Omar Khayyam Moore
being transformed by an extraordinary
and his incredible talking typewriter,
technology. The story begins in an
I decided to replicate the study. My
underdeveloped country during an
target population was five- and six-
interesting time. Throughout the
year-old children with little or no
adventure, I learned, rediscovered, and
reading ability in Tamil. My subjects
maintained some important core values
sat in front of a teaching device
that have stood me in good stead in my
that was the size of a refrigerator (a
interactions with e-learning, training,
reconstructed label, since none of us
performance consulting, and life. I will
had at that time seen a refrigerator).
let you decide whether the story is an
About a third of the front part of the
engaging one and whether its lessons
device was occupied by a membrane
are generalizable.
keyboard that contained thirteen vowels, eighteen consonants, and 234
Tamil Talking Typewriter
My first foray
consonant-vowel combinations—a
into e-learning
total of 265 letters of the entire Tamil
was in 1966, long
alphabet. My learning objective was to
before the label
teach children to read common words
was coined. I was
by sounding them out. My task was
a teacher of physics in a high school in
relatively simple, since Tamil (unlike
Chennai, India, and my wife taught in
other less sophisticated languages) was
an elementary school. Inspired by brief
absolutely phonetic. Each letter had a
reports that I read in three different
single sound (which was also the name
sources at the local U.S. Information
of the letter).
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
250
Michael Allenʼs 2008 e-Learning Annual
This is how each of my subjects
translucency of the muslin, Kumar
interacted with the device. The child
could see which letter was touched
touched any of the letters on the
by the learner. Immediately, he
keyboard. Immediately the letter was
pronounced the letter in a standard
pronounced in a slightly shrill voice.
monotone. Later, we trained (and
After playing around with random
programmed) Kumar to sound out the
letters during the first encounter, the
constructed word whenever the child
child was given a list of short words
touched the space bar.
(the Tamil equivalent of See Spot run).
Three children learned how to read
Now, the device sounded out each
before our device became unglued.
letter touched by the child. At the end
And then a funny thing happened.
of the word, when the child touched a
Kumar ex machina did not faze the
larger key (the space bar), the device
subjects. Even though the black box
sounded out the entire word. By
was no longer a black box, children
working through a controlled set of
were engaged in playing the game. And
words, the child learned to read (as
they succeeded in learning to read.
demonstrated by being able to sound
My associates and I learned
out common words from a first-grade
several powerful principles from this
reader).
adventure:
Now for the disillusioning details. The teaching device was constructed by my high school students out of pieces of cardboard (and decorated with small circles of tin foil from
ÿ Reliable training devices need not be expensive. ÿ Real intelligence is cheaper than artificial intelligence. ÿ It is fun to use human beings to
cigarette packets). The keyboard was
simulate computers. It is more
a translucent piece of muslin with the
fun than doing the simulation the
letters of the alphabet written in India
other way around.
ink (how appropriate!). This keyboard
ÿ The key element in e-learning is
stretched over an appropriate hole cut
not the hardware but a consistent
in the front of the device.
and reliable program.
The CPU was Kumar, an older (and precocious) child squatting inside
ÿ Interactive instruction is a powerful tool.
the box (and breathing through air holes on the side). Kumar was trained
I was fascinated by the fact that
(and programmed) to do the same
programming human tutorial behavior
thing repeatedly. Because of the
results in replicable learning. I
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Cheap But Not Tacky
251
continued to fool around with the
I decided to become a rigid eclectic.
concept, without the expense of
Learning was learning, and I did not
recycled pieces of cardboard.
particularly want to specialize in any
Fast forward some six months: I
one medium, approach, technology, or
met an American professor who was
philosophy. However, I did waste some
visiting Chennai to make presentations
time in trying to differentiate among
about programmed instruction.
types of technology-based training:
Dr. Douglas Ellson’s specialty was
CAI, CMI, CBT, and CIA.
programmed tutoring for teaching
I created a computer program to
reading—which bore an interesting
teach people (and to drill them) to
resemblance to my sloppy studies. To
rapidly square two-digit numbers. I
abbreviate the story, in a couple of
used FORTRAN, punched cards, and
weeks, I turned up in Bloomington,
a mainframe computer. Later, I ported
Indiana, as a research assistant in the
this program to BASIC using paper
psychology department at Indiana
tape and stolen time on a time-share
University. Good-bye, physics.
system. Here’s the lesson that I learned
Welcome psychology and instructional
at this time:
design.
ÿ If I didn’t watch out, I might become addicted and create
It’s Not the Technology; It’s the Learning
In Bloomington,
computer-based training to teach
I had the
all types of useless topics.
privilege of working on Ellson’s project
A Perfect Package
In the mid1970s, I worked
while pursuing a Ph.D. A common
as the assistant
theme in all my work at this time
director
was my passion for bringing about
for Indiana
learning as measured by new and
University’s Center for Innovation in
improved human performance and
Teaching the Handicapped (CITH),
accomplishment. During my early
focusing on teacher training. I
days in the United States, I hung out
designed games, simulations, and
with a lot of Skinnerian behaviorists
role plays (without bothering with
and became an active member of the
computers). I also worked peripherally
National Society for Programmed
on the Computer-Assisted Teacher
Instruction (NSPI). However, from the
Training Systems (CATTS) project
early days in the learning business,
directed by Dr. Melvyn Semmel.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
252
Michael Allenʼs 2008 e-Learning Annual
ÿ The power of the computer should be exploited for performing unique tasks, such as rapidly analyzing dynamic data and feeding the results back to the teacher in a graphic form.
Figure 1. PDP-11 Computer
Here’s the way CATTS worked: A
Playing the Shell Game
Fascinated by the power and potential of
teacher-trainee selected a category
programming,
from one of several behavior
I bought a
observation systems. For example, she
TRS-80 Model 1 computer with the
might have decided to increase the
intent of teaching my twelve-year-
frequency of open-ended questions in
old son, Raja, how to program in
her lessons. While she taught actual
BASIC. Within a week, my son had
students or role-playing cohorts, a
outstripped my programming skills
trained observer sat at the back of the
and started teaching me. A week
classroom with a special keypad and
later, he gave up on trying to teach
coded the teaching behavior, once
me because I was too slow. Twenty
every ten seconds. If I remember
years later, I hired Raja to work on
correctly, these data were analyzed in
a computer game project for Harris
real time by a PDP-11 computer and
Bank in Chicago, one of my favorite
sent to a monitor at the teacher’s desk
clients. I gave Raja specifications for
as a moving line on a graph.
creating a dozen Windows-based
At any given moment, the teacher
game shells. My hypothesis was that
could see how effectively she was
subject-matter experts at the bank
approaching the goal of asking an
should be able to plug in their content,
open-ended question every three
and trainees should improve their
minutes.
fluency with financial and legal facts,
I learned important principles from
principles, and procedures by playing
the CATTS project:
the resulting games. Unfortunately, the
ÿ True training blends different
subject-matter experts did not want
approaches and optimally assigns
to work with computers, so I had to
each instructional task to humans
take the content from them and plug
or to machines.
it in the computer game shells (and
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Cheap But Not Tacky
253
charge them for the service). The
project became a major venture, with
games would have been effective—if
programmers in Romania cranking out
only the learners played them. But
lines of code. Any change that Raja or
they had seen too many video games
I suggested had to be documented in
in action and our training games paled
detail on a web-based spreadsheet and
in comparison.
transmitted through several layers of
Here are the lessons I learned from
project managers, program designers,
this project:
supervisors, and coders. Since my
ÿ Nepotism is wonderful as long as
specialty was building airplanes while
you keep it within the family and
flying them, I found the long delays
hire young people to do computer
to be extremely frustrating. From
programming.
an instructional point of view, while
ÿ For computer-based training and
learners could recall the terminology
games to catch on, we have to
and factual technical information
wait for the current generation
by playing these games, I had the
of subject-matter experts to be
depressing data that showed they
replaced by a new generation for
could not apply their knowledge to
whom computers are not an alien
solving real-world problems.
technology but a normal way of life. ÿ Increases in the popularity of video
Here are some more lessons I learned from this experience: ÿ Nothing kills creative design like
games will decrease the acceptance
delays in the implementation of
of computer-based instructional
changes. I needed to work with
games.
a single programmer who could make on-the-spot changes.
The sequel to the computer game
ÿ The ability to answer closed
shell experiment was funded by
questions is a necessary—but not
my friend, Andy Kimball, at Qube
sufficient—learning outcome.
Learning. Andy paid Raja and me to develop an improved version in
Open Questions
To get a feel for
the form of web game shells that
the learner’s
featured Flash programming, animated
perspective
graphics, and sound effects. Learners
toward e-
enjoyed playing these games and
learning, I
increased their fluency with details of
frequently signed up as a learner for
technical content. The web game shell
different online courses and valiantly
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
254
Michael Allenʼs 2008 e-Learning Annual
completed most of them. The feature
intelligence and decided to trade off
that irked me the most was the
the immediacy of feedback for ease of
exclusive and excessive use of closed
use. We rationalized away the fact that
questions. This phenomenon reminded
our hybrid approaches to processing
me of the bad old days of programmed
open-ended responses required more
instruction, which used “copy frames”
work on the part of the learners by
to present a small piece of information
arguing that it was actually good for
and asked a question that could be
the learners. Data that we collected
answered by mindlessly copying an
later supported this hypothesis.
even smaller piece of information. Most e-learning courses depended on
e-Mail Games
The e-mail
fill-in-the-blank, multiple-choice, and
games format
matching questions. I was irritated
is an early and
by this fact and embarrassed by my
effective method
dependence on the same approach. I
for handling
yearned desperately to use open-ended
open responses. In this inexpensive
questions that reflected the wicked
format, the facilitator and players
unstructured problems and tasks of
communicate with each other by
the real world.
sending e-mail notes. All interactions
The obvious reason for the
are limited to low-technology text
attractiveness of closed questions is
messages. My Australian colleague,
the ease of programming the computer
Marie Jasinski, and I currently use
to process learners’ responses and
twenty-four different e-mail games
to provide immediate feedback. I
associated with different types
decided to work with my associates to
of learning. Most of these games
write programs for processing open
incorporate several rounds of play
questions that can accommodate
spread over a number of days.
a variety of acceptable responses.
Here are brief descriptions of how
Shortly after an enthusiastic start, it
we used two e-mail games—Poll and
became very clear that the project
Predict and Depolarizer—in a course
would require far more financial
on facilitation skills.
resources than we had—and perhaps an entire lifetime to accomplish the
Poll and Predict This activity
goal.
began with an e-mail note that asked
We reminded ourselves that real intelligence is cheaper than artificial
participants to reflect on what makes an effective team facilitator. Each
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Cheap But Not Tacky
255
participant was asked to send a list of
In the first round of Depolarizer, we
as many characteristics of an effective
assigned a negative role to half the
facilitator as possible. We compiled a
participants and a positive role to the
consolidated list that contained such
other half. We asked players whose
items as confidence, empathic listening,
last names began with the letters A
expertise in process skills, flexibility,
through M to send us three or more
integrity, and inclusiveness.
reasons why a facilitator’s confidence
The second round of the game
could negatively impact team
required participants to review
performance. We asked the other half
the consolidated list of facilitator
to send reasons why confidence could
characteristics and to complete two
positively enhance team performance. Here is an example of negative
tasks: 1. Select the three most important
impact: Team members feel that
characteristics of effective
the facilitator is arrogant and begin
facilitators.
questioning her judgment.
2. Predict which three items would be selected by most participants.
Here is an example of positive influence: The facilitator’s confidence may become so contagious that team
Participants sent e-mail notes with their personal choices and predictions.
members begin to feel empowered. We collected all comments, listed
In return, we sent them the result
them with negative and positive
of the poll with the twenty items
items alternating, sent the list back
arranged in order of popularity. We
to participants, and invited them
also identified the participant who
to reflect on the items. The impact
made the most accurate prediction
of Depolarizer was to support and
and declared him to be the winner
challenge beliefs and reconcile
of the Outstanding Psychic Award.
apparently contradictory beliefs by
The impact of Poll and Predict was to
identifying the situations in which
generate different characteristics of
different facilitator characteristics may
an effective facilitator and to compare
produce different results.
and contrast these characteristics. RAMEs: Replayable Asynchronous Depolarizer We began the follow-
Multiplayer Exercises RAMEs are
up game called Depolarizer with the
a streamlined version of e-mail games
highest-rated facilitator characteristic
in which a computer program handles
from the previous game: confidence.
the chores of collecting learner inputs,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
256
Michael Allenʼs 2008 e-Learning Annual
displaying them on a web page, and
A couple of days later, you receive
keeping track of scores. To understand
another e-mail note with instructions
this approach, you can vicariously
for the next round of the RAME.
participate in a RAME on the topic of
You click the link, visit the web page,
conflict management.
and review the six highest-scoring
During the first round, you
suggestions. You are happy to see your
receive an invitation to contribute a
original suggestion included in the list.
suggestion for managing a conflict
Being an objective person, you do not
with a co-worker. You click a link,
automatically select your suggestion
go to a web page, and type your
as the best. You review the other five
contribution inside a text box.
suggestions and reluctantly choose
The RAME program collects all
another suggestion as the best and
contributions from twenty-four
your suggestion as the second-best.
participants, divides them into four
The RAME program once again
groups of six, and displays each group
computes the score for each suggestion
on a separate web page.
and arranges the six suggestions in
A couple of days later, you receive another e-mail note inviting you to
order of their popularity. A couple of days later, you receive
review the suggestions from six other
an invitation to visit a web page. When
participants and to select the top two
you do so, you see the six suggestions
suggestions. You click the web page
arranged in order of the number of
link and see a list of suggestions. Your
votes they received. Your suggestion
suggestion is not included in this list,
received the fourth-highest number
and all of the suggestions are displayed
of votes. The web page also contains
anonymously. Under each suggestion,
a complete list of all suggestions
there is a drop-down menu to record
contributed by different players, this
your choice of the top two. You review
time with the names of the people who
the suggestions carefully and select the
contributed the suggestions.
top two ideas. The RAME program computes scores for each suggestion based on how many participants selected it as one of the top two. It collects the top six suggestions from among the three lists.
Our Current Approach at Integration: The Four-Door Approach
About ten years ago, I began to integrate the lessons that we learned and the components
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Cheap But Not Tacky
257
that we created into an e-learning
lot of time studying in the library
system called the Four-Door
before playing these games. If you
Approach. While we are under no
are impulsive (and eat dessert
delusions that we have arrived at
before the main course), you can
the end of our adventures and found
play a game first (and probably
the treasure chest, our clients agree
get trounced) before reading the
that this approach provides a faster,
related content in the library.
cheaper, and better (that is, more
ÿ Cafe. This is where you can hold
instructionally effective) path to
discussions with your classmates
designing and delivering e-learning.
(in an asynchronous mode,
Rather then describing our e-
meaning that you read and respond
learning system in the abstract, let us
at different times). This area
pretend you are a learner and walk you
contains OQs (Open Questions):
through a course.
You give your answers and read
You have enrolled in a technical course on IT service management.
other people’s answers. ÿ Torture Chamber. This is actually
When you log in, you see a home page
a fairly pleasant test center where
that contains four doors, each with a
you are given two chances to take
brief description of what it contains:
a performance test related to the
ÿ Library. This is a repository of all
training objectives. You pass if
content resources for the course.
your score is 80 percent or better.
When you click here, you see a catalogue. Select the document
Library Being a somewhat systematic
you want and read it at your own
person, you visit the library and read
pace, using your personal learning
the first resource, which is labeled
style. We promise not to interrupt
“The Service Desk.” This piece begins
you with pesky little questions in
with a set of training objectives. The
an attempt to force interactivity on
text is presented in plain language and
you.
in short paragraphs, bulleted lists, and
ÿ Playground. This is a collection
meaningful headings and subheadings.
of different web-based games
However, the text is not “dumbed-
(complete with graphics,
down.” Initially, you are surprised by
animation, and sound effects)
the absence of interspersed questions,
that test your mastery of the
since you were expecting frequent
content from the library. If you
interactivity. Very soon, however, you
are reflective, you can spend a
become thankful that you can read the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
258
Michael Allenʼs 2008 e-Learning Annual
text using your own preferred learning
typing the letter A. A different tone
style and take your own notes. As you
and a red light indicate that your guess
read the material, you focus on the big
is incorrect. After the next guess, you
picture and the relationships among
are sure of the answer. So you type all
the concepts without being distracted
the remaining letters fourteen seconds
by intruding multiple-choice questions
before a timer at the bottom of the
that emphasize factual details.
display counts down to zero. When the answer is complete, the score box
Playground Sometime in the middle of reading the article about the service
displays 9 points. The next question (along with
desk, you catch yourself yawning.
blank circles representing the answer)
You are getting bored with this self-
pops up on the play area. Since you
imposed passivity, and you decide to
know the answer to this question, you
go and play for a while. You click the
rapidly fill in the blank circles. Your
playground button at the bottom of
score increases by 10 more points to
the screen.
19. You use the same procedure to play
You are given a choice among five different games. You click HANGMAN, and you are presented
through the total of seven questions in this round of play. At the end of your first round of
with a set of initial instructions. You
play, you discover that you can play
click the “Play” button to see a short-
the same game repeatedly. Every time
answer question followed by a set of
you play, you are presented with a mix
blank circles representing letters in
of new and old questions in a different
the answer. Some of the circles are
sequence. You also discover that you
already filled in with free letters. You
can play the game at three levels of
read the question and try to guess
difficulty: If you find the game too
the answer. Since this question is on
easy, you can advance to the medium
a topic that you did not encounter
level of difficulty, where you play
during your study time in the library,
without any “free” letter clues. At the
you are forced to make a guess. In
hardest level of difficulty, you have to
desperation, you type the letter E and
spell the answer, one letter at a time,
immediately all occurrences of this
in the correct sequence.
letter in the correct answer appear in
After you get a perfect score of
the corresponding circles. A tone and
100 at the most difficult level of
a green light on the side indicate that
HANGMAN, you move on to the
your guess is correct. You continue by
other games.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Cheap But Not Tacky
259
More About the Playground You
except there are only two buttons
return to the opening screen for the
marked “True” and “False.” Different
playground and check out four other
statements appear on the screen. Your
types of web-based games:
task is to quickly read each statement
TIC TAC. In this game, the computer presents a 3 x 3 tic tac toe
and decide whether it is true or false. After playing these games (and
grid, with a short-answer questions in
replaying some of them), you make a
each box. When you click a question,
mental note to work through the next
you are presented with a text box to
course by having fun in the playground
type your answer. You win this game if
first, then studying the text resources
you provide the correct answers to any
in the library and returning to the
three questions in a straight line (row,
playground again.
column, or diagonal). SEQUENCE. In this game, the
For the present, you return to the library and study the remaining parts
computer presents you with a list
of the material on the service desk.
of seven steps used in processing
You feel confident that you have
requests at the service desk. However,
mastered the topics and achieved the
these steps are not presented in the
learning objectives. So you decide to
correct order. You win this game if
visit the cafe.
you rearrange the steps in the correct order (by dragging and dropping the
Cafe When you visit the cafe, you
steps) before the timer counts down to
are presented with an OQ (Open
zero.
Question) that asks,
CATEGORIZE. In this game, the
What one piece of advice would
computer presents four buttons, each
you give to a new employee who is
labeled with one of the four types of
assigned to the service desk? You type
activities undertaken at the service
a facetious answer (“Ask the employee
desk. Different items (words, phrases,
to bring a giant bottle of aspirin”) and
or sentences) appear on the screen.
click “Send.” Your answer disappears
Your task is to quickly read the item
and you are presented a choice of
and click the appropriate button. Your
three buttons that enable you to
score (displayed on a box) increases
display
every time you classify an item
ÿ A checklist to evaluate your answer
correctly.
ÿ Answers from different experts
TRUE OR FALSE. This game is very
ÿ Answers from other participants
similar to the CATEGORIZE game, Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
260
Michael Allenʼs 2008 e-Learning Annual
You review the earlier answers
The test in the torture chamber
from your fellow participants and feel
contains ten scenarios with multiple-
reassured that you could have given a
choice responses. These items are
good answer if you had taken the task
different from those used in the
seriously.
playground. Each item presents an
In the cafe, you also see a discussion
authentic scenario of something that
forum. You visit this forum and see
happens at the service desk and asks
several discussion threads where other
a question. You have to select the
participants have made comments and
best alternative among four plausible
asked questions. You review some of
choices. With some difficulty, you
the entries, but find them bland and
complete the test and click “Submit.”
boring. So you skip that section and
You immediately receive your score
visit the torture chamber.
(90 percent), a congratulatory message, and a brief preview of the
Torture Chamber When you
next module on capacity management.
enter this area, the first thing you notice is a dire warning that this is
Continuation As the days roll by,
the Examination Hall where your
you work through the other modules
answers will be graded to determine
of the course. The format for all
whether you pass the test. You can
modules is the same, but you keep
pass out of the course by taking the
experimenting with different study
test any time you want—even before
strategies, depending on your mood at
you have studied any of the content
the moment. Eventually, somewhere
or completed any of the activities.
around the seventh unit, you work out
However, you have only two chances
your most efficient strategy of taking
to take the test. You must score at
the test first, failing it, then working
least 80 percent in order to pass.
through the library and the playground
You decide to take the test. If you
to gain mastery, and then taking
pass, you can move on. If you fail, you
the test for the second time. You
can return to the library and study
know that you are living dangerously
some more before taking the test
(because you cannot afford to fail the
again.
test the second time), but you enjoy the twinge of excitement.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Cheap But Not Tacky
261
Current Situation
When I first
ÿ We divide lengthier courses into
began sharing
sections or modules, each with the
my ideas about
four doors.
the four-door
ÿ We have added two more doors
approach with
to some of our courses: (1) a
my e-learning and instructional design
front door explains the overall
colleagues, they were kind enough to
goal for the course and provides
point out several reasons why it would
navigation tips and (2) a showcase
not work. I chose to ignore these
door leads to an archival area with
constructive inputs and continued
torture-chamber performances of
working with my colleagues, Matt (the
earlier students. For example, the
extrovert who enjoys interacting with
showcase area for an online course
humans) and Raja (the introvert who
called “How to Design a Training
loves interacting with computers).
Game in Ten Minutes” contains
Matt designed four-door courses for
more than sixty different games
several corporate clients and Raja
designed by the earlier learners
helped to build these courses using
as a part of their final course
different e-learning platforms based
requirement.
on the clients’ preference. Our current
ÿ In addition to the OQs (Open
list of four-door courses includes
Questions) and discussion forums
both technical and soft-skills topics.
behind the cafe doors, we now
Our clients enthusiastically agree that
conduct e-mail games and RAMES
these courses were developed faster
with synchronous groups of
and cheaper than the earlier ones
learners.
developed through the traditional e-learning approach. Contrary to
Our Future
We don’t know
my colleagues’ gloomy forebodings,
what the future
learners demonstrated mastery of the
has in store
course objectives, as measured by their
for our group,
responses to the performance test in
but we know
the torture chamber. While the current format of the
that we will abide by our philosophy: Cheap but not tacky. We will continue
four-door approach remains essentially
to produce e-learning products
the same, we have implemented some
and programs in the fastest and the
minor modifications:
cheapest way we know. At the same
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
262
Michael Allenʼs 2008 e-Learning Annual
time, we will make sure that nothing
ÿ We will avoid the temptation to
we produce is tacky, trivial, useless,
acquire complex comprehensive
or instructionally ineffective. While
platforms that guarantee to work
we keep searching for innovative
in every stage from the initial
approaches, we will continue using
analysis to the final student
these guidelines:
assessment. We value our agility
ÿ We will avoid analysis paralysis.
too much to lock ourselves into
We won’t spend more than five minutes for the initial analysis.
any rigid approach. ÿ We will partner with other groups
We will build airplanes while
and barter our instructional design
flying them. We will produce
and game design products for
rapid prototypes and test them
technical gimmicks.
immediately with learners. ÿ We will not waste our time
ÿ We will collaborate with the client and use her budget to design new
designing content. Instead, we will
game engines. In return, we let
focus on designing activities that
the client freely use game engines
require learners to interact with
funded by previous clients.
the abundantly available content. ÿ We will avoid large projects
We will keep learning new
because we sincerely believe that
principles and implementing new
small is beautiful. We will come up
guidelines in the foreseeable future.
with flexible breakthrough ideas
Sometimes we may get carried away
from modest projects and let other
with a new guideline and temporarily
groups scale them up to large-scale
forget our basic beliefs. To prevent this
ventures.
from happening, we have a banner in our office that proclaims, “e-Learning is all about learning, stupid!”
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Section Four
Are We Lost? Where Do We Go from Here?
Allison Rossett Allison Rossett is in the Training magazine HRD Hall of Fame and was recognized by ASTD in 2002 for career contributions to workplace learning and performance. She edited the ASTD eLearning Handbook: Best Practices, Strategies, and Case Studies for an Emerging Field. Allison also co-authored a white paper for the American Management Association: “Blended Learning Opportunities” and is the author of four award-winning books, including Beyond the Podium: Delivering Training and Performance to a Digital World and First Things Fast: A Handbook for Performance Analysis. Prior award-winning books are Training Needs Assessment and A Handbook of Job Aids. Some of Rossett’s recent articles are “Performance Support Tools: Delivering Value When and Where Needed,” “Moving Your Class Online,” “Confessions of a Web Dropout,” and “Training and Organizational Development, Siblings Separated at Birth.” She has conducted research and published articles on needs analysis, learner engagement, and persistence in online learning. Her client list includes IBM, HP, the Getty Conservation Institute, Fidelity Investments, Eli Lilly, Deloitte Consulting, BP, the IRS, Amgen, Royal Bank of Scotland, National Security Agency, Veteran’s Administration, and several e-learning start-ups, including Elearning for Kids. Dr. Rossett is a professor of educational technology at San Diego State University. Her 2007 book and website, Job Aids and Performance Support: From Knowledge in the Classroom to Knowledge Everywhere (http://www.colletandschafer.com/perfsupp), reflect the rich possibilities boosted by new technologies and ways of thinking about learning and work. She continues to work on performance support tools devoted to nudging workforce learning professionals towards more strategic roles in their organizations.
Moving e-Learning into the Workplace*
Allison Rossett
In this article, the author discusses and endeavors to define e-learning and performance support, particularly focusing on the issues of integration and tailoring. She then goes on to explore the subject of how best to integrate these learning mechanisms into the workplace, providing this learning to individuals where and when they need it.
What attracted me to e-learning,
to capitalize on what we can do today.
and what I like about it still, is how
Unfortunately, e-learning today is
it empowers employees. Done well,
too often characterized by separation
it allows them to learn and seek and
from the world of work. It happens
find when they want to do so, in the
at a time and in a place distinct from
flow of work, as challenges emerge.
customer inquiry, sales opportunity, or
With e-learning, I saw opportunities
broken equipment. For the most part,
not possible with instructor-led forms.
employees who turn to e-learning are
More independence. More context.
expected to set aside time in order to
More variety. And more community
study up on topics that might be of use
too, when done right. My original
in the future, just in case they need to
hopes were focused on the technology,
know it. e-Learning must be invited
on creating metaphors for learning
into peoples’ busy schedules, because
experiences that would grip adults.
it is distinct from the strategic efforts.
I can remember trying to teach the
The model is familiar . . . and ancient.
scientific method by using a detective
The assumptions are akin to training
agency. Another effort was to define
and development that happens in a
forms of analysis for my grad students
hotel room or a training center where
by using a women’s fitness center that
people come to acquire skills that might
specialized in helping them lose weight
be helpful in the future. But the times,
post-partum. At the least, these were
and e-learning, are changing. That’s
labored efforts that did not even begin
what this article is about.
*Portions of this article are adapted from the new book, Job Aids and Performance Support in the Workplace: Moving from Knowledge in the Classroom to Knowledge Everywhere. San Francisco, CA: Pfeiffer, 2007. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
266
Michael Allenʼs 2008 e-Learning Annual
Three questions provide focus:
found executives who were keen on
1. What is e-learning and where did
e-learning, but employees who were
it come from?
not. A 2006 study done for a large U.S.
2. What is performance support
federal agency found that employees
and how does it expand the
and their supervisors surprised
possibilities?
executives with their eagerness for
3. What threatens the shift of learning into the workplace?
learning, reference, and support in the context of work. e-Learning shifts control to
1. What Is e-Learning and Where Did It Come From?
You would think
individuals. They can make choices
that defining
about what, when, and where, much
e-learning would
in keeping with the self-reliance
be simple. It is
and virtuality that increasingly
not.
characterizes the workplace.
It begins with debate about whether
e-Learning advances the distribution
the word e-learning is worthy of its
of messages far and wide. Executives
own space in the lexicon. Shouldn’t the
want training to be consistent,
focus be on learning, period, without
standard, and accessible. They
calling out a particular delivery system
want messages that are easily and
or technology? Wouldn’t we profit
immediately updated, available where
from focusing less on the “e” and more
employees can put their hands on
on the learner, learning, and context?
them as needs emerge. Few leaders are
Good points, of course, but there
interested in icebreakers, FLASH, and
are reasons to take an independent
objects.
and optimistic look at e-learning.
e-Learning is too delicious to ignore.
e-Learning is what clients want.
The best reason to focus on e-learning
They perceive benefits that start
is that it enables us to do great things
with cost. There is steady and
for people and organizations. What
documented growth in the penetration
are those things? An example makes
of technology-based training and
the point. Consider the sales road
development. ASTD’s State of the
warrior who can fulfill compliance
Industry surveys document growth
requirements even if she can’t get back
and anticipate nothing but more
to the training center. Note that she
advancement in the numbers.
is also able to consult with an e-coach
e-Learning is what employees
who is helping her sell at higher levels
want. Learners do. Early studies
in customer organizations. She checks
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Moving e-Learning into the Workplace
267
in on the sales online community
collaboration. It includes the
devoted to her vertical market and
delivery of content via Internet,
eagerly tunes in from Bangalore,
intranet/extranet (LAN/WAN),
India, to watch a simulation and then
audio and videotape, satellite
an asynchronous briefing on a new
broadcast, interactive TV, and CD-
product. Eager for a promotion to
ROM (www.cybermediacreations.
manager, she signs up for a series
com/elearning/glossary.htm).
of e-learning modules about people
ÿ e-Learning most often means
management. When a customer
an approach to facilitate and
stumps her with a question about
enhance learning through the use
her system and a competitor’s, she
of devices based on computer
points online to a comparison table
and communications technology.
and prints a copy for him on the spot.
Such devices would include
These options serve the employee and
personal computers, CD-ROMs,
the organization and move lessons
digital television, PDAs, and
where they are needed, close to the
mobile phones. Communications
work.
technology enables the use of the Internet, email, discussion forums,
e-Learning Is All This? It can be, but
and collaborative software (en.
mostly it is not. Not yet.
wikipedia.org/wiki/Elearning).
There is no shortage of definitions.
Notice the focus on learning,
They illustrate the progression and
training, and delivery of the
possibilities. Here are just a few
familiar in technological ways.
plucked from the Internet early in
Notice the emphasis on kinds of
2007:
technologies. Cisco, just before
ÿ The delivery of a learning, training,
the turn of the century, defined
or education program by electronic
e-learning as “Internet-enabled
means. e-Learning involves the use
learning.” e-learning in these
of a computer or electronic device
definitions is all about methods for
to provide training, educational,
delivery of education and training.
or learning material (www.intelera. com/glossary.htm). ÿ Covers a wide set of applications
The next two definitions are favorites because they are intent on purposes. They go beyond defining
and processes such as Web-based
e-learning as an opportunity to
learning, computer-based learning,
put classes online. They value
virtual classrooms, and digital
distribution, impact, standards,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
268
Michael Allenʼs 2008 e-Learning Annual
information, and learning out there
How Did We Get Here? Once
where life and work happen.
CEO of SumTotal Systems and now
ÿ e-Learning is any virtual act or
CEO of the Institute for Corporate
process used to acquire data,
Productivity (i4cp), Kevin Oakes
information, skills, or knowledge.
conceptualized the development of
In the context of our research,
technology-enhanced learning and
e-learning is enabled learning,
performance in Figure 1.
learning in a virtual world where
Since the turn of the century,
technology merges with human
learning organizations and
creativity to accelerate and
professionals have worried about
leverage the rapid development
the question of impact and meaning.
and application of deep knowledge
Do your efforts, in rooms or online,
(www.mountainquestinstitute.
matter? How can we be certain? As
com/definitions.htm).
seen in Figure 1, the contemporary era
ÿ e-Learning is the delivery of
is devoted to results, not just learning.
material electronically with
Learning is a big part of it, but not all
the added value of maintaining
of it, by any means.
standards and quality across the
Rossett (2002) expanded the
board without the limitation of a
possibilities by presenting five
specific location (Shoniregun &
purposes for e-learning in the “results
Gray, 2004).
era.”
Figure 1. Oakes’ Eras of e-Learning
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Moving e-Learning into the Workplace
269
1. Learning: Current views support
it are at the heart of KM. Can that
a definition of learning that revolves
happen online, through e-learning?
around a change in the brain, in what
You bet it can—and it is. Visit
we can remember and use, when
the Encyclopedia of Educational
needed. You could say that we’ve
Technology (http://coe.sdsu.edu/
learned when we “get it.” Typically,
eet/), a free resource produced by
that takes some effort, some thought,
educational technology graduate
some mental manipulation, such as
students at San Diego State University.
working on a problem and comparing what you’ve done with an expert, or
3. Collaboration: Even though
looking at what someone else has done
classroom instruction is often
and attempting to identify errors.
remembered for its interactive
Through examination, actions, and
moments, technology is also good
feedback, repeated over time, people
at engagement. It can bring people
learn. Can that happen online, through
together for many purposes, such
e-learning? You bet it can. Take a
as one-to-one development and
look at examples available at Allen
e-coaching, online communities
Interactions (www.alleninteractions.
of practice, pre- and post-class
com).
listservs, and individual engagement with complex simulations, cases,
2. Knowledge management (KM):
and examples. One pharmaceutical
KM is an attempt to maximize the
company brings global teams online
“smarts” that exist within people and
to work on pressing problems; a
organizations. A famous quote about
telecommunications company creates
Hewlett-Packard pinpoints the need:
a collaborative online workspace to
“If HP only knew what HP knows.
manage its product launch; and a
. . .” Successful practices linger in a
group of online graduate students
company for years, unrecognized and
fulfills assignments and nurtures their
unshared. KM is represented by large
teaming skills. Can that happen online,
and small efforts to collect documents,
through e-learning? You bet it can.
practices, and lessons learned in a
It does and it must. Many examples
way that facilitates wide distribution,
are included in Rossett and Marino’s
engagement, and adoption. Getting
article about e-coaching (available
a grip on this knowledge, nurturing
at www.astd.org/NR/rdonlyres/
it, making it accessible, encouraging
73D0090C-0FB4-4DBF-A4A5-EA1C63
conversation about it, and updating
583241/7922/76051146.pdf ).
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
270
Michael Allenʼs 2008 e-Learning Annual
4. Guidance: New technologies
where they are in light of where they
for learning and support enable
and their organization hope to go.
more and better measurement, guidance, assessment, tracking, and
5. Performance support when and
information. Individual contributors
where needed: While learning and
can look at themselves in light of
coaching are directed at enhancing
standards, can test their mettle on
individual capacity, performance
scenarios that reflect organizational
support is an external, organizational
trends and emerging priorities.
resource to which the individual
Organizations, even, can assess
turns for information and guidance
their status and readiness. See, for
at the moment of need. Do you want
example, Senior Corps Tech Center’s
to make sure your family is ready for
Office Self-Assessment Tool (www.
a natural disaster? Want to install a
seniortechcenter.org/info/tso_assess.
motherboard? Want to change the
php?agency_size=med). Best of all,
message on your answering machine?
systems can guide individuals toward
Want to mix a cosmopolitan? Want
critical skills and options to address
to look knowledgeable in front of the
development needs. Managers can
customer about a pesky competitor’s
enjoy a data-based view of employee
products? Want to figure out whether
skills and unit performance.
you are ready to retire? Can this
Executives, too, can capture a view
happen online, through a particular
of organizational skills and needs,
kind of e-learning? You bet it can. Take
looking at what people elect to study,
a look at a performance support tool
what they don’t, where they search
(www.colletandschafer.com/perfsupp/
and what they find, and how they
tool.html) that helps contemplate
participate in online communities
whether your challenge would be well
of practice. In addition, leaders can
served by performance support.
comfort themselves and their legal influencers with technology-based information about compliance and risk avoidance. Can that happen online, through e-learning? You bet it can. Individual and executive dashboards (see Chris Moore’s Zeroed-In
2. What Is Performance Support and How Does It Expand the Possibilities?
Performance support is an informationrich asset that a pilot, nurse, engineer, parent,
Technologies, for example: www.
mechanic, taxpayer, or auditor turns to
getzeroedin.com/) allow people to see
for help in doing things. Performance
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Moving e-Learning into the Workplace
271
support appears in many forms, from
example of the reach of e-learning into
notes on matchbook covers to well-
work and life, into the places where
worn documentation, to posters to
it most belongs. The move is away
ehelp.com and GPS.
from the classroom model and toward
Let’s use a real example. In a project
the provision of dynamic guidance
for a government health agency,
and information where and when it is
performance support was used to help
needed.
patients remember what they had to
Rossett and Mohr (2004) used
do to prepare for their medical and
questions to dramatize the value of
surgical procedures. Could they eat
performance support:
dinner the day before? How about
ÿ Are you willing to send employees
breakfast the morning of? How about
to a class so that they can answer
nail polish? May they continue to
customer’s detailed questions
smoke cigarettes right up to the
about insurance policies,
moment of surgery? What about a cup
cell phone features, or the
of coffee to calm nerves just prior?
demographics of Basra or Bimini?
Developer Betsy Bruce described her
We doubt it.
efforts this way: “The tool is simply a
ÿ After a class about numerous cell
web page created using XHTML. The
phone features, for example, will
logic was created using JavaScript. It’s
your employees remember them
just like any web page, but it has some
later, when queried by a customer?
coding behind it. It just requires a
We doubt it.
modern browser to access—players or plug-ins required!” Couldn’t patients be briefed by
ÿ Are you willing to rely on employee memory when critical, complex, or dangerous actions are
the nurse or doctor? How about
involved on an airplane or in a
having them take a short class? Could
nuclear plant? We doubt it.
they look at a handout? Those are
ÿ Should the organization do what
possibilities, but who can remember
it takes for employees to learn
what they said? Who can find that
such material by heart, especially
handout when you need it? Doesn’t
when there is much to learn, the
it make sense to collect the guidance
content changes often, a mistake
online, in a personal portal devoted to
is dangerous, and employees could
your condition and treatments?
take their expertise with them
Rossett and Schafer (2007) believe that performance support provides an
when they leave the job? We doubt it.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
272
Michael Allenʼs 2008 e-Learning Annual
Performance support matters
Integration, Tailoring, and Finding
because it goes where life and
Your Way Imagine that you have an
work take us and delivers smarts,
important appointment across town,
advice, guidance, and information
in a place you have never been, in
into the field of play. The following
an unfamiliar part of town. Could
descriptions and examples are taken
you study up for the trip online? You
from Rossett and Schafer’s (2007)
could. But do you want to spend
new book, Job Aids and Performance
your time that way? Table 1 presents
Support: Moving from Knowledge in the
performance support systems to
Classroom to Knowledge Everywhere.
take you to your destination. How
Two dimensions are critical in performance support. The first
integrated is each approach? Let’s look next at tailoring. Is the
dimension is the degree of integration.
tool offering up a standard message
Is the performance support inside or
or one that is customized to you and
outside of the task? Is it like TIVO or
your situation? How active is the
is it a computer program that helps
tool? Does it adjust to you and your
decide how to save for retirement?
realities? Does it care which mutual
TIVO support is inside, integrated into
funds you hold, how old you are, how
the task. The retirement guidelines
many you must put through college,
stand apart; they nudge you to think
what products you sell and in which
about this, worry about that, and plan
geography, or if you just bypassed
for future eventualities.
the verbal suggestion to turn left at
The second dimension for
Oak Street? Does it reach out and
performance support is how much
nudge and remind about goals you
tailoring the support offers. Is the
committed to on New Year’s Eve? Does
support standard for all or actively
it provide a statement of operating
tailored to your situation? Does
procedures or model approaches to
it know you and coach differently
customers’ objections? The Yellow
because of that knowledge? Is it a mass
Pages and Technorati’s searchable
mailing from your city government
blog directory are standard, passive,
about fire danger in California or a
valuable resources. You go there to
notice sending you to a unique website
find information on mockingbirds or
because the system recognizes that
local veterinarians who specialize in
your home is on a canyon and you
farm animals. Those trusty resources
must mitigate fire danger in special
wait and serve, but they do not
ways?
customize automatically. You must Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Moving e-Learning into the Workplace
273
Ways of Getting to an Unfamiliar Destination
Commentary
Enhance workforce
The map is a familiar form of performance support, a job
productivity
aid. Note, however, that it is not insinuated into the task and is not tailored to you. It doesn’t know where you are going or offer guidance on the best way to get there. You must make it useful for you, and if it stays on the wall, it will not be close to the challenge you encounter as you go to the goal.
Go to MapQuest; enter your
Here we see the blending of computer-based
address and the destination
performance support and job aids. The online tool is
address. Print the results and
used prior to undertaking the challenge. It is preparation
stuff them in your purse.
for it. It helps you plan for the trip.
Refer to the print results as you
Here we have a conventional print job aid that is
drive.
insinuated into the task, albeit precariously. As you negotiate the highways and byways, a trusty piece of paper points the way. There is no customization in response to detours or distractions.
As you head out the door, ask
This may get you there if you have a good memory and
your brother for directions. He
your brother is reliable. But insinuated with the task?
generously provides them.
Definitely not. And when you get distracted and turn left instead of right, his directions will not adjust to your errant ways.
Dig through the car glove box.
This dangerous support is insinuated into the task. It
Pull out a map. Read the map
is there as the need to decide RIGHT or LEFT. While
while you drive.
integrated into the task, you have to tailor it to you, and at some risk.
Dig through the car glove box.
This dangerous support is insinuated into the task. It
Pull out a map. Read the map
is there as the need to decide RIGHT or LEFT. While
Table 1. Using
while you drive.
integrated into the task, you have to tailor it to you, and
Performance
at some risk.
Support to Reach a Destination
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
274
Michael Allenʼs 2008 e-Learning Annual
know what you want and need and go
might turn to a rubric to help them
for it.
reflect on a recent presentation. How
On the other hand, Fidelity.com is
did they do? Did they make those
different because it is actively focused
points? Did they use these techniques?
on you. If you have accounts with
What do they want to incorporate next
them, they know you, your financial
time around?
holdings, and your goals. Fidelity attempts to help you reach financial
Is Tailoring a Myth? Jupiter
goals in a way that is just for you.
Research (2003) found that only 8 percent of respondents reported that
The Flip Side of Integration and
personalization increased their visits
Tailoring Integration and tailoring
to web content. Think about that. This
have much to commend them. There
number contrasts with 54 percent of
are, however, things to say in favor
consumers who stated that basic site
of less integration and not so much
improvements would encourage more
tailoring. Let’s look at integration first.
visits—54 percent cited faster-loading
The opposite of integration is not disintegration. In this case, it is planfulness, readiness, a reflective
pages, and 52 percent cited better navigation as greater incentives. Personalization sounds good, but
stance. It is information and advice
is it as good as it costs? A single,
consulted right before and soon
standard message costs far less to
after performance. How can we use
produce and maintain. With the cost
delivered knowledge to prepare, to
of personalization estimated at four
mitigate, and to make better decisions
or more times basic costs, Jupiter
about what to do next? Performance
Research and respondents favored
support of this type surrounds
attention to usability over a tailored
effort, rather than intervening in
experience.
the midst of it. Hurricane Katrina
Regulatory content is also an issue.
raised motivation to turn to such
The pharmaceutical industry receives
tools. Consider the many websites
government approvals for what they
that provide lists to help prepare for
may say about medications. One way
hurricanes, earthquakes, floods, and
of controlling the content, of making
fires. All are examples of the value of
certain that there is no gap between
thinking ahead in a systematic fashion.
intended message and received
There is also value in thinking back
message, is to craft, codify and
on a recent effort. Sales professionals
distribute standard content.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Moving e-Learning into the Workplace
275
Two Kinds of Performance
as she creates the new food product.
Support Because there are many
The quarterback glances at his wrist
possibilities, we’ve tamed the domain
in the huddle. The writer pecks away
into two kinds of performance
and smiles at how Wikipedia sports
support: Planners and Sidekicks.
a red line under it in this sentence.
Planners are in our lives just before
Sidekicks vary in how close they are
or after the challenge. They help us
to the task. They might be next to the
decide whether the current health
task, as is the case with the cook and
and terror conditions abroad should
quarterback, or integrated into it, as in
force us to alter trip plans. Similar
the spelling checker.
approaches were used to help many
Now let us add the question of
and diverse Senior Corps managers
tailoring. Will the message be for
look at their unit’s readiness for
all, or will the system actively tailor
technology. That’s a classic example of
it for the user, based on user goals
Planner Performance Support.
and conditions? Is it guidance for
Sidekicks are at our side during the task. The quick food cook reads
everybody interested in retirement, or does it know you and your situation
Performance Support
Standard
Tailored
Sidekicks: They
Here the customer and sales
When the customer picks a product
are with us in the
person look at a PC and
configuration, the sales person
work, as we act.
examine a table that compares
identifies the customer and the
a recommended product and its
system details what it will take
competitors.
to achieve compatibility with this customer’s current installed base.
Planners: They
This is a print or automated
This is an automated program
are there when we
program that reminds a sales
that seeks data about a potential
get ready to act
person what to keep in mind as
customer, qualifies the customer,
and afterwards,
when selling at higher levels in
and then informs the sales person
when we reflect
the organization. Afterwards, the
of the size loan for which he/she will
on our efforts
sales person can reflect on the
qualify. The amount and rationale
Table 2. Kinds
interaction in light of the criteria.
are provided to the sales person to
of Performance
aid in countering objections.
Support
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
276
Michael Allenʼs 2008 e-Learning Annual
and tailor advice accordingly? Table
went unoccupied. Nobody attended.
2 illustrates standard and tailored
When some did, most did not persist.
planners and sidekicks.
To assure success, expectations are
The example in Table 2 raises
redefined, performance management
an important question. Can we
matches the new realities, and
skip training? Can we hand these
supervisors and managers are keen
tools to neophytes and send them
to nudge and coach their employees
into a sales engagement? While
toward new roles and expectations.
performance support delivers
But what happens when the world
immediate information and guidance
is imperfect? Rossett and Frazee
in the workflow, in itself this is not
(2006), Rossett and Schafer (2003),
sufficient. You would not want to take
and Dunlap and Grabinger (2003)
fresh salespersons and hand over the
tackled that question. Stretching the
tools. They need prior development,
literature on self-reliant learning to
online or in rooms. McManus and
the challenges presented to individuals
Rossett (2006) looked at how six
by e-learning in the flow of work,
organizations were using performance
they encourage approaches that
support. In every case, the tools were
nudge individuals to persist, refer,
part of a blend. Education and training
and benefit from an information-
bolstered the use of support, not
rich environment. Will employees
surprisingly.
look things up? Will they call on these support resources, human and
3. What Threatens the Shift of Learning into the Workplace?
The forms of
technological, when confronted with
technology
a problem or irritated customer? Will
application
they contribute to these resources?
discussed
Will they keep coming back for more
above move
ideas and development? Will they
learning away from a controlled classroom environment and into the
engage? There are reasons to assume that
wild and wacky world of work. At
the transition will be halting. Table
work, individuals elect to engage—
3, adapted from Rossett and Schafer
or not. At work, supervisors can
(2007), presents strategies to advance
encourage—or not. Every one of us has
individual success in a performance
heard tales of e-learning “seats” that
centric world. The strategies listed in
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Moving e-Learning into the Workplace
ÿ Provide obviously useful content. ÿ Be explicit about how the
277
ÿ Provide choices in amount of support within the e-learning:
resources advance personal goals;
from hand-holding that does most
provide testimonials.
of the work, to a trusted advisor
ÿ Be explicit about how the resources advance organizational goals; provide results data and testimonials. ÿ Configure the programs so that
there for support, to a provider of very occasional tips and hints. ÿ Ask for suggestions about how the programs could be more useful. ÿ Ask supervisors to encourage
they respond to typical and
employees toward broadly defined
important dilemmas or challenges.
e-learning by remarking on their
ÿ Remind users of all they already know and how their knowledge helps them use the e-learning.
use of the programs. ÿ Build the use e-learning into orientation, training, and coaching.
ÿ Provide examples of how these and similar programs have contributed
Table 3. Helping People Learn in the Workflow
already.
Table 3 are directed at the employee
an immediate sale? Do salespeople
and supervisor. But what of workforce
participate in online communities? Do
and e-learning professionals? Are
they contribute to knowledge bases?
they ready to think in new ways about
Do they learn and seek information
themselves and their work?
continuously? Do they acknowledge
When we look to the question of
the need to learn about products,
what signifies success, we envision
services, and vertical markets
a shift away from counting noses
throughout their days and across their
in classrooms and hits on websites
work lives?
to the search for influences on
It is common practice to look
strategic outcomes and daily habits.
askance at Kirkpatrick’s Leve1
Are salespeople more fluent as they
1 evaluations, the level that
close sales? Do they close more sales?
represents an individual’s reaction
Do customers return to them for
to the experience or program. Such
advice, even if it does not lead to
individual satisfaction has often been
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
278
Michael Allenʼs 2008 e-Learning Annual
cast as less worthy than the higher
to practice. What it does speak for is
Kirkpatrick levels, where outcomes
more attention to guidance systems,
are examined. While results are, of
orientation, and providing directions
course, the reason for the effort,
and worked examples, especially at the
satisfaction and appreciation grow in
get-go.
importance in a performance-centric world. What salesperson or customer service rep or firefighter will look to elearning resources if he or she doesn’t see the worth? As organizations move away from classrooms and into blends delivered at work, employee satisfaction elevates in importance. And there is one other matter with
Is This About Computers, Cell Phones, Wikis, Performance Support, or e-Learning?
The distinctions were once clear. They are not that way today. In this article, the contention is that such
blurring is a good thing because it
special significance for e-learning
puts lessons, information, and ideas
professionals. Recent practice esteems
out there in many ways, almost all the
constructivist approaches. There isn’t
time.
a graduate or certificate program that
A recent visit to a doctor’s office
does not favor scenarios, cases, and
made the point. It was surprising to
problem-based learning. While it is
see a well-thumbed WebMD magazine
hard to argue against these authentic
in the waiting room. But WebMD is
and engaging approaches, Paul
an online health website?!? Or is it a
Kirschner and colleagues (2006) raise
glossy magazine?
questions about the wisdom of using
Newspapers, on the other hand,
minimally guided instruction, such as
are pushing their content online. No
cases, scenarios, and problem-based
problem following San Diego’s pension
activities, for novices or when new
scandals or baseball Padres while on
topics are introduced. They make this
the road in Singapore. The newspaper
statement: “Controlled experiments
is online.
almost uniformly indicate that, when
The content is swinging every
dealing with novel information,
which way. What is a podcast offered
learners should be shown what to do
by a product expert, delivered on an
and how to do it” (Kirschner, Sweller,
MP3 player or streamed on a laptop?
& Clark, 2006, p. 8). This does not
Is it e-learning? Is it knowledge
mean the end of cases, webquests,
management? Is it performance
scenarios, or realistic opportunities
support? If the content supports a visit
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Moving e-Learning into the Workplace
279
to the new Tate Gallery in London
individuals sort options as they press
and helps with understanding and
on daily routines. You would want
appreciating contemporary art, is
e-coaching available 24/7. And you
it e-learning? How about if you add
would want measurement of progress
an online community comprised of
toward better time management, with
experts and visitors? Does that make it
frequent feedback to the individual
e-learning? How about an FAQ? Or a
and the supervisor. How is all this
test? Would some form of assessment
delivered? It comes via the laptop,
convert it to e-learning?
cell phone, MP3 player, and personal
Let’s close with a real challenge confronted in most organizations.
information manager. Michael Allen, editor of this book,
Leadership wants people to manage
spoke to the confusion surrounding
time better. With a world chock full
the definition of e-learning. In an
of classes about time management,
interview with ASTD’s Learning
both online and with instructors,
Circuits in July 2004, Allen said, “I
the question is about impact. Do
didn’t really think this would be much
these classes lead to better time
of a challenge—doesn’t everybody
management? How do we know?
pretty much know what e-learning is?
An executive for a giant technology
But since terms are useful only if they
company once told me that she was
have widely accepted definitions, I
shifting all time management training
thought I’d better do a little research
to e-learning. When asked why, she
rather than just spit something out.
said something along these lines:
I was astounded by the variety of
“Well, it’s not working in classrooms,
definitions I found in various books
so it might as well not work online.
and articles. I really disagreed with
Costs less.”
some, such as those asserting that
If you cared intensely about a group
only Internet applications qualified
or individual’s time management,
as e-learning. I finally settled on
you would want nudging, advice,
ASTD’s definition. While wordy, I
lessons, and guidance where the
found this definition comfortable at
opportunities and distractions are,
the time: ‘e-Learning covers a wide set
at work. You would want messages
of applications and processes, such
integrated into Outlook. You would
as Web-based learning, computer-
want an e-learning program devoted
based learning, virtual classrooms,
to prioritization, supplemented by
and digital collaboration. It includes
sidekick performance support to help
the delivery of content via Internet,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
280
Michael Allenʼs 2008 e-Learning Annual
intranet/extranet (LAN/WAN), audio-
Think back to the array of strategies
and videotape, satellite broadcast,
proposed for time management. Are
interactive TV, and CD¬-ROM.’”
they interactive? You decide.
Michael Allen continues, “What this
They represent e-learning in
definition doesn’t specify, however, is
the context of getting things done.
that e-learning is interactive, or rather,
Admittedly, it is a definitional mess,
provides instructional interactivity.
which is the way work and life are, and
While you can learn from many things
thus a pretty good thing. Employees
in this world, it’s the interactivity that
and their needs and tasks are
differentiates learning from mere e-
everywhere, all the time. That’s where
publishing.”
e-learning must go.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Moving e-Learning into the Workplace REFERENCES Allen, M. (2004, July). Down with boring e-learning! Learning Circuits. Available: www.learningcircuits.org/2004/jul2004/allen.htm. Cross, J. (2004, March 26). The history of e-learning, The future of e-learning. Available: www.internettime.com/Learning/articles/OTH.doc [last accessed January 8, 2007] Dunlap, J.C., & Grabinger, S. (2003). Preparing students for lifelong learning: A review of instructional features and teaching methodologies. Performance Improvement Quarterly, 16(2), 6-25. Jupiter Research. (2003, October 14). Jupiter Research reports that website “personalization” does not always provide positive results. Available: www. jupitermedia.com/corporate/releases/03.10.14-newjupresearch.html [last accessed October 17, 2005] Kirschner, P.A., Sweller, J., & Clark, R.E. (2006, January). Why minimal guidance during instruction does not work. Educational Psychologist, 4(1). McManus, P., & Rossett, A. (2006, February). Performance support: Value delivered when and where needed. Performance Improvement. Rosenberg, M.J. (2003, March). Redefining e-learning. Performance Improvement, 42(3), 38-41. Rossett, A. (2002, March). Overcoming insomnia in the big tent of e-learning. Performance Express. Available: www.performancexpress.org/0203/ Rossett, A. & Frazee, R. (2006). Blended learning opportunities. New York: American Management Association. www.amanet.org/blended/pdf/ WhitePaper_BlendLearn.pdf Rossett, A., & Gautier-Downes, J. (1991). A handbook of job aids. San Francisco, CA: Pfeiffer. Rossett, A., & Mohr, E. (2004, February). Performance support tools: Where learning work and results converge. Training and Development, 58(2), 35-39. Rossett, A., & Schafer, L. (2003, June). What to do about e-dropouts: What if it’s not the e-learning but the e-learner? T+D, 57(6), 40-46. Rossett, A., & Schafer, L. (2007). Job aids and performance support: Moving from knowledge in the classroom to knowledge everywhere. San Francisco, CA: Pfeiffer. Also see: www.colletandschafer.com/perfsupp/ Schmid, R.F., & Gerlach, V.S. (1990). Instructional design rules for algorithmic subject matter. Performance Improvement Quarterly, 3(2), 1-14.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
281
282
Michael Allenʼs 2008 e-Learning Annual
Shoniregun, C.A., & Gray, S. Is elearning really the future or a risk? Ubiquity, 10 (2004). Association for Computing Machinery. Available: www.acm.org/ ubiquity/views/c_shoniregun_3.pdf.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Nick H.M. van Dam Nick H.M. van Dam is an idea generator, visionary, consultant, and thought leader. He is the global chief learning officer for Deloitte and advisor for the Deloitte Consulting Human Capital Practice. As an internationally recognized consultant and thought leader in learning and human resources development, Mr. van Dam has written articles and has been quoted by The Financial Times, Fortune magazine, BusinessWeek, Management Consulting, Learning & Training Innovations magazine, T+D Journal, Bizz magazine, and The India Times, among others. He is a columnist for CLO magazine (U.S.) and Intellectueel Kapitaal Magazine (The Netherlands). He has authored or co-authored a number of books, including Organisation & Management: An International Approach (1991-2007 Dutch and English); Business Simulations—Topsim Series (1995); Change Compass, 2001; The e-Learning Fieldbook (2004 and 2006, Mandarin edition), The Business Impact of e-Learning, 2005 (dissertation), and 25 Best Practices in Learning & Talent Development (2007).
The Business Impact of e-Learning*
Nick H.M. van Dam
In this article, the author discusses the business impact of e-learning, exploring such questions as: “Why do businesses invest in e-learning?” and “What are the elements of a successful e-learning implementation?” Critical success factors include e-learning culture, design, and technology and performance management. Potential benefits of e-learning to businesses include enhanced worker productivity, improved earnings, and enhanced revenue. Economic, social, and technological
members of the Baby Boom generation
forces continue to change organizations
turn sixty-two.
around the world and the global
The arrival of the Internet has led
economy. The adoption of new
to a related innovation in the approach
technologies, the rapid obsolescence
to training and learning, and the term
of knowledge, the need for just-in-
that has been coined to characterize
time training, and the search for cost-
this phenomenon is e-learning. Given
effective ways to meet learning needs
the exponential growth and popularity
of a globally dispersed workforce have
of e-learning over the last decade,
redefined the processes that underlie
one would expect that there would
design, development and delivery of
be a common understanding of what
training and education in the workforce
e-learning encompasses. Yet, the
(Urdan & Weggen, 2000). The Hudson
existence of a broad range of definitions
Institute (2004) suggests that only 20
and applications of e-learning
percent of today’s workforce has the
suggest just the opposite. According
skills for 60 percent of the jobs that
to Brooke Broadbent, “It seems that
will exist in the year 2020. Additionally,
everyone from e-learning theorists and
demographic changes will have an
practitioners, to vendors and learners,
impact on training needs. A wealth
are applying the term e-Learning to
of skills and experience will begin to
very different methods and products”
disappear from the market as the first
(2002, p. 10).
*Based on the published dissertation of Nick H.M. van Dam. The Business Impact of e-Learning (Doctoral Dissertation, Nyenrode Business University, 2005). Intellectueel Kaptiaal. Available at www.ikmagazine.nl/downloads/DissertatiVanDam.pdf.] Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
286
Michael Allenʼs 2008 e-Learning Annual
The main questions that are
inefficiencies with CD-ROM-based
discussed in this article include:
learning that have been eliminated by
ÿ How can e-learning be defined?
net-enabled learning such as:
ÿ How do enterprises justify their
ÿ The costs of production,
investments in e-learning? ÿ What appear to be critical success
distribution, and version control of CD-ROM-based learning. The
factors for the implementation of
development and production of
e-learning?
CD-ROM-based learning is very
ÿ Which elements can make e-learning an effective business
time-consuming. ÿ The limits to tracking and learning management. CD-ROM-
practice? ÿ What is the business impact of
based learning has very limited possibilities for supporting
e-learning?
enterprise-wide tracking of
How Can e-Learning Be Defined?
As mentioned
registration and attendance at
above, there
programs and following progress
are myriad
and completion of learning
definitions
programs
for e-learning
ÿ The limits to storage capacity. It
available. For my purposes, I have
is impossible to put an entire (or
defined e-learning in a business
significant) course library on a
environment as “network-enabled
CD-ROM because of a maximum
learning targeted to achieve business
storage limit.
goals.”
ÿ The inability to pursue integration
In this definition, the network
with other e-learning solutions
includes the Internet and intranet,
such as knowledge management
but does not include CD-ROM-
and performance support.
based learning. One of the key
ÿ The lack of interaction and
characteristics of the Internet is the
collaboration (live) with other
instant dissemination of learning
learners.
content, as well as 24/7 access to the
ÿ A number of different learning
most up-to-date learning content for
delivery media cannot be leveraged
users. I view CD-ROM-based learning,
while using CD-ROM.
although also electronic in nature, as
ÿ It is hard to manage and support
more of a precursor to net-enabled
individual learning experiences of
learning. There are a number of
learners who participate in CD-
Michael Allenʼs 2008 e-Learning Annual Copyright © 200887 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
The Business Impact of e-Learning
287
of these forms or solutions leverage
ROM-based learning. ÿ CD-ROM-based learning provides
the net for learning purposes.
learning content on static medium
The overall goal of learning
that is place-dependent, although
should be an enhancement of human
the learning is more dynamic than
performance, and each of these net-
learning content in a book.
enabled learning solutions should have an ultimate impact on business performance. Donald Kirkpatrick
In my definition for e-learning in business, I include the different forms
developed a framework for evaluation
or solutions of e-learning, such as
of learning about forty years ago
online training and online education,
(Phillips & Pulliam, 2000). Figure 1 shows a Learning
online collaboration, electronic performance, knowledge management,
Terminology Framework that captures
and online learning management. All
my definition of e-learning.
Learning Technology Framework Education
Training
Knowledge
Skills Learning
e-Learning = Net enabled learning targeted to achieve business goals
Knowledge Management
O n li n e
Electronic Performance Support
C
ol
Online Education
n
Online Learning Management
la b o r a ti o
Online Training
Figure 1. Learning Technology Framework
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
288
Michael Allenʼs 2008 e-Learning Annual
What Causes Organizations to Invest in e-Learning?
Organizations
leadership and organizational drivers.
spend an
Business drivers that are most relevant
estimate of $190
to investments in e-learning include:
billion annually
ÿ Enterprise transformation,
on workplace
ÿ Cost reduction,
learning and performance, according
ÿ Time-to-competence, and
to a 2006 ASTD Trend Report.
ÿ People commitment.
Significant investments are made in e-learning. In this section, I will
Enterprise Transformation.
discuss four reasons why companies
Enterprises are experiencing
are investing in e-learning.
increasing pressure to create
First, business drivers are important
shareholder value and therefore need
to investment decisions in e-learning
to transform themselves to achieve
capabilities. Business drivers can
extraordinary goals (Cheese &
be described as those goals, events,
Thomas, 2003; Moore, 2002; Yip 2003).
or issues that urge or compel the
Cost Reduction is a business driver
enterprise or business to move onward
that leads to competitive advantage for
(Webster’s, 2004). Business drivers
enterprises. e-Learning can play a role
are instrumental in business growth,
in reducing the costs for enterprise-
improved competitiveness, and
wide learning.
profitability and increased value of the business (Barksdale, 2002). Barksdale (2002) differentiates
The speed at which enterprises can transfer knowledge and build new capabilities is an important business
between external and internal
performance indicator, according to
business drivers. External business
Arevolo and Lundy (2003). Time-to-
drivers are generally outside the
competence can be increased by using
organization’s control and include
e-learning to support achievement of
economic drivers, human resources
specific business goals.
drivers, government drivers, public
Retaining knowledgeable workers is
perception drivers, and market or
an important competitive advantage
customer drivers. Internal drivers
for enterprises and people commitment
are generated by factors and events
plays an important role in this
within the organization’s control and
area. e-Learning can play a role in
include technology drivers, changes in
maximizing people commitment
systems, processes and key policies,
through the enhancement of learning
shareholder or financial drivers, and
opportunities.
Michael Allenʼs 2008 e-Learning Annual Copyright © 200887 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
The Business Impact of e-Learning
Second, there is evidence that
289
ÿ Time-intensive training needs
business drivers are relevant for
analysis process. To remedy this, a
organizations of different industries,
part of training needs analysis can
geographies, and size. Competition
take place over the Internet, for
for many industries and organizations
example, by using automated skill-
is global in nature, and therefore,
gap assessment tools.
I believe that e-learning may help
ÿ Limited number of training sessions
enterprises to achieve a greater degree
offered. Classroom training
of globalization by developing similar
sessions can only be scheduled
competencies simultaneously in
at a certain frequency, and it is
different geographies with speed and
unlikely that the same training
at lower cost.
sessions are offered on a daily
Third, companies must develop
basis. Due to limitations of group
skills to support critical business
size for classroom training,
goals, while they do not necessarily
employees cannot always attend
have the ability to increase their
the training session that they
education and training budgets. I
desire. As a result, the scheduled
suggest that a competitive advantage
training sessions do not necessarily
might be achieved by executing e-
match the right timing for skill
learning strategies to build employee
development of employees.
competencies faster, better, and
One can argue that often
cheaper. This is consistent with
classroom sessions take place
the results from the ASTD (2006)
either too early or too late. It is
study that shows that expenditure
too early if the skills cannot be
on learning per employee from 1999
practiced and applied immediately
to 2006 has been at best flat for five
after the training session, and it
years, if corrected for inflation.
is too late if the learner has to
Fourth, e-learning can provide opportunities to reduce or take away inefficiencies in the learning business
use skills prior to attending the training program. The Research Institute of
processes, thereby supporting business
America found that three weeks
goals and reducing costs.
after a course is completed, only
The inefficiencies in the learning
18 percent of the knowledge is
business process that I have identified
retained. e-Learning provides
include the following:
learners with the opportunity to develop skills just-in-time because
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
290
Michael Allenʼs 2008 e-Learning Annual
it is available twenty-four hours a
(e.g., Horton, 2001(a); Rosenberg,
day, seven days a week in all time
2001).
zones. ÿ Time away from the job. Each
ÿ Geographic barriers for attending training. Most enterprises use
hour that employees are attending
a (limited) number of training
training or traveling to a training
facilities or conference centers
facility constitutes an hour away
to conduct training. If employees
from the job, which can result
are dispersed over a number of
in lost productivity. According
different locations across the
to Driscoll (2002), e-learning
world, attending training sessions
programs usually take fewer hours
requires significant travel time,
than a classroom event. Reasons
which can be very expensive. This
for this include that the learners
can be a barrier for employees
are in charge of the pace and path
to get approval for attendance.
they use to navigate through the
e-Learning can be taken at any
learning program, and individuals
location providing employees with
are not held up by pace of the
24/7 access to training.
classroom group. Additionally,
ÿ Time-consuming course content
e-learning will eliminate travel
management processes. Training
time to and from a training facility.
programs need to be updated on
Therefore, the consensus among
regular basis. According to an
authors is that e-learning provides
ASTD study (2002), the shelf life
the opportunity to reduce the time
of an average training program
away from the job.
is less than two years. Yet, many
ÿ Course is customized to a group
authors (Carliner, 2002; Horton,
versus the individual. e-Learning
2001(b), 2002) suggest that version
can provide training that is
control and global deployment
customized to the learner’s specific
of courseware can be very time-
needs. For example, by conducting
consuming. One of the benefits
an online pre-assessment, learning
of e-learning courseware is that
gaps can be identified and the
once a course has been updated,
participant can be directed to the
all employees have instant, global
course content that he/she needs
access to the same, most current
in order to master a proficiency
version of a training program.
level in a particular subject area
Michael Allenʼs 2008 e-Learning Annual Copyright © 200887 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
The Business Impact of e-Learning
ÿ Variations in course quality and
291
ÿ Usage of relatively expensive
consistency. Usually classroom
resources for courseware
training material can be modified
development. Classroom
by trainers for deliveries in
training development can be
different geographies. The
expensive because this is done
implications are that different
by local learning professionals or
course versions and qualities
vendors. e-Learning courseware
will exist globally. e-Learning
development can to a large extent
courseware can always be
be accomplished for a relatively
deployed in one version with
low cost in offshore locations, for
consistent quality in the different
example, India.
geographies. ÿ Expensive, time-consuming
ÿ Limited access to training. In most organizations, employees are only
distribution of training material.
allowed to attend a limited number
The coordination, printing, and
of classroom days annually because
logistics of the distribution of
of cost. By comparison, many
classroom training material can
organizations provide employees
be very time-consuming and
with unlimited access to their
expensive. The benefits of e-
entire e-learning catalog because
learning are that all classroom
there are no incremental costs for
material is available online.
attendance in an e-learning course.
ÿ Time-consuming participant
ÿ Collaboration opportunities are
enrollment and administration
limited to people who can attend
processes. Updating employee
training in one location. The
training records, e-learning
collaboration among participants
course enrollment, and e-
in a classroom-based training
learning completion tracking are
model is limited to the classroom
completely automated processes.
attendees. e-Learning provides
Furthermore, there are no resource
an opportunity for people from
needs for classroom scheduling,
multiple geographies to participate
training material logistics, and
in virtual classroom sessions
trainer scheduling in an e-
and collaborate with each other
learning environment, although
without the requirement to be in
synchronous online training still
the same physical location.
requires training scheduling.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
292
Michael Allenʼs 2008 e-Learning Annual
ÿ Limited opportunities to re-take (parts) of a course. Most people
or in e-learning self-study a good alternative.
can take a specific classroom
ÿ Variation in quality in similar
course only once. e-Learning
subject areas. If courses are
courses, or modules of e-learning
developed on a company or
courses, can be re-taken to refresh
geographic basis, there might be a
memory at no incremental costs, at
larger variance in quality because
any time needed.
of investment limitations in course
ÿ Knowledge transfer and
development or due to limited
collaboration stops at the end of
access to experts. e-Learning offers
a course. e-Learning provides the
potential global access to best-in-
opportunity to build communities
class learning content that can be
of practice wherein participants
accessed globally.
can come together after the
management. Compared with
What Appear to Be Critical Success Factors for the Implementation of e-Learning?
classroom training, the Internet
capabilities necessary for e-learning
facilitates knowledge management
success and the factors that support an
by providing the means to
effective e-learning implementation.
training and share knowledge and experience long after the end of a training program (Rosenberg, 2001). ÿ Limited integration with knowledge
As enterprises decide to invest in e-learning, they must understand the organizational
collect and redeploy knowledge
A significant body of literature
more efficiently throughout the
suggests that investment alone in
organization (Geolearning, 2003).
technologies or capabilities does not
ÿ Limited engagement of senior
necessarily provide the anticipated
leadership in training. Due to time
business results. For example,
constraints, it can be a challenge
Pisello and Strassman state about IT
for leadership to allocate time
investment, “It is not just how much
to attend classroom programs.
you spend, but what you invest in and
e-Learning programs require
how well it is managed that counts”
less course time and no travel
(2003, p. 1).
time, as discussed, making their attendance in virtual classrooms
A number of authors have suggested that successful implementation of e-learning requires investments
Michael Allenʼs 2008 e-Learning Annual Copyright © 200887 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
The Business Impact of e-Learning
293
in, and management of, a number
in e-learning to enhance skills and
of organizational capabilities
job performance. A strong learning
(e.g., Horton, 2001b; Masie, 2001;
culture is required for e-learning to
Rosenberg, 2001). The most significant
prosper and be sustained over time.
organizational capabilities for
Unfortunately, the assumptions
e-learning identified include:
and beliefs about e-learning have not
ÿ e-Learning culture,
always been positive among various
ÿ e-Learning design,
learners in many organizations. The
ÿ e-Learning technology, and
resistance expressed to learning on a
ÿ Performance management.
computer is not difficult to understand when we consider that the education
In this section, I will discuss how
experience from kindergarten through
these organizational capabilities can
college until now has always involved
contribute to an effective e-learning
a teacher and a physical classroom.
implementation. Additionally, I will
Sullivan explains that classroom
identify a number of dimensions
training is deeply embedded in the
within organizational capabilities that
corporate educational culture in most
appear to be critical success factors for
organizations “. . . even if we hate
e-learning.
being in a classroom, it’s familiar and not intimidating” (2002, p. 1).
e-Learning Culture A very important
Sullivan states further that, “They
enabler of e-learning is the existence
[employees] feel reassured in the
of an organizational culture that
standard classroom environment and
encourages and provides incentives for
even enjoy going off-site for a day
the adoption and usage of e-learning
to take a course. Off-site training
by its employees. According to Geert
represents a welcome departure from
Hofstede (1991), a culture is a shared
the daily routine, especially if the
set of learned assumptions, values,
training requires travel to a different
and behaviors that influence thoughts,
location” (2002, p. 2). This is one of
feelings, and day-to-day actions.
the reasons why a change from the
Thus, an e-learning culture in an
familiar physical classroom training
organization would support a shared
to the new e-learning environment
positive perspective on the value of
could constitute a major cultural shift
e-learning and the active adoption
(Cross & Dublin, 2002). Therefore,
of the behaviors associated with
gaining positive support for e-learning
engaging frequently and productively
from a number of organizational
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
294
Michael Allenʼs 2008 e-Learning Annual
groups—business leaders, training
of e-learning programs, including
staff, information technology staff, and
whether to build the courseware
learners—is a critical key to success.
in-house, purchase off-the-shelf
In addition, other research
courseware, use an external vendor
reports have indicated the following
to custom-build courseware, or to
organizational supports as critical to
leverage a mix of internal and external
the successful implementation of an
capabilities. Independent of all of
e-learning strategy:
these decisions, the quality of the
ÿ Internal champion and support
e-learning program design retains
from supervisor/manager
importance in and of itself. In fact,
ÿ Time and location for Learning
many authors suggest that e-learning
ÿ Marketing and promotion
design plays a very important role in the overall effectiveness of e-
e-Learning Instructional
learning (Clark & Mayer, 2003; Masie,
Design Another organizational
2002; Mantyla, 2001; Mayer, 2001;
capability is e-learning instructional
Moshinskie, 2000).
design. In the world of e-learning,
The design has a direct influence
content is king. Therefore, e-learning
on the motivation of the learners to
instructional design is central
engage in and complete e-learning
to the success of an e-learning
courses. How can we motivate e-
implementation.
learners to:
This is underscored by results of a meta-analyses conducted by Thomas Russell (2001) of over 355 studies
ÿ Take a course? ÿ Learn the content and develop skills?
since 1928. Russell concludes that the
ÿ Complete the course?
quality of instructional design is the
ÿ Apply knowledge and skills in the
most important factor in the retention
work environment?
of knowledge, rather than the medium (technology or classroom) in which
The answers to these questions
the knowledge is delivered. Schank
are found primarily in the design of
agrees with this and suggests that “e-
the e-learning course. e-Learning
Learning software programs always
instructional designers must focus on
seem to contain major design flaws”
enhancing the extrinsic motivation of
(2005, p. 49).
the learner.
Enterprises must make many sourcing decisions in the development
Additionally, it is important that an e-learning delivery method match
Michael Allenʼs 2008 e-Learning Annual Copyright © 200887 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
The Business Impact of e-Learning
295
Online Learning Continuum People Remember1
More
• Simulations • Games
What they DO What they SAY or WRITE What they HEAR and SEE
• E-courses with audio and video • Recorded live e-learning sessions
• E-coursed with visuals • Online self-study guides • Online PowerPoint presentations • E-mail • E-documents • E-whitepaper
What they SEE What they READ Less
• Interactive like e-classes or e-seminars • Interactive e-courses • E-mentoring or e-coaching
e-reading © Nick van Dam (2003) 1 Edgar Dale (1969)
Low
e-learning
Level of Instructional Design
High
Figure 2. The Online Learning Continuum
different types and levels of learning.
increase as you move further up the
Research results from Edgar Dale
retention pyramid. An online business
(1969) show that people remember
simulation is a powerful learning
more of what they do than what they
tool; however, it is also one of the
read. His learning model was called
most expensive to develop in terms of
The Cone of Dale. I have mapped a
design capability, time, and resources.
broad variety of e-learning delivery
e-Learning design is a very
methods to this learning model and
important capability to support the
termed this The Online Learning
adoption of e-learning in enterprises.
Continuum (Figure 2). The level of desired learning drives many of the
e-Learning Technology The
design decisions. The Online Learning
objective of e-learning is to improve
Continuum indicates some possible
performance through learning, but
delivery methods of e-learning that
the “e” or the technology is the enabler
best support achieving different types
of the learning. Sanders (1998) refers
and levels of learning.
to learning technologies as electronic
As you move along the continuum
technologies that deliver information
from e-reading to e-learning, it is
and facilitate the development of
important to note that the degree of
skills. e-Learning technologies are a
investment in instructional design may
critical organizational capability for e-
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
296
Michael Allenʼs 2008 e-Learning Annual
learning, because learner satisfaction
with the process and content included
is impacted significantly by the type
in the e-learning strategy must be
of e-learning technologies selected,
evaluated and mapped to current
integrated, and supported.
offerings in the marketplace.
Technology factors that drive
It is generally accepted that most
satisfaction with e-learning include:
progressive e-learning strategies
ÿ Appropriate (use of ) technology,
require an open or nonproprietary
ÿ Extent of technology support,
architecture, which allows additional
ÿ Ease of mastery of the technology,
process, content, and technology
and ÿ Speed and responsiveness of the technology.
components to be easily added from multiple vendors with overall ease of integration.
Masie concludes that “the lack
Performance Management
of familiarity or experience with a
Performance management is an
particular learning technology is
important organizational capability
a key obstacle preventing learners
for the implementation of e-
from engaging in e-learning” (2001,
learning. Although performance
p. 26). Furthermore, he recommends
management is a very extensive
that organizations deploy best-of-
subject area, here I will discuss
breed technology and back it up with
its importance related solely to e-
internal support.
learning. Much has been published
Whether an organization chooses
about performance management and
to build the technology backbone
there are different uses of the term
required for its e-learning strategy or
performance management, but the
to outsource the required capabilities
main characteristics of performance
to an outside vendor, it is critical to
management include):
understand the requirements of the
ÿ Appraisal system (including goal
enterprise and to equally understand the capabilities that exist in the marketplace. To build an integrated technology architecture that will optimally support an organization’s e-learning strategy, the technical functionality requirements associated
setting), ÿ Reward system (including compensation), ÿ Assessment and feedback (e.g., 360-degree programs), and ÿ Communication process between employee and employer.
Michael Allenʼs 2008 e-Learning Annual Copyright © 200887 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
The Business Impact of e-Learning
297
Thus, the working definition of
Many studies suggest that high
performance management employed
performers are able to produce high-
here is that put forth by Jon S. Bailey
value accomplishments because they
at the Florida Association for Behavior
have the required competencies and
Analysis, namely the systematic,
they know how to do things more
data-oriented approach to managing
productively than others. For e-
people at work relying on positive
learning to contribute significantly to
reinforcement as the primary factor in
performance improvement, it must
maximizing performance.
develop these requisite competencies.
The success of an organization
Results from a Masie study present
depends on the ability of each of its
evidence that the acceptance of e-
employees to effectively perform and
learning is mostly influenced if the
meet the identified goals. Therefore,
skills taught in an e-learning course
organizations are currently engaging
are valuable to the learner (Masie,
learners in e-learning to support the
2001). Therefore, I argue that, if
achievement of individual performance
organizations used competency-
goals, contributing ultimately to
based performance management
organizational performance, and
systems, and if e-learning solutions
helping the organization to achieve its
were mapped to competencies, then
business goals. Spencer and Spencer
the e-learning program start and
(1993) differentiate two types of
completion rates would increase
performance management systems:
because I anticipate that e-learning
ÿ Performance-based, which focus
courses would be more valuable for
on the what of behavior, and
the learner.
ÿ Competency-based, which focus
Finally, one of the goals of
on the how of performance.
performance management is “to reinforce and reward effective
A competency-based performance
behavior and progress towards
management system is more
goals” (Spencer & Spencer, 1993,
oriented to the future and focused
p. 265). Rewards for performance
on development. Mixed performance
improvement and acquisition of the
management systems combine
desired competencies are another
planning, management, and appraisal
organizational capability critical
of both performance results and
to the success of an e-learning
competency behaviors.
implementation. Organizations
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
298
Michael Allenʼs 2008 e-Learning Annual
can improve performance by
rating for an employee. This would
rewarding employees’ e-learning
enhance the extrinsic motivation to
efforts. Employees may also find
engage in e-learning programs.
that acquisition of new skills
Effective e-learning
and knowledge can lead to more
implementations should ensure that
exciting and rewarding careers.
people are able and motivated to
Personal development plans are
participate in e-learning programs,
important instruments to support
changing personal behavior and having
this. Many studies suggest that
an impact on job performance. If this
effective performance management
does not happen, the organization will
also links rewards to performance.
not benefit from e-learning solutions,
The completion of a number of
and the investments might not provide
required e-learning programs should
an adequate return on investment.
be considered in the performance appraisal when a manager has to
organizational capabilities required for
determine an overall performance
making e-learning a success.
a
• Effective Course Development: - Training Goals - Learner Characteristics - Organizational Environment • e-Learning Industrial - Design Methodology
Implementation of e-Learning
ng rni a e
olo
• Technology Architecture • Selection Process • Maintenance and Support • Vendor Management
Effective e-Learning Implementation
nin ar
g M a na g em
t en
e-L e
e
ng Cult arni ur e -L
• Support From: - Business Leaders - Training Staff - IT Department • Motivated Learners • Time for Learning • Appropriate Location • Marketing and Promotion
Techn
gy
e-L e
D ign
Capabilities for
g Program
es
Organizational
in rn
e-L
Figure 3:
e
Figure 3 summarizes the
• Competency Framework • Learning Curricula • Certification Programs • Incentives and Rewards for Participation and Completion
Michael Allenʼs 2008 e-Learning Annual Copyright © 200887 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
The Business Impact of e-Learning
Which Elements Can Make e-Learning an Effective Business Practice?
299
First, launching
for participation; collaboration and
e-learning
feedback; e-learning culture; e-
within
learning design; e-learning technology;
enterprises
and performance management.
requires
Effective e-learning
investments
implementations must ensure that
that have to
people are able and motivated to
be justified. It has been discussed
participate in e-learning programs,
that business drivers might generate
change personal behavior, and have
investments in e-learning. Business
an impact on job performance. If this
drivers that focus primarily on
does not happen, the organization
knowledge and people and appear
will not benefit from e-learning
to be the most relevant for potential
solutions, and the investments might
e-learning investments include
not provide an adequate return on
enterprise transformation, cost
investment (Cross & Dublin, 2002).
reduction, time-to-competence, and
However, it has been suggested that,
people commitment.
even if organizational capabilities
Investing in technology or
for e-learning are solidly in place in
capabilities alone may not be
organizations and e-learning has been
sufficient. A number of authors
implemented effectively, it is still
have suggested that a successful
not certain that e-learning will have
implementation of e-learning requires
an impact on the business. Further,
investments in, and management of, a
whatever the impact is, it may not be
number of organizational capabilities.
measured in a way that demonstrates
The most significant dimensions of
its value to the business.
organizational capabilities for e-
One of the criteria identified
learning that have been identified by
for evaluating the impact of the
a number of authors include quality
learning is to look at efficiency and
of e-learning; leadership sponsorship;
effectiveness of learning initiatives
technology orientation; assessments
and the learning function. Keuning
and testing; change management;
(2003) suggests that organizations are
learning management system;
effective if they achieve multiple goals.
technology support; personalized
One can say that learning initiatives
content; user friendliness; linkage
are also effective if they support the
to performance management;
achievement of specific business goals.
competency management; incentives
It has been argued that e-learning
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
300
Michael Allenʼs 2008 e-Learning Annual
needs to be aligned with specific
ÿ Dimensions of Organizational
business goals, and the achievement
Capabilities—which can support
of those goals needs to be measured.
e-learning; and
Furthermore, learning initiatives
ÿ Measurement Metrics—which can
are considered efficient if skills and
show the business impact of the
resources are used economically
e-learning initiatives.
(Keuning, 2003). Learning evaluation frameworks have been available for a while. Donald Kirkpatrick is one of the first authors who developed an evaluation
What Is the Business Impact of e-Learning?
As mentioned at the beginning of this article, enterprises around
framework for the training industry, back in 1959. Philips extended the
the world have made significant
Kirkpatrick evaluation model with
investments in e-learning over the last
another evaluation level for measuring
five years. Market evaluators forecast
return on investment (ROI). Daniel
that the e-learning market will
Kirkpatrick’s Four-Level Model
experience significant growth for the
(Kirkpatrick, 1959) of training
coming years.
evaluation, supplemented with Jack
Literature and research results
Philips’ Level 5 (Philips, 1997), has
show that the business impact of e-
dominated the training industry
learning can be comprised of several
when assessing the value of training
components: improved company
initiatives (Active Education, 2003).
earnings, enhanced workforce
However, various sources indicate a
productivity, and enhanced company
very large percentage of companies
revenue. Table 1 indicates the business
have not yet started to measure
impact of e-learning.
the business impact of training and
Company earnings may be increased
development initiatives (e.g., ASTD,
by reducing overall learning costs.
2006).
Potential cost savings may include the
In conclusion, elements that can
following learning cost categories:
make e-learning an effective business
training needs assessments, design
practice include:
and development, acquisition of
ÿ Business Drivers—which can
training programs, learning delivery,
generate investments in e-learning; ÿ Business Goals—which can be
evaluation, travel, and learning function (e.g., Philips, 2001).
supported by e-learning initiatives; Michael Allenʼs 2008 e-Learning Annual Copyright © 200887 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
The Business Impact of e-Learning
301
e-Learning Contribution
Business Impact Improve company earnings
•
Reducing overall learning costs
Enhance workforce productivity
•
Reducing training time
•
Reducing time-to-competence
•
Enhancing knowledge base and skills of people
Enhance company revenue
•
Efficiently trained sales force
•
Selling e-learning courseware to
Table 1. The
customers and partners
Business Impact of e-Learning
Company revenue can be enhanced by providing the sales force with e-learning; because e-learning is
both company earnings and revenue as well. Finally, although most companies
apparently less time-consuming, it
indicate that e-learning has had the
therefore allows sales personnel to
business impact described above, the
spend more time with customers.
hard measurement of the business
Additionally, companies can sell e-
impact of e-learning is largely limited
learning courseware to customers,
to the reduction of overall learning
suppliers, and partners and turn
costs. The financial metrics used
e-learning into a direct revenue
by most companies in my research
generator.
include both cost reduction and cost
Also, the performance of the
avoidance. Only a few companies
workforce can be enhanced by
measure any form of return on
providing people around the world
investment or measure the impact of
with e-learning opportunities that
e-learning on shareholder value.
are more efficient and effective for
In conclusion, since the inception of
the learner. For example, e-learning
e-learning, most discussions have been
provides people with any-time access
focused on e-learning technologies
to courseware. Therefore, people can
and how technology could support
acquire specific knowledge or develop
learning in enterprises. Recently, the
certain skills at the time they need
dialogue about e-learning has matured
it. Additionally, the enhancement of
and shifted to a strategic focus on
workforce productivity may enhance
the impact e-learning can have on an
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
302
Michael Allenʼs 2008 e-Learning Annual
enterprise through support of strategic business goals, such as enterprise transformation and decreasing timeto-competence, with the overall objective of creating a competitive advantage and shareholder value.
Michael Allenʼs 2008 e-Learning Annual Copyright © 200887 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
The Business Impact of e-Learning REFERENCES Active Education Learning Solutions Group. (2003). Unlocking the secrets of ROI. Golden, CO: Author. Arevolo, W., & Lundy, J. (2003). KW predicts learning deployment will expend in 2004. Boston, MA: Gartner. ASTD. (2002). The state of the industry. Alexandria, VA: Author. ASTD. (2006). The state of the industry. Alexandria, VA: Author. Barksdale, S. (2002). Rapid strategic planning. Alexandria VA: ASTD. Broadbent, B. (2002). ABCs of e-learning: Reaping the benefits and avoiding the pitfalls. San Francisco, CA: Pfeiffer. Carliner, S. (2002). Designing e-learning. Alexandria VA: ASTD. Cheese, P., & Thomas, R.J. (2003). How to boost your workforce performance ROI. Boston , MA: Accenture. Clark, R., & Mayer, R.E. (2003). e-Learning and the science of instruction. San Francisco, CA: Pfeiffer. Cross, J., & Dublin, L. (2002). Implementing e-learning. Alexandria, VA: ASTD. Dale, E. (1969). Audio-visual methods in teaching. Albany, NY: International Thomson Publishing. Driscoll, M. (2002). Web-based training (2nd ed.). San Francisco, CA: Pfeiffer. ELITE. (2000). Putting critical chain learning online. New York: Vuepoint. Hofstede, G.H. (1991). Cultures and organizations: Software of the mind. New York: McGraw-Hill. Horton, W. (2001)(a). Evaluating e-learning. Alexandria, VA: ASTD. Horton, W. (2001)(b). Leading e-learning. Alexandria, VA: ASTD. Horton, W. (2002). Using e-learning. Alexandria, VA: ASTD. Furchtgott-Roth, D. (2004), The challenges of the 21st century workforce. Washington, DC: The Hudson Institute. Keuning, D. (2003). Management: A European perspective. Houten, The Netherlands: Wolters-Noordhoff. Kirkpatrick, D. (1996). Evaluating training programs: The four levels. San Francisco, CA: Berrett-Koehler. Mantyla, K. (2001). Blending e-learning. Alexandria, VA: ASTD. Masie, A. (2002, May). The e-learning marketplace survey. Saratoga Springs, NY: The Masie Center. Mayer, R.E. (2001). Multimedia learning. New York: Cambridge University.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
303
304
Michael Allenʼs 2008 e-Learning Annual
Moore, G.A. (2002). Living on the fault line. New York: Harper Business. Moshinskie, J. (2000). e-Learning made e-z workshop. Waco, TX: Baylor University. Phillips, J.J. (1997). Return on investment. Woburn MA: ButterworthHeinemann. Phillips, J.J. (2001). Implementing e-learning solutions. Alexandria, VA: ASTD. Phillips, J.J., & Pulliam, P.F. (2000). Level 5 evaluation: Mastering ROI. Alexandria VA: ASTD. Pisello, T., & Strassmann, P. (2003). Maximizing the ROI from IT investments. New Canaan, CT: The Information Economics Press. Rosenberg, M.J. (2001). e-Learning strategies for delivering knowledge in the digital age. New York: McGraw-Hill. Russell, T. (2001). The no significant difference phenomenon. Available: http:// teleeducation.nb.ca/nosignificantdifference. Sanders, K. (2001). Fast path to success with Centra. Lexington, MA: Centra. Sanders, S. (1998). Learning technologies. Alexandria, VA: ASTD. Schank, R. (2005). Lessons in learning, e-learning, and training. San Francisco, CA: Pfeiffer. Spencer, L.M., & Spencer, S.M. (1993). Competence at work. Hoboken, NJ: John Wiley & Sons. Sullivan, C. (2002). Getting the organization to adopt e-learning. Salem, MA: Hawthorne Associates. Available at: HawthorneWhitePaper_Mktg_Elearning_ April2002.pdf Urdan, T.A., & Weggen C. (2000). Corporate e-learning: Exploring a new frontier. San Francisco, CA: WR Hambrechts + Co. Yip, G.S. (2003). Total global strategy II. Upper Saddle River, NJ: Prentice Hall.
Michael Allenʼs 2008 e-Learning Annual Copyright © 200887 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Allen Avner Allen Avner is co-principal of Avner Associates, a firm specializing in evaluation of technological approaches to increasing worker and workplace effectiveness. He is also principal research scientist, emeritus, at the University of Illinois at Urbana-Champaign. From 1967 to 1993 he was responsible for evaluation at the ComputerBased Education Research Laboratory (CERL) at the University of Illinois. Between 1982 and 1992 he was editor of the Journal of Computer-Based Instruction. He was earlier the chief of the Experimental Psychology Branch of the U.S. Army Aeromedical Research Laboratory, a staff member of the University of Illinois Bureau of Educational Research, and a staff member of the University of Illinois Committee on School Mathematics (UICSM).
Lessons Learned . . . and Relearned
Allen Avner
Learning and the New Technologies Drawing from observation of the e-learning production efforts of over four hundred instructional design teams during the past four decades, the author illustrates eleven commonly seen “lessons learned” with brief casestudy examples. All eleven lessons were the result either of failure to make use of experience from outside the e-learning environment or failure to adapt to the constantly changing capabilities and needs of students. The article concludes with three general rules for designers of e-learning. The date of the first intentional
expensive resource to provide a single
application of computers for e-learning
person with a timed and scored drill on
will probably never be known—because
conversion of hexadecimal to decimal
it was initially a clandestine activity.
numbers was difficult to justify.
Early computer systems were built at great expense for the solution of important and otherwise intractable problems. Yet, if my experience with four different mainframe computer systems in the 1950s was typical, it
Early ComputerMediated e-Learning
The cost barrier to use of computers was broken in the late 1950s by development of
always seemed possible to find someone
techniques for sharing access to a single
eager to demonstrate a computerized
computer among many users. The first
game or instructional drill program.
public demonstration of time-shared
Most were considered programming
computing was made at the University
novelties, shared in secret between
of Illinois Urbana-Champaign
friends and run with caution in brief
campus in 1960 by a group at its
moments stolen from code-check and
interdisciplinary Coordinated Science
maintenance time.
Laboratory (CSL). The CSL PLATO™
There was good reason for
project, begun as an effort to find
considering these early efforts as
cost-effective ways to use computers in
novelties. Real-time communication
education, was able to take advantage
with a computer through punched
of the extensive expertise available at
paper tape, programming switches, and
CSL in both hardware and software
Nixie™ tube displays was difficult. More
development. It was the start of a
important, use of a limited and very
multi-decade effort that provided
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
308
Michael Allenʼs 2008 e-Learning Annual
researchers at Illinois with equipment
CERL. In 1961 it was using computer-
that was often several years ahead of
generated graphics and in 1971 began
that available to groups dependent on
use of plasma displays invented at
commercially available hardware and
CERL. By 1975 CERL had produced
software. In 1967 the PLATO project
a microprocessor terminal to replace
was split off from CSL and became
earlier “dumb” terminals and in 1980 it
the main focus of the new Computer-
was using a microcomputer classroom
Based Education Research Laboratory
to teach a university-accredited
(CERL).
course.
Development of hardware designed
In 1987 PLATO was replaced by
to reduce costs of delivery of
NovaNET™, a low-cost central system.
educational material was continued by
When CERL was closed in 1993 after its operations had been spun off to commercial concerns, it was supporting a total of 3,390 dedicated terminals in addition to unknown numbers of personal computers accessing it by dial-up or Internet. Shared, centralized computing was used by PLATO during its
Figure 1. Early PLATO Terminal
early period solely to reduce the cost of computing. But shared computing demanded real-time communication between terminals that were ultimately distributed worldwide. The presence of real-time communication between terminals had an immediate, unanticipated outcome. PLATO became not only an instructional delivery medium, but also a communications medium. By the 1970s PLATO had features that would be recognized today as versions of instant messaging, message
Figure 3. PLATO III Mercy Hospital Nursing
boards, chat rooms, blogs, and online
School, 1970
support—with distant support staff
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Leasons Learned . . . and Relearned
309
able to view the client’s display while
material, of which about 6,107 student
interacting by instant messaging. Also
hours’ worth had been validated by
present were software innovations
use in accredited instruction. Over a
used in the presentation, management,
thirty-two-year period, PLATO and
and evaluation of instruction.
NovaNET had delivered a combined total of over twenty-seven million contact hours of service.
The Quest for Efficient Learning
One of our oldest technologies is that which facilitates
learning—the collection of techniques that makes it possible for a person to learn without having to independently Figure 2. PLATO V Terminal, 1975
rediscover everything. It is a technology that is basic to advancing a civilization.
Of greatest interest for this article
Communication is a necessary
is the fact that CERL was the home
part of facilitating learning. Hence,
of the multi-decade production
improvements in communications
and use of e-learning materials for
technology often lead to improvements
accredited instruction. Designers from
in the facilitation of learning.
academic, government, and business
Although imitation of the actions
organizations produced materials for
of someone proficient in a skill can
use by students from pre-school age
permit learning in the absence of a
and up in courses taught in elementary
common language, a common spoken
and secondary schools, community
language greatly increases the chances
colleges, universities, medical schools,
that nuances of technique will be
prisons, small businesses, government
communicated. When the spoken
service organizations, industry, and
language is also available as a written
military training sites. By 1993 over
language, we add the capability of
four hundred individual designers and
learning in the absence of a proficient
design teams had produced some 9,371
model. “Distance learning” and “self-
student hours’ worth of instructional
paced learning” thus began thousands
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
310
Michael Allenʼs 2008 e-Learning Annual
of years ago with the first use of
approaches was that all information
written language to communicate
seen by the student was prepared
information.
in advance, and progress through
The invention of moveable type
the materials was prevented unless
made it more economical to distribute
interspersed questions were correctly
the written word. Sound and video
answered. These approaches generally
recording allowed the addition of
increased the quality of learning, since
information best conveyed by sound or
fewer students could complete the
moving images.
materials without actively processing
However, no matter how realistic
the information. But their production
and complete the information,
and maintenance was more expensive
exposure alone does not necessarily
than simple text materials.
lead to learning. The learner may
Invention and development of the
not be motivated to attend to the
programmable computer opened
information, or may fail to understand
new options. While providing all
a crucial point. Through the ages it
of the capabilities of the electro-
has been observed that including a
mechanical teaching machine, it
skilled human tutor in the learning
added the possibility of interactions
process can, through frequent direct
guided by “artificial intelligence” that
interaction with a student, provide
more closely mimicked the actions
both motivation and targeted
of a human tutor, the possibility
assistance. But the cost of providing
of generating content on the spot
every student with a skilled tutor is
through use of rules programmed
rarely feasible.
into the computer, and the possibility
Attempts to force frequent direct interaction of the student with
of easy storage and processing of response and test data.
material to be learned in the absence of a human tutor began centuries ago with structuring of instructional
Lessons Learned
This article is based on an
content into short segments
examination of
interspersed with review questions.
the unpublished
It continued into the 20th Century
internal
with various forms of “programmed
evaluation reports generated for many
text” and mechanical and electro-
of the PLATO design teams as part
mechanical “teaching machines.”
of their efforts to produce effective
A common element in most of these
e-learning materials (see Avner, 1979,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Leasons Learned . . . and Relearned
for a published description of similar studies). Some teams had life expectancies
311
Designers who avoided major design problems tended to seek information on related types of learning rather
measured in months, while others
than assume that they were in a novel
were tracked for several decades.
situation in which nothing was known.
Some were made up of members
Where information was truly lacking,
with minimal instructional and
they tended to use early prototyping to
media experience, while others had
explore the effectiveness of alternative
years of experience in instructional
approaches before committing full
design in a variety of media. All
resources.
faced innumerable problems, and
Designers who were significantly
most learned from overcoming these
hindered by major design problems
problems. If we examine the details
tended either to assume that
of the sorts of problems encountered,
e-learning was a totally new
two points stand out:
environment for which no existing
1. There were no “universal
experience was relevant or to
problems” encountered by every
overestimate the generalizability of
designer. For every instance of
approaches that were successful in
“learning” in which one or more
other settings. At one extreme was
designers new to e-learning
the failure to make use of existing
encountered a major problem
experience with other media, with
requiring them to alter their future
the result that time was wasted in
design approaches, it was possible
independent re-discovery. At the other
to find at least one other designer
extreme was the over-generalization
with similar limited e-learning
of proven techniques without
experience who, faced for the first
understanding the need to fine-tune
time with a very similar situation,
them to new and changing situations
produced an effective design that
to ensure effective learning.
worked with minimal fine-tuning.
Eleven misconceptions or
2. No problems were unique to e-
inappropriate approaches accounted
learning. Even technical issues like
for major amounts of lost time, effort,
the rate and location of display
and effectiveness. Here are the lessons
writing or the variability and
learned:
duration of key echoing had been addressed earlier in fields outside of e-learning. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
312
Michael Allenʼs 2008 e-Learning Annual
Lesson 1: e-Learning Is a
instruction are, at best, an indication
Medium, Not a Treatment All
that it is possible to use e-learning
communications media provide
effectively. They are not guarantees
opportunities for distributing and
that e-learning will automatically be
presenting instructional materials.
effective. A medium as flexible as
Early users of computers in direct
e-learning allows the inexperienced
instruction saw the possibility that the
designer the opportunity to produce
computer could provide:
truly wretched material.
1. Time savings for students (a potential outcome of all self-paced
Lesson 2: Costs Count Reduction of
learning approaches),
the cost of communicating information
2. Time and money savings for
is one of the major contributions of
organizations (a potential outcome
technology to learning. Materials that
of all distance learning approaches
can be distributed at substantially
that reach students who are
lower cost provide an opportunity
geographically dispersed),
for more people to have direct
3. Individualized control of learner-
access to information. Increasing the
material interaction approaching
number of people who have access to
that which is possible with one-on-
information is a gain, even if the new
one human tutoring, and
technology is used only to reproduce
4. More efficient production and
materials already available. Advances
processing of drill and test
in printing technology over the past
materials.
six hundred years have led to the potential low-cost print-on-demand
But these are only potentials. Just
availability of virtually any book that
as media such as the printed word,
has ever been in print. The widespread
photography, sound recording,
availability of low-cost video recording
movies, radio, and television provide
media and playback equipment allows
genuine opportunities for enhancing
personal access to existing educational
instruction, the potential requires
motion picture and TV programs.
effective content and presentation
Even if the video content has not
to exploit its promise. Studies that
been changed, students with the
purport to show that computer
modern hardware can now choose a
mediated e-learning is more
convenient time to view the material
effective than alternative forms of
and can halt or review as many times
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Leasons Learned . . . and Relearned
313
as needed—turning what was once
with students on a one-to-one
expensive group-paced learning into
basis. This included teachers
inexpensive self-paced learning.
of early elementary grades,
A student population that is highly
teachers of music, aircraft flight
motivated and that has a background
instructors, those with tutoring or
appropriate to the printed or video
special education experience, and
material may not benefit from the
classroom teachers at all levels who
added work needed to turn a printed
made extensive use of student-
textbook into a programmed text or
teacher interaction as part of their
a videotaped lecture into a forced
teaching style. Such designers had
sequence of interactions. For that
a head start in knowing how to
reason you should always be aware of
design interactions with students
the true cost and effectiveness of all
that elicited the true state of the
reasonable alternatives. An approach
student’s understanding.
that is slightly less effective than the
2. More effective materials tended to
most effective alternative may be the
be produced by designers who had
best choice if it costs one tenth as
taught the subject matter for five
much.
or more years and who had taught students at both a more elementary
Lesson 3: Experience
and a more advanced level than the
Counts Instructional design and
current material. Such designers
production is labor-intensive. It is
had a better feel for structuring
tempting to cut costs by employing
the design to capture and correct
workers with limited experience.
possible misconceptions picked
But does this pay in the long run?
up in earlier classes and a better
Instructional designers new to e-
understanding of what conceptual
learning, but from a wide range of
simplifications could be made
disciplines and with a wide range of
without harming later learning.
levels of experience, were observed during their first two years of use of
Although technologies such
the medium. The effects of their prior
as e-learning aid the transfer of
experience were evident.
information, skills, and concepts
1. More effective materials tended
to learners, the transfer itself is
to be produced more efficiently
the crucial process. It should be
by designers who had a year or
no surprise that those who have
more of experience in dealing
experience in facilitating learning by
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
314
Michael Allenʼs 2008 e-Learning Annual
individuals will be able to make use
learning numeric skills. It repeatedly
of that experience, even with a novel
presented a subset of items in random
medium.
order, automatically replacing items learned to criterion with new items.
Lesson 4: Your Ability to Judge
The approach seemed like an excellent
the Potential Effectiveness of
aid for medical students faced with
an e-Learning Module Is More
learning numerous anatomical terms.
Limited Than You Think Transfer
Suitably modified, it was presented
of experience from other settings has
to a class of medical students. The
its limits. Suppose that you examine
results were a near disaster, with the
an e-learning module that has no
medical students rebelling against the
obvious errors in content or egregious
program. An approach that was a good
flaws in handling student responses.
fit for students with low motivation
How accurately do you think you can
and poor study skills was accurately
judge its potential effectiveness for
seen by highly motivated students with
teaching its content? If you are an
excellent study skills as holding back
experienced teacher in the subject
their progress.
matter covered by the module, and if
Case 2. A curriculum group
you are very familiar with a specific
underwent a reorganization in which
student population, you will probably
the titles of all of its instructional
be able to make an accurate estimate
modules were renamed. Confusion over
of how effective the module will be for
the new module titles led to one section
that student population. But for any
of a class being assigned an early draft
other population or any other subject
of a module instead of the polished
matter, you will almost certainly be on
final version. Students quickly noticed
shaky ground.
misspellings, ambiguities, and logical
Material that you might consider
errors. The panicked instructor tried to
to be boring can be an exact match to
make a teaching moment of the flawed
the needs of some student population.
module, asking students to record all of
Material that may appear to be
the errors they found and refer to their
exciting, innovative, and a sure-fire
textbooks for appropriate corrections.
path to insightful learning can be
Usually, most students completed a
a dismal failure if presented to the
module before half of the scheduled
wrong population.
class period was over, but this section
Case 1. A drill program showed great success for elementary school students
took the entire period to complete the flawed module. The source of the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Leasons Learned . . . and Relearned
315
confusion was soon identified, and no
be using a new production team that
other section received the draft module.
has no track record? Can you trust a
But what damage had been done to
generic formula to give trustworthy
the section that had to do battle with
predictions? Unfortunately, there
the draft version? Amazingly, when
is good reason to doubt that a valid
tested on their learning of the material
predictor of anything but an upper
in the flawed module, this section
bound on effort is possible.
scored significantly higher than all
Many prediction instruments
other sections on both initial tests
have been developed. The predictors
and a retention test, even though their
typically involve factors such as the
scores on material in other modules
instructional style (e.g., tutorial, drill
were not significantly different from
and practice, simulation), age and
those of other sections. Given the right
educational level of the students,
conditions, even material that most
cognitive level of instruction,
would reject as unacceptable might
experience of the designers, and so
produce effective learning.
forth.
The curriculum design group in
The three best predictors of
Case 2 gained more than one insight
production time found in studies of
from this experience. In addition to
several hundred production teams
being more careful in titling modules,
associated with the University of
they began adding formal “find the
Illinois effort were:
error” practice and test items to all
1. Deadline Time—In practice,
modules.
production takes as much time as is available. An experienced team
Lesson 5: There Is No Valid
or individual under a tight deadline
General Method for Predicting
will use different approaches and
the Time Needed to Produce
standards than might be used
an e-Learning Module Any
under lax limits.
commercial production team that has
2. Designer Experience—A designer
survived for more than a few years
experienced in a particular area
is able to give reasonably accurate
will be better able to select and
predictions of the amount of time it
implement effective approaches
will take them to produce material
while avoiding ineffective
of a familiar type. But what if you
approaches.
are faced with production of an unfamiliar type of material or will
3. Tool Availability—An experienced designer will have developed
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
316
Michael Allenʼs 2008 e-Learning Annual
production tools that facilitate
points in predicting the effort required
production of materials of the
for producing a wide variety of types of
types most often used by them.
modules. What he didn’t realize was
Personal production tools may be
that his production team knew about
as simple as an existing module
the formula and used that information
that is used as a template for future
in their reports to him. If a particular
modules or as complex as a library
module was produced in less time than
of computer routines.
predicted, the “saved time” was set aside. If a module took more time than
The problem is that the second
predicted, some of the “saved time” was
and third of these predictors are
used to reduce the time reported for
difficult to quantify, and any one of the
its production. As the production team
three can have a massive impact on
gained experience and added tools that
production time.
aided the efficiency of their production,
Most generic predictors of
they had less and less need to resort
programming effort gain their
to such time shifting in their reports.
credibility by setting generous
Their work filled the time made
estimates that almost any team can
available by the prediction formula,
meet. Perhaps they do a service by
and their gains in efficiency showed
providing a worst-case estimate of
up as a reduction in time pressure on
startup costs. But that worst-case
the team, rather than as an increase in
estimate should immediately be
productivity.
modified as experience is gained during initial production—always
Lesson 6: Be Cautious of
keeping in mind that a deadline may
Rigidly Structured Approaches
be a self-fulfilling prophecy.
to Instructional Design and
Case 3: A director of instruction
Production The inexperienced
was a firm proponent of a prediction
designer has available a number of
formula developed by a government
models for instructional design and
agency. Although intended for use only
production. Such production models
in the absence of local experience, the
guide the designer or design team
formula had been used for several
through a detailed sequential process
years in scheduling work for his in-
of planning, design, production,
house production team. He had data
and testing. By making one aware of
showing that the formula seldom
techniques found valuable in design of
missed by more than a few percentage
effective instruction, such approaches
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Leasons Learned . . . and Relearned
317
can be an excellent teaching tool for
materials in an efficient manner, but
those inexperienced in instructional
the quality and efficiency probably had
design. But these methods need to be
little to do with the “systems approach,”
applied with caution.
nor were the forms completed in the
Case 4: A new production team
manner expected by the authors of that
made up of graduate students with
approach. The staff ’s procedure was to
little teaching or instructional design
quickly produce a near-final product
experience initially showed promising
by telescoping the planning, design,
growth in design sophistication as a
and production into a single process.
result of following a standard design
Following this, they would do a trial
procedure. Their supervisor was
run using themselves as “students,”
unable to attend sessions in which
make minor adjustments, and then
the initial materials were tested
conduct a final trial with real students.
on undergraduate students, but
After cleaning up the resulting product,
did respond in detail to the reports
they would sit down together and
generated as part of the planning
fill out all of the planning, design,
process. The new production team
production, and testing documentation.
began to spend most of its time in
While the discipline of being forced
planning and little in production and
to attend to requirements for quality
field testing. The production effort was
design can be of enormous benefit
cancelled when it became apparent
to those with little experience, care
that it would not be possible to produce
should be taken that standardized
even one of the modules intended for
production procedures do not become
the following semester.
ends in themselves nor burdens to
Case 5: A government site was
those who have already internalized
required to follow a “systems approach”
the principles upon which they
to production of its courseware. Each
are based. A rigid developmental
step of the process was documented
procedure in which no step can be
by completion of standard printed
begun before all prior steps have
forms. A command inspection would
been completed (the pure “waterfall”
have revealed filing cabinets filled with
model) is rarely practical in real-life
meticulously completed forms.
situations.
The production staff included many experienced instructors who had been
Lesson 7: Early Prototyping—
teaching the same or similar material
Build on Success, Learn from
for decades. They produced excellent
Failure From one viewpoint, “early
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
318
Michael Allenʼs 2008 e-Learning Annual
prototyping” is an alternative to those
materials, including direct observation
varieties of “systems approaches” in
by instructors or designers, paper
which all planning is done before any
tests, and automatically collected
design is begun. An “early prototype”
online measures. The conclusions
combines aspects of planning, design,
listed below were based on analysis of
and evaluation into a single step,
efforts of designers and design teams
much like the procedure described
with less than two years of experience.
above in Case 5. Prototypes of new
Analysis of the efforts of more
approaches are tested as soon as
experienced designers (most of whom
possible and the test results are used
produced high-quality materials) did
in an iterative fashion to improve
not reveal the same clear relationships
the new approach. Early prototyping
between data collection efforts and
can be used as a low-risk step in the
design quality.
gradual introduction of e-learning into
Production efforts that produced
other instruction. If a brief module
poorer quality materials were most
“works,” it can be expanded. If it fails,
often distinguished by the following
it can be modified or augmented. The
characteristics:
best aspect of gradual introduction is
1. Data collection included measures
that it can be done as a part of regular
that had no clear bearing on design
instruction, insuring that information
decisions or treatment of students.
gathered will be representative of that
2. No data were collected during
which would be expected of a more
early testing, or data were gathered
complete implementation.
but not immediately analyzed. 3. Data measures were not assessed
Lesson 8: Look at the Data No matter what approach is used to
for ambiguity or consistency. 4. Data were analyzed at the end of
produce materials, information on
all sessions or analyzed but not
student learning and on the quality
reported to the design team.
of interactions between learners and the material should be collected often
Production efforts that produced
and made available immediately to
better quality materials were most
the designers. This is particularly
often distinguished by the following
true for inexperienced designers. In
characteristics:
the following comparisons, “data”
1. Data collection was limited to
refers to any measure made during the
measures that were related to
course of interaction of students with
specific decisions to be made
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Leasons Learned . . . and Relearned
either in design or in treatment of students. 2. Data were collected from earliest use of materials. 3. Data measures were modified or
319
instructors being supervised? If there is the perception that use of the e-learning materials will result in extra work, reduced income, or reduced power for anyone, you may
augmented as soon as the potential
be in trouble. Even if adopted, there
for ambiguity or inconsistency was
is a strong possibility that use of the
noted.
materials will be sabotaged in any of a
4. Data were analyzed and reported to design teams during or
variety of imaginative ways. We might hope that all would
immediately after each use of the
have a positive view of effective
materials.
e-learning. But that is not always the case. Implementing e-learning
Lesson 9: Even If You Build It,
without thorough
They May Not Come—The Agony
contact with every
Implementing e-learning
of Implementation Assume that
stakeholder (not
without thorough contact with
you have produced an e-learning
just the boss) and a
every stakeholder (not just the
package that is guaranteed to increase
full understanding
boss) and a full understanding
student performance while reducing
of their concerns is
of their concerns is a risky
the number of human instructors
a risky approach.
approach.
needed, thus saving substantially in
Addressing
instructional costs. Further, you have
those concerns effectively from the
identified an environment in which it
beginning can enormously reduce
is sure to be effective. Do you have a
potential problems. Sometimes it
sure sale? Are you guaranteed a set of
is possible to find ways in which
happy users? No.
management can restructure the
Is it to be implemented in a college
organization to meet the needs of all.
to cut down on the need for remedial
Case 6: It was possible to have
classes for incoming freshmen? What
the salary base for the head of an
if the department teaching those
industrial training department
remedial classes depends on these
reclassified to be based on the number
classes as a means of supporting
of trainees processed rather than the
graduate teaching assistants? Is it
number of instructors supervised. In
to be implemented in an industrial
the same organization, several of the
training department where the salary
senior instructors were retained as
of the chief is based on the number of
subject-matter experts for a multi-year
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
320
Michael Allenʼs 2008 e-Learning Annual
design of materials, thus ensuring their
interactive form, that is not its only
employment until retirement, at which
advantage. More important is the
time the remaining instructors would
fact that e-learning makes possible
be able to handle the anticipated
instruction that might not otherwise
reduced human instructional load.
be economically feasible. It is easy to underestimate the
Lesson 10: Simulations Need
extent to which established curricula
Support Simulations of various
are limited to what it is possible to
biological or physical systems were
teach by lecture (or other medium),
an obvious and early application
rather than by what students might
in computer-based instruction.
need. Such limitations can cause a
Mainframe computers had been used
mismatch between what students
for simulations of such systems from
need and what the curriculum can
the beginning and it was easy to
provide. The computer opens another
transfer simplified portions of such
dimension to what can be attempted.
simulations to educational systems.
One obvious change is in the use of
Computerized simulations also
simulations. Computerized simulations
provided impressive demonstrations.
in areas such as aircraft pilot training,
But simulations that are faithful
medical diagnosis and treatment,
to their complex origins can be
and engineering design have enabled
overwhelming to a beginning student.
learning of skills at far lower cost,
Instead of beginning with the most
lower risk, and (potentially) at a higher
realistic version of a real-life system,
degree of mastery than previously
it may be better to start with a
feasible. Simulations rarely totally
simplified version. In simulations that
replace hands-on experience in the
teach diagnostic or trouble-shooting
real environment. But they can bring
skills, most students will benefit from
the student to that real environment
introductory material that covers basic
far better prepared to cope with its
problem-solving techniques.
complexities and far more able to learn from the experience. But note
Lesson 11: e-Learning Can
Lesson 10. Simulations can easily be
Change What Is Learned Rather
overwhelming and unproductive if a
Than How It Is Learned Although
student is dropped into them without
e-learning permits presentation
preparation or real-time support.
of existing instruction in a more
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Leasons Learned . . . and Relearned
Conclusion
321
Of all the
RULE 2. You Do Not Need to Totally
lessons learned
Understand Human Learning to
from designing
Improve Learning
instructional environments
Centuries of accumulated experience in instructional design and
for any medium, the most important
implementation provide an excellent
is the fact that designers must
beginning point for new applications.
themselves be continually learning—
Perfection is not required. With an
both from the experience of others and
understanding of the relative costs of
from the results of their own efforts
alternative media, a good collection of
to facilitate learning. Consider the
alternative instructional approaches,
following rules of e-learning:
and a careful assessment of instructional needs, you should be able
RULE 1. If You Think That You
to make a measurable improvement in
Totally Understand Some Aspect
some aspect of current learning.
of Human Learning, You Need to Collect More Data Our experience is always
RULE 3. Never Forget Rule 1 Identify crucial data, gather it, and
constrained by the types of students
monitor it continually. Even if you are
we encounter, their motivation, the
seeing exemplary learning at minimal
subject matter, and the instructional
costs now, remember that students,
settings we have worked in. Other
subject-matter content, and learning
factors may well be important under
needs change constantly. Anticipate
other conditions. Be cautious in
change, and be ready to deal with it.
assuming that what you know is timeless and can be exported to other settings without unexpected side effects.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
322
Michael Allenʼs 2008 e-Learning Annual REFERENCES Avner, R.A. (1979). Production of computer-based instructional materials. In H.F. O’Neil, Jr. (Ed.), Issues in instructional systems development (pp. 133180). New York: Academic Press.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Will Thalheimer Will Thalheimer is a learning expert, researcher, instructional designer, business strategist, manager, marketer, speaker, writer, and visionary. Dr. Thalheimer has worked in the learning-andperformance field since 1985. He founded Work-Learning Research in 1998 to bridge the gap between research and practice, to compile research on learning, and to disseminate research findings to help instructional designers, e-learning developers, trainers, and performance consultants build more effective learningand-performance solutions. His clients have included giant multinationals, e-learning companies, government agencies, and institutions of higher learning. His research and writings have led the field in providing practical research-based recommendations through his online publications (www.work-learning.com/ catalog), published articles, and his industry-leading blog (www. willatworklearning.com). Dr. Thalheimer speaks regularly at national and international conferences. His conference presentations always receive numerous evaluation-sheet comments like the following: “This was one of the best presentations I attended—solid information delivered in a style that helped me learn.” Will holds a BA from the Pennsylvania State University, an MBA from Drexel University, and a Ph.D. in educational psychology: human learning and cognition from Columbia University.
We’re Professionals, Aren’t We?
Will Thalheimer
What Drives Our Performance? In this article, one of the learning and performance field’s leading visionaries looks back on his twenty-two years in the field with both love and regret, while looking forward to the future by challenging all of us in the field who see ourselves as learning-and-performance professionals. Dr. Thalheimer’s challenge is simple. He asks every person in the field to understand the forces that control their thinking and influence their decision making. It’s as if the author wants to say: the unexamined profession is not worth having. I started in the “training and
my dad had helped me buy at Brooks
development” field in 1985. I was
Brothers—and I walked four blocks into
hired right out of grad school as an
the training field.
instructional designer. I had four
It’s twenty-two years later, and I’ve
courses in instructional design, a
begun to realize how little impact I’ve
master’s degree in business, and a
had on the field—even though I set
bachelor’s degree in psychology. The
out to make big changes in 1998 when
only thing I knew as I was looking
I founded my research and consulting
for a job was that I wanted to build
practice, Work-Learning Research. I
simulations to help people learn.
had decided to start Work-Learning
I still remember paging through
Research when I noticed how our
Training magazine, looking at the
field tended to jump from one fad to
advertisements, trying to find a good
another and hang on sanctimoniously
place to enter the field. And then,
to learning methods that didn’t work.
there it was—an ad for the Strategic
My diagnosis at the time? We had
Management Group, Inc., “world
no anchoring principles to guide us.
leaders in business simulation.”
We didn’t have a common body of
Perfect, I thought. I could combine
knowledge that we could rely on to be
my MBA with my instructional-design
valid and reliable. My thought—which
background. Where were they located?
I now realize was gonzo naïve—was
Unbelievable, just four blocks from my
that I could help move the field toward
apartment in West Philadelphia. The
professionalism by compiling the
very next day, I put on my best business
learning research from the world’s
suit—my only business suit, the one
best refereed journals and by sharing
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
326
Michael Allenʼs 2008 e-Learning Annual
that research through my work and
(Certified Professional in Learning
writings. I was then, and probably am
and Performance).
now, a complete idiot for believing
ÿ Technology for learning has
in the power of good information. It
changed radically: where lean-
takes more than that. Much more.
media computer-based training
This chapter will explore the history
was at the forefront in 1985, today
of our field through my eyes—my
m-learning, e-learning, blogs, and
cynical eye and my optimistic eye.
wikis are out on the bleeding edge. The world is connected via the
The Most Obvious Changes
Let me get the
Internet and wireless technologies.
most obvious
The evidence: the articles and
changes out of
advertisements in our trade
the way quickly.
magazines in 1985 focused on
I’m just going
CBT, audiovisual equipment, and
to list them here, mostly so that you
the like. Today’s media highlight
don’t think I spent the last twenty-two
e-learning and the other Internet-
years as a dentist or forklift operator.
based technologies, along with
ÿ What we called the “training-
authoring tools that go far beyond
and-development field” has morphed into the “learning-
the authoring tools of 1985. ÿ Where training folks hardly
and-performance field.” Today,
had to worry about the legal
everybody knows that training
department in 1985, today a
must be designed to get business
whole new industry has arisen
results, whereas in 1985 there was
around the area of compliance
much less of that mentality. The
training. The evidence: one of
evidence: ISPI changed the name
the most sought after educational
of its magazine from Performance
degrees for HR directors today
and Instruction to Performance
is the law degree. There are now
Improvement. ISPI began offering
industry organizations devoted
a certificate in performance
solely to compliance issues and
technology in the late 1980s or
a high percentage of the vendors
early 1990s. Both ISPI and ASTD
exhibiting at their conferences are
now offer certifications that focus
training companies.
on getting business results, ISPI’s
ÿ For large companies, training
CPT (Certified Performance
often has global requirements,
Technologist) and ASTD’s CPLP
and companies must deal with the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Weʼre Professionals, Arenʼt We?
inherent logistical and learning
327
So now that I’ve gotten the obvious
difficulties surrounding those
stuff out of the way, I want to explore
issues. The evidence: globalization
the critical levers that control the
is a big theme in the popular press
direction of our field. I’m going to
now, but was barely mentioned
blend some of the obvious stuff into
in 1985. Indian companies are
the mix, but will reveal some other
snatching up U.S.-based training
critical factors as well.
companies like crazy. In 1985, nobody in the industry even thought of India. Now almost every large company has a
What Drives Our Performance?
Thinking drives performance. It drives the
significant cohort of employees
performance of
working internationally.
our learners and
ÿ We moved from a lecture model
it drives our performance as learning-
of training to a more “learning-
and-performance professionals. The
centered” model of training.
diagram in Figure 1 illustrates the
Malcolm Knowles’ revolution
processes we typically work in.
around the notion of the adult learner was just beginning to bubble up in 1985. Cognitive psychology hadn’t really made inroads into training practice in the 1980s, and constructivism and the notion that learners build their own memory structures only
Figure 1. The Instructional Design Processes
gathered traction with learning researchers in the 1990s. The
We bring our knowledge, wisdom,
evidence: today, “learner-centric”
and biases to our work. These drive
is a big buzzword. Instructional
everything else we do because they
designers are taught to trust their
control our thinking. They impact
learners—not always a good idea,
the information we notice and
by the way. Wiki’s and other group-
gather. They restrain our design and
knowledge-creation mechanisms
development work. They dictate how
are widely touted to be superior
we deploy our solutions and how we
to older expert-driven top-down
evaluate our work. It is our knowledge,
approaches.
wisdom, and biases that enable us to
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
328
Michael Allenʼs 2008 e-Learning Annual
do our jobs, while constraining our
design teach inflexible procedures
horizons and our success.
and encourage students to believe
Let’s call our “knowledge, wisdom,
that there is one right way. This
and biases” our “thought blocks,” to
type of chicanery is anathema to a
give them a useable label and two
thinking profession. What we need
connotations—one as our building
instead are learning-and-performance
blocks and one as the blocks that wall
professionals who can think critically
in our vision, thinking, and action.
about the myriad of variables that
You’ll notice in Figure 1 how these
affect productivity and performance.
thought blocks dictate the rest of the
We need folks who can evaluate
instructional-design process.
refereed research with diligence and
If our thought blocks control our
skepticism, who can view a vendor’s
output as professionals, then it’s pretty
presentation and see where the truth
obvious that if our thought blocks
lies and know where the lies begin,
can be changed, our performance
who know they can’t rely only on the
can be changed. So one of the most
fragile tools they were given in school
important questions then becomes:
but must also keep learning so they
What controls our thought blocks?
can deepen their wisdom.
Or as I put it more colloquially in the
There is no evidence that the
title, “What drives our performance” as
schooling of our profession has
instructional professionals?
changed much. If anything, our
The rest of this article will outline
learning-and-performance schools
the most powerful forces that control
have become more monolithic
the decisions we make as members
and practice-based. I know of a
of the field. If we understand these
few executives at instructional-
forces, we are more likely to free
development companies who refuse
ourselves from the shackles they
to hire graduates of instructional-
impose. We’re also more likely to get
design programs because those
disgusted enough to help make them
graduates are too rigid, they lack
better.
mental flexibility, and they don’t have enough deep knowledge of learning to
1. Our Educational
make intelligent instructional-design
Preparation Our educational
tradeoffs. Fewer and fewer students
background affects our thought
are learning the tools of the scientist-
blocks in a big way. Unfortunately,
practitioner. This doesn’t mean that
many of our schools of instructional
they should be doing science, but they
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Weʼre Professionals, Arenʼt We?
329
ought to know how to think in the
Level 2 assessments we do are done
critical manner of a person versed in
at the end of the learning event in the
science.
same context as the learning event. Measuring learning when it is top-of-
2. Our Impoverished
mind in situations that provide unfair
Feedback Loops There are basically
hints is a completely bogus enterprise.
two sources of information that can
(See Thalheimer, 2007, for a complete
be brought to bear on any project,
explanation of these measurement
(a) the information we bring the
biases.) These flaws make most Level
project (our thought blocks) and
2 evaluations worthless at best—and
(b) the information we learn from
deceptive in our favor, at worst. Most
doing the project (our feedback
companies do very little assessment of
loops). I’m highly skeptical about
the on-the-job performance (Level 3)
our thought blocks, but I’m even
or of the business results (Levels 4 and
more skeptical about our feedback
5). So the truth is that we don’t receive
loops. The fact is that we receive very
very good feedback about whether our
little valid feedback about how we’re
learning interventions have improved
doing as learning-and-performance
learning, performance, or business
professionals. As evidenced in ASTD
results.
and other annual reports of our
This situation hasn’t changed at all
industry, more than 75 percent of
since 1985, even though ROI, metrics,
companies rely most heavily on Level
learning analytics, and Kirkpatrick’s
1 smile sheets—learner ratings of
four levels are a mainstay in our
instruction. Unfortunately, these smile
conference sessions, books, and
sheets are not predictive of learning
articles. Our impoverished feedback
or on-the-job performance, having
loops leave us in the dark. We simply
correlations that are very weak (for
don’t receive good enough feedback
those of you who are statistically
to improve our performance. We
inclined, correlations (r) are typically
can have all the common sense in
less than .2) (Alliger, Tannenbaum,
the world, we can have forty years
Bennett, Traver, & Shotland, 1997).
of experience, we can hire our
Our measures of learning tend to
field’s best experts, but if all of this
be biased to the measurable, not the
common sense and experience has
authentic, and are additionally biased
been burnished in a dark caldron
toward good results because the
with poor feedback, then it’s a pretty
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
330
Michael Allenʼs 2008 e-Learning Annual
dubious knowledge base we’re talking
Arrogance is one thing. Arrogance
about. And good research only helps
from ignorance simply cannot be
a little in this regard. Research tends
tolerated in a profession as important
to focus on general situations, not
as ours. Arrogance leads to blindness
specific ones. So, although it is a good
and bad results. If you’re arrogant, you
guide, we still ought to be testing our
don’t listen, you don’t challenge your
solutions and obtaining feedback from
long-held paradigms, you don’t learn.
our specific situations.
As professionals devoted to learning, the irony that our arrogance impairs
3. Our Freakin’ Arrogance in Light
our learning is just too grating to
of the Darkness Despite the dark
tolerate.
fog we work in—or maybe because of
Our overall level of arrogance has
it—many of us have an unrelenting
not changed much since 1985. We’re
arrogance that we have all the answers.
still bloody arrogant. But the forces
How many claims have you heard like
that drive our arrogance are a bit
these?
different. Today, with so much more
ÿ Learning objectives should not
information available—the almost
include the word “understand.” ÿ All our courses should be moved to e-learning. ÿ Feedback should be given immediately.
cacophonous deluge of information from the Internet, from our vendors, and from the increasing number of books published each year—there are more sources of information
ÿ Instructional design is irrelevant.
to challenge our mental fortresses.
ÿ The first step of instruction is to
On the other hand, the increasing
gain learner attention. ÿ Instructional designs must account for learning styles.
marketplace competition and drive for the bottom line has made it increasingly difficult for us to have time to reflect on our practices. These
All of these statements are false, misleading, or are gross
forces seem to have cancelled each other out.
oversimplifications that lead to bad learning design—and yet many
4. Trade Organizations,
instructional professionals will look
Conferences, and Magazines Trade
you in the eye with a holy passion and
organizations seem like benevolent
claim their universal truth.
creatures. Ideally, they provide the field with good information, enable
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Weʼre Professionals, Arenʼt We?
331
collaboration between members, and
than there were in 1985. The increased
support interactions between buyers
competition has already improved
and sellers. Unfortunately, most of
the quality of the information being
the trade organizations in our field
exchanged, at least to some extent.
have not reached the ideal. In the
For example, the eLearning Guild
United States, ASTD is the largest
now goes to great lengths to ensure
organization, with annual revenues
that vendors don’t control the
above $28 million. All the other trade
information that is conveyed. The
organizations are small by comparison.
increasing competition enables niche
ISPI brings in about $2 million in
branding opportunities. So ASTD
revenue. Other organizations are
has become the mainstream vanilla
private and so their financials are
provider, ISPI is the rigorous provider,
private. These include The Masie
Masie provides the new and high-
Center, the eLearning Guild, Nielsen
tech, the eLearning Guild provides
Business Media (publishers of
the e-learning community with
Training magazine), and CLO Media
opportunities to learn from each other,
(publishers of Chief Learning Officer
and CLO caters to learning executives
magazine).
and the vendors who chase them.
What all of these enterprises
The business models of these
have in common is that they have
organizations force them to rely
to raise funds to stay in business.
on volunteer speakers, writers,
Thus, the most critical incentives
and collaborators. Because it takes
for these organizations rest in their
a substantial effort to prepare a
ability to bring in money through
conference presentation, write a
membership fees, sales of books
book or article, or share some other
and periodicals, sales of educational
form of intellectual property, trade-
materials, conference admission fees,
organization volunteers come mostly
credentialing fees, and workshop fees.
from those who need to gain visibility
So these trade organizations have a
to survive as commercial entities.
tendency to emphasize information that sells, to the detriment of
5. Consultants and Vendors
information that is proven, time-
Consultants and vendors need sales to
tested, and true.
survive. To get sales, they need to let
There are more magazines, more
buyers know they exist. They can do
conferences, more books, and more
this through advertisements, personal
trade associations to choose among
networking, and visibility. Visibility
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
332
Michael Allenʼs 2008 e-Learning Annual
is available in our industry at reduced
by their commercial interests, not
dollar cost but increased cost of time
necessarily by the truth or certainty of
and effort. Consultants and vendors
the ideas presented.
make themselves visible by writing articles for trade magazines, writing books, and speaking The pressure to be visible is a at conferences. crushing weight on consultants Sometimes they and vendors. can are paid for these efforts, but
The needs of consultants and vendors never change. To maintain their status, they have to find ways to maintain their visibility. This will never change, and certainly hasn’t changed much since 1985. What is different is that there are more
usually they have to write or speak
potential buyers now, more trade
for free. The pressure to be visible is
organizations, and more consultants
a crushing weight on consultants and
and vendors. Having more sources of
vendors; thus they are the ones most
information will tend, in the long run,
likely to endure the proposal-writing,
to improve the best information that is
session-development process. Most of
communicated, while simultaneously
our conference sessions are developed
making it difficult for members of
with the expressed purpose of making
our profession to determine which
a sale. Even if the presenter doesn’t
information is good and which is
seem to be trying to sell him/herself
faulty.
or a product, almost certainly that is a large part of the intention and the
6. Learning Research In refereed
design.
journals such as Cognition &
Sometimes consultants and vendors
Instruction, Educational Psychology,
even have to pay to speak. Conference
and the Journal of Applied Psychology
proposals are accepted at a rate of
there is a wealth of knowledge
about 10 to 20 percent. Thus, many
that might inform our practices
who want to speak are denied. Many
as learning-and-performance
conferences offer paid exhibitors
professionals. Unfortunately, the
speaking slots, asking as much as
researchers who do the research
$30,000 for the privilege.
don’t have the time, the incentives, or
Consultants in particular have a
the capability to take their research
great incentive to write books and
and make it directly usable for
articles. Again, this kind of pressure
practitioners. There are hardly any
increases the likelihood that the
research-to-practice translators—only
messages we receive will be tainted
Ruth Clark and I attempt to do this
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Weʼre Professionals, Arenʼt We?
333
full time. Moreover, practitioners have
for a second company. It doesn’t
been so ill-prepared to be consumers
mean the benchmarked company did
of research-based information that
enough due diligence in the first place.
attempts at translation tend to fall on
It doesn’t mean the benchmarked
deaf ears or be misunderstood.
company did the right kind of
Compounding these problems is the lack of extrinsic incentives
evaluation to prove its effectiveness. But what about research that
for practitioners to build more
has been done on a wide swath of
effective learning-and-performance
companies? For example, surveys
interventions. Most of us want to do
of two hundred companies or so.
good work, but very few of us have to
Unfortunately, most of this research
build effective learning interventions.
is seriously flawed, surveying the
Since we don’t really need to, we
opinions of one or two people in a
don’t go the extra mile to parse the
company, drawing from a sample
sources of information that might help
of the willing, not a representative
us—sources like the learning research
sample of the industry. Year-to-year
or the translated versions of that
data tends to represent a new sample
research.
of respondents, making year-toyear comparisons suspect at best.
7. Industry Research When
Regardless of whether this type of
learning-and-performance
research is done in our largest trade
professionals have to persuade their
organizations or through private
organizations to adopt a certain
research companies, the information is
learning approach, the evidence that
seriously flawed and tends to represent
carries the most weight is industry
mediocre practice,
research and benchmarking against
not leading-edge,
Just because another company
other organizations. The thinking
state-of-the-art
is using a particular learning
goes something like this. “If Nabisco,
thinking.
method—even if they’re
Wal-Mart, and Raytheon are using
Since 1985,
such-and-such, it must be a proven
more and more
that it’s going to be the right
learning method.” Unfortunately,
so-called “research
method for a second company.
the logic doesn’t make sense. Just
reports” have
because another company is using a
been promulgated in our industry.
particular learning method—even if
Unfortunately, when we pay attention
they’re happy about it—doesn’t mean
to these, we utilize valuable time
that it’s going to be the right method
that could be better spent mulling
happy about it—doesn’t mean
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
334
Michael Allenʼs 2008 e-Learning Annual
over better information. When we
the number of dollars spent on
make decisions based on this flawed
training. Or worse, best-of lists can
information, we do ourselves, our
simply be a list of an organization’s
organizations, and our learners
best clients. More than one website in
grievous damage.
our field produces bogus best-of lists based on the amount of money their
8. Contests, Awards, Best-of
advertisers have paid in fees.
Lists, and Other Forms of Thought Control Finally, I must add a section on contests, awards, best-
What This All Means
For the last twenty years or
of lists, and other attention-grabbing
more, hundreds
devices. Each of these devices has four
of millions of
main characteristics: (1) they simplify
employees have
reality, (2) they grab attention, (3)
managed their way through learning-
they distort the truth, and (4) they
and-performance interventions that
are designed to make someone a
we have designed, developed, and
buck. In doing all these things, they
deployed. They have certainly received
control what we think. For example,
great benefits from our work. But
contests of rapid-learning tools are
while we gave them some small seeds
great theater, but poor instructional
of knowledge and inspiration, we
design—especially since the metric is
failed to give them deeper roots of
time-to-develop not effectiveness of
knowledge, wisdom, and competence.
design. e-Learning awards typically
While providing value, we have
ask mediocre volunteer judges to pick
failed to live up to our potential
from entries submitted using non-
as an industry and as individual
interactive video. Submissions are
professionals in that industry. Much
only accepted from companies that
of this is not fully in our control. It is
can afford to pay a hefty entrance fee.
shaped by forces in our industry that
Gold medal winners typically have
capture large shares of our minds,
produced flashy products with new
overload the capacity of our working
technological or visual innovations,
memories to focus on more important
but learning effectiveness does not
things, and make it impossible for
seem to separate gold medal winners
us to make good decisions about our
from the also-rans. Best-of lists can
craft.
utilize the wrong metrics, for example,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Weʼre Professionals, Arenʼt We?
To briefly recap the failings of our
335
6. Learning-and-performance
industry:
research is not utilized when it
1. Our graduate schools prepare
might provide substantial benefits.
technicians, not thoughtful scientist-practitioners who understand learning, think
7. Industry research is severely flawed, but we rely on it anyway. 8. Contests, awards, and best-of lists
critically, and build wisdom over
grab our attention and distort
time.
our thinking about what is most
2. We don’t measure the outcome of
important.
our work in ways that enable us to build effective feedback loops and make improvements that will
Is There Hope for the Future?
Maybe the wasteland of
lead to better learning, on-the-job
the profession
performance, and business results.
I’ve described is the way all
3. The work pressures we face (for example, Internet-induced
professions operate—with imperfect
information overload and business
information and a maddening
demands for cheaper, faster
competition for the limited working-
results)—combined with our
memory capacities of its professionals.
tendency toward professional
Maybe in the next twenty-five years,
arrogance—don’t predispose
nothing much will change in our field.
us to keep learning, to test our
On the other hand, professions do
conjectures, to build a rich and
change. Doctors used to use leeches
complex knowledge base over time.
and snake oil. Fisherman used to kill
4. Our trade associations, magazines,
porpoises while netting tuna. Clothiers
and conferences provide us
used to rely on sweatshops. Painters
with information that sells, not
used to use lead paint. Professional
information that necessarily tells
basketball players used to use
the truth of how we should better
teamwork. Democracies used to act
design our products and services.
as governments of the people, by the
5. Our consultants and vendors are a large source of our information,
people, and for the people. I believe our profession can change
and we tend to think uncritically
for the better. If I didn’t believe this,
about their offerings.
I’d get the hell out and find another vocation.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
336
Michael Allenʼs 2008 e-Learning Annual
But enough about me. Are there any
and new books are coming out
signs that things might change for the
suggesting that measuring our
better? I’m pretty sure there are omens
results is a critical discipline for
that are already signaling a change.
training.
Here’s the list: 1. In the medical profession, a set of practices under the rubric “evidence-based medicine” is sweeping the field. 2. In the K-12 education field, it has become a regulatory requirement to prove the effectiveness of learning interventions. 3. In the management field, people
6. Compliance training’s reliance on course completion is coming to be seen as the joke that it is. 7. New laws (for example, California’s Law AB 1825) are being written that require effective training practices. 8. More and more learning vendors are including evaluations in their project proposals and
are beginning to talk about
contemplating ways to provide
evidence-based management.
themselves with performance
4. In our own field, research-based information—when it is presented
feedback to build cycles of continuous improvement.
with practical wisdom—clearly resonates with professionals.
Am I hopeful? You bet. But I
Business leaders like “e-learning
also worry that we’re just not doing
guru” Kevin Kruse are taking the
enough. I encourage you to get your
evidence-based training message to
ass in gear. We need an army of
the marketplace.
soldiers to win this war.
5. New software is enabling training to be managed from training deployment all the way through to on-the-job implementation,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Weʼre Professionals, Arenʼt We? REFERENCES Alliger, G.M., Tannenbaum, S.I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50, 341-358. Thalheimer, W. (2007, April). Measuring learning results: Creating fair and valid assessments by considering findings from fundamental learning research. Retrieved April 30, 2007, from www.work-learning.com/catalog/.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
337
Final Word The thoughts shared by the
Not all contributors to this Annual
contributors to this Annual are
agree on every point, but they do
inspiring and thoughtful. They shout
appear to agree that we have the
messages that too often fall on deaf
knowledge to achieve much more in
ears. Like Cassandra, who lost her
e-learning—even without increasing
sanity because of her ability to see the
the level of effort or development
future but could find no way to use her
costs. It’s more about working smarter
knowledge to head off misfortunes,
than spending more, although a more
these experienced researchers and
rational perspective would often result
practitioners plead with today’s
in larger budgets. A good start would
e-learning designers and developers
be to become more aware of precedent
to consider the many lessons learned
work and insist that instructional
over years of intensive work with
designers have fitting credentials.
learning technology.
We wouldn’t entrust a person with
The mistakes people make with
no architectural knowledge to design
e-learning today would be useful
a multimillion-dollar building, yet
ones if they were being made for the
many of today’s e-learning designers
first time and there were no history
have very little background in learning
to learn from. But there is extensive
or design. While it may be argued
knowledge about what does and does
that few e-learning projects could
not work in educational technology.
waste millions of development dollars
Witless of the lessons learned, many
because there are rarely such budgets,
of today’s efforts are pointless and
poorly designed applications do waste
destined to fail. Perhaps even more
something very precious, indeed—the
frustrating is that so many pattern
time that learners waste on them.
their work after clueless efforts. There
Wasted time can never be recovered.
has become a broad, common wisdom
Considering lost opportunities, poor
that directs much work and yet has
performance, and wasted learner time,
no basis in either theory or successful
there might actually be quite a few
outcomes.
e-learning applications that do, in fact,
340
Michael Allenʼs 2008 e-Learning Annual
squander away millions of dollars.
how to develop highly effective
But even worse, when children fail to
learning experiences in every project
develop competencies at optimal times
undertaken.
in their intellectual development, they may never develop them to the extent they might easily have.
A Lot Has Changed
We Saw a Different Future
I think most of the authors, if not all, would
The capabilities
agree with me
of today’s
that we labored
e-learning
early under an assumption that may
systems are
never prove valid, and certainly hasn’t
astounding in
proven valid within the timeframe we
comparison to what existed when
expected. We assumed that almost
these authors began their work.
limitless resources could be applied,
Indeed, when I started my own work
justifiably, on many e-learning courses.
in the late 1960s on an IBM 1130,
Because the worldwide audience
programming tools were rudimentary,
of learners was so large, just a few
the display device was an electric
pennies per learner would produce
typewriter, and a noticeable delay
far larger budgets than could ever
occurred before learner responses
be spent. Continual refinement of
could be analyzed. We had no
courses dealing with essential skills,
graphics, no animation, no sound, no
from reading and writing to math,
color, and no ability for learners to
chemistry, the arts, leadership,
communicate with each other or with
financial planning, and thousands of
an instructor. So when we developed
topics, would create superb, powerful
instructional applications, we were
learning experiences—regardless of
very careful to consider each design
the effectiveness of the initial designs.
decision and to evaluate the outcome
I actually expected there to become a
afterward.
competitive economy of courseware
We made many mistakes, of
with ever more wonderful learning
course, but we learned from them.
experiences being invented to intrigue,
Each successive project built on the
motivate, and teach.
knowledge gained from previous
Nearly all of the early efforts
ones. There was a certainty in our
were more carefully thought out
minds that we would eventually
than are most of today’s e-learning
have an articulate knowledge of
applications. No one would have
Final Word
341
considered using such an expensive
time and the importance of providing
system just to present learners a set of
children with stimulating learning
bullet-point slides and then ask a few
experiences at optimal points in their
questions afterward. When a few early
cognitive and physical development,
systems developed audio capabilities,
surely we will begin to offer the
there was no rush to simply add voice-
great learning opportunities that are
over narration to bullet-point slides.
possible.
Nevertheless, many early designs
The field is populated by many
would have been considered weak,
extremely dedicated individuals
if not foolish, by the field’s pioneers,
looking for the opportunity and means
but one has to remember that these
for developing inspiring, fun, and
projects were creating foundational
effective learning experiences. They
experiences. Those experiences need
could most likely make more money
not be re-created again and again.
in other occupations, yet they do truly care about helping learners and keep
Room for Optimism
With all the
striving to make a difference. They are
disappointing
frustrated by what’s happening in so
work that’s going
much of today’s e-learning, if not even
on today, I’m
in their own projects. But they see
still optimistic
the opportunity and know that life-
about the future. While adoption of
changing learning experiences can be
design principles that actually lead to
built, and once built, could be offered
outstanding learning experiences has
to very large numbers of learners at
been slow, and authoring tools seem
very low cost.
to have stagnated at disappointingly
Never before has there been
primitive levels, delivery capabilities
such a great opportunity to share
have surged ahead. We have very
wisdom, knowledge, and skills for the
inexpensive, multimedia-capable,
betterment of human life. We have the
high-speed delivery devices networked
contributors to this first volume of the
with each other and mega data and
e-Learning Annual to thank for much
knowledge bases around the world.
of the opportunity. It is up to us to
As we begin to value each learner’s
make the most of it.
Pfeiffer Publications Guide This guide is designed to familiarize you with the various types of Pfeiffer publications. The formats section describes the various types of products that we publish; the methodologies section describes the many different ways that content might be provided within a product. We also provide a list of the topic areas in which we publish.
FORMATS In addition to its extensive book-publishing program, Pfeiffer offers content in an array of formats, from fieldbooks for the practitioner to complete, ready-to-use training packages that support group learning.
FIELDBOOK Designed to provide information and guidance to practitioners in the midst of action. Most fieldbooks are companions to another, sometimes earlier, work, from which its ideas are derived; the fieldbook makes practical what was theoretical in the original text. Fieldbooks can certainly be read from cover to cover. More likely, though, you’ll find yourself bouncing around following a particular theme, or dipping in as the mood, and the situation, dictate.
HANDBOOK A contributed volume of work on a single topic, comprising an eclectic mix of ideas, case studies, and best practices sourced by practitioners and experts in the field. An editor or team of editors usually is appointed to seek out contributors and to evaluate content for relevance to the topic. Think of a handbook not as a ready-to-eat meal, but as a cookbook of ingredients that enables you to create the most fitting experience for the occasion. RESOURCE M aterials designed to support group learning. They come in many forms: a complete, readyto-use exercise (such as a game); a comprehensive resource on one topic (such as conflict management) containing a variety of methods and approaches; or a collection of like-minded activities (such as icebreakers) on multiple subjects and situations.
TRAINING PACKAGE An entire, ready-to-use learning program that focuses on a particular topic or skill. All packages comprise a guide for the facilitator/trainer and a workbook for the participants. Some packages are supported with additional media—such as video—or learning aids, instruments, or other devices to help participants understand concepts or practice and develop skills. • Facilitator/trainer’s guide Contains an introduction to the program, advice on how to organize and facilitate the learning event, and step-by-step instructor notes. The guide also contains copies of presentation materials—handouts, presentations, and overhead designs, for example—used in the program. • Participant’s workbook Contains exercises and reading materials that support the learning goal and serves as a valuable reference and support guide for participants in the weeks and months that follow the learning event. Typically, each participant will require his or her own workbook.
ELECTRONIC CD-ROMs and web-based products transform static Pfeiffer content into dynamic, interactive experiences. Designed to take advantage of the searchability, automation, and ease-of-use that technology provides, our e-products bring convenience and immediate accessibility to your workspace.
METHODOLOGIES CASE STUDY A presentation, in narrative form, of an actual event that has occurred inside an organization. Case studies are not prescriptive, nor are they used to prove a point; they are designed to develop critical analysis and decision-making skills. A case study has a specific time frame, specifies a sequence of events, is narrative in structure, and contains a plot structure—an issue (what should be/have been done?). Use case studies when the goal is to enable participants to apply previously learned theories to the circumstances in the case, decide what is pertinent, identify the real issues, decide what should have been done, and develop a plan of action.
ENERGIZER A short activity that develops readiness for the next session or learning event. Energizers are most commonly used after a break or lunch to stimulate or refocus the group. Many involve some form of physical activity, so they are a useful way to counter post-lunch lethargy. Other uses include transitioning from one topic to another, where “mental” distancing is important.
EXPERIENTIAL LEARNING ACTIVITY (ELA) A facilitator-led intervention that moves participants through the learning cycle from experience to application (also known as a Structured Experience). ELAs are carefully thought-out designs in which there is a definite learning purpose and intended outcome. Each step—everything that participants do during the activity—facilitates the accomplishment of the stated goal. Each ELA includes complete instructions for facilitating the intervention and a clear statement of goals, suggested group size and timing, materials required, an explanation of the process, and, where appropriate, possible variations to the activity. (For more detail on Experiential Learning Activities, see the Introduction to the Reference Guide to Handbooks and Annuals, 1999 edition, Pfeiffer, San Francisco.)
GAME A group activity that has the purpose of fostering team spirit and togetherness in addition to the achievement of a pre-stated goal. Usually contrived—undertaking a desert expedition, for example—this type of learning method offers an engaging means for participants to demonstrate and practice business and interpersonal skills. Games are effective for team building and personal development mainly because the goal is subordinate to the process—the means through which participants reach decisions, collaborate, communicate, and generate trust and understanding. Games often engage teams in “friendly” competition.
ICEBREAKER A (usually) short activity designed to help participants overcome initial anxiety in a training session and/or to acquaint the participants with one another. An icebreaker can be a fun activity or can be tied to specific topics or training goals. While a useful tool in itself, the icebreaker comes into its own in situations where tension or resistance exists within a group.
INSTRUMENT A device used to assess, appraise, evaluate, describe, classify, and summarize various aspects of human behavior. The term used to describe an instrument depends primarily on its format and purpose. These terms include survey, questionnaire, inventory, diagnostic, survey, and poll. Some uses of instruments include providing instrumental feedback to group members, studying here-and-now processes or functioning within a group, manipulating group composition, and evaluating outcomes of training and other interventions. Instruments are popular in the training and HR field because, in general, more growth can occur if an individual is provided with a method for focusing specifically on his or her own behavior. Instruments also are used to obtain information that will serve as a basis for change and to assist in workforce planning efforts.
Paper-and-pencil tests still dominate the instrument landscape with a typical package comprising a facilitator’s guide, which offers advice on administering the instrument and interpreting the collected data, and an initial set of instruments. Additional instruments are available separately. Pfeiffer, though, is investing heavily in e-instruments. Electronic instrumentation provides effortless distribution and, for larger groups particularly, offers advantages over paper-and-pencil tests in the time it takes to analyze data and provide feedback.
LECTURETTE A short talk that provides an explanation of a principle, model, or process that is pertinent to the participants’ current learning needs. A lecturette is intended to establish a common language bond between the trainer and the participants by providing a mutual frame of reference. Use a lecturette as an introduction to a group activity or event, as an interjection during an event, or as a handout.
MODEL A graphic depiction of a system or process and the relationship among its elements. Models provide a frame of reference and something more tangible, and more easily remembered, than a verbal explanation. They also give participants something to “go on,” enabling them to track their own progress as they experience the dynamics, processes, and relationships being depicted in the model.
ROLE PLAY A technique in which people assume a role in a situation/scenario: a customer service rep in an angry-customer exchange, for example. The way in which the role is approached is then discussed and feedback is offered. The role play is often repeated using a different approach and/or incorporating changes made based on feedback received. In other words, role playing is a spontaneous interaction involving realistic behavior under artificial (and safe) conditions.
SIMULATION A methodology for understanding the interrelationships among components of a system or process. Simulations differ from games in that they test or use a model that depicts or mirrors some aspect of reality in form, if not necessarily in content. Learning occurs by studying the effects of change on one or more factors of the model. Simulations are commonly used to test hypotheses about what happens in a system—often referred to as “what if?” analysis—or to examine best-case/worst-case scenarios.
THEORY A presentation of an idea from a conjectural perspective. Theories are useful because they encourage us to examine behavior and phenomena through a different lens.
TOPICS The twin goals of providing effective and practical solutions for workforce training and organization development and meeting the educational needs of training and human resource professionals shape Pfeiffer’s publishing program. Core topics include the following: Leadership & Management Communication & Presentation Coaching & Mentoring Training & Development E-Learning Teams & Collaboration OD & Strategic Planning Human Resources Consulting
What will you find on pfeiffer.com? • The best in workplace performance solutions for training and HR professionals • Downloadable training tools, exercises, and content • Web-exclusive offers • Training tips, articles, and news • Seamless on-line ordering • Author guidelines, information on becoming a Pfeiffer Affiliate, and much more
Discover more at www.pfeiffer.com
