MMIC ICHA EL LLEN ’S’S HA ELAA LLEN e eL LE EA A RNING NN UA LL RNINGAA NN UA
20 0 88 200
About Pfeiffer Pfeiffer serves the professional development and hands-on resource needs of training and human resource practitioners and gives them products to do their jobs better. We deliver proven ideas and solutions from experts in HR development and HR management, and we offer effective and customizable tools to improve workplace performance. From novice to seasoned professional, Pfeiffer is the source you can trust to make yourself and your organization more successful.
Essential Knowledge Pfeiffer produces insightful, practical, and comprehensive materials on topics that matter the most to training and HR professionals. Our Essential Knowledge resources translate the expertise of seasoned professionals into practical, how-to guidance on critical workplace issues and problems. These resources are supported by case studies, worksheets, and job aids and are frequently supplemented with CD-ROMs, websites, and other means of making the content easier to read, understand, and use.
Essential Tools Pfeiffer’s Essential Tools resources save time and expense by offering proven, ready-to-use materials—including exercises, activities, games, instruments, and assessments—for use during a training or team-learning event. These resources are frequently offered in looseleaf or CD-ROM format to facilitate copying and customization of the material. Pfeiffer also recognizes the remarkable power of new technologies in expanding the reach and effectiveness of training. While e-hype has often created whizbang solutions in search of a problem, we are dedicated to bringing convenience and enhancements to proven training solutions. All our e-tools comply with rigorous functionality standards. The most appropriate technology wrapped around essential content yields the perfect solution for today’s on-the-go trainers and human resource professionals.
Essential resources for training and HR professionals w w w. p f e i f f e r. c o m
A bout T hi s B o ok Why is this topic important? e-Learning is here to stay, for better or worse. Let’s hope it’s for better. The founders of e-learning technology saw a means of improving the quality of individual learning experiences that no other medium offered. They worked to blend what was known about human learning and pedagogy with emerging computer and communication technologies and to study the outcomes to continuously improve effectiveness. Unfortunately, much of today’s elearning fails to build on the vision, the knowledge, and the most important achievements in learning technology. It therefore provides much less learning and performance impact than it could and should, costing organizations immensely and wasting precious learning time. What can you achieve with this book? You can improve the quality of e-learning you buy, create, and use. Contributors to this volume openly and explicitly state the lessons they have learned through decades of first-hand exploratory work. The advice and direction so bluntly shared is applicable whether you work in training or education, in corporations, non-profit organizations, military, or schools. How is this book organized? Each author was asked three questions: What intrigued you about elearning to the point of focusing so much of your career on it? After some decades of working with learning technology, what have you learned (perhaps in contrast to what’s become the general perception)? Looking to the future, what are your current visions, hopes, and expectations? Each article wrestles with these questions in a unique way, but with varying degrees of emphasis. They are organized based somewhat on this emphasis and somewhat on chronology, beginning with authors who were responsible for creating landmark systems and beginning the discussion, moving to lessons learned, and ending with those who focus on critical questions of the day.
Th e P f eif f er A n nua l Seri es The Pfeiffer Annuals present each year never-before-published materials contributed by learning professionals and academics and written for trainers, consultants, and human resource and performance-improvement practitioners. As a forum for the sharing of ideas, theories, models, instruments, experiential learning activities, and best and innovative practices, the Annuals are unique. Not least because only in the Pfeiffer Annuals will you find solutions from professionals like you who work in the field as trainers, consultants, facilitators, educators, and human resource and performance-improvement practitioners and whose contributions have been tried and perfected in real-life settings with actual participants and clients to meet real-world needs. The Pfeiffer Annual: Consulting Edited by Elaine Biech The Pfeiffer Annual: Leadership Development Edited by James Noel and David Dotlich The Pfeiffer Annual: Management Development Edited by Robert C. Preziosi The Pfeiffer Annual: Training Edited by Elaine Biech Michael Allen’s 2008 e-Learning Annual Edited by Michael Allen
Call for Papers In subsequent volumes of Michael Allen’s e-Learning Annual, we will present the most current explorations, thinking, insights, case studies, challenges, guidelines, and other helpful aids for the e-learning professional. We need your requests and your contributions. e-Learning can achieve its full potential to improve the world one learner at a time only through the sharing of best practices and through a continual questioning of ourselves and our work. I’m interested in providing a broad spectrum of articles and other contributions. If you wish to contribute or just have a need an author might address, please contact me at
[email protected] or call me at 651-203-3700.
MMIC ICHA EL LLEN ’S’S HA ELAA LLEN e eL LE EA A RNING NN UA LL RNINGAA NN UA
20 0 88 200
Copyright © 2008 by John Wiley & Sons, Inc. All Rights Reserved. Published by Pfeiffer An Imprint of Wiley 989 Market Street, San Francisco, CA 94103-1741 www.pfeiffer.com Except as specificially noted below, no part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-646-8600, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, 201-748-6011, fax 201-748-6008, or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Readers should be aware that Internet websites offered as citations and/or sources for further information may have changed or disappeared between the time this was written and when it is read. Certain pages from this book are designed for use in a group setting and may be customized and reproduced for educational/training purposes. The reproducible pages are designated by the appearance of the following copyright notice at the foot of each page: Michael Allen’s 2008 e-Learning Annual. Copyright © 2008 by John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com This notice may not be changed or deleted and it must appear on all reproductions as printed. This free permission is restricted to limited customization of the CD-ROM materials for your organization and the paper reproduction of the materials for educational/training events. It does not allow for systematic or large-scale reproduction, distribution (more than 100 copies per page, per year), transmission, electronic reproduction or inclusion in any publications offered for sale or used for commercial purposes—none of which may be done without prior written permission of the Publisher. For additional copies/bulk purchases of this book in the U.S. please contact 800-274-4434. Pfeiffer books and products are available through most bookstores. To contact Pfeiffer directly call our Customer Care Department within the U.S. at 800-274-4434, outside the U.S. at 317-572-3985, fax 317-572-4002, or visit www.pfeiffer.com. Pfeiffer also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. ISBN: 978-0-7879-8743-5 ISSN: 1046-333-X Acquiring Editor: Martin Delahoussaye Director of Development: Kathleen Dolan Davies Development Editor: Susan Rahmeler Production Editor: Dawn Kilgore Editor: Rebecca Taff Editorial Assistant: Julie Rodriquez Manufacturing Supervisor: Becky Morgan Printed in the United States of America Printing 10~NS9~NS8~NS7~NS6~NS5~NS4~NS3~NS2~NS1
Contents Preface
ix
Section 1. Visionary Pioneers Continue to Guide Us Reflections on TICCIT, C. Victor Bunderson
1
Addressing Diversity in (e-)Learning, Patrick Suppes in conversation with Michael W. Allen
31
PLATO, Plasma Screens, and Computer-Based Education, Donald L. Bitzer in conversation with Michael W. Allen
43
Section 2. Learning Technology Takes Shape Back to the Future of Educational Technology, Fred T. Hofstetter
59
It Seemed Like a Good Idea at the Time: The Development of Computer-Assisted Instruction in Schools, Peter G. Fairweather
83
Reflections on e-Learning: The Long History of a New Idea, Wallace Hannum
101
Lessons from the STARS Revisited, Thomas C. Reeves and William D. Aggen
127
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction, M. David Merrill
141
Section 3. What Does It All Mean? Pitfalls and Overhauls Dreams and Responsibilities: Learning and the New Technologies, Tom Boyle
171
Beyond Knowledge Dissemination: Learning to Solve Problems Online, David H. Jonassen
189
What’s Right and What’s Wrong with Online Learning, Greg Kearsley
207
Technology and Learning: A Proposal for Adaptive Tutorial Learning, Alfred Bork, Carole A. Bagley, and Xiwen Zhang
227
Cheap But Not Tacky: Lessons Learned During e-Learning Adventures, Sivasailam “Thiagi” Thiagarajan
249
Section 4. Are We Lost? Where Do We Go from Here? Moving e-Learning into the Workplace, Allison Rossett
265
The Business Impact of e-Learning, Nick H.M. van Dam
285
Lessons Learned . . . and Relearned, Allen Avner
307
We’re Professionals, Aren’t We? What Drives Our Performance? Will Thalheimer
325
A Final Word
339
Pfeiffer Publications Guide
345
Preface Computers needed for e-learning
useful interactive applications.
cost over $20,000 each and need a
Even with sufficient experience,
much more expensive remote support
development of interactions takes
computer as well. It costs about $1,000
considerable time and effort, perhaps
per month to connect each computer
as much as a week to develop a single
to a network. Only more expensive
interaction with feedback.
computers can display any graphics
With all these costs, e-learning
at all, and when they do, they’re not
is obviously much more expensive
very refined, and it takes extra time
than other forms of instruction,
for the graphics to plot on the screen.
yet everyone is excited about it.
The display is monochromatic or
Organizations are finding the money
can display up to eight colors (but
and time to develop courses, even
only four at a time). These expensive
when they sometimes take a year and
computers aren’t always fast, either.
a half to design and develop. They
Sometimes learners must wait up
are finding the money needed for the
to a minute for their answers to be
computers and networking services.
accepted. There’s no animation or
Serious evaluations are scheduled,
video. Sound effects are limited to
and everyone expects revisions to be
beeps.
needed. Organizations sometimes
Development of display graphics
expect preliminary evaluations to
is a very slow and arduous process,
take several months before even more
requiring specification of vector
serious evaluations are undertaken.
coordinates for each line or, more
Because of the high costs, it’s
simply but crudely, translating
important that courses be well
graphics into character graphics
designed and effective. Most
( _, ---, /, \, |, etc.). The development
organizations put their most skilled
language is difficult to learn and
instructors available on projects as
requires about two years of experience
well as their strongest subject-matter
before skills are sufficient to develop
experts. It’s not uncommon for experts
x
Michael Allenʼs 2008 e-Learning Annual
in human learning research to join
started their work. Some went about
the design team. Everyone knows
their explorations and inventions in
that the technology provides new
a carefully studied way. Others were
opportunities, and every aspect of the
pragmatic, while others were simply
instruction will need to be thought
playful. But regardless of the approach,
through carefully to avoid adherence
access to systems was expensive.
to inappropriate or overly constrictive
Those granted access were intent
paradigms. Designs are carefully
about doing something of significance.
considered before development begins,
And because each application was
not only because wasting resources
a major undertaking, it received
would have bad consequences, but
attention and required justification.
also because making changes later is
In these early days, there was
difficult and adds to the already high
perhaps a stronger appreciation than
expenses.
today of the value of better learning
Learner responses are being
experiences in both educational and
logged, and designers analyze these
training venues. Even if costs went
responses in order to improve
up significantly, people, schools,
instructions, questions, and feedback
corporations, the military, and even
and otherwise revise designs. Courses
public funding institutions felt the
are continuously if not systematically
enhanced learning opportunities
improved. While not all courses
would be worth it. Of course, everyone
are successful, even after all this
knew costs would come down if
painstaking work, some demonstrate
systems were successful and adopted
dramatic success and point the
in large numbers.
way to a new era of personalized,
Leaders were inventive. As you’ll
adaptive learning. Others advance our
note from reading the accounts
knowledge of what is ineffective.
provided in this book by some of
If you’re imaging this, then you’re
the most influential leaders of early
envisioning what the e-learning
e-learning efforts, in the face of
arena was like when I entered the
new possibilities, leaders did not
field in the late 1960s (costs roughly
dogmatically follow established
adjusted to today’s dollars). Although
principles of instruction. They did
there are some variants, many of the
not apply practices derived from the
contributors to this first
challenges of classroom instruction
e-Learning Annual encountered
because, obviously, the venue was
similar circumstances when they first
different. But neither did they follow
Preface
xi
what were the established principles of
do add something to the experience
instructional design. The era was new,
on occasion, they do not enhance
and everything was and needed to be
the underlying pedagogy or sustain
questioned.
engagement. Some of the inventiveness is just
Too Much and Too Little Invention
Today, we
too much, primarily because it misses
see too little
the point. An example is the incessant
inventiveness in
drive to create tools that make the
most e-learning.
development of learning experiences
We commonly
quick and easy. While it would clearly
see a regression to simplistic models
be nice if unskilled people could
of instruction that haven’t been very
develop learning experiences in a
engaging anywhere. The best example
blink of an eye—and today’s tools
is a tendency to present a lot of
do make much of the development
information followed by a quiz or test
work far faster and easier than it was
to see whether learners can recognize
when we had only basic programming
correct answers among multiple-
languages—the much more important
choice alternatives. This “tell-and-test”
challenge continues to be how to
model is easy to develop and falls in
make learning experiences effective.
the sweet spot of many so-called rapid
The most unaffordable instruction is
authoring tools. But as convenient
instruction that doesn’t work. Add
and inexpensive as it is to develop,
up all the time learners waste when
this instructional approach is no
they fail to develop needed skills
more attractive or effective delivered
and the opportunities lost from their
via e-learning than it has been in
inadequate performance, and you
classrooms.
almost always have a budget that
Some look to enhance the “tell-and-
far exceeds what anyone spends on
test” approach through technology.
learning interventions, including
Presentations can now have “voice-
development.
over” narration or even contain video.
From my perspective, and I think
Some try to make multiple-choice
the perspective of every contributor
questions more attractive through
to this publication, development costs
animation and novel interaction. Some
are not really the issue; achieving
feel that putting the presentations on
effectiveness is. Indeed, most
a mobile device will enhance learning.
organizations can’t afford their current
Unfortunately, while these attempts
development costs, whatever they are,
xii
Michael Allenʼs 2008 e-Learning Annual
because they are not achieving the
This series of Annuals is one
necessary impact. On the other hand,
attempt to rectify this sad situation of
they could spend many multiples of
today’s hapless e-learning and remind
their current development budgets if
us that, while too rare, some successes
only they were creating the needed
with e-learning have been spectacular
behavioral changes.
and have demonstrated the potential
The expert contributors to this
that lies in wait for all.
volume succeeded in part by breaking rules, by darting out in new directions, by continually looking at their
The e-Learning Annuals
I’m pleased that Pfeiffer
challenges from fresh perspectives.
has elected
Just breaking rules (which novices
to publish an
do without realizing it) isn’t truly
Annual for the
inventiveness or the leadership we
field of e-learning. Growth in the
need. It’s just random variation and
field continues at an impressive rate,
flailing about, which only serves
and few analysts, if any, anticipate a
at best to distract evaluation on
slowdown in the foreseeable future.
parameters that matter. Indeed, we
Many people are employed in some
have too much “rule breaking” going
aspect of e-learning production
on because the majority of e-learning
and delivery, and far more will use
designers and developers are simply
e-learning in schools, as part of
unaware of what has gone before them.
employment training, or as a personal
In contrast but equally ineffective,
elective. With the rapid growth, fueled
many of the field’s new designers
almost completely by the economic
dogmatically follow examples of the
advantages of low-cost delivery,
e-learning applications they’ve seen.
it’s no longer a select few who have
They mimic pedantic applications that
access to e-learning systems and
have become so prevalent without
the opportunity design and develop
realizing there are many alternatives
instructional applications. Everyone
and better options. They need to
has the opportunity, regardless of
think simply about what’s truly
preparation.
needed before they launch into the
On the downside of today’s position,
development of yet another painfully
a painfully small percentage of people
boring learning application.
involved in e-learning have come into
Preface
xiii
the field well prepared. Others struggle
theory. We’ll include case studies,
to use their opportunities well, not
editorials, and a well-rounded view of
having learned the lessons taught so
learning technology as it evolves.
unforgettably to our pioneers by their experiments. Lacking direction, they
A Special Issue
As the first
mimic typical classroom events or the
issue in the
few e-learning applications they’ve
series, it seems
encountered. They apply basic notions
appropriate
of content presentation, operant
to me that
conditioning, TV games (Jeopardy),
we should take a bit of an historical
and post-tests. They may or may not
perspective. The field is young enough
write an instructional objective, but
that some of the pioneers who shaped
if they do, they do so mechanically,
thinking and created groundbreaking
often oblivious of the different types
systems are still available to directly
of objectives needed for different
share their experiences and learning
purposes, sanguine about the benefits
with us. We reached out to them
that listed objectives provide learners,
and found many gracious enough to
and naïve about the situations in
recount major events in their careers
which objectives may actually diminish
and in the formation of learning
learning. This is only one, perhaps
technology.
trivial, example of the areas of ill
I wanted to create, and hope we
preparedness we find so prevalent
have succeeded in creating, a valuable
today.
and interesting book that (1) captures
My hope for these Annuals is that
the excitement and expectations
they will provide truly helpful and
of those people who witnessed the
refreshingly up-to-date information
initial availability of computation
for e-learning designers, developers,
and communication technology for
buyers, and users. Each year we will
instructional uses, experimented
attempt to assemble some of the best
with it, and shaped its beginnings, (2)
insights about teaching and learning
summarizes the major lessons these
with technology as well as thought-
pioneers learned from this work, and
provoking questions and insights.
(3) provides direction for both today’s
We’ll look for a balance between
and future work.
applicable guidance and evolving
xiv
Michael Allenʼs 2008 e-Learning Annual
I asked contributors to recall their
questions, but as you enjoy reading
early visions, hopes, and expectations
David Merrill’s contribution, you’ll
of what computing technologies
encounter the story of his listening to
brought to instruction and learning.
a presentation by B.F. Skinner. I won’t
ÿ What were the new possibilities
steal the thunder of David’s account
and why would they be valuable?
here, but as he learned, thought
ÿ What were the risks and how
leaders of merit change their beliefs
would they be mitigated?
as they continue to explore ways of
ÿ What would learning technology mean for the individual learner? ÿ What did instructional
understanding. Open-mindedness is essential, both in scientific endeavors and creative ones. Regardless of one’s
technologies mean for schools,
perspective of whether instructional
universities, corporations, the
design is an art or a science, as the
military, and government?
leaders convey in their personal odysseys, important ideas come not
For example, did we expect a new
only from focused thought but also
competitive economy of learning
from casual conversations, being in the
opportunities to emerge, leading to
right place at the right time, and being
ever more refined designs and an
receptive to new ideas—regardless of
increased understanding of human
their origins.
learning? (I did expect this, but it doesn’t seem to have happened—yet.) I asked about lessons learned after
Learning what leaders were doing when they derived their principles and perhaps when they revised or
their decades of working with learning
supplanted them gives us a depth of
technology (perhaps in contrast to
understanding we can’t gain through
what’s become the general perception).
most publications. It also encourages
And I asked about future directions.
us to participate in the exploration
Redirecting their view to the future,
of learning technology, knowing that
but cognizant of experience and
powerful insights often come just from
events, what are their current visions,
being active in the field and being
hopes, and expectations?
willing to make mistakes.
Some contributors were a bit
Too often students perceive
shy or modest about sharing their
principles learned in classes and
wisdom within the context of these
from books to have an unassailable
Preface
xv
truth that their authors would never
An Invitation In years subsequent
claim. In e-learning, we often find
to this inaugural and special edition,
student designers fervently defending
we will be bringing our readers
dreadful designs while quite accurately
state-of-the-art work and thinking
quoting principles they’ve learned and
in the field of e-learning. In my
claiming, sometimes quite faithfully,
nearly forty years now of working
adherence to frequently taught
in this field, I continue to stumble
principles. I doubt any researcher
upon amazing accomplishments and
or author would adhere to his or her
powerful insights that have gone
own principle with such veracity,
largely unrecognized and unknown.
especially in the face of the appalling
This series of Annuals represents an
designs they justify. Instead, they
opportunity to unearth those building
would use the disappointing situation
blocks to a better future. If you know
as an opportunity to reexamine the
of work that deserves such exposure,
principle. The accounts contained
whether it’s your own or another’s,
herein, in fact, demonstrate that our
please contact me directly at Annual@
thought leaders have done exactly that
alleni.com or through the Pfeiffer
throughout their careers.
Editorial Staff at 989 Market Street, 5th Floor, San Francisco, CA 941031741 (www.pfeiffer.com).
Section One
Visionary Pioneers Continue to Guide Us
C. Victor Bunderson C. Victor Bunderson’s work began in academic settings at the University of Texas and Brigham Young University (BYU), where he led an early lab and became principal investigator on the $5 million NSF-financed TICCIT computer-aided instruction project. He has been a co-founder of several companies involving advanced computer and information technologies and education and training applications. He also served as vice president for research management at the non-profit Educational Testing Service in Princeton, New Jersey, where he guided research projects designed to develop and investigate a new generation of tests more relevant to learning. He returned to academe in 1991—at BYU—to work with a cadre of talented Ph.D. students in the Department of Instructional Science (later, Instructional Psychology and Technology). His work spans many decades and many types of organizations and has led him to validity-centered design, an interdisciplinary synthesis of measurement science, the learning sciences, and change management. This work has also built the foundations for learning progress systems based on domain theories of learning and growth. He is currently leading an ambitious validation update project for Herrmann International, updating and revising the Herrmann Brain Dominance Instrument for the new millennium. This is an individual differences instrument widely used in many countries and twenty language translations.
Reflections on TICCIT
C. Victor Bunderson
This article presents the early experiences of the author and his colleagues on the courseware development team for the TICCIT computer-aided instruction system. It reviews the history of TICCIT using words of that time, and then assays to explain lessons learned during and since TICCIT times in three main areas: (1) learner control—the high goals and ideals held out for this concept and their difficulty of attainment; (2) progress maps culminating in mastery assessments—use of progress maps with navigation based on well-measured progress information; and (3) differentiated roles and technology integration (how design experiments using differentiated staffing models can serve as a replacement for premature evaluations and can lead to attainment of some of the hopedfor high expectations for TICCIT and other innovations in what CAI has become). These three areas were selected to capture some of the essential issues that drove the early TICCIT project and that remain problematic today. Some History
In the October
lot further. TICCIT and PLATO were
1971 the MITRE
the two large projects in a major $10
Corporation,
million initiative of the National Science
partnering with
Foundation’s Technology Innovations
the University
Group, under the leadership of Arthur
of Texas at Austin (UT Austin) CAI
Melmed. It was an NSF effort to jump-start
lab and Brigham Young University
CAI, after some promising beginnings at
(BYU), submitted a proposal for just
the Illinois Plato Lab and the UT Austin
under $5,000,000. TICCIT (Time-
CAI Lab, which had both received prior
Shared, Interactive, Computer-
NSF funding. With other funding these labs
Controlled Information Television)
had attracted, they were both positioned
was based on innovative concepts in
to undertake these serious efforts. Now,
hardware, software, courseware, and
thirty-six years after that first TICCIT
implementation and was designed to
proposal was funded, it is instructive to
produce a “Market Success for CAI.”
look back at those days and consider what
It is interesting to look back to those
was accomplished. Also, it is sobering to
heady times when this proposal was
remember what dreams and ideals remain,
funded, times when $5,000,000 went a
elusive, apparently just beyond our grasp,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
2
Michael Allenʼs 2008 e-Learning Annual
dancing away to the next decade and then
In that 1971 proposal, some interesting
the next, calling on another generation
claims were made, in several key areas,
to try again. One must have a certain wry
the system, the courseware, and the
humor to view the overconfidence, even
market. The bullet points that follow are
hubris, of those days, and to temper the
selected from a longer list in the proposal’s
accomplishments with the recognition of
executive summary, and are quoted
problems that remain unsolved.
exactly except for one or two minor
One way to summarize TICCIT
paraphrases (MITRE, 1971, p. i):
is to say that it was a synthesis of the contributions of three main
The System
groups—the systems engineering
ÿ Mass dissemination of CAI has
and project management expertise of
been inhibited by the high cost
MITRE Corporation, the CAI learner-
of student terminals, computer
controlled courseware technology
hardware, and courseware.
and team production know-how of
ÿ TICCIT has solved the hardware
Victor Bunderson and his colleagues
cost problem through the
at the University of Texas CAI lab,
imaginative use of commercial
and the instructional theories and
television technology and
design procedures of David Merrill and
low-cost/high performance
colleagues at BYU. The hardware itself,
minicomputers.
as well as the software and courseware,
ÿ The TICCIT terminal (at one-
embodied what became known as
tenth the cost of the nearest
Merrill’s Component Display Theory
equivalent CAI terminal) is able to
of Instruction. MITRE’s disciplines
deliver instructional movies, plus
for engineering and management of
interactive computer-delivered
innovative system solutions permitted
graphics, voice, and text.
it to use these instructional concepts,
ÿ Terminal costs will be very low,
rather than dictate system constraints to a
ranging from a low of 10 cents in
system designed for instruction should be
future home CATV applications to
built. Also, blended in was the courseware
perhaps $1.00 per hour for a profit-
team’s insistence on empiricism mixed
making, turn-keyed system leased
with theory and with a commitment to the
to a school system, including
individual learner in creating the interface
software and commercial cost of
between the learner and the system. The
loans and capital.
centerpiece was giving the learner control of that interface. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
3
The Courseware ÿ The equivalent of five full semester
ÿ Almost 20 percent of all junior college class contact hours are in
courses will be developed in
curricular areas chosen for this
English and in math.
program.
ÿ Learner control over the instructional strategy is a basic
Elsewhere in the proposal, we learn
tenet of the University of Texas’
that The MITRE Corporation had
instructional design philosophy,
invested $750,000 of its internal R&D
as a means to humanize education
funds in this program before the NSF
and minimize the inadequacies of
funding was obtained. In addition,
machinery.
the proposals detailed extensive plans
ÿ The University of Texas’ and
for coordinating with manufacturers,
Brigham Young University’s
publishers, school administrators, and
instructional design concepts
others to assure that the project would
replace courses of conventional
recruit additional investment and
instruction with completely
interest of businesses and customers
modular systems, greatly
alike in actually achieving the “Market
redefining the teacher’s role.
Success” after the NSF funding period
Dramatic increases in effectiveness
was complete. The two community
are sought through the use of a
colleges selected, Northern Virginia
systematic design approach.
Community College near MITRE and Phoenix College in Arizona, were the
The Market
sites where the evaluation occurred,
ÿ The TICCIT curriculum should
and where many lessons were learning
have a major, measurable, and nationally meaningful impact on two junior college student bodies. ÿ A new junior college is being
in subsequent years. MITRE Corporation had been developing a number of technological innovations in TICCIT in hardware
formed in the United States
and software for digital audio and
every two weeks, on average,
graphics. MITRE started with
and approximately 50 percent of
the name TICCET (Time Shared
first-time enrollees in U.S. public
Interactive Computer-Controlled
collegiate institutions in 1969 were
Educational Television) and very soon
in public junior colleges.
thereafter changed it to TICCIT. They changed “Educational Television” to “Information Television,” thus
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
4
Michael Allenʼs 2008 e-Learning Annual
broadening the scope of application of
of the following considerations. Based
the technology to include home-based
on the work of UT doctoral student
information interactions over cable.
Dan McLallen (1971) MITRE and the
Their solution, integrating as it did
UT lab jointly choose community
video, color slide projection, and print,
colleges as the implementation site
was quite advanced for its time. Now,
and the subjects of freshman English
happily (even miraculously, for many
and freshman mathematics as the
writers in this book who pushed the
target courseware projects. McLallen’s
envelope of the ancestors of today’s
dissertation showed a high penetration
technologies at great human and
of credit hours that these two subjects
financial cost), the cost concerns dealt
would encompass out of the total
with in this proposal for hardware and
of most community colleges. The
software have been revolutionized by
penetration would be much greater,
progress in the information technology
and the project would be challenged
industry. The several racks of cards
with more remedial students by
and wiring for digital audio, graphics,
extending both the math and English
and the videotape players used in the
courses downward to encompass more
TICCIT system have been replaced by
building-block lessons from high-
a few chips, with a DVD drive if you
school and remedial levels. When the
wish.
amount of courseware to be developed
The original hardware featured two
was dramatically increased, Brigham
NOVA 800 minicomputers, a terminal
Young University (BYU) was selected
processor, and a main processor. The
to develop these additional lessons.
terminal processor was designed to
As head of the courseware
handle communications with 128 Sony
development and authoring system
color TV sets adapted as terminals.
team, I evaluated the resources needed
TICCIT systems were rolled out
to pull off such a large courseware
over a couple of decades following
development project and found that
the end of NSF funding by a series of
my resources at Texas would be taxed.
corporations, first by Hazletine, then
I looked to a respected colleague,
Ford Aerospace, then Loral. It had a
Dr. M. David Merrill, with whom
relatively long life, more in military
I had developed a strong working
and industrial applications than in its
relationship during his time at the
intended target, schools.
George Peabody School at Nashville.
The courseware team was consolidated at BYU in 1972 as a result
One of my doctoral students at UT Austin, Paul Merrill, was studying a
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
5
computerized version of Dave Merrill’s
impact on the design and features of
hierarchical “imaginary science”
TICCIT. Merrill’s component display
lessons for his dissertation at the
theory of instruction was at the heart
time. When Dave Merrill moved from
of the interaction design, and he
Peabody to BYU, he was alive with
practically single-handedly worked
excitement about the unusual and
out the details of the advisor program
coordinated resources at BYU for the
we had determined was essential
design and development of innovative
in a learner-controlled system with
learning technologies. Darrell Monson
navigational maps. Combining forces
had risen to a high administrative
helped reduce costs substantially for
position at that campus and had under
travel and for duplicate equipment.
his direction an outstanding movie
Also, it reduced maintenance and
studio, learning labs, and radio and
management problems of a split team
TV station. With Merrill, Monson
(it was hard enough to manage it at
was using his influence in setting up
one location!). These were some of
a graduate program in technology
the reasons the consolidation was
in education—we later named it
favored by MITRE and NSF. I brought
Instructional Science. I felt that the
key graduate students and courseware
opportunity to apply these resources
experts from UT, including colleagues
to the TICCIT project through a team
Dr. Gerald Faust and C.J. Ross, math
united in one location would greatly
professor Dr. Ted Boessenroth,
enhance its likelihood of success.
systems programmer Steve Fine and
Instead of subcontracting the
his able wife Claire, Tom McMurchey,
remedial portions to Merrill’s team
and graduate student Fred O’Neal and
under Darrell Monson at BYU, with
his capable wife Harriet. Like Claire,
counsel from MITRE and NSF, and
Harriet became a manager within the
with understanding support from
Courseware Production Factory. Dave
Wayne Holtzman at Texas (who was
Merrill’s team included Drs. Harvey
the real founder of the Texas Lab and
Black, Ed Green, talented graduate
my mentor), I decided to move the
students, including Andy Gibbons and
project to BYU. Up to that time, only
Roland Blake, and many others. These
a planning grant had been given to the
and other people who joined the team
UT CAI Lab. This move was of great
later comprised the Courseware team,
importance, as Dave Merrill and his
which was consolidated at BYU in the
colleagues at BYU had an enormous
summer of 1972. Those joining the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
6
Michael Allenʼs 2008 e-Learning Annual
team after the initial consolidation are
WICAT company. This company later
not listed, but their contributions are
split due to business reasons, but
not forgotten.
showed that business is happy to use
This team worked out the details of
the best of either system. Another
learner-controlled courseware. Steve
instance of this: Bunderson served for
Fine and Tom McMurchey provided
a time as vice president of research
the main technical interface with
and psychometrics for TRO Learning,
the MITRE Corp engineers. With
the company that bought PLATO.
other members of the courseware
Heuston’s Waterford school and
team, they later joined Hazletine
Institute is today a strong contributor
to see the TICCIT system through
to computers in early education.
its generations, which included the
Bunderson also founded what today
migration of TICCIT to PC’s—MICRO
is called Alpine Testing Solutions
TICCIT.
and The EduMetrics Institute. Dave
In reference to such market
Merrill was instrumental in several
successes the TICCIT system
start-up companies, and his consulting
did attain, and the many spin-off
to large training companies, such
companies influenced by TICCT
as NETg, has taken ideas developed
alumni, it can be argued that team
during TICCIT times out in many
production of componentized,
directions. Other TICCIT alumni
model-based courseware built on
have been involved in many e-
the separation of content from logic
learning ventures. Of particular note
has perhaps led to more “market
is Dr. Robert Mendenhall’s career.
successes” in online interactive
He started as an undergraduate pre-
learning projects and in many
law student who helped Bunderson
companies than any particular system
manage the TICCIT lab during and
on which it was embodied. Faust,
after the evaluation period. He gave
Merrill, Ross, and Bunderson and
up law school to became general
others started Courseware Inc. Dr.
manager, then president of WICAT,
Dustin Heuston and Bunderson co-
became head of computer education
founded WICAT Systems and the
at IBM, got a Ph.D. from the BYU
Waterford school, and recruited
department of Instructional Science,
many members of the TICCIT team.
and is today the president of Western
Heuston and WICAT’s president, Bob
Governor’s University, a fairly recent
Mendenhall, later agreed to merge
“market success” as the nation’s first
WICAT and PLATO into the PLATO-
fully competency-based university. As
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
7
discussed below, it embodies several of
lessons, but also their attitudes of
the most important lessons from what
responsibility toward learning and
was right and what fell short with the
their eagerness and confidence in
early TICCIT.
taking on new learning opportunities.
Since hardware and software technology change so rapidly, the disciplines of design-science-based development and team production and the architectural features pioneered in TICCIT have provided some stability to enable predictable and productive
The Evaluation of TICCIT’s First Year in the Colleges
NSF provided an additional $1 million so that researchers at Harvard University and
courseware production. Courseware,
the Educational Testing Service could
after all, is where learning may occur.
evaluate both TICCIT and Plato. This
Many concepts and people from
they did with skill and depth, visiting
TICCIT have indeed had a broad
both before the implementation and
influence in the market for what CAI
later collecting outcome data for
has become.
a year in the two colleges. Patrick
“Learner Controlled Courseware”
Suppes provided a succinct summary
was the byline of TICCIT. It was
of the evaluation of TICCIT in an
meant to suggest that instructional
important article on the state-of-the
materials—Courseware—could be
art of CAI in1979. Suppes was the
developed by a principled process
main consultant called on by Eric
that assured a high likelihood of
McWilliams, the very effective NSF
instructional success, and that
program manager in site visits to
the elements of that courseware—
evaluate the progress of the BYU and
instructional variables—could be
MITRE team. Suppes kept our feet to
placed in the hands of the learners
what he regarded as the appropriate
through a “learner control language”
fire. He did this by drilling into one
implemented through a special
topic during each visit; nailing us to
keyboard. The TICCIT courseware
the wall on the issue during the visit.
team tried to convey with these words
After the NSF visitors left, we got
that the expertise learners could gain
down off the wall and tried to correct
through becoming proficient with
our sins of omission or commission.
the learner control language would
While his summary is terse, it is
accelerate their progress, not just
backed by his extensive personal
through the TICCIT math and English Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
8
Michael Allenʼs 2008 e-Learning Annual
knowledge of the TICCIT project (Suppes, 1979, p. 189).
“(5) The success rate of students who took the TICCIT mathematics more than once seemed to indicate that the courseware did
“The TICCIT project had the responsibility to develop two communitycollege courses, one in English and one in
not provide sufficient remedial depth to teach some of these students. “These results are not terribly surprising.
mathematics. The curriculums of the two
It seems to me important that we do not have
courses are fairly standard and will not be
some immediate evaluation of CAI on the
reviewed here. The more distinctive feature
basis of a single year’s test as in TICCIT or
of the TICCIT courses has been the effort to
PLATO. It is rather as if we had had a similar
use an explicit instructional strategy focused
test of automobiles in 1905 and concluded
on learner-controlled courseware (Bunderson,
that, given the condition of roads in the
1975; Bunderson & Faust, 1976). The
United States, the only thing to do was to stay
Educational Testing Service (ETS) evaluation
with horses and forget about the potential of
of the TICCIT courses, as summarized quite
the internal combustion engine.”
objectively in Bunderson (1977), presents the following conclusions (see also Alderman,
Suppes ends his review of the ETS evaluation appropriately in cautioning
1978). “(1) When used as an adjunct to the
against jumping to premature
classroom, TICCIT (like PLATO) did not
conclusions. Ending an evaluation
produce reliable, significant differences in
after one year is indeed a poor
comparison with classes that did/not use
evaluation strategy. It was expensive
TICCIT (or PLATO).
enough for NSF, but I now recommend
“(2) When used as an integral scheduled
that evaluation strategies should take
part of either mathematics or English classes,
on a design experiment flavor by
TICCIT students did significantly better than
spreading costs over more cycles of
non-TICCIT students.
evaluation and revision.
“(3) Characteristics of the teacher are significant in determining the performance and the attitude of students in both TICCIT
Focus of This Article
Many stories can be told of
and non-TICCIT classes, a conclusion that
these times,
matches much other research of a similar
stories generated
sort.
by the hard
“(4) There was a difference of about 20
and soft technologies, the
percent in completion rate in favor of non-
instructional psychology and research
CAI classes for the TICCIT classes.
adventures, the sobering realities of
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
9
implementation in organizations, for
statement, and what did we learn—
example, the irony that educators who
during TICCIT and afterward?
were often thrilled with the words about learner control and learner
Stories not included here:
initiative but had a need to stamp
Instructional Theory, Component
it out when it actually started to
Display Theory, and Concept
occur. Some of the most interesting
Elaboration Theory, which was born
stories involve the people who passed
during the TICCIT project at BYU.
through the labs at UT Austin, BYU,
Instructional theory stories are best
Courseware Inc., WICAT, Institute
told through the writings of M. David
for Computer Uses in Education,
Merrill, Charles Reigeluth, and others.
and other companies that built on
Other information on TICCIT and its
TICCIT concepts. Except for the few
features can be found in the excellent
side comments, however, in this short
book prepared by Dave Merrill, Ed
article I will be content to tell just
Schneider, and Kathy Fletcher (Merrill,
three stories about lessons that have
Schneider, & Fletcher, 1980). This
endured; about important problems
book provides details of the TICCIT
that remain unsolved. These three
learner control strategy, the testing
stories are about:
strategy, the materials development
1. Learner Control. How far did we
methods, and many other details not
actually go toward understanding
found in this article.
and realizing the depth of what lies beneath this term? 2. Progress Maps Culminating in
Learner
Before TICCIT,
Control
the UT CAI lab
Mastery Assessments. How well
team conducted
did we understand the importance
learner-control
and the science at the base of these
research by giving
artifacts? 3. Differentiated Roles and
the learners more and more control within the hierarchical structures
Technology Integration. We boldly
of prerequisites used at that time to
proclaimed in the proposal that
model the domain. Also, the learner
the teacher would not be replaced,
controls were constrained by the
but that teachers’ roles would be
type of tactical options we had then
greatly differentiated. How little
imagined that learners would need
did we really understand that
(see Judd, Bunderson, & Bessent,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
10
Michael Allenʼs 2008 e-Learning Annual
1970). This research was equivocal
the nature of the “controls” interaction
on comparing the effects of learner
designers give to the learner are vital.
versus program control on criteria
But unless the learner controls act
of efficiency and effectiveness.
within a system that itself is modular
Although the UT team built larger-
and model-based, how can a learner
than-usual systems,, it was doubtful
come to grasp the meaning of the
that the scope of the online learning
controls? Imagine a pilot in a cockpit.
experience was extended enough in
A good pilot has learned to understand
engagement time to determine to what
immediately the meaning of multiple
extent the ability to control one’s own
dials and gauges and instinctively to
learning may be a skill learned slowly
move the steering wheel or joystick,
and requiring substantial practice.
often reaching simultaneously with
Looking back from today’s
the other hand for other controls.
perspective, it is easy enough to ask:
Based on an interpretation of
Why not? Proficient reading itself,
information seen through the window
library searching, and now Google
and from the displays reporting
searching is a skill that manifests
measurements, the pilot can take
very wide differences in individual
just the right action required to
proficiency. In 1971, we had begun to
further a mission in the face of the
suspect that learner control might take
unexpected. The information displays
a long time to develop, but we wanted
present information that permits a
to believe that the TICCIT courseware
knowledgeable pilot to infer actual
experience would be extended enough
states of what key abstract constructs
to show an effect. At least, we thought,
refer to. These constructs include
questions about learner control
altitude, speed, wind direction, and
had a better chance of permitting
geolocation.
proficiency to be developed during
These readings interpreted by the
the TICCIT courses than in earlier
pilot all have a theory behind them
supplementary lesson systems.
and have physical consequences that
When proficiency in directing one’s
will follow lawfully and often very
own learning is seen as a domain
quickly. They are not ad-hoc points of
itself, perhaps this “learning to
decision in a learning content space
learn” proficiency is as valuable as or
that has no coherent structure or
more valuable than the outcomes in
theory behind it. What information
mathematics or English TICCIT was
and controls should be given while
designed to teach. If this is so, then
learners are engaged in a mission
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
11
in a learning space? By analogy,
non-cognitive components: volition,
information should be presented that
motivation, persistence, and voluntary
ties into a coherent set of abstract
approach versus avoidance. Other
constructs related to progress and
terms are conation, affect, attitudes,
location in a learning domain. This
and dispositions. Quoting again from
interpretive framework will reveal
the 1971 proposal to capture the words
a model of the domain. It will
and ideas of that time (MITRE, 1971,
certainly contain learning milestones
p. 73):
and goals and status information, such as TICCIT provided with the
“In summary, minimum evaluation of the
Objectives and Status display. This
success of this project is dependent only on
TICCIT map provided access to
criteria of cost, efficiency, and effectiveness
objectives describing what goals were
relative to baseline data in junior college
to be attained and colored status
math and English. . . . However, the
information depicting progress. Based
courseware design is styled and structured
on possible interpretations of this
in fundamental ways to conform to values
information, the controls the learner
and suppositions about the future which
needs should offer great utility in
be beyond stated performance objectives. .
enabling the learner to reach those
. . The goal is to encourage self-motivated,
goals safely and expeditiously, from
skillful independent learning which will lead
wherever a learner may be within the
to further self-initiated encounters with the
domain at a given time—and despite
subject matter by as many students as can be
whatever difficulties that learner may
induced to accept the challenge.”
have encountered in getting there. The nature of the domain of
How well did we do in achieving
proficiency in learner control is
the non-cognitive goals? The
certainly central in understanding
ETS evaluators discussed them as
learner control. Another issue became
background ideals (Alderman, 1978, p.
the focus of considerable attention
29):
before and during the TICCIT project. It was this: What other characteristics
“Goals, after all, tend always to exceed
besides (cognitive and psychomotor)
our grasp. . . . Since goals influence
proficiency might develop through the
decisions, it seems reasonable that such
use of learner control of instructional
statements balance an enthusiastic optimism
variables? We are dealing here
about potential accomplishments and an
primarily with variables having strong
historical realism about results of prior
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
12
Michael Allenʼs 2008 e-Learning Annual
work. Here we took the developer’s goals as
do to help the students attain these
earnest statements of intent which should
simple fundamentals.
guide, though not dictate, our attention in evaluating the TICCIT program.”
Thankfully, the ETS evaluators
Still, there was enough positive evidence to cheer us, even in that difficult first year that the evaluation
did not seriously try to evaluate our
targeted. Some of the students
attainment of these goals during that
really loved it. Some of the students
first year of learning how the two
succeeded where likely they otherwise
colleges were going to cope with what
wouldn’t have. On average, once they
we had produced. Many adjustments
had completed TICCIT, their scores
had to be made to get the best results
were higher than those in the lecture
out of TICCIT. Now the usual term is
groups.
“Technology Integration.” We went into the implementation
In the TICCIT design experiment described below in connection
thinking that the planning was
with the third TICCIT story, and as
adequate. The systems engineers
implemented in the differentiated
at MITRE put capable Ned Burr
staffing model of Western Governor’s
on the job, and he worked with us
University, we have found approaches
to produce a massive document
that will work well for technology
called an “Implementation Plan.” It
integration. “Technology Transfer” is
seemed to cover everything, training,
not an adequate term. It is not enough
documentation, dealing with people’s
to transfer tools to a group organized
attitudes. We academics were very
in a conventional manner, following
impressed with this evidence of
established roles and rules. The
MITRE’s experience in implementing
biggest role shift is in the students’
large-scale projects, but we learned
role. It is necessary to reorganize roles
how little we had really understood.
and rules substantially and to teach the
Students unfamiliar with TICCIT
students new ways of organizing their
CAI—with any CAI at that time—
time and methods of learning. These
would first have to learn how to
new patterns of learning include both
take action to work on the system
individual work and learning to work
and complete their lessons, let alone
with cohorts of students with similar
change their proficiencies to learn,
goals. These changes result not in
and their attitudes and dispositions.
technology transferred into an existing
Teachers would have to learn what to
system, but in what really should
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
13
be described as a new system. Since
and improved through riding up the
conventionally organized educational
learning curve of technology that they
institutions find it difficult or
begin to compete with the established
impossible to implement such radical
markets.
changes in the roles and rules around
Back to the Learner Control Story:
which they have been organized
Evaluating CAI, including TICCIT
for decades, even for centuries, the
as a disruptive technology does not
technology is too disruptive to use in
take TICCIT’s learner control strategy
the ways that might work best.
off the hook. It is clear that the
Disruptive Technologies: Clayton
controls provided and the information
Christensen uses the term “disruptive
displayed on the maps was not
technology” in his book Innovator’s
sufficient to achieve the broader vision
Dilemma. He explains how a
of what learner control is all about. For
technology that the current system has
one thing, it did not span an adequate
difficulty using can become a market
range of preferences as shown in
success: He points out that the way
different models of thinking and
a disruptive technology might finally
learning preferences. My colleagues
achieve a market success is to skirt
and I have conducted extensive
the existing customer base and go to
validation research with the Herrmann
market niches underserved by the
whole brain model of thinking
existing technologies. In education,
preferences using large international
organizations like University of
databases of surveys taken over the
Phoenix, Western Governor’s
last twenty-five years (Herrmann,
University, and other online
1988—see validation appendix;
universities have found such a niche—
Bunderson, Wiley, & McBride, 2007;
thirty to forty some people with family
Olsen 2007). This model interprets
and work responsibilities who cannot
four clusters of .preference choices,
travel to campuses and take classes
made under scarcity. By scarcity,
scheduled during their prime work
we mean that the respondent has
time. As these organizations achieve
limits on how much can be chosen in
market successes in a new niche that
expressing preferences. The preference
does not require the same definitions
clusters form at opposing ends of two
of roles and rules of procedures as the
bipolar factors. One bipolar factor
campuses, a point is reached when the
favors analytical, logical, mathematical
offerings of the disruptive technology
thinking at one end and interpersonal,
vendors become so well understood
emotional, thinking favoring an
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
14
Michael Allenʼs 2008 e-Learning Annual
interest in human relationships at
synthesizing tasks at a high level of
the other. Experience has shown that
generality. They needed practice going
these preference clusters are relevant
beyond fixed lists of items into larger,
to what type of learning environment
more synthesizing projects, perhaps
and instruction work best for an
even leading to capstone projects that
individual.
tie lower subdomains and objectives
TICCIT designers provided a lot for those who favored the logical left brain. There were plenty of displays TICCIT designers provided a lot for those who favored the logical left brain.
together. The TICCIT English project had its writing assignments, and our
involving logic
greatest documented successes with
and algorithms,
TICCIT came through freeing the
found in the
teacher’s time up by letting TICCIT
rules and helps.
teach the grammar, mechanics, and
Unfortunately,
organization lessons. This enabled the
what many people-oriented students
teacher to spend more time managing
needed was to work in smaller or
and mentoring writing assignments.
larger groups with others. They could
Otherwise, the TICCIT learner control
work around open tables on their own
strategy was all within the organized,
TICCIT terminals, but did not prefer
step-by-step type of thinking favored
the walled-off carrels in the TICCIT
by only one of the four quadrants of
lab. They preferred more human
thinking preference.
interaction, and we eventually learned how to provide it. The other bipolar factor in the
Can we really hope for dispositional changes in students when they are given more choice of controls that
Herrmann model features preferences
really matter and plenty of time
for order, organization, and step-by-
to practice it? The jury is out on
step methods at one end and creative
some of the goals we had, such as
synthesis, taking risks, seeing the big
enhancing a voluntary, eager approach
picture at the other. TICCIT was very
to learning. The doctoral work of
organized and orderly, with a place
Margaret Martinez (1998) poses a
for everything and everything in its
challenge to the supposition that
place. What some of the students
dispositions are really very modifiable.
needed was to see the big picture more
She has identified four “learning
completely than the TICCIT map
orientations,” the transforming,
hierarchies conveyed. They needed
performing, conforming, and resisting
better options to explore and try out
learner. These “orientations” may
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
15
shift through learning, but apparently
the behavior noted above—TICCIT
only with great difficulty. Martinez
students who over-used the advisor to
teaches instead how to maximize
find out just how many more problems
the effectiveness of learning using
they had to work in order to pass out
the orientation you have already
of that lesson.
established. Does Learner
Going way
to genetic nature or nurture and life
Control by Any
beyond the
experiences, a transforming learner
Name Have a
Whether one’s orientation is due
is going to want to see the big picture and jump around a lot, if given the
Future?
highly structured TICCIT approach to learner control,
learner controls. The conforming
do instructional psychologists/
learner wants to be told just what to
technologists today see anything
do, and what steps to follow to do
in the rapidly changing world of
it. The resistant learner is not going
technology—any evidence of the eager
to buy in at all, but when s/he does,
motivation that impels the learning of
look out, this may be a transforming
any content, logic, or skill required?
learner in disguise, one who can
Perhaps. We see the young give
come to love learning. A person in
big parts of their lives to becoming
resistant orientation may be one who
amazingly proficient in role-playing
has refused to buy into seemingly
games. We see incredible Massively
stupid demands and assignments
Multiplayer Online Role-Playing
administered in a structured
Games (MMORPG) like World of
educational system they hate. Some
Warcraft that require the development
kinds of learner control may work
of superior social and leadership skills
to effect a leap of members of this
to organize people and material for
group from resistant to transforming
successful campaigns of conquest
learning.
and acquisition. As John Seeley
In most environments, the
Brown (Brown & Thomas, 2006) has
orientation forming the biggest group
pointed out, the skills attained by a
of learners is called “performing
seventh level Guild Master are the
learners.” They know how to learn and
skills needed to organize people and
will tolerate the system, but their goal
accomplish complex and important
is to minimize effort in checking off
missions in corporate global business.
the attainments the system demands. They have attitudes that could lead to
The motivations are there in games. These are the powerful motivations
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
16
Michael Allenʼs 2008 e-Learning Annual
the TICCIT designers dreamed
It is a broad synthesis of navigational
of for learning environments, but
utility made subject to the gamer’s
could not attain through structured
growing intelligence by a growing
lessons focused directly at learning
knowledge of what the different
objectives. The funny thing is, the goal
measures of attainment mean. Games
of a game is not learning. Learning
might connect in a sort of oblique way
is an incidental by-product of deep
with academic curricula in schools and
investment in the goals of some
colleges, but that connection is hidden
imaginary game-world. These are
and incomplete at best.
larger goals that matter to the people
Perhaps most important, life in an
who choose to make the investment.
MMORPG is life in a meritocracy.
Education has broken the subject
True, you can buy your way to more
matter into neat little packages that
rapid advancement with enough
build on one another to permit a class
money, but you still have to perform to
credit that enables one to take another
stay there, and continue to perform to
class, then another, until one earns a
attain more. Thus complex multiplayer
degree.
games are meritocracies. Schools and
The role restructuring for the individual learner is there in a game world. In an MMORPG, what the
many other human institutions are hardly meritocracies. In searching for the motivational
learners control is a comprehensive
power to attain high levels of
set of functions. They choose from
proficiency and to gain strong
and also shape available goals. They
dispositions toward continued
control the methods for obtaining
learning, the designers of TICCIT
the means needed to seek the goals.
made some very small steps toward
They learn skills
the non-cognitive goals, and toward
and strategies for
the idea of high levels of learning
fighting to keep
proficiency. I am impressed with the
and add to the
online meritocracies in the open-
Gamers have multiple measures of progress at hand.
means toward the goals they value,
source world of the Internet and in the
and so on. Gamers have multiple
game worlds. An increasing number
measures of progress at hand—their
of online communities are structured
level of character attainment in each
as meritocracies, such as Slash
of their avatars, their wealth, health,
Dot, wikis, and user communities
strength, possession of weapons,
associated with online tools,
possession of defenses, and on and on.
(especially open source tools). These
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
17
communities are structured around
and attention, the broadened concepts
pure technology expertise at present,
behind TICCIT maps are likely to
even though more effective and faster
receive increased attention.
transfer of expertise might occur with
After working at WICAT for a
the application of learning design and
time, I had the opportunity to work
measurement disciplines.
at Educational Testing Service as vice
It is easy to lament their lack
president of research management
of recognition of instruction,
during a period of intensive work
measurement, and the like in the new
on the goal of developing new
meritocracies of digital expertise. It is
applications of testing integrated
easy to lament that a little technology
with learning. One ETS initiative in
expertise often trumps experience and
the research division during those
know-how in instructional technology.
days was to develop examples of
But let it grow. Maybe instructional
“Learning Progress Systems.” Such a
technologists can learn as much or
system was conceived of as having a
more from the digital natives as they
map or interpretive framework for
could (but probably won’t) learn from
valid measurements of progress along
the instructional technologists. Is this
meaningful pathways of learning
where the broader mission of learner-
progress. The concept was seen as one
controlled courseware is going to find
of several possible ways to implement
some fulfillment?
Systems which Integrate Learning and Instruction (SILAS; see Snow
Progress Maps Culminating in Mastery Assessments
Progress maps
& Mandinach, 1999). ETS continues
display learning
research with this intention under a
progress and
variety of names and descriptions.
goal information and provide
navigational controls. The TICCIT map (objectives and
I brought to TICCIT a background in measurement from Princeton and from a practical and theoretical education at ETS as a Psychometric
status display, with navigational
Fellow. I tried to assure that the
control) is a concept that has far
measurement systems underlying the
more depth than we realized at the
TICCIT practice files, status displays,
time. While the RULE EXAMPLE
and advisor status information were
PRACTICE EASY HARD HELP
respectable. Dr. Edward Schneider
tactical commands have not sustained
joined the TICCIT team at BYU and
a large amount of continued research
developed an adaptive testing system
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
18
Michael Allenʼs 2008 e-Learning Annual
for the TICCIT tests, based on the
model” that integrated performances
Wald sequential testing approach
at a higher level (Bunderson, Gibbons,
(explained in Merrill-1980). Students
Olsen, & Kearsley, 1981; Gibbons,
did not need to take all the test items
Bunderson, Olsen, & Robertson, 1995;
in the file. They could pass out or fail
and Gibbons & Fairweather, 1999,
out more quickly.
Chapter 15).
The interpretive framework feature
Charles Reigeluth struggled with
of a learning progress map, discussed
the problem of fragmentation versus
as crucial to learner control in the last
integration as a graduate student at
section, depends on having a theory
BYU during TICCIT times. Later, he
by which the measures of progress
has written on Concept Elaboration
can be judged. In TICCIT, we had
Theory (Reigeluth, 1999) in an attempt
produced a Gagne-style “learning
to deal with unfolding sequences of
hierarchy” for each lesson and unit,
increasing complexity, rather than
and indeed, for the entire course. The
fragmented learning hierarchies.
rules for producing these hierarchies
Returning to an academic
were based on analysis into the classes
environment at BYU in 1991, I worked
of learning described by Gagne in
with several colleagues and graduate
his 1985 book, The Conditions of
students in an effort to understand
Learning. These categories were
the measurement foundations for
modified somewhat to fit Merrill’s
progress maps and how to develop
Component Display Theory. Such a
them. Influenced by the work of
learning hierarchy produces one type
Samual Messick on validity, we
of theory of the domain, but I left the
developed concepts of a “Domain
TICCIT experience thinking that this
Theory of Progressive Attainments”
type of learning hierarchy could result
(Bunderson, 2006, Bunderson, Wiley,
in fragmented instructional pathways.
& McBride, 2007). A domain-specific
The TICCIT rule-example-practice
theory of progressive attainments
strategy worked well with concepts
is one candidate of replacing the
and simple rules and procedures, three
learning hierarchies used in producing
of the Gagne/Merrill types of learning,
the TICCIT maps. The definition of
but not so well with performances
validity presented by Messick (1995),
that were integrated at a higher
by the writers of the Joint Standards
level. We had noted this problem of
on Educational Tests (AERA, 1999),
fragmentation versus integration and
and by other validity theorists is the
had developed the concept of a “work
key to understanding the need for an
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
19
interpretive framework in a domain.
map provides, which is the essential
Validity resides in interpretations
meaning of modern conceptions of
and actions taken in response to an
validity.
understanding of what measurements
The idea of navigational control
mean; validity does not reside in a
throughout a courseware domain and
measurement instrument itself, or
the idea of meeting the conditions
in its scores. Therefore, if the users
of conjoint additive measurement in
cannot interpret the meanings of
the measures reported on that map
measures, they cannot respond
to guide learning progress are still
quickly and adroitly through controls
under development at the EduMetrics
whose use depends on a correct
Institute; also, it is found in the work
interpretation of the measures.
of a number of young Ph.D.s who did
Validity is lacking in such a system,
their dissertations on aspects of a
and the broader goals of learner
domain-specific theory of progressive
control discussed in the last section
attainments in areas of interest to
are unlikely to be achieved.
them (Bunderson, Wiley, & McBride,
As this work progressed, we discovered other work in a similar area that also represents a basically compatible
2007; McBride, 2005,; Strong-Krause, 2001; Xin, 2002). One of these former doctoral
and very promising approach to the
students is Dr. Thomas Zane, director
development of progress maps. This
of assessment in the NCATE-
work by a number of investigators has
accredited Teachers College of
been done under the heading Knowl-
Western Governor’s University. As
edge Spaces (see Falmagne, 1990, for
mentioned above, WGU has been able
an early account). A knowledge space
to implement a role-differentiation
is itself a rigorously defined domain-
approach that separates the roles of
specific theory of progressive attain-
teaching, assessment development
ments. It has a promising mathemati-
and administration, mentoring,
cal foundation. Knowledge spaces is a
tutoring, grading, and other matters.
candidate for the theory of the domain
The WGU approach includes social
structure needed as a foundation for
and organizational variables that help
developing the type of learning prog-
solve the problem of low completion
ress maps envisioned here.
rates manifested so sharply in the ETS
The emphasis in our work has
evaluation of TICCIT. Fundamentally,
been on assuring that learners can
WGU shifts the student’s role and
interpret and use the information the
separate a variety of teacher roles.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
20
Michael Allenʼs 2008 e-Learning Annual
WGU is a competency-based university with a method for defining
based on domain-specific theories of progressive attainments.
credible domains and developing aligned assessments. WGU starts
Differentiated
the process of domain definitions
Roles and
This last story describes
in a conventional manner, defining
Technology
how design
domains, subdomains, competencies,
Integration
experiments using
and objectives. Developing a full
differentiated
domain theory of progressive
staffing models as a replacement for
attainments in a domain would
premature evaluations can lead to the
require integration beyond the
attainment of more of the hoped-for
use of objective and competency
high expectations for TICCIT.
statements into pathways that reflect
The news of low completion
integration at higher and higher levels
rates from the junior colleges hit us
of complexity. Feasible steps toward
hard at the TICCIT lab at BYU. Now
this ideal are taken by Tom Zane,
reduced to a fraction of the staff we
whose team is encouraged by the
had during the TICCIT courseware
University officers to use integrative
development effort, we were still
performance measures, not just
burning to know how to make the
objective tests. Zane uses concepts
exciting TICCIT concepts work in
like work models, cited earlier, and
practice. It was clear that students
for assessment, makes heavy use of
didn’t already know how to use a
performance tasks, domain level
system like TICCIT to learn on their
projects, and capstone projects that
own, and that only a relatively small
integrate across domains. Zane’s staff
percentage of students really had the
uses these methods in developing,
characteristics Maggie Martinez later
evaluating, and improving new WGU
identified as those of a “Transforming
assessments. This assessment group
Learner.” These are the learners who
is able to take appropriate steps,
can learn on their own in almost any
sometimes small ones, sometimes
environment. We believed that the
larger, that fit the state of development
innovative TICCIT system was really
of the student population and the state
still in its toddler-hood and that its
of development of WGU. These are
users needed to learn many things to
steps toward realizing a fuller vision
help it grow up.
of measurement of learning progress
To do this, we found that faculty support at BYU was not strong.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
21
English faculty at BYU were polite
organizational conditions. Comparison
but disinterested. Math faculty would
from cycle to cycle depends on the
not use TICCIT in the appropriate
comparability of the measurement—
classes. TICCIT, after all, dealt with
their invariance properties—from
low-level community college material
one cycle to another. The cycle for
and did not honor math faculty
these college English classes was one
members’ vision of the importance
semester in length.
of their unified and dominant role.
During the first semester, several
Most English faculty members were
graduate student instructors taught
interested in teaching literature, and
TICCIT sections. We will examine
this left graduate students free to
the results of two of these student
teach writing with a fair amount of
instructors. The first one (Christine
freedom. The English faculty allowed
Hansen, who with advanced degrees
the TICCIT lab to recruit several
later became head of all the BYU
graduate students who were teaching
writing labs) tried hard to integrate the
sections of English in preparation for
computer. TICCIT had a set of tutorial
the university-wide required writing
lessons in grammar and mechanics.
exam.
It also had writing lessons dealing
All sections at BYU took the same
with audience, purpose, structure,
test at the end of each semester. The
and outlining. The second teacher
test consisted of two parts; a multiple-
did not believe in the computer, but
choice grammar and mechanics
was confident that she could teach
test and a written essay graded by
students to write. The baseline and the
teachers according to generally
results of the first cycle of the design
accepted standards. It is unusual for
experiment are illustrated in
a university to develop one standard
Figure 1. This figure uses the term
measurement system for all sections in a particular domain, but this was one of those unusual times. The tests
TICCIT: The J curve of Implementation in a Design Experiment
were scored by classical means and lacked desirable invariance properties, but were suitably comparable for
100% % Passing General Ed Writing Exam
the large effects noted. A design experiment takes place over repeating
65 – teacher who ignored TICCIT, Taught students to write Baseline: 55%
cycles of implement/evaluate followed 40%
by redesign, revise the social and
Cycles 0 and 1
45 – our teacher who tried hard to use TICCIT
Semester cycles of design experiment
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Figure 1. Results After One Cycle of TICCIT Design Experiment
22
Michael Allenʼs 2008 e-Learning Annual
“J-curve of implementation,” borrowed
the classes taught by all teachers in
from economics. The curve of percent
the study. TICCIT generated extensive
passing drops before starting up again.
data on student progress in every
“Cycle 0” was merely to
lesson and every segment of every
obtain baseline data from all the
lesson. From observation and data,
conventionally taught English
It was easy to see that students had
sections, so future cycles of design
no habit patterns for how to learn on
and improvement could refer back
their own with a computer. As was
to it. The results of Cycle 1 were
found with the community colleges,
not promising for the teacher who
the computer was greatly underused
sincerely tried to use TICCIT as
in the BYU lab, although the BYU
it was supposed it should be used.
college students appeared to have
On average only 45 percent of her
greater success in lesson completion
students passed the GE exam. The
that had the community college
other teacher said to her students
students. The committed teacher and
in a variety of ways: “Go use that
her support group designed better
computer we’re supposed to use in
implementation plans for the second
whatever way you wish, but here in
semester, plans designed to help the
my class I’ll teach you how to write.”
students understand their roles better,
The result of her efforts: 65 percent of
and the teacher to redefine her role
her students passed the GE exam. The
and her new rules of procedure. No
baseline average was 55 percent across
changes were made in the hardware or
the university. After one cycle, the
courseware, just in the implementation
logical conclusion would be to teach
of new roles and new rules. Students
the methods of the second teacher to
were organized into small groups
teachers in the other sections, and not
and scheduled to meet at computer
use TICCIT at all.
terminal tables, where they could see
But the English faculty had
and talk to one another. A “party” was
delegated Freshman English to
held in the computer lab to get them
graduate students, and we were spared
started and to assign tables and small
the risk of pre-judgment or premature
groups of cohorts. At the end of the
judgment.
second cycle, the TICCIT teacher’s
Christine Hansen believed she could
class did better than the second
organize and get much better results
teacher’s had the previous semester,
during the second cycle. The team of
72 percent compared to 65 percent,
researchers studied data available on
and substantially higher than baseline.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
23
Figure 2: TICCIT Individual Carrel (Above) and Open Group of Table-Mounted Terminals (Left)
More TICCIT lessons were completed
beginning of the semester, and they
in Cycle 2 than in Cycle 1.
came to the TICCIT lab at the same time, as though it were a regular class
Figure 2 illustrates one of the environmental adaptations used to create a social environment among the
period. Each group had a name, such as “Emerson” or “Thoreau.” Hansen, backed by the TICCIT
TICCIT English students. The upper
team, still believed she could do
picture features a very private, one-
better. This team also believed that
student TICCIT carrel. Designed with
the students, backed by professional
human factors expertise by Dr. Edward
developers and researchers, could
W. Schneider, this carrel featured no
do much better, so more changes in
glare on the screen, a stand for books,
roles and rules were made. At the
spaces on each side—privacy for
end of the third cycle 84 percent of
only one person. The bottom photo
that semester’s group of students
shows a long table with no partition,
in the TICCIT section passed the
designed by Scheider as well. With
GE exam. The students in these
TICCIT terminals on each side, this
subsequent semesters accepted their
arrangement enabled students at the
role to learn and practice grammar,
table to see their friends and talk
mechanics, audience, and purpose
over the table and to those seated
on the computer, and the teacher
at each side. Their regular seats and
spent more time with small groups
their group of six was set up at the
motivating and mentoring them in
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
24
Michael Allenʼs 2008 e-Learning Annual
how to write. The team made further
for working with small groups. Using
refinements and tried again. At the
the reports, those not making good
end of the fourth semester, 93 percent
progress could be identified early and
of this new group of TICCIT students
encouraged and taught individually,
passed the two-part GE exam.
while most of the class was involved
Significantly more TICCIT lessons
in practicing and studying on the
were being passed, and the scores on
computer or working on their writing.
the objective grammar and mechanics
This large effect was attributable
test were significantly higher. Even
to variables entirely from the
higher was the improvement in writing
social and organizational realm
scores—this was the key to the strong
of implementation or change
performance of the TICCIT-using
management. There were no NSF or
teacher’s students on the writing exam.
University funds left at that time to
See Figure 3 (Below).
redesign the hardware, software, or
The of implementation Implementationinin TheJJ Curve curve of TICCIT TICCIT English—Cycle English – cycle44
TICCIT English – cycle 4
100% % Passing General Ed Writing Exam
93
courseware. Bob Mendenhall was an undergraduate research assistant during these TICCIT studies. He went on to become general manager, then president of WICAT Systems. This company recruited a
Figure 3. TICCIT Design Experiment Results Over Four
good number of the TICCIT team
Baseline: 55%
members. As president of Western 40% Semester cycles of design experiment
Semesters
Governor’s University, he now uses as a standard management
The improvement in Cycle 4 has an effect size of .96 from baseline, and even higher from the low-point in the J curve. This high pass rate for Cycle 4 was attributed to the teacher beginning to use the reports generated by the computer system to identify the number of TICCIT lessons the students were completing. This implementation tactic was added to further refinements in the previous successful implementation methods
strategy the best measures he can get for all aspects of the University’s functioning, and insists on making revisions and following up to see whether the measures improve in the desired direction. It is not as tight as a design experiment, but does not require publication of results, and it is cost-effective for an organization. The WGU model of differentiated staffing is one viable model for elearning in higher education. The
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
25
model is documented at one stage of
or orientations toward learning will
its development Mendenhall’s (2001)
prove difficult or even impossible
dissertation, but due to continuing
to change. It may be better to find
data-based revisions since that time,
strategies to help learners become
the differentiated staffing model and
more successful within the constraints
other aspects of the WGU model have
of different preference profiles,
shifted a bit since that time.
learning orientations, or dispositions. Looking ahead, systems radically
Conclusions
Projects and
different from TICCIT’s highly
systems come
structured approach to learner control,
and go. People
perhaps involving games, may offer an
move on to
essential environment of meritocracy;
other challenges
also, these environments may offer an
and other accomplishments. As in
indirect approach to attaining learning
every field of human endeavor, some
outcomes, but a direct approach to
problems in succeeding with CAI are
attaining greater proficiency to learn
very difficult to solve. Three stories
what is needed to achieve other valued
were told about TICCIT and about
goals.
continued efforts to solve a class
The TICCIT story about
of problem. In the Learner Control
navigational maps with valid progress
story, a key problem is capturing
feedback shows that measurement of
enough motivation in an e-learning
learning progress with deep validity
system to attain the broad goals of
is a high goal, difficult to achieve.
increased, transferable proficiency in
Valid measurement depends on
learning, strong and eager approach
theory-connected interpretations.
toward further engagement with the
Validity of inferences drawn from
subject, and a strong sense of personal
progress measures and goals lies
responsibility for learning progress
in the interpretations and actions,
and learning attainments. It is not
not in the numbers produced by
clear that some of the non-cognitive
the online measurement process. If
traits associated with successful
constructed properly, navigational
learning are teachable and learnable,
maps can provide feedback within
even with an ideal learning language,
an interpretive framework fully
progress mapping interface, and given
understandable to the leaner. Within
long engagements in systems that use
this framework, learners can identify
them. It may be that some preferences
their location in a learning domain and
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
26
Michael Allenʼs 2008 e-Learning Annual
determine where to move next. It is
been attained. Design-experiment-
very hard to put together the tools and
like changes over cycles should
resources in hardware, software, and
replace premature evaluations of
implementation that came together
innovations. This form of data-based
for a brief moment to produce
improvement based on promising
TICCIT. Thus, whether the type of
models and theories can lead to the
measurement-based navigational maps
large effect sizes that the TICCIT
and other tools will work as hoped is
designers hoped for. The design
uncertain. Can a deeper conception
experiment presented in this article
of progress maps be implemented
did occur with TICCIT within two
with navigational controls designed
years of the ETS evaluation. The
for learning proficiency in well
principles it taught, however, can
understood learning domains? Will
be effected with a variety of types of
these tools enable learners to attain
systems that integrate technology with
proficiency and motivation sufficient
learning. Large effects are possible
to use learner controls adroitly
with well-designed and executed
enough to select the best options for
implementations of what CAI has
immediate and longer-term progress?
become. Successful implementations
Perhaps another generation will find
will involve role differentiation for
the answer.
students and teachers, and will involve
The third TICCIT story involved
integrative projects that both stretch
differentiated staffing and cycles of
students, and measure their progress
improvement in competency-based
toward high, educationally valuable,
environments. Both during TICCIT
and intrinsically motivating learning
and later, successes in implementing
challenges.
differentiated staffing models have
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
27
REFERENCES AERA, APA, NCME. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Alderman, D.L. (1978). Evaluation of the TICCIT computer-assisted instructional system in the community college. Final Report, Volume I. Princeton, NJ: Educational Testing Service. Brown, J., & Thomas, D. (2006, April). You play World of Warcraft? You’re hired! WIRED, 14(4). Bunderson, C.V. (1973, September). The TICCIT project: Design strategy for educational innovation. In S.A. Harrison & L.M. Stolurow (Eds.), Productivity in higher education. Washington, DC: National Institutes of Education. Bunderson, C.V. (1977). A rejoinder to the ETS evaluation of TICCIT (CTRC TR 22). Provo, UT: Brigham Young University Press. Bunderson, C.V. (2006). Developing a domain theory: Defining and exemplifying a learning theory of progressive attainments. In M. Garner, G. Engelhard, M. Wilson, & W. Fisher (Eds.), Advances in Rasch measurement, Volume I. Maple Grove, MN: JAM Press. Bunderson, C.V., & Faust, G. W. (1976). Programmed and computer-assisted instruction. In The psychology of teaching methods (75th Yearbook of the National Society for the Study of Education), Part I, pp. 44-90. Chicago, IL: University of Chicago Press. Bunderson, C.V., Gibbons, A.S., Olsen, J.B., & Kearsley, G.P. (1981). Work models: Beyond instructional objectives. Instructional Science, 10, 205-215. Bunderson, C.V., & Newby, V.A. (2006). The relationships among design experiments, invariant measurement scales, and domain theories. In M. Garner, G. Engelhard, M. Wilson, & W. Fisher (Eds.), Advances in Rasch measurement, Volume I. Maple Grove, MN: JAM Press. Bunderson, C.V., Newby, V.A., Olsen, J.B., and Wendt, D.C., (2007) Validity of the HBDI four-quadrant and left-right structure: A structural validation using a new millennium database, Herrmann International Validation Update Technical Report, EduMetrics Institute, Provo, Utah Bunderson, C.V., Wiley, D., & McBride, R.H. (2007). Mapping attainments to enable learner-centered education: Designing for the integration of assessment with instruction. In C.R. Reigeluth, Instructional-design theories and models, Vol. III. Mahwah, NJ: Lawrence Erlbaum Associates. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
28
Michael Allenʼs 2008 e-Learning Annual
Christensen, C.M. (1997). The innovator’s dilemma: When new technologies cause great firms to fail. Cambridge, MA: Harvard Business Press. Falmagne, J. (1990). Introduction to knowledge spaces: How to build, test, and search them. Psychological Review, 97(2), 201–224. Gagne, R. (1985). The conditions of learning (4th ed.). New York: Holt, Rinehart & Winston. Gibbons, A.S., Bunderson, C.V., Olsen, J.B., & Robertson, J. (1995). Work models: Still beyond instructional objectives. Machine-Mediated Learning, 5(3 & 4), 221-236. Gibbons, A.S., & Fairweather, P.G. (1999). Instructional strategy III: Fragmentation and integration. In A.S. Gibbons & P.G. Fairweather, Computer-based instruction: Design and development. Englewood Cliffs, NJ: Educational Technology Publications, pp. 278-296. Herrmann, N. (1988). The creative brain. Lake Lure, NC, Brain Books. Judd, W., Bunderson, V., & Bessent, W. (1970). An investigation of the effects of learner control in computer-assisted instruction prerequisite mathematics (MATHS). Technical Report No. 5, Computer-Assisted Instruction Laboratory, University of Texas at Austin, Texas. Martinez, M. (1998). An investigation into successful learning—Measuring the impact of learning orientation, a primary learner-difference variable on learning. Dissertation for Brigham Young University (UMI# 7972948). www. trainingplace.com/source/research/Martinezdissertation.pdf McBride, R.H. (2005). A domain theory of fluent oral reading. Doctoral dissertation, Brigham Young University, Provo, Utah. McLallen, D. (1974). Analysis of the need and economic feasibility for computer-assisted instruction in high volume junior college courses. Dissertation: The University of Texas at Austin. Mendenhall, R.W. (2003). A model and principles for effective internet-based distance education. Doctoral Dissertation accepted by Brigham Young University, Provo, Utah. Merrill, M.D., Schneider, E. W., & Fletcher, K.A. (1980). TICCIT, in Langdon, D.G., Series Editor, The Instructional Design Library Volume 40, Educational Technology Publications, Englewood Cliffs, New Jersey. Messick, S. (1995). Validity of psychological assessment. American Psychologist, 50(9), 741-49.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on TICCIT
29
MITRE. (1971). Toward a market-oriented computer-based learning system (Phase III— Implementation, Validation, Field Test, and Evaluation). K. Stetten, principal investigator. McLean, VA: MITRE Corporation. Olsen, J.B., Bunderson, C.V., Newby, V.A., and Wendt, D.C., (2007) Validity of the HBDI four-quadrant and left-right structure, A structural validation using a database to Year 2000, Herrmann International Validation Update Technical Report. Provo, UT: EduMetrics Institute, Reigeluth, C.M. (Ed.). (1999). Instructional-design theories and models: A new paradigm of instructional theory. Mahwah, NJ: Lawrence Erlbaum Associates. Snow, R., & Mandinach, E. (1999). Integrating assessment and instruction for classrooms and courses: Programs and prospects for research. Princeton, NJ: Educational Testing Service. Strong-Krause, D. (2001). English as a second language speaking ability: A study in domain theory development. Doctoral dissertation, Brigham Young University, 2001. Dissertation Abstracts International, DAI-A 61/12, 4746. Suppes, P. (1979). Current trends in computer-assisted instruction. Advances in computers, Volume 18. San Diego, CA: Academic Press. Xin, M.C. (2002). Validity centered design for the domain of engaged collaborative discourse in computer conferencing. Unpublished doctoral dissertation. Provo, UT: Brigham Young University.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Patrick Suppes Patrick Suppes has published widely on educational uses of computers and technology in education, as well as in philosophy of science and psychology. Much of his research has been focused on detailed physical and statistical models of EEG- and MEG-recorded brainwaves associated with processing of language and visual images and continued development of computer-based curriculum in mathematics, physics, and English. He is a member of the National Academy of Education (1965), the American Academy of Arts and Sciences (1968), the National Academy of Sciences (1978), and the American Philosophical Society (1991). He has received the American Psychological Association’s Distinguished Scientific Contribution Award (1972), the National Medal of Science (1990), Lakatos Award Prize, London School of Economics (2003) for his 2002 book (Representation and Invariance of Scientific Structures), and the Lauener Prize in Philosophy (2004 Switzerland). Currently, Suppes directs the Education Program for Gifted Youth (EPGY) at Stanford University, which offers online courses in mathematics, physics, and English to pre-college students of all ages, with approximately four thousand part-time students worldwide. The program recently added the creation of an online high school whose first students started in August of 2006. Suppes is also doing research on the brain, with emphasis on language and visual images.
Addressing Diversity in (e-)Learning
Patrick Suppes
In conversation with Michael W. Allen As one of the most revered names in technology-assisted learning, it was an extreme honor that Patrick Suppes took time to talk with me about his career, how the idea of using computing technology to assist learning developed, and how he came to write his seminal Scientific American paper that caught the attention of scientists, educators, and the public alike. I wonder whether it’s a coincidence that both Suppes and Bitzer are physicists, that both demonstrated leading-edge success with learning technology, and both see individualization of the learning experience as a key to the future of an enlightened society. In any case, both scholars had the knowledge, confidence, and capability to shape and harness technology, and after demonstrating phenomenal successes years ago, they’re frustrated that e-learning is so poorly designed today and so often misses the big opportunities. There are important lessons to be learned here, and I’m extremely grateful that Professor Suppes was willing once again to share his insights and profound wisdom. —Michael W. Allen Allen: Dr. Suppes, as someone who
in terms of computer learning in the
has looked seriously at learning with
future. It might be useful to use them as
the support of technology, beginning
a basis and give you my comments on
at the point when computer interface
where I feel we have failed and where
capabilities first made computers
we have succeeded. The first article
usable by learners, I especially
is “Modern Learning Theory and the
appreciate your taking time to
Elementary-School Curriculum,” first
contribute to this first edition of the
published in the American Educational
e-Learning Annual.
Research Journal. This was the paper I gave when I was invited to address the
Suppes: I have some themes to
AERA meeting in 1964. The second is
consider, and I thought we could use
“The Uses of Computers in Education,”
two well-known articles of mine from
first published in Scientific American in
the 1960s to explore them. In these
1966. This was after Dick Atkinson and
articles I talk about what we should do,
I received a grant from the Carnegie
and what we are going to be able to do,
Corporation to establish a laboratory.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
32
Michael Allenʼs 2008 e-Learning Annual
Allen: Okay, now let’s back up a bit.
As we wrote it, we realized we could
Can you tell me what led you up to
give it an educational learning theory
this point?
“twist.” We came up with the idea of computer-based instruction. That’s
Suppes: Dick and I wrote a book
how it all started, actually: out of a
of experiments on learning theory
proposal for an automated laboratory
called Markov Learning Models for
from Carnegie. John Gardner was
Multi-Person Interactions, published
president of Carnegie’s non-profit
in 1960. In this book we anticipated
foundation and very supportive of us.
much of the experimental economics
In our first experiments in late 1962
going on now. We experimented on
and 1963, we brought elementary-
various formats such as two-person
school children to the labs at Stanford.
learning games. We developed what
We soon saw that this wouldn’t work,
we considered to be an automated
as the numbers of learners would in-
laboratory to run experiments initially
evitably be small. Our first step into a
in the area of the psychology of
school was to install a teletype, con-
learning. Carnegie was looking for
nected by telephone to our computer
good projects to support, and we were
at Stanford, in an elementary school
encouraged to send in a proposal.
ten miles from Stanford. That was
Figure 1. (Above) First demonstration of a logic program to third graders in an early CRT terminal, Stanford Lab, 1963 Figure 2. (Right) Fourth grader using teletype machine, Grant School, 1965. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Addressing Diversity in (e-) Learning
33
in 1964. We had to put it in a broom
Suppes: My background was really in
closet, since enrollment then was
physics. I had an undergraduate major
bulging. Students had to go in there to
in physics and was pressured into a
work on it one at a time.
specialization in meteorology during
Out of this first trial came the
World War II, where I served as a
realization that we had to move the
meteorologist. I received my bachelor’s
terminals, because we didn’t have PCs.
degree at the University of Chicago
We actually ran terminals from some
and a U.S. Army Commission as a 2nd
central point until the early 1980s,
Lieutenant at the same time, in 1943.
when PCs became available.
I was then immediately sent overseas,
In 1972, after having taught it
as they needed meteorologists in the
twenty-two years and having written
Pacific. I went to the Solomon Islands,
a textbook, I offered a completely
one of the more remote places on
radical idea: a computerized version
earth, and was even stationed at a
of my Introduction to Logic course.
relatively remote part of those islands.
I was only the master of ceremonies
After the war, I went on to take a Ph.D.
on the first day of class. The students
in philosophy at Columbia. Since then
worked entirely on the computer, and
my philosophy work has been mainly
the results were entirely evaluated by
on the philosophy of science and logic,
the computer program. There were
so in that way I have kept my scientific
no human lectures, only an assistant
boots on. Fairly early I started working
there to help with problems. I taught
in mathematical and experimental
that course every quarter for the next
psychology. It sort of became my
twenty years—sixty times—until I
scientific side, and I have used what
retired in 1992, at the age of 72. The
I have learned from this also in my
course, too, was retired at that time.
philosophy studies.
It was a big experiment. Similarly, from about 1955 I had been teaching
Allen: So, as you devised an e-
another course, Axiomatic Set Theory.
learning course based on your book,
It was a good undergraduate logic and
did you envision it simply as a labor-
math course at a more advanced level,
saving device? What was your goal
and starting in 1974 I taught it three
with the computer-based version of
times a year as a computerized course.
your course?
Allen: Now what was your
Suppes: Several things. We could
background?
turn to my 1966 article in Scientific
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
I C
34
Michael Allenʼs 2008 e-Learning Annual
American. It was about the uses of
Now, what follows from that is an
computers in education. I would stress
idealistic program that has been only
two things. What is striking, when
partly realized. We would like to have
teaching something like logic, is the
the interaction between student and
wonderful, vast individual differences
computer be like that of a very good
in the rate of student learning. Some
dialogue between a student and a
will take to it like ducks to water,
smart tutor. We have made progress
while others have problems. The
on such dialogues, but we have a long
individual differences are huge. The
way to go.
pace of learning varies tremendously across students. The single most
Allen: Before we talk more about the
important cognitive argument for
future. What is your assessment of
using computers, from an intellectual
what is happening today? What are we
standpoint, is the detailed and
doing well and not doing well?
deep accommodation to individual differences in student learning. The second associated point is that
Suppes: Let me mention a few things from the Scientific American article,
sitting and listening to lectures is a
and talk about where we are in regard
passive activity. I have a behavioral
to them. I mentioned being very
norm for the classroom: we teachers
active. That we do well. In the courses
talk too much. I have a rule on
I have been associated with, be it
the computer in I have a behavioral norm for elementary school. the classroom: we teachers No lecture should talk too much. be longer than three minutes. One and
teaching language arts to third graders or teaching Chinese to sophomores in college, we require a good amount of active participation on the part of the student: thinking creatively and
a half minutes is much better. Then
answering questions, not just listening.
you have active exercises to complete.
We have done a pretty good job with
This is even true in the logic course
this activity component.
in the university level. Great stress is
We have also done well regarding
on an active role for the student, not
diversity in learning. I have what I call
passive listening. Individualization for
the “cone of diversity.” Along the x-axis
learning and reducing the passive role
we plot how long a student spends
of the student are the major virtues of
working on the computer in a course,
computerized courses.
and along the y-axis we plot what the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Addressing Diversity in (e-) Learning
35
student has done thus far for each time period on the x-axis. In elementary school, for example, one would talk about grade placement as a progress indicator. At the university level one would talk about how much of a course has been completed. Now the fastest student’s progress would appear on the left. But this student’s curve couldn’t be vertical, because it takes some real time to do anything. The slowest student’s progress would appear almost horizontal, but it couldn’t be completely horizontal, because that
Figure 3. Sample
would be an infinite time. Thus, the
Computer-Time
fastest student adheres closely to
Trajectory
the y-axis, while the slowest student continues close to the x-axis. Everyone falls, in principle, somewhere between the “cone” formed by these two curves. The
sixth or seventh grade. And then you have students who will take longer. Probably my best example of
differences between the fastest and
taking individual differences into
slowest student can be, cumulatively,
consideration is my work in the
very large. For example, in my current
1970s and 1980s with deaf students
activities here at Stanford with the
learning to read. Of course, it is
Education Program for Gifted Youth,
very difficult for them with their
all pre-college, my colleagues and I
auditory language deficiencies. Some
have created a large computer-based
would take hundreds of exercises to
math course for grades 1 through 7.
make progress in elementary school
For these eight years of curriculum,
reading—an amazing amount of
the best students can easily finish
time—but they would make progress.
three years early. The very best are
We are talking about students
phenomenal, taking the Advanced
genetically deaf from birth, but of
Placement Exam in calculus in the
unaffected intelligence. We are talking
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
36
Michael Allenʼs 2008 e-Learning Annual
residential schools around the country.
without any instructional design
Individual differences can be a striking
background became involved, and
phenomenon, and I believe we did a
what these people were fascinated
good job accommodating their needs.
with was the medium’s ability to present information. They would say
Allen: Do you think this is true in the
that the individualization comes into
training environment as well as in
play because you can read it at your
education?
own speed. Well, using that logic, a book is pretty individualized, too.
Suppes: Well, as we both know, there is extensive use of computers to
Suppes: Well, that’s too simple of an
teach people in industry, but I think
idea of instruction!
the answer is no, and I’ll tell you why. I think there is too much tendency
Allen: Right!
in the training industry to lecture people and not enough to check up on
Suppes: In their favor, let me say this.
what they are understanding, or to let
The biggest course and the biggest
the ones advancing
enrollment we have at Stanford in the
I think there is too much whiz ahead, and give
Education Program for Gifted Youth
tendency in the training special attention to
is the K-7 math course. I have been
industry to lecture people those who may not
working on this course since 1992.
and not enough to check be progressing as
This was preceded by the writing of
up on what they are quickly. Most training understanding,... courses, but not all, are not sophisticated
a series of textbooks for K-7 math in the 1960s, followed then by additional computer coursework during the
enough to take into consideration
whole period from 1965 to 1992.
the individual differences involved
So I am saturated in the subject. I
in learning. Perhaps the main
have been constructing the course
reason is cost. It is expensive to
in detail. It’s a huge course, with
develop instruction that is highly
more than forty thousand exercises.
individualized.
Most corporations feel they cannot put that kind of time and money
Allen: I agree. As I began work in the
into perfecting instruction trying to
field, it was individual differences that
do something much simpler to save
fascinated me. But as the Internet
money.
came around, it seemed that people Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Addressing Diversity in (e-) Learning
37
Allen: But then I ask, “What is the
best and worst students may differ,
most expensive form of instruction?”
if you have a criterion of success in
And my answer is “instruction that
learning, by an order of magnitude.
doesn’t work.”
The curriculum has to be organized, and it’s not trivial to organize it to
Suppes: I agree with you. There aren’t
accomplish that.
many educational training groups in corporations that have been producing
Allen: And the benefit of adapting to
any sophisticated learning products
individual differences?
over fifteen to twenty years. There isn’t a culture of really accepting 100
Suppes: Well, like your remark about
percent what can be done with good
the worst thing being not to teach at
instructional design, even in places
all, the worst thing is to be wasting
like IBM.
time. Let me put it this way. I have always been concerned with the top
Allen: Any other comments on your
and bottom 10 percent of students.
assessment of where we are today?
The bottom 10 percent need help and time to get up to speed so they don’t
Suppes: Connected with the notion
fail. The top 10 percent need to be able
of individualization is the question of
to move on and learn more according
how to individualize. To individualize
to their ability. We need to take care
you need sophisticated, dynamic
of their readiness and abilities. The
concepts of curriculum. I have, for
ordinary thing to do when you don’t
example, created technical specs for
have the ability to individualize is to
a program to track student progress
teach to the middle; that’s what good
in the current K-7 math course. It
textbooks do necessarily.
is a complex document describing a
Now let me review a couple
complicated, stochastic motion. To
of things from the other article I
provide for individual differences
mentioned, “The Uses of Computers
requires, on the production side, a
in Education.” These things will
much more complicated conception of
sound familiar, but the problem is
the curriculum. You just can’t provide
to implement them in a curriculum.
for individual differences in some
One point concerns taking advantage
simple way, such as giving learners
of and understanding the nature of
one set of exercises, for example.
transfer. It is a well-studied subject in
The number of exercises done by the
educational psychology and learning
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
38
Michael Allenʼs 2008 e-Learning Annual
theory in general, but not really well
the program, because it’s something
understood, even by curriculum
you really can’t do operationally in
faculty in schools of education. Out
classrooms or in groups. I consider
of these studies come other learning
a criterion of learning based on
criteria such as response latency, the
latency a very important thing—and
time between a test stimulus and
a sophisticated thing—to add, from a
the response. Can a student work
learning standpoint. It’s as important
an exercise fast enough to show real
in language learning and many other
mastery? This is hard to measure in
subjects.
the classroom, but the computer is just tailor-made for it. You can measure
Allen: I remember your talking about
learner responses to the millisecond.
“Old Church Slavonic” years and
You don’t need this, of course, as
years ago.
one-tenth of a second is good enough for ordinary work. Everyone has
Suppes: Yes, I have a 1981 edited
recognized for a long time in learning
volume of the computer-assisted
theory, particularly in education,
instruction we did at Stanford for
that response latency is a much more
the university-level courses. We
sensitive measure of mastery, in many
offered a whole range of language
kinds of learning, than just correct
courses: Armenian, Arabic, Chinese,
responses.
French, German, Russian, and Old
For example, in many basic parts
Church Slavonic. I used to joke
of mathematics, you want students
about Armenian. Stanford wouldn’t
to have answers at their fingertips,
offer Armenian, because there were
so to speak. They should be able to
never enough students. We had, at
do computations with great accuracy,
most, about one student a year for
and also with speed, so they can move
Armenian. But we actually had people
ahead with a competent base. I have
who found it useful, and we received
math races in my K-7 math course. It
a special grant to develop courses.
is assumed the student can calculate
Scholars who were studying that part
the correct answer. The exercises are
of the world or the history of the
only meant to build more competency
revision or who wanted to read more
in terms of swiftness, consolidating
documents in that language needed to
the base of learning. It’s been very
learn Armenian.
successful, and I love that part of
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Addressing Diversity in (e-) Learning
39
Allen: You usually have to have
associated language arts courses at
a large population to justify the
the elementary level, instructors
development costs, but sometimes a
do not like trudging through stacks
very small population can justify them
of individual writing exercises.
just as well.
Everywhere, teachers are backing away from doing this. We are building
Suppes: It will not be an object of
production programs whereby
extensive experimentation. It may
students are given exercises in reading
not be the best possible course, but
comprehension, for example. We give
in the vein of that great line by Oscar
them a fixed vocabulary of no more
Wilde, “If a job’s worth doing, it’s
than thirty-five to forty words to use
worth doing badly.” And here, in the
in composing a sentence. Then we
case of Armenian, the student wanting
write a big context-free grammar,
to learn the language would prefer
formally, as a computer program. The
something that did the job, even
program can do a pretty good job of
though it hadn’t been given the loving
checking for grammaticality. We are
attention, say, of a course that had an
also building a semantic analysis of a
enrollment of a thousand students per
correct paraphrasing that can be used
year.
to assess answers to a question. It’s
I should that mention we are
quite successful already and we will, of
currently working on two language
course, improve it with experience. I
courses that are meant to be
can see why teachers are shying away
completely computer-based. One is
from manual assessments, for we can
a big ESL course, and we’re starting
easily accumulate twenty thousand
from scratch. The other is a mirror
written exercises with a sizable class.
image of this: Chinese. A teaching
We are also using this new
assistant is there to help out with
technology in both the ESL and the
problems. In both of these computer-
Chinese course. We have students
based courses, we do some rather
writing and also writing down what
sophisticated things that match
they listen to in this way. We can do
some earlier predictions of mine.
accurate evaluations with a restricted
Let me amplify this remark. One
vocabulary. We use, as I said, a very
of the things that is coming to
detailed and very large context-
fruition in these language courses is
free grammar. There are some good
sophisticated recognition and analysis
computer parsing programs available
of student-constructed text. In the
for such languages.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
40
Michael Allenʼs 2008 e-Learning Annual
Allen: Are you thinking about
restricted areas of instruction like
integration of voice recognition?
mathematics are a good environment in which to do it. You put your finger
Suppes: Absolutely! We haven’t done
on something I mentioned we needed
much successfully up to now, but we
in 1968 and thought we would have
have a lot of work going on with voice
in another ten years, by the way. I
recognition. One of my objectives,
think it is unlikely there are currently
when I finish some things I am
any big production courses anywhere
working on currently, is to implement
really using voice recognition in any
voice recognition on a production
major way, but it will come. It’s not too
basis. In the K-7 math course, we
difficult to predict future functionality,
have already implemented a test basis.
but timelines are very challenging,
I think this is very important, and
indeed.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Addressing Diversity in (e-) Learning REFERENCES Suppes, P. (1964). Modern learning theory and the elementary-school curriculum. American Educational Research Journal, 1, 79–93. Reprinted in H. C. Lindgren (Ed.), Readings in educational psychology. New York: John Wiley & Sons, 207–222. Reprinted also in R. Ripple (Ed.), Readings in learning and human abilities. New York: Harper & Row, 1971. Reprinted also in H.C. Lindgren & F. Lindgren (Eds.), Current readings in educational psychology (2nd ed.). New York: John Wiley & Sons, 1971, 216-230. Reprinted also in the Bobbs-Merrill Reprint Series in Psychology, P-810, Prod. No. 69065. Japanese translation in W. H. Holtzman (Ed.), Computer-assisted instruction, testing, and guidance. New York: Harper & Row, 1970. Suppes, P. (1966). The uses of computers in education. Scientific American, 215, 206–220. Reprinted in Information: A Scientific American book. San Francisco, CA: Freeman, 1966, 157–174. German translation: Anwendungen elektronischer Rechenanlagen in Unterricht. In Information Computer und Kunstliche Intelligenz. Frankfurt am Main: Umschau, 1967, 157–172. Reprinted also in Mathematical thinking in behavioral sciences: Readings from Scientific American. San Francisco, CA, and London: Freeman, 1968, 213–222. Russian translation in Informatsia. Moscow: Mir, 1968, 165–182. Japanese translation in Information, Scientific American book, 1969. Polish translation: Zastosowania maszyn cyfrowych w nauczaniu-tlum. Tadeusz Wiewiorowski. In Dzis i jutro maszyn cyfrowych. Warsaw: 1969, 231–256. Reprinted also in Computers and computation: Readings from Scientific American. San Francisco, CA: Freeman, 1971, 249–259. Reprinted also in Contemporary psychology: Readings from Scientific American. San Francisco, CA: Freeman, 1971, 257–267. Suppes, P. (Ed.) (1981). University-level computer-assisted instruction at Stanford: 1968-1980. Palo Alto, CA: Stanford University, Institute for Mathematical Studies in the Social Sciences. Suppes, P., & Atkinson, R. (1960). Markov learning models for multi-person situations. Palo Alto, CA: Stanford University. For a full bibliography and PDFs of Patrick Suppes’ articles on e-learning, see http://suppes-corpus.stanford.edu.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
41
Donald L. Bitzer Donald L. Bitzer received his Ph.D. in electrical engineering from the University of Illinois in 1960. He was professor of electrical and computer engineering at the University of Illinois from 1960 to 1989, when he retired from the University of Illinois to become a Distinguished University Research Professor in the Computer Science Department at North Carolina State University. Bitzer’s work has involved applying signal processing and coding theory to a variety of areas, from radar signals and speech processing to the development of software and hardware required for large computer networks. The large educational computer systems “PLATO” and “NovaNet” are results of this research. His research led to the intelligent modems for telephone lines and cable systems as well as the flat plasma display panel now being used for television. He has been granted numerous patents in the computer and electronic areas. Recently his research has focused on using signal processing and coding theory to look for genomic information that controls the translation process in protein production. He is also involved in developing new methods for convolutional coding to improve communications in the presence of noise.
PLATO, Plasma Screens, and Computer-Based Education
Donald L. Bitzer
In conversation with Michael W. Allen I was so fortunate to direct research and development on advanced instructional systems for Control Data and work with the University of Illinois on the PLATO system—an invention of Donald Bitzer and his colleagues at the Control Systems Laboratory. Don’s limitless energy and enthusiasm led him to tackle challenges most people couldn’t even conceptualize, and he inspired many of us, as he continues to do today at North Carolina State University, to take on meaningful challenges, not just tasks that are obviously doable. Professor Bitzer was, as always, gracious and enthusiastic as I interviewed him about his life with PLATO, his hopes, and his evaluation of learning technology today. He continues to quickly connect multidisciplinary knowledge and focus on potential solutions to major societal needs. As a partner in such pursuits with the late William C. Norris, CEO of Control Data Corporation, and a recipient of major federal funding, Bitzer developed learning technology that was a fantasy playground for educators. Knowing that educators limited their vision to what was currently possible and therefore had difficulty imaging, let alone specifying what they needed from interactive technologies, he set out to create the most boundless system for exploration. Even today, some of the capabilities of PLATO have yet to be matched by other systems. Sagacious as he always is, you can’t talk with Professor Bitzer without his raising your sense of importance and obligation to do something significant with your life. It was such a privilege to interview him about what drew him to develop technology for learning, what he feels were the most important things he learned from the experience, and where he feels we need to turn our attention today. —Michael W. Allen Allen: With a background in electrical
without failing. That was a landmark. It
engineering and minors in math and
encouraged us to look into instructional
physics, what attracted you to using
uses. The University of Illinois,
computers as a learning tool?
under the prodding of Shal Sherwin, a professor of physics, set up a fairly
Bitzer: The real reason was that
large committee of scholars, teachers,
computers had started to become
computer engineers, and education
reliable enough to run for forty hours
experts to look into the possibilities.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
44
Michael Allenʼs 2008 e-Learning Annual
They met for several months to
us would end up supporting them in
determine whether computers could
increasingly unsustainable ratios.
be used effectively for learning. I was
Although our funding came from
never invited; I had just finished my
the military, we were free to use the
Ph.D. and was working independently
money on what we wanted as long as
on electronic components directly
we submitted annual status reports.
related to such things.
They would look at what we did the
At the end of the discussions, it
year before and determine whether we
was decided that nothing could be
were to receive the next year’s funding.
done. Daniel Alpert, the head of the
Dan Alpert said, “Let’s take a gamble.
Control Systems Laboratory (CSL),
I think we should do this no matter
was told of the findings, and he wrote
what.” I agreed. We put a little money
a letter to the dean, but then asked
into it.
me to read through it to see whether
At the next meeting of the advisory
I agreed. I read it and said, “Are you
committee, we submitted a report
kidding? Give me a week, and we’ll do
about what we were doing. They were
something useful!”
ecstatic. They said, “We are the largest
So he held the letter up and within a
educational institution in the world,
week or so, we actually had something
and this is something we need as
going. Alpert never sent the letter
part of our military systems.” That, of
and told me, “Well, maybe they can’t
course, encouraged all of us. In a year
do anything, but we can. Get going!”
or so, our PLATO (Programmed Logic
With the help of a colleague, Peter
for Automated Teaching Operations)
Bromfeld, to do some coding and a
project took over the whole lab!
technician half-time, we soon had a useful system. We were interested from the beginning in the use of technology in classroom learning because it seemed to me the majority of our large inner city schools were turning out functionally illiterate students– students who couldn’t really contribute to the workforce in any significant
Figure 1. PLATO III Classroom,
way. That was scary. Such a trend
University of Illinois at Urbana-Champaign,
would take us all down, as the rest of
Sept 1968
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
Again, what drove me
45
in possibilities to synthesize a vision.
fundamentally was trying to solve
The Illiac computer system was, as
some of the problems facing our large
you know, one of the first digital
inner city schools. Even today, the
computers—developed in the 1950s at
majority application of the PLATO
the University of Illinois. Well, as years
system is to keep students in school
went on, more, smarter people, with
and raise their abilities. That is, of
different backgrounds and opinions,
course, where the money is, too, since
were brought in, and they helped grow
schools are paid for keeping students
it into something wonderful. What was
in school at the elementary and high-
needed to get started, however, was a
school levels. One hundred to two
single person with all the right skills in
hundred thousand students each year
place and the single vision in mind.
are brought back and kept in school using this technology.
Allen: Wikipedia says, “Donald Bitzer’s success comes in large
Allen: Why do you think the
part from his rejection of modern
University’s committee said learning
educational thinking.” How do you feel
with computers wouldn’t work?
about this?
Bitzer: I have my own opinion about
Bitzer: What does that mean?
this. The committee members were a
I’ve read quite a bit about my own
mixture of people who weren’t able to
history and found most of it to be
see their common needs as educators
bogus, but benign, so I have just
and the possibilities of technology.
left it alone. True, there are a lot
Each viewed the topic in terms of
modern educational ideas I disagree
his or her own area of study. It was
with: for example, that content
a multidisciplinary meeting, not an
is unimportant, only the delivery
interdisciplinary meeting. They just
matters. That’s nonsense. You are not
never got it together, and they were
going to teach something like math
sure we wouldn’t get it together either
or physics if the teacher doesn’t know
(although they all wanted to be a part
anything about the subjects but is
of it after the fact). I was a professor
very good at presenting slides. That’s
and a computer guy, so I could
not enough. In the early days they
consider the problem from multiple
would say, “Teaching is the medium”
perspectives. I knew how to use the
(implying whatever you teach with
Illiac system we had and could factor
is the teaching, and content is not as
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
46
Michael Allenʼs 2008 e-Learning Annual
important. That is, the medium itself
would come in to see whether things
is the teaching). I never understood
we were working on could be of use to
that. I disagreed with it. I spent zero
them.
time worrying about it. I don’t worry about the things Wikipedia says
Allen: What did they do with your
about me. Well, they didn’t say I was
findings?
an idiot; that would have been more provocative.
Bitzer: Most of the accomplishments that came out of CSL were used in
Allen: You had funding from the
a significant way in the military.
Army/Navy/Air Force group. Did
They ended up having better results
they exert direction or pressure or
by letting us do our thing. A good
guidance, or did they just sit back
example of this is the Electrostatic
and watch what you were doing?
Vacuum Gyro, commonly called the “Star in a Bottle.” It was used for long-
Bitzer: They gave money, sat back,
range guidance for submarines. It was
and watched. That was part of our
a gyroscope with a year’s precession
program. Being a university lab, they
on it or more, precession being the
looked at what we did after we spent
drift of the gyroscope axis. So there
the money. Knowing they could cut
was basically no drift, no torque
the funding for things they didn’t like
suspended in a vacuum. They are still
was a kind of pressure, I suppose.
the standards in submarines. The
But as far as I know, they always
same was true for side-looking radar
came through. Now, when they had
(counting the engines on airplanes
a positive attitude for things they
from fifty miles, even though the
really liked, they would really work
antenna you were using was actually
hard to get additional money from
quite small). We solved these problems
other agencies, as they did for the
because we were interested in them.
educational projects. If they weren’t
These things are still in effect today
really into a project, they had a way
for military use. We did contribute to
of subtly discouraging you from doing
their ability to carry out their mission,
it. But they never once came in and
even though that was not our charge.
said, “Here is what we want you to do.” Basically, we were working on more
Allen: Now these weren’t part of the
general, long-range problems, and they
PLATO Project, were they?
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
47
Bitzer: Same lab, but different
They were into to psychological
projects. There was too much to do to
studies (topics Congress was a little
be working only on one project. That
disturbed with anyway, for example,
was a nice thing about the lab—there
“Does pornography
were bright, creative people working
affect the soldier?”).
They were into... topics
on a variety of things and interacting
They were furious
Congress was a
across projects. We were formal
then when this
little disturbed with.
groups, but we worked together
project came
informally, helping each other, sharing
around. But just then the Mansfield
people and resources. As long as
Amendment came along, which
we were all contributing to moving
required a disclosure on what ARPA
forward . . . that was the important
was working on and its applicability in
thing.
civilian life. Our project was the only thing they had, so we became their
Allen: How about the learning technology. Did the military find useful things there?
hero. The answer is that, all along the way, including the idea of using PLATO for military instruction was
Bitzer: Yes. The government was very
supported by them, but we didn’t write
interested in our finding the “ten most
any of their materials. They wrote
important principles of computer-
their own courseware for medics or
based education.” Many parts of
pilots, and so on. They would later
PLATO were developed by other
contract our help.
people. I was an instructor and taught
In many ways we don’t know what
in the traditional classroom. An ARPA
all they did from an educational
contract to investigate CBE was signed
standpoint. I know they did Chinook
while I was in India. I was a little
vehicle maintenance and medic
upset, but decided I would take part of
training down in Texas. I understand
the money and design a nice model for
the FAA still uses applications you
teaching by computer for the students,
developed at Control Data, Michael.
and take the rest and develop a better,
Control Data Corporation (CDC)
more interactive student display. I
actually took over a lot of service
did that. This money came from the
to the government once you got
behavioral science branch of ARPA
going there. I know more about the
(Advanced Research Projects Agency).
systems that are used in education for
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
48
Michael Allenʼs 2008 e-Learning Annual
bringing kids back. I see that every
Allen: Would these new agencies
day. I see ten thousand students, K-12,
ever give you directives or specific
signing on for basic instruction in
goals?
mathematics, English, etc. Bitzer: No, we would write proposals Allen: Later funding came from the
for what we wanted to do. We only
national Science Foundation (NSF),
fought once, and that was over
right?
evaluation techniques—formative versus summative. Sometimes I
Bitzer: Yes, eventually we received
disagreed on the statistics. Statistics
funding from several places, including
lie, you know. The average heart is
the University itself and the Tri-
in terrific shape, but everyone dies
Service (Army, Navy, and Air Force),
from heart failure. Even there it was
CSL contracts (we had to get our
no real disagreement. NSF supported
own). NSF took an interest and
involvement of the Educational Testing
supported the first real system with
Service to provide evaluation, which
many terminals (twenty to thirty)
was fine with us. But it wasn’t clear
and the cable systems that supported
that ETS was prepared to do good
the distribution. The basis of the
evaluations of this new thing. It’s really
project, I should say, was centered
hard to evaluate new projects and
around the distribution of computer
technologies, because you aren’t really
display signals by cable television to
sure what measurement techniques
a university classroom, a high school
to use or what variables to monitor.
classroom, a nursing school, and a
Measurement of any new process is
community college. More support
inherently challenging. The effects of
came from them and ARPA when we
observation, for example, such as the
started expanding the use of plasma
Hawthorne Effect discovered at GE,
display panels. Various agencies
can invalidate results.
became big supporters after a few
ETS did the job, however. The
years. That would happen often in our
typical result was that teaching on
laboratory, even with other projects.
the computer yielded equal or slightly
After Tri-Service money, particular
better results than teaching in the
agencies would take an interest and
classroom. I would contend if we can
become the primary funder.
do as well as in the classroom, that’s a good start! I mean, you can start with this success and work for sustained
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
49
improvement easier than you can with
successfully answered the question
a classroom.
of how to use learning technology
It all worked out fine. We were all
to greatest advantage, but you
friendly in the end. The government’s
couldn’t have done better to provide a
support was taken over by CDC, who
powerful system with unprecedented
provided expensive equipment, ideas,
capabilities.
software, and courseware. We really had a great relationship with CDC. As you know, we jointly developed
Bitzer: Well, that was my intent. Different people tried different things,
a system with CDC to integrate
and many turned out to contribute
simultaneously developed system
in ways that we alone could not have
software enhancements. Every week
done. People you would never have
we shipped our software changes,
thought of as contributing turned
which had been running for three
out to do so in some way. There was
weeks without error, back and forth
an English teacher for rhetoric, for
to each other. We had about eight
example, a nun named Pauline. Now
hundred terminals on our system,
I thought it would be difficult to
and CDC had about two hundred on
judge rhetoric using a computer, but
your end. It was a remarkable system
she came up with a way. “One way
that allowed rapid development of
of learning to write well,” she said,
advanced capabilities and yet remained
“is to learn to edit carefully. And we
stable for on-going instructional use.
can easily judge editing quality with a computer, even though we can’t judge
Allen: It was amazing how effective
open-ended writing. And for teaching,
the code integration system was.
that will work just as well.”
System capabilities advanced
Then a classic Latin teacher, one of
rapidly, by leaps and bounds, while
the best course writers we had, came
learners continued using courseware
up with a way to teach Latin and the
uninterrupted. Many capabilities
classics in a new way. He could handle
became more sophisticated than
four times more students, teach them
many we have on the Internet today.
more, and get them up to snuff so he
I always viewed what you were
could later teach them the fancy stuff.
doing as taking away functional
Now, I wouldn’t have thought to break
constraints so we educators could
it up the way he did, but he did it! The
explore learning possibilities with
PLATO system allowed creative people
few boundaries. I’m not sure we have
to use their own ideas in a way they
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
50
Michael Allenʼs 2008 e-Learning Annual
would have been restricted from doing
or sixty PLATO terminals in the
with a more rigid system. The whole
classroom, and the teacher could still
was much greater than a sum of the
manage the class as usual. In many
parts, and that’s what made PLATO
ways it was much better. Now when
the success it was.
any system is evaluated as being on par with traditional teaching in the classroom, I consider that a major success, but most don’t see it that way. The computer goes on and its instructional quality is consistent and also subject to instant improvement, but I would say classroom teaching quality is inconsistent. These days are showing how low we can go.
Figure 2. PLATO III system, Washington School, 1970.
e-Learning is successful. Students come back. The overall measurement shows it bringing students back. It has to or the education system can’t pay
Allen: I also read on Wikipedia that,
for it. No one is supporting or funding
“An official evaluation by an external
it now. It is walking on its own legs.
testing agency ended with [the
The whole economic aspect depends
conclusion that though] everyone
on its success rate.
enjoyed using it [PLATO] . . . it was essentially equal to an average
Allen: My hope—my expectation,
human teacher in terms of student
actually—was that a healthy
advancement.” Some saw this as a
market competition would arise.
failure, didn’t they?
It would benefit the learner by forcing organizations to improve
Bitzer: Yes. Obviously they have never
the quality of courseware or lose
done computer-based education.
out to competitors. But I’ve been
There are some things we do
discouraged by the really pathetic
electronically that exceed what a
courseware on most systems. It’s
human can do. We can run a phone
made me wonder if competitive
switching system better than humans,
economic influences will actually start
for example. The PLATO system
to engage.
reduced entropy. You could put fifty Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
51
Bitzer: I think they have in some
Bitzer: I was an optimist. I said,
ways. The problem is the judgment of
“We are going to solve problems, put
what’s good and what isn’t. You can’t
terminals in every home. Things are
expect accurate judgments, unless
really going to work out well.”
educators are very good—unless they
Well, it didn’t happen that fast, but
were outstanding in the classroom
what has happened is as good as I
to begin with. You don’t see a lot of
wanted, just not in size. We do have
advertising of vintage courseware
terminals in every home. I was right
compared to new stuff that is coming
about that, but I didn’t realize how
out. The new stuff is making the
appetizing game distractions and
same mistakes we made long ago
other applications were going to be.
and soaking up resources as it fails.
I shouldn’t complain too much, as
Fortunately, PLATO activities have
one of these distractions, namely TV
expanded in orders of magnitude.
terminals, has been one of my biggest
It all has to be paid for and
successes.
measured against the criterion of
The growth factor for our
whether it is doing the job where
instructional applications is the
the regular system is not. The
most disappointing. Part of that is
measurement has to include the
because of marketing. Remember
dropout rate. Most schools get their
you are marketing against established
funding inversely based on their
institutions of learning, where a lot of
dropout rates. The money they are
protections are built in. They protect
receiving from reduced attrition more
themselves from change in fear they
than pays for the system. In fact,
might be phased out.
it pays for it and other educational
The educational system is one of the
functions as well. I’m in different
hardest things to change. The fact that
pursuits these days, but what I see is
it has been as successful as it has, in
retention rates of 80 to 90 percent. If
light of this attitude of protectionism,
you measure it in any reasonable way,
is pretty amazing in itself. But I really
it’s a success.
wondered when technology-assisted learning took off, how could they turn
Allen: What was your early vision
it down? At some point there would
for what technology could do for
surely be a decree for schools to do
learning?
the job. Then they wouldn’t be able to get around the advantages technology offers.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
52
Michael Allenʼs 2008 e-Learning Annual
I think it will happen. If schools
problems that have to be solved, they
really get on board with “No Child
break down and do it right. We can’t
Left Behind,” they won’t be able to turn
provide effective adult education
e-learning down, and we might see
without having the flexibility to make
something happen. If schools really get on board There is, however, with “No Child Left Behind,” enough inertia in the they won’t be able to current system to turn e-learning down. keep it static. People work hard so they
adults feel comfortable and learn at their own pace and place. To get into the big marketplace, with fifty million or so students, it’s going to take a huge change in attitudes on the part of school
don’t have to worry too much if they’re
districts and everyone else involved.
doing the wrong things.
This is a difficult change. Educational
Nothing is as entrenched as the
leaders are big in their own way,
K-12 education system, and it’s
even if they’re failing. It’s a problem
failing. We are ready with solutions;
I didn’t realize we had—the kind
the educators just need to not be
generated internally. School districts
threatened and embrace what is going
are more interested in politics than
to work. Computer-based systems are
in education. It’s sad, and it shows
expandable, without expanding the
up in the students. Those students
entropy. We are ready.
who come from schools with the right attitude are scoring as well as
Allen: Adult learning opportunities
those from overseas. When they come
may be easier to sell because people
from schools that don’t have needed
are looking for flexible ways to find
values, you almost have to start over
better jobs and further their careers.
with them. I certainly underestimated
Technology-assisted learning has
how to deal with the educational
flexibility of delivery and the ability to
system, but I’m not the only one. The
accommodate individual differences.
President of the United States and the
Do you think it will work backwards
superintendents of school districts
into the schools from success with
have also underestimated how hard it
adults?
is to change the K-12 schools. We have a large number of
Bitzer: Yes, absolutely. We are
distractions in the form of people
going to have niche applications
ready to make money on learning
rather than adoption by the general
technology without actually being
school population. When places have
involved in teaching anything. I can
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
53
teach X-Y-Z. Just take your computer
fundamentals of learning, you’d be
and put this software on it. It will solve
hard pressed to say yes. Games—yes.
your problems. They are beginning
Science, math, etc.—no. They will
to sell a large number of people
not be good contributors to our
on it; it will fail and discourage
culture if they sit around spending
people and schools from employing
time on games. They are not learning
learning technologies. Schools today
what they need to
don’t really use it properly—email,
learn to be effective
They are not learning
billboards, or course outlines. There
contributors these
what they need to learn
is actually very little instruction done
days.
to be effective contributors
online. I wanted to make a system that
these days. Allen: What advice
taught rather than distracted from
would you give to those trying to
teaching. I used to tell the school
advance today’s e-learning industry?
districts that. They used to say, “We are going to get thirty terminals.
Bitzer: Well, to start, “e-learning”
Aren’t we important?” I would say,
is too broad a term. I would say we
“No. It depends on what you do with
have to concentrate on the results
them.” If you have students simply
in the important areas such as math
gathering at terminals, rather than
and science. If you can write an
learning math, English, or science, we
interaction and prove that students
have failed them. The school districts
are learning math, physics principles,
didn’t like to hear that, because they
or editing skills better, then that’s
didn’t know how to do this.
an advancement. I think everyone should be focused on where they
Allen: The younger generation is so
want to head with results. We need
computer savvy, easily using games,
to be result-oriented, have flexibility
music, video, and instant messaging
in the approach, and support the best
skills. Maybe they will create a
designers and developers. Make sure
pressure for change from the bottom
you are working on the right problem.
up.
It’s a learning literacy problem, not a computer literacy problem that we
Bitzer: You are right, but the kind
have to solve.
of pressure it creates is on having the systems available. If you ask
Allen: Any thoughts on training in
yourself, is it helping them with the
corporate environments?
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
54
Michael Allenʼs 2008 e-Learning Annual
Bitzer: In a sense, the challenges
start, if you have a way of measuring
are the same. Anything that helps
quickly and accurately to understand
employees learn the company’s
what the problem is and generating
business, such as how to use
new material to solve that problem
equipment or follow the rules of the
by the computer, then you can adapt
company, would be valuable. But if
as you go along and guarantee that
I were a company, I would think the
students will learn the material in
best investment would be to make sure
a relatively quick fashion. I think
the least of my employees could read,
that’s the kind of applications we
write, and do basic math. That is what
need. When students leave, they have
makes a good employee.
learned the material, and you know it. Tiffany Barnes was one of my
Allen: As an engineer, do you see
students who earned her Ph.D. in
any new inventions or technologies
this field. She is now a professor of
coming along that are going to open
computer science at the University
up new opportunities we should be
of North Carolina at Charlotte. She
looking at?
did a scientific study of how to apply science to fault-tolerant teaching. She
Bitzer: From the hardware perspective
has other projects underway now such
we have more than enough. We just
as how to involve learning games in an
need to get the software sorted out.
effective way. She was one of the best
We have tons of memory we never
students I had in this area.
had before, which allows us to use it
Tiffany did a beautiful analysis of
properly or misuse it. One can develop
how the computer could automatically
measurement and learning strategies
extract the major concepts being
that are scientifically based. You can
taught in the lesson. By using
gather lots of data, process it quickly,
computer-generated questions,
see what concepts are working, and
the computer could determine
then build a scientific, mathematical
which concepts the student did
model of what you want to do and
not understand. Different remedial
make measurements of how well you
strategies were possible. For instance,
are achieving it. By putting feedback
it provided help for those students
and evaluation loops into the system,
who didn’t seem to get the major
you can adapt your processes to what
concepts and different exercises for
I like to call “fault-tolerant” teaching.
those who almost seemed to get it,
Even if the material is weak at the
but not quite. It adapted to learners by
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
PLATO, Plasma Screens, and Computer-Based Education
55
making the measurement accurately,
process. The goal is to make any
even at the concept level, going back
protein quickly, with high yield, using
and correcting misunderstandings to
E coli as the material. For instance, if
bring students along. It worked very
you have some special medication you
well. You can use the advances in
need to fight a particular disease, even
technology available to us these days
though it might be a human protein,
for improving the tools you are using
we can have E coli make that protein
in measurement and delivery of these
instantly and in large quantities. The
ideas for students.
results have been very encouraging. I keep busy, and it’s a lot of fun.
Allen: What are you working on now?
Unfortunately, there is a lag time on this new technology, and the new
Bitzer: I have been working on
medications will not be ready for me
lip-synch processing and speaker
fast enough. I’m getting old, and the
identification by extracting and
two of us won’t converge.
determining what shape the mouth was in. This work has helped with
Allen: Don, thank you so much for
animated figure generation. We have
your time. I am thrilled that you will
also looked at how to break up speech
be a part of this Annual. You have
and recognize a particular speaker,
given so many of us unprecedented
even when that speaker is trying to
opportunities to improve education
disguise his or her voice.
and training.
Currently, I’m working more intensely on genetics—applying single
Bitzer: You are too kind, Mike.
processing theory for extracting
Thank you.
hidden signals in the translation
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
S e c t i o n Tw o
Learning Te c h n o l o g y Ta k e s S h a p e
Fred T. Hofstetter Fred T. Hofstetter is professor of education at the University of Delaware. A specialist in multimedia, he developed the PODIUM multimedia application generator, the GUIDO Ear-Training Lessons, and the Serf Web-based teaching and learning environment. He has received many grants and awards and does consulting work for computer firms. He has published widely and has given lectures and workshops on multimedia and Internetworking in many locations in Europe, Africa, Australia, Canada, the United States, and the Pacific Rim. Hofstetter currently authors four IT textbooks for McGrawHill, including those on the topics of Internet Literacy, Multimedia Literacy, Advanced Web Design, and Internet Technologies at Work. His recent work has been in aligning the School of Education’s educational technology graduate programs with the AECT-SMETS and ISTE-TF standards to create a program assessment plan that is used in the University of Delaware’s NCATE accreditation process. He also chairs several doctoral committees for candidates who are using data-driven strategies to differentiate instruction and improve results in Delaware public school districts.
Back to the Future of Educational Technology
Fred T. Hofstetter
In this article, the author begins by sharing his early computer-assisted instruction experiences in the field of music. He then goes on to discuss the hopes for the larger field of CAI, the risks encountered, what has been learned about learning and the effective use of technology for education, and the implications for the further development of online learning. In the 1970s, we developed e-learning
altered our perception of how the
materials on computer terminals
network should be used in the design of
connected to an expensive mainframe
effective learning environments. This
that delivered courseware to students.
article chronicles the technological
To save costs in the 1980s, we
context in which the principles evolved,
moved these materials to standalone
reflects on lessons learned from thirty-
microcomputers and thereby lost
five years of practice, and discusses
connectivity. This was not a great loss
implications for further work.
for students, however, because most of us had been using the communication features to manage the development of
Where Did We Begin?
My work with educational
the courseware rather than to engage
technology began
students in dialogue online. The science
when I was hired as
of learning had only just begun to
a music instructor
create the design principles that would
at the University of Delaware in 1973.
guide effective use of the network. In
My assignment was to teach ear-
the 1990s the popularity of the World
training, which is a demanding two-
Wide Web created the critical mass
year course that every music student
that led to the Internet becoming the
must pass in order to remain enrolled
global information utility that powers
in a college music program. The
the 21st Century. Having regained the
problem with teaching ear-training is
connectivity we had lost, we are kind
the wide range of individual differences
of back to the future. As it always does,
among college music students. Some
however, the future changes before
students are naturally gifted at rhythm,
it gets here. In this case, the science
while others are more talented in pitch
of learning evolved principles that
perception. Singers with beautifully
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
60
Michael Allenʼs 2008 e-Learning Annual
resonant voices may stumble when
Graded Units for Interactive
trying to vocalize a notated melody.
Dictation Operations GUIDO stands
Some students who excel at melodic
for Graded Units for Interactive
intervals confuse them when the
Dictation Operations. Following
intervals combine to form chords and
Bloom’s mastery learning model,
harmonies.
GUIDO presented a graded series of musical dictation units in the
Bloom’s Mastery Learning Model
domains of intervals, melodies, chord
At the time I was facing this dilemma,
qualities, harmonies, and rhythms. In
Bloom’s (1968) learning for mastery
each domain, the student began with
(LFM) model was coming into vogue.
the first unit and practiced at that
According to this model, students
level until attaining the score needed
progress through graded units of
to advance to the next unit. Bright
instruction and are not permitted to
students progressed quickly through
go to the next level until mastering
the units, while slower students got
the current level. A meta-analysis
extra practice. Because the program
conducted by Kulik, Kulik, and
was new and I did not know exactly
Bangert-Downs (1990) found that LFM
how to sequence the units or how
increases test results by 0.59 standard
difficult to make them, I based
deviations across a broad range of
GUIDO on a table-driven design in
subjects including mathematics,
which all of the instructional variables
reading, science, and social studies.
were put in a table that the instructor
As Saettler (2004) notes, Bloom’s
could edit. Through this table, the
mastery learning model does not
instructor could make the course more
work with all forms of instruction, but
or less difficult, adjust the content and
it is highly applicable if the subject
sequencing of the units, or even create
domain is algorithmic, hierarchical,
new units of instruction.
or procedural. Because music is highly procedural, I decided to design
Analysis of Student
a mastery learning system for ear-
Response Matrices Through
training. I named the system GUIDO
controlled experiments, we found that
in honor of Guido d’Arezzo, the 11th
students who used GUIDO scored
Century monk who invented the
significantly higher on ear-training
musical staff.
examinations than students in control
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
groups using traditional methods of instruction. A series of articles
61
What Were Our Hopes?
Compared to computers today,
published in the Journal of Research
technology was
in Music Education (Hofstetter 1975,
in a primitive
1978, 1979, 1980) documented the
state in the
learning gains. These articles further
1970s. I began my work with GUIDO,
described how GUIDO kept response
for example, on Tektronix display
matrices of all the questions asked and
terminals that were not selectively
responses given. By analyzing these
erasable. If you wanted to erase
matrices, we discovered perceptual
anything onscreen, you had to erase
patterns that changed the thinking
the entire display. Imagine designing
about music theoretical concepts
e-learning modules under such a
that had been misunderstood for
constraint!
hundreds of years. We found melodic and harmonic situations in which
Acquiring a Delivery System
students were not just getting answers
We needed a better platform, so
wrong, but were also agreeing on what
we explored alternative systems.
the wrong answer was. The music
The National Science Foundation
community was highly interested in
had invested heavily in two high-
this research, and I was invited to
profile projects called PLATO and
present my work at many colleges
TICCIT. The PLATO terminals had
and universities. In 1975, Dean
plasma panels that were selectively
Helen Gouldner provided a startup
erasable and transparent. You could
grant that enabled us to begin the
rear project microfiche slides onto
National Consortium for Computer-
the plasma panel, upon which the
Based Music Instruction (Hofstetter,
computer overlaid text and computer
1976), which continues today as the
graphics. PLATO also had a computer-
Association for Technology in Music
controlled random-access audio device
Instruction (ATMI). The significance
that could play audio clips from a
of this work helped me earn tenure
15-inch magnetic disk. In order to
at UD, where I continue to be a
obtain the speed of access needed to
faculty member at one of the finest
display slides and begin audio playback
universities in the world.
promptly, the microfiche and audio devices were powered by compressed
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
62
Michael Allenʼs 2008 e-Learning Annual
air. Even though the technology was
powered PLATO. In 1976, CDC
primitive, we felt PLATO provided
acquired the rights to market PLATO.
more versatility than TICCIT, which
Our goal became achieving the critical
relied on a central bank of computer-
mass of interest needed for UD to
controlled videotape players that was
purchase a PLATO mainframe.
staff intensive and not very portable. PLATO was expensive, however.
Achieving Critical Mass My friend,
The terminals pictured in Figure 1
colleague, and mentor L. Leon
cost $6,000 each and needed to be
Campbell, who was UD’s provost
connected via leased telephone lines
throughout this period, explained
to the PLATO mainframe at the
that, in order to justify the purchase
University of Illinois, where PLATO
of the multi million-dollar mainframe,
was invented. The long-distance
I would have to show widespread
communication costs were prohibitive.
faculty interest in using PLATO
The only way for us to bring down the
across the campus. Thus began the
cost was to develop the critical mass
Delaware PLATO Project, which
of interest needed for UD to be able
lasted for fifteen years from 1974 to
to justify purchasing and installing its
1989. I became the Delaware apostle
own PLATO mainframe. Control Data
of PLATO and gave demonstrations
Corporation (CDC) manufactured
for faculty all across campus. Based
the Cyber mainframe computer that
on effective results from pilot projects in thirty-six academic departments, we were able to convince Provost Campbell to purchase a PLATO mainframe, which was installed in the UD computing center in 1978. Our work with PLATO is documented in a series of Summative Reports in the ERIC database.
Figure 1. The author’s sons Dan (standing)
Moving Beyond Programmed
and Tom (seated) use a PLATO plasma display
Instruction When I reflect on the
terminal in 1979. In the background, a student is touching the screen to place a musical note onto the staff of a GUIDO melodic dictation exercise.
broad range of applications we developed, I realize how pigeonholed the modern view of the classic PLATO system is. I say pigeonholed because
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
63
most contemporary researchers
Ramping Up In order to raise
who write about PLATO use it as an
money to help UD pay these costs,
example of programmed instruction.
we began selling PLATO services to
Lockee, Moore, and Burton (2004, p.
corporations, government agencies,
563), for example, refer to PLATO as
and school districts. Most notable
“the most prolific and long-standing
among the corporations was DuPont,
example of computer-based PI.” They
with whom we partnered to create
are correct in the sense that PLATO
a laboratory technician training
stands for Programmed Logic for
program. In government, our
Automatic Teaching Operations,
largest customer was the Federal
which was how most people used it.
Aviation Administration (FAA),
PLATO was capable of much more,
which used our PLATO system to
however, and we were using it for
teach flight inspection procedures,
more than behaviorally oriented
radar, communications equipment,
PI. We had the PLATO system, for
and electronics. In our local school
example, doing career counseling,
district, which was undergoing
demography, and psychology
desegregation, we used an Emergency
experiments. Our engineering college
School Aid Act (ESAA) grant in
even programmed a model of the
1979 to install PLATO terminals
Delaware Bay that could track the
with the goal of reducing minority
movement of oil slicks in the event of
group isolation by providing basic
a tanker spill.
skills instruction to improve student achievement, enhance student self-
What Were the Risks?
In the 1970s,
concepts, and foster more positive
attempting
interracial interactions. Under a grant
a large-scale
from Control Data in 1981, we were
computer-assisted
funded to create the Lower Division
instruction
Engineering Curriculum (LDEC).
project was filled with risks. The hardware was expensive, and we
Developing Effective Courseware
needed to raise the money to pay
One of the greatest risks in creating
for it. Budget-hungry deans wanted
a computer-aided instruction (CAI)
academic results. If we developed
project in the 1970s was that there
ineffective materials, the deans would
would not be enough quality software
argue that our budget should be spent
available to sustain the project. As
elsewhere.
Nievergelt (1980, p. 11) expressed it,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
64
Michael Allenʼs 2008 e-Learning Annual
“Today it makes no sense to start a
as reasons to reduce our funding.
CAI project unless one is willing to
Figure 2 shows the courseware
write most of the necessary software.”
development process we created
Because courseware development
(Hofstetter, 1981). It contains a variety
was expensive and time-consuming,
of feedback loops, whereby we used
we needed to make sure the
techniques of formative evaluation to
materials we produced hit the mark.
help ensure that the courseware we
Academic deans were watching us,
were developing would be effective.
and they wanted evidence that our
In a contemporary implementation of
materials were effective. We needed a
this concept, Allen (2006, p. 73) refers
mechanism for developing courseware
to this iterative process as successive
that worked from the start. Budget-
approximation.
hungry deans would use any failures
Figure 2. The Delaware Model for Courseware Development Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
65
What Would Learning Technology Mean for the Student?
Scaling Back in Recession In the 1980s, the Reagan tax cut led to an economic recession, and UD began to scrutinize every aspect of its budget. So did DuPont and the FAA, which
The loss of networking was not as devastating then as it would be today because
sought lower-cost alternatives to the
most of us had been using the
expensive mainframe connections.
communication features to manage
Reading the handwriting on the wall,
the development of the courseware
and not wanting to lead UD into a
instead of involve students in online
budget shortfall, I led the effort to
course discussions. The pioneering
convert our most successful PLATO
work of the great Russian psychologist
programs into a microcomputer
Lev Semyonovitch Vygotsky had not
format. At first, we converted
yet made it into the mainstream of
software to micro-PLATO, using
American educational technology.
microcomputers invented by Control
We did not know that the most
Data Corporation. In 1985, we began
important use of the network would
using IBM PCs programmed in
be to scaffold students and provide
TenCORE, which was a standalone
customized coaching when they
version of PLATO’s TUTOR
encountered difficulty. Vygotsky’s
programming language.
work was just being translated into
In 1989, we uninstalled the PLATO
English. Because we did not yet know
mainframe when the Philadelphia
it, we did not realize what we were
Prisons decided to discontinue its
losing when we lost connectivity. In
twenty-six terminal subscriptions and
other words, losing the network was
begin running on IBM PCs instead.
not such a great loss in 1989, because
While we saved cost and avoided
we did not yet know what to do with
a budget shortfall by moving our
it.
programs from the PLATO mainframe to less expensive microcomputers,
Skinner’s Linear Behaviorism
we lost the network. No longer could
American psychology was dominated
instructors communicate with each
by the behaviorists. Chief among them
other through PLATO notesfiles or
was Skinner (1938, 1953), who saw
press “term-TALK” to initiate an
that human behavior is powerfully
online chat.
shaped by its consequences. Moreover,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
66
Michael Allenʼs 2008 e-Learning Annual
Skinner felt that psychology was
without any connectivity. Toward the
essentially about behavior and that
end of his career, Skinner himself did
behavior was largely determined by its
not foresee the vital role computer
outcomes. Instructional designs were
networking would play in education.
based on a stimulus-response (SR)
As late as 1986, Skinner maintained
chain in which positive reinforcement
that the microcomputer is “the ideal
was provided when students answered
hardware for programmed instruction”
correctly. Such designs had no need
and proposed that when used for
for computer networking because all
computer-aided instruction it should
of the feedback was pre-programmed.
be called a “teaching machine” instead
It would be several decades before
of a computer (Skinner, 1986, p. 110).
American educational technologists would recognize the vital role of
Vygotsky’s Triangular
networking in education. As late as
Constructivism Vygotsky (1978)
1989, for example, we received an
provided the missing link by
EDUCOM/NCRIPTAL Best Tutorial
transforming the linear SR model
award for an Introduction to Statistics
of the behaviorists into a triangle
course that did not contain any
that represents how people learn
communication features, neither for
through an extrinsic process in which
instructors to coach students nor for
knowledge is mediated by student
students to help one another. The
interaction with tools, community, and
IBM PC version of GUIDO won a
understandings acquired through prior
Joe Wyatt Challenge award in 1991
learning. Figure 3 shows how Vygotsky drew this triangle by adding to the stimulus-response chain a third node called X, which stands for extrinsic. This extrinsic node is bidirectional, meaning that students can reverse the action of the SR chain by interacting
Figure 3. Vygotsky’s
with tools, instructors, experts,
Interaction
peers, knowledge bases, multimedia, datasets, and professional associations.
Triangle 1
S is the stimulus, R is the response, and X is extrinsic mediation through which students learn by interacting with objects in their environment including tools, instructors, experts, peers, knowledge bases, multimedia, datasets, and professional associations. 1
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
67
Since the learner is portrayed as
the instructor provides graduated
an active processor who explores,
prompting according to the student’s
discovers, reflects, and constructs
needs. At first, the instructor gives
knowledge, the trend to teach from
general help. If the student still
this perspective is known as the
stumbles, the instructor gives more
constructivist movement in education.
specific guidance. By making the student’s thinking visible, the dialogue
Zone of Proximal Development
provides a record of the coaching that
Vygotsky (1978) observed that when
both student and instructor can study
students are learning, inevitably
and reflect on how to improve. Today’s
they stray into a zone in which the
behaviorists consider scaffolding to be
difficulty of the problem they are
an antecedent that plays a vital role in
working on exceeds what they can
invoking the desired response; thus,
accomplish independently. Because the
through scaffolding, behaviorists have
zone is between the student’s current
discovered some common ground with
level of development and the next
Vygotzky and his followers.
level that could be achieved through
Figure 4 is an activity diagram
an expert’s guidance, Vygotsky called
that illustrates the manner in which
it the Zone of Proximal Development
I believe e-learning revolves around
(ZPD).
the zone. Following Engeström (1987),
Online learning creates the zone
the top three nodes of tools, subject,
whenever students submit assignments
and assessment represent mediated
that fail to meet the criteria for a
activity through which the student
high grade. Instead of assigning a
learns to perform the outcome of the
low grade and moving on, I believe
instructional goals. When the student
we have a responsibility to coach the
cannot accomplish this independently,
students and help them achieve the
the instructor provides coaching in
objective. To create such a learner-
the zone, which is represented by
centered environment, I invented a
the bottom inner triangle. The outer
web-based e-learning system called
triangle has a node representing
Serf (Hofstetter 1997, 1999, 2006),
community on the bottom left,
which has a feature called “give the
and professional organizations on
student another chance.” When
the bottom right. The instructor is
the instructor clicks this option, a
responsible for creating curriculum
scaffolding protocol creates a just-
that leads to certification as defined
in-time discussion forum in which
by the appropriate standards body.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
68
Michael Allenʼs 2008 e-Learning Annual
Figure 4. Activities Around the Zone in a Socially Constructed e-Learning Environment
The instructor’s role is bidirectional
three primary principles that have
in that the instructor can also provide
emerged from the science of learning.
recommendations to the professional
These principles are:
organization regarding refinements or additions that may be needed to make
1. People learn by connecting new
the standards meet community needs.
information to concepts already learned.
Have We Learned from the Science of Learning?
At the turn of
2. To learn how to reason, solve
the century,
problems, and augment knowledge
the National
in a field of inquiry, people need to
Research
understand facts and ideas in the
Council (2000)
context of a conceptual framework
published a landmark book entitled
that facilitates application to real-
How People Learn. It is a wonderfully
world problem solving.
written synthesis of developments in
3. People are motivated to learn
the science of learning. A fascinating
when they can set their own goals,
chapter on brain research (Chapter 5)
reflect on their progress, and feel
describes how learning makes physical
in control of their learning.
changes that reorganize the brain. The manner in which these connections
From these principles, it follows that
happen to rewire the brain reinforces
instructional designs should:
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
1. Take into account the learner’s
69
I believe it is important to mention
preexisting understandings and
why it cannot. Because people learn
correct any faulty preconceptions
by connecting new information
in order to prevent future
to concepts already learned, the
misunderstandings;
order in which students encounter
2. Enable students to study multiple
new material is vitally important.
examples of the concept at work
Curriculum planning matters.
in order to learn it in depth in
Students need a course of instruction
authentic contexts; and
that presents materials in a logically
3. Include metacognitive supports
appropriate order with enough real-
that make visible the learner’s
world examples to cover the concepts
reflections and enable an
in depth.
instructor to provide scaffolding and guide revisions to improve
Fitting Course Goals to
student learning and reasoning.
Student Goals In my online courses, I have evolved a process whereby
Learning From Versus
the students buy in to the course
Learning With Hill, Wiley, Miller-
content by making its goals be their
Nelson, and Han (2004) make the
own. Early in the course, I give an
distinction between learning from
assignment in which the students
versus learning with the Internet.
tell me why they enrolled and what
When learning from, the student
they hope to get out of the course.
proceeds with guidance provided
A dialogic protocol enables me to
according to protocols determined
conduct with individual students a
by the instructional design. When
little conversation in which I explain
learning with, on the other hand,
that my purpose is going to be to help
students use the Internet as a tool.
them to master their goals. Most of
Hill and his colleagues reviewed
the students provide a statement of
several studies that examined whether
competencies they hope to develop
people can learn from the Internet
as the course progresses. I respond
without designed instruction. The
to these statements by explaining
overwhelming answer from these
the manner in which the course will
studies is no. Although it amazes me
address the student goals. Later in the
to think that people would expect the
course, I use these goal statements
Internet by itself to produce results,
to help the students propose term
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
70
Michael Allenʼs 2008 e-Learning Annual
paper topics and multimedia projects
about it, normally you do not log on
through which students demonstrate
unless you are in a good mood. Thus,
the extent to which they have met the
e-learning enables faculty to teach,
course objectives. The assignment
and students to learn, when they are
protocol enables me to hold an online
in the mood for it. Negotiating goals
conversation that makes each student’s
when both student and instructor
thinking visible and records it into
are in a good mood leads to more
the course database for subsequent
ambitious projects with a higher
analysis and reflection.
quality of scaffolding
Holmberg (2003) views the didactic nature of distance education as a
Situated Cognition Situated
teaching-learning conversation,
cognition is a term coined by Brown,
in which the teacher bonds with
Collins, and Duguid (1989) to describe
a student by creating empathy.
the kind of learning that happens in
According to Holmberg, it is not only
environments designed to put students
the frequency but also the quality
in real-world contexts. Technology
of communication that is important
plays an important role in enabling
in creating this bond. In the online
learners to study concepts in authentic
environment, I believe one of the
contexts. Through interactive video,
reasons why e-learning is effective is
students can be put into real-life
because people are doing it when they
situations. A film clip of the Tacoma
are in the mood to do so. If you think
Narrows bridge collapse, for example,
Figure 5. A PODIUM Videodisc Overlay for the
Figure 6. Experimenting with Wind Speeds and
Puzzle of the Tacoma Narrows Bridge Collapse
Pulsing Actions to Find the Combination That Created the Standing Wave That Destroyed the Tacoma Narrows Bridge
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
71
has become a classic example used by physics instructors to situate the teaching of wave motion and resonance. Under a grant from the National Science Foundation, Fuller, Zollman, and Campbell (1982) created a videodisc that included the film clip along with a slide bank and full motion videos of the bridge construction, its
Figure 7. The Author Works with Interactive Videodisc in 1988
geographical setting, mathematical formulas, the influence of the wind,
point at which the focus now is on
and standing wave experiments. I
providing students with multimedia
created a program called PODIUM
computers to discover knowledge in
that let the faculty show instantly any
real-world contexts. At the Internet
slide, motion sequence, or experiment
Plasma Physics Education Experience
on the Tacoma Narrows videodisc. By
(IPPEX) Online, for example, students
simply touching the screen of the IBM
can run the same nuclear reactor
Infowindow display shown in Figure
simulation that Princeton University
5, the faculty could show students the
scientists use to determine optimal
historical background and illustrate
settings for tokamak fusion reactors.
the geophysical conditions that led to
By manipulating the sliders that
the catastrophe. Figure 6 shows how
control the plasma density, heating
the faculty could experiment with
power, and magnetic field, students
different wind speeds and pulsing
can explore how these parameters
actions to recreate the standing waves
interact and develop an intuitive feel
that led to the bridge collapse.
for the process scientists go through
Figure 7 shows the assortment
in designing tokamak reactors. The
of equipment used to make
IPPEX simulation is online at ippex.
presentations back then. Multimedia
pppl.gov/tokamak/tokamak.htm.
was so expensive that you could not realistically think about giving each
Learning and Transfer As Bruning,
student a computer. Instead, the focus
Schraw, & Ronning (1995, p. 216)
was on equipping each classroom
explain, “The aim of teaching, from
with a workstation that the teacher
a constructivist perspective, is not
could use in making presentations.
so much to transmit information,
Happily, costs have dropped to the
but rather to encourage knowledge
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
72
Michael Allenʼs 2008 e-Learning Annual
formation and development of
(1998) found that, in eighth grade
metacognitive processes for judging,
mathematics, using the computer
organizing, and acquiring new
for drill and practice is negatively
information.” Rumelhart (1981),
related to student results (-.59 grade
following Piaget, introduced the
levels), while using the computer for
notion of schemata, which are mental
simulations and applications increases
frameworks for comprehension that
results (+.42 grade levels). Too much
function as scaffolding for organizing
drill results in overly contextualized
experience. At first, the instructor
learning and reduces time available for
provides instructional scaffolding that
problem solving and application.
helps the student construct knowledge. Gradually, the instructor provides less
Increasing Motivation According to
scaffolding until the student is able to
Allen (2003), motivation is the most
construct knowledge independently.
important factor in achieving success
In How People Learn, the National
from e-learning. Allen’s law states
Research Council (2000, p. 53)
that e = m 2ci, where m is motivation,
identifies four key transfer principles
c is content, i is interaction, and e is
that govern this process:
e-learning outcomes. If motivation
1. Initial learning is necessary for
is lacking, the m in Allen’s law will
transfer. 2. Knowledge that is overly contextualized can reduce transfer. 3. Transfer is best viewed as an active, dynamic process. 4. All new learning involves transfer based on previous learning.
be zero, and therefore e will be zero, meaning that no learning will occur, regardless of the quality of the content and its interactivity. Social opportunity is important in motivating people to learn (National Research Council, 2000). To create social motivation, I have created a
Instructional designers need to take
gallery feature, which is an option
these principles into account when
the instructor can configure for any
choosing among behavioral, cognitive,
Serf assignment. If the instructor
and social constructivist approaches.
clicks to activate the gallery, students
Too much drill, for example, can cause
can see each other’s submissions.
negative results. In an Educational
Students learn a lot by viewing each
Testing Service study of the 1996
other’s work, which they discuss
National Assessment of Educational
in the online forum. Depending on
Progress (NAEP) database, Wenglinsky
the nature of the assignment, the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
73
instructor can configure the gallery
substantial progress in making e-
to reveal the student names or keep
learning more effective. These three
them anonymous, and display the
areas are:
instructor’s feedback or keep it
1. Reducing transactional distance,
private. In my web design courses, I
2. Making assessment metacognitive,
have a “cool tool” assignment in which I have the students (1) identify the
and 3. Creating communities of learners.
tool they consider most useful and (2) write an essay explaining why they
Reducing Transactional
think it’s cool. Through the gallery,
Distance As Vygotsky (1978) notes,
students explore each other’s tools and
learning inevitably creates a zone in
make discoveries richer than anything
which students encounter problems
I could design on my own.
they cannot solve on their own. At this
The innate human desire to
point, the student needs help from
develop competence is another factor
the instructor. In Serf, I have created
that is important in motivating
a scaffolding protocol that enables the
people to learn (National Research
instructor to identify the zone, provide
Council, 2000). To create competence
just-in-time coaching, and give the
motivation, I let students choose from
student another try. By repeating this
a selection of labs that fit a variety of
process until the student masters the
school or workplace settings. Students
assignment, educators can produce
invariably choose to do the labs that
better prepared students who have
are perceived as aligning with their
truly learned the material.
career paths or helping accomplish tasks in their workplaces.
The problem I have noticed in my current application of the zone, however, is that students do not
What Are Implications for
know that I have provided them
Further Work? In setting future
with feedback until the next time
directions, educational technologists
they log on. This creates the kind of
must consider the extent to which
gap that Moore (1993) refers to as
technology has emerged for addressing
transactional distance. The more time
the problems we see in our current
it takes the student to receive help
work. I believe there are three areas
needed from the instructor, the larger
in which technology has evolved
is the gap of transactional distance.
to the point at which we can make
It is a psychological gap created
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
74
Michael Allenʼs 2008 e-Learning Annual
by communication latency, not a
I plan to create a short messaging
geographical gap caused by physical
service (SMS) to which students can
distance.
subscribe if they would like to be
When I am grading final projects,
notified of course events via TXT
for example, I often send e-mail to
messages that Serf will send to their
notify students that I have found
cell phones.
problems in their submissions and
Eventually, the cell phone interface
have provided feedback along with a
could be extended to TXT a message
chance to resubmit their assignments
whenever someone writes a response
for a higher grade. Needless to say,
in a discussion to which the user has
students place a high value upon being
subscribed. I believe such a cell phone
given a chance to make revisions
interface could substantially increase
and earn a higher grade. It is time-
the amount of interaction in a course,
consuming, however, for the instructor
especially if the student could TXT
to send these e-mails. I plan to
a reply back to the online discussion
automate this process by creating an
forum.
option whereby students can subscribe to a notification service that will
Making Assessment
send them an e-mail automatically
Metacognitive One of the most
each time the instructor leaves them
important things we do in education
feedback on an assignment. This
is help students learn to reflect
e-mail will include the name of the
on whether their current level
assignment, the instructor’s message,
of understanding is adequate. By
and a link to click to go to that point
making the student’s thinking visible,
in the online course. In like manner,
metacognitive tools enable students
I plan to create an e-mail service
to reflect on their progress toward
whereby the instructor can choose
learning to think like an expert.
to be notified (or not) each time a
Through conversational assignment
student responds to the scaffolding.
protocols that record the dialogue
Electronic mail may not suffice,
between students and instructors,
however, to reduce transactional
an e-learning environment can make
distance for the younger generation
student thinking visible. By posing
of students who prefer to use TXT
questions that make students reflect
messaging on cell phones instead of e-
on whether their current level of
mail on personal computers. To reach
understanding is adequate, the
students who live on their cell phones,
instructor can help students learn
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
75
to be aware of the progress they are
to which the student has met each
making toward understanding.
standard. By requiring the students to
In my college-level courses, for
have an expert review their artifacts,
example, students propose and
include the expert’s comments in the
negotiate the topics of their projects
portfolio, and respond to the expert
and term papers. Through an online
by explaining what they plan to do
consultation protocol, I help students
to follow up on the expert’s advice,
create a project that not only satisfies
I believe we can make the student’s
their interests, but also meets national
thinking visible in the context of
standards in their chosen career fields.
professional standards. By making
The dialogue I have with my students
assessment metacognitive in this
is recorded in the course database
context, we can prepare students to
and can be viewed at any time on the
maintain this awareness throughout
course assignment page. I encourage
their careers
students to reflect on this dialogue, think about their progress toward
Building Communities of
meeting the standards, and become
Learners Reflecting on his invention
actively involved in setting their
of the virtual high school (VHS)
learning priorities.
concept, Tinker concludes that
The next assignment dialogue I plan
“Without collaboration, the social
to create is an ePortfolio protocol in
value of networking is lost and online
which students will document and
courses become simply extensions of
reflect on the extent to which they
existing course formats” (2005, p. 413).
meet professional standards in their
Riel (2005) identifies three overlapping
field. Instead of having students use
ways in which online education should
canned software that creates the
be community based: (1) collaborative
portfolio for them, I believe we should
learning in the context of a student
have students learn to use industry
cadre, (2) theoretical learning through
standard web authoring software
community experiences, and (3)
to create their own portfolios and
transformational learning in one’s
thereby acquire strategic 21st Century
community of practice. In Serf, I
multimedia authoring skills. The
have created a community-building
portfolio assignment protocol I plan
component called the affinity cluster.
to create will prompt the students
Instructors who own an affinity
to provide a link to the spot in their
cluster have the power to create one
portfolio that demonstrates the extent
or more affinity groups, each of which
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
76
Michael Allenʼs 2008 e-Learning Annual
can consist of one or more threaded
Conclusion
In the book How
discussion forums, document sharing
People Learn,
libraries, and electronic magazines
the chapter
(a.k.a. newsletters). A role-based
on technology
permissions model enables the
concludes by
instructor to assign privileges to end-
stating that “Much remains to be
users, who can be given read, write,
learned about using technology’s
moderator, or administrator access to
potential: to make this happen,
one or more affinity groups.
learning research will need to become
Affinity groups can have different
the constant companion of software
kinds of purposes and targeted
development” (National Research
audiences. Some groups may be
Council, 2000, p. 230). I personally
intended for experienced users
believe that e-learning will evolve
to participate in multithreaded
to the point at which the computer
discussions, for example, while other
becomes transparent. When that
groups may be intended for novices
happens, we should remove the
who need a simple menu-driven
hyphen from the term e-learning. As
discussion forum. To categorize
Chute (2003) reminds us, Webster’s
affinity groups according to different
unabridged dictionary defines the
purposes, the instructor creates
prefix e as meaning thoroughly, as in
an organizational entity called a
the word evaporize, which means to
“community” and assigns to it affinity
vaporize thoroughly. Thus, we may
groups that share the community’s
define the term elearning as meaning
goals and objectives.
to learn thoroughly, which we will
I believe a logical next step in the
achieve by applying design principles
development of the communities
from the science of learning to create
would be to create a wiki-style
effective multimedia teaching and
knowledge building environment
learning environments.
(KBE). This would help answer
This is what I have been attempting
Romiszowski’s (2005) criticism that, in
since I began working with educational
spite of what is known about creating
technology thirty-five years ago. It
KBEs, IMS vendors have done little
is hard to predict how many decades
to build these kinds of cooperative
remain for me personally to work
learning protocols into their products.
in this exciting field. In 2004, I was
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
77
honored to receive a pioneering award
One of the bodies objects that it is not
for creating the first web-based course
dead yet. Neither am I done yet. God
at the University of Delaware back
willing, I hope to spend a few more
in 1997. In my acceptance speech,
decades inventing, teaching, learning,
I recounted a scene from the movie
mentoring, contributing, and sharing
Monty Python and the Holy Grail in
as we work to get to the point at which
which the Dead Collector carts off to
our field can remove the hyphen from
burial bodies dead from the plague.
e-learning.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
78
Michael Allenʼs 2008 e-Learning Annual REFERENCES Allen, M.W. (2003). Michael Allen’s guide to e-learning. Hoboken, NJ: John Wiley & Sons. Allen, M.W. (2006). Creating successful e-learning: A rapid system for getting it right first time, every time. San Francisco, CA: Pfeiffer. Bloom, B.S. (1968, May). Mastery learning. In Evaluation Comment, 1(1). Los Angeles: University of California at Los Angeles, Center for the Study of Evaluation of Instructional Programs. Brown, J.S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 28, 32-42. Bruning, R.H., Schraw, G.J., & Ronning, R.R. (1995). Cognitive psychology and instruction. Englewood Cliffs, NJ: Merrill/Prentice-Hall. Chute, A.G. (2003). From teletraining to e-learning and knowledge management. In M.G. Moore & W.G. Anderson (Eds.), Handbook of distance education (pp. 297-313). Mahwah, NJ: Lawrence Erlbaum Associates. Engeström, Y. (1987). Learning by expanding: An activity-theoretical approach to developmental research. Helsinki, Finland: Orienta-Konultit. Fuller, R., Zollman, D., & Campbell, T. (1982). The puzzle of the Tacoma Narrows bridge collapse (Videodisc). Hoboken, NJ: Wiley Educational Software. Hill, J.R., Wiley, D., Miller-Nelson, L., & Han, S. (2004). Exploring research on internet-based learning: From infrastructure to interactions. In D.H. Jonassen (Ed.), Handbook of research on educational communications and technology (2nd ed.) (pp. 433-460). Mahwah, NJ: Lawrence Erlbaum Associates. Hofstetter, F.T. (1975, May). GUIDO: An interactive computer-based system for improvement of instruction and research in ear-training. Journal of Computer-Based Instruction, III, 100-106. Hofstetter, F.T. (1976). Foundation, organization, and purpose of the National Consortium for Computer-Based Musical Instruction. Journal of ComputerBased Instruction, 3, 21-33. Hofstetter, F.T. (1978). Computer-based recognition of perceptual patterns in harmonic dictation operations. Journal of Research in Music Education, 26, 111-119. Hofstetter, F.T. (1979). Controlled evaluation of a competency-based approach to teaching aural interval identification. Journal of Research in Music Education, 27, 201¬213.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Back to the Future of Educational Tecnology
Hofstetter, F.T. (1980). Computer-based recognition of perceptual patterns in chord quality dictation operations. Journal of Research in Music Education, 28(2), 83-91. Hofstetter, F.T. (1981). A model for administering computer-based education. In B. Lewis and D. Tagg (Eds.), Computers and education. Amsterdam: NorthHolland Publishing Company. Hofstetter, F.T. (1997). Serf ’s up! Teaching and learning with Serf: Your servant on the internet. In Virtual learning environments: AACRAO Policy Summit (pp. 123-135). Washington, DC: American Association of Collegiate Registrars and Admissions Officers. Hofstetter, F.T. (1999, Fall). Three waves of Serf, a web-based teaching and learning environment. College & university media review, pp. 99-123. Available: http://www.udel.edu/fth/serf/serf1-3.html. Hofstetter, F.T. (2006). Serf 4.0. Newark, DE: Serfsoft Corporation. Available: http://serfsoft.com. Holmberg, B. (2003). A theory of distance education based on empathy. In M.G. Moore & W.G. Anderson (Eds.), Handbook of distance education (pp. 79-86). Mahwah, NJ: Lawrence Erlbaum Associates. Kulik, C.C., Kulik, J.A., & Bangert-Downs, R.L. (1990, Summer). Effectiveness of mastery learning programs: A meta-analysis. Review of Educational Research, 60(2), 265–299. Lockee, B., Moore, D., & Burton, J. (2004). Foundations of programmed instruction. In D.H. Jonassen (Ed.), Handbook of research on educational communications and technology (2nd ed.) (pp. 545-569). Mahwah, NJ: Lawrence Erlbaum Associates. Moore, M.G. (1993). Theory of transactional distance. In D. Keegan (Ed.), Theoretical principles of distance education (pp. 22-38). London: Routledge. National Research Council. (2000). How people learn (expanded edition edited by J.D. Bransford, A.L. Brown, & R.R. Cocking). Washington, DC: National Academy Press. Nievergelt, J. (1980, September). A pragmatic introduction to courseware design. IEEE Computer, pp. 7-21. Riel, M. (2005). Building communities of learners online. In G. Kearsley (Ed.), Online learning: Personal reflections on the transformation of education (pp. 309-320). Englewood Cliffs, NJ: Educational Technology Publications.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
79
80
Michael Allenʼs 2008 e-Learning Annual
Romiszowski, A.J. (2005). Online learning: Are we on the right track? In G. Kearsley (Ed.), Online learning: Personal reflections on the transformation of education (pp. 321-349). Englewood Cliffs, NJ: Educational Technology Publications. Rumelhart, D.E. (1981). Schemata: The building blocks of cognition. In J.T. Guthrie (Ed.), Comprehension and teaching: Research reviews (pp. 3-26). Newark, DE: International Reading Association. Saettler, P. (2004). The evolution of American educational technology. Greenwich, CT: Information Age Publishing. Skinner, B.F. (1938). The behavior of organisms. New York: Appleton-CenturyCrofts. Skinner, B.F. (1953). Science and human behavior. New York: Macmillan. Skinner, B.F (1986, October). Programmed instruction revisited. Phi Delta Kappan, 68(2), 103-110. Tinker, R. (2005). Learning through online collaboration. In G. Kearsley, (Ed.), Online learning: Personal reflections on the transformation of education (pp. 402-414). Englewood Cliffs, NJ: Educational Technology Publications. Vygotsky, L.S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. Wenglinsky, H. (1998, September). Does it compute? The relationship between educational technology and student achievement in mathematics. Princeton, NJ: Educational Testing Service (ETS) Policy Information Center. Retrieved 22 December 2006 from http://www.ets.org/Media/Research/pdf/ PICTECHNOLOG.pdf.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Peter G. Fairweather An elementary school teacher beset with questions, Peter G. Fairweather earned a doctorate in reading and language (psycholinguistics) while puzzling about the processes people use to understand text and what happens as these processes evolve or break down. While exploring models of text comprehension, he served on the faculties of Northwestern University and the University of Texas, extending this thread of research to include cognitive simulation. Seduced by more available resources in industry, he held a number of posts in learning technologies companies, working on device and process simulations with both commercial and academic application. He has also designed interactive elementary and middle school material offered by several publishers. He is the co-author of Computer-Based Instruction: Design and Development, a popular instructional design text that by now both he and his co-author have vigorously repudiated. He has published over fifty papers in the areas of learning technologies, assistive technologies, and accessibility and has been awarded several patents in these areas. Currently, he works with a team on the problem of how to affordably and efficiently develop simulations that infer individuals’ learning status and then adapt appropriately.
It Seemed Like a Good Idea at the Time
Peter G. Fairweather
The Development of Computer-Assisted Instruction in Schools In this article, the author explores the development of computer-aided instruction in schools. Although educators and instructional designers had high hopes for instructional technology, they’ve been disappointed thus far. Various factors converged to force CAI to focus narrowly on particular items or objectives, even though research exists to show this is not the most effective way to teach. The author closes by reviewing a few recent, more successful programs. The midwives of the technology
were first made using the same
revolution in education that began in
vocabulary used for what teachers did.
the late 1960s saw a future of computers
These included:
working as tireless classroom teaching
ÿ Optimal calibration of practice
assistants. For their nascent field,
items to a student’s level of skill,
they adopted the label “computer-
ÿ Determination of the number of
assisted instruction” (e.g., Suppes &
items to be assigned based on the
Morningstar, 1969, 1970), emphasizing
learner’s performance history,
their goal to support the pedagogy and curriculum of the school*. Almost
ÿ Management of the student’s progress through the curriculum,
immediately, however, instructional
ÿ Provision of feedback to the student,
technology and the nurturing context
ÿ Generation of reports about each
into which it was born began to evolve on a path toward failure.
student’s performance, ÿ Determination of optimal limits on the time allowed for practice, and
What We Thought Instructional Technology Could Do
Because
ÿ Maximization of time-on-task.
classroom activities have
Indeed, if a teacher were able to do
always been
these things regularly, at that point in
directed by
history we would have thought that
teachers, it was not surprising that the
teacher to be creditably skilled and
promises of educational technologies
likely to be welcomed into any school
*The focus of this chapter on technology in schools should not be interpreted to mean that schools alone nurtured its development. Other organizations, particularly those involved with training, developed forwardlooking applications in areas ranging from medicine to language learning.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
84
Michael Allenʼs 2008 e-Learning Annual
district. However, the evolution of then-current models of instruction, learning, and even education
ÿ Means for learners to manage their instruction, and ÿ Opportunities for groups of
technology itself combined to
learners to build reflective and
undermine the vision of the pioneers
critical collaborations around
of computer-based instruction.
problem-solving activities.
Early Curriculum: Problems Give Way to Items
The earliest
In contrast, around the same time,
developers of
Patrick Suppes and his co-workers
computer-based
(Suppes & Morningstar, 1969, 1970;
educational
Suppes, Fletcher, & Zanotti, 1976) at
materials
Stanford’s Institute for Mathematical
organized instruction in a variety of
Studies in the Social Sciences
ways. For example, Ludwig Braun and
conceived a different structure
his colleagues used the computer to
altogether for computer-based
interactively manipulate quantitative
instruction—one that dominated the
representations of such phenomena
field for nearly twenty years. Rather
as the near extinction of the buffalo,
than using the computer’s ability to
fluctuations of a population of gypsy
simulate physical, natural, or social
moths, or efforts to manage pollutants
phenomena and providing a means
in a body of water (Braun, 1971a,
for learners to interact with them, the
1971b). This approach to instruction
Stanford team sought to represent the
based on interactive simulations
learner as a traveler on a path through
extended the model-based batch mode
a measured curricular space. Suppes
games used in business schools at the
and his colleagues represented the
time. Indeed, this method anticipated
content and processes of computer-
some of the advantages claimed today
based learning as:
by developers of problem-based
ÿ A set of task items arranged
curricula, including:
in order of difficulty within an
ÿ Authentic problem contexts,
academic subject, sort of a Russian
ÿ Rich, high-fidelity computational
egg that might comprise “division
models supporting instruction, ÿ Reactive problem surfaces allowing learners to deal directly with the
of fractions” within “numerical computation” within “arithmetic” within “mathematics,” for example;
consequences of their decisions,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
ÿ A trajectory through a curriculum
85
was able to provide enough correct
that is composed of these sets of
answers to a set of problems, each of
tasks;
which was described by a horizontally
ÿ An optimization function that
formatted subtraction problem where
advanced learners along their
the minuend is composed of two
prescribed trajectories according
digits each greater than zero and the
to their error rate and, sometimes,
subtrahend is composed of a single
time-on-task;
digit whose value is greater than the
ÿ A set of tasks arranged to
rightmost digit of the subtrahend,
minimize the differences between
he or she was deemed to have a skill
items at one level within a
that might be labeled “subtraction
sequence and those at the next;
with borrowing involving a two-
ÿ A curriculum based on detailed
digit minuend and a single digit
descriptions of the items to be
subtrahend.” While we might admire
presented to learners;
the emphasis on performance-based
ÿ A set of responses associated with
assessment, the definition of particular
each item that the computer could
skills made by identifying them with
judge correct or incorrect;
item descriptions brought about a
ÿ A variably controllable feedback
reductionism that would contribute
mechanism to reinforce or
to the rejection of these curricula by
punish the performance of the
schools.
learners and to provide them with assistance, if needed; and ÿ Different modes of presentation
Later investigators would show that even the simplest subtraction items could evoke any of several different
for items including assessment,
cognitive solution strategies in
practice, tutorial explanation, or
learners (Resnick, Carpenter, Moser, &
demonstration.
Romberg, 1982). Careful chronometric analyses of learners’ solutions showed
Detailed item descriptions
that the skills they applied to problems
identified the corpus of skills to be
sharing a common description could
learned in any segment of curriculum.
differ strongly from one another.
In effect, to have learned a skill was
(Instruction delivered by intelligent
defined as the ability to respond
tutors, such as the PUMP Algebra
correctly at some criterion level
curriculum [Koedinger, Anderson,
to a type of item with a specific
Hadley, & Mark, 1997] try to model
description. For example, if a learner
the mechanisms of problem solving
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
86
Michael Allenʼs 2008 e-Learning Annual
rather than organizing themselves
and Jostens Learning sought ways to
around problem descriptions. It should
span entire “subject areas.” All of them
not be surprising then, that such
developed efficient reusable driver
tutors are the only forms of computer-
programs that operated on sets of data
based instruction in mathematics that
representing the subject being taught.
have proved themselves consistently
While this allowed for the delivery
and significantly effective without
of a huge collection of “items,” each
strong teacher direction).
of them exhibited an unfortunate sameness that, if nothing else, led
That Nasty Problem of “Coverage”
Early computer-
students to expect problem formats,
based
procedures, and types of responses.
instruction
More insidious was the reduction
in schools
of the curriculum into a set of items
concentrated
that could be answered with a single
on arithmetic and language usage
simple response. While multi-stage
(Suppes & Morningstar, 1970). While
problems requiring the evaluation
later developers produced media-
of complex solution strategies
rich thought-provoking examples
can be presented interactively by
in biology, physics, chemistry, and
using only simple easily evaluated
social studies, these tended to address
responses, commercial developers
such narrow curricular segments
usually failed to design to that goal.
that schools could not justify their
Moreover, during the early 1970s, the
purchase. A teacher friend of mine
curriculum recommendations made
expressed his pet criticism as follows:
by organizations such as the National
I can spend $150 on that wonderful
Council of Teachers of Mathematics
program about Brownian motion.
or the National Council of Teachers
The kids love it and learn a lot
of English did not emphasize such
running the simulation over and over.
skills as problem finding and solving
Their passion and its effectiveness
or critical thinking. However,
notwithstanding, I’ve shot my whole
even these organizations seemed
budget on something that doesn’t even
progressive compared to state or
rate a whole class period.
district curriculum departments, so
Conversely, developers like
there was little pressure to go beyond
Computer Curriculum Corporation
simple items requiring equally simple
(CCC), Milliken Publishing, Science
response formats.
Research Associates (SRA), WICAT, Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
87
The requirement of simple
from their influence for another
responses, combined with the
twenty years. While early developers
provision of remedial explanatory
showed their disdain for conditioning
sequences or “helps” that failed
theories of learning (see Skinner, 1953)
to adapt to students’ difficulties,
by rejecting the claim that computer-
further divorced the computer-based
based instruction was simply
curriculum from either classroom or
“programmed learning by computer,”
real-world application. Not only did
nevertheless, they seemed to embrace
the computer have to be able to judge
some of principles of operant
a response, it had to be able to trace
conditioning by designing programs so
for the student a path to a correct
that:
solution. Because artificial intelligence
ÿ Practice involved relatively small
was an expensive and impractical
pieces of behavior with which to
dream at the time, computer-based
gain stimulus control;
instructional applications could not adapt to particular learner skill deficits and generate custom explanations.
ÿ Reinforcement was applied immediately to a correct response; ÿ Small pieces of behavior were
Instead, in most cases, remediation
chained together into more
and explanation sequences only
complex organizations;
recapitulated static solution sequences
ÿ Practice items were reviewed
for simple problems, illustrating them
periodically to prevent extinction
with preconstructed steps. If a student
of acquired skills or to capitalize
did not grasp the approach after one
on the facilitative effect on the
of these static non-generative help
reacquisition of skills by those
sequences, she had no place to turn.
previously learned but forgotten; and
Before Practice Became a Dirty Word
Although signal events marking
ÿ Reinforcers and their schedules could be varied.
the rebirth of cognitive psychology (e.g.,
Rejecting behaviorism, these same developers embraced the newly minted
Miller, Galanter, & Pribram, 1960)
information processing psychology
happened on the same campus where
(e.g., Atkinson & Shiffrin, 1968). It
Suppes and his colleagues worked,
hypothesized a staged pathway for
the central vein of computer-based
pieces of information that ran from
instruction would remain immune
sensory acquisition through a sensory
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
88
Michael Allenʼs 2008 e-Learning Annual
store, through short-term memory,
the language of objectives would
through a working memory where it
provide a common set of descriptors
combined with and was organized
permitting them to be compared.
by material retrieved from long-
Moreover, educators and instructional
term memory, to long-term memory.
designers hoped that objectives, at
It defined information as small
least in their earliest forms, would
packets that could be experimentally
provide statements of the performance
tracked through these stages of
expected of learners.
memory. Computer-based instruction
However well-intentioned this
developers designed with simple
transformation into objectives may
items asking for simple responses,
have been, it helped grind curricula
fitting the information processing
into tiny pieces. Commercial
model nicely. This model supported
curriculum developers embraced
developers’ efforts to explore complex
the “objectives movement” because
learning management strategies while
it enabled them to compare their
maintaining fragmentary instructional
offerings with goals and objectives
items and simple learner responses.
pursued by states. Each vendor
Educational practitioners and
developed technology to enable them
curriculum developers were exposed
to map their objectives to those
to the so-called “behavioral objectives”
of a district to determine degrees
methodology (Mager, 1975) that
of similarity and to identify where
claimed kinship with information
computer-based instruction products
processing theories of learning and
had “holes.” Objectives became
memory. Although learning objectives
marketing weapons for vendors so
came to be defined in different ways,
their reductionism became a continual
early forms generally included only:
feature of and influence on vendors’
ÿ A description of the behavior to be
materials.
demonstrated, ÿ The conditions under which the behavior would be performed, and ÿ A statement of the criterion used to discriminate acceptable from unacceptable performance.
Johnny Learned This Because the Computer Says So
Dicing curricula into smallgrained chunks smoothed learners’ trajectories
Educators hoped that the transformation of curricula into
through the space defined by the items presented. Aficionados of computer-
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
89
based instruction claimed that such
interesting and well-crafted pieces of
atomization enabled precise tracking
instruction.
of what learners had mastered.
Their bloated costs prevented
However, rather than pinpointing
help sequences from modeling
where learners were on their road to
or explaining much. Most of
mastery, learners’ positions within
them involved predetermined
item sequences identified that which
demonstrations of a process or
the learner had to “get past” but not
procedure with which the learner had
necessarily learn.
trouble. Others did even less, resorting
Designers and developers believed
to verbal explanations of the rule,
that if learners successfully performed
concept, or procedure being taught.
on enough items of a particular type,
Most such sequences would cap their
then they must have learned the
explanations or demonstrations by
objective associated with those items.
simply revealing to the learner the
In making this assumption, educators
correct response that she should
became confused and disappointed
have made. While practicing with
because reported results often
previously worked out problems has
exceeded what students could actually
been shown to be an effective means
do.
of reducing cognitive load so that
The brutal economics of
learning can take place (Sweller &
development drove this inconsistency.
Cooper, 1985), that was hardly the
The preparation of the data to present
reason for telling the student the
a single item, to capture a learner’s
answer. Instead, by revealing the
response, and to judge that response
answer, the developer marked the end
cost relatively little in the context of
of the sequence beyond which further
thousands of items in a curriculum.
attempts to model or explain the
In contrast, the machinery to present
objective would cost too much.
a remedial or explanatory sequence,
Because of these static sequences
usually termed a “help,” cost much
that concluded by revealing the
more. Developers averred that their
correct response, students could make
best work lay within such help
progress without actually learning
sequences and expressed frustration
what was required. For example,
that learners who performed well
recently an exquisitely crafted
enough to be judged not in need of
computer-based multimedia early-
help never experienced the most
reading curriculum was evaluated in
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
90
Michael Allenʼs 2008 e-Learning Annual
a large urban district. Observers were
words. Desperately wanting to avoid
stunned with how tightly the children
false negative judgments where the
were riveted to the arresting visual
application told a student that she had
displays and rhythmic audio. Indeed,
erred when she had not, we tailored
when the children were called on to
the feedback to a possibly incorrect
respond to assessment or practice
response by having an animated
items, they sometimes missed items
character, a panda, ask the student to
intentionally in order to re-experience
repeat the response as if it had not
entertaining sequences presented
heard it. After a variable number of
earlier. Help sequences rather briefly
responses judged incorrect, the panda
led them to a point where the correct
would pronounce the word or phrase
response was revealed. By simply
and move on. Although the system
remembering how to respond, learners
would report these problems to the
could work through the curriculum
teacher, the student could continue
without actually learning the
through a series of conversationally
phonological awareness and sound-
entertaining but pedagogically
symbol correspondence skills the
questionable sequences. Indeed, in
developers intended for them to know.
one instance, we observed a student
As another example, my colleagues
happily conversing with the panda
at the IBM T.J. Watson Research
at the back of a classroom. Moving
Center and I developed a practice
over to get a closer look, we found
environment for reading skills
him reading words and phrases to
that used speech recognition (Nix,
the panda, who asked for every single
Fairweather, & Adams, 1998; Williams,
response to be repeated as if the
Nix, & Fairweather, 2000). Recognizing
student were responding incorrectly all
words spoken by children proved
the time. Further examination revealed
challenging, but
that, although his earphones were
Recognizing words spoken by a large speech
properly plugged in, his microphone
children proved challenging. data collection
was unplugged. He “finished” the book
effort helped
without the application detecting even
work the error rate down to just over
one correct response, all because it
5 percent. While that may sound good,
responded so neutrally and because it
consider a situation in which a teacher
wound up pronouncing each word or
working on oral reading skills accepts
phrase for him anyway.
a wrong utterance or, worse, punishes a correct one once every twenty
Although the vendors of the late 1970s and 1980s insistently claimed
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
91
that their materials helped students
effectively remediated the most basic
learn, with equal determination they
skills in those lacking them (see Aeby,
refused to guarantee results. Vendors
Powell, & Carpenter-Aeby, 1999, for
argued that because they had no
use with low-achieving learners).
control of the instructional delivery
Because this effect could be detected
environment, they could not ensure
in schools with a high proportion
that the instructional systems and
of low-achieving students, many
materials were properly used by their
vendors of computer-based instruction
customers. If the materials were not
materials shifted their marketing
properly used, they argued, it was
focus to “Title I” schools, so-called
unreasonable to be held responsible
because of the funds made available
for student learning gains. Although
for disadvantaged students by that
this seemed to be a tacit admission of
part of the Elementary and Secondary
the deficiency of their instructional
Education Act.
strategies to independently enhance
Hersh Waxman and his colleagues
learning, vendors continued to
(2003), in a thoughtful and fairly
promote this specious argument
recent meta-analysis of the effects of
without resistance until the traditional
computer-based instruction, found a
computer-based instruction for
moderately strong effect. However,
schools industry collapsed.
they offer the following trenchant qualification:
Of Course It’s Effective, Right?
From 1970 to
One of the most important issues related
2000, it was
to teaching and learning with technology
easy to find
that needs to be addressed is the soundness
reports of the
of the research for the implementation and
salutary effects
improvement of technology programs. First,
of computer-based instruction (e.g.,
there were few quantitative studies published in
Fletcher, Hawley, & Piele, 1990;
the last five years that included relevant data to
Kulik, Kulik, & Bangert-Drowns,
permit a meta-analysis and calculation of effect
1985; Waxman, Lin, & Michko,
sizes. Scientific journals that use independent
2003). For most of that period, their
peer review in deciding what research merits
combined weight crushed the few
publication are generally considered to be
who questioned whether this emperor
the highest standard of research, yet much of
had clothes (e.g., Becker, 1994). There
the work in the field of teaching and learning
had never been much question that
with technology does not meet that standard
traditional computer-based instruction
[emphasis added]. The lack of quality, refereed
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
92
Michael Allenʼs 2008 e-Learning Annual
quantitative studies points to a serious problem
small-group studies cobbled together.
of research in the field. (p. 16)
Wenglinsky found that it was how computers were used more than how
It seemed that the effectiveness of
much they were used that influenced
computer-based instruction would
their achievement in mathematics. In
forever be sketched by meta-analytic
particular, the time spent by students
studies summarizing effects found
who used the computer to practice
across studies of small groups of
low-level skills did not correlate
students in widely varying learning
positively with better test performance
environments. The effects of computers on The use of student achievement measured at small groups a national level did not warrant of learners the considerable amount of continually money spent to acquire them. weakened studies and forced
and, for some segments of the population, correlated negatively. Economists addressed the issue by examining the return on investment of lottery funds by the government of Israel in educational technology (Angrist & Lavy, 2002).
the reliance on meta-analysis for an
They concluded that the effects of
obvious reason: cost. No one could
computers on student achievement
afford to equip hundreds or thousands
measured at a national level did not
of learners to credibly prosecute this
warrant the considerable amount of
question.
money spent to acquire them and train
Computer-assisted instruction finally had its day in court at a national
teachers to use them. What was the result of these
level. Harold Wenglinsky (1998), then
studies? Hardly anything. At their
of the Educational Testing Service,
publication, the industry built
examined the relationship of the
around directed instruction by
amount of time students in different
computer was on the verge of sudden
grades spent on computers at home
collapse. The low cost of shrink-
and at school with their aggregated
wrapped educational software had
performance on the most recent
already mortally wounded almost
National Assessment of Education
all of the purveyors of large-scale,
Progress (NAEP) available at the time.
broad curriculum computer-based
This relationship, based on thousands
instruction. Moreover, even the
of students assessed with a common
market for shrink-wrapped packages
standardized instrument, contravened
evaporated shortly thereafter,
the conventional wisdom built on
plummeting from $458 million in 2000
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
93
to $152 million four years later, a 70 percent decline (Richtel, 2005). The problem of financial
Frustration with computer-based instructional tutorials in schools contributed to vigorous development
weakness, compounded by increasing
of computer tools for learning. These
pressure from school and teacher
were pressed into the service of the
accountability, reshaped the ways
constructivist perspective on learning.
teachers and principals applied their
Rejecting the “transmission” model
time and budgets. Unless computer-
of learning that depicted knowledge
based instructional materials could
moving from a knowledge source
promise to do at least as good a
into the student, constructivists
job as the teacher, that teacher
believed that effective learning
would be reluctant to turn over
required the student to build his or
a significant amount of time to a
her own knowledge, subjecting it
machine. Moreover, as vendors began
to review, critique, demonstration,
to insist that proper use of their
and reorganization. Computer tools
materials required close coupling
included “productivity” software such
with classroom activities, teachers
as word processors or spreadsheets,
saw themselves being forced to give
visualization tools such as concept
up even more time. For increasing
mappers or function graphing
numberings of them, it simply was not
programs, or even intelligent
worth it.
composition support environments, to name a few. These tools seemed well
Was It a Complete Waste of Time and Money?
Over the first
suited to the self-directed, hypothesis-
thirty years or
driven, problem-oriented learning of
so, traditional
the constructivist enterprise.
computer-based instruction
For example, Geometer’s Sketchpad departed from the traditional tutorial
in schools lost its way. Contrasting
format. Instead of explaining concepts
approaches, such as the intelligent
by choosing words and illustrations
tutoring materials already mentioned,
aimed at the mythical “average
provided rare exceptions that
student,” this program offered learners
delivered on their promises. However,
opportunities to make constructions
their extraordinary development costs
and generate proofs, recapitulating the
keep most of them from successfully
explorations of Euclid, Archimedes,
making the trip from the laboratory to
and Descartes.
the classroom. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
94
Michael Allenʼs 2008 e-Learning Annual
range of domains, but proof eluded them. Geometer’s Sketchpad, on the Figure 1. The
other hand, is almost always used
Geometer’s
by students supported and guided
Sketchpad® is
by a skilled geometry teacher. In a
published by
sense, its effectiveness is an extension
Key Curriculum
or magnification of that of the
Press. For more
teacher. Ironically, to be effective,
information, visit
constructivist methods require a
www.keypress.
bit of the “directiveness” that, in
com/gsp.
overwhelming doses, crippled earlier forms of computer-based instruction. Looking back, even computer-based The Logo computer language,
developed by Wally Feurzeg and others at Bolt, Baranek, and Newman, and famously championed by Seymour Papert of MIT, also contrasted sharply with traditional computerbased instruction by providing learners a means to pose and solve problems through programming. Teachers using Logo believed that students could productively explore mathematical, symbolic, and graphical representations of phenomena ranging from biology to linguistics to physics. However exciting the promised intellectual journey of such programs might seem, it has become clear that effective use of these tools requires considerable guidance by a skilled teacher (Kirschner, Sweller, & Clark, 2006). Adherents of Logo had long claimed that it enhanced learners’ problem-solving abilities in a wide
instruction did not follow the path its creators envisioned: its failures bore fruit. It threw a painfully bright light on what not to do and why and, in so doing, helps illuminate where we might explore next. For example, the structure of problems, their presentation, their adequacy as models of processes and objects in the real world, and their ability to elicit a problemsolver’s understanding of the need to learn something have suffered a stunted evolution in the context of computer-based instruction. How might these problems be transformed in a collaborative virtual world such as Second Life? Consider Donna Dillenberger’s (personal communication, 2007) representation of a data center. Because it feeds on data from real data centers, it presents an authentic model of the problems
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time
95
faced by managers. As one moves
acquired each of those skills. As
about the 3-D representation, one
fine-grained, evidence-based, and
goes through a “cognitive registration”
revolutionary as such models might
process (LeDoux, 1996) that orients
be, their narrow scope and fleeting
the learner and exposes the artifacts of
duration imprisons their promise.
learning within an authentic context.
Imagine if, instead of a model made
If the learner hasn’t mastered the
up of discrete context-free skills,
anatomy of a router, its importance is
one made up of estimates context-
impressed upon the learner caught up
sensitive, interacting abilities and
in the problem of a breach of security.
disabilities guided tool use. What
Moreover, the environment offers
might be inferred if a learner were
itself for reorganization, enabling
able to effectively summarize the
questions such as, “Can we afford to
methods by which an agreement
create a lower bandwidth channel
was negotiated, even if that learner
between these two servers?” How does
did nothing more than observe the
a learner formulate a question to the
process? Is that inference consistent
expert summoned to the scene? Often
with estimates of related learner
the degree to which a question or a
characteristics that have evolved over
statement can be clearly expressed
years of technology-based observation,
reflects the speaker’s understanding.
hypothesis generation, experiment,
While the field has advanced from
and re-estimation? It may be, for
simple problem descriptions to finely
example, that our learner withdraws
articulated models of their solution,
in social situations, real or virtual,
user modeling has only crept along.
but that abstraction, summarization,
This may seem to make little sense,
and communication historically
given that it has advanced so far
have remained exceptional. Should
from the trajectory models through a
the technology aim to ameliorate
curricular space composed of problem
weaknesses or engineer experiences
descriptions. The most effective
that address learner strengths?
cognitive tutors, such of those of
Can we approach the perennial
John Anderson and his colleagues
goal of continuous authentic learning
(Anderson, Corbett, Koedinger, &
by insinuating software interfaces
Pelletier, 1995), model learners as a
between learning experiences and
collection of skills. Problem-solving
their real-world consequences?
performance enables the inference
Reconsider Donna Dillenberger’s
of the degree to which a learner has
(personal communication, 2007)
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
96
Michael Allenʼs 2008 e-Learning Annual
virtual data center and its possible
intervene and “change history” to
evolution. A first phase involved
evaluate such actions against what
user activities such as exploring the
“really happened.” But why stop there?
virtual data center, manipulating the
Why not blend the training experience
environment, sharing observations,
with the actual operational one by
seeking information from one another,
recapitulating the evolution from
or critiquing one another’s ideas.
“canned data,” to “real data,” then
Even operating on “canned data,”
extending it to “the real thing?” With
this virtual data center was able to
the learning interface unified with
unfold a rich manifold of experience
the operational one, members of the
within which a guiding instructional
staff running the center could move
executive could operate. (Even without
back and forth across these stages,
it, the experience affects everyone
using technology, each other, and the
undergoing it). She has implemented
data center itself to fine-tune their
a next phase operating on a stream
performance.
of actual data flowing from operating
So, even though the history of
data centers that enable an authentic
computer-based instruction yielded
mirror to events, processes,
only the faintest image of our initial
procedures, and people. With it,
dreams for it, as with other fields of
one could create training scenarios
human endeavor, a new, sometimes
comparing learner group performance
difficult to recognize set of dreams
to data center staff or the ability to
will guide us now.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
It Seemed Like a Good Idea at the Time REFERENCES Aeby, V.G., Powell, J.V., & Carpenter-Aeby, T. (1999). Effects of SuccessMaker Computerized Curriculum on the behavior of disruptive students. Journal of Educational Technology Systems,28(4), 335-347. Anderson, R., Corbett, A.T., Koedinger, K.R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), pp. 167-207. Angrist, J., & Lavy, V. (2002). New evidence on classroom computers and pupil learning. The Economic Journal, 112, 735-765. Atkinson, R.C., & Shiffrin, R.M. (1968). Human memory: A proposed system and its control processes. In K.W. Spence and J.T. Spence (Eds.), The psychology of learning and motivation: Advances in research and theory, Vol. 2. New York: Academic Press. Becker, J. (1994). How exemplary computer-using teachers differ from other teachers: Implications for realizing the potential of computers in schools. Journal of Research on Computing in Education, 26(3), 291-321. Braun, L. (1971a). Huntington II Simulation Program-POLUT. Teachers guide. Maynard, MA: Program Library, Digital Equipment Corporation. Available from Educational Resources Clearinghouse Center, ERIC number: ED093644. Braun, L. (1971b). Huntington II Simulation Program-POP. Maynard, MA: Program Library, Digital Equipment Corporation. Available from Educational Resources Clearinghouse Center, ERIC number: ED179414. Dillenberger, D. (2007). Personal communication on the evolution of shared virtual worlds as learning tools. Yorktown Heights, NY: IBM Research. Fletcher, J.D., Hawley, D.E., & Piele, P.K. (1990). Costs, effects, and utility of microcomputer-assisted instruction in the classroom. American Educational Research Journal, 27(4), 783-806. Kirschner, P.A., Sweller, J., & Clark, R. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based learning. Journal of Educational Psychology, 41(2), 75-86. Koedinger, K.R., Anderson, J.R., Hadley, W.H., & Mark, M.A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43. Kulik, J.A., Kulik, C.-L. C., & Bangert-Drowns, R.L. (1985). Effectiveness of computer-based education in elementary schools. Computers in Human Behavior, 1, 59-74.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
97
98
Michael Allenʼs 2008 e-Learning Annual
LeDoux, J. (1996). The Emotional Brain: The Mysterious Underpinnings of Emotional Life. New York: Simon & Schuster. Mager, R. (1975). Preparing instructional objectives. Belmont, CA: Fearon. Miller, G.A., Galanter, E., & Pribram, K.H. (1969). Plans and the structure of behavior. New York: Henry Holt. Nix, D., Fairweather, P., & Adams, W.H. (1998). Speech recognition, children and reading. Conference on human factors in computing systems. New York: Association for Computing Machinery. Resnick, L.B., Carpenter, T.P., Moser, J.A ., & Romberg, T.A. (1982). Addition and subtraction: A cognitive perspective, Mahwah, NJ: Lawrence Erlbaum Associates. Richtel, M. (2005, August 22). Once a booming market, educational software for the PC takes a nose dive. New York Times. Skinner, B.F. (1953). Science and human behavior. New York: Macmillan Suppes, P., Fletcher, J.D., & Zanotti, M. (1976). Models of individual trajectories in computer-assisted instruction for deaf students. Journal of Educational Psychology. 68(2), 117-127 Suppes, P., & Morningstar, M. (1969). Computer-assisted instruction, Science, 166, 343-350. Suppes, P., & Morningstar, M. (1970). Four programs in computer-assisted instruction. In W.H. Holzman (Ed.), Computer-assisted instruction, testing, and guidance (pp. 233-265). New York: Harper & Row. Sweller, J., & Cooper, G.A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2, 59–89. Waxman, H.C., Lin, M., & Michko, G.M. (2003). A meta-analysis of the effectiveness of teaching and learning with technology on student outcomes. Naperville, IL: Learning Point Associates. Available: www.ncrel.org/tech/ effects2/ Williams, S.M., Nix, D., & Fairweather, P. (2000). Using speech recognition technology to enhance literacy instruction for emerging readers. In B. Fishman and S. O’Connor-Divelbiss (Eds.), Fourth International Conference of the Learning Sciences (pp. 115-120). Mahwah, NJ: Lawrence Erlbaum Associates. Wenglinsky, H. (1998). Does it compute? The relationship between educational technology and student achievement in mathematics. Princeton, NJ: Educational Testing Service. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Wallace Hannum Dr. Wallace Hannum is a faculty member in the School of Education and coordinator of the educational psychology program at the University of North Carolina at Chapel Hill and associate director for technology of the National Research Center on Rural Education Support. Dr. Hannum teaches graduate-level courses on learning theories, instructional design, and the use of technology in education. Dr. Hannum’s research focuses on instructional uses of technology, especially distance education. Dr. Hannum has consulted on instructional uses of technology with many organizations, both public and private. He has participated in the design and implementation of numerous technology-based programs and projects. He has worked extensively on education projects in Africa, Asia, and Latin America. Dr. Hannum is author of five books and numerous articles on topics related to technology and instructional design.
Reflections on e-Learning
Wallace Hannum
The Long History of a New Idea The revolution in educational technology that was to follow the invention and use of personal computers never happened. But why? The short answer that I learned the hard way is that there is no magic in the machine. Simply placing instructional content on a computer and delivering it to students does not enhance instruction nor guarantee learning. Rather, to achieve effective instruction with technology, educators and designers must pay attention to both pedagogy and past lessons while endeavoring to extend technology’s strengths, rather than simply duplicate what’s already been done in the past. While the specific origins of the term
term in Google), you would think we
“e-learning” are not entirely clear, Jay
all understood the term in a similar
Cross used the term about seven years
fashion. Yet after even a brief analysis
ago to refer to the convergence of
of articles or a discussion with people
learning and networks. He stated that
who use or promote e-learning, it is
e-learning was to traditional training
clear that we do not understand e-
what e-business was to business as
learning the same way.
usual. Computer and communications
What qualifies as e-learning
networks were changing the very nature
according to one person’s concept of
of how we conduct business and how
e-learning fails to qualify according
we conduct training. On the surface,
to another’s. In short, we understand
e-learning seems a recent phenomenon
e-learning in different ways. When
following from the proliferation of the
someone completes a computer-based
Internet, yet writing several years ago
training (CBT) lesson on a CD, is that
Mühlhäuser (2004) entitled his paper
e-learning? Some say yes; others say no.
“e-Learning After Four Decades: What
Some refer to most distance learning
About Sustainability?” It may seem a bit
today as e-learning if it involves the
odd that e-learning can have a history
Internet in any fashion, even if only
of almost five decades, despite seeming
to download class materials. Others
to be new and identified by name only
challenge this being e-learning.
a few years ago. The reason for this
Some use e-learning to refer to any
incongruence follows from the meaning
instruction delivered via computers,
of e-learning. Given how often the term
whether or not they are networked or
is used today (70,300,000 hits for the
use the Internet. Still others include
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
102
Michael Allenʼs 2008 e-Learning Annual
the use of remote exchange and
from the communications metaphor
collaboration in their definition of
in which the Internet is viewed as a
e-learning, whether or not it includes
means for instructors and students
delivering of instructional materials.
to communicate for instructional
If an instructor sends an email with an
purposes (Hannum, 2001). This
attachment for students to download,
definition of e-learning includes the
print, and read, some refer to this as
student working alone through an
e-learning. In essence, this example
interactive tutorial on calculus as well
describes the old vintage WWII
as the student working collaboratively
correspondence study in which the
with others at remote sites on an
instructor sent students materials to
exercise in calculus, perhaps under
read and exercises to complete, and
their instructor’s tutelage. Under the
which they returned to be graded. In
auspices of an educational agency
its more recent version, the Internet
or department, both are organized,
has replaced the post office as the
planned instructional experiences with
means for instructors and students
expected learning outcomes.
to send exercises and exams back
My definition of e-learning does
and forth. While some see this as
not discount the value of an individual
e-learning, others insist that e-
working alone on the Internet
learning must involve the use of “new
searching out something of interest
multimedia technologies,” and not
and possibly learning from it as a
simply the sending of print materials
result. I use Google daily when I am
over the Internet. Those who speak
curious about something. Last week
and write about e-learning know what
I wanted to confirm the nationality
they mean by the term, yet they know
of Archduke Ferdinand, whose
it in different ways. Hence, there is
assassination provoked WWI, and
confusion over exactly what e-learning
I also wanted to find out why diesel
is and when it began.
automobiles made that knocking noise
For purposes of this chapter,
at idle. In both cases I easily found and
I am using e-learning to refer to
learned the answer using the Internet,
instruction delivered at least in part
but was I engaged in e-learning? No,
over the Internet. This instructional
not as I see it. e-Learning involves a
use of the Internet may follow from
planned, coordinated effort involving
the publishing metaphor in which
a sponsoring organization to produce
the Internet is viewed as a means to
specific learning outcomes for a
distribute instructional materials or
specific group of learners. I was using
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
103
the same technology and perhaps even
requires that we situate it historically,
a website or reference they might use,
describe what we know about it, and
but my use of the Internet to research
speculate on what will happen next,
a specific question is not e-learning
despite the obvious risks of such recall
any more than someone running to the
and speculation.
library years ago to look up these same questions in Encyclopedia Britannica
My Journey
My first use
would have been considered to be
of a computer
instruction.
in schools was
The recent furor over e-learning
1967 or 1968
ranges from estimates that it will
(my recall is not
replace all other forms of instruction
more precise about the exact year).
and greatly improve learning
The computer was an IBM 1401,
outcomes, to those who see e-learning
an early transistor-based computer
as a dangerous distraction that is likely
with all of 4K of memory running
to fail. Any reasoned consideration
FORTRAN.
of e-learning, especially its potential impact and future, should first look at antecedents and see what can be learned from them before speculating about what might happen in the future with e-learning. This chapter attempts to explore the rather long history of e-learning, identify some lessons learned from the
Figure 1. IBM 1401
past, and speculate on some possible futures for e-learning. Speculating
The first application the team I
about the future is always risky,
was working with developed was
especially when that speculation
an automated payroll system. As a
centers on technology. Recounting the
result of our work, teachers were
past is also risky, as those who were
able to get their checks two days
there and remember certain events
sooner. Although we were much
don’t agree on what happened, much
more interested in instructional
less those who did not witness the
applications, the decision to do the
recalled events directly. However,
payroll application first was made by
putting e-learning into context
the person who directed research for
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
104
Michael Allenʼs 2008 e-Learning Annual
the school system. He was a crusty,
learning objects, although much of
old, retired Army sergeant who had
the actual content was in an analog
learned how to get things done with
medium such as filmstrips, video,
people through many battles, both
or audio recordings. It was modular
literal and figurative. As a result of
at fine levels of granularity, down to
our automating payroll, many teachers
parts of an individual lesson linked
and administrators said something
to individual objectives, searchable
to the effect of, “I don’t know what
using various ways of matching
computers are, but if they help me get
content to instructional needs, and
paid sooner, I like them.” Thus was our
it was reusable. Teachers appreciated
introduction of computers into a small
the ease with which they could locate
school system in a southern state forty
appropriate instructional resources
years ago.
for specific objectives and for specific students. Had the content itself been
A Foray into Learning Objects
in digital format, I would argue we
Following that, we turned our
were working with learning objects
attention to instructional applications
back then.
and created a classification system
Following this we began
for instructional materials matched
what interested us most—direct
to specific instructional objectives,
instructional applications of
teaching approaches, and student
computers in the form of computer-
characteristics. Essentially this was
assisted instruction (CAI). Our
a large database, although it was
efforts were very elementary—not
hard coded years before any database
much more than automating flash
program like Access or Oracle was
cards—but students were able to
available. A teacher could query
interact directly with a computer that
the database we created to locate
asked them a question, accepted their
instructional materials on a specified
answers, judged the adequacy of their
topic that used a selected teaching
answers, and immediately provided
approach for a student or group of
the appropriate feedback. This allowed
students with certain characteristics.
students to advance at their own pace
We saved teachers considerable time
and provided them with many more
they would have otherwise had to
practice opportunities than a teacher
spend looking for, or creating, such
in a classroom with twenty-five to
instructional content themselves. In
thirty other students could provide.
many ways, this was a forerunner of
Plus, we were able to match the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
105
Figure 2. (Left and Above) IBM 1500
students to individual lessons rather
most anybody’s definition. Here’s how
than the class as a whole. The greatest
it happened. We had an IBM 1500
advantages seemed to accrue to those
system that was specifically designed
students who were either considerably
to deliver CAI to students through
ahead or behind their peers. I suspect
sixteen CRT terminals connected
this is still true today.
locally to the 1500 system by cables. Content was developed, laboriously
Early e-Learning With my interests
I might add, by using a programming
in computer-assisted learning (CAI)
language called Coursewriter that
blooming, I relocated to Florida State
was designed for this purpose. In
University the next year to work in
order to allow remote access to the
a CAI lab that had been established
system and thus the CAI lessons,
there. It was over thirty years later
a Digital Equipment PDP-8 was
before anyone used the term e-
interfaced with the IBM 1500 system
learning and before the Internet was
to send signals over telephone lines
invented, but we were delivering
to terminals at remote locations.
CAI in reading and mathematics
These terminals were teletypes that
to underprivileged elementary age
would print character-by-character
students who were located in another
what was sent by the computer. They
county some twenty-five miles away.
would also accept keyboard input
Reflecting back on that experience
from the students and transmit it
now, I would call that e-learning by
back to the IBM 1500 system as if the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
106
Michael Allenʼs 2008 e-Learning Annual
student were sitting in the adjacent
now. Figuring in all the factors we
room at a terminal hard-wired into
could consider and pricing them
the computer. There were no PCs, no
appropriately, the actual costs of
Internet, no multimedia, no email,
providing CAI once we did a project
and no discussion forums, but we did
would always outstrip our estimates of
have students at a remote location
the cost by a factor of ten. We got so
completing CAI lessons. As we say
we would chuckle after doing a careful
today about e-learning, they were
estimate of the cost of providing CAI
learning at their place, at their pace,
in some situation when somebody
and at the time of their choosing. It
would mention something to effect of,
was any place, any pace, and any time
“Now that you have the estimate done,
learning. Isn’t this how we describe e-
move the decimal over one place.”
learning today?
Invariably this would wind up being a
The economics of early e-learning
better predictor of the real costs as $1
were very different than the economics
at the time of estimate became $10 or
of today’s e-learning. There were few
more when the project was done.
computers capable of supporting CAI back then. Florida State had one of
Blended Learning Largely because
only seven prototype 1500 systems.
of the expense of developing
The cost of computer time was high
and delivering CAI, we began to
in those days of mainframe-only
experiment with another form of
computing, and even simple CAI
instruction in which all the content
required considerable computer time.
was not delivered via the computer.
The time and costs of developing
Rather we gave students packages
a CAI lesson were also quite high.
of printed materials to read and
Estimates ranged from 150 to 400
had them use the computer only
hours of development to create
for testing, obtaining feedback, and
one hour of CAI. The long distance
receiving their next assignment based
charges of a dedicated telephone line
on their test results. If students
were also high back then, making the
mastered the content, as shown by
overall costs of remotely delivering an
their responses to criterion-referenced
hour’s worth of CAI to one student
test items, they were directed to the
prohibitively expensive regardless of
next lesson. If they failed to master
the effectiveness.
the content, the specific objectives
I still recall our estimates of costs to deliver CAI with some humor
they failed were noted and they were directed to additional readings
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
107
and reference materials associated
As before, a student who was not
with those objectives. When they
successful could meet with his or her
completed these assignments, students
instructor. Pick up any journal three
would sign back on the computer
decades later and you will read about
to take an alternate form of the
the advantages of blended learning
assessment for their content before
in which all the instruction does
being allowed to advance. Students
not happen on computers—some is
failing to reach mastery on this second
online, some is face-to-face. As with e-
attempt were directed to schedule an
learning, blended learning is a recent
appointment with the instructor or a
term with a long past.
teaching assistant for some coaching and mentoring before they could
Incorporating Graphics From
continue in the course. We called
the work with computer-managed
this computer-managed instruction
instruction, “blended learning” if
(CMI) to distinguish it from CAI in
you like, we turned our attention to
which all the instruction happened
creating computer-based learning
online. In CMI most of the learning
environments that extended beyond
happened offline when students read
text on screens that had typified
instructional materials or met with
most of the CAI efforts at that
their instructors. The computer was
date. Computer graphics were
used to manage learner progress
possible, but only by using ASCII
through the course. In some CMI
characters or laboriously going in
courses, we used those students who
and turning each individual pixel off
had successfully passed certain lessons
or on to form an image. In general,
as peer-tutors for those who had yet to
it was too time-consuming and too
pass those lessons. Then, if a student
expensive to use much in the way of
was not successful on the exam for a
computer-generated graphics with
lesson, in addition to being directed
the technology
to additional resources, he or she
of the day. What
It was too time-consuming
would also be directed to classmates
was possible, and
and too expensive to use
who had passed that lesson. It was not
what we did, was
much in the way of
mandatory that a student meet with
to interface other
computer-generated graphics.
one of the classmates who had been
display devices and
successful, but we tried to facilitate
bring them under computer control.
their getting in touch with each other
We created hardware and software
for discussion about the content.
that enabled the computer to control
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
108
Michael Allenʼs 2008 e-Learning Annual
a slide projector. Thus, when a lesson
processes have strong antecedents.
called for a color image of a giraffe,
I suspect the next “new thing” in
the computer would turn the slide
educational technology will also have
projector on and advance it to the
strong ties to something that was
next slide to display a slide image of
going on several decades ago. Ignoring
a giraffe on a separate screen that
the lessons of past projects and past
we located next to the computer’s
research holds us back and has us
CRT display. We also rigged filmstrip
making unnecessary missteps when
projectors and audio recorders the
we should be advancing the field, not
same way so that they could be
repeating past errors. To this end I
brought under computer control to
offer some observations about lessons
display images and sounds that were
learned.
essential to the CAI lesson. It was
I start by acknowledging I have a
crude and complex to program, but it
bucketful of seemingly good ideas
was early multimedia.
that didn’t work then and likely won’t work now. I cringe when I see these
The Importance of Antecedents
playing out now in ways that are
The central point in each of these
quite predictable. In the early days
examples is to trace one person’s
of CAI, we thought the ideal was
earlier experiences using technology
one student working alone on one
for instruction to demonstrate that
computer, going through a lesson
most, if not all, of what we think is
that was designed to progress in
contemporary in terms of technology
small steps and through a series of
applications such as e-learning and
questions he or she had to answer
interactive multimedia have been
to receive immediate feedback. This
in existence for several decades. If
was the basic model, whether it was
we expect to use e-learning today
exclusively question/answer sets (as in
to produce significant learning
drill-and-practice CAI) or whether it
improvements, we would do well
contained some initial instruction in
to start by looking back at work
the form of passages to be read before
done four decades ago, not just at
completing the question/answer sets
what people are doing this year.
(as in tutorial CAI). Individualized
Without question, the technologies
instruction reigned supreme, and it
involved have changed greatly—more
was individualized instruction we
than anyone had imagined. Yet the
sought to provide through computer
underlying instructional concepts and
applications to instruction.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
109
We had some interesting findings
were not studying that. I also recall
emerge from our research studies that
some students coming into the CAI
questioned our viewpoint on how to
lab to take part in an experiment
use technology to deliver instruction.
and remarking something to the
We found unexpectedly that students
effect that, “I’m not very good in
could and would learn, even if they
mathematics” or whatever the subject
didn’t have immediate feedback. We
matter involved in the experiment was.
found that repeated practice did not
Usually one of us would try to “pump
lead to better retention. We found that
them up,” saying that “A lot of students
students could learn equally well from
who have said this have done well in
a CAI lesson whose sequence was
the study. Just pay attention and you
randomly scrambled when compared
will learn; everybody has.” At times
with a lesson that was carefully
we would reassure them by saying we
sequenced to promote learning.
weren’t too good at
We found that students who only
mathematics either,
Students could and would
read through a CAI lesson without
but “this lesson is
learn, even if they didn’t have
responding to questions and receiving
very clear and you
immediate feedback.
feedback learned as well as those who
will get it.” Bandura
used the CAI lesson as planned. We
(1997) had not yet described the
found that students who missed many
impact of self-efficacy on learning. In
parts of the lesson by skipping would
hindsight, I think we were attempting
learn as well as those who carefully
to influence self-efficacy, and that is
completed each step in the lesson.
what might have been producing any
I began to appreciate the ability
learning gains we observed, not the
to learn that most students possess.
specific variables under investigation.
They learn even when they shouldn’t!
Causality was and remains a difficult
I remember joking with other
thing to establish, even in controlled
researchers that if we would pin a
experiments.
$10 bill to the top of the computer
Moving forward in time, Apple
terminal and say to the student, “If
and later IBM introduced personal
you pass the test that follows this
computers, and I was excited about
lesson, you can keep the $10,” none of
being able to study computer
the variables we were studying would
applications to instruction without
matter! In hindsight, I think what
all those large costs associated with
I was learning was the importance
mainframes. Finally, CAI would
of motivation in learning, but we
become affordable. Computers
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
110
Michael Allenʼs 2008 e-Learning Annual
were so much less expensive and
chores to create many CAI lessons
so much more widespread. .Newer
across the curriculum. I still recall the
programming languages, such as
day when a faculty member showed up
BASIC and PILOT, allowed for easy
for the initial meeting with the team
content creation. Two decades ago,
assembled to assist him in creating
I worked with several university
CAI and placed a large three-ring
faculty members to create CAI lessons
binder on the table. He said, “Here’s
for their courses. With stronger
my course, call me when you have
programming tools and infinitely less
the computer lessons ready.” Other
expensive computers proliferating,
faculty members were less blunt and
I thought CAI nirvana was at hand!
offered more of their time to the
I knew that soon all students would
effort, yet they continued in a similar
receive most of their instruction in a
manner to take what had been a
very effective and efficient manner by
traditional course and simply place it
having their own personal tutor that
on a computer. The result was seen
patiently would lead them through the
by many as new technology-based
content providing tutoring, support,
instruction—a real advance, several
and reinforcement where needed,
would say.
how needed, and when needed. It
In reality, it was the same old stuff
was all now in reach, but something
ported to a new delivery medium.
happened.
When students went through this
Computers were cheap compared
“new” instruction learning at roughly
with the mainframe computers
the same levels as before, it became
I first used for CAI, and we had
clear that technology alone adds little
many of them. Programming was so
(Clark, 1994). Remember that multiple
much easier and faster as a result
meta-analyses of considerable CAI
of developments in programming
research indicate modest, even trivial
languages. PCs had graphics
differences (effect sizes averaging
capabilities, and you could even get
two to three) when compared with
sound out of them. We had all we
traditional instruction (Jenks &
needed, but the revolution never
Springer, 2002: Kulik & Kulik, 1986;
happened.
Kulik, Kulik & Cohen, 1980; Schacter,
I worked for several years with
1999). Such is also the case with our
different faculty members enlisting
rush into e-learning. Lessons that
the assistance of computer science
are repurposed or ported over will
students to help with the programming
not be better than before. This will
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
111
cause some to question the value of
probably better than most teachers—
e-learning just as some, actually many,
because it could give students more
questioned the value of CAI. When
individualized practice and feedback
was the last time you heard someone
as well as going at each student’s own
talking positively about CAI? Such will
pace. We also recognized that the
be e-learning’s future if it progresses
computer didn’t frown at learners or
along a similar path.
castigate them. It had no knowledge of whether the student sitting at a
Early Hopes
The hopes
terminal was male or female, black or
we had for
white, the mayor’s son or the janitor’s
the earlier
daughter, the fastest learner in the
applications
class or the slowest. The computer
to instruction
worked tirelessly and consistently in
centered around providing each
presenting lessons. We thought we
student with some time each day
could embody the characteristics of
during which she or he would have his
the best teaching in the CAI software
or her own infinitely patient personal
to benefit every learner. We saw the
tutor in the form of CAI that would
weakest students in the class achieving
lead them to master some subject-
well for a change.
matter content. While we were hoping to give the students the best teaching possible, frankly and in retrospect,
Risks and Their Mitigation
We saw our adversaries as
we were working to “teacher-proof ”
the expense
the curriculum as well. We wanted
of mainframe
all students, even those with less
computer
than stellar teachers, to be learning
time and educators who thought
at high rates of mastery. Rather than
using computers would dehumanize
the older model of teacher-to-student
education. We tried to mitigate the
transfer of knowledge, we recognized
costs of computer time by evolving
a new possibility that would bypass
the CMI model in which learners
the teacher, at least for part of the
were online for only a brief time for
day, by transferring knowledge
each lesson, in contrast with CAI in
directly from computer to student.
which learners were online for the
We recognized that the computer per
entire lesson. Later, as PCs emerged,
se did not teach, but we thought the
we realized that computer time for
software we developed would teach—
instruction would become low cost
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
112
Michael Allenʼs 2008 e-Learning Annual
and not a barrier. As for those who
could and should be replaced, at
thought computers would dehumanize
least for that class. If a teacher
education, we tried to explain the
recognizes that all she or he is doing
limitations of having a student sit
is reading notes to the class in the
passively through a lecture that might
form of a lecture, that teacher should
be considerably over his head and
fear replacement by technology. In
frustrate him or considerably below
fact, that teacher should have been
her current level of understanding
replaced by the invention of the
and bore her. Some, perhaps many,
printing press long before computers
teachers were challenged by the
became available. There remain many
technology and had concerns about
uniquely human interactions between
being replaced just as university
faculty and students that even today’s
faculty feared being replaced by the
technology can’t duplicate. Wonderful
printing press centuries earlier and
faculty members continue to inspire,
by the Internet and e-learning today.
challenge, support, and motivate
More than a few faculty today object
students in ways that make technology
to having to put their syllabus online,
look pale. However, too many faculty
much less all their course content.
members continue to do little more
Just as they had previously objected
than present or “cover” content.
to having lectures taped, whether by
Technology does this quite well.
audio of video, today they object to
Our approach to dealing with this
the university “getting” their course
concern early on was to mention that
captured into a digital medium for fear
schools could not afford technology
they will no longer be necessary. That
for all the instruction or even most
is, some faculty members fear being
of it. Faculty were protected by the
replaced by technology.
high costs of providing CAI. We also
My response to fears of faculty
mentioned they could use CAI as a
replacement by technology was then
supplement to help those learners
and is now—those faculty members
needing extra practice they could not
who could be replaced by technology
provide during class. This was what
should be replaced by technology.
has become the blended learning
If a faculty member conducts class
argument—technology should be
by reading his or her notes to the
combined, or blended, with traditional
students or, a more recent trend,
approaches to teaching rather
reading his/her PowerPoint slides to
than delivering all instruction via
the students, that faculty member
technology. We also used the drudgery
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
113
argument that goes something like
In hindsight, I think the endorsement
this: “Find what the teachers don’t
was really for anything but “regular”
like to do and automate it; find what
instruction, not necessarily for the
they enjoy doing and leave it alone.”
CAI lessons themselves. This is like
That is, take the drudgery away from
many a political race in which one
them. Develop CAI for the lessons
candidate has many negatives; votes
they do not like to teach but not for
aren’t so much for the one candidate
the lessons they enjoy teaching the
as they are votes against the other. CAI
most. Then teachers will welcome
had the advantage of not being regular
technology. Such will likely be the case
classroom instruction. e-learning is
with e-learning.
enjoying this advantage today. Perhaps students of today are
Individual Learner Reaction
From the
different, as many
beginning we
have suggested.
Perhaps students of today
tried to measure
They have iPods™
are different, as many
the learners’
and cell phones.
have suggested.
reactions to using
They are used to
computers for instruction because
instant messaging their friends and
we were curious about how they
going online to find information
viewed learning through a computer.
immediately. They have become
One consistent finding was that they
used to immediate gratification,
usually liked it! Their attitudes about
24/7 support, and a fast pace. They
using computers for instruction
hear about a new song at 3:14 one
were positive, as we had hoped.
afternoon and by 3:16 that afternoon
Furthermore, their attitudes toward
they have downloaded that song and
the subject matter itself were more
are playing it on their MP3 players.
positive than before and more positive
Information comes to them through all
than students learning in a traditional
senses using rich, mediated products.
manner. They indicated that they
And it comes quickly. School must
would like to take more lessons and
seem odd and slow to them. First, we
more classes through CAI, and if given
break the content down into separate
a choice between CAI and “regular”
disciplines with sharp boundaries
instruction, they would rather take the
between. Second, we control the pace
class via CAI. We thought this was a
at which they go through lessons,
ringing endorsement from students
even segmenting classes into fixed
for using technology for instruction.
time periods. Third, we start teaching
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
114
Michael Allenʼs 2008 e-Learning Annual
by telling them things, not by asking
organization. A key factor influencing
them things or by allowing them to
instructional use of technology in
control what happens. Fourth, we
organizations is that of opportunity
insist that they pay attention to us
costs. Organizations that value
and only us during a lesson. Fifth,
learners’ time tend to use technology
we allow them no input into what to
for learning when it reduces the time
study, how to study, when to study,
learners spend on a lesson. When
or where to study. How alien all this
corporate employees are taking a
must seem to today’s technology-
class, they are on the payroll and being
using students. They multi-task, move
paid for these hours. Further, when
in many directions at once, initiate
in class, they are not available to the
their own direction for what they
corporation to produce a product
want to know, work in a non-linear
or provide a service. In short, they
manner, bouncing from one topic to
are not bringing a nickel into the
the disconnected next topic, learn by
corporation while they are in classes.
negotiating meaning with peers not by
Thus, anything that can reduce the
one-way communications, consume
time spent in learning and have them
copious amounts of multimedia
back performing their jobs sooner
simultaneously, and expect immediate
is attractive to corporations. Such is
responses to queries. We can’t
also the case in the military services.
compete with this if we stick with
The sooner they can get their men
traditional instruction, but by using
and women through the necessary
technology we can begin to provide
education and training programs,
them with a more comfortable and
the quicker they can be helping
familiar environment in which to
the military accomplish the work it
learn.
does. Technology typically produces just such results by reducing the
Organizational Views of Technology
Organizations,
time required for learning. These
whether school
organizations have a history of
systems,
adopting technology for that reason.
universities, corporations,
On the other hand, educational institutions derive their revenues
governmental agencies, or military
while students are in classes, not
services, adopt, adapt, or reject
when they are out of classes. While
technology in different ways,
corporations and governmental
depending on the nature of the
agencies pay employees while they
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
115
are attending classes, educational
instructor in front of the room talking
organizations do not. In fact, students
and a group of students listening and
pay educational organizations based
taking notes. The role of a learner in
at least to some extent on the amount
schools is being a student. The role
of time they spend in class. When
of a learner in business and industry
we talk about the costs of a public
is being a productive employee. The
education, we rarely figure in the
employees’ job is to produce some
opportunity costs. That is, we assume
product or provide some service.
there is no cost associated with the
If they are in a learning mode, it is
loss of opportunity for students to be
to help them achieve higher quality
doing something other than attending
or efficiency as quickly as possible.
school. Essentially, in K-12 public
Education in a corporate environment
education, we set the opportunity
has a focus and direction grounded
cost at zero and place no value on
in their daily realities as employees of
student time. There is little, if any,
their organizations. What an employee
incentive for schools to reduce class
learns today, he or she is expected
time. In addition, most public schools
to apply right away. Governmental
are required to meet for a certain
agencies and the military services are
number of days each year. University
similar in this regard. The education
classes are required to meet for a
they provide employees is focused
fixed number of hours each semester.
and immediately applied. This allows
Even if technology would enhance the
corporations and governmental
learning process and have students
agencies to try different approaches to
learning and achieving at faster rates,
instruction and quickly see the results
they would still have to meet the
in job performance and even their
required number of hours and days
“bottom line.” This ability to try some
for that class or that semester. When
innovation and see the effects is key
institutions regulate the time required
to accepting and using an innovation
for instruction, they don’t go looking
(Rogers, 2003). Corporations and
for efficiencies. Thus, there is not the
governmental agencies are better
incentive to use technology when its
positioned to try an innovation like
principle advantage is to reduce the
e-learning than is public education.
time required for learning.
Rogers notes that, in addition to
Tradition plays a role in technology
trialability, if an innovation provides
use as well. In educational institutions,
a relative advantage and is compatible
the longstanding tradition is an
with an organization, the innovation
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
116
Michael Allenʼs 2008 e-Learning Annual
is more likely to be adopted. By
there is no magic in the machine.
shortening learning times and
Simply placing instructional content
providing a flexible way to enhance
on a computer and delivering it to
skills of employees, e-learning can be
students does not enhance instruction
a relative advantage to a corporation.
nor guarantee learning. It does
In today’s world of knowledge workers
allow for any time, any place, and
and the creative class, having your
any pace learning, but these are just
employees know more and be better
logistics, not pedagogy. What matters
skilled is what gives any corporation
is pedagogy. A bad lesson placed on
its competitive advantage. e-Learning
a computer is still a bad lesson. It
also fits nicely into many organizations
may be more flexible, but it will not
where time and location have become
produce greater learning. Attention
more flexible, and everybody uses
to the design of instruction remains
technology as an inherent part of
paramount—despite what many
their work. Since e-learning is more
observers have been saying in this
trialable, gives a relative advantage to
high-tech age. It still comes down,
users, and is more compatible with
fundamentally, to a learner trying to
the daily way of life in corporations
make sense out of some new content,
and governmental agencies than it
whether it comes out of the mouth
is in public education, the work of
of a teacher, from the printed words
Rogers would suggest it is more likely
on a sheet of paper, off the screen of
to be adopted in these environments.
a computer, out of the speakers of
I do not expect e-learning to have a
an iPod, as images off the Internet,
significant impact on public schools,
or as video on a DVD. e-Learning
but I do expect e-learning to change
should be about learning, not about
how people learn in non-school
technology. As numerous studies going
situations.
back several decades have shown, technology itself adds very little. I
What Happened?
The revolution
recall a keynote presentation at a
in educational
conference years ago just as interactive
technology that
multimedia was emerging. The
was to follow
presenter, a key figure in the field—
the invention
maybe the key figure—mentioned a
and use of personal computers never
specific interactive multimedia project
happened. Why? The short answer
as setting the direction for the future.
that I learned the hard way is that
Later that afternoon I saw the person
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
117
who was directing the project, an
With considerable effort we
old friend and former colleague, and
developed what seemed to be a
mentioned that she must be thrilled to
pretty good simulation of how an
have such a wonderful endorsement.
anthropologist does fieldwork in a
I asked to see her work so I could see
different culture. When it was time to
the future of interactive multimedia.
test the lessons, we found we had too
It was then that she embarrassingly
many students (it was a large class)
told me they were not funded as well
and too few PCs. Without a blink,
as they would like, so what they had
the anthropology professor placed
done was take some old filmstrips and
three students at each computer and
place them onto an interactive DVDs.
said, “One of you work the keyboard
She said there was no reason to see the
while all of you discuss the simulation
project. We have a long history of the
and decide what to do next at every
latest and greatest being little more
step.” She and her graduate assistants
than old content and old pedagogy
worked in typical anthropologist
delivered by a new technology. Years
fashion by observing the students
of results show the learning outcomes
and taking field notes on what was
are about the same. e-Learning is no
happening. This is certainly not how
different.
I had evaluated CAI lessons; we
I mentioned the high regard we all
always used pretests and posttests
held for individualized instruction. In
to measure knowledge gains. Our
many ways everything we were doing
evaluation would have missed what
and had been doing with technology
was happening as multiple students
for several decades centered on trying
completed each CAI simulation
to deliver individualized instruction to
together. They read the situation
learners. I don’t remember any of us
displayed on the screen and looked at
questioning this direction during the
options that were available for their
early years, but something happened
next move. But before deciding what
twenty plus years ago when I was
to do, they had rich discussions and
working to help an anthropology
debates. One student would describe
professor create CAI in the form
what she thought was happening and
of a simulation for her class of
what the best option was, while others
undergraduates. The experience left
would question her. Then another
me wondering about individualized
student would describe how he saw
instruction.
the situation and the available options and so on. They would talk through
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
118
Michael Allenʼs 2008 e-Learning Annual
their perceptions and thoughts as they
2. Do more, much more than having
negotiated meaning and discussed the
learners read text or view graphics
best option to select next. Once they
in lessons; don’t turn technology
agreed, the designated keyboarder
into a textbook, workbook,
would enter their collective response
videotape, or filmstrip.
to see what would happen, and
3. Design lessons that require human
they would all begin to process
communication, not individualized
the feedback. I saw individualized
instruction; allow and support
instruction destroyed before my very
learners working collaboratively.
eyes, and it was a beautiful thing to
4. Provide flexibility and encourage
see! The sacred goal of providing
learners to use metacognitive
individualized instruction for every
strategies to monitor their thinking
student was replaced by the goal of
processes while completing
providing a collaborative learning
lessons.
environment for all students, at least
5. Avoid fascination with what
in my mind. If we treat e-learning
technology can do, and keep
as a way to deliver individualized
your focus on what is necessary
instruction, I think we have missed the
for learning to occur; too much
real opportunity. People learn quite
multimedia gets in the way of
well from and with other people—
learning as the gee-whiz factor
maybe better.
takes over and learners pay attention to the technology and
Lessons Learned
If I compiled my
not the instructional content.
experience from
6. Provide structure to lessons and
working with
provide guidance to learners as
technology into
they complete lessons, but set
a set of lessons
learners free to do as they wish
learned about e-learning, it would include the following: 1. Engage and involve the learners
during a lesson. 7. Understand as much as you can about the learners, especially their
in active experiences, rather than
prior knowledge of the content,
leading them through carefully
before you begin to design lessons
prescribed lesson sequences.
for them.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
119
8. Provide informative feedback
14. Think minimalist; rather than
to learners that helps them
designing lessons that include all
understand and correct any
the content you can and doing
misconceptualization or erroneous
everything that can be done with
information they may have about
the technology, provide only the
the content; don’t be concerned
content essential to mastering the
about rewarding or reinforcing
objectives; provide technology
them.
effects such as animations only
9. Design interactions that require
when absolutely necessary
more than clicking to continue to
to support specific learning
the next screen.
outcomes. Otherwise, it’s just
10. Design lessons that adapt to learners to keep the level of
distraction. 15. Watch learners, talk with learners,
challenge appropriate by allowing
listen to learners, see how they use
them to skip what they already
technology, see how they go about
know and not encounter that for
learning, see what bothers them,
which they are not yet prepared.
observe them working through
11. Give learners choices, but give
the lessons, pay attention to them,
them advice on selecting from
and involve them in the design of
among choices.
e-learning.
12. Design lessons that include a
16. Design e-learning that encourages,
digital mentor who functions as a
supports, and enables play on part
student’s grandmother would, by
of the learners during lessons; pay
providing support and advice as
attention to the engaging features
needed to guide learners through
of computer games to see what
the e-learning lessons.
works.
13. Remember that a content dump is
17. Elicit cooperation from learners
not a lesson; build in pedagogical
by being clear about why they
functions that have been shown to
are doing this lesson and discuss
enhance learning.
expectations regarding what they will get out of it.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
120
Michael Allenʼs 2008 e-Learning Annual
18. Start your lesson design based on
22. Remember the science of designing
empirically validated principles
lessons for e-learning, but don’t
of learning; then take suggestions
forget the art. Remember the art
from intended learners on how
of designing lessons for e-learning,
to improve the lessons. When in
but don’t forget the science.
doubt, listen to the learners.
23. Expect revisions, plan for
19. Don’t attempt e-learning unless
revisions, allow time for revisions,
you have someone who is a
seek input for revisions, and make
virtuoso with the content, someone
revisions even to your best e-
who is an extremely skilled
learning lessons.
instructional designer with both
24. Interactivity is vital—learners
academic and practical credentials,
need to interact with the content;
someone who has successfully
learners need to interact with
taught the content more than once,
the instructor; learners need
and someone who is skilled in
to interact with other students.
developing for the technology but
Forget any of these and e-learning
understands that pedagogy should
will be less successful.
drive the technology. 20. Do whatever is necessary to keep
25. First figure out what will help someone learn the content, then
from repackaging old content into
figure out how to do that with
e-learning and from making e-
technology—never work the other
learning work just like some older
way around.
technology. 21. Remember that most studies
26. Respect copyrights and protected materials, but use whatever you
comparing technology-based
can find that fits what you are
instruction with traditional
trying to do in an e-learning
instruction find no significant
lesson. Develop new content only
differences. To be worthwhile,
as necessary and be sure that your
e-learning has to be carefully
design, not what you find, drives
planned and crafted, not thrown
what you use.
together, as the technology alone will add no value to the learning.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
121
27. Pay attention to how something
another course that had full online
you are doing with e-learning
facilitation, high quality instructors,
will impact what is going on in
streaming video, and people to
the learners’ heads. If something
respond to any question from
in your e-learning lesson is not
students. Over 80 percent dropped
facilitating an internal mental
out of this e-learning program. By
process associated with learning,
contrast the completion rate for
then don’t do it.
their instructor-led programs was nearly 100 percent. It is not unusual
What Matters
My fascination
to find that, of those who sign up for
with technology
an online course, between 50 and 80
in the form
percent do not complete the course.
of personal
While I can’t know their motivation
computers
to withdraw, I do know that many
remains, but I am more cognizant
report that technology-based courses,
of what matters now. Immediate
from the early CAI courses to the
feedback is not essential. In fact, when
latest e-learning courses, are neither
the learning outcomes are complex,
particularly interesting nor engaging.
immediate feedback is not desirable. It
Many courses are little more than
is not necessary that learners move in
a boring “content dump”—like the
linear sequences through small steps.
professor who brought his three-ring
This is likely to slow down and even
binder and told us to “put this on the
impede learning, possibly contributing
computer.”
to the large numbers of students who
Repurposing
fail to complete lessons delivered via
instruction from
from one medium to another
technology. Technology may allow any
one medium to
has never enhanced the
time and any place learning, but this is
another has never
instruction.
not enough. Such is becoming the case
enhanced the
with e-learning.
instruction. Witness the early
A national provider of IT training
Repurposing instruction
videotaped lectures obtained by
reported that only 30 percent of the
placing a video camera in one of the
people who started an e-learning class
seats when a professor lectured. The
completed it, despite numerous emails
video lecture will certainly not be
and telephone calls from the instructor
better than seeing it live. Such was the
and course manager (Van Liew, 2006).
case with early film in which a filmed
The same provider followed this with
play became a movie. It may have
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
122
Michael Allenʼs 2008 e-Learning Annual
been a decent play, but it certainly
to make optimistic predictions about
made a poor movie. A DVD player
e-learning—and even more optimistic
does not improve a filmstrip, nor does
predictions about m-learning, or
a computer improve on a workbook.
mobile learning. However, I also made
Incidentally, much tutorial CAI that
optimistic predictions about CAI and
was thought to be new and wonderful
even more optimistic predictions
was the same instruction students had
about CMI. Really, how many have
received in schools a half-century ago
even heard of CMI? e-Learning can
as workbooks—simply repackaged.
easily have the same future. The
Of course, the research continues to
path to oblivion for any technology
demonstrate no significant differences:
in education is clear: Ignore the
It’s often simply the same instruction
lessons that could and should have
delivered two different ways.
been learned from working with the
So now we add e-learning to
previous technology and let the focus
the mix. If we deliver the same
shift from learning to the technology.
instructional content through e-
Then it is doomed.
learning, we will get the same results. The future of e-learning does not have to be rosy; it may
Future Directions
The optimistic future of e-
follow a long tradition of technology
learning would
in which early optimism waned as
realize the goals
results began to accumulate until
put forward
finally it was dismissed. Then the
for technology in education several
next technology came and the tune
decades ago: Learners would have a
started again. I know no one excited
rich curriculum delivered to them
about instructional radio, yet many
in a form from which they could
fervently embrace pod-casting. I
readily learn. Technology would
know no one excited about filmstrips,
be there whenever they wanted to
yet many praise multimedia. I know
learn, however they wanted to learn,
no one excited about CAI, yet many
whatever they wanted to learn, at
are rushing to e-learning. Pod-
the pace they wanted to learn, and
casting, interactive multimedia, and
it would treat each student fairly,
e-learning can easily go the way of
providing what they needed without
instructional radio, filmstrips, and
prejudice. So what would that future
CAI if we don’t heed the lessons that
look like?
are there for us to see. I am inclined Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on e-Learning
123
My hope for e-learning is that
content, not just distribute the same
the young person in rural Montana
old content to learners as has been
or rural Mississippi would have
the history of repackaging the same
the same access to superb learning
old content for a new technology. I
environments as the young person
would like to see more conversation
in the well-to-do suburban schools
among people applying e-learning that
of Cambridge or Alexandria has
focuses on teaching and learning and
today. I want the worst schools in the
fewer conversations focused on the
country to be almost as good as the
technology. Rather than seeing many
best schools. More broadly, I hope the
e-learning developers taking the same
young person born in West Africa will
first step, I would like to see some e-
have access to fine instruction, not
learning developers who have watched
just the young person born in Finland.
others take the first step and now take
To some extent technology can be a
the second step to move e-learning
much-needed equalizer in educational
forward. I would like to see the move
opportunity.
from e-learning to m-learning, making
I would like to see e-learning take
access affordable and more portable.
the theory and research on human
I would like to see richer lessons in
learning seriously and design lessons
terms of depth of content and quality
accordingly instead of slapping
of pedagogy. I would like to see people
something up on a webpage and
learning with the enthusiasm for
calling it e-learning. I would like to
learning that a four-year old has. I
see our content management systems
want to see e-learning thrive where
become something more than shovels
CAI and CMI did not. I want to see
to move content around. I would like
learners smile.
to see them add some value to the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
124
Michael Allenʼs 2008 e-Learning Annual REFERENCES Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman. Clark, R.E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53, 445-459. Clark, R.E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21-29. Hannum, W.H. (2001). Design and development issues in web-based training. In B. Khan (Ed.) Web-based training. Englewood Cliffs, NJ: Educational Technology Publications. Jenks, M.S., & Springer, J.M. (2002). A view of the research on the efficacy of CAI. Electronic Journal for the Integration of Technology in Education, 1(2), 43-58. Kulik, C., & Kulik, J. (1986). Effectiveness of computer-based education in colleges. AEDS Journal, 19, 81-108. Kulik, J., Kulik, C., & Cohen, P. (1980). Effectiveness of computer-based college teaching: A meta-analysis of findings. Review of Educational Research, 50(4), 525-544. Mühlhäuser, M. (2004). eLearning after four decades: What about sustainability? In Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2004 (pp. 3694-3700). Chesapeake, VA: AACE. Rogers, E.M. (2003). Diffusion of innovation (5th ed.). New York: The Free Press. Schacter, J. (1999). The impact of education technology on student achievement: What the most current research has to say. Milken Family Foundation. Retrieved July 2, 2007, from: www.mff.org/publications/publications. taf?page=161. Van Liew, R. (2006). e-Learning’s “dirty little secret.” Retrieved July 2, 2007, from: http://traininghott.com/E-learning-Secrets.htm.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Thomas C. Reeves Thomas C. Reeves is a professor of instructional technology at The University of Georgia. He has developed and evaluated numerous interactive multimedia programs for both education and training, and given numerous presentations and workshops in the United States and twenty-three other countries. His research interests include evaluation of instructional technology, socially responsible educational research, authentic learning tasks, and instructional technology in developing countries. In 2003, he was the first person to receive the AACE Fellowship Award from the Association for the Advancement of Computing in Education. His Interactive Learning Systems Evaluation book (with John Hedberg) was published in 2003.
William D. Aggen William D. (Bill) Aggen has over twenty-five years of experience producing various forms of e-learning and computer-based training programs in academic, corporate, and entrepreneurial work environments. He is currently an independent consultant performing project management and instructional design activities for national and international clients. Most recently, Bill was founder of Learnwright, Inc., a training-focused technology leader that creates and delivers web-based, multimedia e-learning courses to pharmaceutical and biotech drug and medical device companies operating globally in FDA-regulated industries following Good Manufacturing Practice (GMP) and Quality System Regulations (QSR). He has developed numerous award-winning interactive multimedia programs in a variety of sectors, including military education, science education, continuing medical education, and compliance training.
Lessons from the STARS Revisited
Thomas C. Reeves William D. Aggen
Using their experience with STARS, a literacy training program developed for the U.S. military, the authors extrapolate eight lessons for teams working on e-learning projects. The lessons—dream, work, plan, communicate, evaluate, reward, manage, and enjoy—are critical to the successful development of e-learning projects as illustrated by the lessons learned by the authors more than a quarter century ago. Introduction
Most young
from the interactive CD-ROMs they
adults born
began to play with as children twenty
after 1980 are
years ago.
considered to be members
As members of the Baby Boomer generation, we can remember the
of the generation known variously as
earliest days of interactive video when
Millennials (Howe & Strauss, 2000),
the “Winky Dink and You” television
Generation Y (Martin & Tulgan, 2001),
show premiered on a Saturday morning
or the Net Generation (Oblinger &
in 1953 (www.tvparty.com/requested2.
Oblinger, 2005). Regardless of what they
html). Special pens and a magic plastic
are called, some of the oldest among
screen placed over the TV screen
them have begun to play significant
allowed children to “interact” with host
roles in the design, development,
Jack Barry’s instructions by drawing
and/or implementation of e-learning
a car or decoding a secret message.
programs, either having completed
The interactions were quite crude by
a master’s degree in instructional
today’s videogame standards, but they
technology or a related field, or perhaps
were creatively interactive, especially
entering directly from an undergraduate
considering that the show was
program in computer science or a
broadcast live.
related field. It is doubtful that many
Our experience with the pioneering
of these younger e-learning designers
days of interactive video may have
are aware that interactive video has
planted seeds that came to fruition
been around for more than fifty years.
in 1980 when we served as members
Instead, we suspect that most of
of the University of Maryland
them conceive of the earliest days of
University College (UMUC) team
interactive multimedia as stemming
that collaborated on the design,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
128
Michael Allenʼs 2008 e-Learning Annual
development, and implementation
(about $60,000 today). The STARS
of the STARS (Space Time Army
delivery system featured a Cromemco
Reconnaissance System), one of
microcomputer and Pioneer videodisc
the earliest large-scale interactive
player integrated with state-of-the-art
videodisc programs. Funded by the
graphics cards to enable single screen
U.S. Army, STARS was an interactive
display of video from the videodisc
multimedia simulation designed to
with computer graphics and text from
help soldiers develop better functional
the computer. Soldiers interacted
literacy skills. Our own needs
with the program primarily using
assessment revealed that the reading
a touch screen, but a keyboard and
levels of the technical manuals used by
mouse were also available. The system
the military exceeded the functional
also integrated digitized audio with
literacy skills of as much as 40 percent
sound recorded on the videodiscs.
of the junior enlisted personnel in the
All programs were stored on the ten
Army at that time.
megabyte hard drive (which seemed
Working most of the time at
huge at the time) that had to be
the Heidelberg, Germany, campus
updated using eight-inch diskettes
of UMUC and other times at the
that were truly floppy. The operating
University of Maryland main campus
system was CPM (Control Program for
in College Park, Maryland, we were
Microcomputers) and the authoring
challenged to develop an interactive
language used to create the interactive
learning environment that could be
program was a prototype version of
deployed at remote sites throughout
PILOT (Programmed Instruction,
Europe where U.S Army units were
Learning, or Teaching).
guarding the frontiers of the Cold
While considerable stress was
War. This functional literacy training
experienced by all of the UMUC
program had to be accessible twenty-
team members at the time of the
four hours a day, highly motivating,
development of STARS, largely
efficient, and, most importantly,
stemming from equipment failures,
effective in helping soldiers learn
programming bugs, budget
to read and apply the contents of
constraints, and unrealistic timelines,
sophisticated technical manuals.
both of us, and we suspect all of the
The STARS program was developed
other members of the STARS team,
for delivery via an early prototype
look back on the twenty-four-month
interactive videodisc system that cost
design, development, implementation,
approximately $25,000 per unit in 1980
and evaluation effort as one of the
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
129
most exciting professional activities
At least part of this problem may
we have ever experienced. Inevitably,
stem from the failure to do what the
we learned many lessons in the
Disney people call “imagineering,” that
process, and thus the purpose of this
is, coming up with original solutions
article is to look back and review those
to design challenges, solutions that
lessons, partly as a reminiscence for
combine the best of what we know
others like us who participated in
with the best we can envisage. To
those trailblazing days of e-learning,
allow us to maximize our creativity in
but more so for members of the Net
the STARS project, we held a week-
Generation who have been challenged
long brainstorming session involving
to create the e-learning counterparts
all design team members and a
to STARS at this time.
professional facilitator. During this session, no ideas were too outlandish
Lesson One— Dream
The first lesson
as we covered scores of large sheets
we learned
of paper with drawings and words
from the STARS
to express our “thinking outside the
project is that
box” design concepts. Of course, this
designing
“imagineering” session was only held
an engaging interactive simulation
after several
requires imagination and creativity.
weeks of needs
Designing an engaging
Ever since the dawn of computer-
assessment,
interactive simulation requires
aided instruction (CAI) on mainframe
learner analysis,
imagination and creativity.
computers in the 1960s, a very
and other front-
frequent complaint has been that
end work (Rossett, 1987, 1999).
interactive learning programs all too
The research literature provides
often simply duplicate what could have
numerous tools and strategies for
been done with less expensive media.
encouraging creativity (Adams, 1987;
Even today, many web-based learning
de Bono, 1992), but the sine qua non is
environments resemble page-turning
freedom from judgment. An important
print materials or at best interactive
aspect of the STARS brainstorming
PowerPoint presentations. Despite the
week was that we designed the ideal
availability of valid design principles
program before considering the real.
(Aldridge, 2006; Allen, 2003; Clark &
We encouraged all of those present
Mayer, 2003; Quinn, 2005; Rosenberg,
during the week to ask themselves
2006), most e-learning programs still
how they would help soldiers learn
fail to be engaging.
functional literacy skills if they were
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
130
Michael Allenʼs 2008 e-Learning Annual
completely unfettered by spatial,
computer graphics were state-of-
temporal, or financial constraints.
the-art at the time. For the World
So what was creative about STARS?
War II scenario, actual World War II
First, the essence of the STARS
footage was edited together with a
program was a time-travel simulation
superimposed robotic assistant, and
that involved soldiers in a series of
for the future scenario, video was
elaborate interactive adventures in the
recorded in buildings that were as
past and the future. In the first part
“futuristic” as we could find in and
of STARS, soldiers traveled back in
around Washington, D.C., in 1980.
time to the Battle of the Bulge during
Our budget only allowed us to hire one
World War II, and in the second part,
professional actor, but we were able to
soldiers leaped three “xergons” into
recruit several volunteer actors from
the future to avert a catastrophic
the University of Maryland campus in
conflict. In both adventures, soldiers
College Park, Maryland, where most of
confronted realistic problem-solving
the actual production was completed.
situations that required application
Additional scenes for the second part
of a range of functional literacy skills
of STARS were filmed on a military
to solve. If a soldier demonstrated
base in Colorado using real soldiers
the required skill and thereby solved
operating their actual equipment.
the problem, he/she advanced in
Interestingly, although the
the simulation, earning “time-warp
production values would seem
energy units” that would be needed
unpolished today, we will make
eventually to return to the present. If
the argument that the pedagogical
the soldier’s performance indicated a
(or instructional) design of STARS
lack of a certain literacy skills, he/she
was advanced and would even be
would be branched into an interactive
seen that way today. The major
lesson keyed to that specific skill
instructional model incorporated
taught within a military context. After
into STARS was learning through
successfully completing a mini-lesson,
authentic tasks, a line of research that
the soldier would return to the STARS
we are still pursuing (Herrington,
adventure for another try at solving
Reeves, & Oliver, 2006). Although
that particular problem.
they were presented in an overall
Although the actual program as
“fantasy” scenario, the authentic tasks
displayed on the screen would seem
incorporated into STARS had real-
somewhat crude if judged by today’s
world relevance for the soldiers who
production values, the video and
used the program. Assessment of their
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
131
literacy skills was seamless with the
Kidder’s award-winning 1981 book,
authentic tasks. Even the lessons to
The Soul of a New Machine, about the
which poor performance in the STARS
development of a new computer at the
simulation would link soldiers were
Data General Corporation, we readily
designed using design principles that
identified with the passion and effort
are still advanced in comparison to
with which the Data General engineers
many of today’s e-learning products.
pursued their ambitious goals. For
At the time of the STARS creation,
example, during the video production
we felt that we were barely scratching
phase of STARS, key staff members,
the surface of what could be done with
including both of us, worked six
interactive multimedia to support
straight weeks without a day off,
learning. Perhaps our proudest
averaging ninety or more hours
achievement is the fact that several
per week. Given the temporal and
award-winning multimedia producers,
financial constraints and the novelty
who came into their own in the mid-
of what we were trying to achieve,
to late-1980s, told us that their work
no amount of planning or time
was partially inspired by what they had
management could have prevented
seen in the STARS program.
these extreme demands from arising. Can effective e-learning programs
Lesson Two— Work
The second
be created within the boundaries of
lesson from
a normal work week? We think they
STARS is
can, especially if time and money are
closely related
not major constraints. However, we
to the first. To
have very rarely
paraphrase Thomas Edison, creating
experienced
Can effective e-learning
an effective e-learning environment
normal work
programs be created within
is 1 percent inspiration and 99
weeks during
the boundaries of a normal
percent perspiration. Throughout
e-learning
work week?
the STARS project, we abandoned
development
the normal forty-hour work week
projects, at least during the most
and often worked far into the night
creative phases of a significant
and throughout our weekends. Most
development initiative. Clearly,
of the key members of the STARS
obtaining individual and institutional
team “signed on” to this project with
support for the kind of commitment
enormous commitment and unflagging
we and our colleagues made to STARS
energy. When we later read Tracy
is more difficult or even impossible
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
132
Michael Allenʼs 2008 e-Learning Annual
in certain types of contexts. For
The PERT chart provided all team
example, in a military training
members with a reference point with
setting, both military and civilian
which to coordinate activities and
personnel are usually responsible for
gauge project progress and, on several
many duties outside the boundaries
occasions, helped us to avert major
of an e-learning development effort
scheduling conflicts that would have
that may prevent their participation
caused significant delays.
in overtime hours. This is why
Today’s e-learning developers have
institutions of higher education and
the advantage of powerful computer-
innovative start¬up companies often
based planning software available
are ideal partners in the design and
from a large range of large and small
development of cutting-edge “proof of
vendors. We have found that the
concept” design projects.
purchase of project management and planning software is only worthwhile
Lesson Three— Plan
The third lesson
if it is matched by an investment
learned from
in learning the ins and outs of
STARS is that a
the software. We are reluctant to
little planning
recommend any specific products
goes a long way
because the market is changing so fast
and more planning goes further. As
and also because frankly we have yet
noted above, no amount of planning
to find the ideal software ourselves.
can avoid unusual demands for time
Adding someone with management
and effort during a truly creative
and planning expertise to an e-
initiative, but inadequate or poor
learning development team, perhaps
planning can foster situations that no
a young M.B.A. graduate willing to
amount of overwork can overcome.
cut his/her teeth on an ambitious
One of the most valuable activities
design initiative, may also be a good
early in STARS was the development
investment. At the very least, hiring an
of an exhaustive PERT (Program
honest and competent consultant for a
Evaluation and Review Technique)
few days or weeks of up-front planning
chart for the project (Cook, 1971).
assistance may also be helpful early
Producing this plan consumed several
in a creative development initiative.
days of key staff time and the services
This was certainly our experience with
of an excellent planning consultant,
STARS.
but it was time and money well spent.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
Lesson Four— Communicate
133
Communication is one of those
“signed on” to the project. As described in Kidder’s (1981)
“buzz words”
book, The Soul of a New Machine,
that finds
“signing-on” is the process by which
its way into
people establish a personal stake in
virtually every human activity, and
a project. The nature of e-learning
thus it seems somewhat trite to
development is that various personnel
state that the STARS team members
will enter and leave the project at
worked hard to maintain open lines
different intervals, while others remain
of communication. And yet, we must
engaged throughout an initiative with
admit that of all the problems we faced
a much greater opportunity to develop
during STARS, the most frustrating
“ownership” of a project. It is no easy
was poor communication between and
task to manage the efforts of ever-
within subgroups of the project team.
changing personnel from an extremely
Although job descriptions were
wide variety of
flexible and several of us fulfilled
backgrounds.
There must be opportunities
several roles simultaneously, the
There must be
for project personnel
STARS team was roughly divided
opportunities for
to meet together to air
into four functional subgroups: (1)
project personnel
misunderstandings.
instructional design, (2) videodisc
to meet together
production, (3) programming, and
to air misunderstandings and vent
(4) implementation. (Management
frustrations. We found that an outside
and evaluation functions were carried
facilitator and a change of location
out by personnel who belonged to all
to a forest retreat encouraged the
the teams.) Numerous efforts were
level of frank communication needed.
made to improve communication,
Exchanging roles from time to
including the establishment of an
time in an effort to walk in another
early form of a listserv, but serious
person’s shoes for a while also helped.
misunderstandings arose from time to
We encouraged an atmosphere of
time. Some of the misunderstandings
mutual teaching and learning on
stemmed from project personnel being
the part of all personnel engaged
located on both sides of the Atlantic
in the STARS initiative, a strategy
Ocean in Heidelberg, Germany, and
that helped to resolve some of the
College Park, Maryland, but the major
communication problems. But again,
contributors were inequities in the
variance in “signing on” led to most
degree to which various personnel had
of our communication problems, and
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
134
Michael Allenʼs 2008 e-Learning Annual
working those differences out is rarely
the development of a creative and
easy.
effective interactive videodisc learning
In today’s increasing “flat world”
system. To be sure, we were aware of
(Friedman, 2005), e-learning
the design principles derived from
projects often involve international
learning, communications, and media
collaborators wherein subject-matter
theories at the time, but they did not
expertise may reside in one country,
add up to instructional formulas or
instructional design in another, and
prescriptions that could be blindly
production and programming in yet
followed. Instead, numerous types
a third country. Although we now
of review, needs assessment, and
have access to sophisticated Internet
formative evaluation were required to
meeting tools such as WebEX (www.
drive the instructional design process
webex.com) and its competitors,
(Reeves & Hedberg, 2003). No amount
communication may be more difficult
of creative effort can yield e-learning
than ever, especially when different
programs immune to the refinement
cultures are involved. We have
process provided by rigorous
reviewed several e-learning programs
formative evaluation.
in recent years developed for Fortune
Most importantly, formative
500 companies by international
evaluation should be focused on
development teams and found that
products, projects, and programs,
the resultant products were seriously
not directly on people. A formative
flawed. We suspect that more than
evaluation focused on products and
a few of the flaws stemmed from
processes rather than personnel was
miscommunications.
the strategy we used in STARS and it is one we have advocated for the past
Lesson Five— Evaluate
Another
twenty-five years and more. Another
important
important principle related to the
lesson from
“evaluate” lesson is that evaluation
STARS is that
methods should be aligned carefully
instructional
with evaluation questions that are
design without continuous evaluation
keyed to specific design decisions
is a waste of time. In 1980, we knew
that the development team faces.
precious little about the science of
Evaluation is not an end in itself, but a
instruction or the art of design that
means to better decision making.
would dictate or even adequately guide
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
Lesson Six— Reward
135
The next STARS
important for us than the financial
lesson concerns
benefits.
how to reward
Long hours and enormous effort
members of
inevitably take their toll, and even the
a creative e-
most dedicated team members require
learning development team for the
respites from time to time. In addition
extraordinary commitments that
to vacations, we have found that
they may be called on to make to a
attending professional conferences
project. Although the first reward
to demonstrate the products of our
that may come to mind is financial
labors has been a great reward for
remuneration, we found this to be
ourselves and others. e-Learning
a relatively insignificant source of
project management personnel should
reward during the STARS project. We
share such professional presentation
were all working for the University
opportunities with as many team
of Maryland University College,
members as possible. In academic
an institution that paid decent, but
contexts, publications are usually
certainly not high, salaries. Of course,
important, and thus opportunities to
psychologists have reported that
publish articles about various aspects
job satisfaction is usually derived by
of an e-learning design initiative
several factors of which money is only
should also be encouraged.
one (Spector, 1997). We found that one of the most
Although it may sound naïve in the 21st Century, we have always
rewarding aspects of the STARS
found that intrinsic rewards are much
project was the freedom to see
more powerful
things in new ways and express our
than extrinsic
Personal satisfaction with a job
creativity (see Lesson One above).
ones. Personal
well done and the simple joy of
The creative experience of designing
satisfaction with a
collaborating with others in a
STARS and numerous e-learning
job well done and
creative design process are the
projects since then has been a major
the simple joy of
best rewards.
factor that has kept us and many of
collaborating with
our STARS teammates in the field
others in a creative design process
of instructional technology for the
are the best rewards. Twenge (2006)
quarter century plus since STARS.
has conducted extensive research
The psychic rewards of collaborating
that supports the conclusion that the
on cutting-edge projects with other
workers in the current Net Generation
creative people have been much more
may not be as turned on by intrinsic
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
136
Michael Allenʼs 2008 e-Learning Annual
rewards as the Baby Boomer
as the failure resulted from sincere
generation was, a trend that may
efforts to achieve something new and
prove to be detrimental to the types of
more effective.
advances needed in the design and use of e-learning.
On STARS, we found that another essential function of management was managing relationships with
Lesson Seven— Manage
One of the
people and organizations outside the
STARS team
development team. Frankly, to be
members said,
creative in an academic environment
“Finding people
sometimes demands stretching, or
who can work
even breaking, “the rules” from time
together as a team is probably the
to time. For example, occasionally
most important thing a manager
purchases must be made in an
can do.” We learned in STARS that
expedient manner that may not
the essence of good management is
conform to state or university
team building. Creative individuals
purchasing procedures. Good planning
inevitably have a wide variety
(see Lesson Three) can eliminate
of personalities, attitudes, work
some of these difficulties, but a truly
styles, and communication skills.
creative project will usually involve
Instructional design, programming,
some unusual “tests” of the system.
video production, and other e-learning
Managers may also have to buffer
development roles often attract widely
the development team from problems
diverse types of people, and there is
involving subcontractors and
also considerable variance within each
commercial vendors. It would be ideal
of these subgroups. Project managers
if e-learning designers could always
must continually strike a balance
count on hardware and software
between individual rewards and team
systems to perform as advertised,
recognition. It is also important that a
but this has not been our experience
manager of any project that demands
in the real world of rapidly evolving
creativity not over-manage. e-Learning
interactive technologies. In some
project managers must be prepared to
ways, STARS was like creating a
take risks and give staff members the
motion picture using film that may
freedom to do the same. When things
not develop that is intended to be
go wrong, as they inevitably will at
shown on a projector that hasn’t been
some point, managers must strive to
invented yet. For example, getting a
support their team members as long
videodisc pressed in those days took
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
137
months! The authoring tools we were
(Friedman, 2005). e-Learning has
using were rife with bugs, many of
the potential to engage learners in
which we were the first to discover.
ways that have barely been imagined
All in all, these and other problems
(Allen, 2003). All that is needed to
added to the excitement of the STARS
help e-learning reach its full, and
project. (Or at least that is what we
as yet unrealized, potential is the
are able to tell ourselves after so many
creative genius and committed effort
years.)
of individuals and teams turned on to the excitement and rewards of meeting
Lesson Eight— Enjoy
The final lesson
this challenge.
from the STARS project is that,
Summary
Dream.
if designing
Work. Plan.
e-learning
Communicate.
ceases to be fun, it is probably time
Evaluate.
to move on to something else. There
Reward.
are few kinds of work that are always
Manage. Enjoy. The STARS team was
enjoyable, but if designers lose the joy
surely not the first group to come
of designing, they become stale and
up with lessons such as the ones
their work shows it. The enjoyment
summarized above. These lessons go
of working in e-learning design flows
back centuries. However, it seems
from several aspects, including the
clear that they must be relearned
camaraderie of teamwork, the sense
to some extent in each new context
of achievement that stems from
and time. Perhaps some wisdom can
meaningful effort, and mutual respect
be passed on from those of us with
for creative ideas. As the Disney
“experience.” We hope that this article
people would have it, there is joy in
serves to stimulate the new Net
“imagineering” (Kurtti, 2006).
Generation of e-learning designers to
The need for more effective education, training, and performance
reach for the stars and go much, much further than we did.
support around the globe is enormous
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
138
Michael Allenʼs 2008 e-Learning Annual REFERENCES Adams, J.L. (1986). The care and feeding of ideas: A guide to encouraging creativity. Reading, MA: Addison-Wesley. Aldridge, C. (2006). Simulations and the future of learning: An innovative (and perhaps) revolutionary approach to e-learning. San Francisco, CA: Pfeiffer. Allen. M. (2003). Michael Allen’s guide to e-learning: Building interactive, fun, and effective learning programs for any company. Hoboken, NJ: John Wiley & Sons. Clark, R.C., & Mayer, R.E. (2003). e-Learning and the science of instruction. San Francisco, CA: Jossey-Bass. Cook, D. L. (1971). Educational project management. Columbus, OH: Charles E. Merrill. de Bono, E. (1992). Serious creativity. New York: HarperCollins. Friedman, T.L. (2005). The world is flat: A brief history of the twenty-first century. New York: Farrar, Straus and Giroux. Herrington, J., Reeves, T.C., & Oliver, R. (2006). Authentic tasks online: A synergy among learner, task, and technology. Distance Education, 27(2), 233 -247. Howe, N., & Strauss, W. (2000). Millennials rising: The next great generation. New York: Vintage Books. Kidder, T. (1981). The soul of a new machine. New York: Little, Brown. Kurtti, J. (2006). Walt Disney’s legends of imagineering and the genesis of the Disney theme park. New York: Disney Editions. Martin, C.A., & Tulgan, B. (2001). Managing generation Y. Amherst, MA: HRD Press. Oblinger, D., & Oblinger, J. (Eds.). (2005). Educating the Net Gen. Washington, DC: EDUCAUSE. Quinn, C. (2005). Engaging learning: Designing e-learning simulation games. San Francisco, CA: Pfeiffer. Reeves, T.C., & Hedberg, J.G. (2003). Interactive learning systems evaluation. Englewood Cliffs, NJ: Educational Technology Publications. Rosenberg, M.J. (2006). Beyond e-learning. San Francisco, CA: Pfeiffer. Rossett, A. (1987). Training needs assessment. Englewood Cliffs, NJ: Educational Technology Publications Rossett, A. (1999). First things fast: A handbook for performance analysis. San Francisco, CA: Pfeiffer.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Lessons from the STARS Revisited
Spector, P.E. (1997). Job satisfaction: Applications, assessment, causes and consequences. Thousand Oaks, CA: Sage. Twenge, J.M. (2006). Generation me: Why today’s young Americans are more confident, assertive, entitled—and more miserable than ever before. New York: The Free Press.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
139
M. David Merrill M. David Merrill is an instructional effectiveness consultant, a visiting professor at Florida State University, a visiting professor at Brigham Young University–Hawaii, and professor emeritus at Utah State University. Since receiving his Ph.D. from the University of Illinois in 1964, he has served on the faculty of George Peabody College, Brigham Young University-Provo, Stanford University, the University of Southern California, and Utah State University. He is internationally recognized as a major contributor to the field of instructional technology, has published many books and articles in the field, and has lectured internationally. Among his principle contributions: TICCIT Authoring System (1970s), Component Display Theory and Elaboration Theory (1980s), Instructional Transaction Theory, automated instructional design and ID based on Knowledge Objects (1990s), and currently First Principles of Instruction. He was honored to receive the AECT Life Time Achievement Award. He and his wife Kate together have nine children and thirty-seven plus two (by marriage) grandchildren, which he claims as his most important accomplishment.
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
M. David Merrill
In this article, the author chronicles his career, from his interest in making education more effective to an epiphany about theories. His extensive experience with computer-assisted learning spans from early efforts to teach concepts to more recent work with automated authoring systems. His recent work attempts to identify underlying principles common to most theories of instruction. The author suggests that there should be less emphasis on merely convey information using technology and more emphasis on how to make instruction effective, efficient, and engaging. The Beginning
This article
change my major from engineering to
is a very
education so that I could learn how to
autobiographical
be an effective teacher.
account of my
My major in secondary education
career. Perhaps
was a frustrating experience. While
my personal quest for understanding
I learned how to minimize liability,
the nature of instruction will be
how to create a resource file, how to
instructive to those of you who are
pass out papers with a minimum of
also pursuing this important question.
confusion, and other necessary skills
While I have learned much, I have also
for a classroom teacher, I can only
realized that there is much more to
recall one or two lectures that dealt
learn. If I had a wish, it would be to
with the topic of effective instruction.
be able to begin my career again while
I had a wonderful experience as an
retaining what I have learned.
unconventional student teacher, but
From July of 1957 until July of
caused considerable consternation
1959 I found myself in Indiana,
for the principal of the school where
Ohio, and Michigan, where I served
I was assigned. I was advised that I
as a missionary for the Church of
probably would not survive the public
Jesus Christ of Latter-Day Saints.
schools. It occurred to me that perhaps
Some of our missionaries were very
my decision to major in education
effective teachers; for others, their
was a mistake and that I should seek
attempts to share our message was an
another career path. As I expressed my
embarrassment. I determined that when
frustration to one of my professors,
I returned from this experience, I would
he gave me a challenge that put me
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
142
Michael Allenʼs 2008 e-Learning Annual
on the career path that I have since
scene. Binary arithmetic, base 8,
pursued. In response to my suggestion
base 16, and other representations of
that I abandon education, he simply
numbers were not in the repertoire of
stated, “You could pursue another
a small-town undergraduate student
career, or you could realize that there
scrambling to complete his bachelor’s
is much that needs to be done. Perhaps
degree.
you could make
This particular class in number
Perhaps you could make
positive change
theory was, for this student, a unique
positive change in the field.
in the field.” He
math class: no problems to work, no
suggested that
homework, a very small textbook. At
funding was available to pursue a
the end of each lecture the professor
Ph.D. Not being one to back away
merely said, “Think about it!” Think
from a challenge, I decided to take up
about what? How do you think about
the gauntlet. Applications to several
mathematics? In desperation, and
major universities resulted in a full-
as a substitute for thinking, I read
ride NDEA three-year fellowship to
the textbook every week. It wasn’t
the University of Illinois.
difficult; it had only ninety-seven
In my final semester of
pages and a bright yellow cover.
undergraduate work, a check with
However, the concepts presented
the registrar showed that I was short
floated over my head like clouds in
one hour of credit for my minor in
the sky. I had no idea what the course
mathematics. Naturally there was
was about or what the text was about.
no one-hour math classes, so it was
Each week a lecture, the injunction
necessary to enroll in a three-hour
to “Think about it!” and another read
class. The University of Illinois had
through the text.
already awarded me a fellowship
The midterm exam was a disaster. It
for my Ph.D. study; therefore
had no problems to work, only a single
the completion of the additional
question: “Invent a number system.”
math class assumed considerable
Invent a number system? What in the
significance for my future. A class
world does that mean? In true survival
in number theory appeared, on
mode I wrote for the whole two hours.
the surface, to be the easiest path
However, it didn’t fool the professor.
to the necessary credit. The year
There were seven students in the
was 1961. New mathematics in the
class; there were seven F’s on the
public schools was still in the future.
midterms. When we objected, the only
Computers were just coming on the
explanation from the professor was,
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
“Think about it!” My anxiety was at an all-time high.
proceeded to define a binary number system with two elements, an oar and
My graduate career was about to be
a rubber boot. I was in the professor’s
terminated before it began by the
office the next day to see if I was going
unnerving command, “Think about
to graduate school or not. He handed
it!” I tried every avenue of escape:
me my paper with a large red A
Another class? Getting the registrar to
written across the top. I thanked him,
waive the credit? Home study? There
breathed a sigh of relief, and vowed to
were no other options. My bachelor’s
never take another math class as long
degree, and hence my entrance to
as I lived.
graduate school, were both riding on a
My first year of graduate school
class in which I had received a failing
was very difficult. Not only was there
grade on the midterm and, worse,
a tremendous amount of work, but
a class that was to me completely
there also seemed to be too many
incomprehensible.
contradictions. The content of
Somewhere in the thirteenth week,
learning psychology challenged many
the light came on. Number systems
of my fundamental beliefs. There
are inventions. They are not natural
were numerous contending systems,
phenomena. Number systems are like
each claiming to explain learning. I
any other invention: an assembly line,
struggled for days trying to explain
an organization. A number system is
learning of the concept green using
merely a system of logic consisting
only S-R (stimulus-response) bonds.
of premises and conclusions. Base 10
I found myself in the basement of the
is only one of many possible number
psychology building feeding rats that
systems. Base 10 numbers are useful
were on a deprivation schedule. Why
for many everyday things, but other
was I feeding rats when I wanted to
systems might be equally useful.
know how to teach children? I was
The day of the final arrived. My anxiety was still high, but at least I thought I understood. You guessed it,
about ready to give up and look for a real job. About this time B.F. Skinner visited
only one question, “Invent a number
the campus. Like my fellow classmates,
system.” Either I understood or it was
I went to hear the great man. I don’t
too late. My future graduate studies
remember any details of his lecture,
depended on my ability to invent
but his response in the question-
a number system. So I wrote, “Let
and-answer period changed my life.
there be an oar and a rubber boot.” I
A member of the audience said, “Dr.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
143
144
Michael Allenʼs 2008 e-Learning Annual
Skinner, in your book (which he
how closely it fits, but none are reality,
named) you said such and such (some
merely inventions.”
detail of Skinner’s theory); but tonight
I returned to my studies with
you seemed to contradict yourself by
renewed enthusiasm. I looked upon
saying such and such”—he quoted a
all theories as artificial systems and
part of Skinner’s speech.
found them fascinating. I stopped
“Hell,” said Skinner, “do you think I believe everything I ever wrote?” This was a great “Do you think I believe everything I ever wrote?”
trying to make all theories agree and force them to form one great truth. It became a game to see if I could
insight for me.
identify the theorist’s1 assumptions
Here was a great
and conclusions. It was fascinating
author saying he
to observe that some systems were
changed his mind and now disagrees
carefully constructed and logical,
with his earlier self. However, what he
while other systems were very loosely
said next changed my life.
constructed and often violated the
“What I’ve tried to do,” continued
canons of logic. I realized that theory
Skinner, “is to make only a few
building is our puny attempt to
assumptions and then see how much
understand our world by inventing
of human learning we can explain with
artificial systems and trying them out
only these assumptions.” He went on
against the world.
to defend his theory and the point he
Later in my graduate career I
made in his speech. I stopped listening
had one additional insight. We
before he ended his explanation.
were studying learning and some
“Good grief,” I thought, “psychology
instructional theories. It was apparent
is just an oar and a rubber boot as
that learning theories tended to
well.
explain how persons acquire and store
Psychological systems are not reality
knowledge, but they have very little
either, but merely logical systems that
to say about how an instructor should
try to explain what we observe in the
structure and sequence knowledge
real world. Behavior is merely one
to promote efficient and effective
logical system that is tested against
learning. It occurred to me that one
reality to see how good a match can
could build a logical system, a theory,
be found. Just like there can be many
about instruction. So I said, “Let there
different number systems, there can be
by an instructional oar and a rubber
many different psychological systems.
boot” (from the preface of Merrill,
Each is tested against reality to see
1994).
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
In Pursuit of e 3 (Effective, Efficient, Engaging) Instruction
Early in
I had begun to prepare a proposal
my career I
for doing research on concept
realized that
teaching. Bob Tennyson, whom I had
instructional
not previously met, walked into my
design consisted
office and announced, “I’ve come to
of two primary
work with you.” I indicated that I had
activities: what to teach and how to
no research funds and would be unable
teach. The goal became to build a
to pay him for his work. He offered to
system of instructional design that
work merely for the experience. “Have
followed Skinner’s strategy, that is,
you ever written a proposal?” I asked.
make as few assumptions as possible,
He had not, but was sure that he
form some prescriptive statements
could. I handed him my notes, the call
about instructional design, and see
for a proposal from NSF, and sent him
whether we can prescribe a wide
away. Two days later he put a draft
variety of instructional strategies from
proposal on my desk. It was terrible,
these fundamental components.
but it was a draft. After considerable discussion and several drafts, we
Teaching Concepts
In 1968, I
submitted the proposal. It funded
returned to
Bob’s graduate education for the next
Brigham Young
three years.
University
Our first study (Tennyson,
from a year
Wooley, & Merrill, 1972) found
as a visiting assistant professor at
exceptional results. The best strategy
Stanford University. I was working on
for concept teaching consisted of
an instructional strategy for teaching
presenting a definition, presenting
concepts. My studies of formal
matched examples and non-
concept learning in graduate school
examples, presenting a divergent
suggested that learning a concept
set of examples, and using an easy-
involved presenting examples and
to-hard sequence of examples. Our
non-examples (Bruner, Goodnow,
measure was correct classification
& Austin, 1956). Most of these
of subsequent randomly sequenced
laboratory studies had used formal
examples and non-examples. We
concepts—red circles, green triangles,
also measured classification errors
and so forth. Very little had been done
of over¬generalization, under-
to investigate teaching concepts using
generalization, and misconception.
real-world tasks.
We compared four treatment groups.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
145
146
Michael Allenʼs 2008 e-Learning Annual
We hypothesized that the group that
first volume of AERA Review of
did not have matched examples and
Research in Education (Merrill &
non-examples would over-generalize—
Boutwell, 1973). As we examined
they did; the group that did not have
the literature, it became evident
a divergent set of examples would
that different investigators often
under-generalize—they did; and the
used the same words in reference to
group that had examples that shared
completely different strategies. In
an irrelevant attribute would form a
order to make prescriptive statements
misconception—they did.
about objectives (what to teach) and
We subsequently conducted
instructional activities (how to teach),
additional studies refining our
it was evident to us that there needed
strategies using different age
to be a descriptive language that
subjects and different concepts
allowed precise description of these
(Merrill & Tennyson, 1977a). Our
two aspects of instructional design.
research continued to support our
A way to precisely describe what was
winning strategy. Tennyson spent
taught and how it was taught was
the next several years continuing the
needed.
investigation of concept teaching
While in graduate school I had the
(Tennyson & Cocchiarella, 1986;
opportunity to read The Conditions
Tennyson & Park, 1980). We described
of Learning in manuscript before it
and illustrated our instructional
was published (Gagne, 1965). My
strategy in a book that has since
search for prescriptive guidelines
guided the effective, efficient, and
for effective, efficient, and engaging
engaging design of concept lessons for
instruction was rewarded. I
many designers (Merrill & Tennyson,
immediately resonated with Gagne’s
1977b; Merrill, Tennyson, & Posey,
categories of learning and his
1992). It is still a very effective
assumption that different kinds of
instructional strategy.
learning required different kinds of strategy for effective presentation
The Task/ Content Matrix
In 1972, it was
and assessment. Most of my previous
a surprise and
explorations in learning theory
an honor to
attempted to explain all learning with
be invited to
a single set of principles. It was also
contribute a
difficult to translate these theories
chapter, Instructional Development:
of learning into prescriptions for
Methodology and Research, for the
instruction.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
Emotional
Emotional (Signal Learning)
Psychomotor
Memorization
Complex Cognitive
147
Topographic
Chaining
Complex Skill
(Stimulus Response)
(Chaining)
(new)
Naming
Serial Memory
Discrete Memory
(new)
(Verbal Association)
(Multiple Discrimination)
Classification
Analysis
Problem Solving
(Concept Learning)
(Principle Learning)
(Problem Solving)
Table 1. Ten Categories of Learned Behavior
As we attempted to apply Gagne’s
more logical to us to separate the
1965 categories, we felt that they were
content to be learned from the
incomplete. I had previously attempted
performance of the student with
to clarify his categories by proposing
regard to the content. We suggested
a two-dimensional scheme (Merrill,
three levels of performance: remember
1971) (see Table 1). On one dimension
content, use content, and find new
I put a unit, a chain, and multiple
content. Following Gagne’s lead, we
chains. On the other dimension I
suggested four kinds of content: facts,
included emotional, psychomotor,
concepts, procedures, and principles.
memorization, and complex cognitive
The result of our reflection on
learning. Gagne’s categories are in
how to describe what to teach was
parentheses.
a performance/content matrix for
As we attempted to describe the
classifying instructional outcomes,
content involved in the studies we
at least in the cognitive domain (See
were reviewing, we felt that there were
Table 2 on next page).
some categories missing in the Gagne
Gagne’s category of concept
scheme. His multiple-discrimination
classification corresponded to our
accounted for remembering facts but
use-concept cell; his rule-using
did not obviously include categories
category corresponded to our use-
for remembering the definition of
procedure cell; and his higher order
a concept, remembering the steps
rule category corresponded to our use-
in a procedure, or remembering the
principle cell. Gagne’s problem solving
statement of a principle. It seemed
or higher order rules did not seem
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
148
Michael Allenʼs 2008 e-Learning Annual
Table 2. The Performance – Content Matrix
Find
Use
Remember
FACT
CONCEPT
PRINCIPLE
PRINCIPLE
sufficient for creative behavior with
analysis approach. In a lecture to my
regard to concepts, procedures, and
instructional design class, I proposed
principles. The learned performance
a whole-task approach in which the
of defining a new concept (an activity
analysis started with the simplest
essential to science), of inventing a
version of a whole task and then
new procedure (an activity common to
with successive cycles of instruction
engineering and especially computer
gradually elaborated this task by
programming), and discovering a new
adding complexity. Charlie Reigeluth
principle (the primary goal of science)
was in the class and volunteered
did not appear to be adequately
to write a version of my lecture for
included in Gagne’s scheme. We
publication. This started our work on
added a find performance category to
elaboration theory (Reigeluth, Merrill,
accommodate these types of learning.
Wilson, & Spiller, 1980). Reigeluth went on to investigate this approach
Elaboration Theory
Gagne’s
in considerable detail (Reigeluth, 1983,
hierarchy
1999a). Elaboration theory started
analysis is a
our thinking about whole tasks that
bottom up
led to the principle of task-centered
prerequisite
instruction in our more recent work.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
My father was an artist—a landscape
teacher stresses this observation in
painter. As a small boy I wanted to
class—but I didn’t realize this until
be an artist like my father. I thought
some years later.) I also observed
that if I could just have a large box
that there were really only two things
of Crayola Crayons, I would be able
that an instructor could do with
to paint beautiful landscapes like my
content: present it to the student
father. For every birthday, I would
or ask the student to remember or
request a large box of crayons. I was
use the content. Combining these
unaware of the precarious financial
two dimensions led to the primary
affairs of my parents. My dad never
presentation form matrix. The
said they couldn’t afford a large box
content dimension had two values:
of crayons. He merely sat down with
generality (for the general case) and
four crayons—red, yellow, blue, and
instance (for the specific case). On
black—and a piece of paper and
the instructional dimension we called
colored a beautiful landscape. By
presenting expository and application
example, he taught me the primary
inquisitory. This led to four primary
colors and mixing pigments. For my
presentation forms (PPF): expository
birthday I would get a box of eight
generality (EG), expository instance
Crayola Crayons. My dad would point
(Eeg), inquisitory generality (IG) and
out that I had four more crayons than
inquisitory instance (Ieg). We used
I needed to paint beautiful landscapes.
the symbols in the parentheses as
I didn’t become an artist, but I never
shorthand for describing instructional
forgot my primary colors and how to
strategies. This allowed us to
combine them to get all the colors of
unambiguously describe almost any
the rainbow.
instructional strategy in terms of the
As I attempted to find a way to
primary presentation forms involved.
describe instructional strategies, it
For example, EG1, EG2, EG3, etc.,
occurred to me that there must be
describes a lecture presenting a series
some primary presentation forms from
of generalities without examples.
which all instructional strategies could
The numbers indicate different
be constructed and thus described. As
generalities, as contrasted with a
I studied the instructional strategies
repeat of the same generality. The
in the literature we had collected,
strategy EG1 Eeg1, EG2 Eeg2, etc.,
it occurred to us that there are two
describes a presentation of a series
levels of content: a general level and
of generalities, each followed by a
a specific level. (Every composition
corresponding example. A discovery
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
149
150
Michael Allenʼs 2008 e-Learning Annual
strategy might be represented by Ieg1,
presentation form comprised a display
Ieg2, Ieg3, IG from which the learner is
to the student. These displays are
to discover the generality from a series
the components of an instructional
of examples that he or she must figure
strategy, hence, component display
out.
theory.
We also realized that there were
CDT, as it came to be called,
secondary presentation forms (SPF)—
consists of three parts: (1) a scheme
teaching activities that enhanced
for describing the content to be
learning but were not directly related
taught–the task/content matrix; (2)
to presenting or applying the content.
a scheme for describing instructional
Our attempts to unambiguously
strategies—primary presentation
describe instructional strategies also
forms, secondary presentation forms,
caused us to recognize a number
and interdisplay relationships; and (3)
of relationships among the primary
a set of rules that relate the two. CDT
presentation forms that we called
identifies which combinations of PPF,
interdisplay relationships (IDR).
SPF, and IDR are most appropriate for teaching each cell in the task/content
Component Display Theory (CDT)
During the 1970s, I was
matrix. Selecting Instructional Strategies
director of the
and Media (Merrill & Goodman, 1972)
Brigham Young
presented an early version of CDT.
University
The Instructional Strategy Diagnostic
Division of Instructional Research,
Profile (Merrill, Reigeluth, & Faust,
Development, and Evaluation. Our
1979; Merrill, Richards, Schmidt,
students during this period conducted
& Wood, 1977), later called The
scores of research studies exploring
Instructional Quality Inventory (Ellis,
many aspects of the task/content
Wulfeck, Merrill, Richards, Schmidt, &
matrix, primary presentation forms,
Woods, 1978), also presented a manual
secondary presentation forms, and
to train people in developing effective
interdisplay relationships. As a result
instructional strategies based on our
of this work, I formalized our content
early work on CDT. The presentation
classification scheme and strategy
of CDT in the Reigeluth Green Books
description into what came to be
(Merrill, 1983, 1987a, 1988) provided
called Component Display Theory
the wide spread dissemination of this
(CDT). The name deserves some
work. The most complete presentation
explanation. We felt that each primary
of CDT is found in Merrill (1994).
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
TICCIT
In 1966, I visited
The challenge of the project was
Vic Bunderson
to design an authoring system. We
at the University
struggled with many ideas. Bunderson
of Texas. He had
proposed a system that had a variety
established a
of different instructional approaches
computer-based instruction laboratory
built into the system. The learner
that involved an IBM 1500 system,
could then select the instructional
one of the early main¬frame CBI
approach they felt was best for them.
systems. This was the beginning of
Harvey Black, a colleague at Brigham
our collaboration. We vowed at this
Young University and a collaborator
time that we would one day work
on the TICCIT project, felt that
together in the same institution. In
students would not have sufficient
the early 1970s, I had arranged an
information to make a wise selection
appointment for Vic at Brigham Young
before the fact. He suggested that
University to work with our laboratory
learners could only make a decision
there. During the interview with the
about what they needed next when
administration, he received a phone
they were involved with the learning.
call from NSF indicating that he had
We came up with the notion of a
received major funding to build a
“momentary comprehension index,”
new CBI system. He turned down
that is, what the learner understands
the appointment at BYU because
at a given moment in time.
he needed his lab at Texas to do the project. Later we shared a room at a
About this time, NSF decided to consolidate the project at a single university and BYU was chosen for
convention, and he indicated that the
the site. Bunderson and many of his
project was bigger than he anticipated.
personnel from the Texas lab moved to
I offered to help. We visited NSF
Provo, Utah, to continue the project.
together, and I returned to BYU with
In the meantime, we were struggling
a contract to work on the TICCIT 1
with how to author for the system. I
project.
was working on Component Display
1
TICCIT is an acronym for Time-Shared Interactive Computer-Controlled Instructional Television. The name was given by MITRE Corporation, who did the hardware development on the project. The original idea was to deliver instructional television into all the homes in Reston, Virginia. The nature of the project changed dramatically, but the name was never changed to more accurately reflect the true nature of the system. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
151
152
Michael Allenʼs 2008 e-Learning Annual
Theory, and we had already identified
lecture, he suggested that the teaching
primary presentation forms. Harvey
machines of the future would contain
Black suggested that only when
a teacher function that would interact
given a rule (expository generality)
with the student and a professor
could students determine whether
function that would monitor the
they needed an example. Only after
teacher’s strategy, and when a given
having studied several examples could
strategy did not seem to be working
students determine whether they
would provide advice as to what to try
were ready for practice. So we put the
next. I remembered this lecture and
primary presentation forms as button
answered this concern by suggesting
commands on a special keyboard.
that we would have an advice key that
Thus, given an objective for a segment
would access an advice function that
of instruction, students could select
would help the student decide which
rule, example, or practice by the
learner control key to select next.
touch of a button. We also added an easy and hard button, which allowed It allowed learner control, not just of content, but of the strategy to be used.
After the meeting, our team strongly suggested that I had gotten
the student to
carried away, that we had no idea
find an easier
how to build an advice system. After
or harder rule,
several members of our team had
example, or
struggled with this problem, the
practice item.
task to design the system fell on my
By means of these buttons, TICCIT
shoulders. Expert systems were not yet
was unique in that it allowed learner
widely available, so I decided to build
control, not just of content, but of the
a decision tree that would provide
strategy to be used.
the required advice to the student.
During a site visit from NSF,
To make this work, we determined
someone raised the concern that
an ideal strategy, “the all-American
students may not know which button
strategy.” This strategy represented
to select next. The help key told
what we thought would be the best
them what was available but did not
use of the learner control keys by
give them any advice about which
a student. We then compared the
presentation form would be most
student’s path through the learner
helpful. I remembered a lecture on
control keys with our ideal strategy.
the future of teaching machines
The adviser then gave “local” advice;
(circa 1963) by my major professor,
that is, it recommended the key that
Lawrence Stolurow (1961). In this
would be the best for the student
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
to try at the time the advice was
an expert system advisor had not yet
requested. It was a very sophisticated
been built.
early version of an overlay expert system. Our team developed a complete
The TICCIT system pioneered a number of developments that are now common on all computer systems.
math zero (basic algebra) curriculum
Although multimedia was still in the
and an English writing program.
future, the system was one of the first
The TICCIT system and curriculum
to have colored text and graphics
were tested at Maricopa Community
(they were pretty primitive by today’s
College in Phoenix and at BYU, and it
standards). The system used an early
continued to run at both locations into
version of Windows, also pretty
the 1990s. At Maricopa Community
limited by today’s standards. The
College, TICCIT was used as mainline
system constructed displays for the
instruction. An evaluation of the
student “on-the-fly” from resource
system showed that English students
files that were combined with text
scored higher on tests of writing
templates to create either presentation
skills and essay tests than did the
or practice displays from the same
comparison group in regular lecture
content files. The design of the
classes. In the math classes students
TICCIT system is described in Merrill,
scored higher on their posttests than
Schneider, and Fletcher (1980).
did the comparison groups in regular classes (Alderman, 1979). The English
Learner Control As far as we
program has been transported to a
were able to determine, no one had
more current programming language
previously provided for learner
and is still available online (http://
control of instructional strategy. The
webclips.byu.edu). I suspect that this
TICCIT system allowed for both
program holds the record for the
content control, selecting the next
longest-running CBI program.
segment of instruction from a menu,
There was an attempt to create a
and strategy control, provided by the
commercial version of the TICCIT
rule, example, practice, easy, hard,
system. It was used for a time by the
and advice learner-control keys. We
military. The advisor system was not
were anxious to learn more about the
included in the commercial version.
effectiveness of this version of learner
To our knowledge, another system that
control. With the TICCIT system, we
allows learner control of strategy with
had a very good laboratory instrument
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
153
154
Michael Allenʼs 2008 e-Learning Annual
for conducting research. Our students
out above all the rest. The syntax
conducted dozens of studies. The
revealed that it was obviously written
results showed (Merrill, 1975, 1980,
by an international student, but the
1984):
content was very insightful. I asked
1. Students can decide how many
Zhongmin Li 2 to stop and see me
examples and practice items they
after class. Thus began a very fruitful
need for mastery;
collaboration.
2. Allowing students to determine
During this period of time, one
the sequence of rule, example, and
of my consulting opportunities
practice forms does not improve
was to review some computer-
performance over those who have
based instruction developed by a
a fixed rule, example, practice
major Air Force training company.
sequence;
The instruction was early CBT
3. Students who have control over
and involved a kluge of equipment
generality help performed more
including two monitors and an
efficiently and effectively than
analog audio system. The instruction
other groups; and
consisted of wall-to-wall text on one
4. Students who have control over
monitor and a very nice graphic on the
example and practice help perform
other monitor. The audio read the text
less efficiently and less effectively
to the student. I suggested that this
than other groups.
was not a very effective instructional strategy, and Mayer (2001) has since
Automating Instructional Design
During the
demonstrated the ineffectiveness
1980s, I found
of this instructional approach. The
myself at the
company agreed to let us try to build
University
some instructional shells that would
of Southern
enable their “designers-by-assignment”
California. I taught a seminar on
to build more effective instruction for
authoring systems. We explored many
different kinds of learning outcomes. I
of the very early authoring systems.
approached Zhongmin about building
One of the assignments of the class
such a shell for naming parts. He
was to design an authoring system. As
agreed. A week later he demonstrated
I read the student projects toward the
for me not only a shell into which
end of the semester, one paper stood
any content for naming the parts of
2
Dr. Li eventually immigrated to the United States, changed his name to James Z. Li, founded LeadingWay Corporation, and has built a very successful business in knowledge management. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
something could be imported, but he
campus and subsequently provided
also demonstrated an authoring system
a very large contract to build ID
that could be used by designers-by-
Expert, a commercial version of our
assignment for importing the content
instructional design expert system
into the shell. Subsequently, we also
(Li & Merrill, 1991; Merrill, 1987b;
designed and programmed shells
Merrill & Group, 1998; Merrill & Li,
for teaching concepts and teaching
1989, 1990). Version 1 of the system
procedures (Li & Merrill, 1990).
was completed when the company
In a contract with the Army
suffered serious financial problems
Research Institute, we proposed to
and discontinued its business. We also
build an instructional design expert
lost another very large contract at this
system. This system would ask a
same time and the work on ID Expert
series of questions of the author and
was discontinued.
then prescribe the most effective
ID Expert was a very ambitious
instructional strategy to use (Merrill
project. The goal was to reduce the
& Li, 1989). It then occurred to us to
labor involved in authoring computer-
combine our authoring shells with
based instruction by an order of
the prescriptive system so that the
magnitude. The system consisted of
system would not only recommend an
several subsystems working together:
appropriate strategy but would also
(1) a set of reusable instructional
select an appropriate authoring shell
strategy algorithms (transaction shells)
and prompt the user to import their
for different kinds of instructional
content into this shell for delivery to
outcomes based on instructional
their students.
principles (CDT); (2) a decoupled knowledge base that allowed the
ID Expert At this time both
designer or system to attach a given
Zhongmin and I moved to Utah
knowledge object to any of the
State University and, with funding
instructional strategy algorithms,
from various government sources
thus allowing for reuse of knowledge
and a major contract with IBM,
objects within the system; and (3) a
founded the ID2 Research Group.
set of instructional parameters that
We were joined by Mark Jones and
allowed the instructional strategies
a number of graduate students and
to be modified by merely selecting a
continued our work on designing an
different parameter value (Merrill,
instructional design expert system.
Jones, & Li, 1992; Merrill, Li, & Jones,
A group from Germany visited our
1992a).
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
155
156
Michael Allenʼs 2008 e-Learning Annual
Initial authoring consisted of
could describe content that could be
identifying the content to be taught
manipulated in a computer system to
and entering it into the knowledge
automatically create instruction from
base. The system contained default
only the content. It was also necessary
strategy parameter values that
to develop a more detailed description
would then allow it to automatically
of instructional strategies on which
format the instructional strategies
our instructional design shells could
for interaction with the student. A
be based.
second level of authoring allowed the
The what-to-teach side of the
author to “open the hood” and adjust
equation was extended to describe
strategy parameter values to modify
knowledge objects (Jones, Li, &
the instructional strategies. The
Merrill, 1990; Merrill, 1998, 2001b,
system allowed for multiple strategies
2001c, 2002b). We identified three
to be assigned to a given lesson or
components of knowledge: entities,
different strategies to be assigned
activities, and processes. We also
to different students. The ultimate
identified properties and defined
goal was to include a “professor”
each of these components in terms
function that would monitor student
of properties. An entity is defined by
performance and automatically adjust
a set of properties each of which has
strategy parameters as necessary to
a set of legal values. Each legal value
accommodate the learning needs
can be represented by a portrayal.
of a given student. This adaptive
Changing the portrayal changes the
instruction feature was planned for
appearance of the entity. An activity
Phase 3 of the project but was never
is defined as some action on the part
completed because of the demise of
of the learner that triggers a process.
the funding company.
When executed, a process results in some consequence that is defined
Instructional Transaction
as a change in the value of one or
Theory In order to develop our tools
more properties of some entity. A
for automating instructional design,
process also has a set of conditions
we found that it was necessary to
that are defined as values on a set of
develop more precise instructional
properties. A process only executes
design theory (Merrill, Li, & Jones,
when the conditions equal the
1990a, 1990b, 1991). My contribution
appropriate property values.
to the work was to try to extend CDT to a more precise level, so that we
We also simplified the Performance/ Content Matrix to identify five kinds
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
of learning outcomes: information-
157
We also identified PEA-nets
about, parts-of, kinds-of, how-to, and
(process-entity-activity –nets) that
what-happens. Information-about and
define the dynamic relationship
parts-of correspond roughly to facts;
among these three components of a
kinds-of corresponds to concepts;
knowledge object. An activity (action
how-to corresponds to procedures;
by a learner) triggers a process. If the
and what-happens corresponds to
conditions are true (the condition
principles in the former scheme.
properties have the appropriate
We also identified an information-
values), then the process executes—
level (generality) and a portrayal-
meaning that it changes the values of
level (instance) for each of the five
the consequence properties.
categories of learned outcomes. In this knowledge representation,
The how-to-teach side of the equation was extended to more
a definition of a concept (kind-of )
precisely identify the relationship
is a list of defining properties and
between our knowledge objects and
the value each must assume for a
the demonstration and application of
portrayal to be a member of the class.
this knowledge (Li & Merrill, 1990;
A step in a procedure (how to) is an
Merrill & ID2 Research Team, 1993;
action performed by the student. A
Merrill, Jones, & Li, 1992; Merrill,
condition is a value on a property. A
Li, & Jones, 1991, 1992a, 1992b). An
consequence is a change in the value
instructional strategy is defined as
of a property.
a primary presentation form, which
INFORMATION PRESENT (TELL)
RECALL (ASK)
DEMONSTRATE (SHOW)
APPLY (DO)
Tell the definition.
Recall the definition.
Show several specific examples.
Classify new examples.
Tell the steps and their sequence.
Recall the steps and their sequence.
Show the procedure in several different situations.
Carry out the procedure in new situations.
Show the process in several different situations.
Predict a consequence or find faulted conditions in new situations
Kinds-of
How-to
What-happens
PORTRAYAL
Tell the conditions and consequence involved in the process.
Recall the conditions and consequence involved in the process.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Table 3. Consistent Information and Portrayal for Categories of Learning
158
Michael Allenʼs 2008 e-Learning Annual
we simplified to TELL, ASK, SHOW,
Instructional Simulator Leston
DO operating on a component of a
was the primary developer of the
knowledge object. Table 3 (previous
Instructional Simulator, a very
page) indicates the consistent
easy-to-use simulation authoring
strategies for kinds-of, how-to, and
and delivery system that combined
what-happens.
instructional strategies with simulation (Merrill, 1999). This tool
Electronic Textbook Mark Lacy
was available for a short time as a
and Leston Drake joined our ID2
commercial product under the name
Research Group shortly before the
IDVisualizer.
demise of the ID Expert project. Using
The Instructional Simulator
largely internal funds, we were able to
implemented our work on knowledge
continue our work to develop systems
objects. It is based on PEA-nets,
for automating instructional design.
as described earlier in this article.
Mark was the primary developer of
Authoring the simulator consists of
the Electronic Textbook, a very easy-
first defining a set of dynamic entities
to-use authoring shell with built-in
that may be parts of a larger system.
instructional design for teaching
For each entity, a set of properties is
naming, concepts, and procedures
identified along with the legal values
(Merrill & Thompson, 1999). This
that this property can assume. For
tool was available for a short time as a
each legal value, a portrayal (usually a
commercial product under the name
graphic representation) is developed
IDXelerator.
and stored in a knowledge-base.
The Electronic Textbook consisted
Attached to each entity is a set of
of three built-in instructional
activities (actions defined by clicking
algorithms, each implementing the
on some part of the portrayal of
rules of Component Display Theory:
the entity), one or more processes
an information-about (facts) lesson,
triggered by each activity, and the
kinds-of (concept) lesson, and a how-to
consequence of the process defined as
(procedure) lesson. Authoring consists
a change in the value of one or more
of putting the appropriate content
properties. The underlying system
into a knowledge base for each kind
is an expert system that executes
of lesson. The system then generated
the PEA-net. Thus, the portrayals
presentation and practice for the
on the display can be manipulated
student appropriate for each type of
by the student. (See Figure 1, which
lesson.
illustrates the interface.) Students
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
159
Figure 1. Interface for Instructional Simulator
interact by clicking on various parts
went on to form their own company,
of the pump to connect or disconnect
LetterPress Software. The ID 2 research
them. The authoring system prompts
group was unable to find additional
the input of the activity-process-
funding to continue this work, and
consequence rules in a very easy-to-
the company that bought the rights
use format. Once the portrayals for
to this produce were unsuccessful in
the different values of the properties
commercializing this work.
of the entities have been developed, the actual simulation can be developed in a matter of a few minutes to a few
First Principles of Instruction
When I developed CDT,
hours, depending on the complexity
I was not trying
of the system. Authoring simulations
to develop an
in the Instructional Simulator was
alternative
considerably more efficient than
method for designing instruction.
programming. The instructional
The primary purpose was to develop
simulator was our most sophisticated
a more precise terminology for
implementation of automated
describing instructional outcomes and
instructional design. Mark and Leston
instructional strategies. The second
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
160
Michael Allenʼs 2008 e-Learning Annual
purpose was to identify relationships
to elaborate, clarify, and explain
between the outcomes of instruction
these principles and to demonstrate
and the strategies thought appropriate
their presence in other instructional
to produce these outcomes. My
design theories and models (Merrill,
intent was to identify underlying
2001a, 2002a, 2006a, 2006b, 2007, In
principles that were common to all
Press-a, In Press-b). I also proposed
models of instructional design. In
a content-first alternative to the
spite of my intentions, CDT was
classic ADDIE instructional design
often characterized as an alternative
model called A Pebble-in-the-Pond
approach to instructional design
approach to instructional development
rather than a more precise way to
(Mendenhall, Buhanan, Suhaka,
characterize existing approaches to
Mills, Gibson, & Merrill, 2006;
instructional design.
Merrill, 2002c, in press-c). Recent
When the second green book was
work has attempted to elaborate the
published (Reigeluth, 1999b), Charlie
content analysis required for task-
indicated in the preface that there
centered instruction and to define an
were many different instructional
effective task-centered instructional
design theories and models, and that
strategy for teaching complex content
designers should learn many of these
(Mendenhall, Buhanan, Suhaka, Mills,
different approaches and use the
Gibson, & Merrill, 2006; Merrill, in
approach that was most appropriate
preparation).
for a given situation. I thought that
TRADOC (the army training and
most of these different approaches
doctrine command) has recently
were all based on the same underlying
adopted a new training methodology
principles and that they differed
called GEL (Guided Experiential
mostly in implementation details.
Learning). GEL is based on First
Charlie challenged me to demonstrate
Principles of Instruction. A task force
my assumption. I set out to determine
of the audiovisual department of the
the fundamental principles that were
Church of Jesus Christ of Latter-
common to many of these different
Day Saints has recently adopted
approaches.
First Principles of Instruction as
The result of this effort was a
the instructional methodology for
set of principles that I called “First
online training materials offered by
Principles of Instruction.” These
the church. The author is humbled
principles are summarized in Table
by these developments and anxiously
4. In the past few years I have tried
awaits data from these two large
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
Task-centered approach
Application principle
ÿ Learning is promoted when learners are
ÿ Learning is promoted when learners engage in
engaged in a task-centered approach, which
application of their newly acquired knowledge
includes demonstration and application of
or skill that is consistent with the type of
component skills.
content being taught.
ÿ A task-centered approach is enhanced when learners undertake a progression of whole
ÿ Application is effective only when learners receive intrinsic or corrective feedback. ÿ Application is enhanced when learners are
tasks.
coached and when this coaching is gradually Activation principle
withdrawn for each subsequent task.
ÿ Learning is promoted when learners activate relevant cognitive structures by being directed
Integration principle
to recall, describe, or demonstrate relevant
ÿ Learning is promoted when learners integrate
prior knowledge or experience. ÿ Activation is enhanced when learners recall or acquire a structure for organizing the new
their new knowledge into their everyday life by being directed to reflect on, discuss, or defend their new knowledge or skill. ÿ Integration is enhanced when learners create,
knowledge.
invent, or extrapolate personal ways to use their Demonstration principle
new knowledge or skill in situations in their
ÿ Learning is promoted when learners observe a
world.
demonstration of the skills to be learned that is consistent with the type of content being
ÿ Integration is enhanced when learners publicly demonstrate their new knowledge or skill.
taught. ÿ Demonstrations are enhanced when learners receive guidance that relates instances to generalities. ÿ Demonstrations are enhanced when learners observe media that is relevant to the content.
Table 4. First Principles of Instruction
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
161
162
Michael Allenʼs 2008 e-Learning Annual
organizations as to the effectiveness
have just scratched the surface of how
of their training based on these
to design what to teach and how to
principles.
teach. What is my greatest concern? I’m
Summary
I hope that
concerned that there is not more
this nostalgic
effort being devoted to this question.
journey through
The current zeitgeist seems to
a few highlights
emphasize communities of learners,
of my career
repositories of content, and electronic
has provided a bit of insight into
communication. There seems to be
how ideas develop and evolve. What
an assumption that information is
have I learned about how to make
sufficient and that e 3 direct instruction
instruction more effective, efficient,
is no longer necessary. The Internet
and engaging (e 3)? Considerably more
is swollen with information, and
than I knew at the beginning of my
amidst this flood, there are only
career. I do believe that we know a bit
isolated islands of e 3 instruction.
more about how to design instruction
I am gratified by the adoption of
that works. I have been gratified by
our work by the Army and the LDS
the many designers who have found
church. Nevertheless, my hope is that
our work helpful;. however, far too
our work may continue to provide a
much instruction is still not effective,
catalyst for further efforts to find ways
not efficient, and not engaging. Do we
to make instruction more effective,
have more to learn? I believe that we
more efficient, and more engaging.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction REFERENCES Alderman, D.L. (1979). Evaluation of the TICCIT computer-assisted instructional system in the community college. ACM SIGCUE Outlook, 13(3), 5-17. Bruner, J.S., Goodnow, J.J., & Austin, G.A. (1956). A study of thinking. Hoboken, NJ: John Wiley & Sons. Ellis, J.A., Wulfeck, W.H., Merrill, M.D., Richards, R.E., Schmidt, R.V., & Wood, N.D. (1978). Interim training manual for the Instructional Quality Inventory SR 78-5. San Diego, CA: NPRDC. Gagne, R.M. (1965). The conditions of learning. New York: Holt, Rinehart & Winston. Jones, M.K., Li, Z., & Merrill, M.D. (1990). Domain knowledge representation for instructional analysis. Educational Technology, 30(10), 23-30. Li, Z., & Merrill, M.D. (1990). Transaction shells: a new approach to courseware authoring. Journal of Research on Computing in Education, 23(1), 72-86. Li, Z., & Merrill, M.D. (1991). ID Expert 2.0: Design theory and process. Educational Technology Research and Development, 39(2), 53-69. Mayer, R.E. (2001). Multimedia learning. Cambridge, UK: Cambridge University Press. Mendenhall, A., Buhanan, C.W., Suhaka, M., Mills, G., Gibson, G.V., & Merrill, M.D. (2006). A task-centered approach to entrepreneurship. TechTrends, 50(4), 84-89. Merrill, M.D. (1971). Necessary psychological conditions for defining instructional outcomes. Educational Technology, 11(8), 34-39. Merrill, M.D. (1975). Learner control: beyond aptitude-treatment interactions. A V Communications Review, 23, 217-226. Merrill, M.D. (1980). Learner control in computer-based learning. Computers and Education, 4, 77-95. Merrill, M.D. (1983). Component display theory. In C.M. Reigeluth (Ed.), Instructional design theories and models: An overview of their current status. Mahwah, NJ: Lawrence Erlbaum Associates. Merrill, M.D. (1984). What is learner control? In R. Bass & C.R. Dills (Eds.), Instructional development: The state of the art II. Dubuque, IA: Kendall/Hunt. Merrill, M.D. (1987a). A lesson based on component display theory. In C.M. Reigeluth (Ed.), Instructional design theories in action (pp. 201-244). Mahwah, NJ: Lawrence Erlbaum Associates.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
163
164
Michael Allenʼs 2008 e-Learning Annual
Merrill, M.D. (1987b). An expert system for instructional design. IEEE Expert, 2(2), 25-40. Merrill, M.D. (1988). Applying component display theory to the design of courseware. In D.H. Jonassen (Ed.), Instructional designs for microcomputer courseware (pp. 61-95). Mahwah, NJ: Lawrence Erlbaum Associates. Merrill, M.D. (1994). Instructional design theory. Englewood Cliffs, NJ: Educational Technology Publications. Merrill, M.D., & ID2 Research Group. (1998). ID expert: A second generation instructional development system. Instructional Science, 26, 243-262. Merrill, M.D. (1998, March/April). Knowledge objects. CBT Solutions, pp. 1-11. Merrill, M.D. (1999). Instructional transaction theory (ITT): Instructional design based on knowledge objects. In C.M. Reigeluth (Ed.), Instructional design theory and models: A new paradigm of instructional theory vol. II. Mahwah, NJ: Lawrence Erlbaum Associates. Merrill, M.D. (2001a). First principles of instruction. Journal of Structural Learning and Intelligent Systems, 14(4), 459-466. Merrill, M.D. (2001b). A knowledge object and mental model approach to a physics lesson. Educational Technology, 41(1), 36-47. Merrill, M.D. (2001c). Toward a theoretical tool for instructional design. Instructional Science, 29(4-5), 291-310. Merrill, M.D. (2002a). First principles of instruction. Educational Technology Research and Development, 50(3), 43-59. Merrill, M.D. (2002b). Knowledge objects and mental models. In D.A. Wiley (Ed.), The instructional use of learning objects (pp. 261-280). Washington, DC: Agency for Instructional Technology & Association for Educational Communications and Technology. Merrill, M.D. (2002c). A pebble-in-the-pond model for instructional design. Performance Improvement, 41(7), 39-44. Merrill, M.D. (2006a). Hypothesized performance on complex tasks as a function of scaled instructional strategies. In J. Elen & R.E. Clark (Eds.), Handling complexity in learning environments: Theory and research (pp. 265281). Amsterdam: Elsevier. Merrill, M.D. (2006b). Levels of instructional strategy. Educational Technology, 46(4), 5-10.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
Merrill, M.D. (2007). First principles of instruction: A synthesis. In R.A. Reiser & J.V. Dempsey (Eds.), Trends and issues in instructional design and technology (2nd ed., pp. 62-71). Upper Saddle River, NJ: Merrill Prentice Hall. Merrill, M.D. (In press-a). Converting e 3 learning to e 3 power learning: An alternative instructional design method for e-learning. In S. Carliner & P. Shank (Eds.), The e-learning handbook: Past promises, present challenges. San Francisco, CA: Pfeiffer. Merrill, M.D. (In press-b). First principles of instruction. In C.M. Reigeluth & A. Carr-Chellman (Eds.), Instructional design theories and models III. Mahwah, NJ: Lawrence Erlbaum Associates. Merrill, M.D. (in press-c). A task-centered instructional strategy. Journal of Research on Technology in Education. Merrill, M.D., & Boutwell, R.C. (1973). Instructional development: Methodology and research. In F. Kerlinger (Ed.), Review of research in education, vol. 1. Ithaca, NY: Peacock Publishers. Merrill, M.D., & Goodman, I.R. (1972). Selecting instructional strategies and media: A place to begin. Syracuse, NY: National Special Media Institutes. Merrill, M.D., & ID2 Research Group. (1998). ID Expert: a second generation instructional development system. Instructional Science, 26, 243-262. Merrill, M. D., & ID2 Research Team. (1993). Instructional transaction theory: knowledge relationships among processes, entities and activities. Educational Technology, 33(4), 5-16. Merrill, M.D., Jones, M.K., & Li, Z. (1992). Instructional transaction theory: classes of transactions. Educational Technology, 32(6), 12-26. Merrill, M.D., & Li, Z. (1989). An instructional design expert system. Journal of Computer-Based Instruction, 16(3), 95-101. Merrill, M.D., & Li, Z. (1990). An instructional design expert system. In S. Dijkstra, B.H.A.M. Van Hout Wolters & P.C.V. Sijde (Eds.), Research in instruction: Design & effects. Englewood Cliffs, NJ: Educational Technology Publications. Merrill, M.D., Li, Z., & Jones, M.K. (1990a). Limitations of first generation instructional design (ID1). Educational Technology, 30(1), 7-11. Merrill, M.D., Li, Z., & Jones, M.K. (1990b). Second generation instructional design (ID2). Educational Technology, 30(2), 7-14. Merrill, M.D., Li, Z., & Jones, M.K. (1991). Instructional transaction theory: An introduction. Educational Technology, 31(6), 7-12. Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
165
166
Michael Allenʼs 2008 e-Learning Annual
Merrill, M.D., Li, Z., & Jones, M.K. (1992a). Instructional transaction shells: Responsibilities, methods, and parameters. Educational Technology, 32(6), 12-26. Merrill, M.D., Li, Z., & Jones, M.K. (1992b). Instructional transaction theory: an introduction. In S. Dijkstra, Z. Li, & M.D. Merrill (Eds.), Instructional models in computer-based learning environments. New York: Springer Verlag. Merrill, M.D., Reigeluth, C.M., & Faust, G.W. (1979). The instructional quality profile: A curriculum evaluation and design tool. In H. O’Neil (Ed.), Procedures for instructional systems development. New York: Academic Press. Merrill, M.D., Richards, R.E., Schmidt, R.V., & Wood, N.D. (1977). The instructional strategy diagnostic Profile training manual. Orem, UT: Courseware Inc. Merrill, M.D., Schneider, E.W., & Fletcher, K.A. (1980). TICCIT. Englewood Cliffs, NJ: Educational Technology Publications. Merrill, M.D., & Tennyson, R.D. (1977a). Concept classification and classification errors as a function of relationship between examples and non examples. Improving Human Performance. Merrill, M.D., & Tennyson, R.D. (1977b). Teaching concepts: An instructional design guide. Englewood Cliffs, NJ: Educational Technology Publications. Merrill, M.D., Tennyson, R.D., & Posey, L.O. (1992). Teaching concepts: An instructional design guide (2nd ed.). Englewood Cliffs, NJ: Educational Technology Publications. Merrill, M.D., & Thompson, B. (1999). IDXelerator: Learning-centered instructional design. In J.v.d. Akker, R.M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design methodology and development research in education and training (pp. 265-277). Dordrecht/Boston/London: Kluwer Academic Publishers. Reigeluth, C.M. (1983). The elaboration theory of instruction. In C.M. Reigeluth (Ed.), Instructional design theories and models: An overview of their current status (pp. 335-381). Mahwah, NJ: Lawrence Erlbaum Associates. Reigeluth, C.M. (1999a). The elaboration theory: guidance for scope and sequence decisions. In C.M. Reigeluth (Ed.), Instructional design theories and models: A new paradigm of instructional theory, volume II (pp. 425-453). Mahwah, NJ: Lawrence Erlbaum Associates.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Reflections on a Four-Decade Search for Effective, Efficient, and Engaging Instruction
Reigeluth, C.M. (Ed.). (1999b). Instructional-design theories and models: A new paradigm of instructional theory (Vol. II). Mahwah, NJ: Lawrence Erlbaum Associates. Reigeluth, C.M., Merrill, M.D., Wilson, B., & Spiller, R.T. (1980). The elaboration theory of instruction: a model for structuring instruction. Instructional Science, 3, 30-31. Stolurow, L.M. (1961). Teaching by machine. Washington DC: U.S. Department of Health, Education, and Welfare. Tennyson, R.D., & Cocchiarella, M.J. (1986). An empirically based instructional design theory for teaching concepts. Review of Educational Research, 56(1), 40-71. Tennyson, R.D., & Park, O.-C. (1980). The teaching of concepts: A review of instructional design research literature. Review of Educational Research, 50(1), 55-70. Tennyson, R.D., Wooley, R.R., & Merrill, M.D. (1972). Exemplar and non exemplar variables which produce correct classification errors. Journal of Educational Psychology, 63, 144-152.
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
167
Section Three
What Does It All Mean? Pitfalls and Overhauls
Tom Boyle Professor Tom Boyle is director of the Learning Technology Research Institute (LTRI) at London Metropolitan University. He has a long history of developing and evaluating innovative multimedia learning technology. Tom led a major project in the development, use, and evaluation of learning objects that won an EASA (European Academic Software Award) in 2004. Currently, Boyle’s work concerns the Centre for Excellence in Teaching and Learning (CETL) in Reusable Learning Objects, which he directs. This multi-million-dollar project is funded by the Higher Education Funding Council for England. It is a collaboration among three universities—London Metropolitan University, the University of Cambridge, and the University of Nottingham—to develop and evaluate high-quality learning resources across a range of subject areas.
Dreams and Responsibilities
Tom Boyle
Learning and the New Technologies This article reviews and comments on major developments in technologyenhanced learning over the past two decades, as the author reflects on how his own work has fit into the landscape of these momentous changes. The central theme of the article is the need for a profound and robust pedagogy to enable us to exploit effectively the opportunities offered by new technologies. “In dreams begins responsibility.”
session I would print a ticker-tape roll
—W.B. Yeats
of paper with punched holes that held my data (there was a neat little printer
The leitmotiv of this article is to
attached to the terminal). You could
outline the dreams of what the
put an elastic band around the roll,
new technologies can deliver,
drop in your pocket, and walk away
counterbalanced by our responsibilities
feeling that your data were secure. This
to learners. A major organizing theme
was mobile computing in the mid-
is the concept of “technological
1970s. We are only talking about thirty
amplifiers” of human ability described
years ago. Never has a technological
by Jerome Bruner (in turn influenced
storm come with so much force, over
by the ideas of Vygotsky).. This idea is
so short a time, and with such impact.
used to establish the conceptual basis
The research group I lead, among
on which the relationship between
other things, now designs multimedia
learning and new technologies is
learning objects for mobile phones.
explored. What manifests is the need
Mobile computing for education has
for pedagogical design principles that
moved a long way in thirty years.
will enable us to exploit the possibilities
Mobile computing is only one of
of the new technologies enabling us to
a huge range of developments: the
creatively utilize the new technology,
micro-computer revolution of the
rather than engaging in surface
1980s, the multimedia revolution of
adaptation to the latest innovation or
the early 1990s, the World Wide Web
fashion.
from the mid-1990s, mobile ubiquitous
My first use of computers was in
computing, and so on. How do we
the mid-1970s doing statistical tests
develop a pedagogy that is other than
for my Ph.D. When I had finished a
in a state of constant reaction to
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
172
Michael Allenʼs 2008 e-Learning Annual
these new developments? How do we
are more profound and have wider
develop a pedagogy that can shape the
implications than others. These
technology to enhance learning in a
deeper questions are often difficult
profound and principled way?
to formulate—let alone resolve. So our discourse in e-learning is often
Questions and Perspectives
Through the
dominated by short-term tactical
example of
considerations, rather than by the
the modern
more profound strategic impact of the
book, Alan
new technology.
Kay points to two broad classes of questions we
Technology and Learning The
can ask about the impact of new
most basic question is not how
technologies: How can they make
technology will impact learning; but
established teaching and learning
rather how the new information and
approaches more effective? and What
communication (ICT) technologies will
is truly new? Kay argues that the
impact it. We have had technology-
modern book did not arrive with the
enhanced learning for several
advent of the printing press, as many
hundred, if not thousands, of years.
assume. In an entertaining story,
The greatest technological invention
Kay recounts how Aldus Manutius
to impact on learning and education
helped to invent the modern book a
has been that of literacy. Literacy
considerable period after the invention
created a new form of permanent
of the printing press by, among other
memory. This transformed how we
things, measuring saddlebags to work
solve problems and how we learn. This
out the optimal size. In this amusing
transition was not without problems.
anecdote, Kay makes a salient point.
Literacy as a technology may have
The first use of new technologies is to
enhanced certain processes, but it
automate the old way of doing things.
seemed to undermine certain other
But the really important breakthrough
abilities. In Plato’s Phaedrus, Socrates
is to find the new ways that are made
produces several strong arguments
possible by the technology.
in defending the oral tradition. He
In tackling any major endeavor, the
criticizes the indirect and removed
most important thing is to ask the
nature of written texts. He contrasts
right questions. In e-learning, different
the “inert” words on a page with the
people ask different questions for
living words of the speaker, which can
different purposes. Some questions
be interrogated and defended. Socrates
Michael Allenʼs 2008 e-Learning Annual Copyright © 2008 John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com
Dreams and Responsibilities
173
further argues that “this discovery
education. The fundamental challenge
. . . will create forgetfulness in the
is to creatively rethink pedagogy to
learners’ souls because they will not
take advantage of the opportunities
use their memories; they will trust to
offered by the new technology.
the external written characters and
One of the most profound insights
not remember of themselves” (Plato,
on the relationship of technology and
1999).
pedagogy is given by Jerome Bruner
In quite an important way, human
(1964) in his paper, The Course of
memory (both individual and social)
Cognitive Growth. Bruner argues that
changed with the advent of literacy.
human evolution has been shaped in
It enabled us to do things that we
a powerful way by the cultural tools
could not do before; yet, in that
we have developed. A long childhood
transition, feats of memory valued in
enables powerful cultural influences
the oral tradition become devalued
on cognitive
and withered. Technologies do not
development.
Technologies do not just
just change the means of learning;
Bruner discusses,
change the means of learning;
they also change the goals. The status
in particular, a set
they also change the goals.
of literacy is now unchallenged,
of technological
but it is refreshing to revisit this
amplifiers of human abilities.
trenchant critique. In the transition
Although our physical abilities, both in
to new technologies, we should think
terms of strength and speed, are quite
freshly about both the established
limited, we have invented such cultural
tec
