1. education
2. distance learning

LGUIDE
Executive
Summary
TM
The E-Learning Experts
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
E-Learning Course Publishers:
A Comparative Analysis
and Industry Directory
“After struggling for 6 months doing internal research around some specific online training needs, I found your report to be extremely helpful. It is formatted in
a way that makes the content very accessible and very concise in comparing
companies. I only wish I had received it sooner!”
Pat Weger
Vice President of Learning & Development, AT&T Broadband
“Unbelievable effort. Your report is well written and offers tremendous depth
and strong analysis. Not only does this report provide the reader strengths and
weakness of each provider by segment, but does it in a simple format that can be
easily referenced. In addition the inclusion of the benchmarks on which to
evaluate the content choices is particularly relevant to a consumer’s needs in
determining the correct content for their employees.”
Peter Martin
E-Learning Research Analyst, Jefferies and Co.
“Lguide is performing a valuable service to the e-learning industry by providing
an objective and articulate call for higher quality content. By offering specific
criticism both in terms of course feature and subject areas, the report provides
developers with effective benchmarks and buyers with a comprehensive roadmap,
which together ought to result in the continued advance of e-learning.”
Trace Urdan
E-Learning Research Analyst for WR Hambrecht + Co
LGUIDE
“Lguide’s unbiased efforts to not only publish a comprehensive directory of the
sector’s publishers, but also evaluate the vast selection of courses is a much
needed tool that should be of tremendous assistance in developing and
implementing enterprise wide e-Learning initiatives more efficiently and cost
effectively.”
Steve Lidberg, CFA
Sr. Research Analyst, Pacific Crest Securities
TM
The E-Learning Experts
601 South Pine Street #201
Tacoma,Washington 98405
Tel: (253) 383-3779
© 2001 LGUIDE
Page 1
LGUIDE
TM
The E-Learning Experts
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
Contents
Introduction:
Why This Report? ......................................................................... 4
Background and Assumptions ....................................................... 6
Methodology Overview .................................................................. 7
Summary of Findings ................................................................... 8
Category Analysis: Business Skills ..................................................... 11
Category Analysis: Desktop Applications ........................................... 26
Category Analysis:
Professional Information Technology............................................ 42
Publisher Evaluations ...................................................................... 55
Full Evaluation: AchieveGlobal ......................................................... 60
Full Evaluation: ActiveEducation ...................................................... 65
Full Evaluation: Catapult, an IBM Company ...................................... 71
Full Evaluation:
Corpedia Training Technologies .................................................... 75
Full Evaluation: DigitalThink ............................................................ 80
Full Evaluation:
Educational Multimedia Corporation ........................................... 88
Full Evaluation: Element K .............................................................. 93
Full Evaluation: ............................................................................. 100
Harvard Business School Publishing ............................................... 100
Full Evaluation:
Intellinex [formerly Teach.com] .................................................. 107
Full Evaluation: KnowHowZone ..................................................... 114
Full Evaluation: Learn2.com .......................................................... 118
Full Evaluation: LearningAction ..................................................... 125
Full Evaluation: MindLeaders......................................................... 129
Full Evaluation: Microsoft Press ..................................................... 138
Full Evaluation: NETg ................................................................... 143
Full Evaluation: PrimeLearning.com ............................................... 152
Full Evaluation: QuicKnowledge.com .............................................. 157
Full Evaluation: SkillSoft ............................................................... 161
Full Evaluation: SmartForce .......................................................... 167
Full Evaluation: Syntrio ................................................................. 175
Full Evaluation: Vital Learning ....................................................... 179
Full Evaluation: Wave Technologies ................................................. 184
Full Evaluation: youachieve.com ..................................................... 189
© 2001 LGUIDE
Page 2
LGUIDE
TM
The E-Learning Experts
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
Publisher Brief Evaluations ............................................................ 193
Brief Evaluation: Bit Learning ........................................................ 196
Brief Evaluation: Columbia Interactive ............................................ 199
Brief Evaluation: Development
Dimensions International (DDI) ................................................ 203
Brief Evaluation: ExperiencePoint .................................................. 205
Brief Evaluation: InfoSource/How to Master ................................... 207
Brief Evaluation: KCI .................................................................... 209
Brief Evaluation: Kenexa Improve It! .............................................. 211
Brief Evaluation: Learning Insights ................................................. 213
Brief Evaluation:
McGraw-Hill Lifetime Learning ................................................ 215
Brief Evaluation:
Pearson Technology Group Interactive........................................ 217
Brief Publisher Overviews ............................................................... 219
Introduction to Brief Overviews....................................................... 220
Brief Overview:
Caliber Learning Network, Inc. ................................................. 221
Brief Overview: Cognitive Arts ........................................................ 223
Brief Overview: Franklin Covey, Inc. ................................................ 226
Brief Overview: Instruction Set ....................................................... 227
Brief Overview: Learnitcorp ............................................................ 229
Brief Overview: Ninth House Network ............................................. 230
Brief Overview: Quisic .................................................................... 231
Publisher Directory ........................................................................ 233
Appendix ...................................................................................... 271
Research Report Methodology ........................................................ 272
Course review process .................................................................... 274
Lguide Review Criteria ................................................................... 276
Authors ......................................................................................... 279
© 2001 LGUIDE
Page 3
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
Introduction:
Why This Report?
We have written this report for two reasons. First, we want to help e-learning
customers identify and evaluate the e-learning courses that best meet their needs and
the needs of their organizations. Second, we believe in the promise of e-learning, but
feel that a great deal of work remains before the products available to consumers
live up to that promise. We want to acknowledge those publishers whose products
best capture the potential of e-learning, and provide information that will help
publishers make their products more useful to users.
For e-learning customers, the past two years have been a time of promise and
confusion. The promise comes from the industry’s rapid growth, which brings with it
an unsurpassed variety of vendors and products from which customers can choose.
The confusion comes from the same place: with hundreds of companies jockeying for
attention, many consumers feel overwhelmed by an onslaught of literature, product
demonstrations, and marketing hype.
At no time and place was this more clear than last September at the Online Learning 2000 exposition hall in Denver. Like many of the e-learning customers we serve,
we circled the hall, moving from booth to booth to speak with company representatives and collect product information. At every stop, we received fistfuls of glossy
brochures that boasted of e-learning courses with innovative technology, insightful
content, and cutting-edge design. Every vendor, it seemed, was “the industry leader,”
offering a “best of breed” solution that was “second to none.”
Each of us left that show with two large items: a duffel bag full of marketing
collateral and product information, and a first-hand appreciation for the confusion
felt by e-learning buyers. It took us weeks to sort through the former, but it has taken
us months to deal with the latter: months of taking courses, testing features, reviewing content, and reporting our findings so that they can be used by others to guide
their own purchases.
This report is the result. In it, we offer profiles of 40 leading e-learning publishers,
summary evaluations of their online courseware, and a directory of hundreds of
other publishers in the industry. For purposes of making this report consistent and
most usable for customers, we have targeted our research. This report focuses on
asynchronous, off-the-shelf, Web-based training courses, and the publishers that
produce them.
For purposes of content comparison, the report also groups these courses into three
major content areas: business skills, desktop applications training, and professional
IT skills (such as programming, networking, and IT certification). This report does
not include evaluations of custom-authored training courses, of synchronous (or
“live”) Web-based training products, or of courses in the health and safety or OSHAcompliance categories.
We chose the 40 publishers based on two criteria. The first and most important
criterion was our ability to gain access to the publishers’ catalogs for purposes of
conducting a full and thorough product review. Our goal is to provide clear and unbiased evaluative information that customers can use to guide their e-learning
purchases, and we believe that rigorous product testing and thorough evaluation of
e-learning courses is the best way of producing that information. (For more on our
review methodology and review criteria, see the Appendix.)
© 2001 LGUIDE
Page 4
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
The second criterion was that of industry significance. As our directory will show,
there are literally hundreds of organizations that currently produce asynchronous
off-the-shelf Web-based training products in some form. In order to make this report
useful for readers, we had to choose a select group of publishers from this wide
array. To do this, we prioritized those publishers that fell into one (or more) of the
following five categories:
· Major players: e-learning publishers with exceptionally large course catalogs,
established market presence, and/or large sales revenues.
· Offline to online: education or media companies with a powerful online
presence that are in the process of moving into e-learning.
· High-quality niche: smaller publishers that, by virtue of focusing their efforts
on a target category, have produced noteworthy products.
· Innovators: publishers whose products incorporate high-quality, leading-edge
instructional design.
· Market makers: publishers that combine their products with an unusual or
creative business model.
The resulting company profiles take three forms. For all 40 of our publisher evaluations, every effort was made to incorporate a comprehensive product evaluation
along with a company profile: these reports are labeled as “Full Evaluations.” In the
few instances where writing a full evaluation was not possible—either because the
product line from an “offline to online” publisher was not yet complete, or because a
publisher only offered access to a limited (and select) group of their courses—we
have noted the circumstances and compiled either a “Brief Evaluation” that offers a
summary product evaluation, or a “Brief Overview” that explains the company’s
background and market position. Our goal is to ensure that consumers have the
widest array of useful information about potential e-learning vendors, and we will
publish full evaluations of these companies’ products as they become available.
Who We Are
Lguide is an independent e-learning research and consulting company. Our mission
is to empower e-learning decisions for corporations, training managers, and business
professionals. We do this by offering in-depth, authoritative analysis of e-learning
products and services. Our team of full-time consultants and analysts have taken and
reviewed thousands of e-learning courses from publishers across the industry, giving
us a unique and detailed perspective on the strengths and weaknesses of leading elearning providers. We leverage this industry expertise in three ways: we provide elearning consulting services to corporate clients, we offer a Website that gives
subscribers access to a database of our research, and we publish research reports
and articles that focus on e-learning product evaluation. The leading print and
electronic journals for the e-learning industry publish Lguide research, including
Online Learning Magazine, Training & Development magazine, and Learning
Circuits.
© 2001 LGUIDE
Page 5
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
Background and Assumptions
As with any research project, this initiative began with our making several assumptions that guided our work. These assumptions were as follows:
Quality counts.
While there are several factors that customers must consider when choosing appropriate e-learning materials—price, learning management system (LMS) compatibility, and end-user technical platform being but a few—this report prioritizes training
effectiveness and quality of the end-user experience. In short, our evaluators try to
identify courses that: a) establish meaningful learning objectives, and b) meet those
objectives by offering users clear and concise information and opportunities for
hands-on skill practice.
Creating excellent courses is difficult...
Our desire to provide users with objective evaluations of online courses requires us
to identify the ways courses fail as well as the ways they succeed. Nevertheless, we
realize that course designers are, like the companies that employ them, subject to
limitations of time, cost, and technology. When we encounter and identify features or
problems that limit a course’s effectiveness, we do not assume that they are the result
of carelessness or ineptitude. Rather, we understand that they are most likely the
result of conscious decisions made by designers who must trade one benefit for
another. We identify these areas because we feel the information helps customers find
courseware that will meet their needs.
...but not impossible.
In light of these concerns, we have made every effort when creating our evaluation
standards to identify actual instances when publishers have successfully met those
standards. Our yardstick, therefore, is not an impossible and idealized one, but one
based on actual “best practices” of courseware publishers with products currently
available on the market. As a result, when we identify courses that lack an important or useful feature, we do so because we know that the feature can and does exist
in other courses.
Evaluation is relative.
This report is not designed to “grade” courses on an absolute scale of success or
failure. Rather, it is designed to help customers make decisions when choosing
between several competing options. As a result, there are courses that receive low
ratings on our scale not because they are “bad” courses per se—they may be
perfectly adequate for most users—but because there are other competing courses
that offer either superior content, more effective design, or a simpler and more
intuitive user experience. Will all of these courses achieve the main goal of training
end-users? In most cases, yes. Are they all equal? No. Our evaluations help users
differentiate between their options to find those products that will work best for
them.
© 2001 LGUIDE
Page 6
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
Course review process
Lguide’s review methodology is the direct result of our experience with thousands of
online courses. The methodology was developed and refined in consultation with
Lguide’s Training Advisory Board (TAB). Lguide’s TAB members are:
Deborah Bauer
Director of Development Services, Dell Learning, Dell Computer Corporation
Kim Church
Chief Information Officer, Preston Gates & Ellis LLP
Chris Lee
Former Managing Editor, TRAINING magazine
Marty Murillo
Manager of Sales Training, iPlanet
Dr. Allison Rossett
Professor of Educational Technology, San Diego State University
Pat Weger
Vice President of Learning & Development, AT&T Broadband
These experts are committed to a fair and disinterested evaluation of every course.
Lguide’s value depends on our integrity. For this reason, we do not enter into
financial relationships that would compromise our objectivity:
• We do not accept money from publishers in exchange for reviewing
their products.
• We do not adjust our ratings to help course providers sell more courses.
• We do not produce or sell courses ourselves.
Lguide Review Methodology
Product reviews are written by Lguide’s staff of experienced consultants and
analysts, as well as a large body of contract reviewers with highly specialized skills.
Our team includes subject matter experts, educators, and IT professionals. We assign
courses to reviewers who are thoroughly familiar with the subject matter. Our
reviewers then act as informed peers, anticipating the questions and problems that
actual learners might face while noting any errors or omissions in the course
material. To date, our staff has reviewed over 1,700 online courses.
While there are other organizations that review online learning products, and other
methodological approaches to reviews, we believe Lguide reviews are unique in
several ways:
They are thorough and experiential
Lguide.com product reviews are founded on a simple premise: we use the products.
Every online course is reviewed by a team of Lguide analysts. We report on course
features that are helpful or frustrating, and we make informed predictions about
which audiences will find the course useful.
© 2001 LGUIDE
Page 7
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
They are team-based
Every review produced by Lguide is the product of a team of reviewers, not just the
opinion of one person. By working together, we are able to offer well-rounded
perspectives on the strengths and weaknesses of individual courses. Our team
approach also ensures consistency across reviews of all products, and enables us to
predict which courses will be useful for different types of learners.
They are contextual
Lguide product reviews offer evaluations of courses in the context of what’s
available on the market. We can offer our users a depth of information and
comparative analysis that is not available elsewhere because our team has studied
over 1,700 online learning products from leading course providers and publishers.
We continually review the latest products as they come to market, and as a result we
offer a breadth of market knowledge that sets us apart.
They are up-to-date and relevant
We re-examine and update our reviews on a regular basis, because the e-learning
industry is changing and growing as fast as the Internet itself. We update and refine
our assessment process as new instructional technologies become available, as
publishers update their course offerings, and as new publishers enter the marketplace. We are committed to identifying the best products and best practices in elearning today.
They evaluate instructional content
Once Lguide has been given access to the products of an e-learning content or
service provider, our editorial team assigns products to teams of analysts and outside
contractors with appropriate subject matter expertise. While it is normally difficult
to find contractors with subject matter expertise in the more complex technical
topics covered by some e-learning products, Lguide has dedicated substantial
resources to developing an extensive network of these specialists.
© 2001 LGUIDE
Page 8
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
Lguide Review Criteria
Lguide review criteria are designed to evaluate those features that learners can
expect from a quality e-learning product. We assess the three major elements of each
product—content, design and delivery, and overall value—by asking literally dozens
of questions about each course. We also test every feature of the course.
When evaluating an online course, our analysts ask the following questions:
Content
•
Does the course provide accurate and useful information? Is it wellorganized? Well-written?
•
Is the information provided appropriate for the intended audience? Are the
most relevant points appropriately emphasized?
•
Are the objectives appropriate for the course subject?
•
Are the objectives met?
•
Does the course provide additional resources and tools? Are these useful?
•
Is the tone of the course conversational and friendly, or dry and
impersonal?
Design & Delivery
•
Is the course optimized for the Web, or is it just an online document?
•
Do all course features function properly?
•
Are there interactive or multimedia elements (audio, graphics, simulations,
video) that enhance learning, or are they merely decorative?
•
Can users practice or apply skills as they learn them?
•
Does the course use realistic simulations or practice scenarios?
•
Can users control the course progress by skipping or repeating sections?
•
Are assessments relevant? Do they test central skills or concepts? Do they
enable customization?
•
Does the course support varied learning styles?
•
Is the navigation clear, intuitive, and easy to use?
•
Is the overall visual aesthetic of the course pleasing?
Value
© 2001 LGUIDE
•
Is this course worth the time and money?
•
How does it compare to available alternatives?
•
For whom is the course more or less valuable?
•
Would we recommend it to a friend?
Page 9
LGUIDE
TM
The E-Learning Experts
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
Summary of Findings
The bulk of the information that will be most helpful to consumers is located in the
sections that follow. These include our profiles of individual publishers, our evaluations of their products, and our analyses of courseware in each of our three major
categories: business skills, desktop application training, and professional IT skills.
The research in these sections yields the following general conclusions about the
state of the e-learning courseware industry.
The first result is not particularly surprising, but it shows that an oft-repeated adage
of instructional design does in fact hold when put into commercial practice: that
users learn more by doing than by watching. In short, we found that the most
effective learning products are those that try to help users acquire concrete
skills, and that do so by offering the ability for hands-on skill practice. After our
team had reviewed thousands of courses, we found that the courses that stayed with
us over time—the ones with lessons we could recall at a moment’s notice even after
weeks or months—were those that incorporated rich, interactive skill practice into
their design.
The second result stems directly from the first, but was far more surprising to our
research team. What we found is that it is possible for users to predict the degree
to which e-learning products will be able to improve skill acquisition before
examining any actual courseware. While this may seem counter-intuitive, our work
with the many courses we studied led us to realize that there are two factors that
consumers can use to evaluate how difficult it will be for course developers to
successfully bring certain training subjects into the online learning medium. This
degree of difficulty then correlates (inversely) with potential for product quality in
any given e-learning subject.
The first factor is the degree to which successful skill acquisition for the subject
is objective and clearly bounded. For subjects such as desktop applications and
professional IT certification, course designers can provide users with a limited
amount of concrete information that is not only finite, but that will yield immediate
positive results when implemented. This is also true for certain business skills such as
negotiation and conflict resolution, but less so for “softer” skills such as coaching,
leadership, and motivation. On a less intuitive note, there are several professional IT
areas, such as programming, that do not always lend themselves as easily to Webbased training because successful skill acquisition is objective (i.e. the code either
works or it does not) but not clearly bounded: experienced programmers realize that
there are multiple ways of solving coding problems. The best IT publishers are those
that use innovative course design elements to address these difficulties.
The second factor is the degree to which the training environment (i.e. a Web
browser and the text, graphics, exercises, and multimedia components within it) can
approximate the actual performance environment for the skills in question. The
best example of this is desktop application training, in which most publishers have
designed creative ways of simulating the actual desktop application within the
boundary of the Web browser. As a result, almost all desktop application courses
offer users the opportunity to practice their skills as they learn them, benefiting from
their mistakes as well as their successes. In contrast, it is much more difficult to take
a business skill such as mentoring and bring it online in convincing fashion. A wellwritten course can offer clear and concise advice about successful mentoring, but
can it offer users the opportunity to practice mentoring? Only the most innovative
and forward-thinking publishers are currently up to this challenge.
© 2001 LGUIDE
Page 10
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
When these two factors are combined and applied to potential e-learning subject
areas, the result is a map of subjects that not only matches our actual findings for
current e-learning product quality (see figure 1), but that can predict potential for
quality even in subject areas for which there is no existing courseware. Our belief is
that customers can use these factors to set and manage their own expectations when
searching for potential training products, and, perhaps more importantly, use them
to recognize when a product defies expectations by providing a new and innovative
solution to these challenges.
The bottom line result of these findings is one that is also useful to buyers of elearning courseware, because it enables them to assess the relative merits of different
publishers’ approaches to e-learning course design. We believe that most subjects
can and will be successfully brought to a WBT environment, but we have found that
the most successful publishers to date are those that acknowledge the difficulty
of bringing certain subjects to online training, and that create appropriate
compensatory strategies when developing courses in these subjects. Publishers
that do not customize their course design to acknowledge this challenge will
generally produce catalogs of inconsistent quality as a result.
Figure 1
Highest
Quality
HIGH
Computer Hardware
Desktop Applications
Interpersonal Communication
IT Certification Exams
Conflict Resolution
Degree to which
skill-related content is
objective and clearly
bounded
Q
l
ua
ity
Leadership
Programming
Coaching
Network Administration
Business Writing
LOW
Lowest
Quality
© 2001 LGUIDE
Degree to which Online Environment
Approximates Real-world Environment
HIGH
Page 11
LGUIDE
TM
The E-Learning Experts
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
Examples of this trend can be seen in our category-by-category analyses of product
quality, which offer evaluations of product strength in our three major categories of
business skills, desktop application training, and professional IT skills. Our conclusions there can be summarized as follows:
For business skills training, overall product quality is disappointing, but there is
hope. We found a marked difference in quality between the courses offered by large,
established e-learning companies, and a smaller group of “second generation”
publishers. While the first group offers more courses covering a greater number of
subjects, their products suffer from a highly template-driven design and from
inconsistent content that is often mediocre. In contrast, the second group offers more
focused catalogs of courses and seems to have targeted their development efforts to
the needs created by particular subjects. They customize their course design to
complement the subject matter, and they address the limitations of those subject
areas that do not lend themselves easily to online training.
For desktop application training, the state of the industry is sound. With a large
number of players competing to build courses for the same relatively small group of
leading desktop applications (notably the Microsoft Office suite), and a performance
environment (the user’s workstation “desktop”) that lends itself well to replication
in a Web-based training course, the courses in this area are very strong. With few
exceptions, users should be able to find high-quality training products for most
subjects from each of the publishers we profile, choosing the one that most clearly
matches their personal learning style. The strengths and weaknesses we identify for
these courses will help users make this match.
For professional IT training, experience counts. As in desktop application training, the performance environment for most professional IT skills matches the Webbased training environment, and the subject matter is always objective. While the
subject matter is not always well-bounded, IT publishers tend to have been involved
with e-learning longer than publishers of courses in other subject areas, and the
experience enables them to deal with these challenges. Furthermore, frequent
turnover in IT certification exams and in the technology itself both work to drive
publishers to update courses frequently. With frequent updates of content come
frequent opportunities to update course design: not surprisingly, many IT publishers
keep their course offerings on the leading edge of course design.
© 2001 LGUIDE
Page 12
LGUIDE
TM
The E-Learning Experts
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
Category Analysis: Business Skills
© 2001 LGUIDE
Page 13
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
First-Mover Disadvantage:
An Overview of Online Business
Skills Training
Overview
Effectively teaching business skills through a Web browser is very difficult, especially for soft skills topics such as coaching or leadership. The majority of the online
business skills courseware available today is of mediocre quality. Nevertheless, there
is hope: a small but growing number of publishers are offering engaging, welldesigned courses that are both enjoyable and effective.
The Mediocre Majority
The online business skills courseware market has been dominated to date by large
publishers such as SkillSoft, MindLeaders, and SmartForce, who offer broad
libraries of courses in all subject areas. Their catalogs are comprehensive, but the
courses themselves are produced using templates: sales courses have the same look,
feel, and simple exercises as customer service or team leadership courses.
These publishers rely on a standard teach-and-test model: like online textbooks, they
present chapters of text followed by multiple-choice quizzes. (SmartForce sometimes
reverses this model by testing, then teaching, but the principle is the same). There is
little interactivity in this model, and courses take little advantage of their online
platforms. The quality of individual courses varies widely across the catalog—you
can be sure the courses will look the same, but not that they will be equally
effective.
The High-Quality Minority
A growing number of business skills publishers are producing smaller and narrower
libraries of courses that are of consistently higher quality. Harvard Business School
Publishing’s e-Learning Interactive series, AchieveGlobal’s courseware, and NETg’s
new management and professional development courses are representative of the
better courseware that is increasingly available. These publishers have eschewed
rigid legacy training models and offer in their place a variety of flexible, user-driven
platforms that are highly interactive.
The bottom line is that not all small publishers produce outstanding courseware, but
we have yet to see a publisher with a very large catalog produce courses that are
consistently excellent. Even their best courses simply can’t compare to the courses
we’ve seen from the new breed of courseware providers.
Interactivity and Authenticity: Hope or Hype?
When it comes to differentiating between mediocre and magnificent courses,
interactivity and authenticity are key.
Computers can be a cold, impersonal medium, and the isolation inherent in Webbased training is its major liability when compared to classroom-based ILT.
Interactive exercises help mitigate these problems by creating an engaging
experience that will not only keep users awake, but also reinforce instruction.
Having said this, it’s important to note that interactivity can take many forms, and
that it is only useful if it is relevant and thought-provoking. Some publishers seem to
believe that mouse-clicking to turn pages constitutes an “interactive” experience. We
encounter empty or meaningless clicking exercises all the time, and they consistently
annoy rather than teach.
© 2001 LGUIDE
Page 14
LGUIDE
TM
The E-Learning Experts
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
The other key component of excellent courses is related to both content and to
design, and it is authenticity: perhaps the hardest element of a course to achieve via
computer screen. Authentic examples, case studies, scenarios, and dialogue are vital
to an effective training experience if the subject involves interpersonal skills.
Regardless of the learners’ level of technological sophistication or subject-matter
expertise, they can understand and detect phoniness—and if they can’t imagine
themselves ever saying or hearing what’s being presented to them as an “example”
of behavior, learners will reject both the example and the lesson it supports.
Interactivity and Authenticity = Engagement
“This course is engaging” is a way of saying that a course has successfully combined
interactivity and authenticity to create a realistic, challenging, and thoughtprovoking experience. Course material can’t and won’t be interesting unless it
challenges learners to think and presents them with problems that test their
capabilities. Yet if you study Web-based training materials and take hundreds of
online courses, as we do, it’s hard not to think that many publishers have little
respect for their users, offering multiple choice or fill-in-the-blank quizzes that
require little more than rote repetition of keywords and phrases from the bulleted
lists that make up the lessons. The best courseware we’ve seen takes the opposite
approach, forcing users to work through challenging but realistic interactive
exercises, and using authentic cases and scenarios to illustrate or model the skills in
question.
© 2001 LGUIDE
Page 15
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
Benchmarks for Choosing a Business Skills Course:
Necessary or Optimal Features
What to Look For
What to Avoid
Practicality
Practical information
Unrealistic or impractical
information
Concision
Concise writing
Wordy, abstract, or vague
writing
Examples
Authentic and relevant examples, No context or application of
case studies, scenarios
concepts
Content
Navigation
Moving between lessons
Detailed table of contents, course Incomplete or missing table of
menu, bookmarking
contents, no bookmarking
Moving within lessons
Intuitive navigation buttons,
including pause, fast-forward,
and rewind, adjustable speed
Structure that requires excessive
clicking, lack of pause, rewind,
or fast-forward buttons, fixed
speed
Interactivity
Exercises
Interactive elements that provoke Pointless or inappropriate
thought, exercises that are
exercises that consist of “empty
relevant to information
clicking”
Simulations
Authentic, interactive simulations Unrealistic scenarios that don’t
that react to your choices and
react to your choices and don’t
give feedback
give feedback
Multimedia
Video
Loads quickly, plays smoothly;
well-scripted and acted
Audio
Clips illustrate points and use
tone or inflection to capture
emotion in scenarios. Seamless
integration without awkward
players and pop-ups
Long downloads, poor image
quality; unrealistic dialogue
with awkward acting
Audio is nothing but narrated
course text, poorly read or
inflected. Play requires separate
clicking and increases download
times
Aesthetics
White space, visual variety,
helpful graphics
Text-dense screens, irrelevant
graphics, repetitive template
Subject-appropriate design
Interface that supports subject
Interface not suited to subject
Challenging questions, helpful
feedback, test acquired
knowledge
Obvious or obscure questions,
tests focus on jargon, little or no
feedback
Printable tips, charts,
worksheets, suggested reading
lists, links to Web sites
Complete, hyper-linked or
searchable
Filler material
Interface
Assessments
Well-written tests
Other Resources
Reusable Resources
Glossary
© 2001 LGUIDE
Irrelevant terms, not searchable or
linked to resources
Page 16
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
Is This It?: The Future of
Online Business Skills Training
If the mediocre courses currently flooding the WBT market represent the future, then
the future of business skills e-learning looks bleak. Poorly designed, poorly written
courses with meaningless interactivity will not sustain an e-learning revolution.
However, if the more recent wave of high-quality second-generation training courses,
often from small publishers, can gain traction in the marketplace, business skills elearning might yet be able to live up to its hype.
Evolution
The majority of publishers who are most successful at hawking their e-learning
wares seem to be ones that moved quickly to transfer existing training strategies and
existing content into a Web-based format. They built large catalogs quickly, gained a
foothold among customers, and generated a good deal of visibility, but their courses
are not aging gracefully. In the space of just a few years, many publishers have
entered the marketplace with new products that are much better adapted to the
online environment, and that take full advantage of the medium. Publishers such
AchieveGlobal, NETg, and Harvard Business School Publishing have all created
courses that maximize Web-based training’s potential while minimizing its
drawbacks. If the trend of intelligent application of technology to subject matter
continues, the next generation of business skills courseware will not only survive in
the online environment, but will thrive.
More Bandwidth = Better Courses
A widely touted theory is that increased availability of bandwidth will solve elearning’s problems. Bandwidth certainly makes for faster delivery, but it does not
by itself guarantee a good course. AchieveGlobal, for example, has some of the best
e-learning business skills courses that we’ve seen, and their products work well for
users with lower-bandwidth connections.
The key to understanding the impact of increased bandwidth is that it’s largely an
issue of increased potential. Increased bandwidth allows designers to incorporate
more robust simulations and assessments, but unless these features are applied
intelligently in support of quality content, they’re nothing more than empty bells and
whistles—shiny, new, and noisy, but ultimately just distracting.
The Bottom Line
Not surprisingly, much of the motive for moving training online has been financial—
training employees with the Web means less travel time, reduced course costs, and
less time away from work. But e-learning’s hype will fade if employees come away
dissatisfied with the online learning experience. It may be true that “If you build it,
they will come,” but that doesn’t mean they’ll stay. If the authenticity and
engagement of courseware is improved, employees will stay—the best of class online
courses are so engaging, they’re even fun.
© 2001 LGUIDE
Page 17
LGUIDE
TM
The E-Learning Experts
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
Category Analysis: Desktop Applications
© 2001 LGUIDE
Page 18
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
E-Learning that Works: An Overview
of Online Desktop Application Training
Overview
The current state of online courseware for desktop application training is promising.
The average desktop course provides a much better learning experience than does
the average course in other subject areas, especially business skills. Several factors
contribute to the relative quality of desktop application courseware:
Training Environment = Performance Environment
The computer is, not surprisingly, an effective training platform for desktop
applications—it’s much easier to effectively re-create the environment of Excel in a
Web browser than to authentically re-create a job interview, a performance
evaluation, or a workplace crisis.
Vendor-Driven Content
Desktop application software is an industry in which just a few prominent vendors
control a huge share of the market. Microsoft is the obvious leader with its Office
application suite. The effect of Microsoft’s dominance is that there are fewer online
publishers who offer courses in desktop applications from other vendors, such as
Corel, Adobe, or Lotus. While this is unfortunate for users of these programs, it
creates tremendous competition in courseware for Microsoft Office applications,
forcing publishers to continually re-engineer their courses. The lessons they learn
subsequently raise the bar for online training. Our report reflects this state of the
market, paying careful attention to the Office training space, but noting where
publishers create courses for other applications.
Well-Defined Knowledge Base
Another reason for the generally high level of quality in desktop courses is the
discrete, binary nature of the topic. There is a limited set of features and functions in
any desktop application, and course developers can measure absolutely within a
course whether a skill has been acquired and a user has done something correctly:
either the user pressed the correct button, or they didn’t; the text is either bold, or it
isn’t. Similarly, when we evaluate desktop application courseware, for each of many
functions, there is an absolute and objective answer to the question “Is it covered?”
That enables us to assess objectively the range of material covered by competing
courses. This stands in stark contrast to business skills subject areas such as
communication, coaching, or leadership, where even experts offer several competing
philosophies about best practices and measures of success.
MOUS Certification
A subset of desktop application courses covers the Microsoft Office User Specialist
(MOUS) Certification exam preparation courses. The MOUS exams certify users in
the desktop applications in the Microsoft Office suite. Microsoft has officially
certified several publishers for MOUS courses, but many other publishers offer full
coverage of MOUS topics within their regular courses. Many publishers offer study
guides specifically geared for MOUS review, and while the exam is not without its
weaknesses (see our report on MOUS preparation at www.Lguide.com), it provides a
shared set of standards for course content that many publishers use to good
advantage.
© 2001 LGUIDE
Page 19
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
Best Practices for Desktop Application Training:
Features to Look for in Online Courses
Necessary or Optimal Features
What to Look For
What to Avoid
Choices and Timesavers
Instruction in all task methods,
including keyboard and toolbar
shortcuts
Single method instruction;
no explanation of keyboard
shortcuts
Context
Case studies, examples, or
logical progression of lessons
Skills taught in isolation or
in an illogical order
Practice on Real or
Realistic Systems
Robust simulations with
feedback; exercises in the actual
application with measured
outputs
Overly sensitive, “picky”
simulations; overly controlled
exercises that march users
through in lockstep progression
Performance-based exercises
Users are asked to complete
tasks or use program features
Users are drilled with multiplechoice quizzes
Performance-based
examinations
Questions ask users to complete
actual tasks, and either specify
method to be used or allow
flexibility in answers
Questions don’t specify method to
be used and penalize users for
alternative (but correct) methods
Pre-assessments
Questions that respect real-world
knowledge and experience, and
use it to generate a customized
learning path.
Obscure or obvious questions that
don’t gauge actual knowledge;
exams that assess and do nothing
with the results
Content
Content
Assessments
Reference Tools and Performance Support
Glossary
Hyperlinked and/or searchable;
provides definitions during and
after the course
Incomplete or nonexistent
glossary; irrelevant or poorly
explained terms
Table of Contents, Menu, Index
Detailed to include topics and
sub-topics; key-word searchable
so users can use course as
ongoing reference
Menus or Tables of Contents
that list only chapter headings;
no index or search functions
Intuitive interface, flexible
learning path
Simple navigation within
lessons; ability to easily jump
between lessons
Counter-intuitive navigation; tiny
navigation buttons; course allows
linear progression only
Progress Tracking/
Bookmarking
Course offers progress indicators
within lessons and in the table of
contents; bookmarks return users
to the page, not just the chapter
No progress tracking or
bookmarking; tracking or
bookmarking only at the broad
chapter level
Navigation
© 2001 LGUIDE
Page 20
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
Getting from Good to Great: The Future
of Online Desktop Application Training
Although the state of online desktop application courseware is currently good, we
believe there remains ample room for improvement.
Simulation Improvements
As courseware simulation technology improves, the first agenda of course designers
should be to build more robust simulations that overcome over-sensitivity problems.
The key to making this work will undoubtedly be a combination of bandwidth
availability and thin client technology that will allow robust simulations to be
offered without crippling download times. After all, to be successful, a simulation in
a Web-based course must not only perform correctly, but also load quickly. It’s also
possible that more publishers will eschew cumbersome simulations in favor of the
DigitalThink/ActiveEducation model of relying on the exercises within the live
application for skills practice.
Performance-Based Training
If MOUS certification continues to gain traction, then WBT publishers will take the
extra steps necessary to make their courses support true performance-based training
and testing, since this is the model used by the MOUS exam itself. Even if MOUS
certification fizzles, a move in this direction will still be the ideal future of Webbased training: it’s simply the best way to master desktop applications.
Audience Differentiation
Many online desktop courses are trying to be all things to all people—
accommodating complete novices and those with some experience alike. The
standard model is to label courses according to rough categories such as “Beginner”
or “Expert,” but the actual course structure isn’t usually differentiated. Instead, most
courses balance comprehensive content suitable for beginners with a custom
learning path for more the experienced user.
The problem is that it’s hard to create tests that correctly gauge user proficiency
with desktop applications without using performance-based assessment, and until
these are available, the best approach might be for publishers to offer different
courses structured for different user bases.
Courses written specifically for brand-new users could include step-by-step
introduction of a large amount of sequentially introduced information, and courses
for more experienced users could offer more flexible navigation with a more layered
structure of information.
The Ultimate Goal
If we look at these three factors—the need for more robust simulations, for
assessments that track and measure actual performance, and for courses that adapt
to offer learning customized to the experience level of the user—what they all point
towards is a solution that would incorporate the best of all these worlds: training
that’s more like a very good help system, or a training application that can hook
into the existing application on the user’s computer to act as a guide or tutor within
the live program.
© 2001 LGUIDE
Page 21
LGUIDE
TM
The E-Learning Experts
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
Clearly, the leading contenders for building this type of product would be the
application vendors themselves. After all, no one is in a better position to create an
application that would integrate with Microsoft Word than Microsoft itself.
Yet discussing application vendors such as Microsoft also raises an important issue
that observers of the e-learning industry should keep in mind. The entire desktop
application training is founded on the premise that these applications are
complicated to use and difficult to master. But if publishers of online training
courses will undoubtedly improve their products over the upcoming years, so too,
will the producers of the applications. The ultimate end of e-learning will be
applications that automatically teach themselves.
© 2001 LGUIDE
Page 22
LGUIDE
TM
The E-Learning Experts
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
Category Analysis:
Professional Information Technology
© 2001 LGUIDE
Page 23
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
Experience Counts: An Overview of
Online Professional IT Training
Overview
Professional IT e-learning is relatively strong. IT trainers moved online earlier than
business skills trainers, so they’ve had longer to work out the kinks and learn how to
design training for the Web. Because the repercussions can be severe and immediate
if information is inaccurate, IT learners tend to hold publishers to a high standard.
Certification programs also impose a certain discipline on IT content, and where
better to learn technological skills than at a computer, via the Internet?
IT Trainers Were Online Early
Early movers in IT training had courses online as early as 1996. Those publishers
and their instructional designers have had plenty of time to adapt and re-adapt
content to the Web.
IT Training Is Well-Developed as a For-Profit Learning Industry
Many of the same issues that are crucial for e-learning—scalability, chunkability,
customization for adult learners—are also crucial for any for-profit learning
industry that targets adults, and IT trainers have had years of experience with these
issues.
IT Training Is a Binary Operation
The nature of technology itself imposes order and quality on IT course content.
Unlike “softer” training areas such as leadership or management skills, there is a
defined body of knowledge in IT, and it’s relatively simple to verify whether a
learner has mastered it: either a program works, or it doesn’t.
Technological Changes Drive Turnover
IT publishers can’t re-sell the same stale courses year after year (or even month after
month), because technological changes drive course turnover. That means that IT
courses on current technologies are also likely to use the latest developments in
interface and instructional design.
Certification Programs Discipline Course Content
Certification programs impose further discipline on IT course content. One of the key
drivers of growth in the professional IT e-learning space is the market demand that
is created by the relatively rapid evolution of standards in hardware and software
development. As manufacturers and software developers continually race to upgrade
their offerings, IT professionals must also increase and maintain their skills so that
they can keep up with the environment and stay ahead of obsolescence. With its
faster development and deployment cycles, e-learning is the ideal medium for this
fast-paced training. The shelf life of these courses is also relatively brief, further
driving course turnover.
The multitude of professional IT certifications creates not just a widespread need for
training, but a highly standardized and regulated body of knowledge for that
training. With product vendors such as Microsoft, Oracle, Novell, and Cisco providing the course content, publishers can focus their attention on effective design and
delivery. Since e-learning courseware development is largely a game of managing
limited resources, the added attention to issues of product design makes a difference.
© 2001 LGUIDE
Page 24
LGUIDE
TM
The E-Learning Experts
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
Training Environment = Performance Environment
The computer is, not surprisingly, an effective training platform for computer-based
skills. For programming and operating system courses, publishers can create Webbased products that provide realistic practice examples and detailed feedback to
guide users through complex IT systems. The best IT publishers incorporate coding
examples into their programming courses and simulate OS features in their administration courses. Capturing the nuances of hardware maintenance is much more
difficult, but even here, IT publishers can create interactive three-dimensional
simulations and illustrations that are superior to two-dimensional textbooks.
© 2001 LGUIDE
Page 25
LGUIDE
E-Learning Course Publishers:
A Comparative Analysis and Industry Directory
TM
The E-Learning Experts
Benchmarks for Choosing a Professional IT Course
Necessary or Optimal Features
What to Look For
What to Avoid
Accuracy
Current information, accurate
details
Dated information, especially in
certification courses
Context
Central definitions, explanations
of relation to other technologies,
current developments
Rote memorization
Appropriate Emphasis
Emphasis appropriate for
learner’s knowledge level
Beginning courses that jump over
central concepts to arcane
features, advanced courses that
dwell on basic definitions
Delineated Steps
Step-by-step breakdowns of
complicated processes
General explanations unsupported by specifics
Clear Sense of Audience
Clear sense of what you need
to know to benefit from the
course
No consideration of background
knowledge necessary (results in
confusion)
Glossary
Easily-accessed glossary that
includes key terms
Courses with no glossary
Realistic Practice
Simulations or exercises that ask
you to perform the technical skill
being taught; simulations that
accurately reflect the functioning
of real systems
Courses that offer only extremelyguided exercises or multiplechoice quizzes; buggy interfaces
that don’t let you practice skills
Interactivity
Engaging graphics, exercises,
simulations, and assessments,
especially in longer courses
Bookmarking
Features that allow you to leave
the course and return to where
you left
Courses consisting entirely of
text; often a problem when
courses have been adapted from
books without being revamped
for the Web
Courses that force you to repeat
entire sections when you resume
the course
Pre-Assessments and Custom
Learning paths
Pre-assessments that accurately
measure your knowledge and
give you a suggested path
through the course
Explanations
Interface
© 2001 LGUIDE
Inaccurate pre-assessments may
lead you to skip topics you need
to know
Page 26