|
 |
|
 |
| ARTICLES OF INTEREST
- ELEARNING |
|
| Beyond
Kirkpatrick: Measuring the Financial Returns of e-Learning |
| by
Kevin Kruse |
| The classic
evaluation model developed by Donald Kirkpatrick looks
at four levels: student reaction, knowledge transfer,
behavioral change, and business results (Kirkpatrick,
1975). Critics of Kirkpatrick model say that it doesn't
take the business impact far enough and that the final
step in any training program should be a "fifth level"
of evaluation -- financial return. This ultimate evaluation
determines the financial return on investment (ROI) of
the training program. |
| New Demands
for Accountability |
For a
long time, organizations paid lip service to the human
resource motto that "people are an organization's
most important asset." Now senior executives have
come to believe that employees and the intellectual
capital they create can uniquely differentiate their
company in the marketplace. Training expenditures now
are viewed as critical investments in human capital,
and an effective method of increasing employee retention.
Because of these factors, the investment in corporate
training programs is large and growing rapidly. Total
corporate spending on training in the United States
was approximately $60 billion in 1998 (Training, October
1998). The fastest-growing segment of the training budget
is expenditures on technology-based training. By the
year 2000, it is estimated that 50 percent of all training
interventions will be delivered via CD-ROM or corporate
Intranets (Ibid.). Specifically, within the Web-based
training marketplace, expenditures are expected to go
from a mere $197 million in 1997 to over $6 billion
in 2002 (International Data Corporation, 1998). This
projected 100 percent annual growth rate in technology-based
training is being driven by increases in the availability
of network bandwidth, the decreasing price of multimedia
computers, and the development of more services and
products from vendors.
Commensurate with this increase in training expenditures,
senior executives are demanding more accountability
from their training departments. In fact, 93 percent
of training professionals surveyed at a 1996 conference
said they are increasingly being asked to show the return-on-investment
of their programs (National HRD Executive Survey, 1997).
Training managers need to be able to answer direct questions
about total costs, benefits, and bottom-line impact.
Visionary training managers embrace cost-benefit analysis
as a way to justify bigger budgets for technology and
new training programs. |
| The
Value of Cost-Benefit Analysis |
Proponents
of technology-based training have long touted its many
benefits: reduction in learning time, increase in knowledge
retention rates, cost savings. Brandon Hall, quoted
in the August 1998 issue of HRMagazine, makes the generalization:
"There's about a 50-percent reduction in time and
cost over classroom training." (Roberts, 1998)
The power of the cost-benefit analysis process is that
it enables you to move from generalizations and assumptions
to proof of the value of each and every program you
develop.
This type of quantifiable measurement of value is critical
in the overall management of a training function, and
a powerful tool that can be used to keep or expand available
training resources. Brandon Hall, editor of Multimedia
& Internet-Based Training Newsletter, has conducted
exhaustive research in the area of return on investment.
His research and that of others have uncovered some
compelling cases:
- A computer storage media company
converted a four-day instructor-led course for 1,500
technicians into a multimedia CD-ROM format. Due to
a reduction in learning time and elimination of travel
expenses, Storage reduced costs over three years by
47 percent and saved $1.5 million (Hall, 1997).
- A major consultancy firm
developed and delivered computer-based training for
7,000 consultants in 50 countries. The cost of the
training was $106 per student, versus an estimated
$760 per student for instructor-led delivery. Over
the five-year life span of the program, technology-based
training saved the firm more than $4.5 million (Ibid.).
- A computer reseller developed
a Web-based training solution for its internal sales
force and value-added resellers. Some 40 online courses
were developed, complete with self-assessment quizzes.
According to their director of strategies technologies,
"We saw a 50 percent increase in sales across
distribution and integration resellers," (Fickel,
1998).
- A branch of the U.S. military
estimated that their technicians' ability to troubleshoot
problems increased by 90 percent after the adoption
of multimedia training. Over a period of five years,
they expect at least a 20-fold return on their investment
(Jerram, 1994).
|
| Key Concepts
of Cost-Benefit Analysis |
At the
simplest level, cost-benefit analysis answers the question
"Was it worth the money?" In other words,
what were the total costs to develop the program, and
what were the total benefits realized? Costs include
direct costs, such as payments to vendors, as well as
indirect costs, such as the value of time. Financial
benefits can be in the form of cost savings, or increases
in productivity or revenue. The following key concepts
are factors in a cost-benefit analysis.
- Life of training. Every
project needs to be measured across some time period.
Technology-based training programs don't last forever.
Their shelf life will be determined by things such
as changes to content, changes in technology, and
changes in business need. According to Hall's research
conducted over the last ten years, most ROI studies
show technology-based training is more expensive to
develop and deliver over the short-term, but pays
off over time. Typically, three-to-five years of use
is an accepted time period to apply for evaluating
a training program.
- Alternate delivery options.
Perhaps the most common method of showing the financial
impact of technology-based training is to compare
it against the costs for other forms of delivery.
To come up with a comparison means, ask the question,
"If we don't deliver the training via the Web,
what would it cost for us to deliver it in a classroom
setting?"
- Size of audience. With
technology-based training, the cost of development
is not dramatically effected by the number of students
using it. The cost is basically the same to develop
a two-hour CD-ROM or Web-based training program for
10 people as it is for 1000 people. The only additional
costs may be in the form of CD-ROM duplication, student
tracking, and end-user support. However, the size
of the target audience is extremely relevant when
comparing the costs against instructor-led delivery.
With live workshops, the number of students has a
direct impact on expenses related to instructors,
locations, and travel.
- Seat time. The total amount
of time students will spend with the course is called
seat time -- how long they will be in their seats.
Seat time is always specified for instructor-led training,
but is an estimate when given for self-paced, technology-based
training. After all, a course that takes one student
two hours to complete, might take another only 90
minutes. Increasingly, effective Web-based training
is blurring the lines between instruction and just-in-time
performance support. This factor makes estimates of
seat time additionally tenuous.
- Burdened costs. This accounting
term refers to the total cost of an item, which may
include some hidden costs. For example, you might
quickly estimate that a classroom facilitator who
earns a $60,000 salary costs $230 per day, simply
by dividing the salary by the total number of weekdays
($60,000 ¸ 52 weeks ¸ 5 days). But the
burdened cost for the instructor will be higher once
you take into account payroll taxes, insurance, and
other benefits. Additionally, when calculating day
rates, make sure to subtract company holidays, vacation
time, and sick days to get an accurate estimate of
the burdened cost for each productive workday.
- Estimated revenue impact.
Often the impact a training program has on sales and
expenses is indirect, or difficult to measure. In
these cases, the impact on revenue is projected or
extrapolated from known data. For example, assume
that a quality control training program was shown
to reduce the number of defective cell phones produced
each year in a factory from 5000 to 3000 (net reduction
of 2000 defective phones a year). Although the training
program directly reduced errors, which led to a drop
in the number of defective phones, you would have
to estimate the revenue impact. To do this, you would
need to research costs associated with wasted materials
in each defective phone, labor time for the manufacturing,
identification of, and disposal of each defective
phone. With this methodology, a defect-reduction number
can be translated into a revenue-savings number.
- Opportunity costs. These
costs are the lost revenues or increased costs associated
with opportunities that will be missed because of
the training program. This measure is increasingly
being used in the competitive world of sales. Traditionally,
for a sales rep spending time in training, a main
measure of cost is the salary of the rep while in
the training program. However, a more advanced analysis
measures the opportunity cost of the rep not being
out in the field. According to Jim DeMaioribus, associate
director of sales training at Knoll Pharmaceuticals,
every day a rep spends in the field is worth approximately
$8,000 in revenue. Therefore a major advantage of
technology-based sales training is its ability to
maximize time in the field and minimize the opportunity
costs of sales training.
|
|
| 1.
10 Tips For Online Learning |
| 2.
Evaluating e Learning |
| 3.
The Magic of Learner Motivation |
| 4.
The Benefits and Drawbacks of e-Learning |
| 5.
Marketing e-Learning |
| 6.
Measuring the Financial Returns of e-Learning |
| |
|
 |
|
 |
|
CoreBiz Business Solutions : |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|
| Receive
periodic updates and announcements on the latest
happenings within CoreBiz Technology and the IT
industry. |
 |
|
|
|