EPM Training Services organizes and runs project management training courses for our clients. We are therefore very interested in the results of those training programs. Are they beneficial to the participants and their employers? How do we measure the results of training in an organization?

Assessing the effectiveness of training is often performed using the four-level model developed by Donald Kirkpatrick, whose ideas were first published in 1959. The four levels of Kirkpatrick’s evaluation model essentially measure:

• Reaction of student (their thoughts, feelings and perceptions about the training). Did they like it? Was the material relevant to their work? This type of evaluation is often called a “happy-sheet” which students complete at the end of any training session.

• Learning (the increase in knowledge or capability as a result of the training). Evaluation at this level attempts to assess how far students have advanced in skills, knowledge, or attitude. Measurement methods include formal to informal testing, team assessment, and self-assessment.This might involve participants taking an assessment before the training (pretest) and after training (post test) to determine the amount of learning that has been transferred.

• Behavior (the extent of behavior and capability improvement and application). Are the newly acquired skills, knowledge, or attitude being used in the everyday environment of the learner?

• Results (effects on the business or environment resulting from the trainee’s performance). This level measures the success of the training program in business terms including increased productivity, better quality, decreased costs, reduced frequency of accidents, and increased sales.

Notice anything missing here? What about Return On Investment (ROI)?

The financial benefits of training can’t be measured in terms of student reactions, nor the amount of  learning that has been achieved, or even the extent to which behavior may have changed. The real benefits come from improved performance – traditionally the hardest training outcome to measure.

Jack J. Phillips proposed that we use ROI as the fifth-level of evaluation. Did the monetary value of the results exceed the cost for the program? This is probably going to be the most difficult thing for the poor overworked Human Resources Department to measure, as it’s only the very senior management who are able to get access to this information. Therefore, senior level support is essential. The cost of measurement is also going to be high. Recognizing this, Phillips proposes a 5% evaluation target for this level of assessment (compared to a 100% target for Level-1). A company would only want to undertake this increased level of assessment for the highest cost, or highest visibility, project management training programs.

Here’s the book:  How To Measure Training Results

ROI is defined like this:

(Net Program Benefits) / (Program Costs) X 100 = ROI

Total benefits include money saved by the organization, and money made.

Total costs include the obvious and the not-so-obvious: Development costs, learner’s time away from work, overhead of HR department, materials, meals, refreshments, facilities, cost of coordination, cost of job coverage during training, and any other cost incurred.

Measuring the outcome of your project management training is an essential part of any training program, and will show you the value of training  and justify the investment.