Kirkpatrick Model-Disadvantages

Level 1: Reaction

Reaction evaluation is how the delegates felt, and their personal reactions to the training or learning experience, for example:
· Did the trainees like and enjoy the training?
· Did they consider the training relevant?
· Was it a good use of their time?
· Did they like the venue, the style, timing, domestics, etc?
· Level of participation.
· Ease and comfort of experience.
· Level of effort required to make the most of the learning.
· Perceived practicability and potential for applying the learning.

Here are some examples of evaluation tools and methods:

· Typically 'happy sheets'.
· Feedback forms based on subjective personal reaction to the training experience.
· Verbal reaction which can be noted and analysed.
· Post-training surveys or questionnaires.
· Online evaluation or grading by delegates.
· Subsequent verbal or written reports given by delegates to managers back at their jobs.

All these can be done immediately after the training ends as it is very easy to obtain reaction feedback. Feedback is not expensive to gather or to analyse for groups. It is important to know that people were not upset or disappointed and to giving a positive impression when relating their experience to others who might be deciding whether to experience the same.

Level 2: Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before to after the learning experience. For example:
· Did the trainees learn what was intended to be taught?
· Did the trainee experience what was intended for them to experience?
· What is the extent of advancement or change in the trainees after the training, in the direction or area that was intended?

Typically assessments or tests can be done before and after the training. It is relatively simple to set up, but more investment and thought required than reaction evaluation. It is highly relevant and clear-cut for certain training such as quantifiable or technical skills but is less easy for more complex learning such as attitudinal development, which is famously difficult to assess. The methods of assessment need to be closely related to the aims of the learning. Measurement and analysis is possible and easy on a group scale. Reliable, clear scoring and measurements need to be established, so as to limit the risk of inconsistent assessment. Cost escalates if systems are poorly designed, which increases work required to measure and analyse.

Here are some examples of evaluation tools and methods:

· Interview or observation can be used before and after the training.
· Hard-copy, electronic, online or interview style assessments are all possible.

Level 3: Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour, and this can be immediately and several months after the training, depending on the situation:
· Did the trainees put their learning into effect when back on the job?
· Were the relevant skills and knowledge used
· Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles?
· Was the change in behaviour and new level of knowledge sustained?
· Would the trainee be able to transfer their learning to another person?
· Is the trainee aware of their change in behaviour, knowledge, skill level?

Observation and interview over time are required to assess change, relevance of change, and sustainability of change. Measurement of behaviour change typically requires cooperation and skill of line-managers.

Level 4: Results

Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test.
Measures would typically be business or organisational key performance indicators, such as:
Volumes, values, percentages, timescales, return on investment, and other quantifiable aspects of organisational performance, for instance; numbers of complaints, staff turnover, attrition, failures, wastage, non-compliance, quality ratings, achievement of standards and accreditations, growth, retention, etc.


References:
http://www.businessballs.com/kirkpatricklearningevaluationmodel.htm


Disadvantages

Level 1: Reaction
Only reflects a quick reading of the participant while they are still in the class. Results should not be used as a solid basis for changing the educational content or strategy

Level 2: Learning
Requires more time and money than level 1. Also requires greater insight to the evaluation process to develop valid measures of learning.

Level 3: Behaviour
Requires significantly more investment of time and money. Requires in-depth insight into performance interventions and root causes of performance deficiencies.

Level 4: Results
Substantial levels of investment and expertise are required to develop level 4. Often hard to decide whether or not this level is required. Linkage from training to org'1 results is hard to establish.

Reference:
http://www.internationalmta.org/1998/9821.html