Introduction
Based on current academic and practitioner work and also the Value of Learning discussion thread results we produced a poll investigating the barriers to evaluating learning and invited members of the network to participate online. In all 392 CIPD members contributed during the period November 2006 to January 2007.
The poll results are below, and are interesting in terms of the extent to which respondents perceived different barriers when taken in relation to one another (as reflected in the ‘always/very often’ sum in Table 1 below). The two propositions which had the highest level of agreement related to evaluation being too costly given other priorities and the lack of interest from line managers in evaluation. The lowest level of support was given to the proposition concerning poor learning objectives as a barrier to evaluating learning.
Table 1: Poll Results
Please indicate on this poll the barriers that you have experienced with ‘serious’ evaluation of learning and training in your organisation:
| Proposition |
Always |
Very often |
Some- times |
Never |
Always/ V often |
| The objectives of learning and training interventions are unclear |
8.7% |
27.9% |
51.1% |
11.1% |
37% |
| We lack effective evaluation techniques |
15.6% |
35.5% |
37.4% |
10.3% |
51% |
| Decisions about future learning and training are not influenced by the results of training evaluation processes |
17.2% |
36.3% |
35.9% |
10.3% |
53% |
| The organisation is more concerned to measure the costs and not the benefits of learning and training |
20.2% |
36.6% |
31.7% |
11.1% |
57% |
| The benefits of learning and training interventions are too difficult to measure |
14.1% |
45.8% |
35.9% |
3.4% |
60% |
| Line managers are not interested in training evaluation; they do not or would not use the data |
19.1% |
47.0% |
29.0% |
4.6% |
66% |
Serious' evaluation is too time consuming/
costly and we have other priorities |
22.5% |
47.3% |
24.8% |
5.3% |
70% |
Comments on the results
The poll results reflected the concerns expressed in the discussion thread. While ‘unclear learning objectives’ was the barrier to learning rated lowest at 37%, this may not be something the LTD practitioner can easily affect given the lack of clarity may come either from the line manager or the learner themselves.
Around half of respondents reported ‘lack of effective techniques’ as a barrier to learning evaluation. The discussion thread also suggested that while Kirkpatrick Levels 1 and 2 can be undertaken, measuring at higher levels is problematic. Indeed some respondents appeared to doubt the possibility of providing any adequate measure, agreeing to ‘benefits are too difficult to measure’ as a barrier (60%). This finding presents food for thought in terms of the potential for developing a measure in the future. Clearly any measure would need to be convincing to a wide range of stakeholders.
The apparent lack of influence from evaluation processes on future decisions was considered a barrier to evaluation by 53% of respondents. This suggests that decisions about LTD may not be influenced by any evaluation outcomes. This is being probed in the current stage of the project which involves interviews with both learning and non-learning decisions makers in organisations. We are also examining the influence of the cost of evaluation, which was seen by 57% as a barrier within the first poll.
The propositions with the highest level of agreement highlight the importance of developing time-effective evaluation practices which are relevant to the needs and priorities of managers.
We welcome your feedback on these results and we would encourage you to contribute your discussion to the Value of Learning discussion thread.