Kirkpatrick's Four Levels Of Training Evaluation In Detail - Connecticut

Kirkpatrick's Four Levels of Training Evaluation in DetailThis grid illustrates the Kirkpatrick's structure detail, and particularly the modern-day interpretation of the Kirkpatrick learningevaluation model, usage, implications, and examples of tools and methods. This diagram is the same format as the one above but withmore detail and explanation:EVALUATIONTYPELEVEL 1REACTIONLEVEL 2LEARNINGEVALUATION DESCRIPTION ANDCHARACTERISTICSEXAMPLES OF EVALUATION TOOLS ANDMETHODS§ reaction evaluation is how thedelegates felt, and their personalreactions to the training or learningexperience, for example:§ did the trainees like and enjoy thetraining?§ did they consider the trainingrelevant?§ was it a good use of their time?§ did they like the venue, the style,timing, domestics, etc?§ level of participation§ ease and comfort of experience§ level of effort required to make themost of the learning§ perceived practicability and potentialfor applying the learning§ learning evaluation is themeasurement of the increase inknowledge or intellectual capabilityfrom before to after the learningexperience:§ did the trainees learn what intendedto be taught?§ did the trainee experience what wasintended for them to experience?§ what is the extent of advancement orchange in the trainees after thetraining, in the direction or area thatwas intended?§ typically 'happy sheets'§ feedback forms based on subjective personalreaction to the training experience§ verbal reaction which can be noted andanalyzed§ post-training surveys or questionnaires§ online evaluation or grading by delegates§ subsequent verbal or written reports givenby delegates to managers back at their jobs§ can be done immediately thetraining ends§ very easy to obtain reactionfeedback§ feedback is not expensive togather or to analyze for groups§ important to know that peoplewere not upset or disappointed§ important that people give apositive impression whenrelating their experience toothers who might be decidingwhether to experience same§ typically assessments or tests before andafter the training§ interview or observation can be used beforeand after although this is time-consumingand can be inconsistent§ methods of assessment need to be closelyrelated to the aims of the learning§ measurement and analysis is possible andeasy on a group scale§ reliable, clear scoring and measurementsneed to be established, so as to limit the riskof inconsistent assessment§ hard-copy, electronic, online or interviewstyle assessments are all possible§§ relatively simple to set up, butmore investment and thoughtrequired than reactionevaluation§ highly relevant and clear-cut forcertain training such asquantifiable or technical skills§ less easy for more complexlearning such as attitudinaldevelopment, which is famouslydifficult to assess§ cost escalates if systems arepoorly designed, which increaseswork required to measure andanalyzeRELEVANCE AND PRACTICABILITY

EVALUATIONTYPELEVEL 3BEHAVIOREVALUATION DESCRIPTION ANDCHARACTERISTICSEXAMPLES OF EVALUATION TOOLS ANDMETHODS§ behavior evaluation is the extent towhich the trainees applied thelearning and changed theirbehavior, and this can beimmediately and several months afterthe training, depending on thesituation:§ did the trainees put their learninginto effect when back on the job?§ were the relevant skills andknowledge used§ was there noticeable and measurablechange in the activity andperformance of the trainees whenback in their roles?§ was the change in behavior and newlevel of knowledge sustained?§ would the trainee be able to transfertheir learning to another person?§ observation and interview over time arerequired to assess change, relevance ofchange, and sustainability of change§ arbitrary snapshot assessments are notreliable because people change in differentways at different times§ assessments need to be subtle and ongoing,and then transferred to a suitable analysistool§ assessments need to be designed to reducesubjective judgment of the observer orinterviewer, which is a variable factor thatcan affect reliability and consistency ofmeasurements§ the opinion of the trainee, which is arelevant indicator, is also subjective andunreliable, and so needs to be measured in aconsistent defined way§ 360-degree feedback is useful method andneed not be used before training, becauserespondents can make a judgment as tochange after training, and this can beanalyzed for groups of respondents andtrainees§ assessments can be designed around relevantperformance scenarios, and specific keyperformance indicators or criteria§ online and electronic assessments are moredifficult to incorporate - assessments tend tobe more successful when integrated withinexisting management and coaching protocols§ self-assessment can be useful, usingcarefully designed criteria andmeasurementsis the trainee aware of their change inbehavior, knowledge, skill level?RELEVANCE AND PRACTICABILITY§§§§§§measurement of behaviorchange is less easy to quantifyand interpret than reaction andlearning evaluationsimple quick response systemsunlikely to be adequatecooperation and skill ofobservers, typically linemanagers, are importantfactors, and difficult to controlmanagement and analysis ofongoing subtle assessments aredifficult, and virtuallyimpossible without a welldesigned system from thebeginningevaluation of implementationand application is an extremelyimportant assessment - there islittle point in a good reactionand good increase in capabilityif nothing changes back in thejob, therefore evaluation in thisarea is vital, albeit challengingbehavior change evaluation ispossible given good support andinvolvement from line managersor trainees, so it is helpful toinvolve them from the start, andto identify benefits for them,which links to the level 4evaluation below

EVALUATIONTYPELEVEL 4RESULTSEVALUATION DESCRIPTION ANDCHARACTERISTICS§ results evaluation is the effect onthe business or environmentresulting from the improvedperformance of the trainee - it is theacid test§ measures would typically be businessor organizational key performanceindicators, such as:§ volumes, values, percentages,timescales, return on investment,and other quantifiable aspects oforganizational performance, forinstance; numbers of complaints,staff turnover, attrition, failures,wastage, non-compliance, qualityratings, achievement of standardsand accreditations, growth,retention, etc.EXAMPLES OF EVALUATION TOOLS ANDMETHODS§§§§§§it is possible that many of these measuresare already in place via normalmanagement systems and reportingthe challenge is to identify which and howrelate to the trainee's input and influencetherefore it is important to identify andagree accountability and relevance withthe trainee at the start of the training, sothey understand what is to be measuredthis process overlays normal goodmanagement practice - it simply needslinking to the training inputfailure to link to training input type andtiming will greatly reduce the ease bywhich results can be attributed to thetrainingfor senior people particularly, annualappraisals and ongoing agreement of keybusiness objectives are integral tomeasuring business results derived fromtrainingRELEVANCE AND PRACTICABILITY§ individually, results evaluation isnot particularly difficult; acrossan entire organization itbecomes very much morechallenging, not least becauseof the reliance on linemanagement, and the frequencyand scale of changingstructures, responsibilities androles, which complicates theprocess of attributing clearaccountability§ also, external factors greatlyaffect organizational andbusiness performance, whichcloud the true cause of good orpoor resultsSince Kirkpatrick established his original model, other theorists (for example Jack Phillips), and indeed Kirkpatrick himself, havereferred to a possible fifth level, namely ROI (Return On Investment). In my view ROI can easily be included in Kirkpatrick's originalfourth level 'Results'. The inclusion and relevance of a fifth level is therefore arguably only relevant if the assessment of Return OnInvestment might otherwise be ignored or forgotten when referring simply to the 'Results' level.Learning evaluation is a widely researched area. This is understandable since the subject is fundamental to the existence andperformance of education around the world, not least universities, which of course contain most of the researchers and writers.While Kirkpatrick's model is not the only one of its type, for most industrial and commercial applications it suffices; indeed mostorganizations would be absolutely thrilled if their training and learning evaluation, and thereby their ongoing people-development,were planned and managed according to Kirkpatrick's model.The use of this material is free provided copyright (see below) is acknowledged and reference or link is made to the www.businessballs.com website. This materialmay not be sold, or published in any form. Disclaimer: Reliance on information, material, advice, or other linked or recommended resources, received from AlanChapman, shall be at your sole risk, and Alan Chapman assumes no responsibility for any errors, omissions, or damages arising. Users of this website are encouragedto confirm information received with other sources, and to seek local qualified advice if embarking on any actions that could carry personal or organizational liabilities.Managing people and relationships are sensitive activities; the free material and advice available via this website do not provide all necessary safeguards and checks.Please retain this notice on all copies. Donald Kirkpatrick's Learning Evaluation Model 1959; review and contextual material Alan Chapman 1995-2007

Level One Evaluation: ReactionIn order to have a good discussion about Kirkpatrick's Level One Evaluation it is helpfulto see Kirkpatrick's complete model of evaluation. Below is a diagram of Kirkpatrick'sFour Levels of Evaluation Model (1994) of reaction, learning, performance, and impact.The Kirkpatrick's Four Levels of Training EvaluationNeedsImprovemenl?-No .Level One:This is the first step of Kirkpatrick's evaluation process where students are asked toevaluate the training the attended after completing the program. These are sometimescalled smile sheets or happy sheets because in their simplest form they measure how wellstudents liked the training. Don't be fooled by the adjectives though, this type ofevaluation can reveal useful data if the right questions asked are: The relevance of the objectives.The ability 0 f the course to maintain interest.The amount and appropriateness of interactive exercises.The perceived value and transferability to the workplace.