\n\t\t\t\t\t\t\t\t\t

The complete story of a training program success or failure<\/p>

Patricia and Jack Phillips, from ROI Institute, included return on investment (ROI) as the fifth level of the traditional Kirkpatrick evaluation levels. According to this new ROI methodology for training and education practitioners, the new evaluation framework includes five levels of data:<\/p>

• • Level 1<\/strong>\u00a0Reaction, Satisfaction, and Planned Action\u2014Data representing participants’ reactions to the program and their planned actions is collected and analyzed. Reactions may include participants’ views of the course content, facilitation, and learning environment. This category of data also includes data often used to predict application of acquired knowledge and skills, including relevance, importance, amount of new information, and participants’ willingness to recommend the program to others.<\/li><\/ul><\/li><\/ul>
    • Level 2<\/strong>\u00a0Learning\u2014Data representing the extent to which participants acquired new knowledge and skills is collected and analyzed. This category of data also includes the level of confidence participants have in their ability to apply what they have learned.<\/li>
    • Level 3<\/strong>\u00a0Application and Implementation\u2014Data is collected and analyzed to determine the extent to which participants effectively apply their newly acquired knowledge and skills. This category also includes data that describes the barriers that prevent application and any supporting elements (enablers) in the knowledge transfer process.<\/li>
    • Level 4<\/strong>\u00a0Business Impact\u2014Data is collected and analyzed to determine the extent to which participants’ applications of acquired knowledge and skills positively influenced key measures that were intended to improve as a result of the program. When reporting data at Level 4, a step to isolate the program’s effects on these measures from other influences is always taken.<\/li>
    • Level 5<\/strong>\u00a0Return on Investment\u2014Impact measures are converted to monetary values and compared with the fully loaded program costs. You can have improvement in productivity, for example, but you must determine the monetary value of that improvement and what that improvement cost you in order to calculate ROI. If the monetary value of productivity’s improvement exceeds the cost, your calculation results in a positive ROI.<\/li><\/ul>

      Each level of evaluation answers basic questions regarding the program success.<\/p>

      Level of Evaluation<\/strong><\/p><\/td>	Key Questions<\/strong><\/p><\/td><\/tr>
      Level 1:<\/strong><\/p> Reaction,<\/strong><\/p> Satisfaction, and Planned<\/strong><\/p> Action<\/strong><\/p><\/td>	Was the program relevant to participants\u2019 jobs and mission?<\/li> Was the program important to participants\u2019 jobs and mission success?<\/li> Did the program provide new information?<\/li> Do participants intend to use what they learned?<\/li> Would participants recommend the program to others?<\/li> Is there room for improvement with facilitation, materials, and the learning environment?<\/li><\/ul><\/td><\/tr>
      Level 2:<\/strong><\/p> Learning<\/strong><\/p><\/td>	Did participants acquire the knowledge and skills presented in the program?<\/li> Do participants know how to apply what they learned?<\/li> Are participants confident to apply what they learned?<\/li><\/ul><\/td><\/tr>
      Level 3:<\/strong><\/p> Application and<\/strong><\/p> Implementation<\/strong><\/p><\/td>	How effective are participants at applying what they learned?<\/li> How frequently are participants applying what they learned?<\/li> If participants are applying what they learned, what is supporting them?<\/li> If participants are not applying what they learned, why not?<\/li><\/ul><\/td><\/tr>
      Level 4:<\/strong><\/p> Business<\/strong><\/p> Impact<\/strong><\/p><\/td>	So what if the application is successful?<\/li> To what extent did application of learning improve the measures the program was intended to improve?<\/li> How did the program affect output, quality, cost, time, customer satisfaction, employee satisfaction, and other measures?<\/li> How do you know it was the program that improved these measures?<\/li><\/ul><\/td><\/tr>
      Level 5: ROI<\/strong><\/p><\/td>	Do the monetary benefits of the improvement in business impact measures outweigh the cost of the program?<\/li><\/ul><\/td><\/tr><\/tbody><\/table> The reason for referring to evaluation data as levels is that it facilitates managing and reporting the data. More important, however, these five levels present data in a way that makes it easy for the audience to understand the results reported. Each level of evaluation provides important, stand-alone data. Reported together, the five-level ROI framework represents data that tells the complete story of program success or failure.<\/p> Reference: Return of Investment Basics, Patricia Pulliam Phillips, Jack J. Phillips, ASTD Press, 2005\u00a0ROI Institute<\/a><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":" The complete story of a training program success or failure Patricia and Jack Phillips, from ROI Institute, included return on investment (ROI) as the fifth level of the traditional Kirkpatrick evaluation levels. According to this new ROI methodology for training…<\/p>\n","protected":false},"author":6,"featured_media":8359,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[51],"tags":[],"class_list":["post-8358","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-return-on-investment"],"_links":{"self":[{"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/posts\/8358","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/comments?post=8358"}],"version-history":[{"count":0,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/posts\/8358\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/media\/8359"}],"wp:attachment":[{"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/media?parent=8358"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/categories?post=8358"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/tags?post=8358"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}