\n\t\t\t\t\t\t\t\t\t

By Jack J. Phillips, Ph.D.<\/p>

Learning and Development professionals often must evaluate their key learning programs, collecting several types of\u00a0data\u2014reaction, learning, application, impact, intangibles and maybe even return on investments.<\/p>

What if the evaluation produces disappointing results? Suppose application and impact were less\u00a0than desired, and the ROI calculation negative. This prospect causes some learning executives to\u00a0steer clear of this level of accountability altogether.<\/p>

For some L&D professionals, negative results are the ultimate fear. Immediately, they begin to think, “Will this\u00a0reflect unfavorably on me? On the program? On the function? Will budgets disappear? Will support\u00a0diminish?” These are all legitimate questions, but most of these fears are unfounded. In fact, negative\u00a0results reveal the potential to improve programs. Here are 11 ways to address negative results and\u00a0use them to facilitate positive transformations:<\/p>

1. Recognize the Power of a Negative Study<\/strong>
When the study results are negative, there is always an abundance of data indicating what went\u00a0wrong. Was it an adverse reaction? Was there a lack of learning? Was there a failure to implement or\u00a0apply what was learned? Did major barriers prevent success? Or was there a misalignment in the\u00a0beginning? These are legitimate questions about lack of success, and the answers are always\u00a0obtained in a comprehensive evaluation study.<\/p>

2. Look for Red Flags<\/strong>
Indications of problems often pop up in the first stages of initiation\u2014after reaction and learning data\u00a0have been collected. Many signals can provide insight into the program’s success or lack of success,\u00a0such as participants perceiving that the program is not relevant to their jobs. Perhaps they would not\u00a0recommend it to others or do not intend to use it on the job. These responses can indicate a lack of\u00a0utilization, which usually translates into negative results. Connecting this information requires\u00a0analyzing data beyond overall satisfaction with the program, the instructor and the learning\u00a0environment. While important, these types of ratings may not reveal the value of the content and its\u00a0potential use. Also, if an evaluation study is conducted on a program as it is being implemented, low\u00a0ratings for reaction and learning may signal the need for adjustments before any additional evaluation\u00a0is conducted.<\/p>

3. Lower Outcome Expectations<\/strong>
When there is a signal that the study may be negative, or it appears that there could be a danger of\u00a0less-than-desired success, the expectations of the outcome should be lowered. The “under-promise\u00a0and over-deliver” approach is best applied here. Containing your enthusiasm for the results early in\u00a0the process is important. This is not to suggest that a gloom-and-doom approach throughout the\u00a0study is appropriate, but that expectations should be managed and kept on the low side.<\/p>

4. Look for Data Everywhere<\/strong>
Evaluators are challenged to uncover all the data connected to the program\u2014both positive and\u00a0negative. To that end, it is critical to look everywhere for data that shows value (or the lack of it). This\u00a0thorough approach will ensure that nothing is left undiscovered\u2014the fear harbored by many\u00a0individuals when facing negative results.<\/p>

5. Never Alter the Standards<\/strong>
When the results are less than desired, it is tempting to lower the standards\u2014to change the\u00a0assumptions about collecting, processing, analyzing and reporting the data. This is not a time to\u00a0change the standards. Changing the standards to make the data more positive renders the study\u00a0virtually worthless. Without standards, there is no credibility.<\/p>

6. Remain Objective Throughout<\/strong>
Ideally, the evaluator should be completely objective or independent of the program. This objectivity\u00a0provides an arms-length evaluation of its success. It is important not only to enter the project from an\u00a0objective standpoint, but also to remain objective throughout the process. Never become an advocate\u00a0for or against it. This helps alleviate the concern that the results may be biased.<\/p>

7. Prepare the Team for the Bad News<\/strong>
As red flags pop up and expectations are lowered, it appears that a less-than-desired outcome will be\u00a0realized. It is best to prepare the team for this bad news early in the process. Part of the preparation\u00a0is to make sure that they don’t reveal or discuss the outcome of the program with others. Even when\u00a0early results are positive, it is best to keep the data confidential until all are collected. Also, when it\u00a0appears that the results are going to be negative, an early meeting will help develop a strategy to deal\u00a0with the outcome. This preparation may address how the data will be communicated, the actions\u00a0needed to improve the program and, of course, explanations as to what caused the lack of success.<\/p>

8. Consider Different Scenarios<\/strong>
Standards connected with the ROI methodology are conservative for a reason: The conservative\u00a0approach adds credibility. Consequently, there is a buy-in of the data and the results. However,\u00a0sometimes it may be helpful to examine what the result might be if the conservative standards were\u00a0not used. Other scenarios may actually show positive results. In this case, the standards are not\u00a0changed, but the presentation shows how different the data would be if other assumptions were\u00a0made. This approach allows the audience to see how conservative the standards are. For example,\u00a0on the cost side, including all costs sometimes drives the project to a negative ROI. If other\u00a0assumptions could be made about the costs, the value could be changed and a different ROI\u00a0calculation might be made. On the benefit side, lack of data from a particular group sometimes drives\u00a0a study into negative territory because of the “no data, no improvement” standard. However, another\u00a0assumption could be made about the missing data to calculate an alternative ROI. It is important for\u00a0these other scenarios to be offered to educate the audience about the value of what is obtained and\u00a0to underscore the conservative approach. It should be clear that the standards are not changed and\u00a0that the comparisons with other studies would be based on the standards in the original calculation.<\/p>

9. Find Out What Went Wrong<\/strong>
With disappointing results, the first question usually asked is, “What went wrong?” It is important to\u00a0uncover the reasons for the lack of success. As the process unfolds, there is often an abundance of\u00a0data to indicate what went wrong. The follow-up evaluation will contain specific questions about\u00a0impediments and inhibitors. In addition, asking for suggestions for improvements often underscores\u00a0how things could be changed to make a difference. Even when collecting enablers and enhancers,\u00a0there may be clues as to what could be changed to make it much better. In most situations, there is\u00a0little doubt as to what went wrong and what can be changed. In worst-case scenarios, if the program\u00a0cannot be modified or enhanced to add value, it may mean that it should be discontinued.<\/p>

10. Adjust the Story Line<\/strong>
When communicating data, negative results indicate that the story line needs to change. Instead of\u00a0saying, “Let’s celebrate\u2014we’ve got great results for this program,” the story reads, “Now we have\u00a0data that show how to make this program more successful.” The audience must understand that the\u00a0lack of success may have existed previously, but no data were available to know what needed to be\u00a0changed. Now, the data exist. In an odd sort of way, this becomes a positive spin on less-thanpositive\u00a0data.<\/p>

11. Drive Improvement<\/strong>
Evaluation data are virtually useless unless used to improve processes. In a negative study, there are\u00a0usually many items that could be changed to make it more successful. It is important that a\u00a0commitment is secured to make needed adjustments so that the program will be successful in the\u00a0future. Until those actions are approved and implemented, the work is not complete. In worst-case\u00a0scenarios, if the program cannot be changed to add value, it should be terminated and the important\u00a0lessons should be communicated to others. This last step underscores that the comprehensive\u00a0evaluation is used for process improvement and not for performance evaluation of the staff.<\/p>

Negative study results do not have to be bad news. Negative results contain data that can be used\u00a0not only to explain what happened, but also to adapt and improve in the future. It is important to\u00a0consider the potential of a negative study and adjust expectations and strategies throughout the\u00a0process to keep the negative results from being a surprise. In the worst-case situation, negative data\u00a0will surprise the key sponsor at the time of presentation.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

By Jack J. Phillips, Ph.D. Learning and Development professionals often must evaluate their key learning programs, collecting several types of\u00a0data\u2014reaction, learning, application, impact, intangibles and maybe even return on investments. What if the evaluation produces disappointing results? Suppose application and…<\/p>\n","protected":false},"author":6,"featured_media":8461,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[51],"tags":[],"class_list":["post-8460","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-return-on-investment"],"_links":{"self":[{"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/posts\/8460","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/comments?post=8460"}],"version-history":[{"count":0,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/posts\/8460\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/media\/8461"}],"wp:attachment":[{"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/media?parent=8460"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/categories?post=8460"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insources.com.au\/qms\/wp-json\/wp\/v2\/tags?post=8460"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}