INSOURCES BLOG

Star InactiveStar InactiveStar InactiveStar InactiveStar Inactive

pyramid61- Lack of Alignment with Business Needs. A training program's payoff comes from the business measures that drive it. Simply put, if a training program is not aligned or connected to a business measure, no improvement can be linked to the program. Too often, training is implemented for the wrong reasons – a trend, desire or perceived need that may not be connected to a business measure.

Initial training needs may be linked to the objectives and evaluation through the use of a consistent four-level concept. If we accept this evaluation framework, four corresponding levels of objectives and needs assessment exist as well. Without the business connection at Level 4, the program will have difficulty in credibly driving any business results

One major telecom firm in the USA faced this problem directly as they reviewed its corporate university's major programs. A first step to check for business alignment was to connect core courses to some business measure or need based on perceptions of the corporate university staff. When the staff could not readily make the connection, they determined the linkage did not exist. The company needed a more detailed up-front analysis.

2- Failure to Recognize Non-Training Solutions. If the wrong solution is implemented, little or no payoff will result.
Too often, training is perceived as a solution for a variety of performance problems when training may not be an issue at all. A recent evaluation of a leading bank's major training program illustrated this problem. In its training program, the bank attempted to prepare the commercial loan officers (relationship managers) to sell products other than commercial loans, such as the bank's capital market products and cash management services. But the training produced little change in the managers' behavior. An impact study subsequently revealed that the culprit was the compensation arrangement. When probed for a reason for the poor results, the bankers clearly indicated that unless their compensation system changed to account for the new product lines, their behavior would not change. They will continue to sell only the products on which their commissions were based.

Attempting to solve job performance issues with training will not work when factors such as reward systems, job design, and motivation are the real issues. To overcome this problem, training staffs must focus on methods to analyze performance rather than conduct traditional training needs assessments – a major shift in performance improvement that has been developing for many years.

Up-front analysis should be elevated from needs assessment, which is based on skills and knowledge deficiencies, to a process that begins with business needs and works through the learning needs.

3 - Lack of Specific Direction and Focus. Training and development should be a focused process that allows stakeholders to concentrate on desired results. Training and development objectives should be developed at higher Kirkpatrick levels than traditional learning objectives. These objectives correspond with six measures that lead to a balanced approach to evaluating training's success. Most training programs should contain objectives at multiple levels, ideally including those at levels 3 and 4.

When developed properly, these objectives provide important direction and focus for a variety of stakeholders at different time frames. For designers and developers, the objectives provide needed insight to focus on application and impact, not just learning. The facilitators need detailed objectives to prepare individuals for the learning experience's ultimate outcome: job performance change.

Participants need the direction provided by level 3 and 4 objectives to clearly see how the training program's outcome will actually help the organization. Sponsors of training and development, the key clients who pay for the program and support it, require such objectives to connect training with important business-unit measures. Finally, evaluators use this type of direction to know what data to collect to determine whether the program has been successful.

Recognizing the importance of multiple objective levels, including business impact, a vice president of corporate training and development at a major package delivery company recently posed an important question to the organization: "How can we expect our management team to support a program when we cannot define the behavior expected from participants and the subsequent business impact driven by the program?"

While not all programs should undergo such detailed up-front analysis, it is a critical issue that needs more attention.

By Jack and Patti Phillips

Write comment (0 Comments)

Star InactiveStar InactiveStar InactiveStar InactiveStar Inactive

methodologypuzzle

For some evaluators, the act of converting data to monetary value inspires fear, misconception, and bewilderment. But remember, all data can be converted to monetary value. You just need to know what techniques are available to you.

Use Standard Values
Many organizations have standard values to measure turnover, productivity, and quality. If a measure has a monetary value developed and accepted by the organization, there's no reason to reinvent it. Standard values are generally grouped into three categories: output to contribution, cost of quality, and employees' time.

When considering output to contribution, look at the value of an additional output. For example, let's say you work at a passport office and your entire role is to process passports. If you can process one more passport, given the resources and time you have available, the value of that one passport is equivalent to the co

st of processing one passport. This one additional output--the passport--times the cost of processing the passport is the monetary contribution of increasing the output to the organization.

Now consider the cost of quality, another standard value in most organizations. Waste, reject rates, and defects often have assigned monetary values. Other measures, such as re-work, can be converted to monetary value by looking at the cost of the work. For example, when employees make mistakes and errors in reporting, the monetary value of those mistakes is the cost incurred in re- working the report.

Employees' time is probably the simplest and most basic approach to data conversion. If time is saved due to a program, the first question to ask is, Whose time is it? Then to convert time to monetary value, take time saved multiplied by labor cost and add the percentage of additional value for employee benefits. (This benefits factor can easily be obtained from Human Resources.) A word of caution: When considering employee time as a gain, remember that the time savings is only realized when the amount of time saved is actually used for productive work.

Turn to Historical Costs
When no standard values exist, go to historical costs. The question to ask is, What has a similar incident cost in the past? An example of using historical costs is the case of a sexual harassment prevention program that was implemented in a large health care organization. The measure of the investigation was formal, internal complaints. The value of the complaint was determined by looking at its historical cost, including litigation, legal fees and expenses, settlement losses, as well as investigation and defense of the organization.

Look to Internal or External Experts
When standard values are unavailable and developing the monetary values through historical costs is not feasible, the next option is to go to internal or external experts. It's important for these experts to fully understand your intent and the business measure you are targeting.

Leverage External Databases
External databases can also provide a wealth of information, including the monetary value of an array of measures. An example of how to use external databases to convert a measure to monetary value is in the case of turnover.

Link with Other Measures
Another technique is to link the value of a measure with others that have already been converted to monetary values. This involves identifying existing relationships to show a correlation between the measure under investigation and another measure to which a standard value has been applied (as in the link between job satisfaction and turnover). Remember, the further you get from the actual monetary value, the lower the credibility of the information.

Use Estimations
Estimates of monetary value can come from participants, supervisors, managers, and even the WLP staff, and can be easily gathered through focus groups, interviews, or questionnaires. The key is to first clearly define the measure so that the people providing estimates have a clear understanding of what you're looking for, and then to determine the most credible data sources.

Consider the case of absenteeism. The table, below, shows supervisors' estimates of the per-day cost of one person not showing up for work, the confidence level in that estimate, and the adjusted per-day cost for one absence at $1,061.

Supervisor Est. Per day cost Confidence Adjusted per day cost
1 $1,000 70% $700
2 $1,500 65% $975
3 $2,300 50% $1,150
4 $2,000 60% $1,200
5 $1,600 80% $1,280
      $5,035
Average adjusted per day cost of one absence $1,061

Since estimates are subjective, we reduce the error by adjusting them with confidence levels. For example, if Supervisor One tells you it costs $1,000 per day for an unexpected absence, then present them with the other supervisors' estimates and ask how confident they are that their estimate is indeed correct. After thinking it over, they may say, "Well, I know what happens when people don't show up for work and I can be pretty sure what it's costing us from a time perspective. Given that it is an estimate and I'm not totally sure, I'll say that I am 70 percent confident in my number." Repeat the process with each Supervisor.

This additional step in the estimation process reduces variability and provides a more conservative value. You have reduced the amount of error and improved the reliability of the value of one absence.

Data Conversion Four-Part Test
For those times when you cannot decide whether you can credibly convert a measure to monetary value, complete this four-part test:

  • If the measure you want to convert has a standard value, then convert it to monetary value.
  • If there is not a standard value, is there a method other than standard values to get there? If there is not a method, then report the measure as intangible.
  • If there is a method to convert the measure, can you do so with minimum resources? If no, then report it as intangible.
  • If you can convert the measure to monetary value using the selected method with minimum resources, can you convince your executive in two minutes or less that the value is credible? If no, then report the measure as intangible. If yes, then convert it.

Five Steps to Data Conversion
Once you've decided to convert a measure to monetary value and have chosen the technique that you're going to use, there are five steps to complete the data conversion process:

  1. Focus on the unit of measure.
  2. Determine the value of each unit.
  3. Calculate the change in the performance of the measure.
  4. Determine the annual improvement in the measure.
  5. Calculate the total monetary value of the improvement.

Finally, remember intangible benefits are those that you choose not to convert to monetary value. Typical intangible benefits are job satisfaction, organizational commitment, teamwork, and customer satisfaction.

Considerations
While all measures can be converted to money, several factors should be considered. One factor is the cost to convert the measure. You don't want to spend more on data conversion than the evaluation itself. Importance of the measure is another consideration. Some measures, such as customer satisfaction and employee satisfaction, stand alone quite well. In that case, you might think twice before attempting to convert the measure to money. Also consider credibility. While most business decisions are made on somewhat subjective data, the source of the data, the perceived bias behind the data, and the motive in presenting the results are all concerns when data is potentially questionable.

Patti Phillips is president and CEO of the ROI Institute and co-author of Show Me the Money, published by Berrett-Koehler. The ROI Institute is a research, benchmarking, and consulting organization that provides workshops, publications, and consulting services on the ROI Methodology.

Write comment (0 Comments)

Star InactiveStar InactiveStar InactiveStar InactiveStar Inactive

ROI logo

Measuring ROI? In business coaching? Yes and yes. Isn't this just a fad? Isn't this impossible? No and no.

As more and more organisations use business coaching as a human resources, performance improvement, and leadership development approach, many executives question its value, particularly as coaching expenditures grow. Whether the engagement takes place in the context of an internal department for coaching or through arrangement with a business coaching firm, coaching assignments and commitments are planned and executed with good intentions. Unfortunately, however, not all coaching engagements produce the value desired by either the individual being coached (participant) or the sponsor who often pays for it. It will be increasingly important that business coaches measure a significant return on investment (ROI) and show the value of business coaching in terms that managers and executives understand.

It's Not a Fad . . .

Measuring ROI enjoys a history of nearly thirty years of application in a variety of human resource and performance improvement processes and across the full spectrum of industries and organisations. Thousands of trained practitioners implement an ROI process in their own settings and thousands of impact studies are generated annually worldwide. The methodology is the subject of many books in many languages.

It's Not Impossible . . .
Successfully measuring ROI for business coaching involves much more than simply assessing results achieved. The most effective ROI processes involve four phases: planning, data collection, data analysis, and reporting.

In the planning phase the coach, the person being coached, his or her manager, and the sponsor (client organization) agree on the evaluation plans and establish a baseline for expectations.

The data collection phase occurs in two time frames. Data is collected first during the coaching experience and then at the conclusion of the engagement or at an appropriate follow up time. The data collected include satisfaction and reaction, learning, application and implementation, business
impact, and ROI.

Evaluation Levels
Level Measurement Focus

1. Reaction &

Planned Action

Measures participant satisfaction with the coaching experience and captures planned actions.
2. Learning  Measures changes in knowledge, skills, and attitudes.

3. Application and

Implementation

Measures changes in on-the-job behavior and progress with application.
4. Business Impact  Captures changes in business impact measures.

5. Return on

Investment

Compares coaching engagement monetary benefits to the program costs.

The third phase in the ROI Methodology—data analysis— isolates the effects of the coaching on the business. The process includes converting data to monetary values using conservative figures (higher figures for costs, lower figures for benefits), capturing costs, calculating the return on investment, and identifying intangible measures and benefits.

Phase four—reporting—requires reaching conclusions, generating reports, and communicating the information to target groups. This new knowledge affords all involved—from the coach and the person being coached to upper level executives in the client organization— the ability to assess the value of the coaching engagement and the opportunity to make adjustments going forward.

Final Thoughts . . .
Developing the ROI in business coaching is not a fad, and it's not impossible. Measuring ROI in business coaching is and will increasingly become an imperative for organisations and coaching firms pursuing the highest standards of accountability.

By Jack J. Phillips, Ph.D. and Patricia Pulliam Phillips, Ph.D.

Write comment (0 Comments)

Disclaimer
Privacy Policy
Terms of Sale
Terms of Use

  • Email: info@insources.com.au
  • Phone: 1300 208 774
  • ABN 74 625 075 041 

SUBSCRIBE TO OUR NEWSLETTER

© 2019 by Insources Group Pty Ltd. All rights reserved.

Search