Program objectives are the fundamental basis for evaluation.
Program objectives drive the design and development of the program and define how to measure success. Program objectives define what the program is intended to do and how to measure participant achievement and system support of the learning transfer process. All too often, however, minimal emphasis is placed on developing objectives and their defined measures.
Defining Program Objectives
Before the evaluation begins, the program objectives must be developed. Program objectives are linked to the needs assessment. When a problem is identified, the needs assessment process begins. Assessments are conducted to determine exactly what the problem is; how on-the-job performance change can resolve the problem; what knowledge or skills need to be acquired to change on-the-job performance; and how best to present the solution so that those involved, the consumers, can acquire the knowledge and skills to change performance to solve the business problem. From here, program objectives are developed to help guide program designers and developers, provide guidance to facilitators, provide goals for participants, and provide a framework for evaluators.
Program objectives reflect the same five-level framework used in categorizing evaluation data. The key in writing program objectives is to be specific in identifying measures of success. All too often, very broad program objectives are written.
While this is acceptable in the initial phases of program design, it is the specific measures of success that drive results and serve as the basis for the evaluation.
Level 1 Reaction, Satisfaction, and Planned Action Objectives
Level 1 objectives are critical in that they describe expected immediate and long term satisfaction with a program. They describe issues that are important to the success of the program, including facilitation, relevance and importance of content, logistics, and intended use of knowledge and skills. But, there has been criticism of the Level 1 evaluation. This criticism surrounds the use of the Level 1 overall satisfaction as a measure of success. The overuse of the overall satisfaction measure has led many organizations to make funding decisions based on whether participants like a program, later realizing the data was misleading.
Level 1 objectives should identify issues that are important and measurable rather than esoteric indicators that provide limited useful information. They should be attitude based, clearly worded, and specific. Level 1 objectives specify that the participant has changed in thinking or perception as a result of the program and underscore the linkage between attitude and the success of the program. While Level 1 objectives represent a satisfaction index from the consumer perspective, these objectives should also have the capability to predict program success. Given these criteria, it is important that
Level 1 objectives are represented by specific measures of success.
A good predictor of the application of knowledge and skills is the perceived relevance by participants of program content. So, a Level 1 objective may be
At the end of the course, participants will perceive program content as relevant to their jobs.
A question remains, however: “How will you know you are successful with this objective?” This is where a good measure comes in. The Table below compares the broad objective with the more specific measure.
| Objective | Measure |
| At the end of the course, participants will perceive program content as relevant to their jobs. | 80% of participants rate program relevance a 4.5 out of 5 on a Likert scale. |
Now, for those of you who are more research driven, you might want to take this a step further by defining (literally) what you mean by “relevance.” Relevance may be defined as:
- knowledge and skills that participants can immediately apply in their work
- knowledge and skills reflective of participants’ day-to-day work activity.
If this is the case, the measures of success are even more detailed.
Objective:
- At the end of the course, participants will perceive program content as relevant to their jobs.
Measures:
- 80% of participants indicate that they can immediately apply the knowledge and skills in their work as indicated by rating this measure a 4.5 out of 5 on a Likert scale.
- 80% of participants view the knowledge and skills as reflective of their day-to-day work activity as indicated by rating this measure 4.5 out of 5 on a Likert scale.
Success with these two measures can be reported individually, or you can combine the results of the two measures to create a “relevance index.”
Breaking down objectives to specific measures provides a clearer picture of success.
Continue in Program Objectives Part 2
Reference: ROI Basics, Patricia Phillips and Jack Phillips, ASTD Press, 2005
