Discuss the training evaluation process.

Chapter 6 Training Evaluation 251 OF THE EVALUATION PROCESS NERVIEW OF THE Before the chapter expla understand the evaluation cussion of formative and scrutinizing the program bot chanter explains each aspect of training evaluation in detail, you need to the evaluation process, which is summarized in Figure 6.1. The previous dis- of formative and summative evaluation suggests that training evaluation involves izing the program both before and after the program is completed. Figure 6.1 sizes that training evaluation must be considered by managers and trainers before mining has actually occurred. As was suggested earlier in this chapter, information gained from the training design process shown in Figure 6.1 is valuable for training evaluation The evaluation process should begin with determining training needs (as discussed in Chapter Three, "Needs Assessment"). Needs assessment helps identify what knowledge, skills, behavior, or other learned capabilities are needed. Needs assessment also helps identify where the training is expected to have an impact. Needs assessment helps focus the evaluation by identifying the purpose of the program, the resources needed (human, financial, and company), and the outcomes that will provide evidence that the program has been effective. The next step in the process is to identify specific, measurable train ing objectives to guide the program. The characteristics of good objectives are discussed in Chapter Four, "Learning and Transfer of Training. The more specific and measur able these objectives are, the easier it is to identify relevant outcomes for the evalua- tion. Besides considering the learning and program objectives in developing learning outcomes, it is also important to consider the expectations of those individuals who sup- port the program and have an interest in it (stakeholders such as trainees, managers, and trainers). If the needs assessment was done well, the stakeholders' interests likely overlap considerably with the learning and program objectives. Analysis of the work environment to determine transfer of training (discussed in Chapter Five. "Program Design") can be useful for determinine how trainine content will be used on the job. Based on the learning objectives and analysis of transfer of training, outcome measures are designed to assess the extent to which learning and transfer have occurred. FIGURE 6.1 The Evaluation Process Conduct a Needs Analysis DAG CO Develop Measurable Learning Objectives and Analyze Transfer of Training Pro Deloping Human Develop Outcome Measures HAH Choose an Evaluation Strategy 1 Alain 5-11-5-230K D.ML W. per Pian and Execute the Evaluation Paining for Training Re Man (Wisie 2008): TABLE Designing Training Outca Criteri Reactic Once the outcomes have been identified, the next step is to determine an evaluation strategy. Factors such as expertise, how quickly the information is needed, change poten- tial, and the organizational culture should be considered in choosing a design. Planning and executing the evaluation involves previewing the program (formative evaluation), as well as collecting training outcomes based on the evaluation design. The results of the evalua- tion are used to modify, market, or gain additional support for the program. The results of the evaluation should also be used to encourage all stakeholders in the training process including managers, employees, and trainers--to design or choose training that helps the company meet its business strategy and helps managers and employees meet their goals. 10 Learning cognitiv COMES USED IN THE EVALUATION OF TRAINING PROGRAMS Behavior skill-base Affective Pesults To evaluate its training program, a company must decide how it will determine the program's effectiveness: that is, it must identify what training outcomes or criteria it will measure Table 6.1 shows the six categories of training outcomesreaction outcomes, learning or cognitive outcomes, behavior and skill-based outcomes, affective outcomes, results, and return on investment." Table 6.1 shows training outcomes, the level they correspond to in Kirkpatrick's evaluation model, a description of each of the outcomes and how they are measured, and the question that each outcome can help answer. Kirkpatrick's original evaluation model included only four levels (reaction, learning, behavior, and results) but recent thinking suggests a fifth level, return on investment (ROD. is necessary to demonstrate the financial value of training. Both level 1 and level 2 outcomes (reactions and leam- ing) are collected at the completion of training, before trainees return to the job. Level 3 outcomes (behavior/skills) can also be collected at the completion of training to deter mine trainees' behavior or skill level at that point. To determine whether trainees are using training content back on the job (ie., whether transfer of training has occurred), level 3, level 4, and/or level 5 outcomes can be collected. Level 3 criteria can be col- lected to determine whether behavior/skills are being used on the job. Level 4 and level 5 criteria (results and return on investment) can also be used to determine whether training has resulted in an improvement in business results, such as productivity or customer satisfaction. These criteria also help determine whether the benefits of training exceed their costs. Keep in mind that the levels do not indicate the importance of the outcomes of that lower-level outcomes cause higher-level outcomes. That is reactions cause leaming which in turn influences skills and results. The outcomes that are collected in evala are based on training needs, program objectives, and the strategic reasons for training We discuss this in the section "Evaluation Practices," later in the chapter. Reaction Outcomes Reaction outcomes refer to trainees' perceptions of the program, including the fact trainers, and content. (Reaction outcomes are often referred to as a measure of comfort.") They are often called class or instructor evaluation. This information cally collected at the program's conclusion. You probably have been asked to complete or instructor evaluations either at the end of a college course or a training programa Reactions are useful for identifying what trainees thought was successful or what in leaming. Reaction outcomes are level 1 (reaction) criteria in Kirkpatrick's framewo Peturn on westent See Honde & program at work Chapter 6 Training Evaluation 251 OF THE EVALUATION PROCESS NERVIEW OF THE Before the chapter expla understand the evaluation cussion of formative and scrutinizing the program bot chanter explains each aspect of training evaluation in detail, you need to the evaluation process, which is summarized in Figure 6.1. The previous dis- of formative and summative evaluation suggests that training evaluation involves izing the program both before and after the program is completed. Figure 6.1 sizes that training evaluation must be considered by managers and trainers before mining has actually occurred. As was suggested earlier in this chapter, information gained from the training design process shown in Figure 6.1 is valuable for training evaluation The evaluation process should begin with determining training needs (as discussed in Chapter Three, "Needs Assessment"). Needs assessment helps identify what knowledge, skills, behavior, or other learned capabilities are needed. Needs assessment also helps identify where the training is expected to have an impact. Needs assessment helps focus the evaluation by identifying the purpose of the program, the resources needed (human, financial, and company), and the outcomes that will provide evidence that the program has been effective. The next step in the process is to identify specific, measurable train ing objectives to guide the program. The characteristics of good objectives are discussed in Chapter Four, "Learning and Transfer of Training. The more specific and measur able these objectives are, the easier it is to identify relevant outcomes for the evalua- tion. Besides considering the learning and program objectives in developing learning outcomes, it is also important to consider the expectations of those individuals who sup- port the program and have an interest in it (stakeholders such as trainees, managers, and trainers). If the needs assessment was done well, the stakeholders' interests likely overlap considerably with the learning and program objectives. Analysis of the work environment to determine transfer of training (discussed in Chapter Five. "Program Design") can be useful for determinine how trainine content will be used on the job. Based on the learning objectives and analysis of transfer of training, outcome measures are designed to assess the extent to which learning and transfer have occurred. FIGURE 6.1 The Evaluation Process Conduct a Needs Analysis DAG CO Develop Measurable Learning Objectives and Analyze Transfer of Training Pro Deloping Human Develop Outcome Measures HAH Choose an Evaluation Strategy 1 Alain 5-11-5-230K D.ML W. per Pian and Execute the Evaluation Paining for Training Re Man (Wisie 2008): TABLE Designing Training Outca Criteri Reactic Once the outcomes have been identified, the next step is to determine an evaluation strategy. Factors such as expertise, how quickly the information is needed, change poten- tial, and the organizational culture should be considered in choosing a design. Planning and executing the evaluation involves previewing the program (formative evaluation), as well as collecting training outcomes based on the evaluation design. The results of the evalua- tion are used to modify, market, or gain additional support for the program. The results of the evaluation should also be used to encourage all stakeholders in the training process including managers, employees, and trainers--to design or choose training that helps the company meet its business strategy and helps managers and employees meet their goals. 10 Learning cognitiv COMES USED IN THE EVALUATION OF TRAINING PROGRAMS Behavior skill-base Affective Pesults To evaluate its training program, a company must decide how it will determine the program's effectiveness: that is, it must identify what training outcomes or criteria it will measure Table 6.1 shows the six categories of training outcomesreaction outcomes, learning or cognitive outcomes, behavior and skill-based outcomes, affective outcomes, results, and return on investment." Table 6.1 shows training outcomes, the level they correspond to in Kirkpatrick's evaluation model, a description of each of the outcomes and how they are measured, and the question that each outcome can help answer. Kirkpatrick's original evaluation model included only four levels (reaction, learning, behavior, and results) but recent thinking suggests a fifth level, return on investment (ROD. is necessary to demonstrate the financial value of training. Both level 1 and level 2 outcomes (reactions and leam- ing) are collected at the completion of training, before trainees return to the job. Level 3 outcomes (behavior/skills) can also be collected at the completion of training to deter mine trainees' behavior or skill level at that point. To determine whether trainees are using training content back on the job (ie., whether transfer of training has occurred), level 3, level 4, and/or level 5 outcomes can be collected. Level 3 criteria can be col- lected to determine whether behavior/skills are being used on the job. Level 4 and level 5 criteria (results and return on investment) can also be used to determine whether training has resulted in an improvement in business results, such as productivity or customer satisfaction. These criteria also help determine whether the benefits of training exceed their costs. Keep in mind that the levels do not indicate the importance of the outcomes of that lower-level outcomes cause higher-level outcomes. That is reactions cause leaming which in turn influences skills and results. The outcomes that are collected in evala are based on training needs, program objectives, and the strategic reasons for training We discuss this in the section "Evaluation Practices," later in the chapter. Reaction Outcomes Reaction outcomes refer to trainees' perceptions of the program, including the fact trainers, and content. (Reaction outcomes are often referred to as a measure of comfort.") They are often called class or instructor evaluation. This information cally collected at the program's conclusion. You probably have been asked to complete or instructor evaluations either at the end of a college course or a training programa Reactions are useful for identifying what trainees thought was successful or what in leaming. Reaction outcomes are level 1 (reaction) criteria in Kirkpatrick's framewo Peturn on westent See Honde & program at work