Human Resource Development

Kinds of Evaluations

Though we often think of evaluation taking place at the end of some experience or program, evaluations can and should occur at various times throughout the life cycle of a program or an organization. The following

section describes how evaluation may provide different kinds of information depending on when it is implemented.

Developmental Evaluation

In the mid-1990s, Patton (1994) introduced the term developmental evaluation to "describe certain long-term, partnering relationships with clients who are, themselves, engaged in ongoing program development" (p. 312). He developed this construct to call attention to the fact that evaluators were increasingly being asked to provide evaluative data during a program’s design process and to be a part of the design team. The developmental evaluator sits side by side with the designers, providing feedback that can be used to increase the likelihood that the program will be successful when implemented. Patton emphasizes that developmental evaluation is not a model; rather, it is a "relationship founded on a shared purpose: development" (1994, p. 313). Questions a developmental evaluation might address are:

• What are the right set of activities and strategies for this program?

• What is the best design?

• What are the appropriate goals and objectives?

• What and whose needs is this program addressing?

• What processes should be in place to make implementation effective?

• What criteria or standards are being used to design this product?

Developmental evaluation has some similarities with the practices of beta testing, pilot testing, or other kinds of program design processes that involve collecting data/information that is fed back into the design process.

Formative Evaluation

Scriven was the first to coin the terms formative and summative as ways of describing evaluation’s main purposes or functions. He explains that formative evaluation is typically conducted for the purposes of program or product improvement by in-house staff. However, many formative evaluations are also conducted by external evaluators. The findings from formative

evaluations are fed into an improvement-focused process that further develops, refines, or revises the object being evaluated. The reports that result from a formative evaluation typically remain internal to the organization. Examples of questions that formative evaluation might address include:

• How well is the program being implemented?

• What are the barriers to implementation?

• How effective are the program’s strategies and activities?

• How might the product be improved to appeal to a larger audience?

• To what extent is the staff prepared to implement the program’s objectives?

• How might the process be refined to make it more user-friendly?

• What aspects of the service are working well?

Summative Evaluation

Summative evaluation is implemented for the purpose of determining the merit, worth, or value of the evaluand in a way that leads to a final evaluative judgment. It "is conducted after completion of the program (for ongoing programs, that means after stabilization) and for the benefit of some external audience or decision maker (for example, funding agency, oversight office, historian, or future possible users)" (Scriven 1991, p. 340). Examples of decisions made based on summative evaluations include the granting of continued funding for the program, the expansion of the program to other sites, the elimination of a program that has outlived its usefulness or is not meeting the current needs of participants, grades in a course, or a decision of which vendor to use. Summative evaluations are often conducted by external evaluators (though not always). Stake (quoted in Scriven 1991) is credited with helping us understand the difference between formative and summative evaluations with the following analogy: "When the cook tastes the soup, that’s formative; when the guests taste the soup, that’s summative" (p. 19). Questions a summative evaluation might address include:

• To what extent did the program meet its goals?

• What were the learning outcomes?

• Were the results worth the project’s costs?

• What components of the program are reproducible in other locations?

• In what ways did participants benefit from the program?

• To what extent is the product viable?

• To what extent has the process improved employee productivity?

Summative evaluations come in a variety of shapes and sizes. The following are four common kinds.

Answer rating: 100% (QA)

Determining the right set of activities and strategies the best design and the appropriate goals and objectives for a program requires careful consideration of various factors including the programs t... View the full answer