Mario Rinaldi, director of the Developmental Disabilities Training Project, called an emergency meeting of the project staff.

Fantastic news! We've Found the answer you've been seeking!

Question:

Mario Rinaldi, director of the Developmental Disabilities Training Project, called an emergency meeting of the project staff. “You’ve all been working really hard on the training manual,” he said. “I hate to pull this on you right now, but I’ve got to tell you that I just had word. The feds are sending in their evaluator. He’s going to be here next week, and we’ve got to be ready.”

Amid the groans, Jane Carline, a staff trainee, spoke up. “What’s the big problem?” she asked. “We’ve been providing a training session every week. We’ve had workshops on developmental disabilities for the teachers, for citizen; we’ve had the TV show, now we’ve got a manual for parents. It seems to me we’re in great shape. So what’s the problem?”

“Well, I was kind of putting this part off,” Rinaldi answered. “They sent me along the new evaluation form so that we can complete the self-study before the evaluator gets here. That’s the tough part. We’re going to really have to dig to get the information ready.”

“What kind of information do we need?” Carline asked. “Remember, we did that pretest and posttest with all the people at the workshops. We’ve got a lot of data on the learning effects from the workshop. Of course, it’s not that easy to do with the TV program.”

“That’s the least of our problems, Jane. Remember, this is a training project. What they want is information on all participants in any training workshop. They want the ages of trainees, their sex, their, their employment, their income-all the demographic stuff. I just didn’t think about all that because the form we used last year didn’t ask for it. You see, we’re using the same kind of evaluation form as continuing education programs use. It doesn’t make any sense, but I’ll bet we can dig up that information somehow.”

“Wait a minute!” George Steinberg called out. “I worked my tail off on that TV program, and there’s no way in the world I’ll ever be able to even guess who watched it. Does that mean the whole thing didn’t happen? Does that mean I get a grade of zero? I flunk?”

“Now George, you know it isn’t like that. They just like to have the information so they can put it together with the data from all the other projects they funded. They’ve got to show results, just as we do.”

“That’s fine to say, John, but if they’re going to evaluate us on how many men, women, and children show up at our sessions, we should do the kind of stuff that can lend itself to what they’re looking for. I’m feeling as if my work just isn’t going to make the grade.”

“I’m starting to feel that way, too,” Carlin said. “What’s the point of doing one thing if they’re evaluation something else?”

“Wait a minute everybody,” Rinaldi responded in frustration. “We’re getting way off track here. These people aren’t here to tell us what we should and shouldn’t do. They’re not going to grade us on what we did. One of the things they ask us about in an open-ended question is the content of the program, the kinds of interventions we did. There’s no problem there. The problem is just in putting together the data they want, the demographic characteristics and all that. Now, I’ve got most of it somewhere. All I need is for somebody to volunteer to help me dig through the files and see what we can find that might relate to some of the questions they’re asking. We’ll be able to get the sex of participants by looking at their names on those address cards we had them fill out. The ages will be hard. That we’ll have to guess at.”

Reflection Questions:

Assume that you are going to help Mario develop an integrated management information system for use in future evaluation work of the training workshops for parents:

What information about the parents (inputs) would you recommend collecting from each workshop participant?
What information about the workshop activities (throughputs) would you recommend collecting for each workshop?
What information about workshop-related outcomes / demonstrable changes in parents (outputs) would you recommend collecting about each parent and what one or two measurement approaches might you use to collect this outcome data?
What two or three other information gathering and workshop evaluation steps would you recommend for integration into ongoing service delivery processes so that the director and staff can feel confident about avoiding a future evaluation emergency.