Hello.

Don't settle for a generic evaluation form. Instead, customize your smile sheets so that they elicit targeted feedback on your performance.

The workshop has come to an end, you have packed up your things, and you're looking forward to a nice meal. You stop to take a quick look at the feedback forms and see high numbers. A success! Or was it? Smile sheets or "happy surveys," common monikers for the evaluation forms used at the end of a training session, typically assess the overall satisfaction that participants felt with the class. Survey items include the ever-popular "The presenter was knowledgeable about the content," "The workshop met my needs" and "I would attend another class with this presenter."

Participants are asked to quickly circle a few numbers on a scale of 1 to 5 and be on their way. Results are tabulated and numbers are analyzed--but what do they really tell us?

Factors influencing survey results

Several years ago, I was scheduled to facilitate a spring workshop for a group of 35 teachers in rural Nebraska. One week prior to the event, the superintendent informed me that the school would be closing at the end of the year and all of the teachers had been given layoff notices. "But we still need to provide training, so let's just go forward as if nothing has changed" he added.

As you might guess, the happy surveys from that workshop showed that the participants were not all that happy.

Compare that with a training session I led more recently, with 100 participants who had voluntarily registered for it. When they arrived, there was a bountiful breakfast buffet, free books, and goodie bags for all.

The training content was practical, relevant, and filled with fun activities, and the supervisor announced that the participants would be allowed to leave after a working lunch. On the survey, participants quickly circled all 5s in the hopes that future training sessions would be just like this one.

As the two examples show, the scores from these surveys don't always reflect just the trainer's performance. Often, they are dramatically influenced by factors beyond the trainer's control.

Limited value

Thomas Guskey, in his book Evaluating Professional Development, makes a strong case for the limited value of post-training surveys. Although they procure immediate feedback and the results are simple to tally, the data usually are not helpful. They often don't lead to improved instruction.

If your goal is to ensure what I call "caffeinated learning"--the kind of learning experience where participants are alert and engaged, and real learning occurs--then the surveys must be tailored to elicit better feedback on your instructional design and delivery skills. You want participants to give actionable insights to your unique areas for improvement.

Meaningful questions

The first step is to design evaluation forms that prompt participants to think more carefully about their learning experience. Consider the following open-ended questions:

* What did you learn today that you are most likely to try on the job?

* Was there anything discussed today that you still do not understand?

* How will what you learned affect your work?

The answers to open-ended questions are harder to quantify, but are much more valuable in improving program design and delivery. A thoughtful presenter will be able to mine these answers for nuggets of information and use them to improve her instructional practices.

Customized evaluation

Rather than a generic form that is used by all the trainers in your organization, class after class, year after year, consider the insightful feedback that could be captured when the forms are designed around the specific needs of each trainer.

The San Diego County Office of Education provides professional development to the employees of 42 school districts and five community colleges. Last year, under the new leadership of Barbara Boone, director of instruction and curriculum, the process associated with evaluation surveys underwent a seismic shift. Boone worked with all the professional developers to write goals for their individual improvement.

The surveys for each person's classes were then customized based on her individual goals. For example, one of Boone's veteran developers wanted to increase participants' active engagement in the class. The developer added a statement to her customized evaluation form that read: This class was structured in a way that kept me actively involved in my own learning.

Boone then sits with the professional developers on a monthly basis to review the evaluation results as they relate to their performance improvement goals. The outcomes have been successful.

"Research shows that specific feedback has a positive impact on growth toward goals," Boone says. "By custom-designing some of the evaluation questions, we are able to support our employees to grow as professional developers. They feel that the feedback is more meaningful and valuable to them as individuals."

Boone and her team began with the following list of evaluative statements, and added to them as they developed individual goals for each employee.

* The structure of the activities stimulated my learning.

* The activities in this class gave me sufficient opportunities for practice and feedback.

* The difficulty level of this training program was appropriate.

* The pace of this class was appropriate.

* There was ample time for questions.

* The class included ample opportunities for active involvement.

* Examples provided were practical and realistic.

* An appropriate amount of material was covered in the time provided.

* The presenter appropriately tapped into the talents and knowledge of the participants.

* Technology was integrated in a way that enhanced the learning experience.

* The presenter was skilled at facilitating adult learning.

* The presenter's voice could be heard clearly.

* The visual aids were clear and effective.

* The handouts and other resources were useful.

Improvement

Independent consultants can easily adapt this process for their own benefit. Although they may not have a supervisor demanding that they meet their annual goals for improvement, independent consultants still want to improve.

When I develop a professional learning activity for a client, we discuss the evaluation process upfront. Together we determine the questions that will provide us with needed data.

Occasionally, I ask to insert a question that is aimed at improving my own skills as a facilitator. This is easiest when the consultant-client relationship is well established.

For example, my personal improvement goal for the year is to honor the needs of introverts in my classes. This is my second year consulting with one of my clients, so I felt comfortable asking to include the following statement in the evaluation survey: The presenter provided adequate time for quiet, personal reflection on the learning material.

By collecting and analyzing responses to this question, I have been able to explore participants' perspectives on this type of learning activity and improve my ability to address the needs of introverts. In turn, this serves the best interests of my clients.

At the end of the day, we all want to feel that we have had the opportunity to learn and grow our talents. By designing evaluation surveys that are customized and meaningful, we can procure data that informs and supports our continual improvement.

1- What are some common problems with training evaluations as they are most often conducted at the end of the training?

2- Describe at least two of the suggested improvements to training evaluation presented in this article.

3-For you personally, what is the most important point made in this article concerning effective training evaluation?

Thanks.