The professor or some other “moderator” will summarize the initial set of ratings for the class. You will then be provided with the results and be allowed to make changes to your initial ratings based on your review of the summary results. However, you need not make any changes if you believe your initial ratings are still an accurate assessment of your evaluation of each of the items. The moderator will then compute a second set of summary statistics based on the second set of ratings. 

1. Did you find the summary ratings helpful to you as you reviewed your initial set of ratings? 

2. If you changed some of your ratings, why did you make changes? 

3. Did you notice any patterns in your ratings compared to the summary ratings? For example, did you tend to be more stringent or lenient in your ratings than the other raters? 

4. Do you feel that frame-of-reference training would have helped you provide more “accurate” ratings? Why or why not? 

5. How do the two sets of summary ratings compare?