Two new methods of assessment provided by the consulting firm were chosen for experimentation. The first is a paper-and-pencil clerical test assessing clerical speed and accuracy. It contains 50 items and has a 30-minute time limit. The second is a brief work sample that could be administered as part of the interview process. In the work sample, the applicant must respond to four different phone calls: a customer who is irate about an out-of-stock item, a customer who wants more product information about an item than was provided in the catalog, a customer who wants to change an order placed yesterday, and a customer who has a routine order to place. Using a 1-5 rating scale, the interviewer rates the applicant on tactfulness (T) and concern for customers (C). The interviewer is provided with a rating manual containing examples of exceptional (5), average (3) and unacceptable ( 1) responses by the applicant. 

A random sample of 50 current CSRs were chosen to participate in the study. At Time I they were administered the clerical test and the work sample; performance data were also gathered from company records for error rate (number of errors per 100 orders) speed (number of orders filled per hour), and customer complaints (number of complaints per week). At Time 2, one week later, the clerical test and the work sample were re-administered to the CSRs. A member of the consulting firm sat in on all the interviews and served as a second rater of performance on the work sample at Time I and Time 2. It is expected that the clerical test and work sample will have positive correlations with speed and negative correlations with error rate and customer complaints.

After reading the description of the study and observing the results above,

1. How do you interpret the reliability for the clerical test and work sample? Are they favorable enough for Phonemin to consider using the "for keeps" in selecting new job applicants?
2. How do you interpret the validity results for the clerical test and work sample? Are they favorable enough for Phonemin to consider using them "for keeps" in selecting new job applicants?
3. What limitations in the above study should be kept in mind when interpreting the results and deciding whether to use the clerical test and work sample? 

Results for Clerical Test Mean score Standard deviation Coefficient alpha Test-retest r r with error rate r with speed r with complaints r with work sample (T) r with work sample (C) Results for Work Sample (T) Mean score Standard deviation % agreement (raters) r with work sample (C) r with error rate r with speed r with complaints Results for Work Sample (C) Mean score Standard deviation Time 1 31.61 4.70 .85 -.31* .41* -.11 .21 .07 Time 1 3.15 .93 SS% .81* -.13 .11 -.37* Time 1 2.91 .99 80% .81* % agreement (raters) r with work sample (T) r with error rate r with speed r with complaints (Note: * means that r was significant at p < .05) -.04 .15 -.40* Time 2 31.22 5.11 .86 .92** -.37" .39* -.08 .17 .15 Time 2 3.11 1.01 79% .77* -.12 .15 -.35" Time 2 3.07 1.10 82% .77* -.11 .14 -.31*