Application 8 Evaluating & Choosing Predictors

Currently, PIP uses the unstructured interview and an unstructured work sample to choose candidates that they think will ultimately perform well, contribute to guest satisfaction, and come to work when scheduled.

Assume that you are interested in how well the selection tools/predictors predict (1) job performance, (2) customer/guest satisfaction, (3) absenteeism, and (4) turnover. First, you conduct a reliability analysis of the current predictors and find the following reliability values:

	Test-Retest	Internal Consistency	Inter-rater
Current Unstructured Work Sample	.92	n/a	n/a
Unstructured Interview	n/a	n/a	55%

Next, you conduct a criterion-related validity study to see the extent to which each current selection tool predicts the outcomes important to the organization. You find the following validity coefficients:

	Job Performance	Guest Satisfaction	Absenteeism	Turnover
Current Unstructured Work Sample	.28, p<.05	.21, p<.10	-.12, p=.20	-.15, p=.15
Unstructured Interview	.07, p=.45	.15, p<.05	-.11, p=.37	-.20, p<.05

1. Do the reliability estimates for each current predictor meet minimum acceptable requirements? Describe in laypersons terms what the reliability estimates are suggesting for these 2 predictors.
2. Discuss which outcomes the selection tools predict based on statistical significance. For these statistically significant outcomes, discuss whether the results are practically significant (how strong are the relationships).

The PIP organization is interested in improving employee selection and is considering the use of new predictors. Some of these predictors (general personality test, numerical ability test, verbal ability test, physical ability test) are available through third parties (consultants). The consultants are able to generalize the validity of these predictors to the PIP job. Other predictors (conscientiousness personality test, job knowledge test) have already been tested in the PIP organization. PIP improved the work sample test and needs to determine the validity of the work sample in predicting criteria.

The table below shows the validity evidence for the general personality test, numerical ability test, verbal ability test, and physical ability test from XYZ Consulting firm. These validity coefficients can be generalized to the PIP job; the jobs on which these estimates are based are substantially similar to the F/M Associate job at PIP. Please assume that reliability for these predictors meets or exceeds suggested levels.

	Job Performance	Guest Satisfaction	Absenteeism	Turnover
General Personality Test	.25, p<.05	.38, p<.01	-.22, p<.05	-.18, p<.05
Numerical Ability Test	.41, p<.01	.19, p=.21	-.11, p=.37	-.13, p=.20
Verbal Ability Test	.32, p<.05	.31, p<.05	-.01, p=.37	-.06, p<.07
Physical Ability Test	.29, p<.01	.08, p<.10	-.19, p<.05	-.21, p<.01

The next table demonstrates the validity evidence for the conscientiousness personality test and the job knowledge test. These tests were developed by the PIP HR department with the assistance of external consultants. Their validity coefficients were obtained through the use of a concurrent criterion-related validation study conducted over time with F/M associates within the PIP organization (multiple properties). Please assume that reliability for these predictors meets or exceeds suggested levels.

	Job Performance	Guest Satisfaction	Absenteeism	Turnover
Conscientiousness Personality Test	.34, p<.01	.33, p<.01	-.30, p<.05	-.31, p<.05
Job Knowledge Test	.40, p<.01	.15, p<.05	-.18, p<.05	-.17, p<.10

PIP HR also has redesigned and improved the work sample. The work sample still requires that candidates address a problem with a leak in a condo, but the methods for evaluating performance have now been standardized. Data have been collected over time of current F/M employees across the PIP properties. The Revised Work Sample meets reliability standards. Here are the validity estimates:

	Job Performance	Guest Satisfaction	Absenteeism	Turnover
Revised Work Sample	.67, p<.01	.21, p=.31	-.30, p=.06	N/A

3. Review the job description (in Canvas) and the reliability and validity information above. Assume, due to budget constraints, that you can choose only 3 predictors for selecting F/M Associates. Which 3 will you choose and why? Make sure to justify your choice based on the data as well as the job, organization, etc.

Note: You may consider the original predictors (original work sample, unstructured interview) as well as the proposed predictors (general personality test, numerical ability test, verbal ability test, physical ability test, conscientiousness personality test, job knowledge test, revised work sample). You may assume that the predictors are not substantially correlated with each other.