Larry had just completed his master’s degree in industrial and organizational (I/O) psychology and obtained his first job with the human resources department of a large local school district. The school district had just had an EEOC (Equal Employment Opportunity Commission) complaint filed against it regarding its physical ability test used to select School Security Officers (SSOs). In particular, both women and older (i.e., those age 40 and older) applicants had complained that the dynamometer test (a test of hand grip strength) was biased against both groups. Therefore, one of Larry’s first projects was to determine if, in fact, the dynamometer test was biased against these two groups. Fortunately for Larry, the school district tested a very large number of applicants each year for the SSO job. In addition, because the dynamometer test had been used for almost two decades, he had data going back almost 20 years and thus had a sufficiently large sample of female and older job candidates who had subsequently been hired, thus allowing him to examine for possible test bias on sex and age. Looking back on his notes from his graduate applied psychological measurement class, Larry remembered that he had to perform a moderated multiple regression analysis to examine for possible test bias. Larry first looked at possible test bias based on gender by entering all the dynamometer test scores in the database for those who had been hired and the gender variable (0 = women, 1 = men) into the regression equation to predict those who successfully completed a 12-month probationary period. Both the dynamometer and the gender variables had significant regression coefficients (i.e., they significantly predicted who passed probation). Therefore, in a second step, Larry entered the interaction term (i.e., gender × test score) into the regression equation. Lo and behold, the regression coefficient for the interaction term was also significant in predicting who successfully completed probation.

Next, Larry reran the multiple regression equation. This time, however, Larry entered the dynamometer test score and age (less than age 40 = 0, age 40 and older = 1) in the first step to predict successful completion of probation. Again, both regression weights were significantly related to who completed probation. However, when Larry entered the interaction term (test score × dichotomous age group) in step 2 of the regression equation, the regression coefficient for the interaction term was not significant. Now, it was time to sit down and look at the results more carefully and try to figure out what was going on with this dynamometer test. 

Questions 

1. Does there appear to be any test bias in terms of gender? If so, what kind of predictor bias seems to be evident? 

2. Does there appear to be any test bias in terms of age? If so, what kind of predictor bias seems to be evident? 

3. If you were Larry, what omitted variables would you investigate? Would you look for different potential omitted variables for gender and age? Why or why not? 

4. What other factors besides the omitted-variable concern might be impacting Larry’s results? 

5. Would drawing a scatterplot (similar to Figures 11.3 and 11.4) help in determining what is happening with the data, or is the moderated multiple regression analysis sufficient? 

6. Does the criterion variable that Larry used (i.e., whether a new hire passed probation) make a difference in whether we are likely to find test bias?

Figures 11.3

Figures 11.4

Transcribed Image Text:

Social Desirability 240 220 200 180 160 140 120 100 80 3 4 5 0 300 00 6 Conscientiousness 7 Op po D 8 ps & of 0 8 D D 30 D 0 9 Gender .male o female