Twenty-nine independent research teams were given an identical dataset and tasked with investigating a research question about racial bias in sports: ‘Are football players with darker skin tone more likely than those with lighter skin tone to receive red cards from referees?’ The results of this crowdsourced research (i.e. data analysis conducted by multiple research teams) were surprising. The analytical approaches varied drastically among research teams, as did their conclusions: 20 teams (69%) found a statistically significant positive effect (nine teams retained the null hypothesis of no differences by skin tone). Read the article (and commentary) and discuss what this means for publication bias and academic research.

Original article: Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E. and Carlsson, R. (2018) Many analysts, one data set: making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science, 1(3), 337–56.

Commentary: Silberzahn, R. and Uhlmann, E. (2015)

Crowdsourced research: many hands make tight work. Nature, 526(7572), 189–91. www.nature.com/news/crowdsourced-

research-many-hands-make-tight-work-1.18508