Case #1: Levitt and List: Challenges with Data in the Hawthorne Effect Reanalysis

One problem in Levitt and List's (2009) reanalysis of the Hawthorne experiments is that the data are incomplete. Many of the old records were lost or not written down well, so the researchers had to work with missing pieces. When the data are not complete, the results can be misleading because they do not show the full story of how workers acted (McDavid, Huse, & Hawthorn, 2019). This makes it hard to know if changes in work were really caused by people being watched or if something else was going on.

Another problem is bias in which cases were studied. Geddes (1990) explains that when only certain cases are chosen or saved, the answers can be one-sided. In the Hawthorne reanalysis, only some of the experiments had enough data to study. This means the results may not apply to all of the experiments. If only the best or clearest records are used, the findings may not show what really happened in the whole set of studies.

Both of these problems, missing data and bias in what was studied, make it hard to know if the Hawthorne effect was real or only the result of weak records. When important information is missing, the results can give a false picture of what actually happened. When only certain cases are studied, the findings may be one-sided and not represent the whole situation. Together, these problems show that program evaluation depends on more than just using the right study methods. It also requires data that are complete, reliable, and strong enough to support fair conclusions (Levitt & List, 2009). Without solid data, even well-designed studies can lead to shaky or misleading results.

This is a colleague post above.

Hello Tutors,

Respond by Day 5toonecolleague and explain whether the problems have to do with design or data collection. What other data collection problems could, in your view, also be important? Explain.

Thanks Tutors.