A randomized controlled experiment involving over 10,000 students enrolled in 4th to 8th grade in Texas examined the impact of arts education on a wide variety of outcomes. Arts education included visual arts, music, dance, and theater. Six different outcomes were measured over 12 different groups, for a total of 72 different hypothesis tests conducted to examine evidence for a positive impact from additional arts education.

(a) If additional arts education has no effect, how many of the 72 tests would you expect to be significant at the 5% level just by random chance? How many at the 1% level? 

(b) In fact, 16 of the tests were significant at the 5% level and 6 tests were significant at the 1% level, but these significant results were not evenly spread out over the different response variables. In each case below, compare the number of significant results to the number expected just by random chance at each level. Does increased arts education appear to have a statistically significant impact in the areas specified? 

(i) Forty-eight of the tests were in four areas (absences, math, reading, and science) and had only 1 result significant at the 5% level and no results significant at the 1% level. 

(ii) The other 24 tests were in the areas of discipline and writing. For these 24 tests, 15 were significant at the 5% level and 6 were significant at the 1% level.