When evaluating research reports in health care, a popular tool is the Assessment of Multiple Systematic Reviews

Question:

When evaluating research reports in health care, a popular tool is the Assessment of Multiple Systematic Reviews (AMSTAR). AMSTAR, which incorporates 11 items (questions), has been widely accepted by professional health associations. A group of dental researchers have revised the assessment tool and named it R-AMSTAR (The Open Dentistry Journal, Vol. 4, 2010). The revised assessment tool was validated on five systematic reviews (named R1, R2, R3, R4, and R5) on rheumatoid arthritis. For each review, scores on the 11 R-AMSTAR items (all measured on a 4-point scale) were obtained. The data, saved in the RAMSTAR file, are shown in the accompanying table.

a. One goal of the study was to compare the mean item scores of the five reviews. Set up the null and alternative hypotheses for this test.
b. Examine the data in the table and explain why a randomized block ANOVA is appropriate to apply.
c. The SAS output for a randomized block ANOVA of the data (with Review as treatments and Item as blocks) appears below. Interpret the p -values of the tests shown.
d. The SAS printout also reports the results of a Tukey multiple-comparison analysis of the five Review means. Which pairs of means are significantly different? Do these results agree with your conclusion in part c?
e. The experiment wise error rate used in the analysis in part d is .05. Interpret this value.