Suppose the nightly TV news shows scientists at a press conference announcing that they’ve discovered evidence that a newly developed chemical can stop the aging process. The work has not yet gone through the peer review process. Analyze this study according to Guideline 2.

Data From Guideline 2:-

Statistical studies are supposed to be objective, but the people who carry them out and fund them may be biased. It is therefore important to consider the source of a study and evaluate the potential for biases that might invalidate the study’s conclusions.

Bias may be obvious in cases where a statistical study is carried out for marketing, promotional, or other commercial purposes. For example, a toothpaste advertisement that claims “4 out of 5 dentists prefer our brand” appears to be statistically based, but we are given no details about how the survey was conducted. Because the advertisers obviously want to say good things about their brand, it’s difficult to take the statistical claim seriously without much more information about how the result was obtained.

Other cases of bias may be more subtle. For example, suppose that a carefully conducted study concludes that a new drug helps cure cancer. On the surface, the study might seem quite believable. But what if the study was funded by a drug company that stands to gain billions of dollars in sales if the drug is proven effective? The researchers may well have carried out their work with great integrity despite the source of funding, but it might be worth a bit of extra investigation to be sure.

Major statistical studies are usually evaluated by experts who are supposed to be unbiased. The most common process by which scientists examine each other’s research is called peer review (because the scientists who do the evaluation are peers of those who conducted the research). Reputable scientific journals require all research reports to be peer reviewed before the research is accepted for publication. Peer review does not guarantee that a study is valid, but it lends credibility because it implies that other experts agree that the study was carried out properly.