.

Before conducting a research project, it is important to have each of the followingexcept

a pre-registered plan so that the line between exploratory and planned analyses is clearly documented.

a clear statement of the flaws in all prior, related research projects.

a clearly stated research question.

a sample-size plan that clearly indicates the extent of the data to be collected.

2.

Once a research project has been completed, the researcher should

immediately revise the plan of analysis to reflect unexpected problems with the collection of data.

check to see if the results were as expected. If not, reanalyze them to coincide with the expectation. Then submit them for publication.

check to see if the results were as expected. If so, submit them for publication. If not, discard the data and consider the project a failure.

make the results publically available, such as in a peer-reviewed journal or in an online repository of research data.

3.

Several recent efforts have been made to conduct large-scale replications of well-publicized findings by psychologists. The results of these efforts suggest that replicability in psychology is currently

perfect. One-hundred percent of the replications resulted in effects very similar to those in the original studies.

better than expected. The replications returned results that, in two-thirds of the cases, were very similar to those in the original studies.

abysmal. None of the replications resulted in effects very similar to those in the original studies.

poor. Although some replications resulted in effects very similar to those in the original studies, this was unusual.

4.

One innovative Open Science practice is

pre-registered review. Peer reviewers examine the proposed research project before the data are collected. Feedback is provided to the researcher while it can still be useful, and, increasingly, if the proposed project is judged to be sound, the journal commits to publishing the project regardless of the results.

pre-planning review. Quarterly conferences are held at several research hubs throughout the world. Prospective researchers are invited to attend and share their ideas for research in a focus-group setting. Those who attend are given a certificate that, if submitted with the manuscript in which they report their results, will fast-track the manuscript to publication.

pre-peer review. There are two levels of review of manuscripts submitted by researchers for publication. The first level is a review of the manuscript by psychology majors throughout the world who have volunteered to do so using a standardized form. After the first 40 reviews are received, the forms are analyzed and, if there is sufficient support, the manuscript is sent to official reviewers.

pre-analysis review. A panel of statistics experts reviews the proposed analysis plan and simulates the likely outcome using hypothetical data. If the simulation does not meet the criteria for acceptance, the panel recommends changes in the plan that are likely to result in meeting the criteria.

5.

The use of archival data presents each of the followingexcept

that a benefit is the possibility of answering new research questions using existing data.

that it can be difficult to draw a firm line between exploratory and planned analyses because all of the data have already been collected.

the virtual impossibility of converting results expressed using the HHST approach to the estimation approach.

that there can be difficulties making data openly available, as participant consent forms may have been signed long before this Open Science practice became the norm.

6.

When reporting one's research, the authors advocate each of the followingexcept

providing figures that feature summary statistics.

telling the full story.

reporting the results using both the estimation approach and the NHST approach, providing CIs and p-values for all important effect sizes.

providing details only when they are relevant for replication.

7.

Crowd-sourced replication refers to

using online polls to identify and prioritize studies that should be replicated.

using the internet to recruit research partners for replication studies.

recruiting participants for replication research at large-scale events.

raising research funds through crowd-sourcing sites on the internet.

8.

In a project with many dependent variables

it is often effective to collapse related DVs into a single DV.

the multiple comparisons problem becomes prominent.

exploratory analysis is preferred.

the estimation approaches becomes inefficient in contrast to the NHST approach.

9.

When research involves a huge number of participants (Big Data), effect sizes

will be estimated very imprecisely, so you will have to guess at the meaning/size of each effect.

may be precise or imprecise. For this reason, the main focus should be onp-values.

are rarely used because the entire population is known.

will be estimated very precisely, but it is still important to interpret the meaning of each effect.

10.

When dealing with non-normal data, the authors recommend

supplementing the data with archival data.

multiplying the number of variables.

avoiding replication.

using trimmed means.