KorvaJoin the waitlist
Science

Half of Social Science Studies Failed an Independent Retest

The papers were all published no later than 2018, so newer studies shaped by stricter transparency rules might fare better.

AI image
science.org · Across the social sciences, half of research doesn’t replicate | Science | AAAS

An international collaboration independently retested 164 social science studies and found that only 49% produced the same result when tried again with new data. The results were reported April 1 in the journal Nature.

The project, called Systematizing Confidence in Open Research and Evidence, drew on a collaboration involving researchers numbering in the hundreds and spread across dozens of nations. It covered prominent journals across six disciplines, from business and economics to psychology and sociology.

That failure rate matches what smaller checks had previously found in single fields such as psychology. The similarity suggests unreliable results are widespread across the social sciences, rather than confined to one corner.

There is one important caveat. All of the papers tested had come out in 2018 or earlier, before many journals tightened requirements for transparency and analytical rigor, so studies published today might hold up more often.

“There is an ongoing need for researchers to establish trust in the research we are producing,” said Kelly Cobey, who studies research practices at the University of Ottawa Heart Institute and did not participate in the effort. She said examining how studies are conducted and analyzed is key to maintaining that trust.

Questions people ask

Does a failed replication mean the original finding was false?

No. A failed replication means a second team did not get the same result under similar conditions, but differences in participants, timing, methods, or chance can explain the gap. Researchers usually treat it as a signal to investigate further, not as proof the first finding was wrong or dishonest.

Why might newer studies do better than the ones tested?

Since the mid-2010s, many journals have required researchers to share data and code, describe their analysis plans in advance, and report sample sizes more clearly. Those practices make errors easier to catch and make it harder to present a chance result as a solid finding.

How was this retest different from earlier ones?

Earlier replication drives often focused on one field, usually psychology, and involved dozens of papers. This effort tested more than 100 papers across six fields at once, using teams in several dozen countries, which makes the 49% figure the broadest cross-disciplinary estimate yet.

More in Science