Here’s a study originally concluding that non-religious kids were more willing to share than kids from Christian and Muslim families. However, the international data was reassessed and a new conclusion was found. Religious (or non-religious) background had no significant effect on outcomes. Rather, it was the country in which the kids lived that was the critical variable.
Psychologist Tania Lombrozzo goes on to praise this event as illustrating how science, as a public enterprise, self-corrects.
This cautionary tale of flawed statistics and questioned claims actually illustrates something quite positive: a virtue of how science works. On the one hand, an initial conclusion was called into question — a move that could erode people’s confidence in scientific claims. On the other hand, the revision was prompted by the kinds of scientific practices that should give us confidence in science: sharing data, revisiting analyses and questioning conclusions in the service of getting things right. Scientific claims can change as we gain access to new data and figure out better ways to make sense of it; that’s a feature, not a bug.¹
I think a very good point is made here. However, her optimism overlooks the seeming fact that many ephemeral, even spurious, scientific claims (especially in pop psychology) have a great deal of influence on how laypeople look at psychological issues within themselves, their relationships and their families. People glimpse the headlines or hear a quick blurb by John Tesh and begin to devise some half-baked, misinformed strategy on how to “fix” problems, often on the basis of a careless, overreaching interpretation and reporting of scientific data.
So I tend to applaud not so much science, itself, but rather, scientists who are willing to admit the limitations of science at every step of the process.
In the study tweeted above, if the researchers messed up once, how can we be sure the revised interpretation is still not egregiously flawed? How many additional uncontrolled, unrecognized variables might continue to influence the observed outcome? As any sociologist, philosopher or theologian worth their salt will tell us, the possibilities here are potentially limitless.