 |
| Neuroscience has its credibility problems, too. Photo: Anna Shvets at pexels |
The "replication crisis" is the term given to the worrying fact that the conclusions in a large proportion of studies cannot be reproduced when other scientists try to confirm the original findings. The paper by John Ioannidis in 2005 that sounded the alarm had the astounding title, "Why Most Published Research Findings Are False".
As The Guardian's science editor, Ian Sample, points out, since then "three major projects have found replication rates as low as 39% in psychology journals, 61% in economics journals, and 62% in social science studies published in the Nature and Science, two of the most prestigious journals in the world".
This is serious stuff:
The influence of an inaccurate paper published in a prestigious journal can have repercussions for decades. For example, the study Andrew Wakefield published in The Lancet in 1998 turned tens of thousands of parents around the world against the measles, mumps and rubella vaccine because of an implied link between vaccinations and autism. The incorrect findings were retracted by The Lancet 12 years later, but the claims that autism is linked to vaccines continue. (Source)
The new study, by the University of California San Diego's Rady School of Management, was published in the journal Science Advances. Ian Sample states that the study highlights the extent of ongoing reference to the false findings:
Studies in top science, psychology and economics journals that fail to hold up when others repeat them are cited, on average, more than 100 times as often in follow-up papers than work that stands the test of time.
The finding ... has led the authors to suspect that more interesting papers are waved through more easily by reviewers and journal editors and, once published, attract more attention.
Put more clearly:
...findings from studies that cannot be verified when the experiments are repeated have a bigger influence over time. The unreliable research tends to be cited as if the results were true long after the publication [of the experiment that] failed to replicate. [Source]
In addition:
"We also know that experts can predict well which papers will be replicated," write the authors Marta Serra-Garcia, assistant professor of economics and strategy at the Rady School and Uri Gneezy, professor of behavioral economics also at the Rady School. "Given this prediction, we ask 'why are non-replicable papers accepted for publication in the first place?'"
Their possible answer is that review teams of academic journals face a trade-off. When the results are more "interesting," they apply lower standards regarding their reproducibility. [Source]
Take note of the next statement, and then the explanation for giving prominence to what is interesting over what is true:
It has been claimed and demonstrated that many (and possibly most) of the conclusions drawn from biomedical research are probably false.
A central cause for this important problem is that researchers must publish in order to succeed [in academia], and publishing is a highly competitive enterprise, with certain kinds of findings more likely to be published than others. Research that produces novel results, statistically significant results ... and seemingly 'clean' results is more likely to be published.
As a consequence, researchers have strong incentives to engage in research practices that make their findings publishable quickly, even if those practices reduce the likelihood that the findings reflect a true ... effect.
The writers of this journal article - it's in Nature and authors include John Ioannidis, cited above - suggest scientists often attempt to game the system in order to achieve high publication numbers against their name to impress superiors.
As Ian Sample states:
The academic system incentivises journals and researchers to publish exciting findings, and citations are taken into account for promotion and tenure. But history suggests that the more dramatic the results, the more likely they are to be wrong.
Methods used can also be fault. The Nature article quoted above is titled, "Power failure: why small sample size undermines the reliability of neuroscience". Wow! What a mountain of implications that statement sets down in the middle of those who are trying to reconcile the brain/mind conflict that a few scientists loudly promote.
A further important issue is highlighted in that key Nature article as part of its conclusion.
Small, low-powered studies are endemic in neuroscience. ... Nevertheless, we should not assume that science is effectively or efficiently self-correcting. There is now substantial evidence that a large proportion of the evidence reported in the scientific literature may be unreliable. Acknowledging this challenge is the first step towards addressing the problematic aspects of current scientific practices and identifying effective solutions.
Finally, we need to consider another challenge to the reliability of what is purported to be a scientific conclusion. And that challenge is wokeism, which arised from the Critical Justice Theory derived from the Frankfurt School's social theory approach in analysing society and the roots of injustice. By way of explanation:
On the political left, wokeness sometimes drifts into wokeism—a system of thought and behavior characterized by intolerance, policing the speech of others, and proving one’s own superiority by denouncing others.
Within American society especially CJT has taken hold of academia, the mainstream news media, and largely, the Democratic Party. A feature of the American practice of this theory is the soft totalitariansm shown in enforcing its manifesto, which covers the diminishing of the status of the traditional family, and the rejection of the male/female reality of each human person.
How this affects the reliability of the scientific endeavour was made clear by the capitulation of a heretofore reputable scientific website to the wokeism afflicting the United States. I'm referring to the website Science-Based Medicine, which has betrayed its previously principled goal of "evaluating medical treatments and products of interest to the public in a scientific light, and promoting the highest standards and traditions of science in health care". (Source)
This is what has happened:
If you read the site’s recent coverage..., you will come away thinking there is a big, broad, impressive body of evidence for youth gender medicine, that there isn’t any actual controversy here at all. Rather than evaluate the available evidence carefully, SBM defaults to just about every activist trope that has come to dictate the terms of this debate in progressive spaces. This is a disturbing example of what complete ideological capture of an otherwise credible information source looks like. Science-Based Medicine has “bought into the hype and failed to ask the hard questions”.(Source)
The topic that gave rise to SBM's loss of reputation is "how to best help gender dysphoric children and adolescents — that is, young people who feel a great deal of distress about their biological sex, which they will often (though not always) describe as a sense of profound identity mismatch and/or being “trapped in the wrong body”. (Source)
This website's meltdown is chronicled by Jesse Singal, who has built up a formidable expertise on this issue. He has written a long account of how SBM's principals ran a book review of Abigail Shrier’s book Irreversible Damage: The Transgender Craze Seducing Our Daughters. The reviewer, Dr Harriet Hall, who had written 700 articles on the website, was favourable in her review.
Lo and behold, in response to pushback to the positive review, the principals "decided to retract the review entirely, replacing it with a statement that it had failed to meet SBM’s normal editorial standards and denying that the move was related to political pressure from the SBM community or anyone else".
Singal states:
SBM has, in the wake of this retraction, published three articles about Shrier, Hall’s review of her book, and the broader controversy over youth gender medicine...
All three articles contain major errors and misunderstandings and distortions, ranging from straightforward falsehoods to baffling omissions to the re-regurgitation of inaccurate rumors first circulated years ago. Activist claims that stretch or violate the truth are repeatedly presented in a credulous manner, while the myriad weaknesses in the research base on youth gender medicine are simply ignored.
And this brings us back to my starting point as to reasons for the sorry state of science. Singal identifies the political, the human, factors in play in doing science, especially in the present climate:
The basic problem here is what Scott Alexander calls “isolated demands for rigor”. This is a standard aspect of human nature, a close sibling of confirmation bias. When it comes to claims we don’t want to believe we will insist the evidence isn’t actually as strong as it appears, demand more and more clarification, shift the goalposts of the debate, and nitpick if necessary; for claims we do want to believe, we’ll wave weak evidence right through the gate without interrogating it too harshly, even if it suffers from exactly the same problems.
In an outstanding piece of scholarship, Singal goes through the "thought pieces" subsequently run on SBM, pinpointing where statements offered in rebuttal to Shrier's information fail to achieve that goal. Here is the link to the second part of Singal's effort to bring accuracy and truth to the debate.
By truth, I refer to Singal's discovery of made-up quotes in one of the pieces that SBM ran after Hall's review was retracted. Though the principals did make some alterations to the pieces SBM ran when notified by Singal, it seems that the professional standing of the principals - one is academic clinical neurologist at the Yale University School of Medicine and the other is professor of surgery and oncology at the Wayne State University School of Medicine (to keep it brief ) - is no safeguard in what should be a clear-cut case of applying science-based medicine to a controversial topic.
Without defenders from within science, the public is on its own. Bravo! Jesse Singal.
Ω If you like this blog, go to my Peace and Truth newsletter on Substack, where you can subscribe for free and be notified when a new post is published.
No comments:
Post a Comment