By Mark Jones
I hit cancel rather than send. The boiling frog cartoon on a friend’s newsletter invoked an urge to correct. I overrode it. It is often said mistakes are how you learn. Everyone is pro-learning, but no one likes making mistakes. Human nature means we like having them pointed out even less. Pointing out errors doesn’t endear you to people. Use of a flawed analogy, complete with cartoon frog, did not impact the newsletter’s conclusions. I elected to let it slide fearing it would be received like a grammar correction, more likely to annoy than to be well-received.
I’ve been on a crusade to get flawed studies retracted. Once I started looking for problematic science to correct, it quickly became overwhelming. Errors are easy to find, far in excess of my capacity to address.
I was taught that science was inherently self-correcting. I came to falsely believe that once pointed out, corrections would replace erroneous conclusions. I’ve come to realize the errors live on, zombie facts, that simply refuse to die.
The case of the boiling frog was adjudicated convincingly more than 20 years ago. As a habituation analogy, it remains startingly common. It is also bunk. A frog in cold water that is slowly being heated will stay in the water and eventually perish while a frog placed in hot water will jump out, so the story goes. The slowly changing temperature, it is said, is imperceptible to the frog. Unable to perceive the danger, cooked frog.
Experiments in the mid-1800s with decerebrated frogs, frogs with their brains removed, seem to be the source. German physiologist Friedrich Leopold Goltz in 1869 published his experiments of slowly heating brainless frogs, reporting twitching observed upon immersion in hot water did not occur in slowly heated water. The fact the frogs must be brainless was lost. Brainless frogs led to an enduring zombie fact that continues to be retold in spite of multiple published corrections.
My recent quests have fared no better. I took on the viral story that toxic flame retardants were found in 85% of analyzed products and exceeded the EPA reference dose, used as an estimate of normal exposure from dust and diet. The reporting led to spatula-geddon, the rampant disposal of black kitchen spatulas. I remain convinced the paper confuses hazard and risk, conflating any presence of a troublesome material with a likelihood of exposure to that troublesome material. My previous column and article failed to stop the paper from being quoted.
I set out to get the paper retracted due to the serious errors of both math and method. The paper now bears two corrections. The first over miscalculating the reference dose. That was initially reported low by 10X. The second correction, prompted by my letter to the editor, deals with miscalculation of exposures. The correlation used to turn concentration into exposure was incorrect. The exposures are much lower than initially reported. My peer-reviewed letter to the editor describing the errors accompanies the paper. I argue, using the paper’s data, that less than 0.5% of the collected samples present a risk of exposure from intended use, a far cry from 85%. In spite of the errors, the paper has not been retracted.
The response I feared for pointing out the boiling frog fallacy is pretty much what I got when I contacted the journal. Human nature fights retraction. Retraction isn’t good for the authors. It is a blackeye for the journal. It doesn’t make money for the publisher. The incentives for retracting a flawed study don’t often lead to retraction. When there is fraud, it is a different story. 2023 set a record with more than 10,000 retractions but the vast majority were for fraud. In spite of my efforts, a paper with errors in both math and methods is still there, still being quoted. Black spatulas remain imperiled by a zombie fact.
Filed Under: Commentaries • insights • Technical thinking
