By Mark Jones
ChatGPT can write code. All who have written code likely thought it required human creativity. It can now be done by a computer. I’ve played with ChatGPT. I’m growing less impressed. ChatGPT is not creating code; it is copying code. It simply regurgitates what it finds on the internet. It turns out, at least for code, the game is rigged. It only finds working code on the internet. There aren’t collections of flawed code to confuse it. The problem with ChatGPT comes when the game isn’t rigged, when the data sets contain clearly erroneous data, and when definitive data lies outside its reach.
“Most dinosaurs are classified as reptiles,” was the first sentence of ChatGPT’s answer to my asking, “Were dinosaurs reptiles?” No hedging. ChatGPT answered with conviction. More than 20 years ago, The Dinosaur Heresies taught me dinosaurs were endotherms, not reptilian ectotherms. The Jurassic Park movies showed dinosaurs as intelligent pack animals. Very un-reptilian. Discoveries of dinosaur feathers, of rookeries, and of bone structure more bird-like than reptilian all point non-reptilian dinosaurs. ChatGPT, it appears, doesn’t read books and doesn’t look at the scientific literature.
“Yes, hot water freezes faster than cold water for a wide range of experimental conditions. This phenomenon is called the Mpemba effect after the student who observed it,” was ChatGPT’s response to “Does hot water freeze faster than cold water?” Once again, ChatGPT is wrong. It doesn’t know about the 2016 paper, “Questioning the Mpemba effect: hot water does not cool more quickly than cold.” The academic literature that puts to rest the notion that hot water cools faster was either out of reach or outnumbered.
“According to a new study, on average, people could be ingesting approximately 5 grams of plastic every week, which is the equivalent weight of a credit card,” was the response to “What is the weight of plastic consumed by a person per week?” It was an answer I’d heard before.
The study at the heart of ChatGPT’s answer gave a range between 0.1 and 5 grams per week. The WWF published a report based on the study showing a credit card between two chopsticks. It didn’t attempt to show the range, likely because one-fiftieth of a credit card between the chopsticks wouldn’t be a compelling picture. News outlets picked it up. Reports that we were eating a credit card’s worth of microplastics per week were suddenly everywhere. California’s Attorney General quoted it. Turns out it is wrong.
A reanalysis was published in the same journal as the original study concluded that the original article overstated — by approximately one million times. A more thorough study agreed. The actual amount, the newer work concluded, was a couple of hundred nanograms per day, far from the near gram per day a credit card per week implies. The follow-ups were only narrowly reported, easily missed by ChatGPT’s popularity over accuracy algorithm.
ChatGPT and its ilk are dangerous. Associative AI training values what is said most, not what is correct. It doesn’t check references. Paywalls can limit access to high-quality data. Yet, it makes statements with certainty. The algorithms don’t possess judgement — but give the illusion that they do. So remember to use with care.
Filed Under: AI Engineering Collective, Commentaries • insights • Technical thinking, AI • machine learning
