By Faye Flam
This spring brought a bounty of dubious headlines. There was the mildly outrageous claim that NASA secretly invented warp drive and the more believable one that using gay canvassers changed minds on gay marriage. And we heard a claim by journalist John Bohannon that he “fooled millions” with a hoax alleging that chocolate aided weight loss.
That last one has a strange irony to it, since Bohannon is positioning himself on the side of the truth seekers, allegedly creating a rigged study and misleading press release to call attention to uncritical thinking and bad journalism. And, yet, he offers scant evidence backing his own clickbait headline on the website io9 “I Fooled Millions Into Thinking Chocolate Helps Weight Loss.” His hoax seems unlikely to have fooled millions of people, but the viral story unveiling his trick may well have fooled millions of people into thinking he’d fooled millions of people.
James Randi, the magician and skeptic, uses the term “honest liar” to refer to those who use trickery but then explain their illusions in the interest of public education and critical thinking. The original hoax might be seen that way, as an honest lie. But now, Mr. Bohannon needs to show he really fooled millions if he wants to rise above the clickbait-writing hucksters he’s trying to condemn.
There are ways readers can think things through to decide what to believe about all these claims. The one that got the most traction among people I knew was the claim that NASA created warp drive, which would, according to stories, allow humanity to start cruising around the galaxy like they do on “Star Trek.” In April, that story got picked up in a couple of British papers and the Huffington Post, with “Has NASA Accidentally Invented The Warp Drive?” Unfortunately, there’s no quote from an engineer who was suddenly and unexpectedly transported to the asteroid belt.
Had the reporter interviewed physicist Sean Carroll, the answer would be almost certainly not. He gave a nice, clear deconstruction of the story in his blog The Preposterous Universe. The evidence presented for the warp drive amounts to someone who works for NASA writing a comment about warp drive on a blog called NASAspaceflight.com, which is not actually produced by NASA.
Carroll uses the incident to illustrate a concept called Bayesian reasoning:
Think of it this way. A friend says, “I saw a woman riding a bicycle earlier today.” No reason to disbelieve them — probably they did see that. Now imagine the same friend instead had said, “I saw a real live Tyrannosaurus Rex riding a bicycle today.” Are you equally likely to believe them? After all, the evidence you’ve been given in either case is pretty equivalent. But in reality, you’re much more skeptical in the second case, and for good reason — the prior probability you would attach to a T-rex riding a bicycle in your town is much lower than that for an ordinary human woman riding a bicycle.
He’s onto something important here, though Bayesian reasoning is easier said than done. He knows a lot about space and time and the laws of physics so it’s easy for him to spot a claim that’s as unlikely as a cycling T-rex. But the rest of us can still look at how many decades of science would have to be undone for the claim to be true.
Another clue can be found in the story’s implications. Why isn’t the space agency commenting? Why on earth would they try to cover this up? NASA has never been shy about its accomplishments. The idea of a warp drive cover-up is highly unlikely.
A key point came up in the comments section. Someone admitted that he bought into the story because he really wanted it to be true. This is an important insight. To apply Bayesian reasoning or old fashioned common sense you’d better get your biases, wishes, and fears on the table. Yes, an accidental discovery of warp drive would be awesome, but the claim is extraordinary and the evidence is weak to nonexistent.
More challenging is a recently debunked story about how to change minds about gay marriage. That one belongs to a different universe of plausibility. The claim: Some people changed their minds about gay marriage after talking to an openly gay person. On the surface that sounds reasonable. The study was published in the journal Science, which is considered the top-tier American journal. It was peer reviewed. Statisticians approved of the analysis.
True, there was some hype. One publication, Business Insider, chose the over-the-top headline, “How to Convince Anyone to Change Their Mind on a Divisive Issue in Just 22 Minutes — With Science.” The study didn’t really say you could convince “anyone.” And now even the more toned-down claims in the paper are suspect. As Fivethirtyeight.com explains, it appears the data actually gathered is not the same as the data used in the paper. One of the researchers is under suspicion of cheating.
Are we helpless in the face of cheating scientists? Or are there ways we could have been more appropriately suspicious from the start? At first glance, Bayesian reasoning would suggest the claim is reasonable. But that may be a biased assessment. I already think gay marriage should be legal. So, I’m imagining that this technique might change other people’s minds. I’m not thinking about whether it could change my mind. What if I flipped it around? Say, the researchers hired canvassers who said the concept of gay marriage threatened their religious traditions? Would I change my mind and oppose gay marriage after talking with such a person for 22 minutes? Would any of my friends? I doubt it. Now suddenly you can use a different prior belief on which to prop a Bayesian argument. From that angle it looks less like a woman on a bicycle and more like a T-rex.
In light of all this, let’s examine the chocolate weight-loss hoax. The original claim doesn’t seem crazy: A tiny amount of chocolate had a tiny influence on the amount of weight lost by a small group of dieters. The difference amounted to 10 percent of an average five-pound weight loss. That’s a difference of 0.5 pounds, which is well within the amount that people fluctuate over the course of a day. To carry out the hoax, Bohannon says he got help to put together a real study but then introduced a deliberate statistical error and falsely claimed “statistical significance.” The right response was to ignore the story completely.
That’s exactly what major U.S. news outlets did! But that wouldn’t have yielded a scandalous viral story with a sexy headline. So where were these millions who were fooled? Bohannon hooked quite a few outlets in the European press, but it’s hard to know what to make of it. We don’t know whether the story was presented as news or entertainment or as a joke.
Bohannon admits that, in essence, most of the takers were bottom feeders:
These publications, though many command large audiences, are not exactly paragons of journalistic virtue. So, it’s not surprising that they would simply grab a bit of digital chum for the headline, harvest the pageviews, and move on. But even the supposedly rigorous outlets that picked the study up failed to spot the holes.
The only “rigorous” he mentions is Shape. I guess rigor is in the eye of the beholder. Men’s Health apparently contacted him but no story has run. Here in the U.S., the vast majority of mainstream newspapers and news magazines ignored the story or sniffed at it and moved on.
Even if the story got picked up in foreign publications read by millions of people, it does not follow that millions of people were fooled. Unless there was a suspicious spike in chocolate sales, it seems more likely that most people said “meh” or maybe even “huh?” but did not rush out to start a chocolate diet. For all we know from the evidence presented, 27 Americans were fooled.
Bohannon’s claim plays to one of the most common and powerful biases: the desire to believe we’re smarter than other people. I’d never heard of the original study, but the exposure of the hoax is causing delight among my Facebook friends. People can’t resist reading about all those millions of inferior thinkers. Perhaps Bohannon fooled millions, or perhaps he fooled himself.
“The Chocolate Warp Drive Diet: Did A Journalist Fool Millions Or Fool Himself?” Forbes, May 29, 2015.
(NASW members can read the rest of the Summer 2015 ScienceWriters by logging into the members area.) Free sample issue. How to join NASW.
.png)
