In 2015, you might remember a whole bunch of articles out there saying that eating dark chocolate can help you lose weight. Everyone was excited, and there are still people who believe it. However, it turned out the whole thing was an experiment intended to expose how junky diet science and medical journalism really is. It was an incredible success.
In 2014, a documentary team contacted John Bohannon with an interesting proposition. They were making a movie about how the diet industry is based on bad science. You know how it seems like every day articles come out saying this or that food is good for you, and then the next day it’s bad? That’s because of poorly-performed studies and bad journalism. Bohannon, who is a biologist in addition to a journalist, understood what the filmmakers were talking about. Only a year before, in 2013, he had done a sting for Science on open access journals, and how they will charge writers to get a paper published, and do little to no peer review. Bohannon had submitted papers filled with mistakes, and still more than half were accepted. He was excited to keep digging into this world.
John Bohannon
This was how he did it: he recruited subjects and ran a clinical trial where people were assigned to different diets. Some were given dark chocolate, and once the study was over, Bohannon and his team added up the results. They intentionally set up the study just like how a “real” diet research study would be, complete with major errors. The main problem was that the study measured too many things, 18, in total, including weight, sodium, sleep quality, etc. There were only 15 people in the study, and when you measure a large number of factors with such a small group, the stats are going to be swayed. The numbers are going to look a lot more significant than they really are, and it made the chocolate seem like a major player.
Once the paper was written, Bohannon submitted it to a bunch of journals. Within 24 hours, it was accepted by many of them. He chose the International Archives of Medicine, which charged 600 Euros. There was no peer review. Once it was out there, the global media went crazy. Bohannon’s hoax had gone viral.
Interestingly enough, the reality of diet science is actually worse than what Bohannon’s chocolate hoax suggests. According to the Health News Review, which critiques how the media covers science, some journalists will cover studies that were never actually done at all. Some scientists, too, will write papers that make a big claim while the actual research doesn’t hold up. This is often because of who is paying the scientist. A scientist on the board of Weight Watchers will say the Weight Watchers diet is the best, and so on. The need for funding overshadows the need for accuracy, which is something many scientists will even admit.
Lots of people were very upset by the hoax when Bohannon revealed the truth. They said it was unethical and that it just added to a field already rife with lies and poorly-framed information. You could compare it to “satire news,” which is sometimes so convincing that people actually believe it, even after the authors say it isn’t real. Do hoaxes and stings actually accomplish anything, or do they just get swept up in a torrent and lost at sea?
