Like a field without weed-killer, the glyphosate-in-your-Cheerios story grew back last week, when the Environmental Working Group published Roundup for Breakfast, Part 2.
In case you missed Part 1, here’s the highlights: EWG tested 45 products containing conventionally-grown oats and found that 43 contained glyphosate! Glyphosate is the active ingredient in Roundup! A California jury recently decided against Monsanto and awarded $289 million to a man who claimed that Roundup caused his cancer! The FDA has been testing foods for glyphosate but has not been releasing its data to the public!
Okay, EWG didn’t use all those exclamation marks. (It did use them in a call-out box titled “GET GLYPHOSATE OUT OF OUR FOOD!,” which encourages readers to “Join EWG and tell General Mills, Quaker and Kellogg’s to get glyphosate out of their products!” and “Take Action!”) Even without the punctuation, the tone and message are clear: glyphosate might kill your kids!
When EWG’s original report was published in August 2018, plenty of reasonable readers pointed out that the killer cereal claims seemed a bit thin. All of the conventionally-grown oat products that “failed” the EWG’s test would have passed the EPA’s standards for glyphosate. EWG’s limit for glyphosate, Susan Matthews points out at Slate, is “one ten-thousandth of what the EPA has deemed to be safe.”
As Matthews and others point out, EWG’s research was not published in a scientific journal. That doesn’t mean that the results of EWG’s research are necessarily flawed, but it does mean that the researchers chose not to subject their work to the scrutiny of any scientific community.
In absence of strong scientific evidence linking glyphosate to cancer, EWG found the next best thing: a splashy court case. Alex Kasprak at Snopes reminds us that juries don’t determine scientific truths. The fact that a jury awarded money to a terminally-ill cancer patient is not proof that a particular substance causes cancer.
These critiques all offer strong reasons to be skeptical of EWG and its research, but they do require a more nuanced understanding the country’s scientific and legal systems. What if you’re a parent just trying to figure out if you should throw away your Cheerios and don’t have time for a thorough investigation of glyphosate or the different organizations that study it? Here are three quick questions you can use the next time you see scary headlines about new scientific findings.
1) Is there a conflict of interest?
Like many organizations, EWG is funded by a mix of individuals, foundations, and companies. According to its own financial statement, EWG is funded by “more than 20 companies, including Stonyfield, Earthbound Farm, Organic Valley, Nature’s Path and Annie’s.”
Some of those organic companies, like Nature’s Path, make cereal and related products that were included in EWG’s study, and were even quoted within the study as evidence of its conclusions.
Is it a problem that Nature’s Path and other companies fund research that eventually sides with organic farming practices? Not necessarily. The problem isn’t taking the money, it’s not disclosing that money at the time of publication so that readers can take potential biases into account. Without a conflict of interest statement, EWG’s claim that non-organic oat cereals are full of glyphosate is kind of like Gob Bluth selling you a frozen banana that won’t make you sick and kill you.
Bottom line: Most organizations have a funding page that can give insight into who’s paying for the research. You’ll want to consider those funding sources when evaluating the organization’s claims. If an organization has a conflict of interest but doesn’t name it, be suspicious.
2) Do the authors discuss limitations?
Part of building scientific knowledge is pointing out what might be wrong with your study, like the drawbacks of using a particular method or ways that the authors may have unintentionally influenced the results of their study. A study’s limitations don’t mean that it’s “bad”; instead, they show that the authors understand their research to be a product of choices, each of which can impact the final result.
Discussing those choices makes it easier for readers to evaluate the study, and, perhaps most importantly, paves the way for future researchers to design studies that overcome those limitations. In other words, including limitations is an essential part of participating in a scientific community. EWG’s report makes no mention of limitations. In leaving out imitations, they effectively silence debate about their methods and results.
Bottom line: If the authors don’t identify any possible limitations, they’re either insincere or delusional.
3) Is the title too interesting?
Let’s say your kid’s already got the cereal box in hand and you have seconds in which to intervene. There’s not time to track a organizations financial statements or consider how self-reflective an article’s authors are. What can you do?
Just look at the title. EWG’s title is “Breakfast with a Dose of Roundup?” Let’s compare that title to other scientific papers about weed killers published in 2018:
Glyphosate induces growth of estrogen receptor alpha positive cholangiocarcinoma cells via non-genomic estrogen receptor/ERK1/2 signaling pathway
Developmental exposure to glyphosate-based herbicide and depressive-like behavior in adult offspring: Implication of glutamate excitotoxicity and oxidative stress
Effects of different concentrations of glyphosate (Roundup 360®) on earthworms (Octodrilus complanatus, Lumbricus terrestris and Aporrectodea caliginosa) in vineyards in the North-East of Italy
Do you notice the trend? Even though the papers are studying associations between glyphosate and breast cancer, depression, and declining reproduction, the headlines are not Does Roundup cause breast cancer, Is Roundup making depressed babies, or Roundup is decimating the earthworm population!
Each of these papers is studying a small question (for example, not breast cancer, but a specific estrogen receptor). To spin those small questions into large claims is bad science, because the findings of most scientific studies are small and incremental. Papers build on papers that build on papers, slowly assembling the knowledge of a field.
Bottom line: If a paper’s title makes a sweeping claim designed to terrify you, it’s probably not good science. Scientific papers aim to build scientific knowledge, not panic.
I’m not here to tell you what to eat for breakfast. (I’ll leave that to Extra Crispy.) Eat Cheerios. Don’t eat Cheerios. Whatever works for you and your family. But if you’re considering making a serious change to your diet based on this cereal killer scare, you owe it to yourself to ask a few questions.