So if you read about health and food, how should you do that? I thought the following Guardian column can provide a nice illustration. My take-away as a non-dietician is that saturated fat and cholesterol is bad, and that you should ignore people like Nina Teicholz who claim that eating fatty products is actually good for you. A keto diet, meaning a diet high in fat, is clearly the worst thing you could ever do, and people like Jordan Peterson who propose an all meat diet are lunatics who should be put away.
Now, I am not here to say which one is right. Again, I'm not a dietician, I'm a linguist. But that also means I'm a scientist, and I can recognize bias in the debate when I see it.
But that's not the real problem. The way in which she frames the debate is as one between a fringe group of science denying idiots and the scientists who've worked for decades in the field and who you should listen to. But this is clearly not a valid argument. She notes that much of the work published about why fat is good, is published in small journals, whereas her camp publishes in the top journals. But all that shows is that her side has control fo the debate, and has for decades.
As much as we scientists may wish to believe we're above bias, we're not. Challenging the status quo is hard, if not impossible, if you haven't already made a name for yourself. There are plenty of examples in the history of science where new and revolutionary ideas were oppressed for decades, before it turns out that the scholars who made the claim were right all along. As long as papers have to be reviewed by the scholars who represent the status quo, and as long as those people decide what is good science, it's going to be hard to get published in major journals.
She also makes another error in this, in that it assumes big journals are better than small journals. Academics are fighting tooth and nail to get right of the ludicrous system of impact factors. When a journal has a high impact factor, when papers in that journal get cited a lot, that must mean it's a good journal. But impact factors don't provide that much information. It's an average. Even in the top journals of science like Nature many papers may receive few to no citations. And papers in small journals may have a far more significant impact. Just because something is published in a big journal, does not mean it's good science—unfortunately.
So to properly understand her column, you need to know how science works. Most people don't, and that's fine, because why would they?! But journalists then shouldn't assume that they do. Now aside from the scientific methodology, there's the issue of the actual science. A large part of her column revolves around cholesterol and whether LDL is bad. For decades science has linked it to heart disease, and saturated fat causes the liver to produce LDL. Sounds simple enough, but cholesterol is more complex than that and should be analyzed on more micro level; dealing with the different molecular structures. But that's way too technical for a general audience. How it all works is thus ignored in service of the larger point. Again, that does not mean it's wrong, but it means you get a limited picture.
This brings us to a bigger problem: who to trust? Should we start ignoring climate change, because a scientist here and there claims that it's all a hoax? Clearly not. There is good reason for established science to be established science: evidence has confirmed it time and again. But as a non-scientist you need to be aware that much is lost in translation between the science and the more lay articles. The only thing you can do is try to be as informed as possible and keep an open mind. Which is basically what scientists (are supposed to) do as well.