Power on Networks: A Disservice to Everyone

Numerous researchers have pointed out the alarming prevalence of poorly conducted studies and irrational conclusions in nutrition research. Without a drastic reform of our research methods, we will continue to drift further and further away from good scientific practice until we lose all respect. John Ioannidis said it best in his “Challenge of reforming nutritional epidemiology research,

“Nutrition research may have had a negative impact on the public’s perception of science. The resources for some of this research could have been better spent on clear, directly controllable health threats, such as smoking, lack of exercise, air pollution, or climate change. Furthermore, the perpetuated nutritional epidemiological model is likely also harming public health nutrition. Unfounded beliefs that justify eating more food, provided that “quality food” is consumed, confuse the public and distract from the agenda of preventing and treating obesity.”

If researchers themselves do not approach the subject critically, media and social media influencers will continue to use studies to generate attention rather than informing the public about the need to interpret this information with caution. These concerns have solid foundations.

Nutrition online

A systematic review published in Public Health Nutrition intended to summarize the level of quality and accuracy [1] for nutrition-related information on websites, for example healthy eating, nutrients, dietary supplements, health outcomes related to foods and dietary patterns. In addition, the review sought to determine whether there was variation between these two factors across websites, social networks or publishers.

The final analysis included 64 studies published between 1996 and 2021.

• There was an increasing trend in the number of publications on the topic: from 1 publication in 1996 to 11 in 2020 (the year with the most publications).
• Most studies did not focus on information published in a specific region; those that did generally evaluated information published in high-income countries
• The most studied topics were disease management (26.6%), general nutrition (23.4%), and maternal and child nutrition (15.6%).
• The number of posts on social media and websites ranged from 4 to 2,770, with an average of 165.7 per post.
• It was identified that certain studies were at risk of bias. [2]

Most studies were rated as neutral (51.6%) or negative (28.1%), mainly because of the potential risk of bias in sample selection. More importantly, some studies did not use or report methods to measure the reliability of the results.

Quality

Forty-one studies assessed study quality using three different methods [3]

• 48.8% of the studies were classified as poor – “the final conclusion of the study was cautious or unfavorable.” Lack of references in the source material was the main factor contributing to this rating.
• Social media posts were more often ‘bad’ (62.5%) than websites (47.1%)
• Topics related to weight loss and supplements were more likely to be rated as “poor” in terms of quality.
• Topics about child and maternal nutrition showed a higher percentage of “good” ratings – “conclusions that were positive or raised no concerns.”

Accuracy

Accuracy was analyzed in 47 studies, where accuracy was assessed against authoritative guidelines, academic literature, or scoring systems.

• 48.9% of studies were classified as poor
• While social media and websites scored equally poorly (each around 50%), YouTube scored even worse.
• Topics related to weight loss and supplements were more likely to be rated as “poor” in terms of accuracy.
• Although only one study has been done, some topics received poor ratings (e.g., immune function and sports nutrition).

Quality and accuracy

Finally, in 27 articles both quality and accuracy were analyzed.

• Government and commercial websites received mixed scores, with some studies rating them high and others low.
• Organizations and academic institutions scored highest in both quality and accuracy.
• Interestingly, Wikipedia was analyzed in two studies: one study rated its quality and accuracy as high, while the other study rated its quality and accuracy as moderate.

Although the review had some important limitations, such as excluding studies that did not provide the specific data sought and omitting readability (a factor that makes reading easier) from the search terms, the authors concluded that low-quality and inaccurate nutrition information is common on websites and social networks. As social media becomes an increasingly important source of nutrition information, this is becoming increasingly concerning. The authors suggest that accredited experts and nutritionists should publish and promote high-quality content, while social networks and online platforms should exercise stricter control over the information shared on their sites.

I remain skeptical that only credentialed professionals can solve this problem. We have all come across profiles of professionals, often with impressive credentials, who use scientific jargon and claim to base their practices on evidence. Yet these individuals selectively present biased studies. Pointing out errors or writing critical posts about such professionals may not change the opinions of their followers. Those who support these professionals are likely to continue to believe in their credibility while rejecting dissenting opinions.

Who supervises the watchdog?

Imagine two online sources posting divergent information on a given topic, both run by reputable professionals. Which one provides high-quality information? You might argue that it is the one that presents the best evidence, and I would agree. However, it is not enough to simply “read” the best evidence; good interpretation is crucial. And more importantly, could either perspective be dismissed simply because it does not conform to the prevailing consensus?

I don’t have answers to these questions, but I believe there is no “final solution” to the problem; there are currently no gold standard guidelines. Fortunately, some authors have investigated the problem and proposed measures to address it.

A judgement in health policy has focused on identifying interventions to manage the abundance of information, both accurate and inaccurate, that complicates the search for reliable sources. [4] The authors offer four interventions.

The interventions were categorized into four prevention levels, as follows:

• Ancient: Interventions aimed at developing resilient communities and health systems that are resistant to misinformation. These are the cornerstones of transparency, encouraging dialogue and providing access to high-quality scientific information – all of which rebuild and maintain trust in government, science and healthcare professionals.
• Primary: broad strategies that reduce vulnerability to disinformation through education and training programs to improve digital health literacy and identify common manipulative techniques, such as emotionally arousing content
• Secondary: identifying and addressing specific false or misleading claims and taking action to curb them.
• Tertiary: This phase focuses on mitigating the negative impacts of infodemics on public health. This includes fact-checking, forming partnerships with the media sector to prevent inaccuracies or exaggerations in reporting, and platform-based interventions that impede the spread of harmful misinformation through content moderation and platform regulation.

Many of these interventions seem promising on paper, but need to be substantiated by their effectiveness in practice. That effectiveness is still uncertain.

There is still a long way to go and we will probably face many challenges with misinformation. If I had to summarize everything in one piece of advice, it would be: be aware of the content you consume, the profiles you follow and especially the information shared by people you trust. In general, your critical sense is less alert when it comes from someone with the same opinions as you. In times of widespread misinformation, asking questions and being skeptical are good ways to avoid being misled.

[1] Quality included reliability of information, balance and impartiality, citation of references and transparency. Accuracy was the factual correctness of information, usually compared to scientific literature or accepted guidelines.

[2] The authors used a modified version of Checklist quality criteria of the Academy of Nutrition and Dietetics with questions about relevance and validation. The authors adopted three questions: Is the review question clear and appropriate? Is the search strategy comprehensive and well described? Are the processes of abstraction, synthesis and analysis described and applied consistently? To be classified as positive, all three questions must have positive answers; otherwise, the assessment is neutral or negative.

[3] The criteria include those developed by the authors, TO SEPARATE“a short questionnaire that provides users with a valid and reliable way to assess the quality of written information about treatment choices for a health problem,” and JAMA’s Standardsusing ‘four core standards to evaluate websites: authorship, attribution, disclosure, and timeliness.

[4] Although I am generally wary of narrative reviews because of the potential for bias and the tendency to tell a story, I believe this work offers interesting ideas that can contribute to the development of a more effective model for health information management.

Sources: Quality and accuracy of online nutrition-related information: a systematic review of content analysis studies. Public Health Nutrition. DOI: 10.1017/S1368980023000873

Beyond Disinformation: Developing a Public Health Prevention Framework for Information Ecosystem Governance. The Lancet Public Health. DOI: 10.1016/S2468-2667(24)00031-8

The challenge of reforming nutritional epidemiology research. JAMA. DOI: 10.1001/jama.2018.11025.