In today’s age, most of us get our information from social media. It’s amazing that we have access to so much free information available at any given moment. However, it can also be a double-edged sword, because with the abundance of information we’re bombarded with daily on social media, it's hard to tell who’s speaking the truth and who’s not.
Especially when so many of these individuals are very credentialed Ph.D.’s, M.D.’s, or R.D.’s, who cite actual research studies to support their claims.
Unfortunately, it is often the case that the studies they're citing don't actually support their claims at at all. This can be very confusing for the general public who doesn’t have a formal education in research. It’s hard for most people to really know whether certain information is valid or not.
Our goal with this article is to help you evaluate the quality of evidence that individuals cite on social media. While understanding all the nuances of research really does take time and formal education, you can, at the very least, learn the basics of what to look for when assessing the credibility of the sources and references shared by your favorite influencers.
After reading through this article, you'll be better equipped to distinguish between accurate information and potentially misleading claims in the ever-evolving landscape of social media content.
The Hierarchy of Evidence
The first thing we need to discuss is the hierarchy of evidence when it comes to research. All research is not created equal. There are studies that carry more weight than others. The evidence from some kinds of studies outweigh the evidence of other studies, even if they are slightly contradictory. So the first thing to understand is, what are the different kinds of studies and how strongly do we value their evidence when making health claims in humans?
The different classifications of studies include:
- cell culture studies
- animal studies
- case studies
- cohort studies
- controlled clinical trials
- meta analysis
Understanding the Different Types of Studies
1. Cell Culture Studies: Cell culture studies serve as the building blocks of research, taking place in controlled laboratory settings. Scientists meticulously manipulate isolated cells to observe their reactions to specific interventions or conditions. These studies offer valuable insights into the intricate mechanisms within our cells. However, their applicability to the complex web of biological processes within the human body is somewhat limited. What we discern in cell cultures may not seamlessly translate to how our bodies function as a whole because our physiology is complex and doesn’t function in an isolated system the way that cell culture studies are conducted. Inherently, cell culture studies carry the least amount of weight when it comes to how translatable the evidence is to humans.
2. Animal Studies: Animal research is often the next step up after cell-culture studies. These studies often involve rodents, insects, and other animals to explore biological processes, interventions, and treatments. Since animals are biological systems that resemble human physiology more closely than individual cells, they provide stronger evidence. Nevertheless, caution is warranted when extending findings from animal studies to humans, given the inherent differences between species and variations in physiology that can affect the relevance of these results.
3. Case Studies: Case studies are the first kind of study we are discussing that is specifically in humans. Case studies are essentially studies conducted in on one person or a small group of people. In the realm of clinical or medical research, case studies serve as a means to explore rare conditions or unique patient experiences. They provide a detailed, firsthand account of specific situations, making them inherently descriptive. Yet, they cannot be used to establish causation or generalize findings to larger populations because they lack the controlled conditions and sample sizes required to draw direct causal relationships between variables. To establish causation, findings have to be replicated using larger sample sizes. Instead, case studies are used to establish hypotheses or bring attention to unusual phenomena, which will later be tested in larger-scale studies.
4. Cohort Studies: In cohort studies, researchers meticulously track participants' exposures to specific factors and monitor their outcomes. For example, researchers may follow a group of individuals for 20 years and see if there is any relationship between the consumption of a particular food and a certain health outcome. Cohort studies are valuable for identifying associations/correlations between variables and providing insights into enduring trends. However, they may not definitively establish causation, as the presence of confounding variables or biases can influence the observed outcomes (more on this later).
5. Controlled Clinical Trials: Controlled clinical trials are considered the gold standard in the world of research. Clinical trials involve human participants, who are randomly assigned to different groups. The pinnacle of clinical research is a placebo-controlled, double-blinded trial. In these trials, both participants and researchers remain unaware of who is receiving the intervention and who is receiving a placebo, effectively minimizing bias. Controlled clinical trials provide robust evidence for establishing cause-and-effect relationships. They offer particularly valuable insights into the efficacy and safety of interventions within a meticulously controlled, systematic framework.
6. Meta-Analysis: While controlled clinical trials are considered the “gold-standard,” they still do not carry the most weight in terms of evidence, meta-analysis does. Think of a meta-analysis as a “study of studies.” In a meta-analysis, researchers systematically review and analyze the results of multiple studies on a specific topic, and use sophisticated statistical techniques to synthesize more robust conclusions about the topic compared to a single clinical study. Meta-analyses stand as highly credible sources for drawing well-informed conclusions, thanks to their ability to synthesize data from various studies. This increases the sample size and minimizes the risk of bias, making them the go-to resource for evidence-based decision-making.
Examples of how research can be misrepresented
Now that you understand the hierarchy of evidence, it’s important to discuss how these different types of studies are often misrepresented by charlatans on social media. Let’s do so by providing specific examples of how individuals might use the wrong kind of study to support specific health claims that simply are not accurate or true.
The first example we want to present is how cell culture studies are often misrepresented. Cell culture studies are usually used to support some sort of mechanistic claim. A "mechanism" simply refers to the detailed explanation of how a specific biological process or phenomenon works at a molecular or cellular level. For example, describing how insulin helps lower blood glucose is a mechanism.
One clear example of misrepresentation is when individuals in the low carb community claim that carbohydrates cause fat gain because insulin inhibits something called lipolysis, which is the mechanism by which our bodies break down fat for energy. This claim is simply based off mechanistic research showing that insulin inhibits lipolysis, but this cannot be extrapolated to conclude that eating carbs makes us fat. To make such a claim, we need clear evidence from clinical trials showing that high carb diets cause more fat gain than low carb diets when calories are the same, which is not what the research shows at all.
By the way, if you want to learn more about why this isn’t the case, make sure to check out our article titled “Do Carbs Make you Fat?” This is simply one of many examples of how charlatans extrapolate findings from cell culture studies to make claims about health that simply are not true. The funny thing is, that most times there’s direct clear human evidence contradicting their claims, yet they choose to ignore it.
Animal research is no different. One of the most prevalent claims made from animal studies is the idea that intermittent fasting can help extend lifespan. Most of the research showing that intermittent fasting can extend lifespan is conducted in Drosophila, AKA flies. Yes, you read that right, flies. If it’s not evidently clear as to why we can’t make conclusions about the effects of fasting on lifespan in humans based off studies conducted in flies, let us explain:
- Biological Differences: Flies and humans have vastly different biology. For instance, flies have a much simpler system for nutrient processing, so what works for them might not work the same way in our more complex human bodies.
- Metabolic Variations: Humans and flies have different metabolic rates. Flies have a much faster metabolism, so their response to fasting may not mirror what happens in our bodies when we try intermittent fasting.
- Evolutionary Variability: Over millions of years, species like flies and humans have evolved distinct ways to adapt to their environments. So, what benefits one species may not necessarily benefit another due to their unique evolutionary histories.
- Difference in Lifespan: Flies have substantially shorter lifespans compared to humans. Flies may only live for a few weeks, while humans can live for several decades. This means that when flies undergo intermittent fasting for 8-10 hours, it represents a much larger proportion of their life than it would for humans fasting for the same duration. The effects of intermittent fasting on extending lifespan in flies might not translate directly to the much longer human lifespan due to this significant time scale difference.
It’s quite obvious that cell culture studies and animal studies do not hold a ton of weight to support health claims in humans. Where it gets tricky is when human studies themselves are misrepresented.
For example, cohort studies are some of the most misrepresented studies because individuals use them as evidence to suggest causation when they can only establish correlation. Just because two variables may have a correlation, does not mean that one causes the other because of confounding variables. Let me explain. There’s a classic correlation example between ice cream sales and incidents of drowning. In general, when ice cream sales go up, incidents of drowning do as well. Does that mean that selling ice cream causes people to drown? No. There’s likely another variable or set of variables (AKA a confounding variables) that can help explain this correlation. In this case, that confounding variable is the season. During summer, people buy more ice cream because it’s hot, so ice cream sales go up. People also tend to go to the beach and pool more often during the summer, thus, there are more drowning incidents.
Correlation does not equal causation.
There are tons of examples of misrepresented correlational data on social media. For example, one that really bothers us is when individuals say that eating sugar causes weight gain so you need to avoid all sugar, which inherently makes people avoid things like fruit which is actually very healthy for you. There is a positive association between sugar consumption and weight gain, but this doesn’t mean that eating sugar makes you fat. It’s the fact that individuals who eat a ton of sugar are likely overconsuming total calories because foods high in sugar tend to be highly processed foods that are hyper-palatable and easy to overconsume. This nuance is important because we don’t want to cause unnecessary fear with our messaging. You can have sugar in your diet, and as long as your overall diet is pretty healthy and you don’t go over on calories, you can still lose weight.
Lastly are clinical studies. These are hard to misrepresent, but there are ways that individuals can intentionally misrepresent controlled clinical studies as well.
The first thing to look for is to ensure that the population used in the study is similar to the population that the claims is being made in. For example, the findings of a study conducted in older, postmenopausal women, may not be relatable to claims made in healthy young males for obvious reasons. It’s similar to the reasons why animal studies can’t be used as support for claims in humans. To give you a specific example, there may a supplement that helps improve bone density in postmenopausal women, however, that same supplement may not be effective at increasing bone density in young men. Why? Well, there are a number of reasons, but for starters, older postmenopausal women are at a greater risk of bone loss compared to young men. Therefore, the supplement may only be effective in people who already have a significant degree of bone loss, which would exclude healthy young men.
Another thing to look out for is similarities in whether the protocol and intervention used in the study resemble that of the claims being made. This is more important for claims that are made about specific foods or supplements. Electrolyte supplementation is a perfect example of this. There is evidence suggesting that electrolyte supplementation can help hydration status and improve performance in individuals who are dehydrated and mostly in endurance sports. Yet, some supplement companies promote electrolyte supplements to help improve performance to everyone, which simply isn’t true. While some individuals may benefit from using them, it’s likely that people who are well-hydrated and doing resistance training in a well-air-conditioned environment, with long rest periods in between sets, won’t benefit from using them. This is why it’s important to ensure that the protocol and intervention used in the study that being cited to support a certain claim is similar to the claim being made.
It’s pretty easy to see how complicated these topics are and why research can be so easily misinterpreted. Hopefully, be reading this article you feel better equipped to tease out false information online when you come across it.
