Clickbait examples – One simple vitamin can reduce your risk of heart disease. Eating chocolate reduces stress in students. New drug prolongs lives of patients with rare disease. Health headlines like these are published every day, sometimes making opposite claims from each other. There can be a disconnect between broad, attention-grabbing headlines and the often specific, incremental results of the medical research they cover. So how can you avoid being misled by grabby headlines? The best way to assess a headline’s credibility is to look at the original research it reports on.
Clickbait – Healthium reduces risk of heart disease.
Let’s start by considering the cardiovascular effects of a certain vitamin, Healthium. The study finds that participants taking Healthium had a higher level of healthy cholesterol than those taking a placebo. Their levels became similar to those of people with naturally high levels of this kind of cholesterol. Previous research has shown that people with naturally high levels of healthy cholesterol have lower rates of heart disease.
So what makes this headline misleading: “Healthium reduces risk of heart disease.”
The problem with this headline is that the research didn’t actually investigate whether Healthium reduces heart disease. It only measured Healthium’s impact on levels of a particular kind of cholesterol. The fact that people with naturally high levels of that cholesterol have lower risk of heart attacks doesn’t mean that the same will be true of people who elevate their cholesterol levels using Healthium.
Clickbait – Eating chocolate reduces stress in students
The relationship between eating chocolate and stress. The hypothetical study recruits ten students. Half begin consuming a daily dose of chocolate, while half abstain. As classmates, they all follow the same schedule. By the end of the study, the chocolate eaters are less stressed than their chocolate-free counterparts.
What’s wrong with this headline: “Eating chocolate reduces stress in students”
It’s a stretch to draw a conclusion about students in general from a sample of ten. That’s because the fewer participants are in a random sample, the less likely it is that the sample will closely represent the target population as a whole. For example, if the broader population of students is half male and half female, the chance of drawing a sample of 10 that’s skewed 70% male and 30% is about 12%. In a sample of 100 that would be less than a .0025% chance, and for a sample of 1000, the odds are less than 6 x 10^-36.
Similarly, with fewer participants, each individual’s outcome has a larger impact on the overall results— and can therefore skew big-picture trends. Still, there are a lot of good reasons for scientists to run small studies. By starting with a small sample, they can evaluate whether the results are promising enough to run a more comprehensive, expensive study. And some research requires very specific participants that may be impossible to recruit in large numbers.
The key is reproducibility— if an article draws a conclusion from one small study, that conclusion may be suspect— but if it’s based on many studies that have found similar results, it’s more credible.
The next time you see a surprising medical headline, take a look at the science it’s reporting on. Even when full papers aren’t available without a fee, you can often find summaries of experimental design and results in freely available abstracts, or even within the text of a news article. It’s exciting to see scientific research covered in the news, and important to understand the studies’ findings.