Study Tests the Validity of Weight Loss Research Claims

Studying the Studies

This is the Weight and Healthcare newsletter! If you appreciate the content here, please consider supporting the newsletter by subscribing and/or sharing!

I am working on several projects that require me to take deep dives into weight science research and they are reminding me of a study that I thought I’d take a look at today.

In 2010 Lucy Aphramor published a paper called “Validity of claims made in weight management research: a narrative review of dietetic articles.” The paper reviewed articles published in The Journal of Human Nutrition and Dietetics from 2004 to 2008. Frustratingly, while one might hope that in 13 years the mistakes that this paper identifies would no longer be made, they are being made very regularly in current research.

Aphramor found that “Although the energy deficit approach [editor’s note: calories in/calories out] to weight management has a high long-term failure rate it continues to dominate research in the field. In the current research agenda, controversies and complexities in the evidence base are inadequately discussed, and claims about the likely success of weight management misrepresent available evidence.”

This paper also tests the common claim that “5-10% weight loss creates clinically meaningful benefits.” We’ve discussed other studies that examined this seriously questionable claim as well.

Aphramor begins by looking at the justifications that are given for recommending weight loss. These include implicit beliefs about weight loss and health, overall benefit, regulation of blood lipids and blood pressure, and prevention of diabetes.

When it comes to implicit beliefs about weight loss and health, Aphramor points out that the basis is often on the assumption that weight loss will create health benefits and/or reduce stigma (and I always want to point out that, whatever someone believes about fat people and their health, telling a group of oppressed people is told to that they should reduce the stigma they face by changing themselves to suit their oppressors is simply wrong.)

Aphramor also points out that “The belief in the value of weight loss is so firmly held that the rationale for intervention may not always be explicitly articulated.” And that sometimes the concern is “that if no action is taken the population will become inexorably fatter.”

Aphramor notes that these justifications typically depend on people simply believing that being higher-weight harms health and that weight loss will improve health and points out that “This contrasts with the usual requirements of the scientific convention that assumes clear justifications for treatment are required.” This is something that I see all the time in weight loss research. The researchers begin with the assumption that being higher-weight is bad (typically based on a correlation between being higher-weight and health issues while aggressively ignoring confounding variable like weight cycling, weight stigma, and healthcare inequalities which are correlated with the same health issues.) and then assumes that weight loss will solve the problem, which is a major leap in logic – like suggesting that since cis-male pattern baldness predicts cardiac incidents, the solution is for those folks to grow hair.

Aphramor points out that “it is not unusual to find claims of non-specific 'health benefits' which are not substantiated.” This goes back to the assumption that weight loss leads to health benefits. It also drives research that only looks at weight loss, but still claims to be about “health” even though no actual measure of health is included. The paper also traces claims and citations of the “5-10% weight loss creates health benefits” and notes that none of the citations offer an original source, positing that “it appears that beliefs about weight and health acquire a truth status so that they circulate as intuitively appealing 'facts', immune from scrutiny and become used, and accepted by editors, without supporting references.” This is absolutely still happening in weight science research, often resulting in me screaming “HOW DID THIS GET PAST PEER REVIEW?!?!?!” when I am analyzing it.

Next the paper hones in on the more specific claim that 5-10% weight loss improvements blood lipids, blood pressure, prevents diabetes and offers an overall reduction in mortality. One study that claimed that weight loss created by dieting improves blood lipids despite their own admission that "it was impossible to identify the independent influence of dietary fat on changes in lipids and lipoproteins." Here again we also see a situation where people make behavior changes and subsequently experience a bit of (at least temporary) weight loss as well as health changes, and then over-eager weight science researchers, forgetting or ignoring everything they learned in school, rush to insist that (even though it was small and simultaneous, the weight loss caused the health changes. In this case, the papers conclusion is also questionable because, as the paper points out, while this was a meta-analysis (which sounds like it would be reliable) further digging found that “of the 70 studies included in their meta-analysis of which 67% had 20 participants or less, 35% lasted for 10 weeks or less and 82% had no control group.”

This is another issue we commonly see in weight science research – they aren’t often in a hurry to admit the (often vast and multiple) shortcomings of the research that they are citing or analyzing.

Aphramor examines another paper that claims to show an impact of weight loss blood lipids and blood pressure that includes medication, exercise, salt restriction and other dietary modification that has no means to isolate the impact of any of these interventions on blood pressure. The “everybody knows” nature of the intersection of weight loss and health let researchers throw extremely basic research methods right out the window. 

In examining the citations, Aphramor writes one of my favorite lines ever in a study: “The scientific merit of a third paper relied upon can be discounted as authors neglected to include any methodology.”

I think the phrase we are looking for here is #lolsob.

In examining the purported link between weight loss and prevention of Type 2 Diabetes Aphramor notes that, again, the claims made by a paper about the specific role of weight loss in diabetes prevention are not supported in their citations. In one of the studies cited the author specific said that their finding didn’t demonstrate the effect of weight loss, but rather of a lifestyle intervention including increased physical activity which is strongly linked to insulin resistance improvements. The second paper they cited “The reduction in the incidence of diabetes was directly associated with changes in lifestyle.”  Aphramor explains “the benefits of health behaviour change are submerged in favour of weight-centred outcomes…in addition, there is no comment on the high quality evidence that weight loss at one year is almost invariably followed by weight regain.”

I’ll say here that, knowing weight science research as I do, whenever a paper includes what seems like an overabundance of citations I become suspicious. Not just because, for example, I know for a fact that there isn’t even one study that shows the majority of people succeed at significant long-term weight loss (let alone the 17 or however many they are citing) but because I am familiar with the trick they are using. Confession time – I recognize this game because I used to play it. Except I wasn’t a professional researcher creating studies meant to dictate healthcare policy. I was an undergrad taking 18 credit hours a semester, working two jobs, and required to write papers in classes outside of my major that, to be perfectly honest, just didn’t seem that important to me. I learned that if I just cited a bunch of sources (that may or may not have actually supported my claims,) there was an assumption that they did. I’m not proud of it, it’s not something I have ever done or would ever do on a paper for publication, unlike, perhaps, a lot of weight science researchers.

Next Aphramor moves on to claims of weight loss efficacy, starting by pointing out that any claim that weight loss causes health improvements would have to rest on the belief that weight loss is possible. (Similarly, even though the ability to fly might reduce joint pain, it’s not an ethical, evidence-based intervention unless it’s actually possible.)

Aprhamor points out that studies that only look at weight loss “with no attempt to measure other specific clinical outcomes, change in diet and exercise behaviours, or to collect qualitative data on participants' views and experiences (or adverse effect)” perpetuate the support of the pursuit of thinness in three problematic ways, which are still prevalent in research today:

1.       Ignoring the fact that health benefits can come directly from behavior changes. Instead, these studies automatically credit any health benefits from weight loss, even if that weight loss was both small and simultaneous to the health issues.

2.       Ignoring the fact that weight loss may come from unhealthy behaviors, including disordered eating and eating disorders. (I’ll add that weight loss treatment typically prescribes behaviors that are consistent with eating disorders, which has become the subject of what I can only refer to as research gaslighting.)

3.       “the focus on attaining one specific body size over another raises ethical issues. Researchers included in this analysis work within a particular framing of fatness as always pathological ie. 'obesity' and some articulation of the value judgments inherent in this position would help the reader assess scientific merit.”

When it comes to effectiveness, Aphramor points out that “Hand in hand with a belief in the indisputable benefits of weight loss is the associated belief that clinically meaningful weight loss is a realistic goal for most people.” Of course we know that a century of research has shown that the vast majority of weight loss attempts fail. Aphramor pointed out that even though most studies in the analysis  admitted the vast failure rate (using wording like “there is increased awareness that short-term interventions are rarely successful” and "long-term success in weight loss treatments for obesity is elusive") they never suggested that perhaps that would be a good reason to stop trying.

Another issue the Aphramor uncovered that is common today is research that takes an “overly optimistic tone, where a more qualified report of findings would increase scientific credibility.” For example, a study that claims that “all diets in 'Diet Trials' were ultimately successful in achieving weight loss in those who complied” glossing over the fact that 64% of the subjects had withdrawn by week 8 of the 16-week study.

Finally when it comes to capturing adverse effects of dieting, Aphramor finds that studies tend to either downplay any adverse effects or omit discussion of them entirely.

It’s incredibly frustrating to know that thirteen years later researchers are still making the same errors, I’m grateful for this paper and the opportunity it provides to demonstrate how long and how consistently weight science research has rested on a weak and broken foundation, full of cracks. You can read the entire paper here.

Did you find this post helpful? You can subscribe for free to get future posts delivered direct to your inbox, or choose a paid subscription to support the newsletter and get special benefits! Click the Subscribe button below for details:

Like the piece? Share the piece!

More research and resources:
https://haeshealthsheets.com/resources/

*Note on language: I use “fat” as a neutral descriptor as used by the fat activist community, I use “ob*se” and “overw*ight” to acknowledge that these are terms that were created to medicalize and pathologize fat bodies, with roots in racism and specifically anti-Blackness. Please read Sabrina Strings Fearing the Black Body – the Racial Origins of Fat Phobia and Da’Shaun Harrison Belly of the Beast: The Politics of Anti-Fatness as Anti-Blackness for more on this.

Note I don’t link to everything I discuss in this post because I don’t want to give traffic and clicks to dangerous media.