Common Terminology and Statistics Issues- Part 2

This is the Weight and Healthcare newsletter! If you like what you are reading, please consider subscribing and/or sharing!

Part one of this was published on December 13, but this piece was pre-empted when the USPSTF put forth dangerous dieting recommendations for children (remember that the public comment period ends January 16.) In the past I’ve written pieces specifically about issues and mistakes that are made with terminology that is used…let’s call it differently in weight science as well as common statistics mistakes and mishaps. In part 1 I offered some additional terminology troubles today in part two we’re discussing statistics shenanigans.

Using a percentage that seems high without proper context:

In one example of this, early in the COVID-19 pandemic I saw a news report claiming that so-called “ob*sity”* was a risk factor for severe COVID because, in a particular city, 25% of severe negative outcomes were in people who are classified as “ob*se.”  At first, that might seem like a large number, but that doesn’t justify calling being higher-weight a risk factor. In order to even begin to be able to draw conclusions from this, we have to at least know the total number of so-called ob*se people who live in the city - otherwise we have no way to know if 25% is higher or lower than the total percentage of this population. I looked it up and that number was 38%.

Several things are issues here.

First, if I were trying to draw conclusions from this (and I wouldn’t, more on that in a moment) I would conclude that being higher-weight is protective, since 38% of the community is higher-weight, but only 25% of the people with severe outcomes were. (Said another way, people who weren’t “ob*se” were 62% of the overall population but 75% of the severe outcomes.) That’s the main statistical issue here. You can’t use a percentage like this without contextualizing it.

Moreover, I wouldn’t draw conclusions from this at all. First, because “ob*sity” is simply a ratio of weight and height. Making assumptions that since a group of people have some physical characteristic in common (like, in this case, height-weight ratio) then that physical characteristic is the reason for the difference in outcomes is on extremely shaky ground, scientifically speaking. In this example, since there are many other factors that can impact this result (including the fact that higher-weight people are at the mercy of a healthcare system in which practitioner weight bias is rampant and, even if that’s not an issue, the tools, best practices, pharmacotherapies and more, that are used are typically developed for thin bodies/excluding fat bodies) we don’t know what number of those severe outcomes were due to healthcare inequalities or other factors.

Relative vs Absolute Risk

Novo Nordisk recently used this one in their manipulative press release about the possible cardiovascular benefits of Wegovy.

Relative Risk Reduction is the percentage decrease of risk in the group who received an intervention vs the group that didn’t receive the intervention. This number can be helpful to determine differences in outcomes between groups, but it’s not that helpful in determining individual risk. For that you need Absolute Risk Reduction.

Absolute Risk Reduction is the actual difference in risk between the group that got the intervention and the group that didn’t. This helps us understand the likelihood that a given individual will benefit from an intervention.

Relative risk reduction can often be a much larger number than absolute risk reduction and those who are trying to manipulate statistics (and those who don’t know about this - like reporters quoting a Novo Nordisk press release) can use relative risk reduction to make people believe a treatment has a greater effect than it actually does.

Let’s use a super simplified example. Let’s say that 200 people who have Condition X are enrolled in a study to see if Medication Y reduces death from Condition X. 100 of them are given the medication (the intervention group) and 100 are not (the control group). At the end of the observation period, 1 person in the intervention group dies and 2 people in the control group die. The relative risk reduction (percent risk of death in the intervention group divided by percent risk of death in the control group, in this case .01 divided by .02) is 0.5 or 50%. That seems like a lot – a company with incredibly poor ethics might issue a press release saying that their drug reduced death by 50% without mentioning absolute risk.

Absolute risk is calculated by subtracting the percentage of risk reduction in the intervention group from the percentage of risk reduction in the control group, or 2%-1% which is a 1% reduction. A much smaller number that more accurately predicts individual experience.

So when a weight loss company gives a percentage of risk reduction, it’s important to ask if it is relative or absolute risk reduction they are talking about.

For example, in the Novo Nordisk press release they wrote that their drug “reduces the risk of major adverse cardiovascular events by 20%”. That is the relative risk. The absolute risk reduction was less than 2%.

These are the basics, if you want to really dig into relative vs absolute risk, there’s plenty more to it. There’s an interesting piece about it here for starters.

LOCF vs BOCF

Last Observation Carried Forward (LOCF) and Baseline Observation Carried Forward (BOCF) are two ways of dealing with dropouts in a trial to determine an endpoint value for those who dropped out.

Let’s look at another example with easy numbers. Let’s say there is a weight loss intervention trial where weight is taken at the beginning, at 1 year, and at 2 years. They start with 100 participants and all 100 participate in the initial weigh-in and the 1-year weigh-in, but only 50 participate in the 2-year weigh-in (just fyi, this is not an uncommon dropout rate in weight loss studies.)

When the study authors are trying to calculate the success of the intervention, how do they handle the 50% of people who dropped out?

If they use LOCF, they take the person’s weight from the 1-year weigh-in and use that as if it were the 2-year weigh-in number.

If they use BOCF, they take the person’s weight from the beginning of the study and use that as if it were the 2-year weigh-in number.

Pop quiz – which do you think the weight loss industry typically uses?

If you guessed LOCF, you are exactly right. And that’s an issue.

Absent actual follow-up to find out the reason that the participants didn’t turn up for the final weigh-in (and that follow-up almost never happens) we don’t know if they didn’t show up because they regained weight that they had lost (and, given 100 years of research showing that this is the outcome for the vast majority of those attempting intentional weight loss that’s not, like, out of the question). So using the 1-year weigh-in number for the 2-year weigh-in may very well artificially exaggerate the success of the intervention.

Just to make the math easy, let’s say that all 100 participants lost 10 pounds at the one year weigh-in. Then let’s say that the 50 people who returned for their year two weight-in had regained 5 pounds (again, a super common occurrence.) Meaning that they were 5 pounds less than their baseline weight at the two-year weigh-in.

Now the authors have three basic choices:

They could just ignore the dropouts as if they never existed, and claim that the average weight loss was 5 pounds (ignoring both the dropouts and the fact that the participants’ weight was climbing at the time that observation ended). Despite this being the kind of thing that would get you a solid “F” in your Freshman Research Methods class, it’s a pretty common occurrence in weight loss research.

But, if they use LOCF, they can use the 10-pound loss for the 50 people who didn’t turn up at year 2. This will boost their total to an average of 7.5lbs per participant. This, again, is quite common in weight loss research.

Now, if they used BOCF (which I would argue is far more appropriate given our base knowledge around weight regain and the fact that they should make every effort not to artificially inflate the success of their intervention) they would have an average of 2.5 pounds lost per participant. This is incredibly uncommon in weight loss research.

And again, they should be honest that, in those who were weighed at year two, weight was being regained.

In an ideal world, weight science research would be transparent and would not use terminology and statistics to mislead or obfuscate. Unfortunately, we don’t live in an ideal world so it’s up to us to know what questions to ask, and to ask those questions.

Did you find this post helpful? You can subscribe for free to get future posts delivered direct to your inbox, or choose a paid subscription to support the newsletter (and the work that goes into it!) and get special benefits! Click the Subscribe button below for details:

Liked the piece? Share the piece!

More research and resources:
https://haeshealthsheets.com/resources/

*Note on language: I use “fat” as a neutral descriptor as used by the fat activist community, I use “ob*se” and “overw*ight” to acknowledge that these are terms that were created to medicalize and pathologize fat bodies, with roots in racism and specifically anti-Blackness. Please read Sabrina Strings’ Fearing the Black Body – the Racial Origins of Fat Phobia and Da’Shaun Harrison’s Belly of the Beast: The Politics of Anti-Fatness as Anti-Blackness for more on this.