15 Comments

The retracted 2020 study that taught me about P-hacking: Dishonesty is more affected by BMI status than by short-term changes in glucose

https://pubpeer.com/publications/341C41AED4E29DBD67D9D3EFE36F66

The post-publication commentary online caught that they were originally looking at whether being hungry or sated affected honesty, and when that didn't pan out, they tried to invent a different "study" around the data collected. It was retracted fairly quickly after that was pointed out.

Expand full comment
author
Jun 11·edited Jun 11Author

Yikes! how have I never heard about this? Thanks for posting it.

Expand full comment

This is hugely helpful for someone (like me) who felt overwhelmed by my statistics class in college and grad school. You lay it out beautifully. I would love to see more pieces like this!

Expand full comment

Ragen, you and other readers may be interested in this editorial, as well as the full issue of the journal American Statistical Association that it is based on.

Moving to a World Beyond p < 0.05 by Wasserstein, Schirm, and Lazar.

https://www.tandfonline.com/doi/full/10.1080/00031305.2019.1583913

Expand full comment
author

This is super interesting, thank you!

Expand full comment

I find this super interesting! Thank you!

Expand full comment

Very well explained article!! Thanks so much for sharing this information!!! 🙂

Expand full comment

Just came here to say I want more like this! So cool.

Expand full comment

Agreed! I'd love to understand more about all of those numbers and protocols in scientific papers. I haven't found a resource that can explain statistics in research papers for someone who doesn't know much about statistics.

Expand full comment
founding

LOVE THIS! Thank you for all your work!

Expand full comment

GenAI is likely to make it a lot easier for those staring down the barrel of "publish or perish" to game the system. Not only does GenAI make P-hacking easier, but it also makes it more feasible for scientists and other researchers to publish in areas outside their actualy expertise. Many publishers and research funders are requiring disclosure for use of GenAI. That could be another "red flag" to keep an eye out for going forward.

Expand full comment

Randall Monroe has a good treatment of this on xkcd 882

https://xkcd.com/882/

Expand full comment

I really like reading your writing on these topics.

Expand full comment

This is exactly the kind of stuff needed right now. I talk with people all the time who say “a study proved” whatever nonsense they are promoting but when you look at the “study” it’s just an opinion they found some actual facts out of context that back up their opinion. Or they confused causation with correlation. My favorite was a “study” that showed that people who drink diet soda are more likely to be heavy, which proves that diet soda makes people heavy. When really what it was saying was that heavy people are more likely to drink diet soda (because marketing has convinced them they need to).

Recognizing bias and being able to discern what is true science and what is marketing couched in scientific words is not only important in health but in all phases of life. Please, please do more of this!!!

Expand full comment

Oooh this makes my head hurt.

Expand full comment