The post-publication commentary online caught that they were originally looking at whether being hungry or sated affected honesty, and when that didn't pan out, they tried to invent a different "study" around the data collected. It was retracted fairly quickly after that was pointed out.
This is hugely helpful for someone (like me) who felt overwhelmed by my statistics class in college and grad school. You lay it out beautifully. I would love to see more pieces like this!
Ragen, you and other readers may be interested in this editorial, as well as the full issue of the journal American Statistical Association that it is based on.
Moving to a World Beyond p < 0.05 by Wasserstein, Schirm, and Lazar.
Agreed! I'd love to understand more about all of those numbers and protocols in scientific papers. I haven't found a resource that can explain statistics in research papers for someone who doesn't know much about statistics.
GenAI is likely to make it a lot easier for those staring down the barrel of "publish or perish" to game the system. Not only does GenAI make P-hacking easier, but it also makes it more feasible for scientists and other researchers to publish in areas outside their actualy expertise. Many publishers and research funders are requiring disclosure for use of GenAI. That could be another "red flag" to keep an eye out for going forward.
This is exactly the kind of stuff needed right now. I talk with people all the time who say “a study proved” whatever nonsense they are promoting but when you look at the “study” it’s just an opinion they found some actual facts out of context that back up their opinion. Or they confused causation with correlation. My favorite was a “study” that showed that people who drink diet soda are more likely to be heavy, which proves that diet soda makes people heavy. When really what it was saying was that heavy people are more likely to drink diet soda (because marketing has convinced them they need to).
Recognizing bias and being able to discern what is true science and what is marketing couched in scientific words is not only important in health but in all phases of life. Please, please do more of this!!!
The retracted 2020 study that taught me about P-hacking: Dishonesty is more affected by BMI status than by short-term changes in glucose
https://pubpeer.com/publications/341C41AED4E29DBD67D9D3EFE36F66
The post-publication commentary online caught that they were originally looking at whether being hungry or sated affected honesty, and when that didn't pan out, they tried to invent a different "study" around the data collected. It was retracted fairly quickly after that was pointed out.
Yikes! how have I never heard about this? Thanks for posting it.
This is hugely helpful for someone (like me) who felt overwhelmed by my statistics class in college and grad school. You lay it out beautifully. I would love to see more pieces like this!
Ragen, you and other readers may be interested in this editorial, as well as the full issue of the journal American Statistical Association that it is based on.
Moving to a World Beyond p < 0.05 by Wasserstein, Schirm, and Lazar.
https://www.tandfonline.com/doi/full/10.1080/00031305.2019.1583913
This is super interesting, thank you!
I find this super interesting! Thank you!
Very well explained article!! Thanks so much for sharing this information!!! 🙂
Just came here to say I want more like this! So cool.
Agreed! I'd love to understand more about all of those numbers and protocols in scientific papers. I haven't found a resource that can explain statistics in research papers for someone who doesn't know much about statistics.
LOVE THIS! Thank you for all your work!
GenAI is likely to make it a lot easier for those staring down the barrel of "publish or perish" to game the system. Not only does GenAI make P-hacking easier, but it also makes it more feasible for scientists and other researchers to publish in areas outside their actualy expertise. Many publishers and research funders are requiring disclosure for use of GenAI. That could be another "red flag" to keep an eye out for going forward.
Randall Monroe has a good treatment of this on xkcd 882
https://xkcd.com/882/
I really like reading your writing on these topics.
This is exactly the kind of stuff needed right now. I talk with people all the time who say “a study proved” whatever nonsense they are promoting but when you look at the “study” it’s just an opinion they found some actual facts out of context that back up their opinion. Or they confused causation with correlation. My favorite was a “study” that showed that people who drink diet soda are more likely to be heavy, which proves that diet soda makes people heavy. When really what it was saying was that heavy people are more likely to drink diet soda (because marketing has convinced them they need to).
Recognizing bias and being able to discern what is true science and what is marketing couched in scientific words is not only important in health but in all phases of life. Please, please do more of this!!!
Oooh this makes my head hurt.