The authors collected every significant clinical study of drugs and dietary supplements for the treatment or prevention of cardiovascular disease between 1974 and 2012. Then they displayed them on a scatterplot.
Prior to 2000, researchers could do just about anything they wanted. All they had to do was run the study, collect the data, and then look to see if they could pull something positive out of it. And they did! Out of 22 studies, 13 showed significant benefits. That’s 59 percent of all studies. Pretty good!
Then, in 2000, the rules changed. Researchers were required before the study started to say what they were looking for. They couldn’t just mine the data afterward looking for anything that happened to be positive. They had to report the results they said they were going to report.
And guess what? Out of 21 studies, only two showed significant benefits. That’s 10 percent of all studies. Ugh. And one of the studies even demonstrated harm, something that had never happened before 2000
Written by
cesanon
To view profiles and participate in discussions please or .
In the research biz, we called this "p hacking" -- combing through the data to find statistically significant differences. I avoided it on my job by requiring a paper beforehand spelling out the model being tested, and what difference will be considered meaningful and not just statistically significant.
There's also another effect that is more recent - NIH now requires that ALL listed clinical trials post results. Previously, authors of clinical trials that had null or negative findings rarely got published. This was because of a combination of two factors: (1) researchers and their sponsors did not want to publicize failures, and (2) journals did not want to feature studies that were not practice changing.
Sorry, but I think we're trying to compare apples and avocados. Clinical studies as done/required today are fine. A goal is set, criteria that need to be met are set and researchers get money to study them. But even there I hope the researchers look beyond the target to see if anything unusual (and good) is happening. THEN such observations need to be subjected to clinical trials. And, in the other case where researchers just collect a bunch of data and see if anything unusual and good is present, is also good, but only as a beginning. Years ago, on one of these now defunct sites, others and I even offered personal results as a first stage indicators to be studied and, maybe, pursued.
Content on HealthUnlocked does not replace the relationship between you and doctors or other healthcare professionals nor the advice you receive from them.
Never delay seeking advice or dialling emergency services because of something that you have read on HealthUnlocked.