Paper showing likelihood of error in medical ra... - Thyroid UK

Thyroid UK
109,140 members126,783 posts

Paper showing likelihood of error in medical randomised clinical trials

diogenes
diogenes

Here is an interesting paper that shows that amalgamating data from subjects as is done in randomised trials can lead to severe errors in interpretation

Lack of group-to-individual generalizability is a threat to human subjects research

Aaron J. Fisher, John D. Medaglia, and Bertus F. Jeronimus

PNAS July 3, 2018. 115 (27) E6106-E6115; published ahead of print June 18, 2018. doi.org/10.1073/pnas.171197...

This is also the essential argument of our recent accepted paper.

15 Replies
oldestnewest
helvella
helvellaAdministrator

Major new word of the day! :-)

In probability theory, an ergodic dynamical system is one that, broadly speaking, has the same behavior averaged over time as averaged over the space of all the system's states in its phase space. In physics the term implies that a system satisfies the ergodic hypothesis of thermodynamics.

A random process is ergodic if its time average is the same as its average over the probability space, known in the field of thermodynamics as its ensemble average. The state of an ergodic process after a long time is nearly independent of its initial state.[1]

The term "ergodic" was derived from the Greek words έργον (ergon: "work") and οδός (odos: "path," "way"). It was chosen by Ludwig Boltzmann while he was working on a problem in statistical mechanics.[2] The branch of mathematics that studies ergodic systems is known as Ergodic theory.

en.wikipedia.org/wiki/Ergod...

Somewhat disappointing that Boltzman died in 1906 and yet this concept has been, apparently, missed by so many since.

Hidden
Hidden
in reply to helvella

Not sure about it being a new word!! :) Yes Boltzmann was heavily involved in probability distributions and is named after several distributions, mainly in physics and mathematics. In thermodynamics and fluid mechanics these techniques, such as ensemble averages, are very common and examples can be found easily. I'm not sure what statistical methods are used in medical research but some papers I've read are very basic analytically. I'm surprised there isn't more collaboration between people from different scientific fields to improve analysis of data and even the design of experiments or trials.

helvella
helvellaAdministrator
in reply to Hidden

:-)

Afraid I only got as far as Poisson, Binomial and Gaussian.

In theory, it would be good if an expert statistician could be attached to every research group in order to avoid mistakes and improve interpretations.

I am particularly concerned where review papers purport to perform meta-analyses of papers which are themselves compromised by inadequate statistical rigour. Sometimes when reading even the introductions you realise that they have shoe-horned papers in which really are not comparable.

diogenes
diogenes
in reply to helvella

Very fortunately, our little team has assembled together, a dreamer of new concepts, though lacking highlevel statistical rigour, a thyroid specialist who can sufficiently understand the statistics to apply the dreams to tests of reality, a computer-literate medical specialist who can put those ideas into formal practice, and a dedicated hospital doctor who supplies the raw data. A rare assemblage I propose.

Hidden
Hidden
in reply to diogenes

It's strange and interesting that your team appear a rarely formed entity within the thyroid world. Well done might I add. In other disciplines it's the norm to have specialists from differing areas collaborating to further research. Admittedly some collaborative research topics receive huge funding from government and industry so it's easy to assemble a good team. I can't imagine much funding goes into thyroid research. Correct me if I'm wrong on this.

PR4NOW
PR4NOW
in reply to diogenes

Diogenes, just curious if I'm right. You are the dreamer, the prof is the thyroid specialist, Dietrich is the medical specialist and Larisch supplies raw data. PR

diogenes
diogenes
in reply to PR4NOW

Yes, that's about it. I claim dreaming, having started the whole odyssey off in 2011.

PR4NOW
PR4NOW
in reply to diogenes

Diogenes, since you and the group have accomplished a breakthrough in thyroid science in the last 7 years, I hope you feel as though your dreams have come true. Quite an accomplishment. PR

diogenes
diogenes
in reply to PR4NOW

The essential problem is that the implications of our physiological studies is lethal to the acceptability of randomized clinical trials. This is as true of comparing T4 only v T4/T3 combination responses. TSH, FT4 and osteoporosis, TSH, FT4 and atrial fibrillation. The paper by Fisher et al I deposited with TUK very recently is a complete rejection of the validity of most medical clinical trials based on RCTs in whatever discipline. I cannot emphasise enough how great a paper this is. Our accepted paper shortly to be published in J Thyroid Research follows exactly the same path in thyroidology and draws exactly the same conclusions. It follows that no longer can one link parameters such as TSH and FT4 to OP and AF in a generalised fashion. The whole corpus of socalled knowledge on which these conclusions rest is essentially swept away - there is no other conclusion, however strongly objectors may complain. All thinking based on these trials has to be completely revisited. The new paradigm is a return to individualised diagnosis and treatment, and not assessing patients by their placement within or without a particular range. Thyroid diagnosis can no longer be a parameter-based acceptancet of normal ranges but the examination of the particular and unique position that a patient occupies, perhaps in some cases outside the range, and their individual presentation. No longer simple biochemistry, but real medicine is needed. This conclusion has only gradually emerged as the disjoint between physiological and clinical trial implications has become clear.

PR4NOW
PR4NOW
in reply to diogenes

Diogenes, as you are all too well aware of, this will be a tough biscuit for allopathic medicine to swallow. The random, double-blind, cross-over trial has become the bedrock of 'Evidence Based Medicine' and they will fight you tooth and nail before admitting the inherent problems with many RCTs. I believe as you do that eventually the truth will prevail but it will be a long painful process to get there. Then there is the problem that the average doctor practices 15-20 years behind the science. We need a better way to keep doctors informed rather then 'educational' events sponsored by pharma.

I, and millions of other thyroid patients who don't even know who you and the group are, appreciate the battle you are about to take on for our right to be treated intelligently so that we may live a 'normal', happy and productive life instead of being a burden on ourselves, our families and society. PR

diogenes
diogenes
in reply to PR4NOW

We put our marker down: if we don't ourselves succeed, others will read and later, act.

Hidden
Hidden
in reply to helvella

You still got a lot further than most. Good stuff :)

I'm by no means an expert statistician but have used/written a lot of statistical data analysis algorithms over the years in R&D. Not medical research by the way!

I definitely agree that some papers' meta-analyses seem flawed. Comparing bananas with apples, etc. Sometimes correlations are made in error since the bigger picture hasn't even been considered. Very blinkered conclusions. Statements like "Increased [variable] A shows a strong correlation with causing condition X. It is the biggest single factor involved." Great news everyone. Or is it? Sorry folks (not that they would admit being incorrect!), we forgot to say we didn't quantify variables B, C, D, ... which could all have an effect on condition X! A lot of the time they haven't even measured B, C, or D. So the conclusion is flawed but is taken as gospel.

As Diogenes mentioned on another post there are huge differences between biochemistry/physiology research and medical-based analysis. I'm surprised some research even gets published. One problem there being the ignorant, like-minded reviewers thinking it's great work! Then the good, sound, rigourous research paper gets flagged as being "suspect" or incorrect. The mind boggles!

Hidden
Hidden
in reply to Hidden

I had had a bit of an interested in the issue of the basis of scientific studys since my first physics lesson when I started comprehsive school and learnt about Baines and his hypothasis that 'for a science to be proved it needs to be consistant with what is already known'. I was removed from the class (re arrogance) for agueing that the sceintists had not understoood him and that what sceintists did was created a lotof red herrings by proving something in a lab that went on to prove some else +something else based on this proof when the intention of baines was that what was seen in the lab needed to be consistant with a much wider feild. It need to be considered against the sum totally of all current knowledge such as history, georgraphy and nature. I said at the time (in temper as upset a being in trouble) that I would spend my life doing this with any facts I heard. I do do this all the time and have kept up my observations primarily because if find it so very useful.my mum laughed at me at the time and said that what I proposed was what every uneducated person does anyway. I never had enough of an academic bent to study sceince much furthur than this but like to think I had a breif moment of genious and dont really understand the above conversation but if it is claiming that sceince is applying to wide a brush and is not individulised enough this is consistant withmy observations as is my noticing that sceintists can be very confident that they have proved something when only a short while later it s clearly incorrect.

Treepie
Treepie
in reply to helvella

Never got the hang of stats,twoway analysis of variance ,regression,non- parametric stats .Only really remember mean,median and mode! But it was 50 years ago.

to be honest you don't really need to understand statistics - you just need to understand some basic logic.

Alice in Wonderland etc is full of examples of how dangerous ignoring the law of the excluded middle is - ie arguing from a specific to a general ... basic formula used is

All X are Y

Some Y are Z

So all X are Z

and that is basically what is going on so much of the time with interpretation of test results etc.

All tests give results

Some results are 100% accurate

So, all results are 100% accurate

And so the Cheshire cat disappears into a grimace.

You may also like...