Here is an interesting cross post from Lymphomation@yahoogroups.com
updated on lymphomation.org: the problems with testimonials
Tue Jul 24, 2018 7:24 am (PDT) . Posted by:
karlamonyc
The Problems with Testimonials PDF version
lymphomation.org/Testimonia...
"For many centuries doctors used leeches and lancets to relieve patients of their blood. They KNEW bloodletting worked.. EVERYBODY said it did. When you had a fever and the doctor bled you, you got better. EVERYONE knew of a friend or relative who had been at death’s door until bloodletting cured him. Doctors could recount thousands of successful cases."
See also:
When Laypersons Give Medical Advice
lymphomation.org/CAM-layadv...
You can't tell in any individual case what would have happened if nothing was done, or something else
- particularly for a type of lymphoma that is known to wax and wane - that has a variable clinical course.
Further, cancers are heterogeneous - the genetic changes vary from person to person within the same type of cancer.
Such accounts can't inform about the number of persons who have used the intervention and did not benefit or were harmed.
There is no denominator - the number of patients studied.
The denominator tells us if the result are likely to occur in 1 of 5, or rarely - in 1 of 30,000. Testimonials cannot provide even an estimate of a rate of effect in others,
or if the effect that is measured was even caused by the intervention.
In clinical science a pre-specified study size is required to provide a rate of an event or treatment effect.
A control group is often needed to establish causality - that the treatment (or part of the treatment) caused the outcome.
This is the reason that expensive and large controlled studies are often needed before a drug can be approved for the treatment of a disease.
People who die cannot testify
Patients who have tried and failed an alternative strategy cannot be accounted for.
Only the "successful" outcomes are reported, which may be 1 in many thousands.
Compare with peer-review clinical trial where the number of patients receiving the treatment
are known up front (prospectively), and the positive and negative outcomes are measured uniformly - and reviewed independently.
The authenticity of the report?
We cannot know if the person reporting the benefit really has the medical condition,
or if he or she is reporting the outcome accurately?
With testimonials there is no follow up, or independent review of the outcome over time.
Did the person relapse shortly after the testimonial?
The biases of the individual reporting the case?
Does the individual have a financial conflict of interest or strong belief?
Do they sell the product or charge a fee for dispensing the information?
Is the testimonial a way of validating their personal decision process and theories?
The specifics of the case, such as the natural history of the disease?
Even for cancers with a very poor prognosis there are case reports in the literature of spontaneous remissions, independent of any intervention.
Some types of cancer behave indolently -- and do not progress to a point requiring treatment.
People sometimes win the lottery , but this does not make playing the lottery a good bet - particularly when betting your life.
How the outcomes were measured, when, and by whom?
Was the reported success objectively measured and validated by independent physicians?
Is it a patient reported outcome?
Was it that the patient felt better?
What tests were used to measure it? What happened later?
.... and did the intervention lead to a lasting and meaningful clinical benefit?
Is the condition self-limiting - does it sometime self-correct with time?
What other medical treatments were given shortly before or after?
A CT scan will often show lesions after standard treatment that are necrotic scar tissue. Credit might be given to an alternative practice used AFTER this treatment, when it was merely the resolution of a scar tissue over time, a normal bodily process.
Was the tumor removed by surgery prior to the alternative practice?
The accuracy of the diagnosis?
Was it a false diagnosis of a cancer, or a cancer of a type with an indolent course?
For all of these reasons it's prudent to regard testimonials with suspicion - particularly if the report is implausible, scientifically. Scientists get cancer too, as do their children. Do scientists think the approach is plausible?
Similarly, case reports have many of the above limitations - cannot establish causality, and can't be the basis for predicting outcomes in others. For example, if one patient receives regular treatment followed by a study drug, it's not possible to be certain that the second part of the treatment improved the outcome.
Better support for soaring numbers of cancer survivors 29 March 2013. Post treatment cancer 'recovery package' urged
Skin problems at the end of a clinical trial
Reducing or Eliminating Alcohol
