An introduction to EBM (session 2)
skills
beginner
evidence-based medicine
critical appraisal
clinical trials
Session outline
- this session is about the randomized control trial (RCT)
- the results of RCTs are particularly important in EBM
- we’ll do a bit of historical work here to show how the RCT developed
- and we’ll identify how key features of RCTs guard against some common errors in medical evidence
- we’ll also look at some ways that RCTs might conceal errors - they’re certainly not perfect!
Exercises
- E1: control in the streptomycin trial
- E2: would you take the treatment?
- E3: the rules of an RCT
- E4: cheat your way through a trial
Introduction: James Lind and clinical trials
Another big sentence
Because the randomised trial…is so much more likely to inform us and so much less likely to mislead us, it has become the “gold standard” for judging whether a treatment does more good than harm.” (Sackett et al. 1996)
Lind
Consequences
The consequence was, that the most sudden and visible good effects were perceived from the use of the oranges and lemons; one of those who had taken them, being at the end of the six days fit for duty. The spots were not indeed at that time quite off his body, nor his gums found; but without any other medicine, than a gargarism of elixir vitriol, he became quite healthy before we came into Plymouth, which was on the 16th of June. The other was the best recovered of any in his condition; and being now deemed pretty well, was appointed nurse to the rest of the sick
Clinical trials
- clinical trial = test of a healthcare intervention
- clinical trials are nothing new:
- James Lind’s anti-scorbotic treatments (Hughes 1975)
- Fibiger’s serum diptheria treatment (Hrobjartsson et al. 1998)
- and lots of similar examples of clinical trials (Clarke 1996)
- perhaps surprisingly, clinical trials have been controversial…
Testing and doing harm
- it’s hard to show benefits without causing harm
- showing inferiority = causing harm
- we usually seek to avoid harms wherever possible
- different ethical frameworks for research and treatment
- we’re in an uncomfortable space of justifying individual harm by group benefits
- and there are too many examples where human rights abuses and scientific research have overlapped
It’s also remarkably hard to find out whether treatments work
It’s also remarkably hard to find out whether treatments work
- as well as the ethical issue, we also have an epistemological issue
- people and diseases vary
- treatments often make small differences to outcomes
- multifactorial models of outcomes
- measurements are imperfect
- vested interests are powerful
- …
Ethics + epistemology give rise to strong demands on clinical trials
- ethical concerns (and others) suggest that we should be careful when designing clinical research
- otherwise the harm done to trial participants is just harm
- we need to find things out properly to obtain benefits
- this is why we want to minimise biases
The 1948 streptomycin trial
MRC 1948 as an example
- “Streptomycin Treatment of Pulmonary Tuberculosis” (1948)
- deals with treatment for a serious illness (TB) - there’s some useful context about what the general landscape of TB was like 75 years ago in Bastian (2006)
- very careful design, and reporting of that design
- interesting data viz - and some context for the trial itself in Crofton (2006)
E1: control in the streptomycin trial
TipTask
- look at the first two pages of the 1948 streptomycin trial
- please identify some control strategies used by the trial authors
Control strategies
- standardising the trial population
- controlling which treatment was used
- random allocation to S- or C-group
- masking
- standardised assessment
- following-up patients comprehensively
- statistical analysis
E2: would you take the treatment?
E2: would you take the treatment now?
E2: would you take the treatment now?
Generic RCT method
The heart of RCTs
E3: the rules of an RCT
- hopefully the last two examples give you a sense of some of the ‘rules’ of an RCT
- some rules about conduct
- some rules about reporting
TipTask
- read the StopAdvisor example on the GDS page about RCTs in digital health
- from that example, what do you think the main rules of RCTs are?
A note on randomising
- why do we randomise patients to groups?
- what might happen if we did not randomise?
- how could we randomise patients to groups?
E4: cheat your way through a drug trial
- imagine that you wanted to promote a new, but ineffective, drug (call it spurosium) - (Lundh et al. 2017)
- how could you bend these RCT rules?
- how could we guard against these rules being bent?
- how could you detect someone trying to bend the rules?
TipTask
- imagine you were an unscrupulous developer. How might you cheat the StopAdvisor trial to make your product look better?
A final thought
- RCTs are a really important source of E in EBM - but not the only one (Jefferson and Jørgensen 2018)
- we’ll encounter many kinds of evidence
- systematic reviews, meta-analyses, and other guidelines are increasingly important
- there are many instances where we’re unable to perform RCTs
- there are also many flawed RCTs out there
- that means that we’ll need a proper method for finding and using evidence
- it’s more complicated than “find a relevant RCT and act on it”
- that’s for next week…
- you might like to have a look at the core text for next time
Pick a paper
With thanks to the Knowledge Network team at NES, especially Derek Boyle and Alan Gillies, for identifying the following group of short and straightforward RCTS:
References
Abraham, Jens, Ramona Kupfer, Anja Behncke, et al. 2019. “Implementation of a Multicomponent Intervention to Prevent Physical Restraints in Nursing Homes (IMPRINT): A Pragmatic Cluster Randomized Controlled Trial.” International Journal of Nursing Studies 96 (August): 27–34. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_proquest_miscellaneous_2213925302.
Ali, Karim, Tanweer Azher, Mahin Baqi, et al. 2022. “Remdesivir for the Treatment of Patients in Hospital with COVID-19 in Canada: A Randomized Controlled Trial.” Canadian Medical Association Journal 194 (7): E242–51. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_8863204.
Bastian, Hilda. 2006. “Down and Almost Out in Scotland: George Orwell, Tuberculosis and Getting Streptomycin in 1948.” Journal of the Royal Society of Medicine 99 (2): 95–98. https://doi.org/10.1177/014107680609900226.
Chen-Hussey, Vanessa, Ilona Carneiro, Hongkham Keomanila, et al. 2013. “Can Topical Insect Repellents Reduce Malaria? A Cluster-Randomised Controlled Trial of the Insect Repellent n,n-Diethyl-m-Toluamide (DEET) in Lao PDR.” PLoS ONE 8 (8): e70664. https://doi.org/10.1371/journal.pone.0070664.
Chlibek, R., J. M. Bayas, H. Collins, et al. 2013. “Safety and Immunogenicity of an AS01-Adjuvanted Varicella-Zoster Virus Subunit Candidate Vaccine Against Herpes Zoster in Adults >=50 Years of Age.” Journal of Infectious Diseases 208 (12): 1953–61. https://doi.org/10.1093/infdis/jit365.
Clarke, M. 1996. “Early Controlled Trials...but "Quasirandom Allocation" of Treatment Was Reported in 1930.” BMJ 312 (7041): 1298–98. https://doi.org/10.1136/bmj.312.7041.1298a.
Crofton, J. 2006. “The MRC Randomized Trial of Streptomycin and Its Legacy: A View from the Clinical Front Line.” Journal of the Royal Society of Medicine 99 (10): 531–34. https://doi.org/10.1177/014107680609901017.
Gallien, Philippe, Gérard Amarenco, Nicolas Benoit, et al. 2014. “Cranberry Versus Placebo in the Prevention of Urinary Infections in Multiple Sclerosis: A Multicenter, Randomized, Placebo-Controlled, Double-Blind Trial.” Multiple Sclerosis Journal 20 (9): 1252–59. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_hal_primary_oai_HAL_hal_01147022v1.
Garbutt, James C., Alexei B. Kampov-Polevoy, Cort Pedersen, et al. 2021. “Efficacy and Tolerability of Baclofen in a u.s. Community Population with Alcohol Use Disorder: A Dose-Response, Randomized, Controlled Trial.” Neuropsychopharmacology 46 (13): 2250–56. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_unpaywall_primary_10_1038_s41386_021_01055_w .
Hrobjartsson, A., P. C Gotzsche, and C. Gluud. 1998. “The Controlled Clinical Trial Turns 100 Years: Fibiger’s Trial of Serum Treatment of Diphtheria.” BMJ 317 (7167): 1243–45. https://doi.org/10.1136/bmj.317.7167.1243.
Hughes, R E. 1975. “James Lind and the Cure of Scurvy: An Experimental Approach.” Medical History 19 (4): 342–51. https://doi.org/10.1017/s0025727300020469.
Jefferson, Tom, and Lars Jørgensen. 2018. “Redefining the ‘e’ in EBM.” BMJ Evidence-Based Medicine 23 (2): 46–47. https://doi.org/10.1136/bmjebm-2018-110918.
Lundh, Andreas, Joel Lexchin, Barbara Mintzes, Jeppe B Schroll, and Lisa Bero. 2017. “Industry Sponsorship and Research Outcome.” Cochrane Database of Systematic Reviews 2017 (2). https://doi.org/10.1002/14651858.mr000033.pub3.
Maddison, Ralph, Jonathan Charles Rawstorn, Ralph A H Stewart, et al. 2018. “Effects and Costs of Real-Time Cardiac Telerehabilitation: Randomised Controlled Non-Inferiority Trial.” Heart 105 (2): 122–29. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_unpaywall_primary_10_1136_heartjnl_2018_313189 .
Sackett, David L, William MC Rosenberg, JA Muir Gray, R Brian Haynes, and W Scott Richardson. 1996. “Evidence Based Medicine: What It Is and What It Isn’t.” British Medical Journal 312 (7023): 71–72. https://doi.org/10.1136/bmj.312.7023.71.
“Streptomycin Treatment of Pulmonary Tuberculosis: A Medical Research Council Investigation.” 1948. BMJ 2 (4582): 769–82. https://doi.org/10.1136/bmj.2.4582.769.
Tadrous, Mina, Kinwah Fung, Laura Desveaux, et al. 2020. “Effect of Academic Detailing on Promoting Appropriate Prescribing of Antipsychotic Medication in Nursing Homes.” JAMA Network Open 3 (5): e205724. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_7251442 .
Tapiainen, Terhi, Tiia Kujala, Marjo Renko, et al. 2014. “Effect of Antimicrobial Treatment of Acute Otitis Media on the Daily Disappearance of Middle Ear Effusion.” JAMA Pediatrics 168 (7): 635. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_proquest_miscellaneous_1544321708 .
Tonelli, Marcello, Natasha Wiebe, Stephanie Thompson, et al. 2015. “Trace Element Supplementation in Hemodialysis Patients: A Randomized Controlled Trial.” BMC Nephrology 16 (1). https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_unpaywall_primary_10_1186_s12882_015_0042_4 .