An introduction to EBM (session 3)

skills
beginner
evidence-based medicine
critical appraisal
clinical trials
Published

September 9, 2024

Session outline

  • this session gives an outline of a set of EBM methods
  • as we’ve seen, the methods of EBM have varied
  • this session gives a single consistent set of methods from one source
  • we’ll dodge most of the statistical content for now

E0: pick a paper

Task
  1. Please make sure you’ve got access to the core reading for this session (Greenhalgh 2019)
  2. Please pick an RCT from the list
  • Abraham et al. (2019)
  • Ali et al. (2022)
  • Chen-Hussey et al. (2013)
  • Chlibek et al. (2013)
  • Gallien et al. (2014)
  • Garbutt et al. (2021)
  • Maddison et al. (2018)
  • Tadrous et al. (2020)
  • Tapiainen et al. (2014)
  • Tonelli et al. (2015)

Exercises

  • E1: formulate
  • E2: beware of browsing
  • E3: three preliminary questions
  • E4: summing up
  • E5: aggregating evidence

Why appraise?

Why do we need this at all?

  • most published papers are bilge, nonsense, and drivel:

for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias (Ioannidis 2005)

  • that means we need effective, quick, and coherent ways of judging published research to answer questions

Spurious causes

  • Leibovici (2001) - see paper abstract
  • Objective: To determine whether remote, retroactive intercessory prayer, said for a group of patients with a bloodstream infection, has an effect on outcomes.
  • Design: Double blind, parallel group, randomised controlled trial of a retroactive intervention.
  • Setting: University hospital.
  • Subjects: All 3393 adult patients whose bloodstream infection was detected at the hospital in 1990-6.

Spurious causes

  • Intervention: In July 2000 patients were randomised to a control group and an intervention group. A remote, retroactive intercessory prayer was said for the well being and full recovery of the intervention group.
  • Main outcome measures: Mortality in hospital, length of stay in hospital, and duration of fever.
  • Results: Mortality was 28.1% (475/1691) in the intervention group and 30.2% (514/1702) in the control group (P for difference=0.4). Length of stay in hospital and duration of fever were significantly shorter in the intervention group than in the control group (P=0.01 and P=0.04, respectively).

These results, however, need to be interpreted with caution. There was no significant difference between the two groups with regard to the most clinically important outcome (mortality), and the median values varied little between prayer and non-prayer on both length of stay (seven and eight days) and duration of fever (two days each). The religious affiliation of the person saying the prayer is not given. Many religious groups do not accept the power of prayer given by those with different beliefs. If real, the effect of prayer shown in this study may be unrelated to supernatural power and hence to a particular belief system, or may be specific to beliefs, reflecting the power inherent in a particular religion. Further work is needed in this area before conclusions can be made. (Thornett 2002)

How to read a paper

Structure of the book

  • formulate (chapter 1)

  • retrieve (chapter 2)

  • appraise (chapter 3 - 15)

  • apply and evaluate (chapter 16, and other work)

  • we’ll mainly discuss appraisal today, with a side-order of formulate and retrieve, and we’ll neglect application and evaluation

Formulate

Formulate a research question

E1: formulate

Task
  1. From the paper that you selected for this session, we’d like you to work backwards, by formulating a question your paper could potentially answer
    1. Whom might this apply to?
    2. Which intervention (or group of interventions) would you know about?
    3. Which outcomes would you know about?
  2. Please be ready to share in the chat

Retrieve

  • contrast three kinds of searching:
    • browsing
    • looking
    • surveying
  • why might Greenhalgh worry about the effects of a casual internet search for evidence?
  • in EBM, we’re generally keen on either systematic looking, or surveying the literature

E2: beware of browsing

Task
  1. Search the topic covered in your selected paper
  2. In the chat, please let us know:
    1. Which evidence appears at the top of your search?
    2. If that evidence aligns with your selected paper?
  3. Can you link this strategy to our earlier concerns about bias?

Retrieve

  • we need an unbiased method for finding evidence
    • or, at least, a retrieval method that is biased in a consistent way
  • colleagues in The Knowledge Network have good specialist training and support available

Core sources

Appraise

  • core ideas:
    • “all evidence…is not necessarily equivalent”
    • methods give us insight into the likely quality of evidence
      • levels of evidence: we should judge evidence by methods, rather than the results obtained

Methods vs evidence

Traditional levels of evidence

Greenhalgh (2019)
  • rule of thumb
  • disagreements about specific rankings, and methods of ranking

E3: three preliminary questions

  1. Why was the study needed - and what was the research question?
  2. What was the research design?
  3. Was the research design appropriate to the question?
Task
  1. Please answer the three preliminary questions for your paper

Research design

Broad fields of research
  • treatment - RCT
  • diagnosis - cross-sectional
  • screening - cross-sectional
  • prognosis - longitudinal
  • causation - cohort / case-control / pluralism

Quality

  1. does it add anything to the literature?
  2. what was the trial population like?
  3. was the comparison between interventions fair and sensible?
  4. what were the outcomes?
  5. how was follow-up?

Intent-to-treat

  • Intent-to-treat is the principle of including all withdrawals as if they had completed the trial
  • Why is this important?

Summing up

An example

“This is a questionnaire survey of 963 general practitioners randomly selected from throughout the UK, in which they were asked their year of graduation from medical school and the level at which they would begin treatment for essential hypertension. Response options on the structured questionnaire were ‘below 89mmHg’, ‘90-99mmHg’ and ‘100mmHg or greater’.

E4: summing up

Task
  1. Greenhalgh gives some nice summary paragraphs (p. 58-9) showing:
    1. what sort of study was performed
    2. n, and where they came from
    3. what treatment(s) were compared
    4. the length and type of follow-up
    5. the outcome measure used
  2. Can you write one for your paper?

Aggregating evidence

  • meta-analysis = join together the results of several RCTs
  • systematic reviews = join together the results of all the RCTs

E5: aggregating evidence

Task
  1. Can you find a meta-analysis or systematic review for your topic?

References

Abraham, Jens, Ramona Kupfer, Anja Behncke, Birte Berger-Höger, Andrea Icks, Burkhard Haastert, Gabriele Meyer, Sascha Köpke, and Ralph Möhler. 2019. “Implementation of a Multicomponent Intervention to Prevent Physical Restraints in Nursing Homes (IMPRINT): A Pragmatic Cluster Randomized Controlled Trial.” International Journal of Nursing Studies 96 (August): 27–34. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_proquest_miscellaneous_2213925302.
Ali, Karim, Tanweer Azher, Mahin Baqi, Alexandra Binnie, Sergio Borgia, François M. Carrier, Yiorgos Alexandroa Cavayas, et al. 2022. “Remdesivir for the Treatment of Patients in Hospital with COVID-19 in Canada: A Randomized Controlled Trial.” Canadian Medical Association Journal 194 (7): E242–51. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_8863204.
Chen-Hussey, Vanessa, Ilona Carneiro, Hongkham Keomanila, Rob Gray, Sihamano Bannavong, Saysana Phanalasy, and Steven W. Lindsay. 2013. “Can Topical Insect Repellents Reduce Malaria? A Cluster-Randomised Controlled Trial of the Insect Repellent N,N-Diethyl-m-Toluamide (DEET) in Lao PDR.” Edited by Philip Bejon. PLoS ONE 8 (8): e70664. https://doi.org/10.1371/journal.pone.0070664.
Chlibek, R., J. M. Bayas, H. Collins, M. L. R. de la Pinta, E. Ledent, J. F. Mols, and T. C. Heineman. 2013. “Safety and Immunogenicity of an AS01-Adjuvanted Varicella-Zoster Virus Subunit Candidate Vaccine Against Herpes Zoster in Adults >=50 Years of Age.” Journal of Infectious Diseases 208 (12): 1953–61. https://doi.org/10.1093/infdis/jit365.
Gallien, Philippe, Gérard Amarenco, Nicolas Benoit, Véronique Bonniaud, Cécile Donzé, Jacques Kerdraon, Marianne de Seze, et al. 2014. “Cranberry Versus Placebo in the Prevention of Urinary Infections in Multiple Sclerosis: A Multicenter, Randomized, Placebo-Controlled, Double-Blind Trial.” Multiple Sclerosis Journal 20 (9): 1252–59. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_hal_primary_oai_HAL_hal_01147022v1.
Garbutt, James C., Alexei B. Kampov-Polevoy, Cort Pedersen, Melissa Stansbury, Robyn Jordan, Laura Willing, and Robert J. Gallop. 2021. “Efficacy and Tolerability of Baclofen in a U.S. Community Population with Alcohol Use Disorder: A Dose-Response, Randomized, Controlled Trial.” Neuropsychopharmacology 46 (13): 2250–56. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_unpaywall_primary_10_1038_s41386_021_01055_w .
Greenhalgh, Trisha. 2019. How to Read a Paper. Hoboken, NJ: Wiley-Blackwell. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/3dhhck/44NHSS_ALMA5146749970003491.
Ioannidis, John P. A. 2005. “Why Most Published Research Findings Are False.” PLoS Medicine 2 (8): e124. https://doi.org/10.1371/journal.pmed.0020124.
Leibovici, L. 2001. “Effects of Remote, Retroactive Intercessory Prayer on Outcomes in Patients with Bloodstream Infection: Randomised Controlled Trial.” BMJ 323 (7327): 1450–51. https://doi.org/10.1136/bmj.323.7327.1450.
Maddison, Ralph, Jonathan Charles Rawstorn, Ralph A H Stewart, Jocelyne Benatar, Robyn Whittaker, Anna Rolleston, Yannan Jiang, et al. 2018. “Effects and Costs of Real-Time Cardiac Telerehabilitation: Randomised Controlled Non-Inferiority Trial.” Heart 105 (2): 122–29. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_unpaywall_primary_10_1136_heartjnl_2018_313189 .
Tadrous, Mina, Kinwah Fung, Laura Desveaux, Tara Gomes, Monica Taljaard, Jeremy M. Grimshaw, Chaim M. Bell, and Noah M. Ivers. 2020. “Effect of Academic Detailing on Promoting Appropriate Prescribing of Antipsychotic Medication in Nursing Homes.” JAMA Network Open 3 (5): e205724. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_7251442 .
Tapiainen, Terhi, Tiia Kujala, Marjo Renko, Petri Koivunen, Tero Kontiokari, Aila Kristo, Tytti Pokka, Olli-Pekka Alho, and Matti Uhari. 2014. “Effect of Antimicrobial Treatment of Acute Otitis Media on the Daily Disappearance of Middle Ear Effusion.” JAMA Pediatrics 168 (7): 635. https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_proquest_miscellaneous_1544321708 .
Thornett, A. M. 2002. “Effect of Retroactive Intercessory Prayer.” BMJ 324 (7344): 1037–37. https://doi.org/10.1136/bmj.324.7344.1037.
Tonelli, Marcello, Natasha Wiebe, Stephanie Thompson, David Kinniburgh, Scott W Klarenbach, Michael Walsh, Aminu K Bello, et al. 2015. “Trace Element Supplementation in Hemodialysis Patients: A Randomized Controlled Trial.” BMC Nephrology 16 (1). https://nhs-scot-primo.hosted.exlibrisgroup.com/permalink/f/16btoc7/TN_cdi_unpaywall_primary_10_1186_s12882_015_0042_4 .