NIHR DC Discover

NIHR Signal Interventions that help and harm patients in the critical care unit

Published on 31 March 2015

doi: 10.3310/signal-000098

This systematic review and expert consensus process found 15 non-surgical interventions affecting adult mortality in critical care. Seven interventions, such as the use of tranexamic acid after severe blood loss, reduced deaths. Eight interventions, such as improving oxygen supply by using a drug, dobutamine, increased them. 2015 UK guidelines only partly reflect these findings. They may need amending to more explicitly identify interventions considered beneficial and harmful in critical care. This interesting study looked at a number of interventions in a particular clinical setting and asked the views of over 500 experts.

The review also found clinicians around the world were not necessarily acting in accordance with the evidence, highlighting an important gap between the evidence and practice. The NIHR Critical Care Clinical Research Network is actively promoting development and use of high quality research.

Share your views on the research.

Why was this study needed?

Critical care is resource intensive and critically ill patients have high mortality rates or high risk of poor long-term outcomes. Research into interventions affecting mortality could save many lives and reduce the high costs of critical care if implemented effectively. However, the extent to which evidence was informing the care of critically ill people was not clear. This research aimed to see whether clinical experts were using evidence on effective and ineffective interventions to inform what they did in clinical practice.

Before this study there was a lack of research on whether critical care trials showing benefits from interventions are designed and implemented differently from trials showing harm. This knowledge would help show how research uptake should change.

What did this study do?

This mixed-methods study started with a systematic literature review of randomised controlled trials (RCTs) on mortality in critical care. Next, the researchers hosted a meeting of experts to prioritise the best quality studies on the most relevant interventions. Finally, an online survey of 555 experts from 61 countries examined their agreement with the systematic review findings. The survey also asked how experts would use the research findings on beneficial and harmful interventions in practice.

The study design contained many aspects that made it reliable, such as including only multi-centre RCTs. However, there may be more recent trials and the online survey of experts may not be representative of all critical care health professionals. Also, the number needed to treat and number needed to harm statistics should be treated with caution as they were pooled across trials.

What did it find?

Based on 24 multi-centre RCTs:

  • Seven interventions decreased mortality. These included: non-invasive positive pressure ventilation, compared with an oxygen mask, for people with chronic obstructive pulmonary disease in acute respiratory failure; lying chest down for people with severe acute respiratory distress syndrome; active cooling of a person (mild hypothermia) after a heart attack; and treating severe blood loss with the drug tranexamic acid. The average (median) number needed to treat in these trials was 7 (interquartile range, 5-8).
  • Eight interventions increased mortality. These included using a haemoglobin-based blood substitute for people with severe, trauma related blood loss, and for critically ill people administering growth hormone or raising oxygen supply to organs by using dobutamine. The average (median) number needed to harm in these trials was 9 (interquartile range, 5-16).
  • The average size of the RCTs was small, involving few centres. RCTs reporting harm were larger, involved more centres, and were more likely to contain blinding compared to those reporting benefits.
  • Experts' views varied on findings from the review and consensus meeting: 81% agreed on interventions that lowered mortality and 82% agreed on interventions that increased it. Of those agreeing with the findings, only 71% followed them in practice.

What does current guidance say on this issue?

The Intensive Care Society has published several guidelines, including the detailed 2015 Guidelines for the Provision of Intensive Care Services. Three of the beneficial interventions are recommended in these guidelines: non-invasive ventilation, lying prone, and ventilation using small volumes of air for acute respiratory distress syndrome. Of the harmful interventions, the guidelines say that two should be avoided or cause harm. These are: hydroxyethyl starch for severe infection, and rapid frequency ventilation. The other six are not mentioned in the recommendations.

Another of the beneficial interventions, mild hypothermia after heart attack, has been recommended by NICE since 2011. However, the authors of this study say there is new evidence that questions the benefit of this intervention.

What are the implications?

This study's findings suggest that 2015 UK guidelines may need amending to reflect this clinical evidence and expert consensus specifying interventions considered beneficial and harmful in critical care.

The finding that clinicians around the world are not necessarily acting in accordance with the evidence highlights the important gap between the evidence and practice. It suggests that critical care practitioners need to engage with the evidence base, and incorporate that knowledge into their practice. In the UK, the NIHR Critical Care Clinical Research Network is actively promoting the development and use of high quality research.

Citation

Landoni G, Comis M, Conte M, et al. Mortality in multicenter critical care trials: an analysis of interventions with a significant effect. Critical Care Medicine. 2015.

Bibliography

British Thoracic Society, Royal College of Physicians, The Intensive Care Society. The use of non-invasive ventilation in the management of patients with chronic obstructive pulmonary disease admitted to hospital with acute type II respiratory failure (with particular reference to Bilevel positive pressure ventilation). London: British Thoracic Society, Royal College of Physicians, The Intensive Care Society; 2008.

Clinical Research Network. Critical Care [homepage on the internet]. London: National Institute for Health Reseach; 2015

The Faculty of Intensive Care Medicine. Core standards for intensive care units. London: The Faculty of Intensive Care Medicine; 2013.

The Intensive Care Society. Guidelines for the provision of intensive care services. London: The Intensive Care Society; 2015.

Institute for Healthcare Improvement. Evidence based care bundles. Cambridge (MA): Institute for Healthcare Improvement; 2015.

NICE. Therapeutic hypothermia following cardiac arrest. Interventional procedure guidance 386. London: National Institute for Health and Care Excellence; 2011.

Tidy C. Numbers needed to treat. PatientPlus [internet]. London: EMIS Group; 2012.

Why was this study needed?

Critical care is resource intensive and critically ill patients have high mortality rates or high risk of poor long-term outcomes. Research into interventions affecting mortality could save many lives and reduce the high costs of critical care if implemented effectively. However, the extent to which evidence was informing the care of critically ill people was not clear. This research aimed to see whether clinical experts were using evidence on effective and ineffective interventions to inform what they did in clinical practice.

Before this study there was a lack of research on whether critical care trials showing benefits from interventions are designed and implemented differently from trials showing harm. This knowledge would help show how research uptake should change.

What did this study do?

This mixed-methods study started with a systematic literature review of randomised controlled trials (RCTs) on mortality in critical care. Next, the researchers hosted a meeting of experts to prioritise the best quality studies on the most relevant interventions. Finally, an online survey of 555 experts from 61 countries examined their agreement with the systematic review findings. The survey also asked how experts would use the research findings on beneficial and harmful interventions in practice.

The study design contained many aspects that made it reliable, such as including only multi-centre RCTs. However, there may be more recent trials and the online survey of experts may not be representative of all critical care health professionals. Also, the number needed to treat and number needed to harm statistics should be treated with caution as they were pooled across trials.

What did it find?

Based on 24 multi-centre RCTs:

  • Seven interventions decreased mortality. These included: non-invasive positive pressure ventilation, compared with an oxygen mask, for people with chronic obstructive pulmonary disease in acute respiratory failure; lying chest down for people with severe acute respiratory distress syndrome; active cooling of a person (mild hypothermia) after a heart attack; and treating severe blood loss with the drug tranexamic acid. The average (median) number needed to treat in these trials was 7 (interquartile range, 5-8).
  • Eight interventions increased mortality. These included using a haemoglobin-based blood substitute for people with severe, trauma related blood loss, and for critically ill people administering growth hormone or raising oxygen supply to organs by using dobutamine. The average (median) number needed to harm in these trials was 9 (interquartile range, 5-16).
  • The average size of the RCTs was small, involving few centres. RCTs reporting harm were larger, involved more centres, and were more likely to contain blinding compared to those reporting benefits.
  • Experts' views varied on findings from the review and consensus meeting: 81% agreed on interventions that lowered mortality and 82% agreed on interventions that increased it. Of those agreeing with the findings, only 71% followed them in practice.

What does current guidance say on this issue?

The Intensive Care Society has published several guidelines, including the detailed 2015 Guidelines for the Provision of Intensive Care Services. Three of the beneficial interventions are recommended in these guidelines: non-invasive ventilation, lying prone, and ventilation using small volumes of air for acute respiratory distress syndrome. Of the harmful interventions, the guidelines say that two should be avoided or cause harm. These are: hydroxyethyl starch for severe infection, and rapid frequency ventilation. The other six are not mentioned in the recommendations.

Another of the beneficial interventions, mild hypothermia after heart attack, has been recommended by NICE since 2011. However, the authors of this study say there is new evidence that questions the benefit of this intervention.

What are the implications?

This study's findings suggest that 2015 UK guidelines may need amending to reflect this clinical evidence and expert consensus specifying interventions considered beneficial and harmful in critical care.

The finding that clinicians around the world are not necessarily acting in accordance with the evidence highlights the important gap between the evidence and practice. It suggests that critical care practitioners need to engage with the evidence base, and incorporate that knowledge into their practice. In the UK, the NIHR Critical Care Clinical Research Network is actively promoting the development and use of high quality research.

Citation

Landoni G, Comis M, Conte M, et al. Mortality in multicenter critical care trials: an analysis of interventions with a significant effect. Critical Care Medicine. 2015.

Bibliography

British Thoracic Society, Royal College of Physicians, The Intensive Care Society. The use of non-invasive ventilation in the management of patients with chronic obstructive pulmonary disease admitted to hospital with acute type II respiratory failure (with particular reference to Bilevel positive pressure ventilation). London: British Thoracic Society, Royal College of Physicians, The Intensive Care Society; 2008.

Clinical Research Network. Critical Care [homepage on the internet]. London: National Institute for Health Reseach; 2015

The Faculty of Intensive Care Medicine. Core standards for intensive care units. London: The Faculty of Intensive Care Medicine; 2013.

The Intensive Care Society. Guidelines for the provision of intensive care services. London: The Intensive Care Society; 2015.

Institute for Healthcare Improvement. Evidence based care bundles. Cambridge (MA): Institute for Healthcare Improvement; 2015.

NICE. Therapeutic hypothermia following cardiac arrest. Interventional procedure guidance 386. London: National Institute for Health and Care Excellence; 2011.

Tidy C. Numbers needed to treat. PatientPlus [internet]. London: EMIS Group; 2012.

Mortality in Multicenter Critical Care Trials: An Analysis of Interventions With a Significant Effect

Published on 31 March 2015

Landoni, G.,Comis, M.,Conte, M.,Finco, G.,Mucchetti, M.,Paternoster, G.,Pisano, A.,Ruggeri, L.,Alvaro, G.,Angelone, M.,Bergonzi, P. C.,Bocchino, S.,Borghi, G.,Bove, T.,Buscaglia, G.,Cabrini, L.,Callegher, L.,Caramelli, F.,Colombo, S.,Corno, L.,Del Sarto, P.,Feltracco, P.,Forti, A.,Ganzaroli, M.,Greco, M.,Guarracino, F.,Lembo, R.,Lobreglio, R.,Meroni, R.,Monaco, F.,Musu, M.,Pala, G.,Pasin, L.,Pieri, M.,Pisarra, S.,Ponticelli, G.,Roasio, A.,Santini, F.,Silvetti, S.,Szekely, A.,Zambon, M.,Zucchetti, M. C.,Zangrillo, A.,Bellomo, R.

Crit Care Med , 2015

OBJECTIVES: We aimed to identify all treatments that affect mortality in adult critically ill patients in multicenter randomized controlled trials. We also evaluated the methodological aspects of these studies, and we surveyed clinicians' opinion and usual practice for the selected interventions. DATA SOURCES: MEDLINE/PubMed, Scopus, and Embase were searched. Further articles were suggested for inclusion from experts and cross-check of references. STUDY SELECTION: We selected the articles that fulfilled the following criteria: publication in a peer-reviewed journal; multicenter randomized controlled trial design; dealing with nonsurgical interventions in adult critically ill patients; and statistically significant effect in unadjusted landmark mortality. A consensus conference assessed all interventions and excluded those with lack of reproducibility, lack of generalizability, high probability of type I error, major baseline imbalances between intervention and control groups, major design flaws, contradiction by subsequent larger higher quality trials, modified intention to treat analysis, effect found only after adjustments, and lack of biological plausibility. DATA EXTRACTION: For all selected studies, we recorded the intervention and its comparator, the setting, the sample size, whether enrollment was completed or interrupted, the presence of blinding, the effect size, and the duration of follow-up. DATA SYNTHESIS: We found 15 interventions that affected mortality in 24 multicenter randomized controlled trials. Median sample size was small (199 patients) as was median centers number (10). Blinded trials enrolled significantly more patients and involved more centers. Multicenter randomized controlled trials showing harm also involved significantly more centers and more patients (p = 0.016 and p = 0.04, respectively). Five hundred fifty-five clinicians from 61 countries showed variable agreement on perceived validity of such interventions. CONCLUSIONS: We identified 15 treatments that decreased/increased mortality in critically ill patients in 24 multicenter randomized controlled trials. However, design affected trial size and larger trials were more likely to show harm. Finally, clinicians view of such trials and their translation into practice varied.

Number needed to treat (NNT) is the number of patients who need to be treated to prevent one additional bad outcome, relative to a control group. 

Number needed to harm (NNH) is the number of people who are exposed to a harmful intervention before one person suffers harm, relative to a control group.

Pooled NNTs (or NNHs) derived from meta-analyses can be misleading because the baseline risk of an event often varies appreciably between the different trials. This explains why the study authors have only reported the average (median) results across the included trials.

Commentary

Critical care is expensive for any health system and is associated with high mortality and poor outcomes for patients. This review provides an international snapshot of important multi-centre studies. It also reveals the gap to be closed to implement evidence based practice. While there are limitations to the review, such as omission of recent large trials, patient reported outcomes and data on costs, it has nevertheless provoked interest and discussion. This will increase critical care practitioners' engagement with research and ultimately help develop the best evidence for improved critical care practice, which is the mission of NIHR's Critical Care Clinical Research Network in the UK.

Dr Paul Dark, NIHR CRN National Theme Lead for Critical Care King's College London, & Honorary NHS Consultant in Critical Care Medicine, Salford Royal NHS Foundation Trust

Commentary

This analysis provides a credible list of beneficial (and harmful) non-surgical interventions in critically ill adult patients. For clinical practice, these interventions are well accepted although recent literature has added uncertainty in one area (therapeutic hypothermia after cardiac arrest). The small number of studies highlights limited investment in evaluating interventions in this area that consumes a substantial amount of healthcare resource. There are very large (>1000 patient) trials in unselected critically ill patients, but the included studies are generally small with more clearly defined clinical problems. To date, targeted complex interventions have been more successful than medicines in critical illness.

Mike Grocott, Professor of Anaesthesia and Critical Care Medicine, University of Southampton

Commentary

This review shines a light on the fundamentals of evidence based medicine in the critical care unit. Twenty four randomised trials evaluating 15 interventions which reported benefit or harm were reviewed. There was a relationship between trial size and effect – the smaller the trial the larger the reported effect. Larger, high quality studies were more likely to report harm. Implementation of findings was variable. The study highlights the importance of large, high quality, multi-centre trials and of ensuring the findings of trials showing harm are translated into practice with greater confidence.

Gavin Perkins, Professor of Critical Care Medicine, Warwick Medical School and Heart of England NHS Foundation Trust