NIHR DC Discover

NIHR Signal Quality improvement collaboratives can improve clinical processes and patient outcomes

Published on 6 February 2018

doi: 10.3310/signal-000546

Quality improvement collaboratives (QICs) were largely effective across a wide range of healthcare problems and settings. Of the 64 studies included in this systematic review, 53 showed improvement in some of the healthcare processes and patient outcomes that they investigated. A small number of studies also showed that collaboratives were cost-effective, and the improvements were sustainable for at least six months. 

Collaboratives originated in the US in the late 1980s. They provide opportunities for teams from different sites to learn from each other and jointly develop improvement initiatives around specific topics, often using a standard method and framework. They have become increasingly popular without strong evidence to support their use.

This well-conducted review goes some way to addressing that gap, although it highlights the limitations in the design of the underlying studies and the quality of the reporting of many collaboratives.

Share your views on the research.

Why was this study needed?

QICs bring together teams from multiple healthcare sites to learn from, share with and motivate each other to improve the quality of health services on a particular topic. Collaboratives use a structured approach, involving target-setting and rapid change cycles, supported by experts.

The theory is that teams learn faster and are more effective when collaborating and benchmarking with other teams. “Collaboratives” are resource intensive processes, regarding funding and staff time. They have increased in number and popularity over the past 20 years, despite there being little evidence about whether they are effective, cost-effective or result in long-term change.

This study aimed to review the evidence about the impact of collaboratives, looking at whether they lead to improvements in healthcare delivery and outcomes, and whether those improvements are sustained for six months or more.

What did this study do?

This systematic review identified a large number of possible studies (over 4,000) from 1995. Researchers carefully excluded many studies, including those without clear data on effectiveness and those that failed to meet strict quality inclusion criteria.

The review finally included 64 studies that reported on healthcare processes and/or patient outcomes from a QICs intervention. A range of study topics, designs and settings were found.

Ten were cluster randomised controlled trials, 24 were controlled before-after studies, and 30 were interrupted time series studies. Thirty-nine studies were carried out in acute hospitals, covering a wide variety of topics. Twenty studies took place in ambulatory care or general practice, most related to managing chronic conditions. Four studies were conducted in nursing homes, looking at pain management, falls prevention and pressure ulcers. One study took place in an ambulance service, covering pre-hospital care for heart attack and stroke.

The review was well-conducted but was limited by the quality of the included studies. The tendency to report positive findings is likely to have resulted in some publications bias. Only six of the studies were from the UK.

What did it find?

Results were based on “primary effect measures” in the studies, which were healthcare processes or patient outcomes:

  • When looking at the results by care setting, 53 out of 64 (83%) of the studies found an improvement for one or more of the study’s primary effect measures.
  • The authors also used a more conservative definition of effectiveness and looked for statistically significant differences in at least half of the primary effect measures in each study. Using this definition, 73% of the studies showed improvements as a result of the collaborative.
  • Only four studies reported on cost effectiveness. All had positive findings.
  • Eight studies continued to collect data for at least six months after the intervention study period. All found that the improvements were sustained.

What does current guidance say on this issue?

The Health Foundation produced a report in 2014 to guide collaboratives in the NHS. They recommend four main components:

  • Focus on who should be included, aiming for multidisciplinary groups but not making participation mandatory as this may reduce enthusiasm.
  • Make sure there is a realistic goal on a specific topic that requires a team approach.
  • Have a structured plan for the collaborative programme, including clear targets, good communication channels, flexibility for tailoring to local needs, support and coaching.
  • Allocate sufficient resources such as IT infrastructure and the ability to collect, measure and report outcomes.

What are the implications?

The findings of this review suggest that collaboratives can be effective in improving healthcare processes and patient outcomes. This appears to be the case across a range of healthcare problems and settings.

Although they are complex and time-consuming interventions that require investment, a small number of studies have shown that they can be cost-effective and produce sustained change.

As promising approaches, the review has highlighted limitations in the design and quality of reporting of QICs. This evidence might inform better studies that could usefully focus on what characteristics of QICs determine their success. Further guidance on the use of QICs in the UK might improve the uptake of successful examples here.

Citation and Funding

Wells S, Tamir O, Gray J, et al. Are quality improvement collaboratives effective? A systematic review. BMJ Quality & Safety. 2017. [Epub ahead of print].

No funding was provided for this study.

Bibliography

de Silva, D. Improvement collaboratives in health care: Evidence scan. London: The Health Foundation; 2014.

IHI. The Breakthrough Series: IHI’s collaborative model for achieving breakthrough improvement. IHI Innovation Series white paper. Boston: Institute for Healthcare Improvement; 2003.

Why was this study needed?

QICs bring together teams from multiple healthcare sites to learn from, share with and motivate each other to improve the quality of health services on a particular topic. Collaboratives use a structured approach, involving target-setting and rapid change cycles, supported by experts.

The theory is that teams learn faster and are more effective when collaborating and benchmarking with other teams. “Collaboratives” are resource intensive processes, regarding funding and staff time. They have increased in number and popularity over the past 20 years, despite there being little evidence about whether they are effective, cost-effective or result in long-term change.

This study aimed to review the evidence about the impact of collaboratives, looking at whether they lead to improvements in healthcare delivery and outcomes, and whether those improvements are sustained for six months or more.

What did this study do?

This systematic review identified a large number of possible studies (over 4,000) from 1995. Researchers carefully excluded many studies, including those without clear data on effectiveness and those that failed to meet strict quality inclusion criteria.

The review finally included 64 studies that reported on healthcare processes and/or patient outcomes from a QICs intervention. A range of study topics, designs and settings were found.

Ten were cluster randomised controlled trials, 24 were controlled before-after studies, and 30 were interrupted time series studies. Thirty-nine studies were carried out in acute hospitals, covering a wide variety of topics. Twenty studies took place in ambulatory care or general practice, most related to managing chronic conditions. Four studies were conducted in nursing homes, looking at pain management, falls prevention and pressure ulcers. One study took place in an ambulance service, covering pre-hospital care for heart attack and stroke.

The review was well-conducted but was limited by the quality of the included studies. The tendency to report positive findings is likely to have resulted in some publications bias. Only six of the studies were from the UK.

What did it find?

Results were based on “primary effect measures” in the studies, which were healthcare processes or patient outcomes:

  • When looking at the results by care setting, 53 out of 64 (83%) of the studies found an improvement for one or more of the study’s primary effect measures.
  • The authors also used a more conservative definition of effectiveness and looked for statistically significant differences in at least half of the primary effect measures in each study. Using this definition, 73% of the studies showed improvements as a result of the collaborative.
  • Only four studies reported on cost effectiveness. All had positive findings.
  • Eight studies continued to collect data for at least six months after the intervention study period. All found that the improvements were sustained.

What does current guidance say on this issue?

The Health Foundation produced a report in 2014 to guide collaboratives in the NHS. They recommend four main components:

  • Focus on who should be included, aiming for multidisciplinary groups but not making participation mandatory as this may reduce enthusiasm.
  • Make sure there is a realistic goal on a specific topic that requires a team approach.
  • Have a structured plan for the collaborative programme, including clear targets, good communication channels, flexibility for tailoring to local needs, support and coaching.
  • Allocate sufficient resources such as IT infrastructure and the ability to collect, measure and report outcomes.

What are the implications?

The findings of this review suggest that collaboratives can be effective in improving healthcare processes and patient outcomes. This appears to be the case across a range of healthcare problems and settings.

Although they are complex and time-consuming interventions that require investment, a small number of studies have shown that they can be cost-effective and produce sustained change.

As promising approaches, the review has highlighted limitations in the design and quality of reporting of QICs. This evidence might inform better studies that could usefully focus on what characteristics of QICs determine their success. Further guidance on the use of QICs in the UK might improve the uptake of successful examples here.

Citation and Funding

Wells S, Tamir O, Gray J, et al. Are quality improvement collaboratives effective? A systematic review. BMJ Quality & Safety. 2017. [Epub ahead of print].

No funding was provided for this study.

Bibliography

de Silva, D. Improvement collaboratives in health care: Evidence scan. London: The Health Foundation; 2014.

IHI. The Breakthrough Series: IHI’s collaborative model for achieving breakthrough improvement. IHI Innovation Series white paper. Boston: Institute for Healthcare Improvement; 2003.

Are quality improvement collaboratives effective? A systematic review

Published on 23 October 2017

Wells, S.,Tamir, O.,Gray, J.,Naidoo, D.,Bekhit, M.,Goldmann, D.

BMJ Qual Saf , 2017

BACKGROUND: Quality improvement collaboratives (QIC) have proliferated internationally, but there is little empirical evidence for their effectiveness. METHOD: We searched Medline, Embase, CINAHL, PsycINFO and the Cochrane Library databases from January 1995 to December 2014. Studies were included if they met the criteria for a QIC intervention and the Cochrane Effective Practice and Organisation of Care (EPOC) minimum study design characteristics for inclusion in a review. We assessed study bias using the EPOC checklist and the quality of the reported intervention using a subset of SQUIRE 1.0 standards. RESULTS: Of the 220 studies meeting QIC criteria, 64 met EPOC study design standards for inclusion. There were 10 cluster randomised controlled trials, 24 controlled before-after studies and 30 interrupted time series studies. QICs encompassed a broad range of clinical settings, topics and populations ranging from neonates to the elderly. Few reports fully described QIC implementation and methods, intensity of activities, degree of site engagement and important contextual factors. By care setting, an improvement was reported for one or more of the study's primary effect measures in 83% of the studies (32/39 (82%) hospital based, 17/20 (85%) ambulatory care, 3/4 nursing home and a sole ambulance QIC). Eight studies described persistence of the intervention effect 6 months to 2 years after the end of the collaborative. Collaboratives reporting success generally addressed relatively straightforward aspects of care, had a strong evidence base and noted a clear evidence-practice gap in an accepted clinical pathway or guideline. CONCLUSIONS: QICs have been adopted widely as an approach to shared learning and improvement in healthcare. Overall, the QICs included in this review reported significant improvements in targeted clinical processes and patient outcomes. These reports are encouraging, but most be interpreted cautiously since fewer than a third met established quality and reporting criteria, and publication bias is likely.

The authors of this review used a set of five core elements to define a quality improvement collaborative. The intervention had to be focused on a specific topic. It had to use a group of clinical and quality improvement experts to bring together evidence, contextual knowledge and quality improvement methods. Multiple teams from multiple sites had to be involved. There had to be a model or framework for improvement that included measurable aims, data collection, implementation and evaluation of changes. And there had to be structured activities that promoted the process of learning and sharing (such as meetings, visits, coaching and feedback).

Expert commentary

QICs may not be familiar to most people, but they fulfil an important, if largely unseen, function in modern health care.

Improving performance and encouraging innovation at pace and scale in a sustainable manner have proved elusive policy objectives. They are examples of ‘wicked problems’ to which there is no magic formula or simple solution. Reliable evidence of what works is hard to produce.

Although studies do suggest QICs can be effective, they require novel approaches to research design if there is to be a more robust and convincing assessment of their adoption and impact.

David Hunter, Professor of Health Policy & Management, Newcastle University