NIHR Signal Participation in health research may be linked to better care and performance

Published on 9 February 2016

This NIHR review found positive evidence that engaging clinicians and healthcare organisations in research is linked to improvements in the delivery of healthcare. In the central part of this three-stage review, most (28/33) papers found a positive impact of research on quality or outcomes.

Seven of these studies were able to show a relationship between research activity and patient outcomes. Other studies related research activity to the quality of care. In these, patients were more likely to get better standards of care at hospitals undertaking research.

There are several suggested ways in which this engagement might work and these probably all work together. The authors thought that the context in which research was undertaken was important. Good engagement seems to combine both formalised arrangements at organisation level and close working between researchers and clinicians day to day. This review points to possible concepts and models, some of which are already being used in the NHS. Despite the acknowledged limitations in the methods, the research does support innovative research models, such as participatory research, research networks and collaborative approaches such as the NIHR Collaborations for Leadership in Applied Heath Research and Care in England.

Participation in health research may be linked to better care and performance

Share your views on the research.

Why was this study needed?

There is a general assumption that engagement in research has a positive impact on healthcare performance. However, there is limited evidence from primary research or existing systematic reviews to support this assumption. Research engagement by practitioners and organisations have long been part of the drive for improving the quality and value of healthcare. This review aimed to look for the best available evidence and establish, as far as possible, whether research engagement by clinicians or organisations improves healthcare performance and to reveal the possible mechanisms behind the successes.

What did this study do?

This was an NIHR funded review in three parts. It involved a scoping part (to define the broad research areas), a systematic review part (focused on the question of whether research engagement improves healthcare performance) and a further part aimed to explore, qualitatively, the mechanisms through which improvement might be achieved.

These mixed methods were chosen as there is an acknowledged lack of research in this complex area. Following the scoping phase, the researchers counted the number of studies with positive, negative impacts or both. In the interpretive part the possible mechanisms were researched and explored. This produced some ideas for future action.

Broad and specific impacts of research engagement were measured in some trials, for example, as change in attitudes and behaviours or greater awareness of research findings. The outcome impacts of interest were related to reduced mortality. The systematic review used standard methodology with multiple database searches and quality assessment of the included studies using appropriate tools. Thirty-three studies were selected including randomised controlled trials, controlled before and after studies, qualitative studies and surveys. The studies were conducted in nine countries, with 15 papers from the US, five from Canada and three from the UK. The papers covered 15 clinical fields, including 10 cancer research papers and 6 cardiovascular studies.

As the underlying studies varied so much in how findings were reported, it was not possible to combine results quantitatively. The researchers provided an overview by counting the number of studies with positive, negative or mixed impacts. The researchers acknowledge that the review has limitations due to this approach, including the fact that there may have been some bias from the studies with positive results being preferentially published. They tried to counteract this by identifying unpublished literature from grey literature and key authors.

What did it find?

  • Over 10,000 records were identified by searching databases. Of these, 473 were assessed at full text and 33 papers included in the focussed review
  • Twenty-eight studies found that research engagement improved healthcare performance (six of these studies were positive/mixed). Seven of these studies reported improved healthcare outcomes for patients, the rest of the studies reported on improved processes of care.
  • Four of 33 studies included in the systematic review had the explicit intention to produce improvements in healthcare performance as a direct consequence of research engagement. In the remaining studies, the improvement was a by-product of research when the primary aim had been to test a specific therapy or approach. The mechanisms for improvement in these four studies included collaborative approaches and quality improvement initiatives at organisational level.
  • More than 80 papers were included in the later ‘wider’ interpretive review. This aimed to explore the numerous plausible mechanisms acting to achieve improvement. These included changes in clinicians’ attitudes and behaviours or the infrastructure created to support a particular trial, for example. Use of networks to provide more effective collaboration among clinicians and to create supportive contexts was found to be a common path for improvement of healthcare performance.

What does current guidance say on this issue?

The Department of Health mandate states that the NHS Commissioning Board has a duty to promote research and innovation. Its objective is to ensure that the commissioning system promotes and supports participation by NHS organisations and NHS patients in research and importantly the goal is to improve patient outcomes. NHS organisations are also required to meet benchmarking standards for delivery of trials to time and target recruitment levels. Health organisations are supported in this by clinical research networks in the UK. Further strategic support and coordination is provided by 15 Academic Health Science Networks in England and similar initiatives in other parts of the UK.

What are the implications?

Most of the improvements in the 33 included papers were process improvements (such as following best practice according to guidelines) but seven papers reported an improvement in a health outcome such as better survival.   

The mechanisms through which research engagement might improve healthcare performance overlap and rarely act in isolation. Their effectiveness often depends on the context in which they operate. The authors also note possible confounding effects in interpreting the relationship between research activity and outcomes.  For instance, organisations such as teaching hospitals which carry out many research trials, may have other institutional features associated with good patient care.    Identifying the actual mechanisms of achieving improvement in complex healthcare systems is not straightforward.  Given that many of the findings from this review are in some way already being implemented in the NHS, this review provides the best level of currently available supporting evidence to promote this collaborative research approach.

Citation and Funding

Boaz A, Hanney S, Jones T, et al. Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review. BMJ Open. 2015;5(12):e009415.

This project was funded by the National Institute for Health Research Service Delivery and Organisation (SDO) programme (project number HS&DR - 10/1012/09) now incorporated into the NIHR Health and Service Delivery (HS&DR) programme.

Bibliography

DH. Academic Health Science Networks: expression of interest. London: Department of Health; 2012.

DH. The mandate: a mandate from the Government to the NHS Commissioning Board: April 2013 to March 2015. London: Department of Health; 2013.

Hanney S, Boaz A, Jones T, et al. Engagement in research: an innovative three-stage review of the benefits for health-care performance. Southampton (UK): NIHR Journals Library; 2013.

Why was this study needed?

There is a general assumption that engagement in research has a positive impact on healthcare performance. However, there is limited evidence from primary research or existing systematic reviews to support this assumption. Research engagement by practitioners and organisations have long been part of the drive for improving the quality and value of healthcare. This review aimed to look for the best available evidence and establish, as far as possible, whether research engagement by clinicians or organisations improves healthcare performance and to reveal the possible mechanisms behind the successes.

What did this study do?

This was an NIHR funded review in three parts. It involved a scoping part (to define the broad research areas), a systematic review part (focused on the question of whether research engagement improves healthcare performance) and a further part aimed to explore, qualitatively, the mechanisms through which improvement might be achieved.

These mixed methods were chosen as there is an acknowledged lack of research in this complex area. Following the scoping phase, the researchers counted the number of studies with positive, negative impacts or both. In the interpretive part the possible mechanisms were researched and explored. This produced some ideas for future action.

Broad and specific impacts of research engagement were measured in some trials, for example, as change in attitudes and behaviours or greater awareness of research findings. The outcome impacts of interest were related to reduced mortality. The systematic review used standard methodology with multiple database searches and quality assessment of the included studies using appropriate tools. Thirty-three studies were selected including randomised controlled trials, controlled before and after studies, qualitative studies and surveys. The studies were conducted in nine countries, with 15 papers from the US, five from Canada and three from the UK. The papers covered 15 clinical fields, including 10 cancer research papers and 6 cardiovascular studies.

As the underlying studies varied so much in how findings were reported, it was not possible to combine results quantitatively. The researchers provided an overview by counting the number of studies with positive, negative or mixed impacts. The researchers acknowledge that the review has limitations due to this approach, including the fact that there may have been some bias from the studies with positive results being preferentially published. They tried to counteract this by identifying unpublished literature from grey literature and key authors.

What did it find?

  • Over 10,000 records were identified by searching databases. Of these, 473 were assessed at full text and 33 papers included in the focussed review
  • Twenty-eight studies found that research engagement improved healthcare performance (six of these studies were positive/mixed). Seven of these studies reported improved healthcare outcomes for patients, the rest of the studies reported on improved processes of care.
  • Four of 33 studies included in the systematic review had the explicit intention to produce improvements in healthcare performance as a direct consequence of research engagement. In the remaining studies, the improvement was a by-product of research when the primary aim had been to test a specific therapy or approach. The mechanisms for improvement in these four studies included collaborative approaches and quality improvement initiatives at organisational level.
  • More than 80 papers were included in the later ‘wider’ interpretive review. This aimed to explore the numerous plausible mechanisms acting to achieve improvement. These included changes in clinicians’ attitudes and behaviours or the infrastructure created to support a particular trial, for example. Use of networks to provide more effective collaboration among clinicians and to create supportive contexts was found to be a common path for improvement of healthcare performance.

What does current guidance say on this issue?

The Department of Health mandate states that the NHS Commissioning Board has a duty to promote research and innovation. Its objective is to ensure that the commissioning system promotes and supports participation by NHS organisations and NHS patients in research and importantly the goal is to improve patient outcomes. NHS organisations are also required to meet benchmarking standards for delivery of trials to time and target recruitment levels. Health organisations are supported in this by clinical research networks in the UK. Further strategic support and coordination is provided by 15 Academic Health Science Networks in England and similar initiatives in other parts of the UK.

What are the implications?

Most of the improvements in the 33 included papers were process improvements (such as following best practice according to guidelines) but seven papers reported an improvement in a health outcome such as better survival.   

The mechanisms through which research engagement might improve healthcare performance overlap and rarely act in isolation. Their effectiveness often depends on the context in which they operate. The authors also note possible confounding effects in interpreting the relationship between research activity and outcomes.  For instance, organisations such as teaching hospitals which carry out many research trials, may have other institutional features associated with good patient care.    Identifying the actual mechanisms of achieving improvement in complex healthcare systems is not straightforward.  Given that many of the findings from this review are in some way already being implemented in the NHS, this review provides the best level of currently available supporting evidence to promote this collaborative research approach.

Citation and Funding

Boaz A, Hanney S, Jones T, et al. Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review. BMJ Open. 2015;5(12):e009415.

This project was funded by the National Institute for Health Research Service Delivery and Organisation (SDO) programme (project number HS&DR - 10/1012/09) now incorporated into the NIHR Health and Service Delivery (HS&DR) programme.

Bibliography

DH. Academic Health Science Networks: expression of interest. London: Department of Health; 2012.

DH. The mandate: a mandate from the Government to the NHS Commissioning Board: April 2013 to March 2015. London: Department of Health; 2013.

Hanney S, Boaz A, Jones T, et al. Engagement in research: an innovative three-stage review of the benefits for health-care performance. Southampton (UK): NIHR Journals Library; 2013.

Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review

Published on 9 December 2015

Boaz A, Hanney S, Jones T, Soper B

BMJ Open , 2015

Objective There is a widely held assumption that engagement by clinicians and healthcare organisations in research improves healthcare performance at various levels, but little direct empirical evidence has previously been collated. The objective of this study was to address the question: Does research engagement (by clinicians and organisations) improve healthcare performance? Methods An hourglass-shaped review was developed, consisting of three stages: (1) a planning and mapping stage; (2) a focused review concentrating on the core question of whether or not research engagement improves healthcare performance; and (3) a wider (but less systematic) review of papers identified during the two earlier stages, focusing on mechanisms. Results Of the 33 papers included in the focused review, 28 identified improvements in health services performance. Seven out of these papers reported some improvement in health outcomes, with others reporting improved processes of care. The wider review demonstrated that mechanisms such as collaborative and action research can encourage some progress along the pathway from research engagement towards improved healthcare performance. Organisations that have deliberately integrated the research function into organisational structures demonstrate how research engagement can, among other factors, contribute to improved healthcare performance. Conclusions Current evidence suggests that there is an association between the engagement of individuals and healthcare organisations in research and improvements in healthcare performance. The mechanisms through which research engagement might improve healthcare performance overlap and rarely act in isolation, and their effectiveness often depends on the context in which they operate.

In this study ‘Engagement in research’ was a deliberate set of intellectual and practical activities undertaken by healthcare staff (including conducting research) and organisations (including playing an active role in research networks, partnerships or collaborations). Practical measures of this were such things as the number of registered trials, publications or affiliations to research networks.

‘Healthcare performance’ was understood to reflect the consequences of clinical activity, and was primarily taken to mean improvements in the processes and/or outcomes of care, rather than other measures of healthcare performance such as efficiency.

‘Mechanisms’ were seen in relatively simple terms as levers that instigate and sustain activity, for example, research collaborations between researchers and healthcare staff who are potential users of the findings.

Expert commentary

Clinical research provides the evidence base for improving healthcare.  In addition to this, it is suggested that organisations and clinicians that engage in research improve their overall performance because of that engagement, generating additional benefits for patients and services. However, it is difficult to obtain hard evidence to support this view. The question does not lend itself to controlled trials. The present review rigorously and critically examined all of the evidence available. The conclusions are encouraging and may help to promote research participation as one of the mechanisms by which a research intensive healthcare system brings benefits. However, there is a need for caution. The studies describe associations and it is often difficult to infer causative links confidently. Large, carefully designed quantitative studies are still needed to add confidence to the conclusions.

Professor Peter Selby, Director – NIHR Diagnostic Evidence Cooperative at Leeds

Expert commentary

Boaz and colleagues have uncovered and organised a great deal of valuable information; both what studies in the literature show and where analysis and evidence is lacking. While research engagement is generally found to support better care for patients at both individual clinician and organisation levels, there is much still to do to understand how this happens, which types of engagement are most beneficial to patient care and which policies and practices might improve on that impact. They have not only increased our understanding but pointed to where future research on this subject can be sure to add value.

Jon Sussex, Chief Economist RAND Europe, Co-Director of the Cambridge Centre for Health Services Research (CCHSR)

Expert commentary

Healthcare organisations strive to improve patient outcomes, safety and the patient experience and system efficiency. Current approaches focus on implementing quality improvement methodologies supported by strong clinical and managerial leaders. Professional development of health care teams is recognised as a key factor in improving the quality of healthcare. Engagement requires development of new team processes, systematic collection of patient data, and networking with other clinicians. Such engagement often challenges clinician views on best practice.

More rapid uptake of research innovations may also contribute to better outcomes. But the subtle impact on professional team culture and behaviour may be a more powerful mechanism. Medical research saves lives; perhaps through mechanisms not always recognised by research and healthcare funders.

Professor Gary Ford, Chief Executive Officer, Oxford Academic Health Science Network