NIHR DC Discover

NIHR Signal Introducing a primary care risk prediction tool did not reduce emergency admissions

Published on 20 February 2018

doi: 10.3310/signal-000557

Predicting emergency admissions paradoxically increased hospital admissions from primary care across all risk groups by about 3% overall.

The Predictive Risk Stratification Model (PRISM) was evaluated in a trial in general practices in Wales, and there is little evidence it benefits patients by reducing deaths or improving quality of life either.

The number of people living to older age with chronic health conditions is growing. Various risk stratification tools have been introduced across the NHS aiming to improve planning and delivery of care for people in the poorest health. 

This study, funded by the NIHR Health Services Delivery and Research Programme, evaluated the tool during implementation of a quality and outcomes framework incentive to encourage identification and better management of high-risk patients. The lack of benefit may be because of poor uptake of the tool, lack of available services to help those at risk or the concentration of efforts on too few high-risk individuals. General practices still seem to have found few alternatives to hospital admission for many patients.

Share your views on the research.

Why was this study needed?

There were over five million emergency hospital admissions in 2012/13 in England, for instance, costing around £12.5 billion. About half of these admissions can be accounted for by only 5% of the population. Identifying these patients and ensuring they receive optimal preventive care may, in theory, improve their health outcomes and save NHS resources.

Risk stratification tools are designed to identify people at high risk of adverse events, such as hospital admission within the next 12 months. The Predictive Risk Stratification Model (PRISM), developed by Informing Healthcare and Health Dialog for NHS Wales, is one of several implemented in the UK. Its introduction coincided with a new payment under the Quality and Outcomes Framework (QOF) for GPs to identify people at high risk of emergency admission.

The intention is to use risk scores to target primary and community-care resources where they are most needed. This research aimed to evaluate the use of the tool alongside an incentive in the quality an outcomes framework to see if patient outcomes improved and track resource use.

What did this study do?

The stepped-wedge cluster randomised trial took place in 32 general practices in south Wales. The design meant all practices introduced the intervention in successive phases over 12 months.

Practices received PRISM software and one training session, with representatives providing two additional support sessions per month. The practice population was assigned to one of four risk categories from lowest to the highest risk of emergency admission.  It is unclear whether GPs had clear plans to intervene with people at high risk.

Outcomes for around 230,000 patients were compared before and after introduction using routine linkage data. Analyses were adjusted for length of time in each phase. Researchers also interviewed staff and sent postal questionnaires to a sample of patients, preferentially sampling those at highest risk.

A separate systematic review identified 11 observational studies on predictive risk tools but found no comparison data by which to review effect on outcomes.

What did it find?

  • PRISM increased the proportion of people with one or more emergency admissions, overall (7.1% intervention vs 4.7% control) and in each of the four risk categories. In highest risk group four, 55.4% were admitted compared with 44.7% in control phases (odds ratio 1.71, 95% confidence interval 1.40 to 2.08). 
  • Mortality remained unchanged: 9.25 vs 9.58 per 1,000 patients per year.
  • GPs used PRISM to fulfil QOF targets. They felt it made them more aware of high-risk patients but had doubts whether it could change practice. Community care services are limited, and there was often little alternative to admission.
  • Cost of PRISM introduction was low at £822 per practice in the first year and £474 in subsequent years, or £0.12 per person per year. However, when combined with the additional GP work and hospital attendances, this amounted to £76 per patient per year. At higher cost and with no benefit, it was clear the intervention was not cost-effective.

What does current guidance say on this issue?

A 2015 discussion paper by NHS England noted the downfalls of risk stratification tools: modest predictive accuracy, potential to worsen rather than address healthcare inequalities and increase costs.  

In 2011 the Department of Health (DH) advised that two DH commissioned risk prediction tools – Patients at Risk of Re-hospitalisation and the Combined Predictive Model – were outdated and would not be upgraded. Trusts were advised either to upgrade models themselves or seek alternatives. In response Nuffield Trust issued a guide, helping commissioners to choose from available tools. Regular conference on applying predictive risk models to high-risk populations has followed.

An enhanced service specification by the NHS Commissioning Board in 2013/14 encouraged GPs to undertake risk profiling and stratification of their registered patients. 

What are the implications?

PRISM appears accurate in categorising people according to their risk of admission. So that leaves the question why this didn’t help.

Responses from practice staff are notable. The tool brought focus to high-risk patients as intended. But perhaps current services limit the extent to which complex illness can be managed in the community. The tool itself had variable uptake by GPs particularly when QOF payments were not available

Citation and Funding

Snooks H, Bailey-Jones K, Burge-Jones D, et al. Predictive risk stratification model: a randomised stepped-wedge trial in primary care (PRISMATIC). Health Serv Deliv Res. 2018;6(1).

This project was funded by the National Institute for Health Research Health Services Delivery and Research programme (project number 09/1801/1054).

Bibliography

DH. Risk Stratification and next steps with DH Risk Prediction tools – Patients at Risk of Re-hospitalisation and the Combined Predictive Model. London: Department of Health; 2011.

Lewis G, Curry N, Bardsley M. Choosing a predictive risk model: a guide for commissioners in England. Briefing. Nuffield Trust; 2011.

NHS Commissioning Board. Enhanced service specification: risk profiling and care management scheme.

NHS England. Next steps for risk stratification in the NHS. NHS England; 2015. 

NHS Wales Informatics Service. GP practices trial tool to identify patients at risk. NHS Wales Informatics Service; 2009.

Whitfield L. Predictive risk: an idea whose time has come? Nuffield Trust comment. 2013.

Why was this study needed?

There were over five million emergency hospital admissions in 2012/13 in England, for instance, costing around £12.5 billion. About half of these admissions can be accounted for by only 5% of the population. Identifying these patients and ensuring they receive optimal preventive care may, in theory, improve their health outcomes and save NHS resources.

Risk stratification tools are designed to identify people at high risk of adverse events, such as hospital admission within the next 12 months. The Predictive Risk Stratification Model (PRISM), developed by Informing Healthcare and Health Dialog for NHS Wales, is one of several implemented in the UK. Its introduction coincided with a new payment under the Quality and Outcomes Framework (QOF) for GPs to identify people at high risk of emergency admission.

The intention is to use risk scores to target primary and community-care resources where they are most needed. This research aimed to evaluate the use of the tool alongside an incentive in the quality an outcomes framework to see if patient outcomes improved and track resource use.

What did this study do?

The stepped-wedge cluster randomised trial took place in 32 general practices in south Wales. The design meant all practices introduced the intervention in successive phases over 12 months.

Practices received PRISM software and one training session, with representatives providing two additional support sessions per month. The practice population was assigned to one of four risk categories from lowest to the highest risk of emergency admission.  It is unclear whether GPs had clear plans to intervene with people at high risk.

Outcomes for around 230,000 patients were compared before and after introduction using routine linkage data. Analyses were adjusted for length of time in each phase. Researchers also interviewed staff and sent postal questionnaires to a sample of patients, preferentially sampling those at highest risk.

A separate systematic review identified 11 observational studies on predictive risk tools but found no comparison data by which to review effect on outcomes.

What did it find?

  • PRISM increased the proportion of people with one or more emergency admissions, overall (7.1% intervention vs 4.7% control) and in each of the four risk categories. In highest risk group four, 55.4% were admitted compared with 44.7% in control phases (odds ratio 1.71, 95% confidence interval 1.40 to 2.08). 
  • Mortality remained unchanged: 9.25 vs 9.58 per 1,000 patients per year.
  • GPs used PRISM to fulfil QOF targets. They felt it made them more aware of high-risk patients but had doubts whether it could change practice. Community care services are limited, and there was often little alternative to admission.
  • Cost of PRISM introduction was low at £822 per practice in the first year and £474 in subsequent years, or £0.12 per person per year. However, when combined with the additional GP work and hospital attendances, this amounted to £76 per patient per year. At higher cost and with no benefit, it was clear the intervention was not cost-effective.

What does current guidance say on this issue?

A 2015 discussion paper by NHS England noted the downfalls of risk stratification tools: modest predictive accuracy, potential to worsen rather than address healthcare inequalities and increase costs.  

In 2011 the Department of Health (DH) advised that two DH commissioned risk prediction tools – Patients at Risk of Re-hospitalisation and the Combined Predictive Model – were outdated and would not be upgraded. Trusts were advised either to upgrade models themselves or seek alternatives. In response Nuffield Trust issued a guide, helping commissioners to choose from available tools. Regular conference on applying predictive risk models to high-risk populations has followed.

An enhanced service specification by the NHS Commissioning Board in 2013/14 encouraged GPs to undertake risk profiling and stratification of their registered patients. 

What are the implications?

PRISM appears accurate in categorising people according to their risk of admission. So that leaves the question why this didn’t help.

Responses from practice staff are notable. The tool brought focus to high-risk patients as intended. But perhaps current services limit the extent to which complex illness can be managed in the community. The tool itself had variable uptake by GPs particularly when QOF payments were not available

Citation and Funding

Snooks H, Bailey-Jones K, Burge-Jones D, et al. Predictive risk stratification model: a randomised stepped-wedge trial in primary care (PRISMATIC). Health Serv Deliv Res. 2018;6(1).

This project was funded by the National Institute for Health Research Health Services Delivery and Research programme (project number 09/1801/1054).

Bibliography

DH. Risk Stratification and next steps with DH Risk Prediction tools – Patients at Risk of Re-hospitalisation and the Combined Predictive Model. London: Department of Health; 2011.

Lewis G, Curry N, Bardsley M. Choosing a predictive risk model: a guide for commissioners in England. Briefing. Nuffield Trust; 2011.

NHS Commissioning Board. Enhanced service specification: risk profiling and care management scheme.

NHS England. Next steps for risk stratification in the NHS. NHS England; 2015. 

NHS Wales Informatics Service. GP practices trial tool to identify patients at risk. NHS Wales Informatics Service; 2009.

Whitfield L. Predictive risk: an idea whose time has come? Nuffield Trust comment. 2013.

Predictive risk stratification model: a randomised stepped-wedge trial in primary care (PRISMATIC)

Published on 19 January 2018

Snooks H, Bailey-Jones K, Burge-Jones D, Dale J, Davies J, Evans B, Farr A, Fitzsimmons D, Harrison J, Heaven M, Howson H, Hutchings H, John G, Kingston M, Lewis L, Phillips C, Porter A, Sewell B, Warm D, Watkins A, Whitman S, Williams V & Russell I T.

Health Services and Delivery Research Volume 6 Issue 1 , 2018

Background With a higher proportion of older people in the UK population, new approaches are needed to reduce emergency hospital admissions, thereby shifting care delivery out of hospital when possible and safe. Study aim To evaluate the introduction of predictive risk stratification in primary care. Objectives To (1) measure the effects on service usage, particularly emergency admissions to hospital; (2) assess the effects of the Predictive RIsk Stratification Model (PRISM) on quality of life and satisfaction; (3) assess the technical performance of PRISM; (4) estimate the costs of PRISM implementation and its effects; and (5) describe the processes of change associated with PRISM. Design Randomised stepped-wedge trial with economic and qualitative components. Setting Abertawe Bro Morgannwg University Health Board, south Wales. Participants Patients registered with 32 participating general practices. Intervention PRISM software, which stratifies patients into four (emergency admission) risk groups; practice-based training; and clinical support. Main outcome measures Primary outcome – emergency hospital admissions. Secondary outcomes – emergency department (ED) and outpatient attendances, general practitioner (GP) activity, time in hospital, quality of life, satisfaction and costs. Data sources Routine anonymised linked health service use data, self-completed questionnaires and staff focus groups and interviews. Results Across 230,099 participants, PRISM implementation led to increased emergency admissions to hospital [ΔL = 0.011, 95% confidence interval (CI) 0.010 to 0.013], ED attendances (ΔL = 0.030, 95% CI 0.028 to 0.032), GP event-days (ΔL = 0.011, 95% CI 0.007 to 0.014), outpatient visits (ΔL = 0.055, 95% CI 0.051 to 0.058) and time spent in hospital (ΔL = 0.029, 95% CI 0.026 to 0.031). Quality-of-life scores related to mental health were similar between phases (Δ = –0.720, 95% CI –1.469 to 0.030); physical health scores improved in the intervention phase (Δ = 1.465, 95% CI 0.774 to 2.157); and satisfaction levels were lower (Δ = –0.074, 95% CI – 0.133 to –0.015). PRISM implementation cost £0.12 per patient per year and costs of health-care use per patient were higher in the intervention phase (Δ = £76, 95% CI £46 to £106). There was no evidence of any significant difference in deaths between phases (9.58 per 1000 patients per year in the control phase and 9.25 per 1000 patients per year in the intervention phase). PRISM showed good general technical performance, comparable with existing risk prediction tools (c-statistic of 0.749). Qualitative data showed low use by GPs and practice staff, although they all reported using PRISM to generate lists of patients to target for prioritised care to meet Quality and Outcomes Framework (QOF) targets. Limitations In Wales during the study period, QOF targets were introduced into general practice to encourage targeting care to those at highest risk of emergency admission to hospital. Within this dynamic context, we therefore evaluated the combined effects of PRISM and this contemporaneous policy initiative. Conclusions Introduction of PRISM increased emergency episodes, hospitalisation and costs across, and within, risk levels without clear evidence of benefits to patients. Future research (1) Evaluation of targeting of different services to different levels of risk; (2) investigation of effects on vulnerable populations and health inequalities; (3) secondary analysis of the Predictive Risk Stratification: A Trial in Chronic Conditions Management data set by health condition type; and (4) acceptability of predictive risk stratification to patients and practitioners. Funding The National Institute for Health Research Health Services Delivery and Research programme.

Expert commentary

Case managing at-risk patients has become almost gospel as the way to reduce emergency admissions in the NHS. However, there is little evidence that these interventions work and this is not the first study to show they sometimes actually do the reverse of what was intended, i.e. increase admissions.

What should policymakers and practitioners make of this? Few would disagree that NHS patients with complex problems need better coordinated proactive care.

One message from this and similar studies is that we should look to see how these interventions can improve care rather than expecting them to save money.

Martin Roland, Emeritus Professor of Health Services Research; Fellow, Murray Edwards College, University of Cambridge