NIHR Signal Accuracy of staff who read mammograms doesn’t decline over time

Published on 2 August 2016

Changing the order in which readers examine screening mammograms has no effect on breast cancer detection rates, rates of recall for further tests or rates of disagreement between readers.

In the UK each woman’s mammogram is examined by two independent readers (radiologists, radiographers or breast clinicians) who review each batch of mammograms from about 40 women in the same order. Research in other fields has indicated that vigilance can decrease with increasing time spent performing a particular task, but this effect has not been tested previously for the repetitive task of screening mammography.

This large multicentre trial therefore examined whether changing the order in which the second reader looks at the mammograms could improve cancer detection rates. Contrary to expectation it had no effect. In fact, it found that the performance of individual readers improved and they recalled fewer women for further tests if they read the mammogram last in the batch compared to reading it first. This didn’t affect their cancer detection rates.

The findings are generalisable only to breast cancer screening programmes where two readers examine mammograms separately, which happens in most of Europe but not in the US.

Accuracy of staff who read mammograms doesn’t decline over time

Share your views on the research.

Why was this study needed?

Breast cancer screening detects about 18,000 cancers per year in the UK (8.6 per thousand women screened). About 6,030 cancers per year are detected between screening rounds. This is due to cancers missed at screening and those developing in between screening rounds. An additional 69,700 women each year experience false positives at each screening round, where the result wrongly indicates that breast cancer might be present, and women are recalled for further tests.

Interpreting mammograms is a difficult and visually repetitive task that can result in missed cancers and false positives. Findings from settings, such as airport baggage screening, have suggested that increased time spent performing a task may reduce ability to accurately detect abnormalities.

In the UK National Health Service Breast Screening Programme two readers separately examine each woman’s mammograms for signs of cancer. Women have mammograms taken of both breasts. Both readers scrutinise batches of around 35 women’s mammograms. Current practice is that both readers examine the x-rays in the same order. The study questioned whether reader fatigue towards the end of a batch has a part to play in detection of breast cancers. They therefore looked at the effect of reversing the order in which the second mammography reader views batches. If vigilance fell towards the end of their batch it could be expected that the rates of disagreements in the group where the order was reversed might occur.

What did this study do?

This randomised controlled trial (CO-OPS trial) was carried out across 46 specialist NHS breast cancer centres over one year.

A total of 360 qualified mammogram readers were included – 186 radiologists, 143 radiography advanced practitioners and 31 breast clinicians – who reviewed mammograms from almost 1.2 million women.

In the control group, two readers examined each batch of mammograms in the same order. In the intervention group, two readers examined each batch in the opposite order to one another.

There were between 16 and 46 mammograms per batch, and each reader examined between one and 26 batches in a single day.

The main outcome of interest was the cancer detection rate – the number of women with cancer detected as a proportion of all women screened. Other outcomes were rates of recalling women for further tests and disagreement between mammography readers.

What did it find?

  • There was no significant difference in breast cancer detection rates when two readers reviewed mammograms in the opposite order (5272 cancers, 0.88%) compared with the same order as each other (5212 cancers, 0.87%) (difference 0.01% points, 95% confidence interval [CI] -0.02 to 0.04% points).
  • There was also no significant difference in the rate of recall of women for further tests in the opposite order review group (24,681 women, 4.14%) compared with the same order group (24,894, 4.17%) (difference -0.03% points, 95% CI -0.10% to 0.04% points).
  • Also there was no significant difference in rates of reader disagreements when two readers reviewed in the opposite order (20,471 disagreements, 3.43%) compared with the same order (20,793, 3.48%) (difference -0.05% points, 95% CI -0.11% to 0.02% points).
  • Increasing the time that individual readers spent reviewing a batch of mammograms had no effect on their cancer detection rates, but they did recall fewer women for further tests towards the end of reading the batch.

What does current guidance say on this issue?

Women in England aged between 50 and 70 are offered breast mammography screening every three years through the NHS Breast Screening Programme. The NHS is in the process of extending the programme as a trial to offer screening to some women aged 47 to 73.

What are the implications?

This large trial does provide evidence that two readers examining mammograms in a different order to each other, rather than the same order as each other, has no effect on breast cancer detection rates or rates of recall for further tests.

This suggests that, up to about 40 readings, there is no loss of “vigilance” or carefulness.  These findings are contrary to those from research in other fields which suggest that performance on a repetitive task may worsen over time. The study did show that over the course of reviewing around 40 mammograms, an individual reader’s performance improved and they recalled fewer women for further tests and this did not affect the cancer detection rates.

The findings are generalisable only to breast cancer screening programs that use two readers to examine mammograms separately. It was noted by these researchers that programmes in the US rarely incorporate double reading.

Citation and Funding

Taylor-Phillips S, Wallis MG, Jenkinson D, et al. Effect of Using the Same vs Different Order for Second Readings of Screening Mammograms on Rates of Breast Cancer Detection: A Randomized Clinical Trial. JAMA. 2016;315(18):1956-65.

This project was funded by the National Institute for Health Research through a fellowship and the CLAHRC West Midlands initiative and the University of Warwick.

Bibliography

NHS Choices. Breast Cancer Screening. London. Department of Health; updated 2015.

Why was this study needed?

Breast cancer screening detects about 18,000 cancers per year in the UK (8.6 per thousand women screened). About 6,030 cancers per year are detected between screening rounds. This is due to cancers missed at screening and those developing in between screening rounds. An additional 69,700 women each year experience false positives at each screening round, where the result wrongly indicates that breast cancer might be present, and women are recalled for further tests.

Interpreting mammograms is a difficult and visually repetitive task that can result in missed cancers and false positives. Findings from settings, such as airport baggage screening, have suggested that increased time spent performing a task may reduce ability to accurately detect abnormalities.

In the UK National Health Service Breast Screening Programme two readers separately examine each woman’s mammograms for signs of cancer. Women have mammograms taken of both breasts. Both readers scrutinise batches of around 35 women’s mammograms. Current practice is that both readers examine the x-rays in the same order. The study questioned whether reader fatigue towards the end of a batch has a part to play in detection of breast cancers. They therefore looked at the effect of reversing the order in which the second mammography reader views batches. If vigilance fell towards the end of their batch it could be expected that the rates of disagreements in the group where the order was reversed might occur.

What did this study do?

This randomised controlled trial (CO-OPS trial) was carried out across 46 specialist NHS breast cancer centres over one year.

A total of 360 qualified mammogram readers were included – 186 radiologists, 143 radiography advanced practitioners and 31 breast clinicians – who reviewed mammograms from almost 1.2 million women.

In the control group, two readers examined each batch of mammograms in the same order. In the intervention group, two readers examined each batch in the opposite order to one another.

There were between 16 and 46 mammograms per batch, and each reader examined between one and 26 batches in a single day.

The main outcome of interest was the cancer detection rate – the number of women with cancer detected as a proportion of all women screened. Other outcomes were rates of recalling women for further tests and disagreement between mammography readers.

What did it find?

  • There was no significant difference in breast cancer detection rates when two readers reviewed mammograms in the opposite order (5272 cancers, 0.88%) compared with the same order as each other (5212 cancers, 0.87%) (difference 0.01% points, 95% confidence interval [CI] -0.02 to 0.04% points).
  • There was also no significant difference in the rate of recall of women for further tests in the opposite order review group (24,681 women, 4.14%) compared with the same order group (24,894, 4.17%) (difference -0.03% points, 95% CI -0.10% to 0.04% points).
  • Also there was no significant difference in rates of reader disagreements when two readers reviewed in the opposite order (20,471 disagreements, 3.43%) compared with the same order (20,793, 3.48%) (difference -0.05% points, 95% CI -0.11% to 0.02% points).
  • Increasing the time that individual readers spent reviewing a batch of mammograms had no effect on their cancer detection rates, but they did recall fewer women for further tests towards the end of reading the batch.

What does current guidance say on this issue?

Women in England aged between 50 and 70 are offered breast mammography screening every three years through the NHS Breast Screening Programme. The NHS is in the process of extending the programme as a trial to offer screening to some women aged 47 to 73.

What are the implications?

This large trial does provide evidence that two readers examining mammograms in a different order to each other, rather than the same order as each other, has no effect on breast cancer detection rates or rates of recall for further tests.

This suggests that, up to about 40 readings, there is no loss of “vigilance” or carefulness.  These findings are contrary to those from research in other fields which suggest that performance on a repetitive task may worsen over time. The study did show that over the course of reviewing around 40 mammograms, an individual reader’s performance improved and they recalled fewer women for further tests and this did not affect the cancer detection rates.

The findings are generalisable only to breast cancer screening programs that use two readers to examine mammograms separately. It was noted by these researchers that programmes in the US rarely incorporate double reading.

Citation and Funding

Taylor-Phillips S, Wallis MG, Jenkinson D, et al. Effect of Using the Same vs Different Order for Second Readings of Screening Mammograms on Rates of Breast Cancer Detection: A Randomized Clinical Trial. JAMA. 2016;315(18):1956-65.

This project was funded by the National Institute for Health Research through a fellowship and the CLAHRC West Midlands initiative and the University of Warwick.

Bibliography

NHS Choices. Breast Cancer Screening. London. Department of Health; updated 2015.

Effect of Using the Same vs Different Order for Second Readings of Screening Mammogramson Rates of Breast Cancer Detection. A Randomized Clinical Trial

Published on 10 May 2016

S Taylor-Phillips, M Wallis, D Jenkinson, V Adekanmbi, H Parsons, J Dunn, N Stallard, A Szczepura, S Gates, O Kearins, A Duncan, S Hudson, A Clarke

JAMA Volume 315 Issue 18 , 2016

IMPORTANCE Interpreting screening mammograms is a difficult repetitive task that can result in missed cancers and false-positive recalls. In the United Kingdom, 2 film readers independently evaluate each mammogram to search for signs of cancer and examine digital mammograms in batches. However, a vigilance decrement (reduced detection rate with time on task) has been observed in similar settings. OBJECTIVE To determine the effect of changing the order for the second film reader of batches of screening mammograms on rates of breast cancer detection. DESIGN, SETTING, AND PARTICIPANTS A multicenter, double-blind, cluster randomized clinical trial conducted at 46 specialized breast screening centers from the National Health Service Breast Screening Program in England for 1 year (all between December 20, 2012, and November 3, 2014). Three hundred sixty readers participated (mean, 7.8 readers per center)—186 radiologists, 143 radiography advanced practitioners, and 31 breast clinicians, all fully qualified to report mammograms in the NHS breast screening program. INTERVENTIONS The 2 readers examined each batch of digital mammograms in the same order in the control group and in the opposite order to one another in the intervention group. MAIN OUTCOMES AND MEASURES The primary outcomewas cancer detection rate; secondary outcomes were rates of recall and disagreements between readers. RESULTS Among 1 194 147 women (mean age, 59.3; SD, 7.49) who had screening mammograms (596 642 in the intervention group; 597 505 in the control group), the images were interpreted in 37 688 batches (median batch size, 35; interquartile range [IQR]; 16-46), with each reader interpreting a median of 176 batches (IQR, 96-278). After completion of all subsequent diagnostic tests, a total of 10 484 cases (0.88%) of breast cancer were detected. There was no significant difference in cancer detection rate with 5272 cancers (0.88%) detected in the intervention group vs 5212 cancers (0.87%) detected in the control group (difference, 0.01% points; 95%CI, −0.02%to 0.04%points; recall rate, 24 681 [4.14%] vs 24 894 [4.17%]; difference, −0.03%points; 95%CI, −0.10% to 0.04%points; or rate of reader disagreements, 20 471 [3.43%] vs 20 793 [3.48%]; difference, −0.05%points; 95%CI, −0.11% to 0.02%points). CONCLUSIONS AND RELEVANCE Interpretation of batches of mammograms by qualified screening mammography readers using a different order vs the same order for the second reading resulted in no significant difference in rates of detection of breast cancer.

Standard UK practice is for two readers to examine mammograms in batches of approximately 40. In some centres the readers can access the other reader’s decision but in others the reader cannot. Disagreements are resolved by a third reader or a group of readers reviewing the suspicious mammogram.

Expert commentary

This study of patterns of reading screening mammogram batches, reverse reading versus reading in same order, within the NHS Breast Screening setting, fails to confirm a priori premise for repetitive tasks, that reader fatigue at the end of the batch results in a reduced cancer detection rate. An interesting parallel observation that recall rate decreases with time spent on task may relate to individual awareness of target rates for recall. It is gratifying that this reduced recall rate does not translate into a reduced cancer detection rate. Film readers should however remain aware of the potential negative impact of fatigue and short breaks between batch reads are likely to remain beneficial.

Dr Caroline Rubin, Consultant Radiologist, University Hospitals Southampton, Medical Director Education and Training, Clinical Radiology, Royal College of Radiologists