PATHOLOGY QUALITY ASSURANCE REVIEW

Chaired by Dr Ian Barnes

Leadership Graduates Challenge Event

Challenge Event I
ABOUT THE REVIEW
London, 25 September 2013 Who attended? The independent review of Quality Assurance in pathology was commissioned by Sir Bruce Keogh, the NHS Medical Director for England. It is chaired by Dr Ian Barnes. The Review has been asked to examine all aspects of the current quality assurance system for NHS-commissioned pathology in England. It is examining both explicit quality assurance processes and the governance which makes use of them. In a context of increasing focus on culture in the NHS, it is also addressing the implications of safety science and continuous quality improvement for NHS-commissioned pathology. The Review has 3 core workstreams: 1. Workforce, led by Jo Martin 2. Provider Quality Activity, led by Ian Sharp 3. National system, including commissioning and incident reporting, led by Peter Huntley This event was for the graduates of the Leadership Development Scheme of the National Pathology Programme. These are people who have moved into or are preparing to move into senior roles in pathology services. Typically they are in mid-career. The event was advertised by email and was filled on a first come, first served basis. 15 delegates attended. What was the purpose of this event? This was the first in a series of challenge events (see box on next page) This audience has a deep knowledge of the culture and processes in English laboratories. As relatively recently senior people, they are for the most part inheritors of these rather than creators of them. They are thus well placed to take an informed yet critical view of the status quo. As leaders, however, they will be responsible for implementing any changes that may be triggered by the Reviews recommendations, and more generally, for responding to the cultural and organisational challenges currently facing the NHS in the wake of structural change and of the Francis report and its successors. The expectation of the Review was that this group would be particularly well positioned to offer challenge to: the Reviews findings the feasibility of possible change
th

How was the event run? The event began with a briefing from the Review Chair, Ian Barnes. After giving an overview of the Reviews remit and structure, Dr Barnes proceeded to outline its main findings, and the themes that would inform its eventual recommendations. Themes were arranged into three groups reflecting the area of current risk they were designed to address.

Page 1. Company name

Leadership Graduates Challenge Event

Page 1

After this briefing, the group was asked to: consider the implications of these themes for pathology services and for the system as a whole reflect on the likely impact of proposed changes consider what risks were attendant on each proposal, in terms of feasibility and likely impact suggest ways of framing the approach to reduce these risks and maximise the potential benefits of any changes

CHALLENGE EVENTS

At the end of the day, the group was asked to crystallise their reaction to the discussion by framing in precise terms a short set of recommendations for the Review to consider. Findings Ian Barnes gave an overview of the current state of quality assurance processes in England. Although there are accountable institutions at every level (see Figure 1), in practice there is a lack of: transparency integration verification of performance oversight enforcement through appropriate sanctions

The Review is conducting a series of Challenge Events in order to improve and refine its recommendations. Challenge events are not consultation in the usual sense. As an independent review, we dont have to consult formally on our recommendations, since they are not binding. Challenge events are best seen as a form of structured listening. Challenge events bring together a small but representative group of people who have some form of expertise on the matters under review. At each event, we will share the current state of thinking within the Review: a summary of our methods, the evidence we have received, our provisional findings, and the emerging themes that will form the raw material for our recommendations in due course. We will then invite comment and challenge. The responses we receive will contribute significantly to the evidence base, relevance and feasibility of the Reviews findings and recommendations.

The impact of these weaknesses poses risks to: the standards used to assess quality in pathology quality assessment within pathology quality assurance of the service

Emerging themes Ian Barnes set out a number of themes currently investigated by the Review. Reflection on these generated a wide-ranging discussion. Energy and debate focused on the following topics: quality standards in pathology system-level oversight of pathology quality assurance transparency of performance-related information commissioning internal clinical governance

Page 2. Company name

Leadership Graduates Challenge Event

Page 2

Figure 1

During the conversation, delegates were naturally keen to offer their own solutions to the challenges emerging from the Reviews work . Towards the end of the day, delegates were in fact invited to crystallise their own sense of the direction of the discussion by contributing recommendations of their own. These gave the Review valuable insight into the kinds of ideas that this particular group felt were important and feasible, and will be extremely helpful as the Review reaches its own conclusions in the coming months. The nature of a quality standard in pathology Delegates asked the Review about its definition of standards, and the nature of a pathology quality standard. Delegates felt that there is currently no authoritative set of standards; there was not even agreement over the key metrics that such a set of standards would use. Delegates recognised that there were clear standards of scientific accuracy. However, delegates felt that scientific standards alone, while useful within the pathology service, had much less relevance to its users and customers. It was important, delegates said, to choose a set of standards that had a broader relevance. As one delegate remarked, Surely, commissioners should measure outcomes, not how well Pathology can measure Sodium to seven decimal places? Delegates also suggested that standards should be applied in the wider context provided by benchmarking. This will require co-operation between peers and between local and regional pathology services and external bodies, to ensure standards have the same meaning and the same impact across the country. Oversight of the quality assurance system The Review wanted to understand delegates sense of how the progress of quality assurance in pathology might be overseen. Delegates recognised that it was important, in the light of the evidence presented to the Review, to maintain vigilant about the effectiveness of the quality assurance system. If there were any changes to the system as a result of the Reviews recommendations, it would naturally be important to monitor the effectiveness of those changes and to ensure that the impact on quality and patient safety was as intended.

Surely, commissioners should measure outcomes, not how well Pathology can measure Sodium to seven decimal places?

Delegates were concerned that such oversight should not have unintended consequences of its own. In particular, some delegates expressed anxiety that such oversight might lead to a duplication of roles between organisations. Others were concerned that oversight might lead to direct performance management of pathology: in effect, another management layer. The Review team noted this concern, but stressed that the role of oversight was to ensure system effectiveness rather than service effectiveness. Transparency of performance-related information The Review is keen to explore ways in which information relating to the performance of pathology services can be made appropriately and meaningfully available. Such transparency is increasingly seen as an important driver of quality in the wider NHS. However, the Review has been made aware that the application of transparency to the pathology context raises some non-trivial challenges. The discussion started with the observation that the most immediately available and comparable stream of quality information is the results of External Quality Assurance (EQA). Most laboratories use EQA and it is a requirement for laboratory accreditation, so this is an information stream that is already in wide use. However, the results of EQA are currently confidential to laboratories, who are the customers of EQA providers.

The discussion among delegates therefore focused on the impact and possible benefits of publishing EQA results. Some delegates suggested that EQA results could provide a minimum, externally validated standard for pathology services. Others were worried that this might discourage continuous quality improvement: laboratories might in this case aim for the minimum standard rather than seeking to be the best they can. Some delegates were enthused by various forms of ranking, at laboratory or service level. Some were attracted to the idea of banded achievement levels, ranging for example from Red (inadequate) through Green (Safe) to Blue (exemplary). Others were attracted to the idea of league tables, arguing that they might drive up performance. While the idea of publication of such performance at service level appeared uncontroversial, there was some debate as to whether this might be extended to assess the performance of individual pathologists. Many delegates felt that this was, in principle, possible. There was discussion of how this would apply to the various disciplines within pathology, with time precluding detailed resolution of this issue. However, some delegates wanted to set a five-year timeframe for the extension of performance standards to individuals, which was considered sufficient time to allow the development of appropriate solutions across all disciplines. Delegates pointed to the need to standardise not only the application of EQA to allow meaningful publication, but also the operation of EQA schemes to reduce the risk of gaming. Commissioning The Review is keen to understand how the quality assurance of commissioned services might be strengthened and developed in pathology.

In Screening, because of market forces, people comply, as stuff can be taken away.

Delegates felt that the starting point for such activity ought to be laboratory accreditation. At present, commissioners are free to commission services from laboratories which are not part of the accreditation scheme or have not achieved fully accredited status. Delegates urged the Review to take a view on mandatory accreditation. While accreditation offers a relatively easy way for non-specialist commissioners to be assured of quality, delegates identified a role for specialist commissioning. Specialist commissioners would understand the components of a quality service and the effect of various service levels and configurations on the pathway as a whole: the clinical utility of the service. Commissioners need to know what it is that they are buying, not just what it costs. There was an appreciation among delegates for the power and effectiveness of the commissioning lever. Used well, they argued that it can drive up standards without the need to legislate or deploy cumbersome oversight mechanisms. In Screening, because of market forces, people comply, as stuff can be taken away.

Page 5. Company name

Leadership Graduates Challenge Event

Page 5

Internal clinical governance The Review was keen to gather from delegates their experience of internal clinical governance and its role in quality assurance. In particular, the Review is keen to understand the role played by Trust and provider Boards in providing assurance of the quality of the pathology service. In response to this question, most of the discussion focused on two functions which sit at the interface between the pathology directorate and the Trust Board: error reporting and the quality manager role. Many delegates argued that there is not currently any agreed definition of error. In particular, some delegates were unclear what constituted a useful definition of error or how to draw a line between sub-optimal but acceptable events, and reportable errors. Others challenged the need to draw this line. They argued that the emphasis needed to change from policing the lower end of acceptability towards a culture of continuous improvement where all forms of sub-optimal performance are reported, analysed and amended. One delegate observed that in their experience of regional quality monitoring, good units are reporting more errors than those where there is an intuitive concern [about quality]. Delegates had varied experience of the expertise of Trust boards in handling error reporting and continuous improvement processes. Some boards still seemed to be inappropriately reassured by silence:

Good units are reporting more errors than those where there is an intuitive concern [about quality].

no news is good news. There was a vigorous debate about adopting the airline approach seen as an emphasis on prevention of errors. Culture is about prevention it is a cultural thing. Such a culture would take deviations from standard practice which had not resulted in harm or risk to safety almost as seriously as explicit untoward incidents. Airlines care about planes being late as well as about near misses and crashes. Some delegates were worried that safety culture, which relies on the open sharing of process and errors, was difficult to achieve in a competitive environment. Is error reporting shared among competitors? they wondered aloud. Many delegates voiced concern about the usefulness of current incident-reporting tools. Nine times out of ten they dont give the Pathology department en ough information, so we have to use our own information. There was also concern that Departments that are good on reporting incidents receive very little feedback in return. Some delegates were concerned about the growth of the Quality Manager role as a separate function from routine professional practice. These delegates felt that the Quality Manager role should be gradually replaced by explicit quality processes which are the responsibility of every specialist. These processes should include not only error reporting and analysis but also addressing risk: A lot of emphasis should be placed on having outputs that are actionable the report that actually leads to action. A recurring theme in the discussion was the interaction between Page 6. Company name

A lot of emphasis should be placed on having outputs that are actionable the report that actually leads to action.

Leadership Graduates Challenge Event

Page 6

good processes and good culture. Delegates highlighted a virtuous feedback mechanism in which useful error reporting leads to more robust scrutiny and better solutions, which encourages more transparency. More transparency and a more permissive culture for error detection in turn would encourage more error reporting, it was argued. Such an outcome, delegates argued, would generate a holistic culture impacting the frontline MLA as much as the Trust Chief Executive. Conclusion and follow up activity The discussion during the day was extensively captured by the Review team. As well as the summary provided here, further detail was noted, which will form material for analysis by the Review as it moves to its conclusion. In his closing statement, Ian Barnes, having thanked the delegates for their engagement, encouraged them to stay in touch with the review. This is possible by contacting the Review on england.pathqareview@nhs.net or by following the Reviews twitter account @pathqareview. The Review also blogs at www.pathqareview.wordpress.com. Comments or tweets in response are welcomed and read.

Em@il Contact the Review by email on england.pathqareview@nhs.net

Twitter account @pathqareview

Page 7. Company name

Leadership Graduates Challenge Event

Page 7