Children and Young Peoples Project

Children and Young Peoples Improving Access to Psychological Therapies Outcomes and Feeedback Bulletin - March 2013
For more information please see: http://www.iapt.nhs.uk/cyp-iapt
1

Children and Young Peoples Project

Welcome and Introduction
Welcome to the CYP IAPT data bulletin, a special edition newsletter dedicated to data, feedback, outcomes and evaluation. This issue will give you an overview of what has been happening and what were setting out to achieve in 2013.
Id like to start by thanking all staff in the collaborating sites for their hard work and dedication, particularly the data managers whose efforts have made this possible. All phase 1 sites have successfully uploaded data during year 1 of the project and the phase 2 sites have been incredibly busy getting their IT systems in place and ready for the start of their Routine Outcomes Monitoring (ROM). The collection and analysis of outcomes data is crucial in the CYP IAPT project. Miranda Wolpert Outcomes monitoring helps us to understand, quantify and demonstrate how National Informatics Lead, our treatment impacts on the lives of the young people using our services. This CYP IAPT information enables our young clients to reflect on and track their progress over the course of therapy. It also enables clinicians, young people and families to make decisions about what treatment is needed and how helpful particular treatments have been, as well as helping individual practitioners and services review their impact and make changes as necessary. This will ensure that we are able to deliver the best possible treatment to children and young people. We have learnt a lot during the first year of the project. To harness this knowledge we surveyed partnerships about their experiences so that we can make improvements. You can find a report from Anne OHerlihy, the Extended Scope Project Lead, on page 3. We also ran a consultation on data collection, the specification and the measures in the first version of the dataset. Thank you to everyone who took the time to complete the survey, your feedback has been invaluable. Weve taken all of the comments into account, and a number of important changes have been made to version 2 of the specification. Version 3, which includes a number of new measures, is due to be implemented in April 2013 please see page 7 for more details. Many of the collaboratives and partnerships have already been promoting the use of ROM in their services. Hertfordshire, for example, ran a one-day training with Scott Miller, one of the pioneers of service user feedback and outcomes in clinical practice. You can read a report about this day on page 5. The Salford collaborative also ran a series of workshops on ROM for all its partnerships; you can find further details on page 6. We are just beginning to analyse data from the first three quarters from phase 1 sites. Please click here to see the CORC report which outlines the results so far You can find an overview of what weve achieved to date, as well as the timeline for future uploads on page 10. As ever, we very much welcome feedback and comment so please do not hesitate to get in touch.

For more information please see: http://www.iapt.nhs.uk/cyp-iapt

Children and Young Peoples Project

Outcome measures in CYP IAPT
The CYP IAPT dataset identifies a number of validated outcomes measures tools and specifies when these should be used. Some tools are completed by young people, some by parents and some by both. We selected measures that have a proven value, are quick to complete and are clinically useful, and which are compatible with and build on the Child and Adolescent Mental Health Services Minimum Dataset (CAMHS MDS). In particular, the CYP IAPT programme set out to choose outcome measures that could be used to: I  nform and support meaningful clinical conversations for individual cases. l P  rovide rigorous outcome data to allow consideration of service level outcomes.
l

Measures were selected by the Outcomes and Evaluation Group and were agreed by the CYP IAPT Expert Reference Group. Please see http://www.iapt.nhs.uk/cyp-iapt/ for more details. Some measures were selected to be primarily used as clinical tools, others to be used to allow meaningful comparison across cases.A range of options was provided for session by session tracking to allow clinicians to select measures that were most relevant and meaningful collaboratively with young people and families.

Call for contributions

Please share your top tips, advice and expertise on using feedback and outcomes tools
The next version of A Practical Guide to Using Service User Feedback & Outcome Tools to Inform Clinical Practice in Child and Adolescent Mental Health is due to come out over the next couple of months. We would really welcome contributions from clinicians with experience of using the tools. The next edition will include tips, advice and expertise to help other clinicians and practitioners make best use of the CYP IAPT feedback and outcomes tool kit to enhance clinical practice. Contributions could be (anonymised) examples of where the tools had been particularly helpful, tips on helpful language to introduce the tools to families and young people, ideas on feeding back information, using information from the tools in supervision etc. The next guide will be used for CYP IAPT and beyond to develop other useful resources and training materials for clinicians and practitioners, all of which will be made available free of charge. Please send contributions by the end of March 2013 to Duncan Law at Duncan.law@hertspartsft.nhs.uk The Editors reserve the right to edit and adapt contributions. Please title your emails CYP-IAPT feedback and outcomes guide (and add your name). For more information please see: http://www.iapt.nhs.uk/cyp-iapt
2

Children and Young Peoples Project

Progress in using ROM
Anne OHerlihy, Extended Scope Project Lead, reports on feedback from partnerships about the implementation of ROM
Over the last year, 18 CAMHS partnerships from year 1 have reported progress in the implementation of routine outcome monitoring (ROM). The quarterly reports from sites show that between 73-98% of CBT and parenting trainees are collecting and using ROM and the percentage of the wider clinical team also using ROM has increased from an average of 20% in September to 54% in December 2012. This includes five partnerships where 80-100% of their clinicians are reported to be using ROM.

Concerns about ROM

One of the main difficulties for ROM implementation is posed by IT infrastructure. All partnerships would like an electronic system that allows data to be entered once with immediate upload to the patient record system, and that can provide a data report with graphs in real-time. Staff have expressed concerns about the time ROM use will add to their clinical sessions and the knock on effect this will have on waiting times, plus feeling overwhelmed by the additional paperwork. They are also concerned about how the ROM data would be used by managers and supervisors, and a number of clinicians remain unconvinced about the clinical benefits. A further issue emerged about identifying the most appropriate measures when working with young people who present with a range of issues and the clinicians tendency to opt for global measures that were not always clinically relevant.

New technologies and web-based entry

Sites are reporting that the capital funds given by the Department of Health for handheld technology and the changes to allow children and young people to enter data via the web will mitigate some of the IT infrastructure risks, not least because IT departments have engaged with clinical teams to support their bid.

Reported benefits of using ROM on clinical practice

Despite the difficulties and concerns, most partnerships describe a shift in culture and a growing understanding of how ROM can benefit clinical practice and service development. Foremost are the comments from young people and their families, who value the opportunity to give immediate feedback and report their concerns and issues regularly, as well as being able to visualise how they are doing on a graph.

It means if we go off track or get a bit lost along the way, we can both figure out how to find the way back again.
Young person from YoungMinds Very Important Kids Group

For more information please see: http://www.iapt.nhs.uk/cyp-iapt

Children and Young Peoples Project

[It] makes us feel like this is more of a shared experience between us and the clinician... like were in this together.
Young person from YoungMindsVery Important Kids Group

It supports us to feel more in control of its direction to understand when things are working, when things are not and what we can do about it.
Young person from YoungMindsVery Important Kids Group

Services report that ROM supports clinical team discussions and decisions about the appropriate level of care needed by presenting an unbiased view of the young persons wellbeing, and has enhanced supervisory practice, further encouraging reflective practice.

Developing practitioner skills at the same time as changing the way services are delivered through ROM has kick started a change in culture.
Gloucester CAMHS

Feedback is that they [trainees] are finding it very useful and it has helped improve patient outcomes and end treatment at an agreed time, based on the presenting outcome data.
Derby CAMHS

Reported benefits of using ROM on service development

ROM is being used to support service design and development plans. Some partnerships are in discussions with commissioners about how they can build outcomes into a review of their performance. The patient experience measures such as how are you doing SRS and Chi are being used to inform consultations with young people and guide service development.

The service has been able to demonstrate to commissioners collaborative engagement with young people and their families, which has contributed to securing the contract for the next 3 years.
Hertfordshire CAMHS partnership

For more information please see: http://www.iapt.nhs.uk/cyp-iapt

Children and Young Peoples Project

Example of service experience
Duncan Law, Hertfordshire Specialist CAMHS, talks about his partnerships experience of the implementation of CYP IAPT
It has certainly been a huge challenge rolling out the service user feedback and outcomes framework across Hertfordshire, but the data is starting to come through thanks to a great team effort, in particular from Amy Turner, a very committed and competent data manger, and CYP IAPT Project Manger, Janet Arris. The flow of data is testament to the way our services have begun to embrace the CYP IAPT philosophy, supported by trainees, supervsiors and service managers. Commissioners who see the value in having clinicians trained to use service user feedback and outcomes in clinical work have supported additional training and clinical discussion forums. At a local team level, Siri Wooster, Divisional Manager of East Hertfordshire Specialist CAMHS, where the Hertfordshire CYP IAPT roll out began, has ensured that CYP IAPT and transformation have been high on the agenda and, most importantly, allowed time for discussion within the team about the anxieties, practicalities and clinical issues around all aspects of service user feedback and outcomes. Just over a year on, we have made good progress but still have a way to go to get and keep clinicians on board and spread the message that feedback is about positive collaborative practice and not just questionnaires. The feeling in the partnership is that all this effort will be worth it.

Outcomes workshops
In January Hertfordshire CAMHS flew over Scott Miller, one of the pioneers of service user feedback and outcomes in clinical practice, to present an Outcomes Masterclass. Simon Shattock, a Systemic Family Therapist from Hertfordshire Partnership NHS Foundation Trust, attended the training. Heres what he thought of it.
I was rather sceptical about what this day would achieve for me as I have been using regular outcome monitoring in my work since participating in the CYP IAPT course. The day helped me understand how to use the session by session tools in a meaningful way to achieve feedback on the therapeutic relationship. Scott Millers work challenges the orthodox view about the dominant discourse of evidence-based practice. This is not to say we should not undertake treatments that are useful to children and families, but should consider that real change with clients should happen in the first 4-6 sessions, and that the greatest predictor of positive outcome for clients is the therapeutic relationship regardless of the model. The most effective therapists are those who seek negative feedback from clients earlier on in treatment. and then tailor his or her approach to the client accordingly. I have used his ideas in therapy with a teenage girl and was able to find out that she did not like the silences in the room between us and would prefer if I talked more and asked lots of questions. I have altered my position as a therapist accordingly and hope now to be more effective in my therapy now.

For more information please see: http://www.iapt.nhs.uk/cyp-iapt

Children and Young Peoples Project

Outcomes workshops
Recognising routine outcomes monitoring as a key driver for the successful implementation of CYP IAPT, the Manchester Collaborative ran a series of workshops, delivered to all Partnerships. The key aims were to: l Raise awareness and support implementation of ROMs in practice l Ensure a baseline knowledge of CYP IAPT across the workforce l Introduce and increase awareness of nationally agreed CYP IAPT ROMs l Provide an opportunity to experience and test application in practice l I  dentify relevant internal/external networks and the necessary resources for successful implementation The workshops were delivered over a full day with a total of152 staff members across the participating partnerships. The attendance of senior managers and clinicians was particularly valuable in ensuring visible strategic and clinical ownership and commitment to the successful implementation of CYP IAPT.Whilst challenging, the implementation of ROM is an exciting part of the process to support service transformation in CAMHS and all should be commended for starting this work. Thanks go to all the partnerships within the Manchester Learning Collaborative for their support, recognising that continued successful implementation will to a great degree depend upon the goodwill of the committed workforce.The implementation of ROMs requires a cultural shift for many organisations; it is a process that will take time. There is clearly a significant degree of enthusiasm for the use of ROMs in practice.

Gill Walker Service Transformation Lead; Manchester collaborative

Barry Nixon NHS North West CYP IAPT Lead

The session made me excited & passionate about the transformation of our service.
Trainee attending the workshop

Really informative, much better understanding of what IAPT means in real terms and outcome measures.
Trainee attending the workshop

Helped me understand the national context of IAPT and the use/ importance of routine outcome measures.
Trainee attending the workshop

For more information please see: http://www.iapt.nhs.uk/cyp-iapt

Children and Young Peoples Project

Consultation on the CYP IAPT dataset
To see what those working at all of the CYP IAPT sites thought of the data we carried out a consultation in July 2012 about the process, data specification and measures. A full report on the consultation can be found here.We also asked for feedback via emails and in meetings.The majority of responses to the formal consultation came from CAMHS clinicians (80%) and below is a selection of comments.

A child was able to highlight in a session the issues we were NOT talking about more easily than if we had not had it clearly identified on the SRS/ORS Family members like the visual aspects of this process it shows the change not just talking
Feedback from clinician

Changes to the dataset

Version 2
Following feedback from the consultation we have made several changes, and version 2 of the dataset was released in December 2012. Changes include: A  ddition of a refined assessment questionnaire (Current View) which includes additional presenting problems, contextual factors, complexity factors and educational information l U pdating of event records to improve the quality of data recorded about resource need, for instance indirect contacts are now captured, event duration, and how many professionals were present at appointments
l

Version 3

Version 3 of the dataset will be released in April 2013, and the following measures will be added (subject to final sign off by measure developers) to track change in:
l l l l l

Family relationships (SCORE 15) Eating disorders (EDE-Q) Parenting ( Parental Efficacy Scale) Management of learning disability (SLDOM) General wellbeing (Warwick-Edinburgh scale)

In addition, the group are looking at whether a) some costed measures can be supported (but this requires discussion on copyright issues); b) whether CORE YP and CORE OM can be used (but this requires discussion around data sharing.) For more information please see: http://www.iapt.nhs.uk/cyp-iapt
7

Children and Young Peoples Project

The CORC Central Team
The CORC Central Team provides a range of support for the collection, analysis and reporting of data, as well as general support for members. The team provide site visits, and telephone and email support where members have queries, need information, or have problems with implementation.

Miranda Wolpert CORC Director

Andy Fugard Research Lead

Rachel Argent Research Assistant

Jenna Bradley Senior Research Officer and Project Co-ordinator

Data Uploads and Analysis

The CORCNexus consortium have been commissioned by the DH to collate and analyse data for CYP IAPT and support and train sites to allow appropriate data flow. CORCNexus is a collaboration between the CAMHS Outcomes Research Consortium (CORC), the CAMHS Evidence Based Practice Unit (EBPU), and the secure data storage company MegaNexus (MN). Every quarter, outcome and activity data flows to the secure data store, as outlined below.

For more information please see: http://www.iapt.nhs.uk/cyp-iapt

Children and Young Peoples Project

Since January 2012 a total of 789 children, young people and/ or their families have been seen by CYP IAPT trainees in phase 1 sites or others using the CYP IAPT measurement approach. Six hundred and sixty six of these cases are still open and 121 have been closed (as of 30th September 2012).

What are we doing with the data?

The CYP IAPT data will be used by the central CYP IAPT project team to inform our understanding of the outcomes achieved by young people and their families after accessing services. We intend to make appropriate data publically avail- Number of cases by presenting problems of cases able as soon as possible, but at this stage in the project we are seen as identified at first contact focussing on reviewing data quality and identifying any issues or areas for further work. CORC have carried out preliminary analysis of very high level summary intake data. Click here to view the CORC report. Please be aware that the data analysis must be taken as provisional until all data quality checks have been complete. Moreover, when considering information such as presenting problems, complexity, participation in education or training, it is important to note that most of the data is from trainees working with less complex cases which are appropriate for their skills and training as they proceed through the course.Trainees have been working with cases that suit the therapies in which they are being trained. It is very likely that aspects of the data will change once all practitioners in CYP IAPT sites use the outcome monitoring.

The frequency of child and young person-rated session by session measures completed at first time point (Q1-Q3 2012).

For more information please see: http://www.iapt.nhs.uk/cyp-iapt

Children and Young Peoples Project

Data collection going forward
The CYP IAPT project has achieved a great deal in a very short time-frame, but theres lots more work to be done. Below is a schedule for data uploads going forward.

Data collection Dataset Phase 1 Phase 2 sites period version sites End of year col- v2 All service Trainees only lection (Jan to March 2013) Verification of the outcomes tools proposed in the data set consultation and development of associated guidance Q1 (Apr to Jun) v3 All service Trainees only 2013 Q2 (Jul to Sep) v3 All service Trainees only 2013 Q3 (Oct to v3 All service Trainees only. All service Dec) 2013 gearing up 1st Dec 2013

Central submission period 1st to 15th April 2013

Dec 2012 to Feb 28th 2013 1st to 15th July 2013 1st to 15th Oct 2013 1st to 15th Jan 2014

CYP IAPT YouTube Channel

On the CYP IAPT YouTube channel you can find a range of videos of children and clinicians talking about routine outcomes monitoring and service user feedback. The latest video is of Duncan Law discussing how goal setting and sacking your therapist has encouraged better service user participation. Please click here to view the videos.

Any Questions for CORC?

We very much welcome feedback, so please get in touch with comments and/or questions. Our email is CORCiapt@annafreud.org. For more information please see: http://www.iapt.nhs.uk/cyp-iapt
10