Accountability Agreements in Ontario: How can we make them more effective instruments for change? U of T HSPRN Seminar November 19, 2008 Dr. Ben Chan, MD MPH MPA CEO, Ontario Health Quality Council 1 OHQC Mandate – A Dual Role An independent body, created by the Government of Ontario to: • Report directly to Ontarians on the state of our publicly funded health system; and • Support continuous quality improvement 2 Reporting on Attributes of Quality & High Performing Health System • • • • • • • Effective Efficient • Equitable • Accessible Safe Patient-centred Focused on population health Integrated Appropriately resourced 3 4 Accountability “By “accountable” I mean making sure that the government and our health partners clearly agree on what outcomes we need to achieve together. Accountability means being answerable for our actions, not just our good intentions. We need clearer performance targets, greater transparency, and better lines of communication. And let me be clear: accountability isn’t a burden we place on others, it’s a responsibility we all accept and share ― and I include this government and my ministry.” ― George Smitherman, Ontario Minister of Health and Long-Term Care in 2004 5 Timelines Year Key Milestones 2003 Work through JPPC to develop a multi-year funding and accountability framework for Ontario hospitals 2004 Commitment to the Future of Medicare Act introduces accountability agmts 2005 First accountability agreements are negotiated between Ministry and hospitals 2006 Local Health System Integration Act establishes LHINs 2007 LHINs responsible for service accountability agreements with local providers 2007 Ministry-LHIN accountability agmts enter into force for 3 years 2008 LHIN-hospital 2 year agreements 2009 LHIN agreements to be introduced in CHCs; community mental health and addiction services, community service agencies, CCACs 2010 LHIN agreements to be introduced in long-term care facilities 6 Indicator Development Process • Indicator development “reference groups” developed • Identify indicator candidates from previous reporting activities, health system scorecard, etc. • Apply decision tree with defined criteria 7 Selection Criteria Primary Criteria Secondary Criteria • Direct measure (or potential measure) of Ministry strategic goal or priority • Construct validity • Evidence basis • Within hospital control • Responsiveness to change • Availability and timeliness of data • Data quality and reliability • Acceptability and familiarity 8 Classification of Hospital Indicators • Performance indicators • have targets and consequences if hospitals miss their targets • Monitoring indicators • No targets or consequences; might graduate to performance indicators in future • Developmental indicators • No targets or consequences • Data quality or methodological limitations; require further development before useable as performance indicators • Explanatory indicators • provide operational information and may provide context for the interpretation of the performance or monitoring indicators • Not considered candidates for performance indicators 9 Classification of Indicators • Four domains: – – – – – Financial Organizational Patient access & outcomes System integration Patient experience (identified by no indicators yet) 10 Ministry-LHIN Agreements Agreement Performance Indicators Agreement Pilot Indicators (2007-08) Access • 90th %ile wait times for surgeries (cancer, CABG, cataract, hip & knee repl), MRI, CT Quality • Readmission rates for AMI Integration • Rate of ED visits that could be managed elsewhere • Hospitalization rate for ambulatory care sensitive conditions • Median wait time for LTC placement • %a of alternate level of care days (no target for 07/08) • • • • • • • • 11 Change in hospital productivity % of chronic/complex continuing care patients with new stage 2 or greater ulcers Perception of change in quality of care In-hospital cancer deaths as a %age of all cancer deaths Psych readmission rates in hospitals Time to first post-acute home care visit Readmission rates of CCAC clients referred by hospitals % of individuals with multiple psych hospitalizations in the past fiscal year Hospital-LHIN Agreement – Performance Indicators FINANCIAL • Total margin • Current ratio ORGANIZATIONAL • Percentage of full-time nurses 12 Hospital-LHIN Agreement – Performance Indicators ACCESS & OUTCOMES • Readmissions to own facility for specified CMGs (AMI, stroke, COPD, CHF, pneumonia, GI, diabetes) • new stage 2+ skin ulcers (complex ctg care) • Volume indicators 13 The Study • How are accountability agreements working in Ontario? • Are they effective tools for promoting better quality, system management? • Do they reinforce other system activities aimed at performance improvement? 14 Who We Talked To • • • • • • • Key Ministry officials LHIN representatives Researchers & institutes (ICES, CIHI, OHRI) Cancer Care Ontario Quality Improvement groups, experts Associations (OHA, OACCAC) Jt Policy & Planning Cmte staff 15 What is the Overall Picture of Quality within Accountability Agreements? 16 Missing Attributes of Quality • Population health • Equity • Appropriately resourced 17 Quality – or Utilization?? • Heavy emphasis on throughput as proxy for access • Hospital-LHIN performance indicators on volume: – – – – – – – Total (inpatient and day surgery) weighted cases Mental health inpatient days Elderly Capital Assistance Program rehab inpt days Complex Ctg Care weighted pt days Ambulatory care visits (outpatient and ED) Emergency visits Other volumes 18 Measuring Whole System Quality vs Narrow Slices? • Current accountability agreements measure only a small component of a particular attribute of quality – Example: Complex Ctg Care – safety: pressure ulcers • Other safety issues? Med errors? Missed dx? • Danger: divert attention from other NB areas not being monitored 19 Ideas for “Big Dots”? • Safety • A measure of global hospital adverse event rates – Trigger tools? • Integration – Improved measure of ALC days using objective criteria – Measures of continuity and co-ordination between primary care and hospitals • Access – Global measures of access for all surgeries – Wait times for a broad basket outpatient and communitybased services 20 Data Quality Concerns • lack of data • incomplete data • lack of standardized definitions and data inconsistencies across sites – e.g. ALC bed days – physician discretion in coding • over-reliance on administrative data 21 Data Quality • Ideas for improvement – More systematic assessments of data quality – More investment in standardized tools • e.g. for ALC days – include data quality indicator in future accountability agreements? 22 Data Collection Burden • • • • too many indicators? not clear how information will be used lack of dedicated resources for data collection too much emphasis on reporting, not enough on quality improvement • multiple, uncoordinated reporting requirements for: – – – – Accreditation, accountability agreements, CCO, wait times strategy, emerg department reporting, trauma hospital reporting, radiation therapy, LHIN growth funding 23 Targets & Corridors • Targets for performance indicators – negotiated between each hospital and LHIN – Accounts for hospital’s past performance and the hospital’s capacity to manage risk • Corridors: – set for each performance indicator – typically ± 2.5 and 3 standard deviations from the target • Example: corridor for 30-day readmission rates for specified case mix groups is the target plus three times the standard deviation of that number • Performance met if target missed but within corridor 24 The Goal … in Industry 25 Approaches to Quality • Six sigma – Relentlessly shrink your variation in processes – keep your defects to 3.4 per million opportunities • Ontario’s accountability agreements – Huge variation is tolerated – Performance at the lowest 1% is a pass 26 What If You Miss the Target? • For financial indicators – Intense scrutiny at multiple levels – Precedents of trustees appointed for management • Wait times strategy – Financial penalties • For quality indicators – “Discussion with the Board” 27 Indicator Cascade • What is strategy for moving system-wide indicators of performance? • What needs to happen at meso, micro levels of system to get macro level change? • Is there a clear “line of sight” or “chain of accountability” between leaders’ goals and front-line staff goals? • Does measurement system support this top to bottom strategy? 28 Hypothetical Indicator Cascade 29 Accountability Agreements: Are They Aligned Today? 30 Challenges for Future Design • Map accountabilities in one sector which affect another • Address shared accountabilities – Primary care, specialty clinic, cmty services impact chronic dis mgt & hospitalizations • How to handle accountabilities to primary care for downstream impacts – E.g. primary care access => ED visits 31 Alignment with QI Campaigns • Existing campaigns: – Wait Times Strategy • ↓ waits for cancer, caaract, hip & knee surgery, CT, MRI – Safer Healthcare Now • Medication reconciliation, infections (surgical site, central line, ventilator associated pneumonia, improve AMI care, rapid response teams (↓ preventable codes / deaths) – FLO Collaborative • Improve patient flow from hospital to cmty, LTC, home care – Quality Improvement & Innovation Partnership • Improve diabetes, colorectal screening, access in FHTs 32 Alignment with QI Campaigns • Indicators, targets to support wait times strategy • FLO Collaborative – Little reflection in accountability agreements • ALC in partial use in some LHINs; data quality concerns • Safer Healthcare Now – No accountability indicators to reflect campaign • AMI readmits relate to post-discharge, not care in hosp 33 Opportunities to Reinforce QI Initiatives • Track similar indicators in initiative & AAs? • Set targets for improvement in AAs, to reflect targets in QI projects? • Set accountabilities for participation or active engagement? 34 Public Reporting Landscape 35 Public Reporting Activities • Not well aligned with accountability agreements – (with exception of wait times strategy) – Information on actual performance results in AAs not available nor not easily accessible to public – Some inconsistent definitions • e.g. hospital readmissions – Public reporting on many issues not covered in AAs 36 Public Reporting - Ideas • More centralized reporting • More transparency of accountability agreement indicator data to public • Better coordination of reporting efforts 37 Key Points • Problems with alignment of indicators at different levels, QI, public reporting, AA’s • Stronger mechanisms of accountability for financial vs quality indicators • Indicators capture “slices”, not big picture – Driven by availability of data 38 Aim for Strategic Alignment of Indicators • • • • • Public reporting Corporate dashboards of organizations Accountability agreements Accreditation Major QI initiatives – Safer Healthcare Now, FLO Collaborative, QIIP Family Health Team/CHC Initiative 39 Contact Us • Email: [email protected] • Website:www.ohqc.ca – Download 2008 Report and Summary – Accountability Agreements white paper – Free quality improvement tools & resources 40