Site Improvement through
Monitoring Systems (SIMS)
Josef Amann
Interagency SIMS Core Group Member
Introduction to Quality Improvement Course
Cape Town, South Africa November 2014
Delivering a Sustainable AIDS-free Generation with Accountability, Transparency, and Impact
Site Improvement through Monitoring System (SIMS)
Site Improvement through
Monitoring System (SIMS)
2
Site Improvement through Monitoring System (SIMS)
Goals and Objectives
Goal
The goal of SIMS is to increase the impact of PEPFAR programs on the HIV epidemic through
standardized monitoring of quality, at the site, community, and above-site level, focusing on
key program area elements
Primary Objectives
1. Monitor capacity at sites, community and above-site to provide high-quality HIV/AIDS
services in all program areas
2. Facilitate use of these data and quality outcomes to improve services
3. Provide foundational data for regional, national, and global programmatic decision
making
Secondary Objectives
1. Demonstrate PEPFAR and its implementing agencies’ accountability for funded activities
2. Emphasize in-country technical staff role in accountability, monitoring, and improvement
3. Improve capacity building by:
• Providing guidance to assess the content of services through standards-based
monitoring
• Recognizing successes to improve morale and accountability of staff
3
SIMS TIMELINE
June –July 31st
• Developing modular
toolkit for “sites” where
PEPFAR services are
delivered (facility and
community)
• Orientation meeting for
20 countries
June-September
• Developing above-site
and TA modules for all
program areas and
budget codes
September 30th
• First quarter of data
collected
• Minimum 1 site visit
per mechanism
January 2015
• All OUs
collecting on all
modules
October 2014
• Meeting for
remaining OUs
March 2015
• Opportunity
to revise all
CEEs
4
Where in the world is SIMS?
Oriented to SIMS Facility and Community tools at Johannesburg Workshop;
implementing SIMS; will be oriented to Supportive Functions/Above-Site tool virtually
Will be oriented to all SIMS tools at Bangkok Workshop in October
What is a SIMS Site Visit?
• A rapid assessment of services provided at a facility or
community or service-delivery point of care or program
intervention
• The assessment uses a tool of core essential elements
(quality standards) to assess all PEPFAR-funded
program areas of a facility, community and above-site
• The SIMS assessment consists of:
– Discussions with key facility and implementing partner
staff
– A facility walk-through, and chart review (if applicable)
• Conducted by USG staff (different from quarterly
Implementing Partner visits required in the MER)
6
Defining a ‘Site’
• Sites are defined within the PEPFAR Monitoring, Evaluation and Reporting
(MER) indicator reference sheet PEPFAR Site Support (SITE_SUPP)
• Site definitions are consistent among SIMS, MER, and integrated PEPFARsupported Site List (iPSL)
• Facility Site: SIMS adopts the existing definitions for facilities from the
MER
• Community Site
– The lowest geographic administrative unit
– Could be a “ward” or “sub-district”
– Population-level data should be available
– Multiple domains/types of interventions can take place in a site
• Above-site/supportive function site assessment point
7
SIMS has 3 modular toolkits
1. Facility
2. Community
3. Above-Site/Supporting Functions
All modules are finalized and should be adapted,
translated and implemented in-country
8
Above-Site/Supporting Functions
standards
Facility
standards
Community
standards
Improved quality of
PEPFAR-supported
services
Sustained HIV Epidemic Control
AFTER THE SIMS ASSESSMENT IS
CONDUCTED – WHAT’S NEXT?
OR HOW ARE WE MAKING SURE THINGS CHANGE –
IMPROVEMENT HAPPENS
10
Actors/Players
• Health Care Facility, Community-based Staff
– Run or operate the health care facility, CBO, intervention point, etc.
– provide the HIV services
• Implementing Partner
– supports the health care facility staff or community site/intervention point
(technical assistance, training mentoring, salary support, commodities etc)
– has to be physically present at the site at least once per quarter (MER
requirement)
• USG in-country staff
– Oversight of the implementing partners activities
– Monthly meeting with the implementing partner
– Conducts the SIMS assessment
• MOH, Provincial or District Health Management Teams
– Supervision/mentoring functions
Guiding Principles
• SIMS is conducted to ensure high quality HIV
services and programs and to improve the quality
• SIMS is a major part of oversight of the
implementing partner by the USG in-country staff
• SIMS Data is the most useful in-country and
should be used primarily in-country by USG staff
when monitoring/working with the
implementing partner
• Implementing partner is main facilitator of
improvement
Quality Assurance vs. Quality Improvement
13
Quality Assurance vs. Quality Improvement
SIMS
14
Quality Assurance vs. Quality Improvement
SIMS
15
Improvement – at different times in
different forms
• Site feedback and debrief
• Quality improvement project at the site
• Implementing partner monitoring and oversight
by the USG staff
• Program-level review and improvement
16
IMPROVEMENT: SITE FEEDBACK AND
DEBRIEF
17
Site Feedback – Improvement starts here
• Sites or Intervention Points should receive sameday feedback in the form of a Facility or
Community Dashboard and a Debrief
• Site involvement and feedback critical
components to SIMS
• Promotes staff ownership and accountability for
services they are providing and for improving
them
• Non-punitive approach
• Provide accompanying tool as explanation
18
SIMS Dashboard
SIMS
Dashboard
• Left at Site
• No Site
composite
score
indexed or
CEEs
differently
weighted
19
Debrief -- Improvement
•
•
•
All actors/players are there
Some issues can be address or fixed right there, no
need of a root cause analysis or a formal QI project
Some issues just have to be brought to the attention of
the health care facility staff or intervention point staff
IMPROVEMENT: QUALITY
IMPROVEMENT PROJECTS AT SITE
21
Quality Improvement Projects at a site
• Issues or gaps identified through the SIMS
assessment may become the topic of a Quality
Improvement (QI)Project at the health facility site
Example:
Liwale District Hospital scored RED for Cotrimoxazole during the SIMS
assessment
2.1 Cotrimoxazole
All patients should have documented prescription of cotrimoxazole (CTX), according to national guidelines.
Q1
Question
Pull 10 charts. What percent of charts reviewed
have documentation of CTX prescription within
an appropriate interval per the national
guidelines at the last clinical assessment?
SCORE
Response
Y
N
Scoring
If <60%=Red
If 60-79%=Yellow
If 80-89%=Light
Green
if ≥90%=Dark
Green
Quality Improvement Projects at a site
QI Team Liwale District Hospital
Quality Improvement Projects at a site
QI Team Liwale District Hospital
• Will identify Cotrimoxazole as
topic for a QI project
Quality Improvement Projects at a site
QI Team Liwale District Hospital
• Will conduct a root cause
analysis
Quality Improvement Projects at a site
QI Team Liwale District Hospital
Quality Improvement Projects at a site
• QI projects are supervised by the
implementing partner
– Documentation of results/improvement
– Report to USG staff in monthly partner meeting
– Dissemination of best practices identified in one
site to all other sites they support
IMPROVEMENT: IMPLEMENTING
PARTNER MONITORING AND OVERSIGHT
BY USG STAFF
28
Implementing Partner Monitoring
System/Process in USG-Country offices
• Formal SIMS reports to the implementing
partners
• Meetings (e.g. monthly) of the Activity
Manager/Project officer with the
Implementing Partner
• Meetings (e.g. quarterly) with all
Implementing Partners around specific
topics/issues
Implementing Partner Monitoring
System/Process in USG-Country offices
• Formal SIMS reports to the implementing partners and the
MOH
Implementing Partner Monitoring
System/Process in USG-Country offices
• Formal SIMS reports to the implementing
partners with Action Plan
All Sites of one IP in one Region
32
IMPROVEMENT: PEPFAR COUNTRY
PROGRAM OVERALL
33
Look at the PEPFAR Program Overall
• Look at the HIV program overall (across all partners)
– Improvement could mean re-focus of program activities/resources
Overall SMS Score Card of one CDC-country program
Thank You!