1. finance
2. accounting and auditing

How to perform clinical audit?
Dr. Mithali Abdullah @ Jacquline Sapen
MSc. OSHM (UUM), M.B.B.S (Aust.), OHD (NIOSH), CMIE (NIOSH)
Timbalan Pengarah (Perubatan)II
Hospital Sultan Abdul Halim, Sungai Petani, Kedah
WHAT
HOW
WC
WHY
What is your cause?
What motivates you
to do clinical audit?
Source: Start with Why (2009) Simon Sinek
Road Map of talk
Revisit of Clinical audit
 Clinical audit cycle
 Stages in clinical audit

INTRODUCTION
“Quality is not an accident, it is the result of
high intentions,
sincere efforts,
intelligent direction and
skillful execution”
Clinical Audit

Clinical audit is a specific form of audit that
involves measuring clinical practice against
standards. An audit within the clinical setting
is not necessarily a clinical audit project.

Clinical audit would measure process( are
we doing the things we should do?)

Clinical audit could also look at outcomes:
such as monitor the success of a treatment
which is known to work, rather than to find
out whether it works.
Clinical audit vs Research
Clinical
Audit
Research
• Ask the question „are we following
agreed or evidence based practice?‟
• Seek to find out if we are achieving the
things we agreed we should achieve
• Ask the question ‘what is the best
practice?‟
• Researches will seek to find new
knowledge
Audit Cycle

The clinical audit process is known as the „audit cycle‟.
Choose a
topic
Re-audit to
look for
improvement
Review
standard
Implement
change to
improve
Collect data
on current
practice
Compare data
collected with
standard
Source: NICE 2002
STAGES IN
CLINICAL AUDIT
Stages in clinical audit
Stage 1: preparation: management & methodology
Stage 2: selecting criteria and standard
Stage 3: measuring level of performance
Stage 4: making improvements
Stage 5: sustaining improvements
Stage 1: preparation
Good
preparation is crucial to the success of an audit
project.
“If
you fail to plan, you plan to fail.”
The
audit team should decide in advance the audit project
for the forthcoming year, time frame for data collection,
design and format of the necessary data collection tools.
Ref: Principals for best practice in clinical audit. National Institute for
Clinical Excellence. 2002. Radcliffe Medical Press Ltd.
2 areas to be considered in
preparation
Project management
•
•
•
•
•
Team
Leader
topic selection,
planning and resources,
communication
Project methodology
• Study design
• Data issue: collection,
analysis, storage
• Implementability
• Stakeholder
involvement: patients,
other HCW
• Support for local
improvement
Stage 1: Step 1
PROJECT MANAGEMENT
The team



1.
2.
3.
4.
5.
6.
Identifying the skills and people needed to carry out the audit, and
training of staff and encouraging them to participate.
Involve the right people with the right skill from the start.
Certain skills are needed, which include:
Project leadership, Project management, project organization
Clinical, managerial, and other service input and output
Audit method expertise
Change management skills
Data management: data collection, data entry, data analysis, &
data presentation
Facilitation skills.
The Team


1.
2.
3.
4.
5.
6.
Audit team comprises :
staff from all relevant groups involved in the care delivery, audit staff
All audit team members:
must understand the processes of clinical care concerned
Must have a basic understanding of clinical audit
Understand the purpose of the audit
Committed to the plan and objectives of the audit
Understand what is to be expected of them – specific roles &
responsibility: leader, data management, communication, etc.
Must understand the ground rules for meeting
The leader
The leader
“Leadership is the art of getting someone else to do
something you want done, because he wants to do it.”
~ Dwight D. Eisenhower
Topic selection
1.
2.
Involve users (health care providers or patients) in the
process
Based on SMART crtiteria
Involve users (health care providers or
patients) in the process

1.
2.
3.
4.
5.
Sources of user information:
Critical incident reports
Complaints or comments
Direct observation of care
Direct observations
Focus group discussion
Selection of topic: SMART





Seriousness: patient safety issues, high volume, high
risk, high cost
Measurable: any evidence-based standard, feasibility of
data management
Appropriateness: is the problem important? Is the
problem a priority?
Remediable: Is the problem concern amenable to
change?
Timely: should we deal with the problem now or later?
Defining the purpose of audit



1.
2.
3.
4.
The aims of the audit must be defined.
Key words that could be used include: to improve, to enhance, to
ensure, to change.
E.g. :
To improve blood transfusion process in the hospital
To ensure that Safe Surgery Save Life Initiative is implemented
To enhance the referral of stroke patients in the hospital
To increase the percentage of patients with controlled blood
pressure.
Purpose of audit
What aspects of care that concern us?
 Which aspect of care that you would like to
audit?

Effectiveness
• Is the treatment being administered
correctly & does it have the desired
effect?
Efficiency
• Is this approach achieving the desired
outcome with minimal effort, expense, &
wastage ?
Equity
• Do all patients have equal access to care?
Purpose of audit
Accessibility
• Is it easy for patients to get
treatment? Waiting time?
Appropriateness
• Is this the right management
approach?
Acceptability
Timeliness
• Is the treatment acceptable to
patients? As per standard/guideline?
• Is the care being provided at the
correct time?
Providing the necessary structure
1.
2.
A structured audit program is needed: committee structure,
feedback mechanism, regular audit meeting
A team of well-qualified audit staff.
Funding is important.This include resources for staff training &
administration, knowledge of clinical audit technique, data
management.
Time is another important barrier in successful audit. Participants
should be given the time to participate in the project.
Communication
Inform all those who are going to be involved.
 Platforms for communication include:
memo/letter, e-mail, meeting

Stage 1: Step 2
PROJECT
METHODOLOGY
Project methodology





Design of study: retrospective , prospective, cohort, case
studies, case review
Data collection: what, where, when, how
Sample : subjects to be selected. Inclusion & exclusion
criteria
Sampling technique & sample size
Type of data analysis: qualitative, quantitative
Study design: retrospective or cross
sectional data collection
1.
2.
Retrospective data collection:
look at previous records.
Could be difficult to trace relevant records.
Time consuming
Relevant information required may not be available: incomplete
records
Cross sectional data collection:
gives audit team more immediate feedback on its current
performance
Relevant information required would be easier to retrieve:
health care providers informed prior to study on documentation
of information.
Data collection
What: collect data that is related to audit criteria
 Who(samples) from whom these data is gathered?
Patients? Healthcare providers? Inclusion/Exclusion
criteria
 Who: by trained staff.
 Where: unit/department

Data collection



When: inpatient, outpatient, on admission, after
discharge, during administration of medicine, during
observed procedure?
How: include observational checklist, questionnaires
(self-completed or interviewer completed), patient
medical records, interviews
Data storage: data collection format
An Audit on the
Use of Granisetron Injection for
Chemotherapy-Induced Nausea and Vomiting
in Hospital Sultanah Bahiyah
Pharmacy Department
Supervisor : Malathi Sriraman @ Jayaraman
Group Leader : Chan Huan Keat
Group members: Khor Seau Ting
Tan Say Li
Data collection pertaining to audit criteria
Quality of Care Selected
Rational use of granisetron injection in terms of:
(a) Indication.
(b) Dose.
Audit Criteria & Standard
NO
Quality of Care
Criteria
Standard
1.
Indication of
granisetron
injection
Only prescribed in moderately and highly
emetogenic chemotherapy unless it is justified.
100 %
2
Dose of
granisetron
injection
Only prescribed at the lowest effective dose, which
is 1mg or 0.01mg/kg unless it is justified.
100%
Ref. Systemic Therapy of Cancer 2nd Edition, Ministry of Health Malaysia, 2007.
34
Objective 1: To determine the adherence of prescribers to recommended
indications and doses of granisetron injection by the updated guidelines.
Study
Design
Prospective study:
To check prescriptions for adherence to the
recommended indications & doses.
Inclusion
Criteria
All adult chemotherapy prescriptions
prescribed with granisetron injections
received from Jan-Mac 2011.
Exclusion
Criteria
Prescriptions with incomplete information.
Population
Size
N = 432 prescriptions.
All included.
Data
Collection
1. Receive & screen the prescriptions.
2. Record the required information
into Data Collection Form A.
Data
Analysis
SPSS 16:
(a) Descriptive analysis.
35
Data collection


You may like to know the factors that contribute to
substandard performance such as knowledge,
perception and practice.
Thus, design data collection tool to collect such data.
Include questionnaire (patients or HCW) &
observational checklist (to determine if knowledge is
translated into practice)
Objective 2: To assess prescribers’ perception of the granisetron
injection utilization in the prevention of CINV.
Study
Design
Cross-sectional study:
To study the prescriber’s perception using a
validated questionnaire modified from
Tajunisah M.E. et al (Cronbach’s α = 0.72). 8
Inclusion
Criteria
All doctors in 10 wards that provides
chemotherapy during Jan-Mac 2011.
Sampling
Method
Convenient sampling.
Sample
Size
n = 63 doctors.
Data
Collection
1. Distribute Data Collection Form B (selfreported questionnaire) to wards.
2. Collect the forms back after completely
answered by doctors.
Data
Analysis
SPSS 16:
1. Descriptive analysis.
2. Inferential analysis: Chi Square tests.
8. Tajunisah M.E. et al, Physicians’ Perception and Adherence to Guideline on the Management of Chemotherapy
Induced Nausea & Vomiting (CINV).
Hospital Pulau Pinang, 2009.
37
PATIENT FALLS AUDIT FORM. 003/HSAH/UKL
Names of Auditors:___________________
Ward/Department:____________________
Date:_______________
1.
2.
3.
4.
5.
6.
7.
8.
BHT
1
BHT
2
BHT
3
BHT
4
BHT
5
BHT
6
Has fall risk assessment been completed
within 48 hours of admission?
Is level of risk for fall documented in the
record?
Is there an appropriate care plan for the
patient based on the level of risk?
Is there evidence that the patient has been
reassessed accordingly?
Is there evidence that there is action plan to
reduce risk of falling?
If a fall occur, is there any evidence of the
circumstances surrounding the patient?
For a patient who has fallen, is there any
evidence that other strategy have been
implemented?
Is there any evidence that doctors/
physiotherapist/occupational therapist have
been involved?
Total √
%√
Adapted from Barnet, 2001.
3 August 2009
BHT
7
BHT
8
BHT
9
BHT
10
Total √ %
√
Data protection




Data collection procedure must be ethical
When data is obtained directly form patients, auditors must explain
why the information is needed and what will happen to it, before
asking the patient‟s consent.
If the information is obtained from medical records:
i) patients consent to the retrieval of records and data being used
or
ii) data should be made anonymous before the audit start
or
Obtained ethical approval from national ethic committee.
Confidentiality of data
Stage 2
SELECTING CRITERIA
& STANDARD
Stage 2: selecting criteria & standard



Clinical audit is a specific form of audit that involves measuring
clinical practice against standards
So, there must be criteria and standard.
Criteria: i) explicit statement that define what is being measured
ii) represent elements of care that can be measured
objectively
Criteria: an item or variable which enables the achievement of a
standard & the evaluation whether it have been achieved or not
Standard: level of care that need to be achieved for the particular
indicator.
Criteria

1.
2.
3.
Classified into:
Structure: what you need
Process: what you do
Outcome: what you expect
Criteria

1.
2.
3.
Structure: include
staff: no. of staff, skill mixed, ratio of staff to patients.
Equipment: technology, number of equipment
Space provided: number of OT functioning, room for
counseling
Criteria

1.
2.
3.
4.
5.
6.
7.
8.
Process refer to actions & decision by healthcare providers
Communication
Assessment
Education
Investigation
Prescribing
Therapeutic intervention
Evaluation
Documentation
The importance of process criteria is determined by the extent
to which they influence outcome.
Department
KPI
Anesthesiology Percentage of
Elective And
Emergency Surgery
Patients Who
Receive Acute Pain
Service
Standard
Achievement
>10%
5.1%
(204/4001 x
100%)
Process Criteria: pain assessment
Criteria

Outcome criteria are typically measures of physical or
behavioral response to an intervention, reported health
status, level of knowledge and satisfaction

More often proxy or intermediate outcome criteria are
used.
Outcome Criteria
Department
Dietetic &
Catering
NIA
Standard
Incidence of Physical
Contamination of Food
Served to Patients
Sentinel events
Indicator: Delay in
Response to In-patient
Referral by Dietitian
(for critical case)
Achievement
2 cases
Standard : ≤ 5
%
Process Criteria: referral process
6.1%
(25/407 X 100%)
(Repeat SIQ. 11.8%:
Jul-Dec 2009)
How to develop valid criteria?

1.
2.
3.
4.
5.
6.
Valid criteria must meet the following:
Evidence based
Related to important aspects of care
Measurable
Specific
Achievable
Relevant
How to develop valid criteria?

1.
2.
3.
Sources for developing criteria:
Existing guidelines: clinical practice guidelines (CPG),
national policy
Systematic literature review relevant to topic
Review of other practices elsewhere (when you
cannot find information in the literature)
Stage 3
MEASURING LEVEL OF
PERFORMANCE
Stage 3: measuring level of performance




Data storage
Data analysis
Comparing performance with standard
Any gap in performance (short fall in quality, SIQ)
Data analysis: compare with standard
NO Quality of Care
1.
Indication of
granisetron
injection
Low Emetogenic
(Level 1&2)
38.0%
(164 Rx)
Criteria
Standard
Only prescribed in moderately and highly
emetogenic chemotherapy unless it is justified.
100 %
Moderate Emetogenic
(Level 3)
19.9%
(86 Rx)
38% non-compliant to
indication
42.1%
(182 Rx)
High Emetogenic
(Level 4)
52
Data analysis: compare with standard
NO Quality of Care
2
Dose of
granisetron
injection
Criteria
Standard
Only prescribed at the lowest effective dose,
which is 1mg or 0.01mg/kg unless it is justified.
100%
0.7% Granisetron 1mg
(3 Rx)
Only 0.7% compliant to
dose of 1 mg
99.3%
(429 Rx)
Granisetron 3mg
53
Stage 4
MAKING
IMPROVEMENT
Stage 4: making improvement
Analysis of problems: why is there a gap in
performance?
 Identify areas to improve
 Formulate strategies to overcome problems
 Communicate action plan
 Implement strategies
 Evaluate strategies

SIQ



Is there gap in performance?
Is the gap in performance real?
Exclude error in data collection: inclusion criteria,
exclusion criteria, data analysis, data entry, data
calculation
Identify areas to improve
Conduct root cause analysis
 Could be done earlier during data collection: has been
discussed in project methodology.
 Literature review helps in identifying causes for
substandard performance.
 Data collection tools designed to measure these causes:
knowledge, perception & practice

To assess prescribers’ perception of the granisetron injection utilization
in the prevention of CINV.
Prescribers’ perception:
Why are guidelines not widely-adapted in HSB
practice?
Lack of awareness
33.3% (21)
Lack of guideline availability
36.5% (23)
Lack of guideline enforcement
42.9% (27)
n=63 prescribers
There are no association between prescribers’ perception and all the factors tested including
departments and clinical experiences (p>0.05).
58
Formulate strategies to overcome
problems
Targeted
root cause
What to
change
How to
change
When to
change
Who is
responsible
Stage 1 of strategy for change
Targeted contributing
factors
AWARENESS
What to change?
To improve the awareness among the
prescribers and pharmacists.
How to change?
At INDIVIDUAL level:
۩
CME to update the prescribers on the
latest guidelines.
۩
CPE to educate the pharmacists on the
chemo prescriptions screening.
When to change?
2 months (Nov – Dec 2011).
Who is responsible?
۩
۩
CDR Pharmacy unit.
DIC Pharmacy unit.
Stage 2 of strategy for change
Targeted contributing factors
GUIDELINE AVAILABILITY
What to change?
To ensure updated guidelines are
available in all the related wards and
daycare clinics.
How to change?
At PRACTICE SITE level:
۩
To distribute the updated
antiemetic guidelines in poster
form to facilitate the rational
prescribing among the prescribers.
When to change?
2 months (Jan – Feb 2012).
Who is responsible?
۩
۩
CDR Pharmacy unit.
Hospital Health Education Unit.
Stage 3 of strategy for change
Targeted contributing factors
GUIDELINE ENFORCEMENT
What to change?
To ensure the adherence to the
antiemetic guideline in the hospital.
How to change?
At SERVICE level:
۩
Order form to require justification
from oncologists/ specialists in
non-indicated or high dose cases.
۩
CDR unit to prepare granisetron
injection in 1mg syringe form to
encourage the use of low dose.
When to change?
2 months (March – April 2012).
Who is responsible?
CDR Pharmacy unit.
Communicate action plan





What to tell: why audit is done, problem statement,
criteria & standard, major & important findings,
strategies for change
How to communicate
Who to tell
When to tell
Where to tell
Communicate action plan
Many ways to do this:
1. meeting: intra-department, inter-department, hospital
level, state level
2. Training session: CME/CPD, seminar
3. Memo/letter
4. Bulletin, newsletter
5. Information board
6. E-mail

Present clinical audit and findings & what to do to improve
the performance
Implement strategies
Strategies for CHANGE is all about CHANGE
 Observe the stages for change

Awareness
Agreement
Acceptance
Assimilation
Accountability
Action
Stage 5: sustaining improvement
Do another cycle of data collection and analysis
 Do trending of performance

4 kinds of motivation
“Do this clinical audit
& you’ll get a bonus”
“I really want to do this
clinical audit ”
“I really don’t want to
“Do this clinical audit or
do this clinical audit ”
you’ll get low mark on
your annual appraisal”
Negative
Motivations away from something
+
Intrinsic
You want to do it
Extrinsic
Someone wants you to do it
Positive
Motivations towards a goal
+ only this one
create positive,
sustainable
motivation