Document 193086

Definitions
‰ Assessment
: student assessment
Introduction
to Student Assessment
‰ Evaluation = Measurement + Judgement
g
: course evaluation
: curriculum evaluation
‰ Measurement
Danai Wangsaturaka, MD PhD
The Faculty of Medicine, Chulalongkorn University
[email protected]
Danai Wangsaturaka, MD PhD
Spotlight on…
Why Assess ?
•
•
•
•
To classify student achievement
- pass/fail
- A,
A B,
B C,
C D,
D F
To motivate student learning
To provide feedback to students
To provide feedback to teachers
WHEN
Danai Wangsaturaka, MD PhD
Why Assess ?
Summative
Formative
Making judgement
Provide feedback
Danai Wangsaturaka, MD PhD
Danai Wangsaturaka, MD PhD
Danai Wangsaturaka, MD PhD
How to Assess
Assessment methods
or
Assessment instruments
Danai Wangsaturaka, MD PhD
1
Written Exams
Written Exams
‰ Selected Response Questions (SRQs)
‰ Selected Response Questions (SRQs)
‰ Constructed Response
p
Q
Questions ((CRQs)
Q)
‰ Constructed Response
p
Q
Questions ((CRQs)
Q)
‰ short answer questions
Any questions requiring a candidate to generate
an answer rather than select from a small set of
options
(Bennet & Ward, 1993)
‰ essays, referral letters, etc.
‰ MEQs, PMPs
Danai Wangsaturaka, MD PhD
Danai Wangsaturaka, MD PhD
How to Assess
Assessment methods
• written examination
• computer-based
p
exam
• oral examination
• long case examination
• OSCE
• direct observation
• portfolio assessment
Assessment instruments
• MCQs
• EMIs
• MEQs
• Checklist
• Rating scales
• Scoring rubrics
Danai Wangsaturaka, MD PhD
Some Methods/Instruments
•
•
•
•
•
•
•
•
•
Checklist
Yes/No
Rating scales
1 = very poor, 10 = very good
Scoring rubrics
1 = ……………………………
2 = ……………………………
3 = ……………………………
Danai Wangsaturaka, MD PhD
Who is best placed to assess ?
Depend on what and why we assess
• paper-based/computer-based
• self
lf
• teacher
• peer
• co-worker
• customer
360 degree
incognito customers
mini-CEX
DOPs
OSCE
portfolio
Danai Wangsaturaka, MD PhD
Danai Wangsaturaka, MD PhD
How to Assess
Danai Wangsaturaka, MD PhD
2
When to Assess
Depend on why we assess
• end of course
• continuous assessment
• flexible
• frequency
Principle of
Educational Assessment
Danai Wangsaturaka, MD PhD
The Faculty of Medicine, Chulalongkorn University
Danai Wangsaturaka, MD PhD
Reliability (Reproducibility)
Reliability (Reproducibility)
When you measure something repeatedly and
you always get the same results, your results
are reliable.
li bl
It can be measured by:
- different investigators/assessors
- the same assessor but on different occasions
Danai Wangsaturaka, MD PhD
Reliability (Reproducibility)
Danai Wangsaturaka, MD PhD
Reliability (Reproducibility)
Some suggestions
Why does it matter to our students?
Danai Wangsaturaka, MD PhD
Danai Wangsaturaka, MD PhD
 Do not rely on short tests
 Do not rely on tests with few items
Danai Wangsaturaka, MD PhD
3
Validity
test
test result
Developing blueprint
• One-dimension
- discipline-based
- use of cognitive level
interpretation
(recall, interpretation, problem-solving)
intention
• Two-dimension
Does it measure what it is intended to measure?
Choose the test according to the objective
Danai Wangsaturaka, MD PhD
Danai Wangsaturaka, MD PhD
Hx PE Dx Ix Mx Total
Hx PE Dx Ix Mx Total
1.
2.
3.
4.
5.
Infection
Neoplasm
RS
CVS
GI
1.
2.
3.
4.
5.
Thyroid
Breast
Esophagus
Stomach
S
Small bowels
6.
7.
8.
9.
GU
Neuro
Psychiatry
Endocrine
6.
7.
8.
9.
Large bowels
Pancreas
Liver
Vascular
TOTAL
TOTAL
Danai Wangsaturaka, MD PhD
Principles of Educational Assessment
•
•
•
•
•
Reliability
Validity
A
Acceptability
t bilit
Educational impact
Cost
Danai Wangsaturaka, MD PhD
Principles of Educational Assessment
U = Rw x Vw x Aw x Ew
Cw
( adapted from Van der Vleuten, 1996)
Danai Wangsaturaka, MD PhD
Danai Wangsaturaka, MD PhD
Danai Wangsaturaka, MD PhD
4