1. No category

Response to Intervention
Predicting Success in General
aandd Spec
Special
a Education:
ducat o How
o to
Select and Chart 'Ambitious But
Realistic' Student Academic
Realistic
Goals
Jim Wright
www.interventioncentral.orgg
www.interventioncentral.org
Response to Intervention
Workshop PPTs and handout available at:
http://www.interventioncentral.org
www.interventioncentral.org
Response to Intervention
Workshop Agenda
RTI and Goal-Setting: Introduction
How to Set Goals for Students at the Acquisition
Stage of Learning
How to Estimate ‘Typical
Typical Peer Performance’
Performance
How to Set ‘Off-Level’ Goals for Students With
Significant Reading Delays
How to Structure Data Collection to Make Sense:
Baseline, Goal, Progress-Monitoring
How to Set Up Progress-Monitoring Charts to
Create Visual Displays of Data
www.interventioncentral.org
Response to Intervention
Goal-Setting: Key Concepts
www.interventioncentral.org
4
Response to Intervention
RTI ‘Pyramid of
Interventions’
Tier 3
Tier 2
Tier 1
Tier 3: Intensive interventions.
interventions
Students who are ‘nonresponders’ to Tiers 1 & 2 are
referred to the RTI Team for more
intensive interventions.
Tier 2 Individualized
interventions. Subset of students
receive interventions targeting
specific needs.
Tier 1: Universal interventions.
interventions
Available to all students in a
classroom or school. Can consist
of whole-group or individual
strategies or supports.
www.interventioncentral.org
5
Response to Intervention
Educational Decisions and Corresponding Types of
Assessment
•
•
•
•
SCREENING/BENCHMARKING DECISIONS: Tier 1: Brief
screenings to quickly indicate whether students in the general
generaleducation population are academically proficient or at risk.
PROGRESS-MONITORING DECISIONS: At Tiers 1, 2, and 3,
ongoing ‘formative’ assessments to judge whether students on
intervention are making adequate progress.
INSTRUCTIONAL/DIAGNOSTIC DECISIONS: At any Tier,
Tier detailed
assessment to map out specific academic deficits , discover the root
cause(s) of a student’s academic problem.
OUTCOME DECISIONS: Summative assessment (e.g., state tests) to
evaluate the effectiveness of a program.
Source: Hosp, M. K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBM: A practical guide to curriculum-based
measurement. New York: Guilford Press.
www.interventioncentral.org
6
Response to Intervention
Interventions: Potential ‘Fatal Flaws’
Any intervention must include 4 essential elements. The absence
of any one of the elements would be considered a ‘fatal flaw’
(Witt VanDerHeyden & Gilbertson,
(Witt,
Gilbertson 2004):
1. Clearly defined problem. The student’s target concern is stated in
specific, observable, measureable terms. This ‘problem
id tifi ti statement’
identification
t t
t’ is
i th
the mostt iimportant
t t step
t off th
the
problem-solving model (Bergan, 1995), as a clearly defined
problem allows the teacher or RTI Team to select a well-matched
intervention to address it.
2. Baseline data. The teacher or RTI Team measures the student’s
academic skills in the target concern (e
(e.g.,
g reading fluency
fluency, math
computation) prior to beginning the intervention. Baseline data
becomes the point of comparison throughout the intervention to
help the school to determine whether the intervention is effective.
effective
Source: Witt, J. C., VanDerHeyden, A. M., & Gilbertson, D. (2004). Troubleshooting behavioral interventions. A systematic
process for finding and eliminating problems. School Psychology Review, 33, 363-383.
www.interventioncentral.org
7
Response to Intervention
Interventions: Potential ‘Fatal
Fatal Flaws
Flaws’ (Cont.)
3. Performance goal. The teacher or RTI Team sets a specific,
p
duringg the intervention
data-based ggoal for student improvement
and a checkpoint date by which the goal should be attained.
4. Progress-monitoring plan. The teacher or RTI Team collects
student data regularly to determine whether the student is onon
track to reach the performance goal.
Source: Witt, J. C., VanDerHeyden, A. M., & Gilbertson, D. (2004). Troubleshooting behavioral interventions. A systematic
process for finding and eliminating problems. School Psychology Review, 33, 363-383.
www.interventioncentral.org
8
Response to Intervention
Avg Classroom Academic
Performance Level
Discrepancy 1: Skill Gap
Discrepancy 2:
(Current
Gap in Rate of
Performance Level)
Learning (‘Slope
( Slope
Target
of Improvement’)
Student
‘Dual
Dual-Discrepancy
Discrepancy’:: RTI Model
of Learning Disability (Fuchs 2003)
www.interventioncentral.org
9
Response to Intervention
Big Ideas: The Four Stages of Learning Can Be
S mmed Up in the ‘Instructional
Summed
‘Instr ctional Hierarch
Hierarchy’’
(Haring et al., 1978)
Student learning can be thought of as a multi
multi-stage
stage process
process. The
universal stages of learning include:
• Acquisition: The student is just acquiring the skill.
• Fluency:
Fl
The
Th student
t d t can perform
f
the
th skill
kill bbutt
must make that skill ‘automatic’.
• Generalization: The student must pperform the skill
across situations or settings.
• Adaptation: The student confronts novel task
demands that require that the student adapt a
current skill to meet new requirements.
Source: Haring, N.G., Lovitt, T.C., Eaton, M.D., & Hansen, C.L. (1978). The fourth R: Research in the classroom.
Columbus, OH: Charles E. Merrill Publishing Co.
www.interventioncentral.org
10
Response to Intervention
Goal-Settingg in Your School
At your table:
• Discuss examples of when teachers in your school set
academic goals for specific students for instruction or
intervention.
• How do yyou or the teachers that yyou work with ggo
about setting student academic goals? Do you see a
need for improvement?
www.interventioncentral.org
11
Response to Intervention
Goal-Setting:
Acquisition
Focus of Inquiry: When
ACQUISITION is the
t
target,
t hhow ddo we sett
individual student
academic goals?
www.interventioncentral.org
12
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals packet 3 pp
pp. 11-44
The focus of classroom interventions is often to help
students to acquire a fixed set of academic-skill
academic skill items
(e.g., naming numbers 1-10). When the intervention
supports the acquisition of a finite set of items, timelines
tend to be short (e.g., 1-8 weeks) and the goal is
typically mastery of all items in the academic-item set.
www.interventioncentral.org
13
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
1. Select a Set of Academic Items as the Intervention
Target The teacher decides on a finite set,
Target.
set or 'pool'
pool , of
academic items to be targeted in the intervention.
Examples of possible academic
academic-item
item sets suitable for
intervention are naming of all mixed-case letters;
answering 2-term multiplication math facts 0-12; and
giving definitions for 20 key biology terms.
www.interventioncentral.org
14
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
2. Establish Criteria for Item Mastery. The teacher next
defines the criteria that allow him or her to judge when
the student has mastered any particular item from the
academic-item
academic
item pool. Along with the expectation of a
correct response, mastery criteria usually include
expectations for speed of responding.
Creating criteria for determining item mastery is useful
b
because
th
these criteria
it i allow
ll th
the tteacher
h bboth
th to
t bbe
more consistent and to have greater confidence in
judging whether a particular item has been mastered.
mastered
www.interventioncentral.org
15
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
2. Establish Criteria for Item Mastery-Cont. As an
example of criteria for item mastery
mastery, a first
first-grade
grade
teacher decides that mastery on a mixed-case letternaming intervention should be defined as: "When
When
shown a flash-card with an upper- or lower-case letter,
the student will correctly name the letter within 3
seconds."
To cite a second example, a high-school science
t h whose
teacher
h
iintervention
t
ti iis iintended
t d d tto promote
t
definitions of 20 key biology terms defines mastery as
follows: "When
When shown a biology term,
term the student will
correctly state the definition orally within 10 seconds."
www.interventioncentral.org
16
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
3. Collect Baseline Data. Before beginning the
intervention the teacher determines the student
intervention,
student'ss
baseline level of performance. The easiest way to
collect baseline data is to present each of the items
from the item-pool to the student in random order, have
the student respond, apply the mastery criteria
(developed in the previous step) to determine whether
each item is correct or incorrect, and record the
student's
t d t' responses.
www.interventioncentral.org
17
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
3. Collect Baseline Data-Cont. TIP: If a student tends to
have a high degree of variability in responding—e.g.,
responding e g
on some days the student answers items correctly and
on other days he or she gets those same items
wrong—the teacher may want to inventory the
student's skills across 2-3 successive days and count
as 'known' for baseline only those items the student
can correctly answer across all sessions.
www.interventioncentral.org
18
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
3. Collect Baseline Data-Cont. For example, a firstgrade teacher collects baseline data by showing her
student flash-cards with all 52 mixed-case letters while
applying her mastery criteria: The teacher sorts each
card whose letter the student can correctly name within
3 seconds into a 'known' pile and sorts into an
'unknown' pile those flash-cards that the student
identifies incorrectly or hesitates in responding beyond
3 seconds.
d At th
the endd off th
the session,
i th
the tteacher
h tallies
t lli
the student's responses and discovers that at baseline
he can correctly identify 38 of a possible 52 mixedcase letters.
www.interventioncentral.org
19
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
4. Set an Intervention Exit Goal. The teacher next sets
a student exit goal that defines a successful
intervention. In most cases, the teacher will probably
decide that the intervention is to be judged a success
when the student has met the standard for mastery on
all items in the academic- item pool.
For example, a high school science teacher may set,
as an iintervention
t
ti exitit goal,l th
thatt a student
t d t will
ill bbe able
bl
to correctly define all of the items from a list of 20 key
biology terms.
terms
www.interventioncentral.org
20
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
4. Set an Intervention Exit Goal-Cont. Occasionally,
however the teacher may decide that an alternative
however,
outcome goal is acceptable.
For example, a fourth-grade teacher may set as an exit
goal that a student whose intervention focuses on 2term multiplication facts 0-12 will be able to answer at
least 90 percent of those math facts correctly. In this
t h ' judgment,
teacher's
j d
t 90 percentt proficiency
fi i
on thi
this
collection of math facts will permit the student to
experience sufficient success on math class- and
homework to discontinue the intervention.
www.interventioncentral.org
21
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
5. Decide on the Frequency and Session Length of
the Intervention.
Intervention The teacher decides how long each
intervention session is to last and how many
intervention sessions the student will receive per week.
For students with mild academic deficits, intervention
sessions can be as short as 20 minutes per day, 3
days per week. For students with greater deficits,
intervention sessions may last 30-45 minutes per
session
i andd occur as often
ft as 44-55 days
d
per week.
k
www.interventioncentral.org
22
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
6. Set a Timespan for the Intervention. The teacher
estimates the number of instructional weeks the
intervention should be attempted and sets an end-date
by which the student is predicted to attain success. An
intervention that targets the student's acquisition of a
specific set of academic items is typically of short
duration: between 1 and 8 instructional weeks.
www.interventioncentral.org
23
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
6. Set a Timespan for the Intervention-Cont. Predicting how
l
long
an acquisition
i iti intervention
i t
ti should
h ld last
l t iis more off an artt
than a science. The teacher must exercise professional
judgment selecting a timespan that is both ambitious and
judgment,
realistic.
The frequency and session length of a particular intervention
will affect the timespan. For example, a student whose
intervention is scheduled at a higher 'dosage' (e.g.,
(
daily for
f
40-minute sessions) can be expected to reach the exit goal
faster than a similar student whose intervention is at a lower
'dosage' (e.g., 3 times per week for 20-minute sessions).
www.interventioncentral.org
24
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
7. Monitor the Student's Progress. Throughout the
i t
intervention,
ti the
th teacher
t h can monitor
it th
the student's
t d t' progress
periodically (e.g., weekly or even more frequently) by having
the student attempt all of the items in the item-pool and
recording the results.
For example, the first-grade teacher whose intervention
targets a student's letter-naming skills for mixed-case letters
measures her student's progress by reviewing all 52 letter
flash-cards once per week and, each time, tracking the
number of letters that the student is able to name correctly
within 3 seconds of being shown the flash-card.
www.interventioncentral.org
25
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals
7. Monitor the Student's Progress-Cont. As a second
example,
l th
the high
hi h school
h l science
i
tteacher
h working
ki with
ith a
student on acquiring 20 key biology terms and their definitions
ends each intervention session by having the student attempt
to define all terms, with each vocabulary word counted as
correct if the student defines it correctlyy within 10 seconds.
www.interventioncentral.org
26
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals: EXAMPLE: RICK
At your table:
• Review the case of Rick (next
slide).
)
• Complete the CASE 1 goal
planning form for this student
(packet 3, p. 3).
• Be prepared to report out.
out
www.interventioncentral.org
27
Response to Intervention
When ACQUISITION is the Target: How to Set
Individual Student Academic Goals: EXAMPLE: RICK
• A 5th-grade teacher, Mrs. Fallows, decides to use Cover-Copy-Compare
(CCC) to increase the spelling skills of her student
student, Rick
Rick. Because Rick has
spelling difficulties, the teacher selects for the intervention a 50-item basic
vocabulary list from the third-grade book of the school's reading series. Mrs.
F ll
Fallows
decides
d id that
h mastery ffor Rick
Ri k means that,
h after
f hhearing
i a spelling
lli
word spoken and used in a sentence, Rick can write it correctly within 10
seconds.
• Because she doesn't want to overwhelm Rick with too many words, Mrs.
Fallows chooses to give him 10 new words at a time from the list (baseline)
and to ensure that they are mastered before giving more words.
words
• The teacher is unsure about how often to use the intervention. While she
could have Rick do CCC daily,
y, she wonders if twice weeklyy is sufficient. She
also estimates that he may need 1-3 weeks to learn each set of 10 words but
again isn't sure.
www.interventioncentral.org
28
Response to Intervention
Cover-Copy-Compare:
Cover
Copy Compare: Spelling
• DESCRIPTION: In this intervention to promote
acquisition of spelling words,
words the student is given a
spelling sheet with the target words correctly spelled.
The student looks at each correctly spelled word,
covers the word briefly and copies it from memory, then
compares the copied word to the original correct model
(Skinner, McLaughlin & Logan, 1997).
• GROUP SIZE: Whole class, small group, individual
student
• TIME: Variable up to 15 minutes per session
www.interventioncentral.org
29
Response to Intervention
Cover-Copy-Compare:
Cover
Copy Compare: Spelling
MATERIALS:
• Worksheet:
W k h t C
Cover-Copy-Compare
C
C
• Spelling Log: Mastered Words
www.interventioncentral.org
30
Response to Intervention
Cover-Copy-Compare:
Cover
Copy Compare: Spelling
INTERVENTION STEPS: Here are the steps of CoverCopy Compare for spelling:
Copy-Compare
1. [Teacher] Create a Cover-Copy-Compare Spelling
Sheet The teacher selects up to 10 spelling words for
Sheet.
the student to work on during the session and writes
those words as correct models into the left column
('Spelling Words') of the Worksheet: Cover-CopyCompare. The teacher then pre-folds the spelling
sheet using as a guide the vertical dashed line ('fold
line') bisecting the left side of the student worksheet.
www.interventioncentral.org
31
Response to Intervention
product
laughter
string
Cover-CopyCompare
p Spelling
p g
Student
Worksheet
summer
distract
neighbor
stable
geography
h
spool
strict
www.interventioncentral.org
product
Response to Intervention
Cover-Copy-Compare: Spelling
2.
•
•
•
•
•
[Student] Use the Cover-Copy-Compare Procedures. During the
Cover-Copy-Compare intervention, the student follows these selfdirected steps for each spelling word:
Look at the correctly spelled target word that appears in the left column
of the sheet.
Fold the left side of the page over at the pre-folded vertical crease to
hide the correct model ('Cover').
S ll the
Spell
th wordd from
f
memory, writing
iti it in
i the
th first
fi t response bl
blankk under
d
the 'Student Response' section of the spelling sheet ('Copy').
Uncover the correct model and compare it to the student response
('Compare')..
Continue until all words on the spelling list have been spelled and
checked against the correct models.
www.interventioncentral.org
33
Response to Intervention
Cover-Copy-Compare: Spelling
3. [Teacher] Log Spelling Words Mastered by Student. The
teacher should select an objective standard for judging that
the student using Cover-Copy-Compare has 'mastered' a
spelling word (e.g., when the student is able to copy a
specific word from memory without error on three
successive occasions). The teacher can then apply this
standard for mastery to identify and log spelling words in
each session, using the Spelling Log: Mastered Words
sheet.
www.interventioncentral.org
34
Response to Intervention
Spelling Log:
M t dW
Mastered
Words
d
Sheet
www.interventioncentral.org
Response to Intervention
When ACQUISITION is the Target: How to Set Individual
Student Academic Goals: EXAMPLE: ANGELA
At your table:
• Review the case of Angela
((next slide).)
• Complete the CASE 2 goal
planning form for this student
(packet 3, p. 4).
• Be prepared to report out.
out
www.interventioncentral.org
36
Response to Intervention
When ACQUISITION is the Target:
g How to Set
Individual Student Academic Goals: EXAMPLE: Angela
• A social studies teacher,, Mr. Franklin,, selects Q
Question Generation (Q
(QG)) as
an intervention for his student, Angela, who has difficulty finding or
generating main idea sentences for paragraphs. The instructor decides that,
when given a randomly chosen paragraph from course readings
readings, Angela
should be able to either locate the main idea sentence or write her own 'gist'
sentence with at least two key supporting details within 5 minutes.
• The teacher is not sure how to collect baseline data but would welcome your
suggestions.
• Mr.
Mr Franklin plans to assign Angela short passages (3-4
(3 4 paragraphs) at least
3 times per week on which to use the strategy and will collect her note cards.
He also plans to assess the student's use of QG once per week on a sample
passage, to be completed in class. Mr. Franklin hopes that, in six weeks, the
student will have made noticeable improvements to her comprehension.
www.interventioncentral.org
37
Response to Intervention
Question
Generation
“Identifying or Constructing
Main Idea Sentences”
Students are taught to boost
their comprehension of
expository passages by (1)
locatingg the main idea or keyy
ideas in the passage and (2)
generating questions based on
that information.
f
http://www.interventioncentral.org/academicinte entions/ eading comp ehension/q estion gene ation
interventions/reading-comprehension/question-generation
www.interventioncentral.org
Response to Intervention
Estimating
E
ti ti 'Typical
'T i l P
Peer A
Academic
d i
Performance': A Review of Methods packet 1
pp. 2-6
www.interventioncentral.org
39
Response to Intervention
Estimating 'Typical Peer Academic Performance': A Review of Methods
Type of Peer Comparison: Description
Strengths/Weaknesses
Research Norms Based on Fall/Winter/Spring
Screenings. The ideal source for performance
i f
information
ti in
i any academic
d i area is
i a sett off highhi h
quality research norms that:
• are predictive of student success in the targeted
( )
academic area(s)
• are drawn from a large, representative student
sample
• include fall, winter, and spring norms
• provide an estimate of student risk for academic
failure (e.g., that are divided into percentile tables
or include score cut-offs denominating low
risk/some risk/at risk).
Examples of publicly available academic research
norms can be found on these websites:
EasyCBM.com: http://www.easycbm.com
DIBELS NEXT: http://dibels
http://dibels.org/next.html
org/next html
Appropriate Use(s) for This Performance Data
Source: Research norms based on
f ll/ i t / i screening
fall/winter/spring
i data
d t can be
b usedd for
f
the full range of instructional decision-making,
including setting student performance outcome
ggoals for core instruction and/or anyy level of RTI
intervention in general education and for setting
performance goals on IEPs.
These norms can also be useful at Special
Education Eligibility Team meetings to verify
whether a student has moved into a lower level of
academic risk as a result of RTI interventions.
Limitations of This Performance Data Source:
There are no significant limitations in using these
research norms.
norms
www.interventioncentral.org
40
Response to Intervention
Sample Rationale for…
Research Norms Based on Fall/Winter/Spring Screenings
“
"We chose to use these norms because they
provide
id th
the highest-quality
hi h t
lit iinformation
f
ti available
il bl
about student academic performance. They are
accurate
t predictors
di t off student
t d t success, hhave
been created using a representative student
sample,
l are bbroken
k outt iinto
t ffall/winter/spring
ll/ i t / i
norms, and do a good job of estimating the
l l off academic
level
d i risk
i k ffacedd bby any iindividual
di id l
student."
www.interventioncentral.org
”
41
Response to Intervention
Example: Research Norms Based on Fall/Winter/Spring
Screenings packet 2,
2 pp. 1
www.interventioncentral.org
42
Response to Intervention
Estimating 'Typical Peer Academic Performance': A Review of Methods
Type of Peer Comparison
Strengths/Weaknesses
Research Norms Based on a Single
Academic Performance Sample. Norms
generated from research studies or other
sources may provide estimates of student
academic performance based on a
sampling
li from
f
a single
i l point
i t in
i titime, rather
th
than a more comprehensive sampling
across separate fall, winter, and spring
screenings.
i
Appropriate Use(s) for This Performance Data Source:
Norms drawn from a single 'snapshot' student sample can be
useful in general education for setting student performance
outcome goals for core instruction and/or any level of RTI
intervention. Similarly, these norms can be used to set student
performance
f
goals
l on IEP
IEPs. In
I bboth
th cases, hhowever, singlei l
sample norms would be used only if more comprehensive
fall/winter/spring screening norms are not available.
Limitations of This Performance Data Source: SingleThese norms may also have been compiled sample academic norms can be used for RTI decision-making
from a relatively small student sample that if there is no better normative information available. However,
is not demographically representative of a
Special Education Eligibility Teams should be cautious in
diverse 'national' population. Nonetheless, interpreting these norms, as they do not reflect typical student
these norms are often the best information growth across fall, winter, and spring screenings; and are
that is publically available for skills such as likely to be based on a small student sample that may not be
mathematics computation—and so have a demographically representative of the school's or district's
definite place in RTI decision-making.
students.
www.interventioncentral.org
43
Response to Intervention
Sample Rationale for…
Research Norms Based on a Single Academic Performance
Sample.
“
"We
We used these single-sample
single sample norms to set
student performance goals because they are
drawn from research and were the best
information available to us. Because the norms
were collected at one point in time,
time however,
however
and were drawn from a small student sample,
we realize that these norms should be
interpreted cautiously—especially when used
for decisions about special education eligibility.
eligibility "
www.interventioncentral.org
”
44
Response to Intervention
Example: Research Norms Based on Single Academic
Performance Sample packet 1.
1 p.
p 18
www.interventioncentral.org
45
Type of Peer Comparison
Response
to Intervention
Strengths/Weaknesses
Local Norms: Multiple Classrooms. Local
norms across classrooms are generated
when a district or school administers an
academic screener in multiple classrooms
at a grade level and compiles the data into
norms that display the range of student
abilities in that local setting.
Appropriate Use(s) for This Performance Data Source:
Schools can use data from local norms across classrooms to
identify which students are struggling relative to their
classmates, to quantify the academic gap that separates
these struggling students from the majority of their local peers,
peers
and to match them to appropriate interventions to close that
gap. Local academic norms can also be useful for special
educators to estimate the minimum skill levels that a student
For example, a district may administer a
with an IEP will need to survive instructionally when
brief CBM in writing to the entire third grade
mainstreamed in a particular general-education setting.
in its three elementary schools to produce
l l norms bby di
local
district
t i t andd building.
b ildi
Li it ti
Limitations
off This
Thi P
Performance
f
Data
D t Source:
S
L l norms
Local
cannot provide an external, objective standard for minimum
Local norms can help to identify which
competency in the academic skill because they vary,
students in a school stand out from local
depending on the demographic and other characteristics of
peers because of academic skill deficits and
the school or district being normed. Local norms would not
require additional intervention support.
give the Special Education Eligibility Team an absolute,
However, unlike research norms, local
research derived cut-off
research-derived
cut off for academic competence
norms do not provide an absolute standard
independent of the district's average student skill level
of student academic competence.
necessary to certify that the student is an RTI 'non-responder'.
www.interventioncentral.org
46
Response to Intervention
Sample Rationale for…
Local Norms: Multiple Classrooms
“
“Our school used these local norms from multiple
classrooms because they helped us to identify
which students fell farthest from the local
academic average and thus are likely to need
additional intervention support to be successful in
their classrooms
classrooms. We also understand that local
norms give us information limited to the student's
performance in this school
school. Only research norms
can provide an objective, research-based view of
the student's
student s skills compared to a national
average."
www.interventioncentral.org
”
47
Response to Intervention
Defining ‘Discrepant’ Academic Performance: Do
We Use External Norms or Local Norms?
External (Research or Benchmark) Norms: Used to
compare the performance of a student or instructional
program to objective external/research/national norms.
External norms can helpp to answer these qquestions:
– Is the school’s core program successful (comparison of
local to research norms)?
)
– Is a child performing at a minimum level of competency
in the academic skill to allow us to predict future
success?
– What objective academic performance cut-off should be
set to determine student entry into and exit from Tier 2
and 3 intervention programs?
www.interventioncentral.org
48
Response to Intervention
Defining ‘Discrepant’ Academic Performance: Do
W Use
We
U External
E t
lN
Norms or Local
L l Norms?
N
?
LLocall Norms:
N
Rank-ordered
R k d d compilation
il ti off scores off students
t d t
within a particular grade level/school. Local norms are
used to help answer these questions:
– What is the typical range of student ability in the grade
level or school?
– How is a particular student performing relative to other
children in the grade level or school?
– How much effort must a teacher exert to instruct this
student relative to other students in the class?
www.interventioncentral.org
49
Response to Intervention
Baylor Elementary School : Grade Norms: Correctly Read Words Per Min : Sample Size: 23 Students
Group Norms: Correctly Read Words Per Min: Book 4-1: Raw Data
31 34 34 39 41 43 52 55 59 61 68 71 74 75 85 89 102 108 112
115 118 118 131
LOCAL NORMS EXAMPLE:
EXAMPLE Twenty-three
T
t th
4th-grade
d students
t d t
were administered oral reading fluency Curriculum-Based
passages
g at the 4th-grade
g
level in their school.
Measurement p
In their current number form, these data are not easy to
interpret.
So the school converts them into a visual display—a boxplot —to show the distribution of scores and to convert the
scores to
t percentile
til fform.
When Billy, a struggling reader, is screened in CBM reading
fluency he shows a SIGNIFICANT skill gap when compared to
fluency,
his grade peers.
www.interventioncentral.org
50
Response to Intervention
Baylor Elementary School : Grade Norms: Correctly Read Words Per Min : Sample Size: 23 Students
January Benchmarking
Group Norms: Correctly Read Words Per Min: Book 4-1: Raw Data
31 34 34 39 41 43 52 55 59 61 68 71 74 75 85 89 102 108 112
115 118 118 131
National Reading
Norms: 112 CRW
Per Min
Group Norms:
G
N
C
Converted
t d
to Box-Plot
Median (2nd Quartile)=71
1st Quartile=43
3rd Quartile=108
Billy=19
Hi Value=131
Low Value=31
0
20
40
Source: Tindal, G.,
Hasbrouck,, J.,, &
Jones, C. (2005).Oral
reading fluency: 90
years of
measurement
[Technical report
#33]. Eugene, OR:
University of Oregon.
60
80
100
120
140
160
Correctly Read Words-Book 4-1
www.interventioncentral.org
51
Type of Peer Comparison
Response to Intervention
Strengths/Weaknesses
Local Norms: Single
Classroom Teachers can
Classroom.
develop informal academicperformance norms by screening
all students in their classroom.
classroom
Students are administered a
standardized screening measure
(e g timed Maze reading
(e.g.,
comprehension screening lasting
for 3 minutes). The teacher next
scores the screener and rank
rankorders the student results. The
teacher can then set a cutpoint
((e.g.,
g , the lowest 20 ppercent of
scores on the Maze task) to
select students to receive
additional (Tier 1) coreinstruction and/or intervention
support.
Appropriate Use(s) for This Performance Data Source: Local norms
compiled from a single general-education
general education classroom can be very efficient
in identifying general-education students who would benefit from extra
teacher support in core instruction or may even need specific teacherdelivered interventions.
interventions Because local classroom norms estimate the
range of current skill levels in a room, they can also be valuable in aiding
educators to better plan to support students with special needs included
in those settings.
settings
Limitations of This Performance Data Source: Local classroom norms
may not be representative of average skill levels in other classrooms—
even in
i th
the same school-h l so interpretation
i t
t ti off suchh norms should
h ld bbe
limited to the classrooms from which they were derived.
Also, self-contained special education settings might find that local
norms compiled in their classrooms are not particularly useful. This is
because students in such a program are likely to have a range of special
education classifications and a correspondingly wide range of academic
skills. With such widely discrepant academic skills among students,
classroom norms may not yield a meaningful group-level estimate of
'average'
performance.
www.interventioncentral.org
52
Response to Intervention
Sample Rationale for…
Local Norms: Single Classroom
“
"We screened students in our single classroom
using measures of basic academic skills.
skills These
local norms have helped us to be proactive in
finding students in the room who need additional
core instruction or intervention support. However,
we realize that norms from one classroom can be
meaningfully applied only to that classroom. To
come up with a shared standard of average local
student performance across a whole grade level,
our school will need to screen multiple classrooms
and combine the results."
www.interventioncentral.org
”
53
Type of Peer Comparison
Response to Intervention
Strengths/Weaknesses
Local Norms: Small Group.
The most informal (and lowlow
cost) means for developing
local norms is for the teacher to
select a small number (e.g.,
(e g 33
5) of students who—in the
teacher's estimation--possess
average abilities on the
academic skill to be normed.
The teacher administers this
small group a standardized
screening measure (e.g., 3minute Maze passage. The
p
teacher rank-orders the ggroup's
screening results and selects
the median student score to
provide a rough estimate of a
'typical' level of peer academic
performance.
Appropriate Use(s) for This Performance Data Source: Local smallgroup norms provide an informal but useful estimate of typical classroom
academic performance. The teacher at Tier 1 can use these small-group
norms to determine how severe a struggling student's academic delays
are That is
are.
is, the teacher can administer the same screening measure used
to compile the small-group norms to a particular student experiencing
academic delays. The teacher then compares the target student's
screening result to the informal small
small-group
group peer norm to calculate that
struggling student's current skill gap. Small-group academic norms may
also be a useful tool for special educators, providing a low-cost means for
estimating the skill levels that a student with an IEP will need (e.g., in oral
reading fluency) to be mainstreamed in a particular general-education
setting.
g p local norms
Limitations of This Performance Data Source: Small-group
provide at best only a rough estimate of classroom academic skill levels.
They should be used for Tier 1 (classroom) core-instruction and
intervention planning only when information of higher quality (e.g.,
research norms, grade-wide local norms, class-wide local norms) are not
available. These informal norms would also not be appropriate for higherstakes, more
intensive interventions at Tiers 2 and 3.
www.interventioncentral.org
54
Response to Intervention
Sample Rationale for…
Local Norms: Small Group
“
"We compiled local norms with a small group of
students in our classroom because there were no
better norms available and we did not have the
resources or time to screen an entire class
class. We
used these small-group norms to help us to
identify and set intervention goals for students
who needed extra classroom academic support.
We also recognize that use of these informal
group norms should be restricted to generaleducation Tier 1 problem
problem-solving
solving."
www.interventioncentral.org
”
55
Type of Peer Comparison
Response to Intervention
Criterion-Referenced Performance Goal. Proficiency-based performance goals can be described as
'criterion referenced' when they link to important academic skills and have clear definitions of 'mastery'
'criterion-referenced'
but are not backed by research-based or local norms. This proficiency level may be created by the
teacher; may reflect a school-, district-, or state-defined standard; or may be derived from other sources
of expert opinion.
opinion
Criterion-referenced performance goals are often linked to the assessment of discrete academic subskills that may be mastered in a few days or weeks. For example, a teacher may decide that, in his
classroom, students must be able to correctly answer at least 20 math fact problems (single-digit times
single-digit) within 3 minutes to be proficient (teacher-developed standard).
Criterion-referenced goals may also be used to gauge student progress over longer periods, such as a
full school year. For example, a district may include a curriculum expectation that, by the end of grade 1,
students will know all elements of a preselected set of sight words taken from the Dolch Word List
(district standard).
Sometimes criterion-reference goals include cutscores that indicate when a student has attained
mastery. For example, a science instructor may assess students' knowledge of 50 key biology terms and
provide additional instructional and intervention support to students who know fewer than 90 percent of
those terms (teacher-developed standard; 90% cutscore).
www.interventioncentral.org
56
Strengths/Weaknesses
Response to Intervention
Criterion-Referenced Performance Goal: Appropriate Use(s) for This Performance Data
S
Source:
C it i
Criterion-referenced
f
d goals
l are applicable
li bl tto allll grade
d llevels,
l can bbe applied
li d tto virtually
i t ll
all academic content areas, and can be employed in both general- and special-education
settings. They are most useful when developing performance expectations either for short-term
academic sub-skills that the student may attain in a matter of weeks (e.g., recognition of all
mixed-case letters) or for academic-skill targets for which no local or research norms are
available.
Limitations of This Performance Data Source: A significant limitation of criterion-referenced
goals is that they are based largely on the 'expert opinion' of teachers, curriculum writers, and
other
th sources—rather
th th
than being
b i dderived
i d empirically
i i ll ffrom research-based
h b d or local
l l norms. IIn
any individual case in which criterion-referenced goals are used, it is likely to be unclear how
the target student is performing relative to other students in his or her local or national peer
group or even whether that student's falling short of these goals is actually predictive of
academic failure. It is recommended, then, that criterion-referenced goals be used only when
g q
y research-based or local norms cannot be obtained.
higher-quality
www.interventioncentral.org
57
Response to Intervention
Sample Rationale for…Criterion-Referenced Performance Goal
“
"We developed
"W
d l d our own criterion-referenced
it i
f
d
performance goals for this student because there
were no research-based
h b d or llocall norms available
il bl
for the academic skill that we were measuring. We
b li
believe
th
thatt th
the standards
t d d ffor mastery
t th
thatt we
developed for the student are both ambitious and
realistic.
li ti However,
H
we also
l recognize
i that
th t we lack
l k
information both about the degree to which this
student's
t d t' peers have
h
mastered
t d th
these criterionit i
referenced goals and about how strongly these
goals
l might
i ht correlate
l t with
ith other
th measures off llongterm academic success."
www.interventioncentral.org
”
58
Response to Intervention
Types of Peer Comparisons: Example
A 5th-grade teacher, Mrs. Stevens, collects a Curriculum-Based
Measurement writing probe from her class and scores it for correctly spelled
words. She finds that the class average is 67 correctly spelled words, while
her target student, Ray, writes only 16 correctly spelled words.
Ray has been referred to the Special Education Eligibility Team for a
possible learning disability in writing. Mrs. Stevens wants to present her
writing data to the Team but is nervous about what questions they may ask.
ask
What type of norms did Mrs. Stevens collect?
Write a brief explanation about the strengths and limitations of these norms
p
that the teacher could ppresent when she attends the Special
Education Eligibility Team meeting.
www.interventioncentral.org
59
Response to Intervention
Types of Peer Comparisons: Example
A 3rd-grade
grade teacher
teacher, Mr
Mr. Brady
Brady, has a student,
student Veronica,
Veronica who has significant
reading delays and lacks fluent recognition of sight words. Mr. Brady meets
with the RTI Team to develop a reading intervention. They decide as one
focus of the intervention to work on mastery of a list of common sight words.
To helpp him to set an intervention ggoal for the student,, Mr. Bradyy meets with
a teacher from 1st Grade and asks her to estimate who long it would take a
typical student from her classroom to master the sight word list. The Grade
1 teacher says that most students would master the list in about six
weeks—so Mr. Brady uses mastery of the list within 6 weeks as his goal for
Veronica. However, Mr. Brady knows that his principal really likes teachers
to use only the AIMSweb norms to set student outcome goals.
yp of ggoal did Mr. Bradyy set for Veronica?
What type
Write a brief explanation laying out the strengths and limitations
of this goal that Mr. Brady could share with his principal.
www.interventioncentral.org
60
Response to Intervention
RTI Data Challenge:
S tti IIndividual
Setting
di id l RTI
Academic Goals Using
Research Norms for
Students Receivingg ‘OffLevel’ Interventions
Source: Shapiro, E. S. (2008). Best practices in setting progressmonitoring monitoring goals for academic skill improvement. In A.
Thomas & J. Grimes ((Eds.),
) Best practices
p
in school psychology
p y
gy V
(pp. 141-157). Bethesda, MD: National Association of School
Psychologists.
www.interventioncentral.org
Response to Intervention
AIMSweb Cut-Points: Using National
A
Aggregate
t Sample
S
l
• Low Risk: At or above the 25th ppercentile: Core
instruction alone is sufficient for the student.
• Some Risk: 10th to 24th ppercentile: Student will benefit
from additional intervention, which may be provided by
the classroom teacher or other provider (e.g., reading
t h )
teacher).
• At Risk: Below 10th percentile : Student requires
i t i intervention,
intensive
i t
ti which
hi h may bbe provided
id d bby the
th
classroom teacher or other provider (e.g., reading
teacher)
teacher).
www.interventioncentral.org
62
Response to Intervention
63
Source: AIMSweb® Growth Table Reading-Curriculum Based Measurement: Multi-Year Aggregate: 2006-2007 School Year
www.interventioncentral.org
Response to Intervention
How to Set a Goal for an ‘Off-Level’
Off Level Intervention
1. Comparing Student Performance to Benchmarks and
Identifying Severe Discrepancies.
Discrepancies The student is
administered reading fluency probes equivalent to his or her
current grade placement (during the Fall/Winter/Spring
schoolwide screening) and the results are compared to peer
norms. If the student falls significantly below the level of
peers, he or she may need additional assessment to
determine whether the student is to receive intervention and
assessmentt ‘off
‘ ff grade
d llevel’.
l’
www.interventioncentral.org
64
Response to Intervention
Example of Progress-Monitoring Off-Level: Randy
In January, Mrs. Chandler, a 4th-grade teacher,
receives her classwide reading fluency screening
results. She notes that a student who has recently
transferred to her classroom
classroom, Randy
Randy, performed at 35
Words Read Correct (WRC) on the 1-minute
AIMSweb Grade 4 fluency probes.
probes
Mrs. Chandler consults AIMSweb reading-fluency
research norms and finds that a reasonable minimum
reading rate for students by winter of grade 4 (25th
percentile)
til ) iis 89 WRC.
WRC
www.interventioncentral.org
65
Response to Intervention
Example of Progress-Monitoring
Off Level: Randy
Off-Level:
AIMSweb Norms:
‘Typical’ reader (25th
percentile) in Gr 4 at
mid-year (winter
norms): 89 WRC
Target Student Randy:
35 WRC
Conclusion: Randy’s
grade-level
d l l
performance is in the
g
‘frustration’ range.
He requires a SurveyLevel Assessment to
find his optimal
Source: AIMSweb® Growth Table Reading-Curriculum Based Measurement: Multi-Year Aggregate:
2006-2007 School
Year
‘instructional’
level.
www.interventioncentral.org
66
Response to Intervention
How to Set a Goal for an ‘Off-Level’ Intervention
2. Conducting a Survey Level Assessment (SLA). For
students with large
g discrepancies
p
when compared
p
to
benchmarks, the teacher conducts a SLA to determine the
student’s optimal level for supplemental intervention and
progress-monitoring.
it i
•
•
The teacher administers AIMSweb reading probes from
successively earlier grade levels and compares the student’s
performance to the benchmark norms for that grade level.
level
The student’s ‘instructional’ level for intervention is the first
grade level in which his reading-fluency
reading fluency rate falls at or above
the 25th percentile according to the benchmark norms.
www.interventioncentral.org
67
Response to Intervention
Example of Progress-Monitoring Off-Level: Randy
Because Randy’s reading fluency rate is so far below
the grade-level norms (a gap of 54 WRC), his teacher
decides to conduct a Survey Level Assessment to find
the student
student’ss optimal grade level placement for
supplemental reading instruction.
www.interventioncentral.org
68
Response to Intervention
Example of Progress-Monitoring Off-Level: Randy
69
On
GradeLevel
3-level
2-level
Survey
Assessment.
The a
probes,
Randy attains
median
teacherscore
conducts
of 64
48a
WRC.
Survey Level
Assessment with
The
Randy,
AIMSweb
assessing
winter
him
th percentile)
norm
using(25
CBM
reading
rd grade
for
fluency
a 23nd
grade
probesstudent
student
from
issuccessively
69 WRC. earlier
53
grades until he
The
performs
student
at isorstill
now
above
inin
the
the‘instructional’
‘frustration’
25th percentile
range
and
range
according
theand
Survey
the
to the
Survey
Level
Assessment
Level
AIMSweb
Assessment
norms.
continues.
ends.
Source: AIMSweb® Growth Table Reading-Curriculum Based Measurement: Multi-Year Aggregate: 2006-2007 School Year
www.interventioncentral.org
Response to Intervention
How to Set a Goal for an ‘Off-Level’ Intervention
3. Selecting an ‘Off-Level’ Progress-Monitoring Goal.
To set a pprogress-monitoring
g
g ggoal, the teacher looks
up the benchmark WRC for the 50th percentile at the
student’s off-level ‘instructional’ grade level previously
d t i d through
determined
th
h the
th Survey
S
LLevell A
Assessment.t
www.interventioncentral.org
70
Response to Intervention
Example of Progress-Monitoring Off-Level: Randy
71
Goal-Setting.
To find
the progressmonitoring
it i goall ffor
Randy, his teacher
looks upp the
benchmark WRC for
the 50th percentile at
Grade 2 (his off
off-level
level
‘instructional’ grade
level)—which is 79
WRC.
WRC
This becomes the
progress-monitoring
goal for
the School
student.
Source: AIMSweb® Growth Table Reading-Curriculum Based Measurement: Multi-Year Aggregate:
2006-2007
Year
www.interventioncentral.org
Response to Intervention
How to Set a Goal for an ‘Off-Level’ Intervention
4. Translating the Progress-Monitoring Goal into
Weeklyy Increments.
The teacher’s final task before starting the progressmonitoring is to translate the student’s ultimate
intervention goal into ‘ambitious but realistic’ weekly
i
increments.
t
One useful
O
f l method
th d for
f determining
d t i i weekly
kl growth
th rates
t
is to start with research-derived growth norms and to
then use a ‘multiplier’
multiplier to make the expected rate of
weekly growth more ambitious.
www.interventioncentral.org
72
Response to Intervention
How to Set a Goal for an ‘Off-Level’ Intervention
4 T
4.
Translating
l ti th
the Progress-Monitoring
P
M it i G
Goall iinto
t W
Weekly
kl
Increments. (Cont.)
• The teacher first looks up the average rate of weekly student
growth supplied in the research norms. (NOTE: If available, a
good rule of thumb is to use the growth norms for the 50th
percentile at the ‘off-level’ grade at which the student is
receiving intervention and being monitored.)
• The teacher then multiplies this grade norm for weekly growth
by a figure between 1.5 and 2.0 (Shapiro, 2008). Because the
original weekly growth rate represents a typical rate student
improvement, using this multiplier to increase the target
student’s weekly growth estimate is intended accelerate
learning and close the gap separating that student from peers.
www.interventioncentral.org
73
Response to Intervention
Example of Progress-Monitoring Off-Level: Randy
74
Randy s ultimate goal is 79 WRC (the
Randy’s
50th percentile norm for grade 2).
Determining
g Weeklyy Rate of Improvement
p
D i the
During
th Survey
S
Level
L lA
Assessment,t
(ROI). Randy is to be monitored on
Randy was found to read 64 WRC at the
intervention
at grade 2. The teacher finds—
nd
2 grade level.
according to AIMSweb norms—that a typical
student in Grade 2 (at the 50th percentile) has
There is a 15-WRC gap to be closed to
a rate of improvement of 1.1 WRC per week.
get Randy to his goal.
goal
She multiplies the 1.1 WRC figure by 1.8
At 2 additional WRC per week on
((teacher judgment)
j g
) to obtain a weeklyy ggrowth
intervention Randy should close the gap
intervention,
goal for Randy of about 2.0 additional WRCs.
within about 8 instructional weeks.
Source: AIMSweb® Growth Table Reading-Curriculum Based Measurement: Multi-Year Aggregate: 2006-2007 School Year
www.interventioncentral.org
Response to Intervention
How to Set a Goal for an ‘Off-Level’ Intervention
5 Advancing the Student to Higher Grade Levels for
5.
Intervention and Progress-Monitoring
The teacher monitors the student’s
student s growth in reading fluency at
least once per week (twice per week is ideal).
When the student’s
student s reading fluency exceeds the 50th percentile in
Words Read Correct for his or her ‘off-level’ grade, the teacher
reassesses the student’s reading fluency using AIMSweb
materials at the next higher grade.
If the student performs at or above the 25th percentile on probes
from that next grade level, the teacher advances the student
and begins to monitor at the higher grade level.
The process repeats until the student eventually closes the gap
with peers and is being monitored at grade of placement.
www.interventioncentral.org
75
Response to Intervention
Example of Progress-Monitoring Off-Level: Randy
76
Advancing the Student to
Hi h Grade
Higher
G d Levels
L l (Cont.).
(C t )
So Mrs. Chandler assesses
y on gAIMSweb
reading
Randy
Advancing
the Student
tog
fluency
probes Levels
for Grade
Higher Grade
of 3 and
finds
that he reads on average
72
Progress-Monitoring.
His
WRC
—exceeding
exceeding
the Grade
teacher
teacher,
Ms
Ms. Chandler,
Chandler
notes 3
25
cut-off
thatth percentile
after 7 weeks
of of 69 WRC.
intervention, Randy is now
Th di f 82 Randy
Therefore,
RWRC—exceeding
d is
i advanced
d di d tto
reading
WRC
Grade
progress-monitoring
and
the 79 3WRC
for the 50th
intervention
materials
are
percentile
phis
of students
in Grade
adjusted
2 (winter accordingly.
norms).
Source: AIMSweb® Growth Table Reading-Curriculum Based Measurement: Multi-Year Aggregate: 2006-2007 School Year
www.interventioncentral.org
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example Packet 1 pp. 11-13
• Comparing Student Performance to Benchmarks and
Identifying Severe Discrepancies.
Discrepancies A school adopted universal
RTI screening/progress-monitoring tools from EasyCBM
(http://www.easycbm.com). When the school-wide screening was
conducted
d t d iin th
the ffall,
ll th
the school
h l di
discoveredd th
thatt a 5th-grade
d
student, Alicia, was substantially delayed on the Passage
Reading Fluency (PRF) task. Alicia read only 45 words per
minute on the passage.
• Q: Consult the Grade 5 Fall screening table (packet 2 p. 4) and
find the Passage Reading Fluency norm for the 20th percentile.
percentile
Then subtract Alicia's reading score from that normative value.
How much of a gap separates Alicia from her peers in reading
fluency?
www.interventioncentral.org
77
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example
• Q: Consult the Grade 5 Fall screening table and find the Passage
Reading Fluency norm for the 20th percentile.
percentile Then subtract Alicia's
Alicia s
reading score from that normative value. How much of a gap separates
Alicia from her peers in reading fluency?
• A: The peer norms in the fall screening in Grade 5 at the 20th
percentile are 111 Words Read Correct (WRC). Alice read 45 WRC.
So the g
gap
p between Alice and her g
grade peers
p
is 66 WRC.
• Q: Do you judge that this student needs a
survey-level assessment to determine the
optimal
ti l ''off-level'
ff l l' grade
d ffor matching
t hi Ali
Alicia
i
to appropriate progress-monitoring and
intervention materials? If so, why?
www.interventioncentral.org
Response to Intervention
‘Off-Level’ Intervention Goal-Settingg Example
p
• Conducting a Survey-Level Assessment. Alicia's performance on the fall
Passage Reading Fluency screening triggered a Survey
Survey-Level
Level Assessment.
Assessment
Table 1 below shows how Alicia performed on successively easier PRF
passages drawn from earlier grades below her current grade placement:
Table 1: Survey-Level Assessment for Student Alicia (Fall Norms) on Passage Reading Fluency
Task
Grade Level
Words Read Correct/1
Minute
Grade 4 PRF Passages
51 WRC
Appropriate Off-Level Grade
for Supplemental
Intervention?
___ Y ___ N
Grade 3 PRF Passages
63 WRC
___ Y ___ N
Grade 2 PRF Passages
68 WRC
___ Y ___ N
www.interventioncentral.org
79
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example
• T
Task:
k Compare
C
th
the results
lt off Ali
Alicia's
i ' S
Survey-Level
L lA
Assessmentt tto the
th ffallll
easyCBM screening norms. For each grade level at which the assessment
was conducted, decide whether the student score falls between the 20th
and 50th percentiles ('instructional' range). If so, mark the 'Y' blank under
the 'Appropriate Off-Level Grade for Supplemental Intervention' column;
otherwise mark the 'N'
otherwise,
N blank.
blank
Table 1: Survey-Level Assessment for Student Alicia (Fall Norms) on Passage Reading Fluency
Task
Grade Level
Words Read Correct/1
Minute
Grade 4 PRF Passages
51 WRC
Appropriate Off-Level Grade
for Supplemental
Intervention?
___ Y _X_ N
Grade 3 PRF Passages
63 WRC
_X_ Y __ N
Grade 2 PRF Passages
68 WRC
_X_ Y ___ N
www.interventioncentral.org
80
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example
• Selecting an ‘Off-Level’ Progress-Monitoring Goal. Once the
school had determined the student
student'ss optimal 'off-level'
off level grade for
intervention and progress monitoring, it next needed to
determine the student's progress-monitoring goal for that grade
l l
level.
• Task: For the grade level at which you have chosen for Alicia's
'off-level' intervention and monitoring,
g, look upp the fall 50th
percentile cutscore from the easyCBM norms. Subtract Alicia's
current PRF score at this grade level (taken from the Survey
Level Assessment) from the 50th percentile cutscore to determine
the amount of progress the student will need to make to
'graduate' to the next grade level.
www.interventioncentral.org
81
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example
• T
Task:
k For
F the
th grade
d level
l l att which
hi h you have
h
chosen
h
f Ali
for
Alicia's
i '
'off-level' intervention and monitoring, look up the fall 50th
ppercentile cutscore from the easyCBM
y
norms. Subtract Alicia's
current PRF score on at this grade level (taken from the Survey
Level Assessment) from the 50th percentile cutscore to determine
the amount of progress the student will need to make to
'graduate' to the next grade level.
• A: Alice read 63 WRC on the
Grade 3 grade-level passages,
while the 50th percentile cutscore
for Grade 3 on the fall screening
is 83 WRC. So Alice will need to
add 20 WRC before she is ready to
‘graduate’ to Grade 4 reading
passages.
www.interventioncentral.org
82
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example
• Translating the Progress-Monitoring Goal into Weekly
Increments Now that the school had determined the overall
Increments.
gap that Alicia needed to close to advance to the next grade
level on Passage Reading Fluency, the school needed to break
th t overallll goall iinto
that
t weekly
kl iincrements.
t
• First, the school had to find an expected rate of weekly growth on
the PRF usingg the easyCBM
y
norms. Unfortunately,
y, the norms do
not list weekly rates of growth. Therefore, the school calculated
this figure on its own (described below) using information taken
from the norms chart.
chart
• The school then converted this 'average' rate of progress to a
more ambitious one by multiplying the original weekly increment
by a figure between 1.5 and 2.0 (Shapiro, 2008).
www.interventioncentral.org
83
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example
• Because Alicia's intervention to improve her reading fluency was
scheduled to last eight instructional weeks
weeks, the school then
multiplied the ambitious rate of weekly progress by the number of
intervention weeks and added that product to Alicia's baseline to
predict
di t the
th student's
t d t' progress if th
the intervention
i t
ti were successful.
f l
www.interventioncentral.org
84
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example
• Task: Compute the 'typical'
typical rate of weekly progress at the student's
student s
'off-level' grade by (1) subtracting the fall screening results (50th
percentile) for the off-level grade from the spring screening results
(50th percentile)
til ) andd (2) di
dividing
idi th
the diff
difference bby 32
32--representing
ti
the typical 32 weeks that separate fall and spring screenings in most
schools. The resultingg qquotient represents
p
'average'
g expected
p
rate of
student progress per instructional week.
A: The spring screening norm for
the 50th percentile is 115.
115 When the
fall 50th percentile screening norm
of 83 is subtracted from the spring
screening
i norm off 115,
115 the
th difference
diff
is 32 WRC. When that difference of 32
WRC is divided byy 32 weeks,, the
typical rate of student reading-fluency
progress is 1 additional WRC.
www.interventioncentral.org
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example
• Task: Transform the above 'typical' rate of weekly progress to an
ambitious rate by multiplying the original weekly increment by a
figure between 1.5 and 2.0.
• A: If the 1.0 WRC rate of ‘typical’ Grade 3 student progress
in reading fluency is multiplied by 2.0, the new ‘ambitious’
rate of progress is 2.0
2 0 additional WRCs per week.
week
NOTE: Your expected rate of student progress goal will be
l
lower
if you selected
l t d a less
l
ambitious
biti
multiplier
lti li than
th 2.0.
20
www.interventioncentral.org
86
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example
• Task: Multiply the ambitious rate of weekly progress that you
calculated by the eight weeks of Alicia's
Alicia s intervention
intervention. Then add
that product to the baseline that you had previously assessed for
this off-level grade during the Survey Level Assessment to
produce
d
an'' iintermediate'
t
di t ' off-level
ff l l iintervention
t
ti performance
f
goal.l
• A: When the 2.0
2 0 WRC calculated as Alice
Alice’ss ambitious
expected rate of weekly progress is multiplied by the 8
weeks of the intervention, she should be adding 16 WRCs to
h reading
her
di per minute
i t by
b the
th endd off a successful
f l
intervention. When the 16 WRCs is added to Alice’s baseline
in Grade 3 of 63 WRC, it is expected that—if the intervention
works—Alice will be reading 79 WRCs.
www.interventioncentral.org
87
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example
• Advancing the Student to Higher Grade Levels for
Intervention and Progress
Progress-Monitoring
Monitoring. Once Alicia
Alicia'ss
intervention had been in place for 8 weeks, the school
discovered that she was now reading 84 Words Read Correct on
th Passage
the
P
R
Reading
di Fl
Fluency assessmentt att hher ''off-level'
ff l l'
grade.
• Because it was still mid-autumn, the school continued to consult
the fall easyCBM norms to see if the student had exceeded the
50th percentile
til ((graduation
d ti goal).
l) If so, the
th student
t d t could
ld bbe
considered for advancing to the next grade-level for intervention
and progress-monitoring. If not, she would continue to be
monitored at her current level.
www.interventioncentral.org
88
Response to Intervention
‘Off-Level’ Intervention Goal-Setting Example
• Task: Look up the easyCBM norms for the fall screening in
Passage Reading Fluency for Alicia
Alicia'ss 'off-level'
off level grade.
grade Decide
whether her performance of 84 Words Read Correct exceeds the
50th percentile and therefore whether she would be considered a
candidate
did t tto advance
d
tto th
the nextt grade
d llevell ffor iintervention
t
ti
support.
• A: Because Alice
Alice’ss current
reading rate of 84 WRC
exceeds the 50th percentile
f ll norm off 83 WRC
fall
WRC, she
h
should be considered to
advance to Grade 4 materials
for intervention and
progress-monitoring.
www.interventioncentral.org
89
Response to Intervention
Activity: Group
Di
Discussion
i
• Review the steps
outlined in this
workshop for setting
goals
l ffor students
t d t on
off-level interventions.
• How prepared is your
district to use research
norms to guide goal
goalsetting for a student
whose intervention is
‘off-level’?
1
1.
2.
3
3.
4.
5.
Setting Individual RTI Academic
Goals Using
g Research Norms for
Students Receiving ‘Off-Level’
Interventions
C
Comparing
i St
Student
d t Performance
P f
to
t
Benchmarks and Flagging Extreme
Discrepancies
Conducting a Survey Level
Assessment (SLA).
S l ti an ‘Off
Selecting
‘Off-Level’
L l’ P
ProgressMonitoring Goal.
Translating a Progress
Progress-Monitoring
Monitoring
Goal into Weekly Increments.
Advancing the Student Who Makes
P
Progress
to Higher
Hi h Grade
G d LLevels
l ffor
Intervention and Progress-Monitoring.
www.interventioncentral.org
90
Response to Intervention
Placing Data in a ‘Data
Data Context’
Context
Focus of Inquiry: What simple organizing tool
can teachers use to help them to structure their
d t collection—to
data
ll ti
t iinclude
l d bbaseline,
li goal,l andd
progress-monitoring?
www.interventioncentral.org
91
Response to Intervention
The Structure of Data Collection
• Teachers can use a wide variety of methods to
assess student academic performance or behavior.
behavior
• However, data collection should be structured to
include these elements: baseline,
baseline the setting of a
goal for improvement, and regular progressg
monitoring.
• The structure of data collection can be thought of as
a gglass into which a wide varietyy of data can be
‘poured’.
www.interventioncentral.org
92
Response to Intervention
Classroom Data Collection Methods: Examples
•
•
•
•
•
•
•
•
Existing data
Gl b l skill
Global
kill checklist
h kli t
Behavioral frequency count/behavior rate
Rating scales
Academic skills: Cumulative mastery log
Work products
Behavior log
Curriculum-based measurement
www.interventioncentral.org
93
Response to Intervention
www.interventioncentral.org
Response to Intervention
Setting Up Effective Data Collection: Example
Example: Mrs. Braniff, a 3rd-grade teacher, decided to use a math
time drill intervention to helpp her student Brian to increase his
fluency with basic multiplication problems (0-9).
• To measure Brian’s progress on the intervention, Mrs. Braniff
decided to use Curriculum-Based Measurement Math
Computation worksheets (created on
www.interventioncentral.org).
i t
ti
t l )
• She used the RTI Classroom Progress-Monitoring Worksheet to
organize her data collection.
collection
www.interventioncentral.org
95
Response
to Intervention
RTI Classroom
Progress-Monitoring
Worksheet
www.interventioncentral.org
Response
to Intervention
RTI Classroom
Progress-Monitoring
Worksheet
www.interventioncentral.org
Response to Intervention
www.interventioncentral.org
Response to Intervention
Baseline: Defining the Student Starting Point
• Baseline data provide the teacher with a snapshot of
the student’s
student s academic skills or behavior before the
intervention begins.
• An estimate of baseline is essential in order to measure
at the end of the intervention whether the student made
significant
g
pprogress.
g
• Three to five data-points are often recommended—
because student behavior can be variable from dayy to
day.
www.interventioncentral.org
99
Response to Intervention
Baseline: Using the Median Score
If several data points are collected
collected, the middle
middle, or median
median,
score can be used to estimate student performance.
Selectingg the median can be a ggood idea when student
data is quite variable.
www.interventioncentral.org
100
Response to Intervention
Baseline: Using the Mean Score
If several data points are collected, an average, or mean,
score can be calculated by adding up all baseline data and
di idi bby the
dividing
th number
b off ddata
t points.
i t
13+15+11=39
39 divided by 3=13
Mean = 13
www.interventioncentral.org
101
Response to Intervention
www.interventioncentral.org
Response to Intervention
www.interventioncentral.org
Response to Intervention
Intervention ‘Timespan’: How Long is Long Enough?
Anyy intervention should be allowed sufficient time to
demonstrate whether it is effective. The limitation on how
quickly an intervention can be determined to be ‘effective’ is
usually the sensitivity off the measurement tools. As a rule,
behavioral interventions tend to show effects more quickly than
academic interventions—because
interventions because academic skills take time to
increase, while behavioral change can be quite rapid.
A good rule of thumb for classroom interventions is to allow 4-8
instructional weeks to judge the intervention.
www.interventioncentral.org
Response to Intervention
Performance Goal
The outcome ggoal for an intervention can be estimated in several
ways:
• If there are research academic norms or local norms available
(e.g., DIBELS), these can be useful to set a goal criterion.
• The teacher can screen a classroom to determine average
performance.
• The teacher can select 3-4 ‘typical’ students in the class,
administer an academic measure (e.g., curriculum-based
measurement writing) to calculate a ‘micro-norm’.
• The teacher can rely on ‘expert opinion’ off what is a typical
level of student performance.
www.interventioncentral.org
Response to Intervention
www.interventioncentral.org
Response to Intervention
www.interventioncentral.org
Response to Intervention
www.interventioncentral.org
Response to Intervention
www.interventioncentral.org
Response to Intervention
www.interventioncentral.org
Response to Intervention
www.interventioncentral.org
Response to Intervention
EXAMPLE: RTI Classroom Progress-Monitoring
g
g Worksheet
Mr. Brady, a 3rd grade teacher, plans an intervention for
his student, Veronica, who lacks mastery of Grade 1
sight words.
Mr. Brady plans to monitor Veronica’s sight word
recognition
g
weekly,
y usingg curriculum-based
measurement (CBM) Word Reading Fluency probes
from EasyCBM.
With this information, fill out sections A and B of the RTI
Cl
Classroom
P
Progress-Monitoring
M it i W
Worksheet
k h t (use
(
blank
bl k
sheet in packet 3 on p.7).
www.interventioncentral.org
112
Response to Intervention
www.interventioncentral.org
Response to Intervention
RTI Classroom Progress-Monitoring Worksheet
Mr. Brady decides that he will collect 3 baseline data-points on
Veronica. He also pplans to take the median of those baseline datapoints.
With this information, fill out section C of the RTI Classroom
Progress-Monitoring Worksheet, including calculating the actual
baseline figure.
Baseline Data for Veronica
1/13/2012
1/17/2012
1/20/2012
8
12
9
www.interventioncentral.org
114
Response to Intervention
www.interventioncentral.org
Response to Intervention
RTI Classroom Progress-Monitoring Worksheet
The teacher decides that the intervention for Veronica will last 7
instructional weeks,, endingg on Fridayy March 9,, 2012.
Mr. Brady also consults Word Reading Fluency norms from
easyCBM and decides to set an outcome goal for Veronica (at the
end of the intervention) of 23 Correctly Read Words.
With this information, fill out sections D & E of the RTI Classroom
Progress-Monitoring Worksheet.
www.interventioncentral.org
116
Response to Intervention
RTI Classroom Progress-Monitoring Worksheet
www.interventioncentral.org
117
Response to Intervention
RTI Classroom Progress-Monitoring Worksheet
Mr. Brady decides that he
will summarize Veronica’s
progress by taking the
median of the final 3
progress-monitoring
observations. Progressmonitoring data appear to
the right. With this
information, fill out the
remaining sections of the
RTI Classroom ProgressMonitoring Worksheet.
Progress-Monitoring Data for
V
Veronica
i
1/25/2012
2/1/2012
2/7/2012
2/14/2012
2/22/2012
2/29/2012
3/7/2012
www.interventioncentral.org
12
9
14
17
22
26
21
118
Response to Intervention
RTI Classroom Progress-Monitoring Worksheet
www.interventioncentral.org
119
Response to Intervention
Chartingg Data
Focus of Inquiry: How
can progressmonitoring data be
converted to a visual
di l to
display
t hhelp
l tteachers
h
to make instructional
and intervention
decisions?
www.interventioncentral.org
120
Response to Intervention
Sample Peer Tutoring Chart
www.interventioncentral.org
121
Response to Intervention
Sample Peer Tutoring Chart
www.interventioncentral.org
122
Response to Intervention
Source: New York State Education Department. (October 2010). Response to Intervention: Guidance for New York State
School Districts. Retrieved November 10, 2010, from http://www.p12.nysed.gov/specialed/RTI/guidance-oct10.pdf; p. 21
www.interventioncentral.org
123
Response to Intervention
The Intervention Central Guide to…Setting
to Setting
Up and Interpreting Time-Series Charts
Response to Intervention requires that schools collect
data on student progress over time to demonstrate
whether an academic or behavioral intervention is
working. It is much easier to see the student
student’ss overall
rate of progress when data are converted to a visual
display. The time-series chart is the type of visual
display most commonly used to graph student progress.
www.interventioncentral.org
124
Response to Intervention
Components of Time-Series Charts
LLabels
b l off Vertical
V ti l (‘Y’) andd Horizontal
H i t l (‘X’) A
Axes. Th
The vertical
ti l axis
i off th
the
chart is labeled with the ‘behavior’ that is being measured. The
horizontal axis of the chart displays
p y the timespan
p duringg which pprogressg
monitoring took place.
www.interventioncentral.org
125
Response to Intervention
Components of Time-Series Charts
Phase Changes. The chart is divided into phases— time periods in which
data are collected under similar conditions. Phases are separated with
vertical lines. Each phase is labeled to indicate the intervention condition in
effect (e.g.,
(e g ‘Baseline:
Baseline: Teacher whole
whole-group
group math instruction
instruction’)). Data
collected within a phase are plotted as a series of connected data points.
www.interventioncentral.org
126
Response to Intervention
Components of Time-Series Charts
Baseline Data.
Data RTI Teams will often collect baseline data to determine a
student’s starting point before an intervention is begun. Baseline data
provides a snapshot of the student’s level of academic or behavioral
f ti i before
functioning
b f an individualized
i di id li d iintervention
t
ti iis putt iinto
t place.
l
www.interventioncentral.org
127
Response to Intervention
Components of Time-Series Charts
Progress-Monitoring Data.
Data Once an individualized academic or behavioral
intervention has been put into place for a student student, the RTI Team
then monitors the intervention frequently (e.g., weekly) to track that
student’s
t d t’ response to
t the
th iintervention.
t
ti
www.interventioncentral.org
128
Response to Intervention
Components of Time-Series Charts
Plotting Goal Line and Aimline.
Aimline When charting student progress
progress, visual
indicators show the goal that the student is striving to reach and the
expected rate of progress that the student is predicted to make. The goal
li is
line
i drawn
d
as a horizontal
h i t l line
li th
thatt represents
t a successful
f l level
l l off
performance.
www.interventioncentral.org
129
Response to Intervention
Components of Time-Series Charts
The aimline is a sloping line connecting baseline and goal that shows the
rate at which the student is predicted to make progress if the intervention is
successful. By plotting both goal line and aimline, the RTI Team can
visually
i ll compare th
the student’s
t d t’ actual
t l performance
f
on any dday tto th
the
expected rate of progress (aimline) and eventual goal for improvement
(goal line).
(g
)
www.interventioncentral.org
130
Response to Intervention
ChartDog GraphMaker
Provides teachers with a tool to create single-subject timeseries graphs.
graphs The free application allows the user to save
his or her data and store online. ChartDog also allows the
user to:
–
–
–
–
–
–
enter up to four data series on one graph
enter and label phase changes
set goal-lines and aimlines
compute trend-lines for any data series by phase
compute percentage of non-overlapping data points
compute
t No-Assumptions
N A
ti
Effect
Eff t Size
Si (NAES) bbetween
t
2
phases
www.interventioncentral.org
131
Response to Intervention
ChartDog
www interventioncentral org
www.interventioncentral.org
www.interventioncentral.org
Response to Intervention
ChartDog Graph Maker
www.interventioncentral.org
133
Response to Intervention
Review the ‘Quality Indicators for Progress-Monitoring’ below from the New
York State RTI Guidance Document. Decide on 22-33 key ‘next
next steps
steps’ that you
would like to take to make use of the resources / recommendations on data
collection shared at today’s workshop.
Source: New York State Education Department. (October 2010). Response to Intervention: Guidance for New York State School
Districts. Retrieved November 10, 2010, from http://www.p12.nysed.gov/specialed/RTI/guidance-oct10.pdf; p. 22
www.interventioncentral.org
134
Response to Intervention
Predicting
P
di ti Success
S
in
i General
G
l andd Special
S i l
Education: How to Select and Chart
'A biti
'Ambitious
But
B t Realistic'
R li ti ' Student
St d t Academic
A d i
Goals: Next Steps
In your teams:
•
Review the resources and ideas shared at this
workshop.
•
Decide on 22-33 key ‘next
next steps
steps’ that you plan to take
to apply workshop content in your classroom or
school.
www.interventioncentral.org
135