Leading
DataDriven
Schools
Kids First … New Mexico Wins
Acknowledgements
 This training is based on Paul
Bambrick-Santoyo’s Driven by
Data: A Practical Guide to
Improve Instruction
Kids First … New Mexico Wins
Acknowledgements
 Through our collaboration with the University of Virginia,
School Turnaround Specialist Program (STSP), we have had the
opportunity to learn from the STSP team.
 Mike Kight has graciously shared his resources and work on
leading data-driven schools that set the foundation for our
work.
Kids First … New Mexico Wins
NY State Public School ELA 4th Performance
vs. Free-Reduced Rates
100%
Percent Proficient
90%
80%
70%
60%
50%
40%
30%
20%
10%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percent Free-Reduced Lunch
Kids First … New Mexico Wins
NY State Public School ELA 4th Performance
vs. Free-Reduced Rates
100%
Percent Proficient
90%
80%
70%
60%
50%
40%
30%
20%
10%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percent Free-Reduced Lunch
Kids First … New Mexico Wins
Norms
 Start/end on time
 Hand raised
 All technology is on task (no
mid-session breaks)
 Hold each other accountable
 Dive in to make this your own
 Write down burning questions
as we go
 Ask clarifying questions
relevant to all
Kids First … New Mexico Wins
GETTING POOLSIDE: Examining Student Data
Kids First … New Mexico Wins
Man on Fire
What were the key moments in Creasy’s
attempt to help Pita?
What made Creasy’s analysis effective?
How did Creasy get Pita to see the error?
Kids First … New Mexico Wins
Analysis
Core Idea:
If you are not at the pool,
you cannot do the analysis.
Kids First … New Mexico Wins
ANALYSIS
Kids First … New Mexico Wins
GETTING STARTED
Kids First … New Mexico Wins
Part 1 – Global Impressions
 How well did the class do as a whole?
 What are the strengths and weaknesses in the
standards? Where do we need the most work?
 How did the class do on old versus new standards?
Are they improving on or forgetting old material?
 How were the results in different question types
(multiple choice, open-ended, written response)?
 Who are the strong and weak students?
Kids First … New Mexico Wins
Part 2 – Deep Analysis
 Squint: Bombed questions – did all students choose the
same wrong answer? Why or why not?
 Break down each standard: Did students do similarly
on every question or were some questions harder?
Why?
 Compare similar standards: Do results in one influence
the other?
 Sort data by student scores: Are there questions that
separate proficient from non-proficient students?
 Look horizontally by student: Are there any anomalies?
Kids First … New Mexico Wins
Interim Assessment Teacher Reflection –
Question-level Analysis
Kids First … New Mexico Wins
Interim Assessment Teacher Reflection –
Standards-level Analysis
Kids First … New Mexico Wins
Analysis
CORE IDEA:
Test-in-hand analysis is not one
possible way to analyze student
error – it is the only means by
which to do effective analysis.
Kids First … New Mexico Wins
Four Steps for Data-Driven Analysis
Praise
Probe
Kids First … New Mexico Wins
Role Play Cheat Sheet
Kids First … New Mexico Wins
ROLE PLAY
Kids First … New Mexico Wins
Role Play Reflection
 What did you learn about the teachers?
 How was this interim assessment analysis meeting
different from a post-observation conference?
Kids First … New Mexico Wins
BREAK
Kids First … New Mexico Wins
IMPACT OF DATA-DRIVEN INSTRUCTION
Kids First … New Mexico Wins
North Star Middle Schools 2003 vs. 2008
Kids First … New Mexico Wins
SAT Best Combined Scores – 2005 to 2012
1150
1100
1050
National
1000
North Star
950
900
850
2005
2006
2007
2008
2009
2010
2011
2012
Kids First … New Mexico Wins
Percent of Juniors and Seniors
Taking AP Exams Over Time
70%
65%
60%
49%
50%
43%
38%
40%
30%
20%
17%
19%
10%
0%
2007
2008
2009
2010
2011
2012
Kids First … New Mexico Wins
AP Results – Six Year Score Summary
90%
81%
75%
73%
62%
60%
60%
75%
74%
73%
72%
61%
60%
60%60%
75%
60%
47%
NJ Statewide
45%
Global
30%
30%
15%
North Star
33%
11%
0%
2007
2008
2009
2010
2011
2012
Kids First … New Mexico Wins
Dodge Academy: Turnaround through
Transparency
90.0%
79.4%
80.0%
69.7%
70.0%
58.1%
60.0%
50.0%
2005
2008
40.0%
32.5%
30.0%
21.6%
20.5%
20.0%
10.0%
0.0%
Reading
Math
Science
Kids First … New Mexico Wins
Baltimore Bombshell
Kids First … New Mexico Wins
Principals Pursuing Excellence –
One Year of Growth, 2013-14
 Pojoaque Valley Intermediate increased reading proficiency
by 11.1% and math proficiency by 18%.
 Jornada Elementary in Las Cruces increased reading
proficiency by 7.7% and math proficiency by 15.8%.
 White Mountain Elementary in Ruidoso increased reading
proficiency by 4.9% and math proficiency by 20%.
Kids First … New Mexico Wins
Quick-Write Reflection
 From what you know right now, what are the most
important things you would need to launch a datadriven instructional model in your school?
Kids First … New Mexico Wins
The Four Keys to Data-Driven Instruction:
ASSESSMENTS
ANALYSIS
ACTION
In a Data-Driven CULTURE
Kids First … New Mexico Wins
The Power of the Question – Math
1. Read the six assessment questions.
2. Discuss the prompts with a partner.
Kids First … New Mexico Wins
Assessment
CORE IDEAS:
Standards (and objectives) are meaningless
until you define how to assess them.
Because of this, assessments are not the
end of the teaching and learning process;
they’re the beginning.
Kids First … New Mexico Wins
The Power of the Question – Reading
1. Read the four assessment questions.
2. Discuss the prompts with a partner.
Kids First … New Mexico Wins
Assessment
CORE IDEAS:
In an open-ended question, the rubric
defines the rigor.
In a multiple choice question, the options
and text difficulty define the rigor.
Kids First … New Mexico Wins
Assessment
 Common Interim Assessments
 Transparent Starting Point
 Aligned to State Tests, Instructional Sequence,
and College Ready Expectations
 System of Teaching, Assessment, Reteaching,
Reassessment
Kids First … New Mexico Wins
The Four Keys to Data-Driven Instruction:
ASSESSMENTS
(Interim, Transparent, Aligned, Re-Assess)
ANALYSIS
ACTION
In a Data-Driven CULTURE
Kids First … New Mexico Wins
“Man on Fire” – A Video Case Study
 After a solid analysis, what
made Creasy’s action plan
effective?
Kids First … New Mexico Wins
Assessment & Analysis
CORE IDEA:
How do we find more time
to teach our students?
Spend less time teaching what they already
know and more on what they need.
Kids First … New Mexico Wins
Analysis
 Immediate turnaround of results and reports
provide user-friendly, succinct item-level
analysis, standards-level analysis, and
bottom-line results.
 Teacher-owned analysis
 Test-in-hand analysis
 Deep analysis moves beyond what students
got wrong to answer why they got it wrong.
Kids First … New Mexico Wins
The Four Keys to Data-Driven Instruction:
ASSESSMENTS
(Interim, Transparent, Aligned, Re-Assess)
ANALYSIS
(Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep)
ACTION
In a Data-Driven CULTURE
Kids First … New Mexico Wins
Looking at a Model of Deep Analysis
 Review one of the following teacher analysis/action
plans:
– 2nd grade Math
– 7th grade English
– 7th grade Math
– 10th grade English
 What makes these teacher analysis/action plans
effective?
Kids First … New Mexico Wins
One-and-a-half-pager: Pre-Work for
Effective Analysis Meetings
Kids First … New Mexico Wins
Deep Analysis in Practice
 Revisit your analysis from the first role play – where
could your analysis have been deeper?
 Pick 1 standard from the assessment: analyze deeply
in preparation for giving feedback to a teacher:
– Generate a hypothesis backed with evidence from the
questions
– Identify how you would test your hypothesis
– Identify action plan for whole group (explicit instruction,
assignments, assessments/checks for understanding)
– Identify action plan for struggling students
Kids First … New Mexico Wins
Lunch
Kids First … New Mexico Wins
Opening Praise – Julie
 What makes Julie’s praise effective?
Kids First … New Mexico Wins
Praise – Narrate the Positive
 Genuine: heart-felt, authentic
 Reinforce Positive Actions: particularly those that
are connected to the teacher’s goals for
development
 Teacher-Driven: teacher does the reflection and
narrates why they were successful
Kids First … New Mexico Wins
Probe on Analysis – Beth
 What prompts does Beth use to guide Steve to a
deeper analysis of his results?
 What makes these prompts effective?
Kids First … New Mexico Wins
Probe on Analysis
 Narrow the Focus: look at the standards that merit deeper
analysis or better action plans
 Begin with the End Goal: start with what you want students
doing to answer the question correctly
 Look at the Gap: what is the gap between what students
should do and what they actually did?
 Analyze the Error: analyze what students wrote/did that led
to the error
Kids First … New Mexico Wins
Action Plan & Follow-Up – Beth
 What makes Beth’s prompts/questions so effective?
Kids First … New Mexico Wins
Action Plan & Follow-Up
 Detailed, specific actions: “What would that [lesson,
worksheet, exit ticket] look like?”
 Time-saving: teacher leaves with more
accomplished, rather than more action steps
Kids First … New Mexico Wins
The Four Keys to Data-Driven Instruction:
ASSESSMENTS
(Interim, Transparent, Aligned, Re-Assess)
ANALYSIS
(Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep)
ACTION
(Action Plan, Ongoing, Accountability, Engaged Students)
In a Data-Driven CULTURE
Kids First … New Mexico Wins
Preparing to Lead an
Effective Analysis Meeting
 Read through the onepager on leading effective
analysis meetings
Kids First … New Mexico Wins
Preparing to Lead an
Effective Analysis Meeting
 Script out your questions
and the prompts you will
use to guide the teacher
through each step of the
analysis meeting
Kids First … New Mexico Wins
Analysis Simulation, Round 1 – Probe
 IDENTIFY ROLES: Teacher, Leader
 ROLE PLAY ANALYSIS MEETING (4 min):
– 1 min – Praise
– 3 min – Probing Analysis
• Get teacher to a deep analysis
 DEBRIEF THE ROLE PLAY (2 min):
– 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the
leader
– 1 min - Teacher gives feedback
• What did the leader do most effectively? Where can s/he improve?
 MINI RE-DO (3 min):
– 2 min – Select the hardest part of the feedback meeting and re-do that section
– 1 min – Teacher gives quick feedback
Kids First … New Mexico Wins
Analysis Simulation, Round 2 – Probe
 SWITCH ROLES
 ROLE PLAY ANALYSIS MEETING (4 min):
– 1 min – Praise
– 3 min – Probing Analysis
• Get teacher to a deep analysis
 DEBRIEF THE ROLE PLAY (2 min):
– 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the
leader
– 1 min - Teacher gives feedback
• What did the leader do most effectively? Where can s/he improve?
 MINI RE-DO (3 min):
– 2 min – Select the hardest part of the feedback meeting and re-do that section
– 1 min – Teacher gives quick feedback
Kids First … New Mexico Wins
Analysis Simulation, Round 3 –
Action and Follow-Up
 IDENTIFY ROLES: Teacher, Leader
 ROLE PLAY ANALYSIS MEETING (4 min):
– Assume you have completed the deep analysis
– 4 min – Begin from the Action Step
• Get teacher to make explicit action plan for the classroom
• Schedule the follow-up
 DEBRIEF THE ROLE PLAY (2 min):
– 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the leader
– 1 min – Teacher gives feedback
• What did the leader do most effectively? Where can s/he improve?
 MINI RE-DO (3 min):
– Select the hardest part of the feedback meeting and re-do that section (2 min)
– Teacher gives quick feedback (1 min)
Kids First … New Mexico Wins
Analysis Simulation, Round 4 –
Action and Follow-Up
 SWITCH ROLES
 ROLE PLAY ANALYSIS MEETING (4 min):
– Assume you have completed the deep analysis
– 4 min – Begin from the Action Step
• Get teacher to make explicit action plan for the classroom
• Schedule the follow-up
 DEBRIEF THE ROLE PLAY (2 min):
– 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the leader
– 1 min – Teacher gives feedback
• What did the leader do most effectively? Where can s/he improve?
 MINI RE-DO (3 min):
– Select the hardest part of the feedback meeting and re-do that section (2 min)
– Teacher gives quick feedback (1 min)
Kids First … New Mexico Wins
Role Play Reflection:
 What are your biggest takeaways for leading effective
analysis meetings?
 What are the actions we need to take to improve the
depth of our teacher action plans and the quality of
the analysis meetings?
Kids First … New Mexico Wins
ACTION, REVISITED
Kids First … New Mexico Wins
Mr. Holland’s Opus
 What made the difference? How did Lou Russ finally
learn to play the drum?
 What changed Mr. Holland’s attitude and actions?
Kids First … New Mexico Wins
 Teachers plan lessons collaboratively based
on data analysis
Action
 Teachers write action plans to apply insights
from analysis meetings
 Specific timeline for implementation
 Accountability – leaders check for alignment
 Engaged Students
Kids First … New Mexico Wins
Increasing Rigor Through the Lesson
 Tool found on pages 70-73 in your training manual
 TASK
– Put a question mark next to activities you want to
understand more deeply in order to implement
effectively.
– Put a star next to activities that sound particularly
doable for you that you want to implement.
– Double star the activities you want to implement
first
Kids First … New Mexico Wins
Case Study: Douglass Street School
1. Did Krista win the bet? What percentage of
teachers’ classes do you think achieved gains of 15
percentage points on the state test?
2. Based on your answers, name the biggest stumbling
blocks to the school’s success.
3. Based on your answers, name the most important
drivers of school improvement.
Kids First … New Mexico Wins
BREAK
Kids First … New Mexico Wins
Poll the Crowd
 What percentage of teachers’ classes do you think
achieved gains of 15 percentage points on the state
test?
– 0-25%
– 26-50%
– 51-75%
– 76-99%
– 100%
Kids First … New Mexico Wins
The Four Keys to Data-Driven Instruction:
ASSESSMENTS
(Interim, Transparent, Aligned, Re-Assess)
ANALYSIS
(Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep)
ACTION
(Action Plan, Ongoing, Accountability, Engaged Students)
In a Data-Driven CULTURE
(Leadership, PD, Calendar, Build by Borrowing)
Kids First … New Mexico Wins
Data-Driven Culture
CORE IDEA:
Any initiative that requires buy-in from the
beginning will fail.
When done well, data-driven instruction
doesn’t require buy-in, it creates it.
Kids First … New Mexico Wins
Data & Culture
Kids First … New Mexico Wins
 Introductory Professional Development: what
(assessments) and how (analysis and action)
 Implementation Calendar: with time for assessments,
analysis and action
Data-Driven Culture
 Highly Active Leadership Team: teacher-leader data
analysis meetings, maintain focus
 Ongoing Professional Development: aligned with
calendar, flexible to adapt to student learning needs
 Build By Borrowing: identify and implement best
practices from high-achieving teachers and schools
Kids First … New Mexico Wins
The Four Keys to Data-Driven Instruction:
ASSESSMENTS
(Interim, Transparent, Aligned, Re-Assess)
ANALYSIS
(Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep)
ACTION
(Action Plan, Ongoing, Accountability, Engaged Students)
In a Data-Driven CULTURE
(Leadership, PD, Calendar, Build by Borrowing)
Kids First … New Mexico Wins
Data-Driven Instruction
CORE IDEA:
Data-driven instruction shifts the focus
from the teaching to the learning.
Kids First … New Mexico Wins
Roadside Distractions
 The pursuit of total buy-in
 Reliance on poorly
implemented professional
learning communities (PLCs)
 Year-end assessment analysis
Kids First … New Mexico Wins
Eight Mistakes that Matter
 Inferior Interim
Assessments
 Secretive Interim
Assessments
 Delayed Results
 Separation of Teaching
and Analysis
 Infrequent Assessments
 Ineffective Follow-Up
 Curriculum-Assessment
Disconnect
 Not Making Time for
Data
Kids First … New Mexico Wins
Pitfalls
Kids First … New Mexico Wins
Pitfalls
 Giving teachers too much data that they are not sure
where they should focus
 Too many forms
 Not engaging teachers who teach content areas for
which data is not available
Kids First … New Mexico Wins
Pitfalls
 Assumption that plans are being implemented
 A cycle is seen as an “event” instead of a process
 Compliance Syndrome
 Action Plans developed from one assessment
Kids First … New Mexico Wins
Pitfalls
 Analysis alone will not help teachers translate ideas
into classroom practice
 Underinvesting in building the capacity of teachers to
deepen their use of effective teaching strategies
 Disproportionate attention for students on the
“bubble”
Kids First … New Mexico Wins
But…
Kids First … New Mexico Wins
Finding the Time
 Meaningful data analysis, identifying student
learning problems by triangulating numerous data
sources, analyzing student work samples, finding
root causes for emerging problems, and creating a
plan to tackle these problems takes time.
 We suggest 90 minutes of uninterrupted time.
Kids First … New Mexico Wins
Finding the Time
 Restructured or rescheduled time – Altering the
school day, school year, or teaching schedules.
 Using time better – Using time that is already built in
for other less important tasks or those that could be
handled through electronic communication.
 Hiring subs or finding volunteers to free teachers
from regular instructional time for interim
assessment analysis.
Kids First … New Mexico Wins
Finding the Time
 Reallocating funds to provide teachers with stipends
for staying after school or giving up other time.
 Stagger teacher and student schedules to have one
group report early or later in the day.
Kids First … New Mexico Wins
DDI Implementation Rubric
 INDIVIDUAL:
– Score your school on the rubric
– Where are the weakest areas that could be
addressed first?
 SCHOOL TEAM (OR SMALL GROUP):
– Identify highest-leverage area of focus
Kids First … New Mexico Wins
Results Meeting Protocol
 Tool found on page 74 in the
manual
 Originally set up by the
Brazosport (Texas) School
District
 Highlighted by Mike Schmoker in
The Results Fieldbook
 Designed to be implemented in
a 50 minute block (can be
lengthened or shortened)
Kids First … New Mexico Wins
Results Meeting Protocol
 Identify roles: Timer, facilitator, recorder (2 minutes)
 Identify objective to focus on (3 minutes unless objective is
given)
 What worked so far (5 minutes) [Or: What teaching strategies
did you try so far?]
 Chief challenges (5 minutes)
 Brainstorm proposed solutions (10 minutes)
 Reflection: Feasibility of each idea (5 minutes)
 Consensus around best actions (15 minutes)
 Put in calendar: When will the tasks happen? When will the
teaching happen? (10 minutes)
Kids First … New Mexico Wins
Brainstorming Protocol
 Go in order around the circle: each person has 30 seconds to
share a proposal
 If you don’t have an idea, say “Pass.”
 No judgments should be made; if you like the idea, when it’s
your turn simply say, “I would like to add to that idea by…”
 Even if 4-5 people pass in a row, keep going for the full
brainstorming time.
Kids First … New Mexico Wins
Reflection Protocol
 1 minute—silent personal/individual reflection on the list:
what is doable and what isn’t for each person
 Go in order around the circle once: depending on size of
group each person has 30-60 seconds to share their
reflections.
 If a person doesn’t have a thought to share, say “Pass” and
come back to him/her later.
Kids First … New Mexico Wins
Consensus and Calendar Guidelines
 Identify key actions from brainstorming that everyone will agree to
implement.
 Make actions as specific as possible within the limited time.
 Identify key student-teacher guides or tasks needed to be done to be
ready to teach.
 Identify who will do each task.
 Identify when each task will be done.
 Put date for re-teaching on calendar.
 Spend remaining time developing concrete elements of lesson plan:
– Do Nows
– Teacher guides (for example, what questions to ask the students or how to
structure the activity)
– Student guides, homework, and so on.
Kids First … New Mexico Wins
Key Tips to Making a
Results Meeting Productive
 GET SPECIFIC to the question itself – “We can teach
10 lessons on this standard. What is the set of
lessons these students need based on the data?”
 AVOID PHILOSOPHICAL DEBATES about theories of
Math/Literacy; focus on the small, specific challenge
of the moment. That’s where the change will begin!
 IF THE GROUP IS TOO LARGE, split into two groups.
You’ll generate more ideas and you can share your
conclusions/action plans at the end.
Kids First … New Mexico Wins
Practice: Results Meeting (25 min)
Simulate parts of results team meeting
– Topic: your biggest area of growth from the DDI
rubric
– Pick a major challenge that you think you will face
as school leaders leading this
– Focus on brainstorm, reflection, and begin
consensus building
Kids First … New Mexico Wins
Reflection:
 What was effective about this protocol?
 What was challenging?
Kids First … New Mexico Wins
Visualizing Success
Kids First … New Mexico Wins
Final Reflection
 What is my personal commitment to making datadriven instruction work more effectively in my
school/district?
Kids First … New Mexico Wins
Relentlessness
Through a relentless
commitment to
observation, feedback,
courageous
conversations, and
customized support, we
are seeing LEAs within
New Mexico improve
outcomes for our
children.
Kids First … New Mexico Wins
Kids First … New Mexico Wins
Contact information
Hannah Peria
School Improvement Specialist
Priority Schools Bureau
[email protected]
(505) 827-8005
Howard Oechsner
School Improvement Specialist
Priority Schools Bureau
[email protected]
(505) 827-6580
Kids First … New Mexico Wins