Leading Data- Driven Schools - New Mexico State Department of
Leading DataDriven Schools Kids First … New Mexico Wins Acknowledgements This training is based on Paul Bambrick-Santoyo’s Driven by Data: A Practical Guide to Improve Instruction Kids First … New Mexico Wins Acknowledgements Through our collaboration with the University of Virginia, School Turnaround Specialist Program (STSP), we have had the opportunity to learn from the STSP team. Mike Kight has graciously shared his resources and work on leading data-driven schools that set the foundation for our work. Kids First … New Mexico Wins NY State Public School ELA 4th Performance vs. Free-Reduced Rates 100% Percent Proficient 90% 80% 70% 60% 50% 40% 30% 20% 10% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percent Free-Reduced Lunch Kids First … New Mexico Wins NY State Public School ELA 4th Performance vs. Free-Reduced Rates 100% Percent Proficient 90% 80% 70% 60% 50% 40% 30% 20% 10% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percent Free-Reduced Lunch Kids First … New Mexico Wins Norms Start/end on time Hand raised All technology is on task (no mid-session breaks) Hold each other accountable Dive in to make this your own Write down burning questions as we go Ask clarifying questions relevant to all Kids First … New Mexico Wins GETTING POOLSIDE: Examining Student Data Kids First … New Mexico Wins Man on Fire What were the key moments in Creasy’s attempt to help Pita? What made Creasy’s analysis effective? How did Creasy get Pita to see the error? Kids First … New Mexico Wins Analysis Core Idea: If you are not at the pool, you cannot do the analysis. Kids First … New Mexico Wins ANALYSIS Kids First … New Mexico Wins GETTING STARTED Kids First … New Mexico Wins Part 1 – Global Impressions How well did the class do as a whole? What are the strengths and weaknesses in the standards? Where do we need the most work? How did the class do on old versus new standards? Are they improving on or forgetting old material? How were the results in different question types (multiple choice, open-ended, written response)? Who are the strong and weak students? Kids First … New Mexico Wins Part 2 – Deep Analysis Squint: Bombed questions – did all students choose the same wrong answer? Why or why not? Break down each standard: Did students do similarly on every question or were some questions harder? Why? Compare similar standards: Do results in one influence the other? Sort data by student scores: Are there questions that separate proficient from non-proficient students? Look horizontally by student: Are there any anomalies? Kids First … New Mexico Wins Interim Assessment Teacher Reflection – Question-level Analysis Kids First … New Mexico Wins Interim Assessment Teacher Reflection – Standards-level Analysis Kids First … New Mexico Wins Analysis CORE IDEA: Test-in-hand analysis is not one possible way to analyze student error – it is the only means by which to do effective analysis. Kids First … New Mexico Wins Four Steps for Data-Driven Analysis Praise Probe Kids First … New Mexico Wins Role Play Cheat Sheet Kids First … New Mexico Wins ROLE PLAY Kids First … New Mexico Wins Role Play Reflection What did you learn about the teachers? How was this interim assessment analysis meeting different from a post-observation conference? Kids First … New Mexico Wins BREAK Kids First … New Mexico Wins IMPACT OF DATA-DRIVEN INSTRUCTION Kids First … New Mexico Wins North Star Middle Schools 2003 vs. 2008 Kids First … New Mexico Wins SAT Best Combined Scores – 2005 to 2012 1150 1100 1050 National 1000 North Star 950 900 850 2005 2006 2007 2008 2009 2010 2011 2012 Kids First … New Mexico Wins Percent of Juniors and Seniors Taking AP Exams Over Time 70% 65% 60% 49% 50% 43% 38% 40% 30% 20% 17% 19% 10% 0% 2007 2008 2009 2010 2011 2012 Kids First … New Mexico Wins AP Results – Six Year Score Summary 90% 81% 75% 73% 62% 60% 60% 75% 74% 73% 72% 61% 60% 60%60% 75% 60% 47% NJ Statewide 45% Global 30% 30% 15% North Star 33% 11% 0% 2007 2008 2009 2010 2011 2012 Kids First … New Mexico Wins Dodge Academy: Turnaround through Transparency 90.0% 79.4% 80.0% 69.7% 70.0% 58.1% 60.0% 50.0% 2005 2008 40.0% 32.5% 30.0% 21.6% 20.5% 20.0% 10.0% 0.0% Reading Math Science Kids First … New Mexico Wins Baltimore Bombshell Kids First … New Mexico Wins Principals Pursuing Excellence – One Year of Growth, 2013-14 Pojoaque Valley Intermediate increased reading proficiency by 11.1% and math proficiency by 18%. Jornada Elementary in Las Cruces increased reading proficiency by 7.7% and math proficiency by 15.8%. White Mountain Elementary in Ruidoso increased reading proficiency by 4.9% and math proficiency by 20%. Kids First … New Mexico Wins Quick-Write Reflection From what you know right now, what are the most important things you would need to launch a datadriven instructional model in your school? Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS ANALYSIS ACTION In a Data-Driven CULTURE Kids First … New Mexico Wins The Power of the Question – Math 1. Read the six assessment questions. 2. Discuss the prompts with a partner. Kids First … New Mexico Wins Assessment CORE IDEAS: Standards (and objectives) are meaningless until you define how to assess them. Because of this, assessments are not the end of the teaching and learning process; they’re the beginning. Kids First … New Mexico Wins The Power of the Question – Reading 1. Read the four assessment questions. 2. Discuss the prompts with a partner. Kids First … New Mexico Wins Assessment CORE IDEAS: In an open-ended question, the rubric defines the rigor. In a multiple choice question, the options and text difficulty define the rigor. Kids First … New Mexico Wins Assessment Common Interim Assessments Transparent Starting Point Aligned to State Tests, Instructional Sequence, and College Ready Expectations System of Teaching, Assessment, Reteaching, Reassessment Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS (Interim, Transparent, Aligned, Re-Assess) ANALYSIS ACTION In a Data-Driven CULTURE Kids First … New Mexico Wins “Man on Fire” – A Video Case Study After a solid analysis, what made Creasy’s action plan effective? Kids First … New Mexico Wins Assessment & Analysis CORE IDEA: How do we find more time to teach our students? Spend less time teaching what they already know and more on what they need. Kids First … New Mexico Wins Analysis Immediate turnaround of results and reports provide user-friendly, succinct item-level analysis, standards-level analysis, and bottom-line results. Teacher-owned analysis Test-in-hand analysis Deep analysis moves beyond what students got wrong to answer why they got it wrong. Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS (Interim, Transparent, Aligned, Re-Assess) ANALYSIS (Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep) ACTION In a Data-Driven CULTURE Kids First … New Mexico Wins Looking at a Model of Deep Analysis Review one of the following teacher analysis/action plans: – 2nd grade Math – 7th grade English – 7th grade Math – 10th grade English What makes these teacher analysis/action plans effective? Kids First … New Mexico Wins One-and-a-half-pager: Pre-Work for Effective Analysis Meetings Kids First … New Mexico Wins Deep Analysis in Practice Revisit your analysis from the first role play – where could your analysis have been deeper? Pick 1 standard from the assessment: analyze deeply in preparation for giving feedback to a teacher: – Generate a hypothesis backed with evidence from the questions – Identify how you would test your hypothesis – Identify action plan for whole group (explicit instruction, assignments, assessments/checks for understanding) – Identify action plan for struggling students Kids First … New Mexico Wins Lunch Kids First … New Mexico Wins Opening Praise – Julie What makes Julie’s praise effective? Kids First … New Mexico Wins Praise – Narrate the Positive Genuine: heart-felt, authentic Reinforce Positive Actions: particularly those that are connected to the teacher’s goals for development Teacher-Driven: teacher does the reflection and narrates why they were successful Kids First … New Mexico Wins Probe on Analysis – Beth What prompts does Beth use to guide Steve to a deeper analysis of his results? What makes these prompts effective? Kids First … New Mexico Wins Probe on Analysis Narrow the Focus: look at the standards that merit deeper analysis or better action plans Begin with the End Goal: start with what you want students doing to answer the question correctly Look at the Gap: what is the gap between what students should do and what they actually did? Analyze the Error: analyze what students wrote/did that led to the error Kids First … New Mexico Wins Action Plan & Follow-Up – Beth What makes Beth’s prompts/questions so effective? Kids First … New Mexico Wins Action Plan & Follow-Up Detailed, specific actions: “What would that [lesson, worksheet, exit ticket] look like?” Time-saving: teacher leaves with more accomplished, rather than more action steps Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS (Interim, Transparent, Aligned, Re-Assess) ANALYSIS (Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep) ACTION (Action Plan, Ongoing, Accountability, Engaged Students) In a Data-Driven CULTURE Kids First … New Mexico Wins Preparing to Lead an Effective Analysis Meeting Read through the onepager on leading effective analysis meetings Kids First … New Mexico Wins Preparing to Lead an Effective Analysis Meeting Script out your questions and the prompts you will use to guide the teacher through each step of the analysis meeting Kids First … New Mexico Wins Analysis Simulation, Round 1 – Probe IDENTIFY ROLES: Teacher, Leader ROLE PLAY ANALYSIS MEETING (4 min): – 1 min – Praise – 3 min – Probing Analysis • Get teacher to a deep analysis DEBRIEF THE ROLE PLAY (2 min): – 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the leader – 1 min - Teacher gives feedback • What did the leader do most effectively? Where can s/he improve? MINI RE-DO (3 min): – 2 min – Select the hardest part of the feedback meeting and re-do that section – 1 min – Teacher gives quick feedback Kids First … New Mexico Wins Analysis Simulation, Round 2 – Probe SWITCH ROLES ROLE PLAY ANALYSIS MEETING (4 min): – 1 min – Praise – 3 min – Probing Analysis • Get teacher to a deep analysis DEBRIEF THE ROLE PLAY (2 min): – 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the leader – 1 min - Teacher gives feedback • What did the leader do most effectively? Where can s/he improve? MINI RE-DO (3 min): – 2 min – Select the hardest part of the feedback meeting and re-do that section – 1 min – Teacher gives quick feedback Kids First … New Mexico Wins Analysis Simulation, Round 3 – Action and Follow-Up IDENTIFY ROLES: Teacher, Leader ROLE PLAY ANALYSIS MEETING (4 min): – Assume you have completed the deep analysis – 4 min – Begin from the Action Step • Get teacher to make explicit action plan for the classroom • Schedule the follow-up DEBRIEF THE ROLE PLAY (2 min): – 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the leader – 1 min – Teacher gives feedback • What did the leader do most effectively? Where can s/he improve? MINI RE-DO (3 min): – Select the hardest part of the feedback meeting and re-do that section (2 min) – Teacher gives quick feedback (1 min) Kids First … New Mexico Wins Analysis Simulation, Round 4 – Action and Follow-Up SWITCH ROLES ROLE PLAY ANALYSIS MEETING (4 min): – Assume you have completed the deep analysis – 4 min – Begin from the Action Step • Get teacher to make explicit action plan for the classroom • Schedule the follow-up DEBRIEF THE ROLE PLAY (2 min): – 1 min – Review the Feedback Cheat Sheet – ID the top feedback to give the leader – 1 min – Teacher gives feedback • What did the leader do most effectively? Where can s/he improve? MINI RE-DO (3 min): – Select the hardest part of the feedback meeting and re-do that section (2 min) – Teacher gives quick feedback (1 min) Kids First … New Mexico Wins Role Play Reflection: What are your biggest takeaways for leading effective analysis meetings? What are the actions we need to take to improve the depth of our teacher action plans and the quality of the analysis meetings? Kids First … New Mexico Wins ACTION, REVISITED Kids First … New Mexico Wins Mr. Holland’s Opus What made the difference? How did Lou Russ finally learn to play the drum? What changed Mr. Holland’s attitude and actions? Kids First … New Mexico Wins Teachers plan lessons collaboratively based on data analysis Action Teachers write action plans to apply insights from analysis meetings Specific timeline for implementation Accountability – leaders check for alignment Engaged Students Kids First … New Mexico Wins Increasing Rigor Through the Lesson Tool found on pages 70-73 in your training manual TASK – Put a question mark next to activities you want to understand more deeply in order to implement effectively. – Put a star next to activities that sound particularly doable for you that you want to implement. – Double star the activities you want to implement first Kids First … New Mexico Wins Case Study: Douglass Street School 1. Did Krista win the bet? What percentage of teachers’ classes do you think achieved gains of 15 percentage points on the state test? 2. Based on your answers, name the biggest stumbling blocks to the school’s success. 3. Based on your answers, name the most important drivers of school improvement. Kids First … New Mexico Wins BREAK Kids First … New Mexico Wins Poll the Crowd What percentage of teachers’ classes do you think achieved gains of 15 percentage points on the state test? – 0-25% – 26-50% – 51-75% – 76-99% – 100% Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS (Interim, Transparent, Aligned, Re-Assess) ANALYSIS (Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep) ACTION (Action Plan, Ongoing, Accountability, Engaged Students) In a Data-Driven CULTURE (Leadership, PD, Calendar, Build by Borrowing) Kids First … New Mexico Wins Data-Driven Culture CORE IDEA: Any initiative that requires buy-in from the beginning will fail. When done well, data-driven instruction doesn’t require buy-in, it creates it. Kids First … New Mexico Wins Data & Culture Kids First … New Mexico Wins Introductory Professional Development: what (assessments) and how (analysis and action) Implementation Calendar: with time for assessments, analysis and action Data-Driven Culture Highly Active Leadership Team: teacher-leader data analysis meetings, maintain focus Ongoing Professional Development: aligned with calendar, flexible to adapt to student learning needs Build By Borrowing: identify and implement best practices from high-achieving teachers and schools Kids First … New Mexico Wins The Four Keys to Data-Driven Instruction: ASSESSMENTS (Interim, Transparent, Aligned, Re-Assess) ANALYSIS (Quick, User-Friendly, Teacher-owned, Test-in-hand, Deep) ACTION (Action Plan, Ongoing, Accountability, Engaged Students) In a Data-Driven CULTURE (Leadership, PD, Calendar, Build by Borrowing) Kids First … New Mexico Wins Data-Driven Instruction CORE IDEA: Data-driven instruction shifts the focus from the teaching to the learning. Kids First … New Mexico Wins Roadside Distractions The pursuit of total buy-in Reliance on poorly implemented professional learning communities (PLCs) Year-end assessment analysis Kids First … New Mexico Wins Eight Mistakes that Matter Inferior Interim Assessments Secretive Interim Assessments Delayed Results Separation of Teaching and Analysis Infrequent Assessments Ineffective Follow-Up Curriculum-Assessment Disconnect Not Making Time for Data Kids First … New Mexico Wins Pitfalls Kids First … New Mexico Wins Pitfalls Giving teachers too much data that they are not sure where they should focus Too many forms Not engaging teachers who teach content areas for which data is not available Kids First … New Mexico Wins Pitfalls Assumption that plans are being implemented A cycle is seen as an “event” instead of a process Compliance Syndrome Action Plans developed from one assessment Kids First … New Mexico Wins Pitfalls Analysis alone will not help teachers translate ideas into classroom practice Underinvesting in building the capacity of teachers to deepen their use of effective teaching strategies Disproportionate attention for students on the “bubble” Kids First … New Mexico Wins But… Kids First … New Mexico Wins Finding the Time Meaningful data analysis, identifying student learning problems by triangulating numerous data sources, analyzing student work samples, finding root causes for emerging problems, and creating a plan to tackle these problems takes time. We suggest 90 minutes of uninterrupted time. Kids First … New Mexico Wins Finding the Time Restructured or rescheduled time – Altering the school day, school year, or teaching schedules. Using time better – Using time that is already built in for other less important tasks or those that could be handled through electronic communication. Hiring subs or finding volunteers to free teachers from regular instructional time for interim assessment analysis. Kids First … New Mexico Wins Finding the Time Reallocating funds to provide teachers with stipends for staying after school or giving up other time. Stagger teacher and student schedules to have one group report early or later in the day. Kids First … New Mexico Wins DDI Implementation Rubric INDIVIDUAL: – Score your school on the rubric – Where are the weakest areas that could be addressed first? SCHOOL TEAM (OR SMALL GROUP): – Identify highest-leverage area of focus Kids First … New Mexico Wins Results Meeting Protocol Tool found on page 74 in the manual Originally set up by the Brazosport (Texas) School District Highlighted by Mike Schmoker in The Results Fieldbook Designed to be implemented in a 50 minute block (can be lengthened or shortened) Kids First … New Mexico Wins Results Meeting Protocol Identify roles: Timer, facilitator, recorder (2 minutes) Identify objective to focus on (3 minutes unless objective is given) What worked so far (5 minutes) [Or: What teaching strategies did you try so far?] Chief challenges (5 minutes) Brainstorm proposed solutions (10 minutes) Reflection: Feasibility of each idea (5 minutes) Consensus around best actions (15 minutes) Put in calendar: When will the tasks happen? When will the teaching happen? (10 minutes) Kids First … New Mexico Wins Brainstorming Protocol Go in order around the circle: each person has 30 seconds to share a proposal If you don’t have an idea, say “Pass.” No judgments should be made; if you like the idea, when it’s your turn simply say, “I would like to add to that idea by…” Even if 4-5 people pass in a row, keep going for the full brainstorming time. Kids First … New Mexico Wins Reflection Protocol 1 minute—silent personal/individual reflection on the list: what is doable and what isn’t for each person Go in order around the circle once: depending on size of group each person has 30-60 seconds to share their reflections. If a person doesn’t have a thought to share, say “Pass” and come back to him/her later. Kids First … New Mexico Wins Consensus and Calendar Guidelines Identify key actions from brainstorming that everyone will agree to implement. Make actions as specific as possible within the limited time. Identify key student-teacher guides or tasks needed to be done to be ready to teach. Identify who will do each task. Identify when each task will be done. Put date for re-teaching on calendar. Spend remaining time developing concrete elements of lesson plan: – Do Nows – Teacher guides (for example, what questions to ask the students or how to structure the activity) – Student guides, homework, and so on. Kids First … New Mexico Wins Key Tips to Making a Results Meeting Productive GET SPECIFIC to the question itself – “We can teach 10 lessons on this standard. What is the set of lessons these students need based on the data?” AVOID PHILOSOPHICAL DEBATES about theories of Math/Literacy; focus on the small, specific challenge of the moment. That’s where the change will begin! IF THE GROUP IS TOO LARGE, split into two groups. You’ll generate more ideas and you can share your conclusions/action plans at the end. Kids First … New Mexico Wins Practice: Results Meeting (25 min) Simulate parts of results team meeting – Topic: your biggest area of growth from the DDI rubric – Pick a major challenge that you think you will face as school leaders leading this – Focus on brainstorm, reflection, and begin consensus building Kids First … New Mexico Wins Reflection: What was effective about this protocol? What was challenging? Kids First … New Mexico Wins Visualizing Success Kids First … New Mexico Wins Final Reflection What is my personal commitment to making datadriven instruction work more effectively in my school/district? Kids First … New Mexico Wins Relentlessness Through a relentless commitment to observation, feedback, courageous conversations, and customized support, we are seeing LEAs within New Mexico improve outcomes for our children. Kids First … New Mexico Wins Kids First … New Mexico Wins Contact information Hannah Peria School Improvement Specialist Priority Schools Bureau [email protected] (505) 827-8005 Howard Oechsner School Improvement Specialist Priority Schools Bureau [email protected] (505) 827-6580 Kids First … New Mexico Wins
© Copyright 2024