Building the Capacity of Schools, Districts and States to Implement School-wide PBIS Rob Horner University of Oregon www.pbis.org Goals • Define current status of PBIS implementation in the U.S. • Summarize features of Schools that are successful at implementing and sustaining PBIS with functional outcomes for students. • Define features of Districts that establish the capacity to implement PBIS at scales of social significance. • Define features of States that establish capacity to implement PBIS at scales of social significance Why SWPBIS? • The fundamental purpose of SWPBIS is to make schools more effective and equitable learning environments. Predictable Positive Consistent Safe SWPBIS: Building Effective Schools Main Messages • PBIS works. Effective (academic, behavior) Equitable (all students succeed) Efficient (time, cost) Main Message: Build Capcity • Schools • Implement with high fidelity at all three tiers • Expect more from your districts and states • District/ Region • Build Training, Coaching, Evaluation and Technical Expertise needed • Build capacity to sustain PBIS • Adapt to geography and size • States • • • • Provide functional leadership Implement with a full “slice” of the educational system Align initiatives Provide the data systems, training, coaching and evaluation needed School-wide Positive Behavioral Interventions and Supports (SWPBIS) • The social culture of a school matters. • A continuum of supports that begins with the whole school and extends to intensive, wraparound support for individual students and their families. • Effective practices with the systems needed for high fidelity and sustainability • Multiple tiers of intensity What is School-wide Positive Behavior Intervention and Support (PBIS)? • School-wide PBIS is: • A multi-tiered framework for establishing the social culture and behavioral supports needed for a school to achieve behavioral and academic outcomes for all students. • Evidence-based features of SWPBIS • • • • • • • Prevention Define and teach positive social expectations Acknowledge positive behavior Arrange consistent consequences for problem behavior On-going collection and use of data for decision-making Continuum of intensive, individual intervention supports. Implementation of the systems that support effective practices Establishing a Social Culture Common Language MEMBERSHIP Common Experience Common Vision/Values School-wide PBIS Culturally Equitable Academic & Social Competence OUTCOMES Culturally Valid Decision Making Culturally Relevant Support for Student Behavior SYSTEMS Culturally Knowledgeable Staff Behavior Outcomes Practices Systems Data School-wide behavior expectations School-wide Instruction on Expectations Team-based Universal Screening Class-wide Behavior expectations and routines Class-wide Instruction on Routines Supportive Leadership Progress Monitoring Academic Success Active Supervision Selection, Training, Coaching Implementation Fidelity Social Emotional Competence Effective Recognition Multi-tiered Support Corrective Consequences Policies and funding Functionbased Support Standardized Assessments Experimental Research on SWPBIS Bradshaw, C.P., Koth, C.W., Thornton, L.A., & Leaf, P.J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10(2), 100-115 Bradshaw, C.P., Koth, C.W., Bevans, K.B., Ialongo, N., & Leaf, P.J. (2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23(4), 462-473. Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133-148. Bradshaw, C.P., Reinke, W. M., Brown, L. D., Bevans, K.B., & Leaf, P.J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children, 31, 1-26. Bradshaw, C., Waasdorp, T., Leaf. P., (in press). Effects of School-wide positive behavioral interventions and supports on child behavior problems and adjustment. Pediatrics. Horner, R., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A., & Esperanza, J., (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133-145. Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42(8), 1-14. Ross, S. W., Endrulat, N. R., & Horner, R. H. (2012). Adult outcomes of school-wide positive behavior support. Journal of Positive Behavioral Interventions. 14(2) 118-128. Waasdorp, T., Bradshaw, C., & Leaf , P., (2012) The Impact of Schoolwide Positive Behavioral Interventions and Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial. Archive of Pediatric Adolescent Medicine. 2012;166(2):149-156 Bradshaw, Pas, Goldweber, Rosenberg, & Leaf, 2012 Number of Schools Implementing SWPBIS since 2000 January, 2014 19,960 Number of Schools Implementation SWPBIS (Tier I) by State January 2014 14 States with more than 500 schools Number of PBIS schools (Green) Implementing, (Red) measuring fidelity and (Blue) at Tier I fidelity by state >75% ------------------Connecticut Total number of schools using SWPBIS Florida Illinois Iowa Kentucky Total number of schools measuring fidelity Michigan Minnesota Schools at Tier I fidelity Missouri North Carolina Oregon South Carolina Vermont Wisconsin Building Capacity: Schools • Focus on “core features” that deliver valued outcomes. • PBIS is a framework for organizing practices that deliver core features. The core features should be documented to produce valued outcomes. Framework Practice Core Feature Valued Outcomes PBIS Selection and teaching of school-wide Expectations School-wide Expectations Improved Social and Academic Competence for Students Schools • Define and distinguish between • Practices • Core features • Valued outcomes Procedures  Core Features Effective Procedure Effective Procedure Effective Procedure Technology Core Features Science Values Valued Outcomes Effective Procedure Cultural/ Contextual Fit Implications • Certify, and Promote “core features” • Do not certify people • Do not certify manuals or programs • Measure “Core Features”… use for decision-making • Measure fidelity by assessing if “core feature” is in place • Provide examples of multiple practices (ways) to achieve core features • Focus on “contextual fit” variables that guide selection of effective practices. Building Capacity: Schools • Anticipate implementation error patterns SCHOOL-WIDE POSITIVE BEHAVIOR SUPPORT ~5% ~15% Primary Prevention: School-/ClassroomWide Systems for All Students, Staff, & Settings 27 Tertiary Prevention: Specialized Individualized Systems for Students with High-Risk Behavior Secondary Prevention: Specialized Group Systems for Students with At-Risk Behavior Main Ideas: 1. Invest in prevention first 2. Multiple tiers of support intensity 3. Early/rapid access to ~80% of Students support Math Remember that the multiple tiers of support refer to our SUPPORT not Students. Behavior Avoid creating a new disability labeling system. Health Reading Building Capacity: Schools • Measure “fidelity of implementation” • As a DV to assess implementation practices • As an IV to improve level of adoption. • Fidelity measures should focus on the “core features” of any practice. 1 1 1 2 2 2 3 3 3 4 4 4 4 4 Start Up Full Implementation 4 5 5 5 Start Up Part Implementation 5 6 6 6 7 7 7 03-Jun-05 08-Mar-05 08-Nov-04 03-Aug-04 01-Mar-04 01-Nov-03 01-Sep-03 06-Feb-04 07-Nov-03 11-Sep-03 05-Aug-03 05-Nov-03 01-Sep-03 21-Apr-03 28-Feb-03 31-Oct-02 12-Sep-02 01-Mar-05 24-Nov-04 12-Aug-04 02-Jun-05 01-Feb-05 22-Jan-04 23-Feb-04 05-Nov-03 05-Aug-03 Iowa Checklist 01-05, PK-6 % Fully & Partially Implemented 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 7 October 2014 School-wide PBIS Tiered Fidelity Inventory OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports 2014 Algozzine, B., Barrett, S., Eber, L., George, H., Horner, R., Lewis, T., Putnam, B., Swain-Bradway, J., McIntosh, K., & Sugai, G (2014). School-wide PBIS Tiered Fidelity Inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. www.pbis.org. PBIS Tiered Fidelity Inventory • Assesses PBIS implementation at all three tiers. Building Capacity: Schools • Focus on “efficiency” of practices • • • • Time Money Expertise of personnel Match with existing organizations/ systems. • 1. Efficiency for adoption • 2. Efficiency for sustained performance NOTE: Differences in Efficiency across Multiple Tiers of Support Time Cost of a Discipline Referral (Avg. 45 minutes per incident for student 30 min for Admin 15 min for Teacher) 1000 Referrals/yr 500 Hours 2000 Referrals/yr 1000 Hours 250 Hours 500 Hours Student Time 750 Hours 1500 Hours Totals 1500 Hours 3000 Hours Administrator Time Teacher Time T o ta l O ffic e D is c ip lin e R e fe rr Kennedy Middle School 1500 1200 900 600 300 0 Pre PBIS 95-96 Year 1 Year 2 96-97 97-98 School Years Year 3 98-99 What does a reduction of 850 office referrals and 25 suspensions mean? Kennedy Middle School  Savings in Administrative time  Savings in Student Instructional time  ODR = 15 min  Suspension = 45 min  ODR = 45 min  Suspension = 216 min  13,875 minutes  231 hours  43,650 minutes  728 hours  29, 8-hour days  121, 6-hour school days Building Capacity: Schools WHAT • Use Implementation Science • Implementation Drivers • Stages of Implementation • Improvement Cycles Interventions WHEN WHO HOW HOW Stages Teams Drivers Cycles Implementation Drivers An Active Implementation Framework Reliable Benefits Consistent Uses of Innovations Performance Assessment (fidelity) Systems Intervention Coaching Training Integrated & Integrated Compensatory & Selection Facilitative Administration Compensatory Leadership Drivers Technical Adaptive Decision Support Data System Stages of Implementation Implementation occurs in stages: • • • • • • Exploration Installation Initial Implementation Full Implementation Innovation Sustainability Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 2 – 4 Years Stages of Implementation Focus Should we do it Stage Exploration/ Adoption Decision regarding commitment to adopting the program/practices and supporting successful implementation. Installation Set up infrastructure so that successful implementation can take place and be supported. Establish team and data systems, conduct audit, develop plan. Initial Implementation Try out the practices, work out details, learn and improve before expanding to other contexts. Elaboration Expand the program/practices to other locations, individuals, times- adjust from learning in initial implementation. Continuous Improvement/ Regeneration Make it easier, more efficient. Embed within current practices. Getting it right Making it better Description Steve Goodman Improvement Cycles Main Messages • Sustained and High Fidelity Implementation of SWPBIS requires active District Support. • Especially for Tiers II and III ----------------------------------------------------------Student = unit of impact Schools = unit of analysis District = unit of implementation Visibility Funding Political Support Policy Leadership Team Active Coordination Training Coaching Behavioral Expertise Evaluation Local School/District Teams/Demonstrations Sugai et al., www.pbis.org Implications for Schools • Build commitment from Administration, Faculty, Students and Families that attention to social culture is important • Implement SCHOOL-WIDE, multi-tiered systems. • Build on what you already do well • Never stop doing what already works • Always implement the smallest change that produces the largest effect. • Never adopt something new without defining what you will STOP doing to create the resources needed for new adoption. • Measure fidelity of implementation as well as impact • Measure fidelity frequently, and use the information to guide improvement. • Report outcomes to families, faculty, community and administration. Implications for Schools • Expect more support from your district (or regional unit) • Initial personnel orientation • Data systems • Fidelity • Universal Screen and Progress Monitor • Standardized Assessments • Support for Tier II, and Tier III implementation • Role of school psychologist, counselor, social worker Tier II Tier III Increased structure Assessment: FBA, Mental Health, Academic, Physical Increased frequency of recognition/ feedback Comprehensive support plan Self-assessment Fidelity measures Link home and school Outcome measures Building Capacity: Districts/ Regions Building Capacity: Districts/ Regions • Three different conditions: • Stand alone district • Urban district • Clusters of rural / small districts Building Capacity: Districts/ Regions • Initial Implementation • • • • Build commitment (focus on valued outcomes) Establish leadership team Invest in Exemplars… but build capacity as you do this Invest in building district capacity to • • • • Implement with fidelity Implement with depth Implement with breadth (scale) Implement with sustainability • Full Implementation • Use of evaluation data • Iterative commitment events. Building Capacity: District/ Region • Real implementation means providing the technical assistance to establish durable systems. • • • • • • • Selection of Personnel Training Coaching Performance Feedback Data systems for effective decision-making Problem solving by teams and administration Effective engagement of families and community Position Description Faculty Evaluation Annual Orientation Building Capacity: District/ Region • Data systems • • • • Fidelity of implementation Universal Screening Progress Monitoring Standardized student outcomes • Stages of Implementation • • • • Exploration Installation Initial implementation Full implementation Measure District Capacity District Capacity Assessment www. Scalingup.org Building Capacity: States • Lead with clarity • Establish a leadership team with the goal of improving the capacity for implementation • Implement to change the full system • Focus on a slice of the full system as your implementation target • Guide adoption of practices • Define core features expected in schools • Align initiatives to avoid competition and conflict • Braid initiatives at the point of common budget • Provide that data systems needed for capacity development • Fidelity, and Impact at the school level • Implementation capacity at the district level • Invest in functional capacity for implementation • Training, Coaching, Evaluation, Technical Expertise Building Capacity: States • State Implementation Stages • Exploration • Initial Implementation (Exemplars). • Scaling paper (100-200 schools) • Evaluation data State • Reinvestment • State capacity • Evaluation data • Policy change District/ Region Schools Building Capacity: States • 1. Selection of effective practices • 2. Establish expectations • Schools should create a coherent social culture that promotes learning. • Students should graduate with academic AND social skills • 3. Establish iterative improvement system • Report on social culture of school • 4. Build the training, coaching and evaluation capacity at the state level. • 5. Align initiatives and expectations to promote efficiency and outcomes. Oregon Promising Practices • • Standard Operating Procedure: Promising Practices • • Promoting Educational Effectiveness in Oregon: • Standard Operating Procedure for Identifying and Implementing Educational Innovations • -------------------------------------------------------------------------• Practices may be (a) Standard, (b) Emerging, (c) Scaleworthy or (d) Not recommended Cascade of Competence State State Conferences Regions State Dept Trainers Local Content Specialists National Trainers Districts Evaluation/ Strategic Planning District/Regional Trainers Schools Local Coaching Alignment: Align at the common budget point Effective Procedure Core Features Effective Procedure Effective Procedure Core Features Valued Outcomes Effective Procedure Core Features Alignment Teacher Effectiveness PBIS Define and teach positive behavior Schoolwide Support 1. Expectations 2. Recognition 3. Consequences 4. Data System Appropriate Classroom Behavior Restorative Practices Consequences 1. Questions 2. Restore 3. Teach Building a Coherent Decision System • Building State capacity to gather information • Documenting outcomes for students • Documenting fidelity • Documenting capacity School-level decision-making Individual Decision-systems Cumulative Mean ODRs 12 Jennifer Frank, Kent McIntosh, Seth May 10 Cumulative Mean ODRs Per Month for 325+ Elementary Schools 08-09 8 0-1 6 2-5 6+ 4 2 0 Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun PBIS Tiered Fidelity Inventory • Assesses PBIS implementation at all three tiers. Assessing Capacity • www.sisep.org • State Capacity Assessment • District Capacity Assessment An Example Implementation Fidelity (SET) Elementary and Middle 2009-10 Implementation Fidelity (SET) By Factor for Elem and Middle 2009-10 Summary • Implementation at scale is possible • Consider the cluster of core features needed for scaling • Admin support, Technical capacity, 100-200 demonstrations • Small demonstrations may be necessary but insufficient • Build in system for adapting the program to fit the local context while retaining the core features. • Consider an implementation plan with established procedures for improving efficiency of implementation • Measure fidelity of implementation as a part of effective practice. • Sustained implementation requires continuous regeneration • Always emphasize, measure and report on valued outcomes Reflection • Schools • 1. Do we have a regular way to assess if we are using PBIS? • 2. Do we have a regular way to assess if we are benefiting students • 3. Do we have clear expectations for the District/Regions • District • 1. Does our district have the “capacity” to select and implement effective practices…. Like PBIS. • 2. Does our district have the capacity to sustain effective practices (data , training, coaching, evaluation) • State/ Commonwealth (build district capacity… • 1. Do we have a way to help districts/ state offices select effective practices and align federal/state initiatives? • 2. Do we have a way to Train/Coach/ Evaluate across the three types of districts/regions. • 3. Do we have “Decision Systems” that promote implementation and improvement.