1. No category

5/1/2015
Student Data & Accountability
Scorecards: Connections
Michigan Department of Education
MPAAA Conference
May 5, 2015
Welcome!
• Introductions
• Assessment  Accountability  Reporting Cycle
• Your Data Components Included in Accountability
Mich. Dept. Ed.
(MDE)
Div. Of
Accountability
Services (DAS)
Assessment
• How MDE/CEPI Reports Data & Results Back
Accountability
Systems
1
5/1/2015
The Flow of Data for Reporting
MSDS
MISchool
Data.org
Schools
Schools
Secure
Site
Communities
Fed.
Gov’t.
Accessing State Applications
Single Sign-On
(SSO)
EEM
Secure Site
MSDS
REP
ISD-Obtained Access
MISchoolData.org
Michigan Educ. Info.
System (MEIS)
GAD
SID
FID
2
5/1/2015
Accountability Landscape in Michigan
• “Accountability” exists in 2 primary categories:
Diagnostic reports that combine data points to issue an overall
outcome of performance.
These
include the Accountability Scorecards and the Top-to-Bottom
School Rankings
Transparency reports on individual variables reported in state
applications/systems.
These
include student counts, staff counts, dashboard displays, etc.
Our Focus Today
• Brief overview of Scorecards and Ranking
• What/how data elements used for Scorecards,
School Rankings
• How information is reported back to
administrators, schools, communities
3
5/1/2015
Accountability Scorecards
Scorecards are a 5 category, color-coded system combining performance
and compliance information for an overall school status.
Top-to-Bottom School Rankings
• A multi-year assessment performance system
that more measures school performance relative
other schools
• Ranks schools statewide on a 0-99 scale
• Still relevant in the work that MPAAA members
perform
4
5/1/2015
Scorecard Elements
Grouped by Year of Source Data
• Current Year
Participation Rates
Full Academic Year (FAY)
Students’ Demographics
(race/ethnicity, ED, EL, SWD,
grade)
Compliance Factors
Student Residency
Shared Educational Entities/
Specialized Shared Educational
Entities (SEEs/S2E2s)
• Prior Year
Attendance Rates
Graduation Rates
Teacher-Student Data Link (TSDL)
for Educator Evaluations
Intersection of Scorecard Elements & MSDS Collections
5
5/1/2015
School Year Time
Action: System
Element(s)
PRIOR
School Year
Sept. – Aug.
Collect: MSDS
TSDL Collection: TSDL
June
Collect: MSDS
EOY Collection: Attendance, Graduation, EL Exits
July – Mid-Sept. Collect: MSDS
CURRENT October
School Year
Late Jan./Feb.
SRM: Graduation appeals, EL Exits
Collect: MSDS
Collect checkpoint 1 of 3 for Full Academic Year (FAY)
Report: MI School Data
Prior Year Graduation Rates
February
Collect: MSDS
Collect checkpoint 2 of 3 for Full Academic Year (FAY)
4/20 – 4/29
Report: Secure Site
Collect: MSDS
Verify Enrollment & Demographics for WIDA, ACT, & WorkKeys
5/12 – 5/20
Report/Collect: Secure Site
WIDA Not Tested
5/18 – 6/15
Report: Secure Site
Collect: MSDS
•
•
•
Subgroups (race/ethnicity, ED, EL, SWD)
WIDA Access invalidation if not EL
WIDA Alternate Access invalidation if not both EL and SWD
Verify of Enrollment & Demographics for M-STEP & MI-Access
•
•
•
•
Full Academic Year (FAY), checkpoint 3 of 3
Subgroups (race/ethnicity, ED, EL, SWD)
Out of Level invalidation if enrolled/tested grades differ
MI-Access invalidation if not SWD
5/18 – 6/15
Report/Collect: Secure Site
M-STEP & MI-Access Not Tested
Late-Sept./Oct.
Report: Secure Site
Preliminary Scorecard
Late Fall
Report: mi.gov/amao
AMAOs (EL accountability)
A Closer Look at…
Scorecard/Accountability Elements
6
5/1/2015
Participation Rates
• Scorecard Description
Thresholds
• 95.00% or more students tested
• OR 2 or less students untested
Consequences
Schools’ overall scorecard color capped at YELLOW
• MSDS Connection: the denominator of participation rates are
determined using enrollment/exit records from the Spring General and
SRM Collections
Assessments
WIDA, ACT, & WorkKeys
M-STEP & MI-Access
SRM AS OF Deadline
3/27/2015
6/5/2015
SRM Cert. Deadline
4/29/2015
6/15/2015
Full Academic Year (FAY)
• Scorecard Description: Only results of FAY students are included in
scorecard percent proficient calculations.
• MSDS Connection: A student has full academic year (FAY) status
when they are present in the same entity for three snapshots: Fall
General Collection, Spring General Collection, and the enrollment and
demographic snapshot for that assessment.
Assessments
WIDA, ACT, & WorkKeys
M-STEP & MI-Access
SRM AS OF Deadline
3/27/2015
6/5/2015
SRM Cert. Deadline
4/29/2015
6/15/2015
*FAY students are those that are used for school ranking calculations.
7
5/1/2015
Demographic Subgroups
• Scorecard Description: Participation and proficiency results are
reported disaggregated by subgroups for each race/ethnicity,
Economically Disadvantaged (ED), English Learners (EL), and
Students With Disabilities (SWD).
• MSDS Connection: Students’ membership in subgroups is determined
using the enrollment and demographic snapshot for that assessment.
Assessments
WIDA, ACT, & WorkKeys
M-STEP & MI-Access
SRM AS OF Deadline
3/27/2015
6/5/2015
SRM Cert. Deadline
4/29/2015
6/15/2015
Demographic Subgroups
SWD special consideration
• Scorecard Description: Students who take the alternate assessment
(MI-Access) MUST have a Student With Disabilities (SWD) flag or
the test will NOT be scored.
• MSDS Connection: Students’ membership in the SWD subgroup is
determined using the enrollment and demographic snapshot for that
assessment.
Assessments
WIDA, ACT, & WorkKeys
M-STEP & MI-Access
SRM AS OF Deadline
3/27/2015
6/5/2015
SRM Cert. Deadline
4/29/2015
6/15/2015
8
5/1/2015
Demographic Subgroups
EL special consideration
• Scorecard Description: English Learner (EL) students in the US for 12
months or less may be exempted from M-STEP testing if they have
been flagged as EL and took the EL assessment (WIDA ACCESS).
• MSDS Connection: Students’ membership in the EL subgroup is
determined using the enrollment and demographic snapshot for that
assessment and their time in the US is determined using all of their
historical MSDS records.
Assessments
WIDA, ACT, & WorkKeys
M-STEP & MI-Access
SRM AS OF Deadline
3/27/2015
6/5/2015
SRM Cert. Deadline
4/29/2015
6/15/2015
Demographic Subgroups
EL & SWD special consideration
• Scorecard Description: EL students in the US for 12 months or less
who are also SWD may be exempted from M-STEP/MI-Access
testing if they have been flagged as BOTH EL AND SWD and took
the EL assessment (WIDA ACCESS or WIDA Alternate ACCESS).
• MSDS Connection: Students’ membership in the EL and SWD
subgroups are determined the enrollment and demographic snapshot
for that assessment and their time in the US is determined using all of
their historical MSDS records.
Assessments
WIDA, ACT, & WorkKeys
M-STEP & MI-Access
SRM AS OF Deadline
3/27/2015
6/5/2015
SRM Cert. Deadline
4/29/2015
6/15/2015
9
5/1/2015
Demographics: Out of Level Invalidation
• Scorecard Description: Students must be administered the grade level
assessment that matches their enrolled grade. If the tested grade and
enrolled grade do not match the test is flagged as “Out of Level” and
will be invalidated.
• MSDS Connection: Students’ enrolled grade is determined using the
enrollment and demographic snapshot for that assessment.
Assessments
WIDA, ACT, & WorkKeys
M-STEP & MI-Access
SRM AS OF Deadline
3/27/2015
6/5/2015
SRM Cert. Deadline
4/29/2015
6/15/2015
Attendance Rates
• Scorecard Description
Thresholds
Consequences
• 90.00% or more for full points
• OR a 10% improvement over the prior year for partial
points
Schools’ meeting neither threshold will have their overall
scorecard color capped at YELLOW
• MSDS Connection: Attendance rates are determined using [Days
Attended] and [Total Possible Attendance] from the Attendance
Component for students in grades K-11in the prior year’s End-of-Year
(EOY) collection.
10
5/1/2015
Graduation Rates
• Scorecard Description
Thresholds
• 80.00% or more for full points
• OR reduce the year-over-year rate gap by 25% for partial points
Consequences
Schools’ meeting neither threshold will have their overall
scorecard color capped at YELLOW.
• MSDS Connection:
Graduation cohorts are set when students are first reported in MSDS with a
[Grade or Setting] of 09.
Graduates determined using [District Exit Date] and [District Exit Status]
Graduation appeals are handled by SRM uploads generally occurring July
through Mid-September.
*Graduation rates are also worth 10% of a school ranking for those that receive them.
Teacher-Student Data Link (TSDL) for
Educator Evaluations
• Scorecard Description
Thresholds
Report TSDL for at least 95% of students
Consequences
Schools’ overall scorecard color capped at YELLOW if one or
both of the TSDL/Educator Effectiveness objectives are not met.
• MSDS Connection: TSDL is reported through MSDS. Any student expected to
continue in the fall, spring, or end-of year collections and not in grades 20 or 30
and not homeschooled or private schooled would be expected to have at least 1
course record submitted.
11
5/1/2015
Educator Effectiveness Labels for
Educator Evaluations
• Scorecard Description
Thresholds
Consequences
Report Educator Effectiveness for 100% of
educators/administrators with certain assignment codes.
Schools’ overall scorecard color capped at YELLOW if one
or both of the TSDL/Educator Effectiveness objectives are
not met.
• REP Connection: Educator Effectiveness labels are reported in the REP.
Any educator or administrator with assignment codes described on page
73 of the REP field descriptions guide here:
http://www.michigan.gov/documents/cepi/REP_Data_Descriptions_EOY_2015_470158_7.pdf
Compliance Factors
• Scorecard Description
Thresholds
Submission of the required reports:
• School Improvement Plan (SIP)
• EdYes! reports
• School Systems Review (SSR)
• Self-Assessment
• Interim Self-Assessment
• Program Evaluation
Consequences Schools’ overall scorecard color capped at YELLOW
• MSDS Connection: By state law, schools identified as the PEPE for
even one student reported in MSDS with a [Grade or Setting] code
00-12 (Kindergarten through grade 12) are required to fulfill state
accreditation system requirements.
12
5/1/2015
Student Residency
• Scorecard Description: Accurately reporting student residency codes
ensures homeschooled and nonpublic school students attending public
school for non-essential curriculum or advanced placement (AP)
courses are not included in the public school’s/district’s accountability
calculations.
• MSDS Connection: Residency codes are reported through MSDS.
Homeschooled or nonpublic school students should be reported with
one of the following Residency Codes:
04 (Non-Resident Nonpublic Student)
07 (Non-Resident Home-Schooled)
08 (Resident Nonpublic Student) or
15 (Resident Home-Schooled)
Shared Educational Entities/Specialized
Shared Educational Entities (SEEs/S2E2s)
• Scorecard Description: SEEs/S2E2s allow student results from multidistrict programs to be sent back to the sending districts for
accountability rather than having the results be attributed to the
district hosting the program.
• MSDS Connection:
SEEs are reported in MSDS using the proper entity code in the [School
Facility Number] characteristic
S2E2 are reported in MSDS using the [S2E2 Code] characteristic
13
5/1/2015
A Closer Look at…
Reporting Your Data Back to You
Next Steps
• After collecting your data from the various
systems, we calculate and aggregate various
accountability and transparency reports.
• These reports then sometimes get a district
“preview” or “appeal” window.
14
5/1/2015
Where Reports Come Back
• For Scorecards, we allow a preview/appeal window in the Secure Site prior to any public
releases.
 This has usually occurred in mid-summer.
 Will occur later this year due to changes in assessment.
• For School Rankings, those with the lowest school rankings are notified early of their status.
 Rankings also delayed this year.
• Neither Scorecards or Rankings are anticipated to be released publicly, only as diagnostic to
schools.
• For transparency reporting elements, such as student/staff counts, information is loaded into
MISchoolData.org after collections have been closed out and certified/audited.
Detailed Reports Provided Back to Schools
• For Scorecards, during and after a preview/appeal window in the
Secure Site, we provide student data files reflecting the demographics
and enrollment details, etc. that we captured from the MSDS and other
systems earlier in the year.
This is done so that districts are provided the ability to verify our calculations
and utilize data we calculate that they might find useful for instruction and
analysis (e.g. bottom 30% identification).
15
5/1/2015
What’s Next
• We are continuing the assessment – accountability reporting cycles
with the timelines described and anticipate beginning calculations for
Scorecards and Rankings (that will be provided only to schools and
districts for this assessment transition year).
• Most data transparency elements typically reported should be
available on MISchoolData.org currently. Some current year may be
reported there as well.
• www.MISchoolData.org (Demo, if time…)
Accountability Contact Information
Support regarding Scorecards can be reached at:
(877) 560-8378
OR
[email protected]
16