1. No category

What To Expect From
The AzMERIT TEST
Arizona’s Measurement of Educational Readiness to Inform
Teaching
Joe O’Reilly
Mesa Public Schools
Today’s Topics
Ø AzMERIT basics
Ø AzMERIT items compared to AIMS items
Ø Moving to online testing
Ø How they will set what is passing?
Ø How will students do on the new test?
Ø How will this impact school letter grades and Title I
accountability?
DISCLAIMER
The information contained in this PowerPoint is the best information publically available at this time.
This was not created in conjunction with the Arizona Department of Education or the State Board of
Education.
AzMERIT Basics
Ø 
It is unique to Arizona, based on an AZ test blueprint
Ø 
Computer based (March 30th – May 8th) or on paper (April
14th –22nd); 40% of schools are online, 60% on paper.
Ø 
It replaces most of AIMS (not AIMS Science 4, 8, Biology)
Ø 
It measures mastery of the Arizona College & Career
Ready standards
Ø 
Arizona educators have reviewed and approved every item
used as being appropriate for Arizona students
AzMERIT Compared to AIMS
AzMERIT will be provided by AIR – American Institutes for Research
Comparing Item Formats
Ø AIMS was all multiple choice plus an essay
Ø AzMERIT is 80-85% multiple choice items and short
answer plus a writing assessment. The computer based
tests include technology enhanced items.
Ø AIMS multiple choice items had 4 options and you
chose the correct one
Ø AzMERIT multiple choice items often are four options
and you choose the correct one, but they are not limited
to 1 choice and there are open ended short answer
questions
AIMS Math Item
AzMERIT Style Math Item
Aims
http://demo.tds.airast.org/AIRAssessment/default.aspx?
disableJava=true&client=AIR&messages=AIR&acc=Content\\acc_default.json&config=Content\
AIMS Math Item
AzMERIT Style Math Item
AIMS Reading Item
Hot Text Item
“Hot Text” Reading Item
Hot Text
AZ is Moving To Online Testing
• 
• 
• 
• 
• 
• 
New test features – tools, reference sheets, cross off, etc.
New item types, not just multiple choice
More accommodations
Less expensive than paper tests
More engaging for students
Requires at least 1 computer per 10 students and adequate
bandwidth to send responses to AIR
•  But paper tests still will be offered for the life of the contract
Needed Technology Skills
Additional skills students must know
Score = Content Knowledge + Motivation + Familiarity With Format and Tasks
Typing
Drag and Drop
Highlight
Copy and paste
Scroll bar
Online graphic calculator
Zoom
Online protractor
Rotate
Drawing
How is your district ensuring that students have a chance to learn these skills?
But Can Online & Paper Tests Be
Comparable? (statistically, yes)
Ø Most test items on AzMERIT are identical multiple
choice items -- 84% to 100% of the test items are the
same.
Ø When an innovative item was not able to be translated
to paper, it was replaced with a comparable, more
traditional item.
Ø ADE/AIR will conduct a “mode comparability study” to
see how students did on paper compared to online and
place them on the same scale.
Challenge Ahead:
What Is Passing?
Ø 
The performance needed to pass or reach the
other three levels will be set this summer.
Ø 
Passing should indicate college and career
readiness. This will be a much tougher
standard than AIMS, which was measuring
tenth grade mastery.
ADE is Recommending Four
Performance Levels
It is proposed that there be no descriptors, just numbers.
 Level 4 –
 Level 3 –
 Level 2 –
 Level 1 –
You can read the proposed descriptors in the State Board Meeting agenda Items:
http://www.azed.gov/state-board-education/files/2015/03/board-agenda-and-materials-3.23.15revised.pdf , p79 – on
If Names Were Given To The Levels,
What Would You Want to See?
We asked educators, Board members and others about
what descriptions would sound best to them. So far we
have found…
  ighly Proficient, Proficient, Approaching Proficient,
H
Below Proficient - 70% ranked it 1 or 2
  dvanced, Proficient, Partially Proficient, and Not
A
Proficient - 47% ranked it 1 or 2
  ighest level, first passing level, highest non-passing
H
level, lowest level (these were the descriptors on the
PowerPoint for the State Board) - 5% ranked it 1 or 2
Want to Share What You Think
The Labels Should Be?
Go to: https://www.surveymonkey.com/s/
Suggest_AzMERIT_Performance_Descriptors
Survey closes Tuesday, April 21st
Each Level Will Have a
Performance Level Descriptor
Ø 
Each level will have an extended Performance
Level Description that describe the skills and
abilities that students must demonstrate for that
performance level.
Ø 
ADE is using volunteers to help with the
process. Have your educators volunteered to
serve on ADE committees?
How to volunteer for ADE committees:
http://www.azed.gov/assessment/2014/05/28/assessment-section-committee-application/
How Will They Set
The Passing Score?
The Bookmark Method
(the Reader’s Digest Condensed Version)
Ø Groups of teachers and curriculum specialists who really
know student performance at their grade level(s) will come
together.
Ø Questions for each grade/subject will be listed in a booklet
from hardest to easiest
Ø Teachers will place a bookmark below the item a student
who is just barely proficient has a 67% chance of getting it
right
Ø They then discuss with the group where they placed
bookmarks and come to a consensus placement, then
different groups come to a consensus
Ø Repeat for each performance level
The Committees Will Be Informed By
Comparisons To Other Tests
Proposed Standard Setting Timeline
Ø Teachers and curriculum experts set initial bookmarks
or standards July 13-17
Ø ADE will make recommendations to the State Board as
early as July 27th, but it may be at the August meeting
Ø The State Board of Education makes the final decision
about cut points between performance levels
How Will Students Do on AzMERIT?
Higher standards, which students may have only been
exposed to for a couple of years
Tougher Questions
Less familiarity with
the new item formats
Not so well
this year
In Other States the Percent Passing
Dropped With The New Test
In 22 “Smarter Balanced” States The
Percent Passing Will Drop
Smarter Balanced Assessment Estimated Percent Passing
The Percent Passing Will Drop …
…and That Is a Good Thing
Ø It shows expectations have changed from adequate
10th grade skills (AIMS) to prepared for the expectations
of post-secondary education or work
Ø In a world where jobs and opportunities can go to the
best prepared, we are ensuring that Arizona students
are being held to similar high standards as students
across the country
And Governor Ducey Would Agree – He
Just Called For More Rigorous Measures
“…we know we are below the mark in relation to other states in student
achievement, particularly in reading and math. And our own measurements of
success do not provide us the real picture. …
* Arizona tells our citizens that 70% of our 8th graders are proficient in reading,
when according to NAEP only 28% of our 8th graders are proficient at reading.
That’s an astounding difference—and 6% lower than the national average—we can
do better. Also, we are told our 8th-grade proficiency in math is 59% of kids; NAEP
says it’s 31% AND 3% lower than the national average. The disparity is smaller,
but still way too large.
But there are two major problems with the data:
* First, Arizona’s scores are too low – unacceptably low.
* Second, we’re giving false assurance to too many parents that their kids
are well prepared for life or college--when in fact they are not.
So we need to commit ourselves to achieving excellence.
(State Board Meeting, 3/23/2015)
…But It May Not Be Seen As A Good
Thing By Parents
Ø Many students who have been “Meeting The Standard”
for years will suddenly be labeled as “Not Proficient”
Ø The student’s ability has not changed. The quality of
the teaching has not changed. But it will seem that way
to parents.
Ø You should have a strategy to prepare parents before
results are released, and a strategy for responding after
the results are sent home
Your Challenge
Ø Lead your community in understanding low passing
rates are expected as we reset what students should
know and what schools should teach
Ø Frame it as a new, higher challenge to be met. Don’t
see it as a failure on the part of students, teachers or
parents.
Ø Ensure that your district improves performance over the
next few years to meet the new expectations
When will we get results?
Ø 
Parent reports will be released in late fall,
probably November
Ø 
In future years results will be available by the
end of May
What does AzMERIT mean for
students?
Ø 
High school students do not have to pass to
graduate, but in the future the results may be
incorporated into course grades.
Ø 
There is no consequence for younger
students this year and next using AzMERIT.
“Move on When Reading” performance
levels for third graders will be determined
using others methods.
What does AzMERIT mean for
schools?
Ø 
A bill was passed to suspend letter grades for two
years. All accountability data will be made public,
but a letter grade will not be awarded in 2014/15
and 2015/16.
Ø 
Arizona also uses the letter grade system as part
of the federal accountability system. We will still
have to meet 95% tested and ADE will still identify
Priority and Focus schools based on schools with
the lowest passing, lowest graduate rates and/or
largest achievement gap.
In Summary
Ø AzMERIT is a more challenging and more engaging
test than AIMS
Ø It has new question formats that require students to
show what they know, not just guess
Ø Paper and computer based exam results will be
comparable
Ø What it takes to reach each performance level will be
set this summer (ADE is using teacher volunteers)
Ø We expect passing rates to be about half of what we
had with AIMS – and that is a good thing, but a shock
to parents and the public
Ø There will be limited use of the test for accountability
purposes in the first two years (but after that…)
Your Homework
Ø Communicate with your parents & your community
Ø Make sure your community expects very different
results than we have had in the past
Ø Let them know it is because expectations have been
raised, not because your students and teachers are
doing worse than in the past
Ø They have a role in rising to those expectations
Ø Make sure your district has made the curriculum,
instruction and resource allocation decisions that will
improve the scores going forward
Questions?
[email protected]
480.472.0241