Download Report

CALIFORNIA
Assessment of Student Performance and Progress
Online Testing Appeals
User Guide
Winter/Spring 2015
Smarter Balanced Summative Assessments
for English Language Arts/Literacy and Mathematics
Published April 13, 2015
Prepared by the American Institutes for Research©
California customization prepared by Educational Testing Service®
© Smarter Balanced Assessment Consortium, 2014
Descriptions of the operation of the Test Delivery System, Test Information Distribution Engine, and related systems are
property of the American Institutes for Research® (AIR) and are used with permission of AIR.
CAASPP System
Contact Information
California Technical Assistance Center
2731 Systron Drive
Concord, CA 94518
Phone: 800-955-2954
Fax: 800-541-8455
E-mail: [email protected]
Web site: http://caaspp.org/
ii
CAASPP System
Table of Contents
Work with Appeals .............................................................................................................................................................. 1
Understanding Appeals......................................................................................................................................................... 1
User Roles ........................................................................................................................................................................ 1
Create an Appeal ................................................................................................................................................................ 2
Approving, Rejecting, or Retracting Appeals ........................................................................................................................ 5
To retract an appeal: ......................................................................................................................................................... 5
Creating Appeals through File Uploads ................................................................................................................................ 6
Understanding the Appeal Upload File Format ................................................................................................................. 6
Submitting an Appeal Upload File ..................................................................................................................................... 6
Resolving File Upload Errors ............................................................................................................................................. 7
View an Appeal ................................................................................................................................................................... 9
List of Figures
Figure 1. TOMS Dashboard ................................................................................................................................................. 2
Figure 2. Appeals Home screen .......................................................................................................................................... 2
Figure 3. Appeals tabs ......................................................................................................................................................... 3
Figure 4. Step 1 to create an appeal (request type) ............................................................................................................. 3
Figure 5. Step 2 to create an appeal (select criterion) ......................................................................................................... 4
Figure 6. Step 3 to create an appeal (enter criterion) .......................................................................................................... 4
Figure 7. View Appeals page ............................................................................................................................................... 4
Figure 8. Selection fields in the View Pending Appeals page .............................................................................................. 5
Figure 9. Retrieved Pending Appeals page ......................................................................................................................... 5
Figure 10. Sample Appeals Upload file ................................................................................................................................ 6
Figure 11. Upload Appeals screen ....................................................................................................................................... 7
Figure 12. Sample error messages ...................................................................................................................................... 7
Figure 13. TOMS Dashboard ............................................................................................................................................... 9
Figure 14. Appeals Home screen ........................................................................................................................................ 9
Figure 15. Appeals tabs ..................................................................................................................................................... 10
Figure 16. Step 1 to view an appeal (request type) ........................................................................................................... 10
Figure 17. Step 2 to view an appeal (select status) ........................................................................................................... 10
Figure 18. Step 3 to view an appeal (select filtering criterion) ............................................................................................. 11
Figure 19. Step 4 to view an appeal (additional values)..................................................................................................... 11
Figure 20. View Appeals screen ........................................................................................................................................ 11
List of Tables
Table 1.
Table 2.
Table 3.
Table 4:
Table 5.
Table 6.
Table 7.
Table 8.
User functions in the Appeals module .................................................................................................................... 1
Appeal types .......................................................................................................................................................... 3
Search criteria ........................................................................................................................................................ 4
Columns in the Appeals Upload File ...................................................................................................................... 6
File upload error icons and resolutions .................................................................................................................. 7
Columns in the “Error Messages” report ................................................................................................................ 8
Appeal status and descriptions ............................................................................................................................ 10
Fields in the View Appeals page .......................................................................................................................... 11
iii
CAASPP System
Acronyms and Initialisms Used in the Online Testing Appeals User Guide
CAASPP
CALPADS
CalTAC
CDE
LEA
SC
SSID
TA
TDS
TOMS
California Assessment of Student Performance and Progress
California Longitudinal Pupil Achievement Data System
California Technical Assistance Center
California Department of Education
local educational agency
CAASPP Test Site Coordinator
Statewide Student Identifier
Test Administrator
test delivery system
Test Operations Management System
iv
Work with Appeals | Understanding Appeals
Work with Appeals
In the normal flow of an online testing opportunity, a student takes a test in the test delivery system
(TDS) via the secure browser and then submits it. The TDS then forwards the test for scoring and
reporting.
For security incidents that result in a need to reset, reopen, invalidate, or restore individual online
student assessments, the request must be approved by the California Department of Education
(CDE). In most instances, an appeal will be submitted to address a test security breach or irregularity.
In some cases, an appeal may be submitted to address incidents that are not security related such as
allowing an extension of the grace period—the allowable period between the time a test is paused
and when the test is resumed—to reopen an assessment for a student who becomes ill, has paused
the test, and is unable to resume testing until after testing has expired. The local educational agency
(LEA) California Assessment of Student Performance and Progress (CAASPP) Coordinator or the
CAASPP Test Site Coordinator (SC) may submit appeals in the Test Operations Management System
(TOMS).
Appeals are a way of interrupting this normal flow when there are testing exceptions or circumstances
that interfere with the planned completion of a testing opportunity. For example, a student may need
to have a test reset, or a Test Administrator (TA) may need to invalidate a test because of a security
incident or impropriety. This section describes how you may view, create, and approve appeals.
All individual submitted appeals are available for retrieval and review by appropriately credentialed
users within a given organization. Your view of appeals will be restricted according to your user role
as established in TOMS.
Understanding Appeals
This section describes the types of appeals that may be created, and the statuses that may be
assigned to these appeals.
User Roles
Table 1 shows the three user roles in the Appeals module and the functions they are permitted to
perform.
Submit means that a user has the rights to request an appeal.
View means that a user can see a student’s appeal.
Edit means that a user has the rights to process, retract, or reject an appeal with a comment.
Table 1. User functions in the Appeals module
Role
LEA CAASPP Coordinator
CAASPP Test Site Coordinator
TA
April 2015
Submit



Function
View



Edit



2015 CAASPP Online Testing Appeals User Guide  1
Customization Copyright © 2015 by the California Department of Education
Create an Appeal | Understanding Appeals
Create an Appeal
Note: Appeals may be created by LEA CAASPP Coordinators and CAASPP Test Site
Coordinators only.
Open TOMS.
1.
2.
3.
Open an Internet browser.
Open TOMS by going to http://caaspp.org/ and selecting the [Test Operations Management
System] button to access the TOMS Sign In Web form.
When you have logged on and the TOMS Dashboard has opened, select the [Appeals] button
in the left navigation bar (shown in Figure 1).
Figure 1. TOMS Dashboard
Select the Function.
1.
The Appeals Home screen, shown in Figure 2, opens.
Figure 2. Appeals Home screen
2.
3.
Select the [Appeals] tab.
Select the tab to choose a function—either to create, view, or upload an appeal. Tabs are
shown in Figure 3.
2  2015 CAASPP Online Testing Appeals User Guide
Customization Copyright © 2015 by the California Department of Education
April 2015
Create an Appeal | Understanding Appeals
Figure 3. Appeals tabs
Create an Appeal.
1.
2.
Select the [Create Appeals] tab, which is indicated in Figure 3.
Select an appeal type to request from Submit a request to: drop-down list, shown in Figure 4.
Table 2 lists the available types of appeals.
Figure 4. Step 1 to create an appeal (request type)
Table 2. Appeal types
Type of Appeal
Invalidate Test
Reasons for Appeal
 Test security breach
 Test administered inconsistently
with the Online Test
Administration Manual (TAM)
 Student deliberately did not
attempt to respond appropriately
to items
Results of Approved Appeal
Reset Test
 A student with an individualized
Resetting a student’s test removes
that test from the system and
enables the student to start a new
test from the beginning.
education program or Section
504 plan started test without his
or her designated supports or
accommodations
 Correct test was not available
 Incorrect test originally opened
Reopen a Test
 Allows for a submitted or
expired test to be reopened
April 2015
While invalidated tests will be
scores and scores will be reported
(with a note that an irregularity
occurred), these scores will not be
included in percent-proficient
reporting for federal accountability.
Reopening a test allows a student
to access a test that has already
been expired or been submitted:
Expired—If the test has expired,
it reopens where student left off.
Submitted—If the test has been
submitted, it reopens at the last
page of the test; student can
review items in the current
segment, but cannot return to
previous segments
2015 CAASPP Online Testing Appeals User Guide  3
Customization Copyright © 2015 by the California Department of Education
Create an Appeal | Understanding Appeals
Type of Appeal
Restore Test (from Reset)
Reasons for Appeal
 A test was inadvertently or
inappropriately reset
Results of Approved Appeal
Grace Period Extension
 Student became ill and the test
Allows the student to review
previously answered questions
upon logging back on to the test
following expiration of the pause
rule period.
expired
 Technological difficulty resulted
in expiration of the test
 Unanticipated excused absence
or school closure resulted in
expiration of test(s)
A test that has been reset in error
can be restored to its previous
status and restarted where the
student left off.
Warning: Submit test reset and restore requests at least one day prior to the end of a
test window to allow students to complete their test opportunity.
3.
Select a criterion, listed in Table 3, by which to search for a student from the Search by dropdown list (shown in Figure 5).
Figure 5. Step 2 to create an appeal (select criterion)
Table 3. Search criteria
Field
Result ID
Session ID
SSID
4.
Description
Result ID is a unique internal identifier assigned by the system and
associated with a student’s test opportunity; for system use only. The
result ID is not the same as the SSID.
Session ID associated with the test opportunity.
Student’s Statewide Student Identifier (SSID) associated with the result.
Once the criterion is selected, enter the ID number for the field type, shown in Figure 6.
Figure 6. Step 3 to create an appeal (enter criterion)
5.
Select the [Search Student Results] button, indicated in Figure 6. The system displays the
results at the bottom of the View Appeals page (see Figure 7).
Figure 7. View Appeals page
4  2015 CAASPP Online Testing Appeals User Guide
Customization Copyright © 2015 by the California Department of Education
April 2015
Create an Appeal | Approving, Rejecting, or Retracting Appeals
6.
For each result for which you want to create an appeal, enter the reason in the text box, and
then select [Create]. The system displays a confirmation message.
Note: Please add sufficient detail in the Reason box to provide enough information
to facilitate a decision about the appropriateness of the type of appeal. For example,
if the student’s test expired due to illness, please indicate that in comment area.
You may retract any appeal that you have created within the system; see Approving, Rejecting, or
Retracting Appeals in the next section for additional details.
Approving, Rejecting, or Retracting Appeals
Appeals require CDE approval or rejection before the TDS can process them.
Caution: You cannot retract or delete an approved or rejected appeal.
To retract an appeal:
1.
Retrieve the desired student appeals by following these steps:
a. Select the Appeals tab, and then select View Pending Appeals. The View Pending
Appeals page appears (see Figure 8).
Figure 8. Selection fields in the View Pending Appeals page
b. From the drop-down lists and in the text field, select or create the search criteria.
c. Select Search. The system displays the found appeals at the bottom of the View Pending
Appeals page (see Figure 9).
Figure 9. Retrieved Pending Appeals page
2.
3.
4.
Optional: If needed, you may review your original reason for
initiating the appeal by selecting show comment.
Mark the checkbox for the appeals you want to retract. A
comment box and approval buttons appear.
As California users may only retract their created appeals,
select [Retract] (other options will be unavailable to select).
The system removes the retracted appeal from the list.
April 2015
2015 CAASPP Online Testing Appeals User Guide  5
Customization Copyright © 2015 by the California Department of Education
Create an Appeal | Creating Appeals through File Uploads
Creating Appeals through File Uploads
If you have many appeals to create, it may be easier to create them at once using a file upload
procedure. This task requires some familiarity with composing comma-separated value (CSV) files or
working with Microsoft Excel. The following sections describe how to compose the files and then
upload them to the appeals interface.
Understanding the Appeal Upload File Format
The upload file consists of a Microsoft Excel or CSV file with a dedicated header row and subsequent
data rows containing the detail for each of the appeals to be created. Table 4 describes the columns
presented in the upload file, and associated valid values.
Table 4: Columns in the Appeals Upload File
Column
Name
Type*
Description
Type of Appeal.
Valid Values
One of the following:
 Invalidate Test
 Reset Test
 Re-open Test
 Grace Period Extension
 Restore Test
Search
Type*
Student field to perform a search on.
One of the following:
 Result ID
 Session ID
 SSID
Search
Value*
Search value corresponding to the search
type.
Up to 1,000 alphanumeric characters. The
value must exist in the test delivery system
(TDS). For example, specifying a result ID of
123456 requires that this result ID exist in the
TDS. Note, too, that an SSID needs to be in
the format entered, i.e., CA-XXXXXXXXXX.
Up to 1,000 alphanumeric characters.
Reason*
Reason for submitting an appeal.
*Required field.
Figure 10 is an example of a simple upload file that reopens a test for a student with the SSID number
99999999 (which uses the format CA-XXXXXXXXXX).
Figure 10. Sample Appeals Upload file
Submitting an Appeal Upload File
1.
Select the [Appeals] tab, and then [Upload Appeals]. The Upload Appeals page, seen in
Figure 11, appears.
6  2015 CAASPP Online Testing Appeals User Guide
Customization Copyright © 2015 by the California Department of Education
April 2015
Create an Appeal | Creating Appeals through File Uploads
Figure 11. Upload Appeals screen
2.
3.
4.
5.
6.
Download one of the file templates by selecting [Download Excel Template] or [Download
CSV Template].
Open the file in a spreadsheet application or text editor, and add a row for each appeal you
want to add. Be sure to follow the guidelines in Table 4. Save the file on your computer.
To upload the file you created, select [Browse] and navigate to the upload file you created in
step 3.
Select [Upload File]. The system displays a preview of the uploaded file. Use this preview to
verify you uploaded the correct file and whether there are any errors present. For information
about resolving error messages, see Resolving File Upload Errors.
Select [Commit]. The system commits those records that do not have errors and sends a
confirmation e-mail.
Resolving File Upload Errors
During validation, the appeals system displays messages for the errors it detects, as in the example in
Figure 12.
Figure 12. Sample error messages
The first column in the table of error messages shows an icon that indicates the error’s severity.
Table 5 describes those icons and associated resolutions.
Table 5. File upload error icons and resolutions
Icon
April 2015
Description
Indicates the record causes the entire upload file
to fail.
Indicates a record that will not be processed due
to an error. The record containing the error will
be ignored; the remaining records will process.
Resolution
Select [Cancel] to abort the file upload. Make the
indicated correction, and upload the file again.
One of the following:
Select [Commit] to submit all records that have
no errors. Repair those records that have errors
and submit them in a separate file.
Select [Cancel] to abort the file upload. Make the
indicated corrections, and upload the file again.
2015 CAASPP Online Testing Appeals User Guide  7
Customization Copyright © 2015 by the California Department of Education
Create an Appeal | Creating Appeals through File Uploads
Icon
Description
Warning that a record contains an error; The
record will be accepted as entered.
Resolution
One of the following:
Select [Commit] to submit all records that have
warnings. Review the warning messages, repair
the records as necessary, and upload them
again in a separate file.
Select [Cancel] to abort the file upload. Make the
indicated correction, and upload the file again.
Table 6 describes the columns appearing in the “Error Messages” report.
Table 6. Columns in the “Error Messages” report
Column Name
Record Number
Field Name
Field Value
Validation Message
Description
Line in the upload file where the error occurred.
Name of the column in which the error occurred.
Value that caused the error.
Message describing the error.
8  2015 CAASPP Online Testing Appeals User Guide
Customization Copyright © 2015 by the California Department of Education
April 2015
View an Appeal | Creating Appeals through File Uploads
View an Appeal
Open TOMS.
1.
2.
3.
Open an Internet browser.
Open TOMS by going to http://caaspp.org/ and selecting the [Test Operations Management
System] button to access the TOMS Sign In Web form.
When you have logged on and the TOMS Dashboard has opened, select the [Appeals] button
in the left navigation bar (shown in Figure 13).
Figure 13. TOMS Dashboard
Select the Function.
1.
The Appeals Home screen, shown in Figure 14, opens.
Figure 14. Appeals Home screen
2.
3.
Select the [Appeals] tab.
Select the tab to choose a function—either to create, view, or upload an appeal. Tabs are
shown in Figure 15.
April 2015
2015 CAASPP Online Testing Appeals User Guide  9
Customization Copyright © 2015 by the California Department of Education
View an Appeal | Creating Appeals through File Uploads
Figure 15. Appeals tabs
View an Appeal.
1.
Select an appeal type to view from the Submit a request to: drop-down list, shown in Figure 16.
Table 2 lists the available types of appeals.
Figure 16. Step 1 to view an appeal (request type)
2.
Select an appeal status from the Select status: drop-down list, shown in Figure 17. Table 7 lists
the possible statuses that may be displayed following initiation of an appeal.
Figure 17. Step 2 to view an appeal (select status)
Note: An appeal’s status can change throughout its life cycle. Some of the statuses have a
corresponding reason or description that you may view in the Comments column of the View
Appeals page (see Figure 20).
Table 7. Appeal status and descriptions
Appeal Status
Error Occurred
Pending Approval
Processed
Rejected
Rejected by System
Requires Resubmission
Retracted
Submitted for Processing
Description of Status
A technical or system interruption occurred while the appeal was being
processed which caused an error. The appeal may need to be resubmitted.
Appeal is pending approval in the approver queue.
Appeal was successfully processed and the test opportunity has been updated.
Appeal was rejected by the reviewing party.
TDS was unable to process the appeal. This may occur when an incongruent
appeal request is submitted based upon the current student test status (e.g.
attempting to reopen an unexpired test).
This appeal must be resubmitted. This may occur when the appeal creation
process does not complete correctly due to a system or other technical
interruption; the appeal was logged.
Originator retracted the appeal.
Appeal submitted to TDS for processing.
10  2015 CAASPP Online Testing Appeals User Guide
Customization Copyright © 2015 by the California Department of Education
April 2015
View an Appeal | Creating Appeals through File Uploads
3.
Select a criterion from the Filter by drop-down list shown in Figure 18.
Figure 18. Step 3 to view an appeal (select filtering criterion)
4.
If you selected something other than “All” in step 3, enter a value for the search field you
selected.
Figure 19. Step 4 to view an appeal (additional values)
5.
6.
Select [Search]. The system displays the found appeals at the bottom of the View Appeals
screen (see Figure 20).
Optional: To review additional information about an appeal, select show comment, indicated in
Figure 20.
Figure 20. View Appeals screen
Table 8 describes the columns in the Retrieved Appeals screen.
Table 8. Fields in the View Appeals page
Field
Case Number
Request Type
Action
Status
Request Date
Comments
School IRN
SSID
Result ID
First Name
Last Name
Test
Test Opp #
April 2015
Description
ID number associated with the appeal.
One of the appeal types listed in Table 2.
A [Resubmit] button will appear here if the action is to resubmit the
appeal.
One of the status codes listed in Table 7.
Date appeal was created.
Comment or explanation added by user who created the appeal.
14-digit code that identifies county, district, and school (i.e., CDS
code).
Student’s SSID associated with the result.
Unique internal identifier assigned by the system and associated with
a student’s test opportunity; for system use only. The result ID is not
the same as the SSID.
Student’s first name.
Student’s last name.
Name of the test associated with the appeal.
Test opportunity associated with the result ID.
2015 CAASPP Online Testing Appeals User Guide  11
Customization Copyright © 2015 by the California Department of Education
View an Appeal | Creating Appeals through File Uploads
Field
Test Status
Description
Approved
Completed
Denied
Expired
Invalidated
Paused
TA approved the student for the session, but the
student has not yet started or resumed the test.
Student submitted the test for scoring.
TA denied the student entry into the session. If the
student attempts to enter the session again, this
status changes to Pending until the TA approves or
denies the student.
Student did not complete the opportunity, and cannot
resume the test because the test opportunity expired.
The test result was invalidated.
The test is currently paused as a result of one of the
following:
 Student or TA selected the Pause button on the
test.
 Student or TA idled for too long and the test
automatically paused.
 Test Administrator stopped the student’s session.
 Test Administrator paused the individual student’s
test.
 Student’s browser or computer shut down or
crashed.
Pending
Requested By
Requested By Email
Test Start Date
Date of Last Activity
Student is awaiting TA approval for a new test
opportunity.
Reported
Student’s score for the completed test has been
submitted.
Review
Student has answered all test items and is currently
reviewing answers before submission for scoring.
Scored
Test delivery system processed the student’s answers
on the test.
Started
Student started or resumed the test and is actively
testing.
Submitted
Test was submitted for quality assurance review and
scoring before it is sent downstream for reporting.
Suspended
Student is awaiting TA approval to resume a test
opportunity.
User who created the appeal.
Single sign-on username of the person who made the request.
Date student started the test opportunity.
The date the appeal was last acted upon.
12  2015 CAASPP Online Testing Appeals User Guide
Customization Copyright © 2015 by the California Department of Education
April 2015