Classroom Assessment A Practical Guide for Educators by Craig A. Mertler Chapter 9 Subjective Test Items Introduction  Subjective test items are very versatile tools for assessing student learning.  Again, there is a tendency for educators to believe that their development is simple and straightforward, but writing high-quality items requires a lot of practice. General Characteristics of Subjective Test Items  Subjective test items: Those items that typically do not have a single correct response. • Subjective implies that subjective judgments of the scorer are an integral part of the scoring process. • Also known as “free-response,” “constructedresponse,” and “supply-type” items. • Include short-answer and essay items. • Require students to produce what they know, as opposed to merely recognizing the best response from a set of options. General Characteristics of Subjective Test Items  Subjective test items (continued) • Allow students to search for ideas and concepts that are not restricted to a predetermined set of responses. • Relatively easy to construct. • Time consuming for students to answer. • Can be time consuming and quite subjective to score, depending on the format. General Characteristics of Subjective Test Items General guidelines for writing subjective test items:  Creating Subjective Tests: General Guidelines 1. 2. 3. Subjective test items should cover important content and skills. The reading level and vocabulary of each item should be as elementary as possible. Each subjective item should be stated in an unambiguous manner, and confusing sentence structure and wording should be avoided. 4. Subjective items should not consist of verbatim statements or phrases lifted from the text. 5. Clues to the correct answer should not be provided. 6. Vary the types of items that appear on classroom tests. 7. Group items similar in format together so that each type appears in a separate section. 8. Each section should be preceded by clear directions. 9. Within each section, order the items from easiest to most difficult. 10. Although all item types will not appear on every test, they should be arranged in the following order: true-false, matching, short-answer, multiple-choice, and subjective items. 11. Provide adequate space for students to respond to each item. 12. Avoid splitting an item between two pages. Types of Subjective Test Items  Short-Answer Items • Require students to supply a word, short phrase, number, or other type of brief response. • Formatted as either questions (known as short-answer items) or incomplete sentences (known as completion or fill-in-the-blank items). • Virtually any completion item can be rewritten as a short-answer item (and vice versa). • Assess similar content and skills as multiple-choice items, but students must recall or create their answer (as opposed to simply identifying it from a list). Types of Subjective Test Items  Short-Answer Items (continued) • Most efficient for assessing lower-level thinking skills. • Relatively easy to construct . • Relatively easy to score (not as easy as objective items since responses must be handwritten). • Two important issues related to scoring:  Spelling and sentence structure.  Students must be informed if these will be assessed on the test. Types of Subjective Test Items  Short-Answer Items (continued) • Two additional issues related to scoring:  Students may provide a response that is correct but not what the teacher had intended.  Teachers must determine if these answers are the result of poor writing or lack of learning (misconceptions). • Variation—supplying a list of possible responses.  No longer a subjective item; merely a variation of a matching item. Types of Subjective Test Items  Short-Answer Items (continued) • Guidelines for development: Creating Short-Answer Items: General Guidelines 1. Short-answer items should be worded specifically and clearly. 2. In completion items, place the blank near or at the end of the statement. 3. Avoid copying statements verbatim. 4. Omit important words only. 5. Use only one or two blanks. 6. Standardize the length of blanks. Types of Subjective Test Items  Short-Answer Items (continued) • Advantages  Relatively easy to construct.  Easier than multiple-choice (no options needed).  Probability of guessing is substantially reduced. • Limitations  Primarily used only with lower-level cognitive skills.  Scoring can be made difficult by spelling errors and poor sentence structure. Types of Subjective Test Items  Essay Items • Questions or prompts that require students to write paragraphs or develop themes as responses. • Involve a wider variety of thinking skills (students must recall, select, organize, and apply). • Responses may range from a few sentences (restricted-response items) to several pages (extendedresponse items); range along a continuum… Restrictedresponse Extendedresponse Types of Subjective Test Items  Essay Items (continued) • Appropriately used to assess complex thinking skills (i.e., analysis, synthesis, and evaluation). • Sometimes used exclusively or as part of a larger performance assessment. • Not appropriately used to assess lower-level skills. • Scoring is complex and typically unreliable (due to influence of response characteristics). • Scoring can be quite time consuming. Types of Subjective Test Items  Essay Items (continued) • Scoring should resemble scoring of performance assessments, using holistic or analytic rubrics. • Scoring can be made more objective by:  Defining ahead of time what constitutes a correct answer.  Using a carefully defined rubric or checklist.  Deciding whether spelling, grammar, etc. will be scored.  Scoring student responses anonymously.  Scoring all responses to a given item before moving on to the second item. Types of Subjective Test Items  Essay Items (continued) • Guidelines for development: Creating Essay Items: General Guidelines 1. Essays should consist of the application of essential knowledge to new situations. 2. Essays should present a clear and focused task to students. 3. Specify for students the desired length, time limits, and evaluative criteria. 4. Develop a model response. Types of Subjective Test Items  Essay Items (continued) • Advantages  Can be used to elicit a wide variety of responses.  Students are permitted to create their own responses.  Probability of guessing is substantially reduced. • Limitations  Bluffing can be a problem.  Very time consuming to score.  Content coverage may be restricted. Validity and Reliability of Subjective Test Items   Validity • Must be able to answer the following:  Am I measuring what I intend to measure?  To what degree do I have confidence in the decisions I will make based on those measures? • Of primary interest is content evidence of validity. Reliability • Cannot be subjected to item analysis. • Subjective judgments will always be made (lower reliability) but reliability can be improved by following guidelines.