Download Report

Hello, this is Karina Vanderbilt and I am a teacher in the Santa Fe Public School District. In this video, I
am going to describe how your individual observation rubric points are converted into the final score
that is incorporated into your summative evaluation.
Before we go into the detailed calculations of the observations, let’s take a look at the bigger picture.
This diagram maps the theory of action for the evaluation system, which is characterized by the
relationship between Teacher Effectiveness, Instructional Practice, and Student Achievement. This
model assumes that Teacher Effectiveness affects what instructional decisions a teacher uses as part of
her practice, which in turn impacts student achievement.
The overall goal of the evaluation is to recognize educator quality and effectiveness though the use of
various measures. Educator effectiveness is measured using the NMTeach Observation Rubric, Multiple
Measures, and Student Achievement Measures. Student achievement is assessed through different
measures such as teacher-created, formative tests; interim assessments; other non-cognitive measures;
and summative assessments including student-based assessments (SBA), Partnership for the Assessment
of Readiness for College and Careers (PARCC), and end-of-course exams (EOCs). We can see that the
most difficult diagram element to measure is the teacher’s instructional practice—the day-to- day
decision-making about curriculum, lesson planning, classroom management, and all the large and small
pieces that coalesce to create effective instruction. The NM Teach observation rubric attempts to
glimpse inside this black box to understand how teachers impact student learning.
This video focuses on this last measure, the NMTeach Observation Rubric. The use of this observation
rubric begins to provide a window into your instructional practice, though it is understood that, without
the use of something like daily diaries to glean the hundreds of decisions educators make on a minuteby-minute basis to facilitate student learning, instructional practice will always be difficult to completely
delineate.
On the first page of your summative evaluation, you will see a table that looks something like this. This is
an example evaluation for a teacher, who we will call Mrs. Smith. You can see that each of the
evaluation categories in the first column corresponds to one of the measures of teacher effectiveness
from the NMTeach Theory of Action. The second column lists the individual measures that specify the
categories. In Mrs. Smith’s sample case, there is only one measure for Student Achievement—SBA, and
one for Observations—Domains 2 and 3, which include creating an environment for learning and
teaching for learning. There are three measures included in the Multiple Measures portion of this
teacher’s evaluation—Domains 1 and 4, which include teacher preparation and professionalism; the
student survey; and teacher attendance.
The measures included in Mrs. Smith’s evaluation are chosen by the district, based on their plan for Mrs.
Smith’s group and level. You can find all district plans by following the link on the screen, which is also
listed in the description for this video.
The third column displays the number of possible points that Mrs. Smith could earn according to her
district’s plan. For all teachers, these possible points will sum to 200. The right-most column shows the
number of points earned for each category. Summed, these points equal the final summative score that
determines Mrs. Smith’s effectiveness rating.
With the overview in mind, we turn our focus to the observation portion of the summative evaluation.
However, before making this pivot, I’ll just point out that you will likely notice that the calculations
shown for computing Observations, Domains 2 and 3 are completely analogous for how the summative
score points earned are determined for the Multiple Measures, Domains 1 and 4. We can see that,
based on her observations, Mrs. Smith earned 41.17 points out of 65 possible for Domains 2 and 3. Let’s
see how those points were determined.
Depending on the option selected by your district, you are observed two or three times a year on four
domains—Domain 1: Preparation and Planning, Domain 2: Creating an Environment for Learning,
Domain 3: Teaching for Learning, and Domain 4: Professionalism.
These four domains are divided into two different categories. Domains 1 and 4 are included in the
Multiple Measures category, and Domains 2 and 3 are used for the observational piece of the
evaluation.
As you know, Domains 2 and 3 are each divided into five elements or indicators, A through E. I won’t go
into the details about each element here, but you can find all of the definitions by following the link on
the screen, which is also in the description. And after each classroom observation, you receive a score
from 1 to 5 on each element (1 being the lowest score and 5 being the highest). At the end of the year,
if you collect all of your individual rubric scores, your observations for the year might look something
like what is illustrated in this slide.
As you can see, Mrs. Smith was observed three times and was scored from one to five on each of the 10
elements.
Mrs. Smith’s final observation raw score is calculated by adding up the rubric points earned for each
observation and then finding the average across all three observations. For Mrs. Smith, we can see that
her total observation score for observation 1 was 30; for observation 2, it was 31; and for observation 3,
it was 34. We can find the overall observation score by summing those total scores and dividing by three
to find the average. Mrs. Smith’s average observation score is 31.67.
If Mrs. Smith had been observed only twice, the overall observation score would be an average of the
two sessions in which she had been observed. If scores were missing for a few elements, the total score
would be weighted based on the number of scores that were missing so that the total scores were
comparable. Finally, we compute the average of the weighted scores in order to determine an overall
average.
We know now that Mrs. Smith had an average of 31.67 observation rubric points. But what proportion
of the total points did she earn?
Because the maximum number of points that can be earned on each element is 5, there is a maximum
of 50 points possible across the 10 elements. Because Mrs. Smith earned 31.67 points over those 10
elements, she received 0.6334 or 63.34 percent of the total possible points.
Depending on how your district allocates total points, the number of possible observational points might
vary from one evaluation to another and from one district to another. However, the proportion of
possible summative observation points you earn always be equal to the proportion of rubric points that
you earn. Therefore, since Mrs. Smith earned 63.34 percent of her rubric points, she also earned 63.34
percent of the possible summative score points allocated to observations.
Based on Mrs. Smith’s district’s plan, she can earn up to 65 points for her observations. Because she
earned 63.34 percent of her possible rubric points, she earns 41.171 observational points.
Let’s look at a couple more teachers:
We can see that Ms. Garcia received the same number of rubric points as Mrs. Smith and therefore also
earns 63.34 percent of her total possible summative score points. However, based on her district plan
and she earns 63.34 percent of 70 possible points, or 44.338 points toward her summative score.
Mr. Vigil earned fewer points on his rubric than did Mrs. Smith or Ms. Garcia. He earned 23.50 points, or
47 percent of the possible 50 points. His district plan and graduated considerations determine that he
can earn a possible 65 points on his summative score from his observation, so he earns 30.55 points.
To summarize, the observation score points that you see on your summative form are determined based
on the proportion of observation rubric points that you earned during your observations this year.
We hope this video was helpful to you in understanding how observation points are calculated. Thank
you for watching! For more videos that illustrate how other aspects of your summative evaluation are
calculated, visit http://ped.state.nm.us/ped/NMTeach_EvaluationVideos.html