Standard 11, Outcomes Assessment
Outcomes of the DVM program must be measured, analyzed, and considered to improve the program. Student
achievement during the pre-clinical and clinical curriculum and after graduation must be included in outcome
assessment. New graduates must have the basic scientific knowledge, skills, and values to provide entry-level
health care, independently, at the time of graduation.
The school/college must develop relevant measures and provide evidence that students/graduates have attained
the following competencies:
1.
2.
3.
4.
5.
6.
7.
8.
9.
comprehensive patient diagnosis (problem solving skills), appropriate use of clinical laboratory testing, and
record management
comprehensive treatment planning including patient referral when indicated
anesthesia and pain management, patient welfare
basic surgery skills, experience, and case management
basic medicine skills, experience and case management
emergency and intensive care case management
health promotion, disease prevention/biosecurity, zoonosis, and food safety
client communications and ethical conduct
critical analysis of new information and research findings relevant to veterinary medicine
Colleges must have annual and cumulative NAVLE passing percentages that statistically fall within the range of the
upper 95% of all accredited colleges.
Colleges must evaluate five-year trends in student attrition. Significant evidence of a rising trend in the annual
proportions of attrition must be explained and corrective remediation measures implemented and described.
Colleges must evaluate five-year trends in one-year post-graduation proportionate employment. Significant
evidence of a declining trend in the annual proportions of graduates achieving employment within one year of
graduation must be explained and corrective remediation measures implemented and described.
The following information is provided to explain why changes in use of the NAVLE pass
rate are being proposed and the underlying methodology:

A nonparametric method of examining the distribution of scores is advisable because the current NAVLE
pass rates are not normally distributed among accredited colleges (Shapiro-Wilk P-value < 0.00001)

There are two ways to construct an interval that captures 95% of the data from the 2010-2011 sample
 Based on the 2.5th and 97.5th percentiles
 Based on the 5th and 100th percentiles
The latter was chosen because there are colleges with 100% passing rates.

A college with passing rate of 77% (77 out of 100): 95% confidence interval = 67.5% - 84.8%
 2010-2011 NAVLE scores for accredited colleges 95% confidence interval is 88% - 100%
 This college would not have an acceptable passing rate

A college with passing rate of 80% (80 out of 100): 95% confidence interval = 70.8% - 87.3%



2010-2011 NAVLE scores for accredited colleges 95% confidence interval is 88% - 100%
This college would just barely miss having an acceptable passing rate
Advantages of this approach
 Uses actual accredited colleges’ data to establish a “reference range” for comparison with
candidate colleges’ NAVLE passing rates
 Does not require arbitrary cutoffs (e.g., 80%)
 Does not make assumptions about underlying probability distributions of accredited colleges’
NAVLE scores
 Uses defensible and replicable exact Clopper-Pearson binomial confidence intervals
 95% confidence intervals - while arbitrary - are widely used and accepted in statistical analyses
If the proposed changes to Standard 11are finalized in current form, the following
revisions to the self-study requirements listed in the COE manual, Section 12.11
will be revised as follows (additions are underscored, deletions struck).
Data to demonstrate outcomes of the educational and institutional program(s) may be collected by a
number of means including, but not limited to, surveys, interviews, focus groups, self-assessments,
third-party provider, information held by the college, and other. Where appropriate, the data must be
analyzed/summarized for brevity.
 Except for North American Veterinary Licensing Examination (NAVLE), the Council does not
assign numerical values to document levels of achievement for students in any of the outcome
delineators, but closely analyzes trends for the college. Trends that imply significant decrease(s) in
student achievement over a five-year period may imply deficiencies in the program. The trends are
used by the Council in its analysis of the compliance of the college with the Standards. In the case of
declining trends in the delineators, the college must provide an explanation for the decline(s), and must
provide a plan to reverse the trend(s).
12.11.1.
Student educational outcomes must include, but are not limited to:
12.11.1.a. NAVLE school score report data and passage rates over the past five years
(Table A),
Each college must submit a copy of the annual NAVLE School Score Report
with the AVMA-COE Interim Report each year. The Council on Education
expects that 80% or more of each college's graduating senior students sitting for
the NAVLE will have passed at the time of graduation. Colleges with recurring
passing percentages less than 80% the upper bound of an exact binomial 95%
confidence interval of the annual passing percentage will exceed the 5 th
percentile of NAVLE passing percentages of all accredited colleges each year. If
the confidence interval of the cumulative passing percentage for two successive
years fails to include the 5th percentile of all accredited colleges, the college will
be placed on Limited Accreditation. Colleges with passing percentages less than
80% If the exact binomial 95% confidence interval of the cumulative passing
percentage fails to include the 5th percentile of passing percentages of all
accredited colleges for four (4) successive years will, for cause, be placed on
Terminal Accreditation.
12.11.1.b. student attrition rates with reasons (Table B),
Each college must submit data on absolute attrition every year. The Council on
Education expects that a significantly increasing (positive) trend in absolute
12.11.2.
attrition from the college will be explained, and that the college will implement
steps for arresting the trend. Five-year annual proportions in absolute attrition
will be tested for positive trend using the exact Cochran-Armitage test;
statistical significance is defined as a two-sided probability value  0.05. If the
five-year trend remains positive for three successive years after corrective
remediation measures have been implemented, the Council will take action that
may include, but not be limited to, finding the college not in compliance with
Standard 11, Outcomes Assessment.
12.11.1.c. the learning objectives for each of the nine listed competencies, and a
summary of the analysis of evidence-based data collected for each of the nine
listed competencies used to ensure that graduates are prepared for entry level
practice (please note that a listing of core and elective blocks does not
constitute evidence of learning). Evidence of student learning outcomes for
clinical competencies must be obtained by direct measures. These may
include capstone experiences, student portfolios, standardized clinical
proficiency exams, or other evaluations of clinical performance based on
measurable and published program objectives. Indirect measures should not
be used as the sole determinants of clinical competency outcomes. Examples
include employer surveys and student course or rotation grades
12.11.1.d. employment rates of graduates (within one year of graduation) (Table C),
Each college must submit data on one-year post –graduate employment every
year. The Council on Education expects that a significantly declining
(negative) trend in proportionate employment from the college will be
explained, and that the college will implement steps for arresting the trend.
Five-year annual proportionate employment will be tested for positive trend
using the exact Cochran-Armitage test; statistical significance is defined as a
two-sided probability value  0.05. If the five-year trends remain negative for
three successive years, the Council will take action that may include, but not
be limited to, find the college to be not in compliance with Standard 11,
Outcomes Assessment.
12.11.1.e. assessments of graduating seniors; and assessments of alumni at some postgraduation point (for example, three and/or five years post-graduation)
assessing educational preparedness and employment satisfaction,
12.11.1.f. assessments of employers of graduates to determine satisfaction with the
graduates,
12.11.1.g. assessments of faculty (and other instructors, for example interns and
residents) related to such subjects as adequacy of clinical resources, facilities
and equipment, library and information resources, etc.; and preparedness of
students entering phases of education, and
12.11.1.h. additional assessment that might assist the college in benchmarking its
educational program.
Institutional outcomes.
12.11.2.a. Describe how the college evaluates progress in meeting its mission (for
example, benchmarking with other institutions, etc.).
12.11.2.b. Describe the adequacy of resources and organizational structure to meet the
educational purposes (dean should provide).
12.11.2.c. Describe outcomes assessed for college activities that are meaningful for the
overall educational process (for example, scholarly activity of the faculty,
faculty awards, faculty and staff perception of teaching resources, student
satisfaction with the educational program, teaching improvement benchmarks,
and others). If your program assesses other outcomes, briefly describe the
results.
12.11.2.d. Describe how outcomes findings are used by the college to improve the
educational program (give examples)
Table A – NAVLE
Year
Students taking exam(s)
Students passing
exam(s)
Average scores
Table B – Attrition
Entering
Class
Attrition*
Reason for Relative Attrition
Academic
Failure/Additional
Personal
Program
Absolute Attrition**
Number
Percentage
* Students that are either withdrawing from the program or moving to a different (earlier) class
** Students who leave and never return
Table C – Employment Rates
Graduating Class
# Graduates Who Received Employment or
Advanced Training Offers / # Completing This
Question, and (%)
Mean # Employment or Advanced
Training Offers Received