Academic Programs Appraisal Manual University Appraisal Committee (UAC), Implementation Plan, and Biennial Progress Report CONTENTS Contents ____________________________________________________________________ ii Editorial Note ________________________________________________________________ 3 Glossary ____________________________________________________________________ 4 Overview of Program Appraisals _________________________________________________ 6 Introduction _____________________________________________________________________ 6 Scope of the appraisal _____________________________________________________________ 6 Appraisals results _________________________________________________________________ 7 Coordination with accreditation reviews _______________________________________________ 7 Timeline ____________________________________________________________________ 8 University Appraisal Committee (UAC) ____________________________________________ 9 Role ____________________________________________________________________________ 9 Membership _____________________________________________________________________ 9 Meeting planification _____________________________________________________________ 10 UAC Report _____________________________________________________________________ 10 Implementation Plan _________________________________________________________ 12 Biennial Progress Report ______________________________________________________ 13 Appendix E: UAC Report Templates _____________________________________________ 14 ii Concordia University Academic Programs Appraisal Manual: Contents EDITORIAL NOTE This is the 5th edition (March 2012, revised January 2014) of the Academic Programs Appraisal Manual; it differs from the 3rd edition (May 2004) and the 4th edition (July 2005) in the following ways: Assessment of progress As part of the appraisal process, the Office of the Provost will initiate a Biennial Progress Report 24 months after the Appraisal Dossier has been completed. Departments and units responsible for the implementation of recommendations will be asked to report on the progress of recommendations made in the appraisal, utilizing updated program data from the Institutional Planning Office. Curriculum mapping The new appraisal guidelines introduce the use of curriculum mapping. Departments will be given the opportunity and the tools to conduct this part of the process in advance. Please refer to Appendix C for more information on curriculum mapping. Departmental data package In order to use quantitative data to support the analysis of programs, departments will be presented with a data package that includes a wide range of key performance indicators. Please refer to The DAC Report: Departmental data package for more information. Guidelines on length Word limits for each section of the Department Appraisal Committee Report (DAC) have been added to the templates in order to encourage departments to submit a brief, but complete, report. INDI programs (previously SIP) The Office of the Dean of Graduate Studies will appraise individualized Programs (INDI). Joint programs CREPUQ’s L’évaluation périodique des programmes d’études à l’Université Concordia1 (March 2003) mentions that Concordia should update the appraisal procedure concerning the programs offered in collaboration with other institutions. The last edition of the Manual (July 2005) did not consider joint programs; this document meets CREPUQ’s recommendation by including all existing joint programs in the assessment. The Office of the Dean of Graduate Studies will be directly involved in the evaluation of graduate joint programs. Membership of the DAC Since each department is responsible for the appraisal of its programs, it is not necessary to include a faculty member from another similar discipline as was previously asked in the 2004 and 2005 Aca-‐
demic Programs Appraisal Manuals. All academic programs appraisals are to be conducted using the 5th edition revised of the Manual. The Academic Planning and Priorities Committee (APPC) has approved these guidelines and any departure from them will be presented as an exception to APPC. 1
crepuq.qc.ca/documents/cvep/rapports/concordia/concordia.htm Concordia University Program Appraisal Manual: Editorial Note 3 GLOSSARY Academic Programs Summary Table This table is a summary of the Department Appraisal Committee’s (DAC) recommendations and imple-‐
mentation schedule. APPC Academic Planning and Priorities Committee. This Senate standing committee, chaired by the Provost and Vice-‐President, Academic Affairs, provides recommendations concerning academic strategies and implementation to Senate. APPC’s role in the appraisal is to review the process (including the Manual) and recommend updates or modifications as necessary. The Appraisal Coordinator will provide the committee with regular updates on the ongoing activities and with a summary of each Appraisal Dossier for the committee’s information. Appraisal Appraisals are periodic evaluations conducted by University-‐approved units and are a thorough, objec-‐
tive and constructive review of every academic program leading to a degree (Major, Specialization, Hon-‐
ours, Masters and Doctoral programs) or to an official denomination (Certificates, Graduate Diplomas). Appraisal Coordinator The Appraisal Coordinator implements the schedule of appraisals, provides support to departments, forwards relevant documentation, plans External Evaluator visits, and retains archives. The Coordinator acts in consultation with the Provost and Vice-‐President, Academic Affairs; the Vice-‐Provost, Teaching and Learning; the Faculty Deans; and the Dean of Graduate Studies. Appraisal Dossier This includes the DAC, UAC and EE Reports, as well as the Implementation Plan from the Dean(s). The executive summary of the latter is the only document that will be made available to the public. Associate Dean The appraisal is part of the portfolio of the Associate Dean, Academic Affairs from each Faculty, and the Associate Dean, Academic Affairs in the School of Graduate Studies. The Associate Deans have a gui-‐
dance and approval role. CREPUQ Conférence des recteurs et des principaux des universités du Québec. CREPUQ, among other responsibili-‐
ties, strives to facilitate communication between Universities and the use of collective tools for evalua-‐
tion and administration of the appraisal process. CREPUQ’s guidelines are established in the Policy of Quebec Universities for the Periodic Evaluation of Current Academic Programmes.2 Curriculum Mapping Curriculum mapping involves articulating the targeted learning outcomes that students achieve by virtue of completing an academic program, and tracing the curricular mechanisms put in place to reach these outcomes. 2
crepuq.qc.ca/documents/cvep/politiques/guide2004_en.htm 4 Concordia University Academic Programs Appraisal Manual: Glossary CVEP Commission de vérification de l’évaluation des programmes. CVEP works closely with CREPUQ to imple-‐
ment the Policy of Quebec Universities for the Periodic Evaluation of Current Academic Programmes. Department While some units responsible for managing programs are not officially called departments, this Manual uses the term to designate both departments and units. Some units may be joined for the appraisal (in consultation with the appropriate Faculty or School and the Vice-‐Provost, Teaching and Learning) to facil-‐
itate the process and avoid repetition. DAC Department Appraisal Committee. Formerly known as the Self-‐Appraisal Committee (SAC), the DAC is responsible for the preparation of the DAC Report. Membership is suggested in this Manual and con-‐
firmed by the Chair of the department undertaking the appraisal. Dean and Associate Dean (Academic Affairs), School of Graduate Studies References to these positions are included in parentheses, e.g. Dean(s) or Associate Dean(s), whenever graduate programs are involved in a department’s appraisal and action at the Dean or Associate Dean’s level is required. EE External Evaluators. Invited to evaluate a program by the Appraisal Coordinator in consultation with the DAC, UAC and Dean(s), the EE visit the department under appraisal, evaluate the recommendations in the DAC Report and send the EE Reports to the UAC. Faculty The term Faculty is used in reference to any of the four Faculties: Arts and Science; Engineering and Computer Science; Fine Arts; and the John Molson School of Business. IPO Institutional Planning Office. The IPO provides strategic information and analyses on issues related to the academic planning and mission of the University. Before the appraisal begins, the IPO prepares the De-‐
partmental data package for the use of the different committees involved in the process. UAC University Appraisal Committee. The UAC is responsible for reviewing the DAC and EE Reports and writ-‐
ing the UAC Report. The UAC includes faculty members, representatives from academic support units such as the IPO, the Office of the Vice-‐President, Research and Graduate Studies, and Student Services; it also includes representatives from the Concordia Student Union and/or Graduate Students’ Association. The UAC Report is sent to the Faculty Dean and to the Dean of Graduate Studies when applicable. VPRGS Vice-‐President, Research and Graduate Studies. The VPRGS will be asked to review the Dean of Graduate Studies’ section of the Implementation Plan (when graduate programs are part of the appraisal). VPTL Vice-‐Provost, Teaching and Learning. The VPTL drafts the calendar for appraisals, updates the Manual with the help of the Appraisal Coordinator, reviews the list of EE candidates with the help of the Associ-‐
ate Dean(s), and verifies compliance at every step of the process. Concordia University Academic Programs Appraisal Manual: Glossary 5 OVERVIEW OF PROGRAM APPRAISALS Introduction Academic programs appraisals are an accountability requirement of the Quebec government for all uni-‐
versities. Beyond this, appraisals are an important opportunity for departments to assess, redirect if nec-‐
essary, and clarify their aims and objectives. Departments and other stakeholders should also consider how to advance a program’s leadership position in its field. As an objective assessment of a program’s performance, appraisals enable departmental and University administrators to determine how well a program is doing academically, which is vital information for decision-‐making on how best to maintain and improve it. Departments are invited to analyze their programs in order to determine existing opportunities to achieve focus and distinctiveness, possibilities for future program niches, or complementarity with re-‐
spect to competing programs; departments must also consider the potential to engage in innovative pedagogy or research, improve the curricular structure, etc. Therefore, the appraisal dossier will include descriptive, analytical, and aspirational components: de-‐
partments, Faculties, and all other participants in the process must first describe the department and programs as they see them, then offer their analysis on that description, and finally offer recommenda-‐
tions on the means to achieve these aspirations. This Manual aims to facilitate the process by encouraging concision and attention to specific data and trends, as well as going beyond a simple description of programs and departments. Appraisals should be succinct, honest, and consistent with the stakeholders’ academic missions and long-‐range goals. With respect to provincial accountability, CREPUQ reviews the evaluation processes of all Quebec uni-‐
versities at regular intervals. On a random basis, CREPUQ also conducts detailed audits of a number of program reviews in each university to ensure that guidelines are followed. Scope of the appraisal Theoretically, every program leading to a degree (Major, Specialization, Honours, Master’s Degree, Doc-‐
toral Degree), as well as Certificates and Graduate Diplomas, must be appraised. In practice, programs with fewer than 10 currently registered students (five for graduate programs) should not be appraised. Evaluation of process and performance includes examining program pedagogy; curricular arrangements; targeted learning outcomes; rates (admission, retention, failure and withdrawal, and graduation rates); research and creation; and administrative processes. 6 Concordia University Academic Programs Appraisal Manual: Overview of Program Appraisals Appraisals results Appraisal Dossier 1.
Department Appraisal Committee (DAC) Report Evaluation of a department and its programs; departments are asked to submit a single report. 2.
External Evaluators’ (EE) Report Evaluation conducted by two consultants external to the University who are appointed by the Vice-‐Provost, Teaching and Learning (VPTL) and the Faculty Associate Dean, in consultation with the appraised department. External Evaluators are invited to submit a collaborative report if pos-‐
sible. 3.
University Appraisal Committee (UAC) Report Review of the DAC and EE Reports. The UAC Report includes a short description of the University administrative processes as they apply to the department and programs under appraisal. 3a. DAC Responses The DAC has the opportunity to respond to the EE and UAC Reports. 4.
Implementation Plan After reviewing the Reports and responses, the Dean(s) prepare an Implementation Plan, which includes recommendations and an implementation schedule. The Dean(s) may forward the draft of the Plan to the Provost and VPRGS for their feedback. 5.
Executive Summary The Executive Summary of the Implementation Plan is the only document made available to the public by the Appraisal Coordinator. Biennial Progress Report The status of the implemented recommendations will be assessed by using updated departmental da-‐
ta, which will be provided by the Institutional Planning Office 24 months after the final Appraisal Dossi-‐
er has been submitted. Coordination with accreditation reviews Appraisal and accreditation are distinct processes, but departments requiring professional accreditation reviews are encouraged to use all pertinent information gathered during the accreditation review for the appraisal. A comparison grid, available in Appendix A, will be used by the Appraisal Coordinator (in col-‐
laboration with the department responsible for the program) to determine which sections of the accredi-‐
tation report may be used for the appraisal. CREPUQ warns that “one must be careful to ensure that the objectives of the two procedures [accredita-‐
tion and appraisal] are not confused and that all the requirements of periodic evaluation are fulfilled.”3 Nonetheless, every attempt will be made to ensure that redundancy is minimized. 3
Guide for the Application of CREPUQ Policy Related to the Periodic Evaluation of Current Programmes (2004), CREPUQ, p. 11. crepuq.qc.ca/documents/cvep/politiques/guide2004_en.htm Concordia University Academic Programs Appraisal Manual: Overview of Program Appraisals 7 TIMELINE Deadline Activity Responsibility Preparatory work February 1 The Appraisal Coordinator contacts the Faculty Dean to initiate the appraisal. With the Faculty Dean’s permission, the Appraisal Coordinator sends all necessary pro-‐
February 15 cedural documentation to the department. The Appraisal Coordinator sends a request to CUPFA for the nomination of the DAC’s February 15 part-‐time faculty member. The Chair of the Department confirms the DAC membership to the Appraisal Coordina-‐
April 1 tor. June 1 Deadline to make requests for additional data to the Appraisal Coordinator. Academic programs appraisal Early October Submission of the list of EE candidates to the Office of the Provost Early October Suggested: completion of the first draft of the DAC Report November 15 Submission of the DAC Report January 15 Membership of the UAC is confirmed by the Office of the Provost Mid-‐February Completion of the EE visit to the appraised department March 1 Submission of the EE Report Mid-‐March DAC’s response to the EE Report if needed April 15 Submission of the UAC Report April 30 DAC’s response to the UAC Report if needed June 1 The Dean(s) send the Implementation Plan draft to the Provost and VPRGS for feedback August 31 Submission of the Dean(s)’s Implementation Plan August 31 Online availability of the Executive Summary of the Implementation Plan September 15 The summarized Appraisal Dossier is made available to APPC for their information Two years after the end of the academic programs appraisal: Biennial Progress Report September 1 Documentation on the Biennial Progress Report is sent to the department October 31 Submission of the Biennial Progress Report November 30 Dean(s) respond December 1 Biennial Progress Report and Dean’s Response(s) are sent to APPC for their information 8 Deliverable Appraisal Coordinator Departmental data Appraisal Coordinator package Appraisal Coordina-‐
tor/CUPFA Department DAC membership Department DAC DAC DAC Office of the Provost Appraisal Coordinator EE DAC UAC DAC Dean(s) Dean(s) Appraisal Coordinator Appraisal Coordinator EE candidates list DAC Report UAC membership EE Report DAC Response UAC Report DAC Response Implementation Plan Executive Summary Appraisal Dossier Appraisal Coordinator Department Dean(s) Appraisal Coordinator Biennial Progress Report Dean’s Response(s) Concordia University Academic Programs Appraisal Manual: Timeline UNIVERSITY APPRAISAL COMMITTEE (UAC) Role The role of the UAC is one of an unbiased assessor, neither prosecuting nor advocating any programs. The UAC is working in consultation with the DAC and the EE and is responsible for reviewing and com-‐
menting on the following: • DAC Report; • EE Report; and • The University’s administrative processes and their impact on the appraised program(s). Membership The Provost and Vice-‐President, Academic Affairs, is responsible for approving the membership of the UAC. The committee includes: • One representative selected by the Office of the Provost (acts as Chair); • One representative selected by the Office of the Vice-‐President, Research and Graduate Studies; • One representative selected by the Concordia Student Union; • One representative selected by the Graduate Students’ Association; • One faculty member (tenured), not directly involved in the appraisal, having already served as Chair or undergraduate program director; • One faculty member (tenured), not directly involved in the appraisal, having already served as Chair or graduate program director; • One faculty member (extended-‐term appointment, tenure-‐track or tenured); • One part-‐time faculty member (with at least 24 credits of seniority), selected by the Presi-‐
dent of the Concordia University Part-‐time Faculty Association; and • One representative from the Institutional Planning Office (non-‐voting member); • Appraisal Coordinator (non-‐voting member, acts as committee secretary). In addition, one representative from Student Services will provide consultative assistance to the UAC as needed. The members of the UAC should refrain from any informal contact related to appraisals with the Associ-‐
ate Dean(s), the DAC and the department until the deliberations of the UAC are complete. The UAC Chair will be available to comment on the appraisal in the relevant Faculty Council and the Council of the School of Graduate Studies. Concordia University Academic Programs Appraisal Manual: University Appraisal Committee (UAC) 9 Meeting planification All necessary documentation (Manual, DAC Report, strategic plans, etc.) will be forwarded to the UAC members before the first meeting. At the first meeting, the UAC Chair should: 1.
2.
Introduce the process; and Provide a rough timeline for completion. Between the first and second meeting, the UAC Chair should ask each UAC member to write a short list of questions or comments that they wish the EE to address during their visit. The list of questions will be compiled by the Appraisal Coordinator and sent back to the UAC for approval. At the second meeting, the UAC Chair should: 1.
2.
3.
Review the general situation of the department in the Faculty and the University. Review the salient points of the DAC Report, EE Report, and DAC Response to the EE Report. Discuss the recommendations to be included in the UAC Report. Following the second meeting, the UAC Chair will prepare a draft of the UAC Report and distribute it to the committee for feedback and modifications. Additional meetings may be scheduled as required by the UAC Chair. UAC Report The ideal length for the UAC Report is five pages or less. 1. Cover page 2. Administrative processes (University) The UAC must connect the University process to the reality of the department and programs un-‐
der appraisal. A more detailed list of topics may be found in Appendix E. a)
b)
c)
d)
Support from Faculty, School of Graduate Studies, and University Services, facilities and resources available Summary of the administrative structures Admission and retention at the University level 3. Summary and recommendations a) Response to the DAC Report, EE Report, and DAC Response b) Response to the Academic Programs Summary Table c) Numbered recommendations The UAC Chair signs the UAC Report, and sends both the printed and electronic copies to the Appraisal Coordinator, who sends the electronic version to the DAC should they wish to respond. 10 Concordia University Academic Programs Appraisal Manual: University Appraisal Committee (UAC) The DAC may prepare a written response to the UAC Report and send an electronic copy to the Appraisal Coordinator for consideration by the Dean(s). This response will be included in the Appraisal Dossier. After verifying that all of the requisite information is included, the Appraisal Coordinator forwards an electronic copy to: • Provost and Vice-‐President, Academic Affairs • Vice-‐President, Research and Graduate Studies • Dean(s) • Associate Dean(s) • DAC Concordia University Academic Programs Appraisal Manual: University Appraisal Committee (UAC) 11 IMPLEMENTATION PLAN The Dean(s), after reviewing the DAC, EE and UAC Reports, along with any response from the DAC, pre-‐
pares an Implementation Plan, which includes a schedule for implementing recommendations. In response to recommendations put forward by the DAC, EE, or the UAC, the Dean(s) comment on the recommendations from the Reports in regards to their feasibility and, if applicable, the rationale for pur-‐
suing or not pursuing a recommendation. The Dean(s) may discuss this draft with the Provost and VPRGS before finalizing the text; once finished, the Dean(s) sends a signed electronic copy of the Implementation Plan to the Appraisal Coordinator, who forwards electronic copies of the accepted finalized recommendations of the Implementation Plan to: • Provost and Vice-‐President, Academic Affairs • Vice-‐President, Research and Graduate Studies • Vice-‐Provost, Teaching and Learning • Associate Dean(s) • UAC Chair • DAC Chair The final Appraisal Dossier (electronic) is submitted to the Dean(s), who use the Implementation Plan as the basis for planning and making resource allocations, as well as to APPC for their information. 12 Concordia University Academic Programs Appraisal Manual: Biennial Progress Report BIENNIAL PROGRESS REPORT The Office of the Provost will notify the relevant department(s) to begin the work on the Biennial Pro-‐
gress Report no more than two years after the end of the appraisal process, considered as the moment when the Executive Summary of the Implementation Plan is made available online. The department and other units responsible for the implementation of recommendations will be given access to an updated data package and to a chart detailing the recommendations from the Implementa-‐
tion Plan as a basis for analysis. The format of the Biennial Progress Report will include the following for each recommendation listed in the Dean’s Implementation Plan: •
•
•
Initial implementation schedule and responsibility Status of the recommendation: target met, ongoing, delayed, or dropped If the recommendation is delayed or dropped, the unit identified as being responsible for the recommendation will be invited to provide a short rationale. The time frame for submitting the Biennial Progress Report is three months. The Appraisal Coordinator will include the completed Progress Report in the Appraisal Dossier and send a copy to APPC for their information. Concordia University Academic Programs Appraisal Manual: Implementation Plan 13 APPENDIX E: UAC REPORT TEMPLATES The following templates are used to write the UAC Report. Some variations will occur due to the nature of the department and programs under appraisal; therefore, an answer to every ques-‐
tion is neither mandatory nor expected. The UAC should remain attentive to providing a portrait that is as accurate as possible of the department’s situation and contribution and that responds to issues raised in the DAC and the EE Reports. For clarity and brevity’s sake, it is recommended that the UAC try as much as possible to limit its Report to five pages (excluding appendices) and to use a bullet-‐point or numbered list format whenever possible. This document includes the following templates: 1. Cover page 2. Administrative processes (University) a) Support from Faculty, School of Graduate Studies and University b) Services, facilities and resources available c) Summary of the administrative structures d) Admission and retention at the University level 3. Summary and recommendations a) Response to the DAC Report, EE Report, and DAC Response b) Response to the Academic Programs Summary Table c) Numbered recommendations 14 Concordia University Academic Programs Appraisal Manual: UAC Report Templates UAC Report 1. Cover page Department of UAC members and signatures Year of appraisal Name of UAC member Representing Signature Office of the Provost Office of the VPRGS Full-‐time faculty Full-‐time faculty Full-‐time faculty Part-‐time faculty Student (CSU) Student (GSA) Institutional Planning Office List of programs under appraisal List of programs excluded from the appraisal The UAC met on the following dates: This report was received by: Concordia University Academic Programs Appraisal Manual: UAC Report Templates On: 15 UAC Report 2. Administrative processes (University) The following items should be assessed within the context of how they impact the depart-‐
ment and the programs under appraisal. a) Support from Faculty, School of Graduate Studies, and University b)
Services, facilities and resources c)
Summary of the administrative structures d)
Admission and retention at the University level 16 Concordia University Academic Programs Appraisal Manual: UAC Report Templates UAC Report 3. Summary and recommendations The UAC is invited to include the following in its Report: a) Response to the DAC Report, EE Report, and DAC Response b) Response to the Academic Programs Summary Table c)
Numbered recommendations Concordia University Academic Programs Appraisal Manual: UAC Report Templates 17