ACCA National Conference September 27, 2013 New Orleans, LA Advanced CAS: How to utilize the new Counseling Standards. Laura A. Dean, Ph.D. M.J. Raleigh, Ph.D., NCC, LCPC CAS President Associate Professor College Student Affairs Administration University of Georgia [email protected] ACCA Representative to CAS Director, Counseling & Psychological Services University of North Carolina at Pembroke [email protected] How many of you… • Were in CAS I workshop earlier today? • Have executed a self assessment using any set of standards, CAS or institutional? • Have looked at the CAS Standards for at least one functional area? • Have participated in a department/program review process using CAS standards? • Have used CAS standards for other purposes? Participant Outcomes After this session, participants will be able to … • Describe the purpose of CAS and the counseling area standards • Explain how to perform a CAS self-study and assessment of Counseling Services • Discuss how IACS and CAS work in conjunction to improve mental health practice on campus. • Other goals that you have for this session? CAS Mission • The mission of the Council for the Advancement of Standards in Higher Education (CAS) is to promote the improvement of programs and services to enhance the quality of student learning and development. • CAS is a consortium of professional associations who work collaboratively to develop and promulgate standards and guidelines and to encourage self-assessment. CAS Overview • • • • Based in philosophy of self-assessment 40 member associations 43 sets of functional area standards Standards of practice informed by wide range of professionals and approved by consensus • Self-assessment guides (SAGs) for program selfstudy • 35 years of history; span across higher education Uses of CAS Standards • Credibility, accountability, improvement       Program & service improvement Measures of quality and effectiveness Measures of impact on learning and development Design of new programs & services Institutional self-studies Preparation for accreditation or program review • Staff development • Academic preparation Standards & Guidelines Understanding Standards and Guidelines Standards Guidelines Indispensible Requirements Clarify & Amplify Standards Achievable by any & all programs of quality Guide enhanced practice beyond essential function Appear in Bold type Appear in light-faced type Use verbs Must & Shall Use verbs Should & May General & Specialty Standards General Standards • Common across all functional areas Specialty Standards • Address issues specific to the functional area • Programs & services must develop, disseminate, implement, and regularly review their mission. • The primary mission of Counseling Services (CS) is to assist students in defining and accomplishing personal, academic, and career goals. Documentation to be Indexed Across the 14 Component Areas 14 Components for Index 1. Mission 8. Diversity 2. Program 9. Organization and Management 3. Leadership 10. Campus and External relations 4. Human Resources 11. Financial Resources 5. Ethics 12. Technology 6. Legal Responsibilities 13. Facilities and Equipment 7. Equity and Access 14. Assessment and Evaluation Types of Documentation • • • • Student Marketing Materials Program Documents Website Rhetoric and Materials Institutional Administrative Documents • Research, Assessment, and Evaluation Reports and Data • Annual Reports • Committee Meeting Minutes 9 Scope of Counseling Services • To accomplish the mission, the scope of CS must include o individual and group counseling services to students who may be experiencing psychological, behavioral, or learning difficulties o programming focused on the developmental needs of students to maximize their potential to benefit from the academic environment and experience o consultative services to the institution to help foster an environment supportive of the intellectual, emotional, spiritual, and physical development of students o advocacy for a healthy and diverse learning community o assessment services to identify and address student needs through appropriate services and referrals o crisis response, including threat assessment Why and How To Embark on a self study. CAS Fundamental Elements About Self-Assessment • • • • • • • • • • Internally driven Provides useful information Provides reasonably accurate information Is systematic & regular Is effective in terms of time, cost, etc. Charts quality program development & professionalism Develops a shared vision among constituents Guides solid staff development Relies on honesty with meticulous evaluation Assembles results into an action plan for improvement Component of Self Study 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. Mission Program Organization & Leadership Human Resources Ethics Law, Policy, & Governance Diversity, Equity, & Access Institutional & External Relations Financial Resources Technology Facilities & Equipment Assessment & Evaluation Decisions to be Made • Determine why using? – Assessment outcome to improve support and commitment to Student Affairs. – Used to support IACS accreditation process. • How much do you want to know? – Extensive study or snap shot? Discussions to have • Do you have other things on the agenda? – Enhance collaboration – Enhance consistency between campus sites – Need evidence to increase funding, facilities, staffing • Who is asking for the assessment? – What will be done with the information after it is completed? – How will the outside source use the assessment results? Balanced Assessment • Current attention focuses on outcomes o Shift from industrial economy focusing on inputs, consistency of product o Information economy focuses on outcomes, fit of product to consumer (Art Levine) • Important to know both sides: o Are our programs & services organized and run effectively to achieve the intended outcomes? o Are the intended outcomes (program & individual) achieved? o In counseling, are we focused on outcomes for individuals or for our overall program/service? CAS Self-Assessment Process Self-Assessment The CAS Perspective Ground Rules for Self-Assessment Management team committed to program improvement Decisions will be in the best interest of the institution Recognition & rewards for exceptional programs All team members valued & can contribute to decisions Mutual trust among participants Self-Study Overview • Purpose & scope of study is defined • Self-study team is identified & trained • Evaluative evidence is collected and reviewed • Self-study is conducted; ratings are compiled • Discrepancies are identified • Appropriate actions are determined • Special actions for program enhancement are recommended • Action plan is developed and communicated CAS Self-Assessment Process Six Steps of the Program Review Process 1. Assemble Your Team(s) 2. Educate Your Team(s) 3. Review Evidence & Conduct Rating 4. Complete Action Plan 5. Prepare a Descriptive Report 6. Close the Loop CAS Self-Assessment Process Step 1: Assemble Your Teams • Work Teams &/or Evaluation Teams • Size: – 3-5 members for a single functional area comprised of stakeholders including students – 8-10 members for a diverse department or division comprised of stakeholders including students • Composition: internal, external • Coordinator/Leader CAS Self-Assessment Process Step 2: Educate Your Teams • Watch CAS CD/PowerPoint Presentation or attend training session • Establish team ground rules • Review Standards and Guidelines • Discuss meaning of each standard • Establish team’s inter-rater reliability – Discuss, consider & set criteria – Build a common language, i.e. “well met”, “full met”, standards, guidelines • Encourage team discussion; expect disagreements; commit to consensual resolution • Adapt to institution Self-Assessment Guides (SAGs) Self-Assessment Guides • Translates CAS standards into an effective workbook format to enable self-assessment • Is used for self-assessment of a program’s strengths and weaknesses and the development of an action plan • Are organized into the fourteen components of the CAS Standards SAGs offer a ready format for evaluation • In most instances, there are multiple criterion statements for each standard • Each criterion measure focuses on a particular aspect of the standard, allowing raters to express more detailed and specific judgments Part 8. Institutional and External Relations - Criterion Measures 8.1 CS reaches out to relevant individuals, campus offices, and external agencies to: 8.1.1 establish, maintain, and promote effective relations 8.1.2 garner support and resources for programs and services as defined by the mission statement 8.1.3 disseminate information about the programs and services 8.1.4 coordinate and collaborate, where appropriate, in offering programs and services to meet the needs of students and promote achievement of student learning and development outcomes. 8.2 CS establishes and maintains close working relationship with the community mental health resources where adequate mental health resources are not available on campus. 8.3 CS has procedures for the referral of student who require counseling beyond its scope. Criterion Measure Rating Scale ND Not Done 1 Not Met 2 Minimally Met 3 Well Met 4 Fully Met NR Not Rated 26 27 CAS Self-Assessment Process Step 3: Review Evidence & Conduct Rating • Should rate all standards (in bold type), but sometimes a standard won’t apply (rarely) • Before process begins, decide whether to include guidelines or other measures that go beyond the standards – No criteria have been written for guidelines, but they can provide additional evidence of good practice – Review with team for further clarification of a standard – Write your own or disregard • Design process for compiling evidence • Gather evidence and conduct rating • Team uses scale based on established criteria • Individuals rate each and every criterion measure and then gather consensus • Identify quality indicators • Document all strengths & deficiencies • Complete the assessment & ratings Choosing an Assessment Method Match between learning and assessment: • Overall, your assessment method should be a reflection of the learning that you are seeking to assess • Is what you are asking students to do going to provide you with the evidence you need to make a statement about the learning that occurred? • Thinking about Bloom’s taxonomy, the different levels of thinking would require different assessment methods. • More in-depth thinking level = more in-depth assessment Choosing an Assessment Method The data: • How you plan to/need to use the data can often drive the whole assessment process • Think beyond your comfort level (chance to learn!) Considerations: • Direct vs. indirect • Quantitative vs. qualitative Measuring Learning Direct Methods Indirect Methods Any process employed to gather data which requires subjects to display their knowledge, behavior, or thought processes. Any process employed to gather data which asks subjects to reflect upon their knowledge, behaviors, or thought processes. Where on campus would you go or who would you consult with if you need mental health counseling, therapy or emotional support? I know where to go on campus if I need therapy, counseling and/or emotional support. Strongly agree Moderately agree Neither agree or disagree Moderately disagree Strongly disagree Quantitative Data • Summary of objective responses on a questionnaire or program evaluation • Statistics about use by students and various population subgroups • Needs assessments, follow-up studies, and self-study reports • Institutional research reports and fact books • Comparative or benchmark data at local, regional or national level Qualitative Data • Focus group information • Written summaries of responses to openended questions in interviews and on evaluations • Client satisfaction surveys, self-reports, and written comments, both solicited and unsolicited Evaluative Evidence PUBLISHED MATERIALS: – Brochures & other program information – Participation policies & procedures PROGRAM DOCUMENTS: – Mission statements; program purpose & philosophy statements – Brochures, pamphlets & related materials – Training manuals; policies & procedures manuals ADMINISTRATIVE DOCUMENTS: – Organization charts; student & staff profiles – Financial resource statements & budgets – Annual reports ACTIVITY REPORTS: – Curriculum vitae & resumes; professional activity – Outreach and consultation to other programs, departments, or community RESEARCH & EVALUATION DATA: – Needs assessments & self-studies – Program evaluation; graduate & follow-up studies – Assessment of therapy, individual and group therapy evaluation – Outreach and consultation evaluation CAS Self-Assessment Process Step 4: Complete Action Plan • Respond to the Overview Questions at the end of each rating section – Designed to stimulate summary thinking about overarching issues • Identify areas of program strength – Where excellent performance or accomplishment exceeds criterion and is viewed as excellent or exemplary • Identify areas of program weakness – Program shortcomings that fail to meet criterion measures and/or rating discrepancies among raters of two point or more – Viewed as unsatisfactory by at least one rater • Describe practices requiring follow-up – Note criterion measure numbers deemed less than satisfactory and describe practice shortcomings that need to be strengthened CAS Self-Assessment Process • Summarize & prioritize actions required for program to meet standards – List each criterion measure and/or related practices that the selfstudy process identified as being “Not Done,” “Unsatisfactory,” or where rater discrepancies were noted. – List specific actions identified in the self-study that require implementation – Prioritize the list by importance, need, and achievability of the desired change • Write program action plan for implementing program changes – Prepare a comprehensive action plan – Indentify resources (i.e., human, fiscal, physical) that are essential to program enhancement – Set dates by which specific actions are to be completed – Identify responsible parties to complete the action steps – Set tentative start-up date for initiating a subsequent self-study CAS Self-Assessment Process Step 5: Prepare a Descriptive Report • • • • Explain the mission, purpose, and philosophy of program Summarize the available data Recommend specific plans for action Include resources needed, dates for completion & identify responsible persons Step 6: Close the Loop • • • • • Special actions for program enhancement are recommended Action plan is communicated Specific actions are aligned with strategic plans Go through channels to request resources needed Thank members of the evaluation team Final Report Format • • • • Executive Summary Copy of Standards Summary of Initial Findings Copy of Original Action Plan Scores, strengths, & recommendations • List of Actions Taken with Completion Dates & Evidence of Completion • Lessons Learned to be Used for Next Program Review • Final Comments Translating Assessment Outcome to Learning Outcomes Standards & Outcomes • Standards serve a purpose in leading toward intentional outcomes • CAS identifies 6 learning and developmental outcome domains • Each functional area either directly influences, contributes to, or makes outcome possible • Some outcomes may be more salient to a program/service than others, but all should be on radar • In programming, it’s critically important to think first of desired outcomes & then design programs that will achieve the outcome • Standards, outcomes, assessment lead us to accountability Student Learning & Development: Part of the Program Programs and services must • assess relevant and desirable student learning and development • provide evidence of impact on outcomes • articulate contributions to or support of student learning and development in the domains not specifically assessed • articulate contributions to or support of student persistence and success • use evidence gathered through this process to create strategies for improvement of programs and services Outcome Domains • Knowledge Acquisition, Construction, Integration, and Application • Cognitive Complexity • Intrapersonal Development • Interpersonal Competence • Humanitarianism and Civic Engagement • Practical Competence Summary CAS is all about professionals coming together to promote… • Standards and guidelines to design quality programs & services • Targeted learning and developmental outcomes • Self-assessment of both programs/services and student outcomes A final thought • Don’t ignore the reality & usefulness of retrospective sense-making • Even if you have not been engaged in assessment formally, you have probably been using the principles in your work • Figure out how what you have been doing fits with what you want to be doing, and then fill in the rest of the plan • Just start… CAS Resources • CAS Professional Standards for Higher Education (7th Edition) (Blue Book) – Includes contextual statements and standards – Next book released in August 2012 • Self Assessment Guides CD (Version 4.2) (SAGs) – E-Learning Course – Conducting a Self-Assessment • Frameworks for Assessing Learning & Development Outcomes (2006) (FALDOs) – Available on CD • CAS Statement of Shared Ethical Principles • CAS Characteristics of Individual Excellence • Web Site: www.cas.edu Council for the Advancement of Standards in Higher Education www.cas.edu One Dupont Circle NW, Suite 300 Washington, DC 20036 Laura A. Dean, Ph.D. CAS President, ACCA Alternate Representative Associate Professor, College Student Affairs Administration University of Georgia [email protected] M.J. Raleigh, Ph.D., NCC, LPCP ACCA Representative to CAS Director, Counseling & Testing Center University of North Carolina at Pembroke [email protected]