Minnesota State University Moorhead 2007-2008 Assessment Report Cover Sheet (An electronic version of this form can be accessed at http://www.mnstate.edu/assess) Note: All non-accredited programs are required to complete this form. Include Assessment Reporting Forms for each learning outcome assessed. Academic Program: Bachelor of Science, Mass Communications Department: Mass Communications College: College of Business and Industry Date: October 19, 2007 1. Name(s) of Department Assessment Coordinator and/or Assessment Committee Members Jody Mattern, Mark Strand, Dan Johnson, Martin Grindeland, CT Hanson 2. List of All Student Learning Outcomes. (List all outcomes, placing an asterisk (*) by the outcomes you are assessing this year.) 1. Recognize the theoretical concepts in one's chosen emphasis. 2. Demonstrate an understanding of the ethics governing one's chosen emphasis. 3. Demonstrate verbal competency in one's chosen emphasis. 4. Demonstrate visual competency in one's chosen emphasis. 5. *Demonstrate ability to work collaboratively. 6. *Demonstrate an ability to create a professional document that meets industry standards. 7. *Demonstrate entry level competence inone's chosen emphasis through successful completion of an internship or other professional exeprience. 3. Describe how your program has addressed the comments from the Student Learning Outcomes Assessment Committee during the past two academic years? (If you have made changes to your plan, file a revised Assessment Plan Cover Sheet and Assessment Planning Form(s).) The assessment committee suggested that we provide more specific criteria for the different categories in the rubric. We revised the internship rubric by including category clarifications and adding two questions to better follow our student learning outcomes. We revised the teamwork rubric by adding category clarifications related to amount of time a specific behavior occurred (some of the time, none of the time, always, never, etc.). These changes were not in place for our assessment measures for 2006-07, but are in place for future assessments. 4. If you have received an Instructional Improvement Grant in the past two years, identify the outcomes on which the grant was based and provide a summary here of the results from your grant. Instructional Improvement Grant Report Jody Mattern Learning Outcome Addressed: Demonstrate an ability to create a professional document that meets industry standards. The instructional improvement grant provided funding for ten duty days for the development of a portfolio class. The description of the project from the grant application follows. We proposed development of a senior level class for advertising and public relations students called “Senior Portfolio.” The class has a dual function: 1) to act as a departmental assessment tool for the learning outcome related to creating professional documents, and 2) to help students develop a professional-level portfolio for use in job searches after graduation. During the summer of 2006, I conducted 11 interviews with advertising professionals at local, regional and national levels. I also reviewed several books for possible use as textbooks and class resource materials. After developing separate evaluation rubrics for copywriters, graphic designers, and art directors, I worked with the instructional technology department to develop special software for use in this class. With this software, I am able to upload student projects, pair them with a selected evaluator, and send them off in the form of an email. I also recruited a network of 25 evaluators to evaluate portfolio materials and maintained contact with them throughout the semester. During the semester, a total of 126 evaluations were solicited. After the semester has ended, the average score of all projects evaluated was tabulated.This score will be compared from year to year to track changes. I also asked evaluators to indicate overall weaknesses they saw in the student work. I will compile these comments and share with others in the department who teach courses in the advertising emphasis. For example, several evaluators have indicated that the typography is weak in many of the pieces they see. I have shared this with the two faculty members who teach layout and typography and they are exploring ways of including more work with typography in their courses. I have included some materials to document what is stated in this report including examples of the interview questions and answers, a list of evaluators, screen shots of the evaluation software, examples of evaluation rubrics, and a list of the projects sent out. 5. Signatures Department Chair or Program Director Dean or Director Required Attachments: 1. Assessment Reporting Forms 2. Records of department meetings when Assessment Report was discussed and approved. Minnesota State University Moorhead 2007-2008 Assessment Reporting Form (An electronic version of this form can be accessed at http://www.mnstate.edu/assess) Instructions: Include this form for each student learning outcome assessed during the previous year. Include Assessment Report Cover Sheet. Academic Program: Bachelor of Science, Mass Communications 1. Learning outcome assessed (please include the number of the outcome to correspond with the list on the cover sheet). 5. Demonstrate ability to work collaboratively. 2. Describe assessment measure used for this learning outcome (attach instrument or rubric) A teamwork rubric was administered to CT Hanson's classes. Approximately 70 students completed the evaluation of others in their group. Results were tabulated and averages obtained for each criteria measured. See Appendix D for results. 3. Expected/satisfactory student results (from assessment plan) The department expected 80% of its student interns to rate at least 4 on a 5-point scale. 4. Actual results from the past year (attach additional information, if necessary) The overall average score from fall 06 and spring 07 was 4.6, which exceeds our expectations. See Appendix D for a breakdown of results by question. 5. Describe and explain available trend data for student performance on this outcome over the past several years. In other words, describe how the results of this measure have changed over the past several years. Since this was the first year we tracked this measure, we will use this figure as a baseline to track trends in future years. 6. Proposed action in response to results. (Please note if improvements can be made with existing department resources. If improvements cannot be made with existing department resources, consider applying for an Instructional Improvement Grant.) We are satisfied with these results and have determined that no additional steps to improve scores for this learning outcome are needed other than to revise the rubric to better clarify categories. Minnesota State University Moorhead 2007-2008 Assessment Reporting Form (An electronic version of this form can be accessed at http://www.mnstate.edu/assess) Instructions: Include this form for each student learning outcome assessed during the previous year. Include Assessment Report Cover Sheet. Academic Program: Bachelor of Science, Mass Communications. 1. Learning outcome assessed (please include the number of the outcome to correspond with the list on the cover sheet). 6. Demonstrate an ability to create a professional document that meets industry standards 2. Describe assessment measure used for this learning outcome (attach instrument or rubric) An online rubric was developed that showed a portfolio piece and asked professionals to evaluate the piece compared to other junior-level portfolio pieces they may have evaluated in the past (see Attachment E for evaluation rubric). Two assessments were evaluated. One was the overall average of scores. The second was the overall number of "yes" answers when evaluators wee asked if this piece was of sufficient quality to be a portfolio piece. 3. Expected/satisfactory student results (from assessment plan) We expected 90% of our student work to receive average or better scores (3.0 or better). We expected that 50% of the pieces would be considered portfolio quality. 4. Actual results from the past year (attach additional information, if necessary) The average of all ad evalautions was 3.13. The percentage of ad portfolio pieces deemed to be of sufficient quality was 66%. The average of all graphic design evaluations was 3.63. The percentage of graphic design portfolio pieces deemed to be of sufficient quality for a portfolio was 74%. See Appendix E for a breadown of scores in all categories. 5. Describe and explain available trend data for student performance on this outcome over the past several years. In other words, describe how the results of this measure have changed over the past several years. Since this is the first year, we used this evaluation, these figures will serve as a baseline for future activities. 6. Proposed action in response to results. (Please note if improvements can be made with existing department resources. If improvements cannot be made with existing department resources, consider applying for an Instructional Improvement Grant.) This program was implemented as a pilot program. It is our goal to develop outside evaluation processes for other areas of emphasis so that each area has a consistent assessment procedure using outside evaluators. The instructor will pay closer attention to copywriting since that score was the one area where we failed to achieve an average or above score. Also, the comments indicated an overall weakness in typography. This fact has been shared with the faculty who teach design courses. We are also exploring the possibility of bringing in an expert in typography to speak to design students. Minnesota State University Moorhead 2007-2008 Assessment Reporting Form (An electronic version of this form can be accessed at http://www.mnstate.edu/assess) Instructions: Include this form for each student learning outcome assessed during the previous year. Include Assessment Report Cover Sheet. Academic Program: Bachelor of Science, Mass Communications 1. Learning outcome assessed (please include the number of the outcome to correspond with the list on the cover sheet). 7. Demonstrate entry level competence in one's chosen emphasis through successful completion of an internship or other professional experience. 2. Describe assessment measure used for this learning outcome (attach instrument or rubric) A 20-question evaluation rubric is completed by the intern's supervisor at the company where the intern is serving (see Attachment B). 3. Expected/satisfactory student results (from assessment plan) The department expects 80% of its student interns to rate at least 4 on a 5-point scale. 4. Actual results from the past year (attach additional information, if necessary) The overall average for all questions was 4.39 (see Attachement F for breakdown by question). None of the individual averages for questions fell below the 4.00 (excellent) average. 5. Describe and explain available trend data for student performance on this outcome over the past several years. In other words, describe how the results of this measure have changed over the past several years. Since this is the first year we have officially tracked these numbers, this year will serve as a baseline for future trend analysis. 6. Proposed action in response to results. (Please note if improvements can be made with existing department resources. If improvements cannot be made with existing department resources, consider applying for an Instructional Improvement Grant.) Since the average exceeded our departmental goal, no further action will be taken at this time except for a revision of the rubric to better clarify categories. Appendix D Teamwork Evaluation Fall/Spring 2006-07 Rating Scale: 5 = Superior 4 = Excellent 3 = Good 2 = Fair 1 = Poor Total Students Evaluated: 69 Question Average Team Oriented 4.7 Shared in Workload 4.5 Reliable 4.5 Cooperative 4.6 Supportive of Others 4.7 Overall Average 4.6 Appendix E: Results, Professional Documents MC 490 Ad Portfolio Development Evaluations Semester: Spring 2007 Rating Scale: 5 = Significantly Above Average 4 = Slightly Above Average 3 = Average 2 = Slightly Below Average 1 = Significantly Below Average Ads Number of Evaluations: 61 Question Average 1) Demonstrates Conceptual Thinking (creativity) 3.39 2) Layout 3.10 3) Headline/Body Copy 2.87 4) Overall Impact 3.15 5) In your opinion, is this piece of sufficient qualityto be included in a portfolio? Yes 66% No 34% Overall average for ads: 3.13 Graphic Design Number of Evaluations: 39 Question Average 1) Composition 4.31 2) Typography 3.26 3) Demonstrates Conceptual Thinking (creativity) 3.56 4) Use of Color 3.38 5) In your opinion, is this piece of sufficient quality to be included in a portfolio? Yes 74% No 26% Overall average for graphic design 3.63 Attachment F – Internship Results Intern Supervisor Evaluations Summer 2006 - Spring 2007Rating Scale: [5 Superior, 4 Excellent, 3 Average, 2 Fair, 1 Poor] No. Question Ave Mode Range N/A 1. Grasps professional situation 4.33 4 3-5 1 2. Displays technical knowledge 4.25 5 3-5 0 3. Demonstrates an ability to organize 4.31 4 3-5 0 4. Takes responsibility for tasks 4.42 5 2-5 0 5. Requires minimal supervision once the task is explained 4.38 5 3-5 1 6. Completes tasks in a professional manner 4.46 4 3-5 0 7. Meets deadlines 4.32 5 3-5 0 8. Demonstrates initiative 4.15 4 3-5 0 9. Accepts constructive criticism 4.39 5 2-5 0 10. Takes pride in assigned work 4.54 5 3-5 0 11. Uses good judgment 4.29 4 2-5 0 12. Exhibits adaptability 4.46 5 3-5 0 13. Demonstrates an ability to work as a member of a team 4.67 5 2-5 1 14. Exhibits maturity 4.31 4 3-5 0 15. Exhibits good listening skills 4.37 4 3-5 0 16. Exhibits good written communication skills 4.16 4 2-5 3 17. Exhibits good oral communication skills 4.31 4 2-5 0 18. Exhibits professional demeanor with clients 4.47 5 3-5 2 19. Respects workplace colleagues 4.68 5 4-5 0 20. Exhibits positive personal decorum 4.58 5 3-5 0 Total: 4.39 5 2-5 8 Assessment Committee Meeting Agenda September 21, 2007 2:15 p.m. 1. Discuss revisions 2006-2007 plan. 2. Discuss report for 2006-2007 3. Discuss 2007-2008 Plan a. Expand internship evaluation b. Writing pre-post test c. Evaluation day d. Exit survey Mass Communications Departmental Meeting October 5, 2007 Present: Mark Strand, chairperson: Martin Grindeland, Wayne Gudmundson, Shelton Gunaratne, Bill Hall, CT Hanson, Dan Johnson, Jody Mattern, Aaron Quanbeck, Regene Radniecki, Camilla Wilson 1) Jim Anderson, Assistant Director Admissions and Shawn Soderberg, Assistant Registrar joined our meeting to review MSUM’s Dragon Core requirement and eservices tools. 2) Assessment reports are due at the end of the month. Jody is coordinating our departmental assessment plan and also represents the College of Business and Industry on the university’s assessment committee. She distributed to the faculty the attached assessment materials and noted that for 2006-07 the department has met its goals for the following student learning outcomes: Demonstrate ability to work collaboratively. Demonstrate the ability to create a professional document that meets industry standards. Demonstrate entry-level competence in one’s chosen emphasis through successful completion of an internship or other professional experience. Following review and discussion of the attached 2007-2008 assessment plan, Mark moved to submit the plan to the university assessment committee. Martin seconded the motion. Motion carried. 3) Announcements: a) Bob Mikkelson will be in town and meeting with classes on October 8th and 9th. b) Faculty development and Dille grant applications are due October 31st. c) The Red River Interscholastic Press Association (RRIPA) fall workshop is scheduled for Thursday, Nov. 1st in the CMU Ballroom. d) Jody noted that she is reading Rebekah Nathan’s book “My Freshmen Year: What a Professor Learned” which focuses on actual experiences of students. Anyone wishing to read the book can borrow it from her. e) Recycled computer requests are due to the Student Technology Fee Committee by October 17th for first consideration, however, there are no macs available at this time. C.T. indicated that he would like to request a pc for use with the perception analyzer. Respectfully submitted, Deb Hval