Minnesota State University Moorhead 2007-2008 Assessment Report Cover Sheet (An electronic version of this form can be accessed at http://www.mnstate.edu/assess) Note: All non-accredited programs are required to complete this form. Include Assessment Reporting Forms for each learning outcome assessed. Academic Program: BS Physics Department: Physics and Astronomy College: Social and Natural Sciences Date: Oct. 31, 2007 1. Name(s) of Department Assessment Coordinator and/or Assessment Committee Members Linda Winkler, Assessment coordinator The following faculty members contributed to assessment: Matthew Craig, Ananda Shastri, Linda Winkler 2. List of All Student Learning Outcomes. (List all outcomes, placing an asterisk (*) by the outcomes you are assessing this year.) 1.* Demonstrate basic knowledge of experimental and theoretical physics. 2.* Apply analytical thinking, mathematical analysis, and computational techniques to the solution of problems. 3. Reach both quantitative and qualitative conclusions from experimental and observational data. 4. Apply analytical thinking, instrumentation skills, and computer techniques to perform experiments. 5. Design and carry out an independent research. 3. Describe how your program has addressed the comments from the Student Learning Outcomes Assessment Committee during the past two academic years? (If you have made changes to your plan, file a revised Assessment Plan Cover Sheet and Assessment Planning Form(s).) Our assessment plan was completely revised during Spring of 2006 and passed through SLOAC. Suggestions from SLOAC were incorporated for rubrics in goals 2 and 5. Our first goals assessed were #1 ,#3, and #5 during academic year 2005-2006. 4. If you have received an Instructional Improvement Grant in the past two years, identify the outcomes on which the grant was based and provide a summary here of the results from your grant. Title of Project: Projects in Optics Based on assessment of Goals #1 and #5, we purchased a Newport Optics projects set so that our students in PHY 202, 305, and 306 will have more experiments involving optics. The original proposal is attached with more detail concerning the outcomes used to determine the need. Status of the project: the Newport optics set has been purchased and assembled. This semester (Fall 2007) is the first semester in which we have been using the equipment as part of PHY 305. We plan to continue with two more experiments in PHY 306. The department will assess the experimental outcomes during this semester of PHY 306. 5. Signatures Department Chair or Program Director Dean or Director Required Attachments: 1. Assessment Reporting Forms 2. Records of department meetings when Assessment Report was discussed and approved. Minnesota State University Moorhead 2007-2008 Assessment Reporting Form (An electronic version of this form can be accessed at http://www.mnstate.edu/assess) Instructions: Include this form for each student learning outcome assessed during the previous year. Include Assessment Report Cover Sheet. Academic Program: BS Physics 1. Learning outcome assessed (please include the number of the outcome to correspond with the list on the cover sheet). GOAL #1: Demonstrate basic knowledge of experimental and theoretical physics. 2. Describe assessment measure used for this learning outcome (attach instrument or rubric) ETS Major Field test in Physics 3. Expected/satisfactory student results (from assessment plan) Each student will achieve 80% or better in each subfield tested. 4. Actual results from the past year (attach additional information, if necessary) Cohort seniors Spring 2006 Student no. Program Total %-ile 1 BS Physics 25 2 BS Phys/Teach 25 3 BS Physics Math 40 Intro %-ile 25 10 25 Advanced %-ile 20 40 40 Cohort juniors Spring 2007 Student no. Program 1 BS Physics 2 BS Physics 3 BS Physics 4 BS Physics Total %-ile 20 75 50 25 Intro %-ile 40 67 35 15 Advanced %-ile 25 80 55 40 Cohort seniors Spring 2007 Student no. Program 1 BS Phys/Math 2 BS Physics 3 BS Phys/Teach Total %-ile 80 65 15 Intro %-ile 70 45 15 Advanced %-ile 80 75 15 NOTE: Scores on the major field test are reported as the total, and then broken into subscores for introductory topics (1st and 2nd year level) versus advanced topics (3rd and 4th year level). The scores reported are NOT scores indicating correct answers, but provide ranking among all students and institutions participating in the test. Thus a 40% score indicates the student performed as well as 40% of all students who took the test. In this way, the score is similar to the physics GRE, and so we have a way of estimating how well our students would do on the physics subject GRE. 5. Describe and explain available trend data for student performance on this outcome over the past several years. In other words, describe how the results of this measure have changed over the past several years. We have data for only two years, so cannot extract a trend. We will be continuing to give the mft to both junior and senior cohorts each year, as we believe the information provided is valuable for our department and our curriculum. We will have to re-evaluate what we consider acceptable for our program, since the scores do not represent percent of correct answers. 6. Proposed action in response to results. (Please note if improvements can be made with existing department resources. If improvements cannot be made with existing department resources, consider applying for an Instructional Improvement Grant.) We are still discussing amongst ourselves what to do in response to these scores. However, two changes are being implemented using current resources: A. Less take-home exams are being given in advanced classes. We believe students are pushing themselves less to learn material if they know they can use written resources and lots of time to answer questions on take-home exams. B. More encouragement is given to sophomores through juniors to serve as tutors/ SIs, and lab assistants. We are thinking of implementing a physics teaching internship , with the understanding that students learn the most while teaching other students. Minnesota State University Moorhead 2007-2008 Assessment Reporting Form (An electronic version of this form can be accessed at http://www.mnstate.edu/assess) Instructions: Include this form for each student learning outcome assessed during the previous year. Include Assessment Report Cover Sheet. Academic Program: BS Physics 1. Learning outcome assessed (please include the number of the outcome to correspond with the list on the cover sheet). GOAL 2:Apply analytical thinking, mathematical analysis, and computational techniques to the solution of problems. 2. Describe assessment measure used for this learning outcome (attach instrument or rubric) We assessed problem-solving skills in our upper division courses by looking at a representative problem from a take-home exam in Physics 430, Quantum Mechanics and a computer-based problem from the final in Physics 350, Introduction to Computational Methods in the Physical Sciences. Student solutions to each problem were rated in four different categories: • General Approach, which rated the completeness and appropriateness of the physical principles used in the solution to a problem; the scale in this category is 0 through 6. • Specific Application of Physics, which rated how general principles were applied to the specific problem; the scale in this category is 0 through 9. • Logical Progression, which rated the organization and coherence of the solutions; the scale in this category is 0 through 7. • Appropriate Mathematics, which rated the sophistication and appropriateness of the math used in the solution; the scale in this category is 0 through 5. The attached rubric describes the meaning of each rank in each category in detail. Within each category, a higher ranking represents better work. Three faculty members ranked each student’s work after student names had been removed from the student work. Problem-solving rubric attached. 3. Expected/satisfactory student results (from assessment plan) All students will score Acceptable or higher for each category on the rubric. 4. Actual results from the past year (attach additional information, if necessary) Mathematics: The charts below show student performance in Physics 430 in each of the categories. In each chart the vertical axis is the fraction of student work that fell into each rank, averaged over the three faculty who ranked the solutions. Computation: The problem we chose to look at in assessing the computational abilities of our students had two parts. One was manipulation of data that required the use of a computer, and the other was a description of the data before and after the manipulation. We found the rubric used in Physics 430 to be less helpful for assessing computer work. Instead, we placed the 10 student solutions into these groups: • Incorrect computer work, no description (1 student) • Correct computer work, no description (2 students) • Correct computer work, description present but limited (3 students) • Correct computer work, good description (4 students) As with problem solving skills, almost all of the students handled the technical part of the problem correctly, but some struggled with communicating the meaning of their results, or omitted a description altogether. 5. Describe and explain available trend data for student performance on this outcome over the past several years. In other words, describe how the results of this measure have changed over the past several years. None 6. Proposed action in response to results. (Please note if improvements can be made with existing department resources. If improvements cannot be made with existing department resources, consider applying for an Instructional Improvement Grant.) Mathematics: It is clear from the charts that most of our students use mathematics appropriately. The students who had difficulty with the mathematics were the same students who had difficulty identifying and applying the physics principles relevant to the problem. Although not separately rated, reviewing faculty felt that almost none of the students did a thorough job of explaining what they were doing and why. Even solutions that were well organized, mathematically correct, and logical contained very little prose describing what the students were thinking. Requiring the students to more fully explain their thinking might aid in identifying early in the semester which students are struggling and why. Computations: Again, students are weak at verbally communicating the meaning of the results and the structure of their solution. We will require studnets to document their work more fully. Assessment report, attached supplementals I. II. III. IV. Email exchanges regarding assessment. Proposal for Instructional Improvement Grant, Spring 2006 Charts regarding SLO-2, problem-solving Problem-solving rubric Minnesota State University Moorhead Instructional Improvement Grant Application Form Academic Program: BS Physics Amount Requested: $6850 (Minimum: $500; Maximum: $7500) TITLE=Projects in Optics Title of Project: 1. Indicate need for project, evidence of need, and student learning outcome addressed. Proposal must tie to results of departmental assessment projects and may include other evidence. During last review period (2004), we reached the conclusion from student performance on the Senior Project, that our graduating seniors had very little familiarity with modern physics and modern instrumentation. This has been reinforced by student comments on the exit interview that they believe our laboratory equipment is obsolete and in bad shape. This internal assessment was reinforced by our recent 6th year review consultant, Dr. Gubi Sudhakaran. His report spoke to the need for updating and revamping our physics laboratory facilities. He also spoke to the need for increasing the number of experiments our students perform in the PHY 305/306 experimental sequence. Assessment data for the 2006 report were gathered for the two following revised goals: GOAL 1. Demonstrate basic knowledge of experimental and theoretical physics. GOAL 5: Design and carry out independent research. These data, included in our report, verified the above statements. There are areas of undergraduate physics our students are not too familiar with. Also, our students need help in analyzing data and relating this data to physics principles. We continue to address these issues by improving the experimental offerings in our sophomore to upper-level physics classes. 2. Summarize proposal and how it will increased expected student performance on one or more student learning outcomes. We are requesting funds to purchase a Newport "Projects in Optics" educational kit. This kit comes packaged with professional-grade optical components and detailed instructions for 10 experiments for undergraduates. Eight of these projects develop students' understanding of the wave nature of light, light and its interaction with matter, and using light waves for image formation and image processing. The fact that this kit introduces 8 new experiments means that we will be able to offer more experiments in PHY 305/306. Both courses were recently upgraded from 1 credit courses to 2 credit courses. Exposure to more experiments means that our students will have better experimental skills. Because these experiments entail modern optics, our students will have a better grasp of waves and electromagnetism. 3. Describe what will be done and by whom. Include estimate of number of students served. The requested equipment will be used by faculty teaching PHY 202, PHY 305, PHY 306 during the next 4 years of assessment. We estimate 16 to 20 students in these courses will complete experiments in diffraction, polarization, and other light wave phenomena during the 2 years of assessment we focus on GOAL 4. PHY 202: 2 experiments per year with requested equipment PHY 305: 2 experiments per year with requested equipment PHY306: 2 experiments per year with requested equipment 4. Indicate how results will be assessed as part of the program’s next biennial report. PHY 306: Oral and written report evaluation by all faculty according to department approved rubrics (Spring 2007 and Spring 2008). 5. Signatures Department Chair Dean or Director Required Attachments: 1. Provide a detailed budget that indicates amount required for supplies, equipment, salaries, travel and/or reassigned time replacement costs and other necessary budget categories. If salaries are included, the amounts needed for fringe benefits must be included. Justify costs. For SLOAC Use: SLOAC Hearing Date: SLOAC Decision: Suggestions to Department (see attached) Recommend Not Recommend Table 1: General Approach 0 Nothing written 1 Physics approach is inappropriate. Successful solution is not possible 2 Physics approach is appropriate, but the manner of its application indicates a fundamental misunderstanding. 3 Physics approach is appropriate, but a wrong assertion is made as a serious misinterpretation of given information. 4 Physics approach is appropriate, but neglects one or more other principles necessary for the solution 5 Physics approach is appropriate and all necessary principles included, but errors are evident. 6 Physics approach is appropriate and all necessary principles included without any conceptual errors. Table 2: Specific Application of Physics 0 Nothing written. 1 Difficult to assess (GA#2). 2 Solution does not proceed past basic statement of concepts. 3 Vector/scalar confusion. 4 Specific equations are incomplete. 5 Confusion resolving vectors into components. 6a. Wrong variable substitution: Poor variable definition. 6b. Wrong variable substitution: Difficulty in translating to a mathematical representation. 7a. Careless use of coordinate system without a coordinate system defined. 7b. Careless use of coordinate system with a coordinate system defined. 8 Careless substitution of given information. 9a. Specific equations do not exhibit clear inconsistencies with the general approach, but hard to tell due to poor communication. 9b. Specific equations do not exhibit clear inconsistencies with the general approach and the solution is clear. Table 3: Logical Progression 0 Nothing written. 1 Not applicable - one step problem. 2 The use of equations appears haphazard and the solution unsuccessful. Student may not know how to combine equations. 3a. Solution is logical to a point, then an illogical jump is made. Student may abandon earlier physics claims to reach answer. 3b. Solution is somewhat logical, but frequent unnecessary steps are made. Student may abandon earlier physics claims to reach answer. 4 Solution is logical, but unfinished. Student may stop to avoid abandoning earlier physics claims. 5 Solution meanders successfully toward answer. 6 Solution progresses from goal to answer. 7 Solution progresses from general principles to answer. Table 4: Appropriate Mathematics 0 Nothing written 1 Solution terminates for no apparent reason 2a. When an obstacle happens, "math magic" or other unjustified relationships occurs 2b. When an obstacle happens, solution stops. 3 Solution violates rules of algebra, arithmetic, or calculus 4 Serious math errors 5a. Mathematics is correct, but numbers substituted at each step 5b. Mathematics is correct, but numbers substituted at last step.