Learning Analytics in Practice Dr. Rebecca Yates : Visiting Assistant Professor, Speech and Hearing Sciences Dr. Rob Petros : Assistant Professor, Chemistry Dr. Jason Simon : Associate VP Institutional Research and Effectiveness Dr. Mike Simmons : Senior Associate Director, CLEAR A Solution to the Efflux of STEM Majors • Redesign has increased enrollment from 145 to 192 (32%), while reducing actual class size from 145 to 96 ($120K+ in annual tuition). • Retention rate has increased dramatically ($250K?) # of students # of students retention registered receiving D, F, or W rate (%) 2013/2014* 39 377 90 2012/2013 83 285 71 2011/2012 83 291 71 2010/2011 87 248 65 2009/2010 77 218 65 * first year incorporating engaged learning activities (peer group learning) • Percent effort for teaching course has decreased dramatically Student Performance During Redesign 60 % of class 50 40 Results from ACS Standardized Final Exam NextGen launched podcasts introduced lecture eliminated 30 20 10 0 May-11 May-12 May-13 May-14† May-14‡ May-14* Above Natl. Average Bottom quartile † All students taking second semester with Petros ‡ Students that were in Petros' section for both OChem I and II * Students that were in Petros' section for only OChem II Assigning Teams and Assessing Soft Skills www.catme.org (Comprehensive Assessment of Team Member Effectiveness) Becky Yates – Course improvement Goal 2. The student will understand normal communication and major communication disorders • 2.1 The student will demonstrate knowledge of normal modes of communication • 2.1.1 Define communication, speech, hearing and language • Question: What is the definition of hearing? • 2.1.2 Differentiate terminology to describe atypical communication, speech, hearing and language • Question: What are the two major groups of sounds produced for speech? Becky Yates – Course improvement Utilize item Analysis • Used item bank to revise questions, adjust scoring, and improve assessment • Assess higher level thinking by classifying questions as low, medium, and hard to ensure exam question coverage allows for the exam to discriminates students learning effectively Becky Yates – Course improvement Provide Feedback to Student to Improve Performance • Enabled Secure Exam Review • Released Customized Reports to Students • Displayed Exam Score Upon Exit Making Campus Data Easy • Result of an 18 month long process involving Faculty, Staff, and Associate Deans who wanted to reduce confusion around where to find data and promote better access to data for internal UNT purposes • A one stop site for data regarding enrollment, retention, degrees, etc. from multiple sources across campus • A FERPA protected community resource and access is granted after training completion and FERPA permission authentication – no exceptions • NOT designed for, nor should be used for, Doctoral dissertation sampling or panel creation • Users are audited for FERPA compliance Student File Structures: 3 Tabs included in each Excel workbook: 1. Variable names and definitions • Consistent with multiple sources 2. Data with filtering capability • Sort on variables to create lists or view individuals 3. Pivot table template (with links to video tutorials) • Obtain aggregated and summary data • Logically organized with the user in mind • Meta-tagging for optimal categorization and understanding • Multiple pathways to a report • Personalization with bookmarking and notes How can I use this information? • What are the majors of the students in a specific course? • Potential to discuss career opportunities in that field, double majoring, or other relevant topics • How did my students do last term in the pre-req? • How did students in ENGL 1320 perform in ENGL 1310? • I need to quickly create a short list of students for a scholarship • Real example: UG Math Majors who have recently taken MATH 3000 or FINA 3770 and earned an A or B in that course, produced a short list of 12 to check for qualifying GPAs • Check on your majors after each semester to check for low semester GPA and overall GPA • Real example: During a training we looked to see how the 1st semester GPA for all FTIC ANTH majors and identified 2 students who had below a 2.0 to contact for immediate intervention Key Points • Technology helps • Setup required – figure out what you want before you start • Strategy required – determine how you’ll use information • Improve course? • Feedback to student? • Reporting or data? • Seek examples and see what others have done