Enhancement of Academic Programs Using Digital Technology
PROPOSAL COVER SHEET
Project title: Developing a Browser of Student and Course Objects (BoSCO) as an Easy to
Use Learning Analytics Tools to Facilitate Effective and Efficient Course and Curriculum
Design
Project lead
·
Name: Robert Dunbar
·
Title: Associate Professor
·
Phone: 507.258.8209
·
Email: [email protected]
Undergraduate academic degree program: BSHS University of Minnesota Rochester
Number of students graduating in program: The Bachelors of Science in Health Sciences
(BSHS) Program at the University of Minnesota Rochester is a new program, and is on track to
graduate its first full class of students in the Spring of 2013. It has a current enrollment of 371
students across the academic years.
Number of faculty associated with program: 10 Tenure Track; 25 Non-tenure track
Number of course offerings associated with program: There are currently 71 courses
associated with the program.
Are there a significant number of students taking classes associated with this degree
program who are pursuing a different degree program?
No
Please summarize briefly the key improvements in academic outcomes that would result
from support for your proposal:
This grant would fund the development of a learning analytics tool that would: 1) allow faculty
to intelligently and dynamically explore a large amount of course and student data, 2) facilitate
curricular integration and improved course design, and 3) leverage currently available University
resources.
1
Background: While designing an integrated curriculum for a Health Science degree, the faculty
at the University of Minnesota Rochester have encountered many challenging questions. Among
these are: Does an integrated curriculum improve student performance? What content in
mathematics is necessary for a Health Science degree? What content and skills in humanities
courses should we include? What are the early indicators in student grades, attitudes and
background that allow us to predict student success? To start to answer these questions and
others, 10 faculty members amassed a preliminary (IRB compliant) dataset containing the
gradebook information from 72 courses, which were offered over five consecutive semesters,
and represent a variety of disciplinary areas (BIOL, BIOC, CHEM, CLI, HUM, MATH, PHYS,
SOC). However, our collective research is hampered by a lack of easy ways to update, sort,
represent, and analyze the data. It has become clear that our faculty, and perhaps faculty
throughout the University of Minnesota system, would benefit from having a tool that permitted
the efficient collection and analysis of course and student data.
Learning analytics, which Siemens (2010) defines as “the use of intelligent data, learnerproduced data, and analysis models to discover information and social connections, and to
predict and advise on learning,” is emerging as a strategy with the potential to draw meaningful
connections among a large and growing body of learner-centered data, analyses of those data,
and student performance. Information gathered using well-designed learning analytics tools
might also be extremely useful for course and curriculum design with the intent of improving
student learning outcomes. To that end, several tools have shown promise in visualizing how
students navigate course content in specific learning management environments (Ali, 2012;
Ferguson, 2012; Macfadyen and Dawson, 2010). However, the results from these programs also
serve to highlight two major problems faced by the emerging field of learning analytics. The first
problem is how to determine what data is pedagogically useful (Ali, 2012; Ferguson, 2012) and
the second is synthesizing the data in a way that will encourage faculty to use them (Ali, 2012).
We believe that one of the best ways to encourage faculty to connect learning analytics with
course design is to have them drive the process of identifying the variables that are significant to
them, and then provide them with a meaningful way to interact with student and course related
data in a dynamic fashion. We are in a unique situation to develop and use such a tool by
bringing together faculty, IT, and administration to share data, knowledge, and resources and
build a platform for learning analytics across the curriculum.
The BSHS program at UMR has a number of qualities that align with the goal and will
increase the probability of success for the proposed project. First, there are no discipline specific
departments and all design faculty are required to do research on learning as part of their tenure
requirements. As a result, our faculty are accustomed to discussing a variety of different
disciplinary and cross-disciplinary indicators for assessment and research. In fact, we already
have approved IRB protocols (#1008E87333; #0908S71602) in place for collecting student
grades and course work across all courses offered in our department. Second, much student data
is already stored in relatively central locations. All instructors use our in-house curriculum
management system, iSEAL, which contains student assignments and course grade books. We
are also fortunate that, as part of the University of Minnesota system, we have the ability to share
and access assignments and activities that are stored within both Google Drive and Moodle.
Third, formal and informal pathways for communication among faculty and IT staff are already
established and are continuously reinforced as a result of the ongoing development of iSEAL.
Finally, the small size of our faculty population and general openness to sharing deidentified data
3
means it would be relatively easy to test the preliminary dashboard across a wide array of
disciplines. Because of our highly integrated curriculum, we can analyze how performance on
specific assignments in a course correlates with another assignment in a different course. As an
example of the latter, Dr. Dunbar and Dr. Prat-Resina are currently involved with a funded
project to investigate how students' performance on writing papers in Humanities courses
correlates with the writing in Laboratory reports in Biology and Chemistry.
Goals: For this project, our primary goal is to bring together an interdisciplinary group of faculty
to identify a set of relevant research questions, indicators, and databases in order to develop and
test a tool that would allow faculty to connect learning analytics with course design. We propose
that developing an effective and easy to use learning analytics tool that allows faculty to
intelligently and dynamically explore the large amount of student and course data would increase
the probability of faculty adoption. This Browser of Student and Course Objects (BoSCO; See
example schematic in Figure 1) would therefore help faculty connect course design and learning
outcomes at both the course and curricular level and, therefore, increase both the efficiency and
effectiveness of course and curriculum design.
Figure 1: A schematic representation of the BoSCO interface showing how it allows the
combination of different kinds of student and course data for analysis.
Partnerships and Internal Resources: The PI’s have the skills and knowledge to design a
prototype for the tool, and collectively have expertise in collaborative learning, collection and
analysis of quantitative and qualitative data, MySQL database management, and graphical user
interface design.
UMR Partnerships- In addition to support from our Vice Chancellor/Director, we have
also worked directly with Michael Olesen, UMR IT Director, who has reviewed the proposal and
agreed to partner with us in the event of funding. UMR IT’s Web and Software Development
Group currently consists of four full-time IT professionals with extensive skills in software and
web development, database administration, and data analysis. Staff from this group will be
dedicated to this project if it is funded, and their expertise would be critical toward the goal of
connecting the tool to iSEAL.
4
UMN Partnerships - UMR, in developing its information technology infrastructure, has
worked closely with the Office of Information Technology (OIT) to leverage central resources to
support UMR programs. The iSEAL curriculum delivery system and its data are hosted on OIT
virtual servers. OIT provides UMR services ranging from networking and storage infrastructure
to academic technology support. It is our intention, in coordination with the UMR IT Director, to
leverage OIT and other UMN IT resources in supporting this project. This will involve seeking
expertise interfacing with and extracting data from Google Drives, Google Sites, and in data
analytics. Discussions are currently underway to determine the resources that are available and
interest in collaboration.
Timeline: Reaching our goal will require a coordinated effort between design faculty and IT
staff with expertise in software development and database analysis. The efforts of the individuals
involved in the project will be managed by the PI’s through a series of formal meetings and the
summarized outcome of each meeting will mark specific milestones of progress throughout the
funding period. The timeline of the two years of funding will be as follows: Early in the first six
months, the PI's will meet with a multidisciplinary group of CLI faculty collaborators. In this
meeting the group will identify a list of indicators associated with pedagogical questions relevant
to the evaluation and design of their courses. During this meeting, the group will also discuss the
location (iSEAL, Moodle, Google Drive, etc.) and accessibility of the data associated with each
indicator. The PI’s will then work together to: summarize the outcome of the meeting, generate a
hierarchical list of questions with the associated indicators, and identify the databases in which
the data is stored. PI’s and collaborators will then meet to finalize the prioritized list as well as
begin to discuss possible statistical tools that might be useful. The prioritized questions, the
associated indicators, the relevant databases, and suggested statistical tools that are finalized in
this meeting will all be used in months seven through twelve by the PI’s and a software
developer/database analyst to build a mock-up of the learning analytics tool (BoSCO). The beta
version of BoSCO will be completed by the end of the first year of funding. Early in funding
months 13-18, the PI’s will present BoSCO to our collaborators for feedback. The PI’s and IT
developer will then make changes to BoSCO based on collaborator feedback. An updated
version of the analytics tool will then be distributed to our collaborators for use early in funding
months 19-24. During this testing period, we plan to collect data on how frequently and in what
way our collaborators use the tool. In the last two months of year two, the PI’s will generate a
summary of the tool, its capabilities, how it was used, and examples of data analyzed. This
summary will represent the final milestone of this stage of the project and we have every
expectation that the results will be distributed as a presentation or publication.
References:
Ali, L., Marek, H., Gasevic, D., Jovanovic, J. (2012). A qualitative evaluation of evolution of a
learning analytics tool. Computers & Education 58, 470-489.
Ferguson, R. and Buckingham Shum, S. (2012). Social Learning Analytics: Five Approaches.
Proc. 2nd International Conference on Learning Analytics & Knowledge, (29 Apr-2 May,
Vancouver, BC). ACM Press: New York
Macfadyen, L.P., & Dawson, S. (2010). Mining LMS data to develop an “early warning
system” for educators: A proof of concept. Computers & Education 54, 588-599.
Siemens, G. (2010). What are Learning Analytics? Retrieved January 5, 2013, from
http://www.elearnspace.org/blog/2010/08/25/what-are-learning-analytics/
5
Proposed Budget:
Year One:
PI summer salary and fringe (x 3)
=
$12,300
Collaborators summer salary and fringe (x 7)
=
$12,700
Developer/Analyst (1/8 FTE; 5hrs/week) salary and fringe (x 1)
=
$10,000
Total for Year One:
=
$35,000
Year Two:
PI summer salary and fringe (x 3)
=
$11,000
Collaborators summer salary and fringe (x 7)
=
$10,000
Developer/Analyst (1/8 FTE; 5hrs/week) salary and fringe (x 1)
=
$10,000
Travel (Dissemination of findings)
=
$4,000
Total for Year One:
=
$35,000
6
Molly J. Dingel, Ph.D.
Assistant Professor, Center for Learning Innovation
University of Minnesota Rochester
Education/Professional Preparation:
B.A. Grinnell College, May 1998, Mathematics and Sociology
M.A. University of Kansas, May 2000, Sociology, with honors
Ph.D. University of Kansas, December 2005, Sociology
Appointments:
2005-2007: Postdoctoral Fellow, Biomedical Ethics Research, Mayo Clinic College of Medicine
2007-2008: Assistant Professor, Sociology, North Dakota State University
2009 - present: Assistant Professor, University of Minnesota Rochester
Select Publications/Presentations:
Dingel, Molly J., Wei Wei, and Aminul Huq. Forthcoming. “Cooperative learning and peer
evaluation: the effect of free riders on team performance and the relationship between
course performance and peer evaluation.” Journal of Scholarship on Teaching and
Learning. Accepted January 9, 2013.
Dingel, Molly J., Rachel Hammer, Jenny Ostergren, and Jennifer B. McCormick. 2012. “Chronic
Addiction, Compulsion, and the Empirical Evidence.” American Journal of Bioethics
Neuroscience. 3(2):58-59.
Dingel, Molly J., Ashley Hicks, Marguerite Robinson and Barbara A. Koenig. 2012. “Integrating
Genetic Studies of Nicotine Addiction into Public Health Practice: Stakeholder
Perspectives on Challenges, Barriers, Opportunities.” Public Health Genomics. 15(1):4655.
Dingel, Molly J. 2013. “Teaching Critical Thinking: A Review Of and Reflection On Classroom
Activities.” Accepted for presentation at the Midwest Sociological Society annual
meeting. Chicago, IL, March 27-30, 2013.
Metzger, Kelsey and Molly J. Dingel. 2012. “Preparing health sciences undergraduates for the
complexities of individualized medicine: a curriculum overview.” Individualizing
Medicine Conference 2012. Rochester, MN. October 1-3.
Huq, Aminul, Molly J. Dingel, and Jered Bartels. 2011. “Integrating Introductory Statistics and
Sociology Courses through Group Projects in a Health Sciences Program.” Joint
Statistical Meetings. Miami Beach, FL, July 30-August 4, 2011.
Dingel, Molly J., Rebecca Bamford, and Robert L. Dunbar. 2010. “Optimizing groupwork to
advance learning: exploring collaborative work in undergraduate courses.” Midwest
Sociological Society annual meeting. Chicago, IL, April, 2010.
7
Robert L. Dunbar, Ph.D.
Associate Professor, Center for Learning Innovation
University of Minnesota Rochester
Education:
B.S.S. Cornell College, May 1993, Biology and History
M.A. Drake University, May 1995, Biology
Ph.D. University of Minnesota, January 2002, Neuroscience
Appointments:
2002-2003: Postdoctoral Fellow, Duke University, Neuroscience Research
2003-2008: Assistant Professor,Biology, Buena Vista University
2009 - present: Associate Professor, University of Minnesota Rochester
Select Publications/Presentations:
Dunbar, R. L., & Nichols, M. D. Fostering empathy in undergraduate health science majors
through the reconciliation of objectivity and subjectivity: An integrated approach.
Anatomical sciences education, 5(5), 301–308, 2012.
Aryal, B., Dunbar, R.L., Muthyala, R.S. “Assessment of Group Learning in Interdisciplinary
Environments”. Proceedings of the 2012 National Association for Research in Science
Teaching Annual Meeting, 2012.
Petzold, AM, Nichols, MD and Dunbar RL. Teaching artful and accurate scientific presentation
skills at the undergraduate level: A multidisciplinary approach. ACUBE annual meeting.
October 19-20, 2012.
Dingel, Molly J., Rebecca Bamford, and Robert L. Dunbar. 2010. “Optimizing groupwork to
advance learning: exploring collaborative work in undergraduate courses.” Midwest
Sociological Society annual meeting. Chicago, IL, April, 2010.
Dunbar, R.L. “Integrative courses: anatomy and beyond.” Anatomical Sciences Education,
3(2):73-6, 2010.
Berglund, K., Dunbar, R.L., Lee, P., Feng, G., and Augustine, G.J. “A Practical Guide: Imaging
Synaptic Inhibition with Clomeleon, a Genetically Encoded Chloride Indicator.” Imaging
in Neuroscience and Development: A Laboratory Manual, eds. A. Konnerth, F. Lanni, and
R. Yuste, Cold Spring Harbor Laboratory Press, 2005.
Dunbar, R.L., Chen, G., Gao, W., Reinert, K.C., Feddersen, R., and Ebner, T.J. “Imaging
Parallel Fiber and Climbing Fiber Responses and Their Short-term Interactions in the
Mouse Cerebellar Cortex in vivo.” Journal of Neuroscience, 126(1):213-27, 2004.
8
Xavier Prat-Resina, PhD
Assistant Professor, Center for Learning Innovation
University of Minnesota Rochester
Education/Professional Preparation:
BSc. Chemistry. Universitat de Barcelona (Spain) 1999
PhD. Theoretical Chemistry, Universitat Autonoma de Barcelona (Spain) 2004
Postdoctoral Associate. University of Wisconsin Madison. 2005-2006.
Appointments:
2007: "Juan de la Cierva" researcher at the Barcelona Biomedical Research Park (Spain) 2008 2010 : Research Associate for the "Chemical Education Digital Library", Chemistry Department,
University of Wisconsin at Madison.
2010 : Associated Lecturer, Chemistry Department, University of Wisconsin at Madison.
2010 - Present: Assistant Professor, University of Minnesota at Rochester
Select Publications/Presentations:
(in chemical education)
Author of ChemEd DL application "Molecules 360"
http://www.chemeddl.org/resources/models360 , 2008-2012
Author of ChemEd X Data web platform
http://chemdata.umr.umn.edu/chemedXdata/, 2012
Xavier Prat-Resina and Bernadette Caldwell. JCE Concept Connections: Computational
Molecular Modeling, . J. Chem. Educ., 86 (8): 958, 2009.
(in theoretical chemistry)
Andrea Bottoni, Giampietro Miscione, Juan Novoa, and Xavier Prat-Resina. A DFT
computational study of the mechanism of allyl halides carbonylation catalyzed by nickel
tetracarbonyl. J. Am. Chem. Soc., 125(34):10412-10419, 2003.
Xavier Prat-Resina, Josep Maria Bofill, Angels González-Lafont, and José Maria Lluch.
Geometry optimization and transition state search in enzymes: Different options in the
microiterative method. Int. J. Quant. Chem., 98 (4):367-377, 2004.
Xavier Prat-Resina, Angels González-Lafont, and José Maria Lluch. Reaction Mechanism of the
Mandelate Anion Racemization Catalyzed by Mandelate Racemase Enzyme: A QM/MM
Molecular Dynamics Free Energy Study J.Phys.Chem.B, 109 (44): 21089-21101, 2005.
Demian Riccardi, Partricia Schaefer, Yang Yang, Haibo Yu, Nilanjan Ghosh, Xavier PratResina, Peter Koenig, Guohui Li, Dingguo Xu, Hua Guo, Marcus Elstner and Qiang Cui.
Development of effective QM/MM methods for complex biological processes J.Phys.
Chem.B, 110 (13):6458-6469, 2006 Feature Article, (Cover).
Nilanjan Ghosh, Xavier Prat-Resina, M. R. Gunner and Qiang Cui
Microscopic pKa Analysis of Glu286 in Cytochrome c Oxidase (Rhodobacter
sphaeroides): Toward a Calibrated Molecular Model Biochemistry, 48 (11): 2468–2485,
2009
9