ISCAP Proceedings: Abstract Presentation
Towards an Online Learning Top Level Dashboard
Christopher Njunge
California Lutheran University
Paul Witman
California Lutheran University
Abstract
With the growth of Higher Education online programs, institutions, departments and faculty are seeking ways to monitor, analyze and improve the experiences and outcomes of learners. This study proposes a Top Level Dashboard that captures the interactions that learners have with faculty, departments and institutions and presents them as a dashboard.These interactions can then be evaluated for the extent to which they meet or do not meet their desired outcomes (rating of positivity vs negativity). The total number of interactions, and percent of positive interactions can be useful in understanding and evaluating online programs and courses.
Many institutions already collect and use this data through various systems such as: enrollment management, course evaluation, learning management etc. These systems often operate as silos and don’t allow for visibility of learner interactions at faculty, department or institution level. We propose that aggregate data from these systems can be captured into a top level dashboard to visualize learners' online experiences and assess the extent to which they are positive or negative. This can provide various insights including: variations in learner experiences by course, department, cohort or day and time of course offering. Exploratory data analysis can be carried out to understand the data further, and machine learning can be useful in predicting certain outcomes or suggesting certain interventions.
The Top Level Dashboard can be operationalized in a number ways depending on the values of the organization and the learning outcomes sought. The key questions would be how to define an interaction, and how to distinguish positive from negative interactions. Thematically, the dashboard can be set up to visualize learning interactions such as aggregate assignments completed on time, course evaluations such as work load, course and instructor ratings, or responsiveness to mentorship initiatives. These can then be compared across departments, terms/semesters or faculty.
As our understanding of online learning communities in Higher Education grows, studies on what to monitor, how to measure it, and how to visualize it will continue to evolve. A Top Level Dashboard approach will be helpful in encouraging institution-wide discussions and collaborations that provide shared understandings of data within the institution. It will surface baselines of interactions that online learners are engaging in, and alert stakeholders to trends and variations relative to such baselines. Finally, the ability to review the interactions at various levels of granularity will allow for insights to be gained and target interventions implemented and assessed.
One limitation of this approach is that it leverages aggregate data, and thus it may take some time to identify the root causes of any issues identified. However, this makes it possible to implement without exposing protected learner data, and can create a baseline for expected types and levels of learner interactions.