Abstract
This paper presents a novel approach for generating actionable recommendations from educational event data collected by Campus Management Systems (CMS) to enhance study planning in higher education. The approach unfolds in three phases: feature identification tailored to the educational context, predictive modeling employing the RuleFit algorithm, and extracting actionable recommendations. We utilize diverse features, encompassing academic histories and course sequences, to capture the multi-dimensional nature of student academic behaviors. The effectiveness of our approach is empirically validated using data from the computer science bachelor’s program at RWTH Aachen University, with the goal of predicting overall GPA and formulating recommendations to enhance academic performance. Our contributions lie in the novel adaptation of behavioral features for the educational domain and the strategic use of the RuleFit algorithm for both predictive modeling and the generation of practical recommendations, offering a data-driven foundation for informed study planning and academic decision-making.
Practical problem
University staff who design degree programs must decide which courses to recommend in which semester, how heavy a course load to allow, and which prerequisites to enforce, yet they usually do this from regulations and intuition rather than from evidence about what actually helped past students. Campus management systems record every exam attempt of every student, but existing analyses of such data look at single courses or grades in isolation and miss the order in which courses are taken and how they interact. The one prior attempt to mine course sequences relied on decision trees, which overfit and react strongly to unusual students, so the rules they produce are hard to trust and act on. Program designers therefore lack reliable, readable rules linking study behavior to outcomes such as grade point average.
Research questions
- Which behavioral features derived from campus management event data capture students’ academic histories, course sequencing, and attempt patterns in a way that is useful for study planning?
- Can an interpretable rule-ensemble model predict academic outcomes such as grade point average more accurately and robustly than the decision-tree approach used so far?
- How can the rules of such a model be turned into actionable recommendations for study program designers?
Contributions
- A three-phase approach that defines objectives, engineers behavioral features from educational event logs, and distills recommendations from an interpretable predictive model.
- A catalogue of educational behavioral features adapted from a general process mining framework, spanning control-flow (occurrences, first and last occurrence, directly-follows, eventually-follows), data (credits, grades), and time (study and course duration) perspectives.
- Uses the RuleFit algorithm (gradient-boosted rules plus a Lasso-regularized sparse linear model) for predicting grade point average and ranking rules by importance and coefficient.
- A recommendation generation procedure that interprets high-importance rules by coefficient sign and magnitude into guidance on course load and course sequencing.
- An empirical study on campus management data of 3,190 computer science bachelor students at RWTH Aachen University, with per-semester datasets and five-fold cross-validation.
Key findings
- On five semester-stage datasets (Data-S2 to Data-S6) predicting grade point average up to semester N, RuleFit yields lower mean squared error than the decision-tree baseline in every case under five-fold cross-validation.
- The gap is largest on Data-S3, where the decision tree’s mean squared error exceeds 2 while RuleFit stays well below; variance of results is higher on Data-S2 and Data-S5, attributed to immature behavior patterns and a smaller cohort.
- The number of extracted rules, each a candidate recommendation, ranges from 65 for the third semester to 143 for the fifth; most semesters produce more positive than negative rules, the fifth semester being the exception.
- For fourth-semester students, the top rule (completing SAP by the second semester with more than 77 total credits) has importance 0.0898 and coefficient -0.1799, i.e. it is associated with a better grade point average.
- Limitations: the evaluation covers a single program at one university, rules learned from history can become obsolete as curricula change, and personal, socio-economic, and instructor factors are not captured in the event data.