Student-centric Model of Learning Man- agement System Activity and Academic Per- formance from Correlation to Causation

2025-05-02 0 0 1.2MB 43 页 10玖币

侵权投诉

Student-centric Model of Learning Man-

agement System Activity and Academic Per-

formance: from Correlation to Causation

Varun Mandalapu

University of Maryland Baltimore County

varunm1@umbc.edu

Lujie Karen Chen

University of Maryland Baltimore County

lujiec@umbc.edu

Sushruta Shetty

University of Maryland Baltimore County

sshetty2@umbc.edu

Zhiyuan Chen

University of Maryland Baltimore County

zhchen@umbc.edu

Jiaqi Gong

Univerisy of Alabama

jiaqi.gong@ua.edu

In recent years, there is a lot of interest in modeling students’ digital traces in Learning Man-

agement System (LMS) to understand students’ learning behavior patterns including aspects of

meta-cognition and self-regulation, with the ultimate goal to turn those insights into action-

able information to support students to improve their learning outcomes. In achieving this

goal, however, there are two main issues that need to be addressed given the existing literature.

Firstly, most of the current work is course-centered (i.e. models are built from data for a speciﬁc

course) rather than student-centered (i.e. models are built taking the perspective of students by

analyzing data across courses); secondly, a vast majority of the models are correlational rather

than causal. Those issues make it challenging to identify the most promising actionable factors

for intervention at the student level where most of the campus-wide academic support is de-

signed for. In this paper, we explored a student-centric analytical framework for LMS activity

data that can provide not only correlational but causal insights mined from observational data.

We demonstrated this approach using a dataset of 1651 computing major students at a public

university in the US during one semester in the Fall of 2019. This dataset includes students’

ﬁne-grained LMS interaction logs and administrative data, e.g. demographics and academic

performance. In addition, we expand the repository of LMS behavior indicators to include

those that can characterize the time-of-the-day of login (e.g. chronotype). Our analysis showed

that student login volume, compared with other login behavior indicators, is both strongly cor-

related and causally linked to student academic performance, especially among students with

low academic performance. We envision that those insights will provide convincing evidence

for college student support groups to launch student-centered and targeted interventions that

are effective and scalable.

Keywords: Learning Management System, Student behavior modelling, Causal Analysis

arXiv:2210.15430v3 [cs.CY] 30 Mar 2023

1. INTRODUCTION

Understanding student learning behaviors is a complex process as it is affected by a

multitude of factors like self-regulation, instructional design, and social environment

(Ikink, 2019). However, it is not efﬁcient to study all the inﬂuencing factors as some

factors are not feasible to intervene compared to others. Some have a higher impact

on the performance at an individual level than others. Earlier research in education

identiﬁed self-regulation as a highly impactful and intervenable factor (Zimmerman

and Kitsantas, 2014;Schapiro and Livingston, 2000). Self-regulation, in general terms,

is deﬁned as one’s ability to control their own behaviors, thoughts, and emotions to

achieve their goals to drive their learning experiences successfully.

Studying student self-regulation capabilities is an arduous task as it involves multidi-

mensional and complex factors like planning, goal setting, time management, and self-

monitoring. Many of these factors are hard to quantify without student self-reports or

other psychometric evaluations, and these reports are prone to biases like social desir-

ability and reference bias (Toering et al., 2012;Rosenman et al., 2011). To mitigate these

biases, researchers in education focused on extracting data from computer-based sys-

tems like Learning Management Systems (LMS) as they are used to deliver course con-

tent and have the capability to capture student interaction and assessment behaviors

non-invasively (Tempelaar et al., 2015;Salehian Kia et al., 2021). Student interaction

data with LMS proved to be a valuable resource in identifying learning strategies and

track patterns among students that strongly support their academic performance. The

data logs collected by LMS systems are analyzed with scientiﬁc techniques published

in the Educational Data Mining (EDM) domain. In their study, Romero and Ventura

(Romero and Ventura, 2007) described that EDM methods rely on clustering and pat-

tern recognition techniques to categorize students into various groups based on their

interaction patterns. Categorization of students using clustering and pattern recogni-

tion supports instructors in making changes for a set of students. Teaching practices

that impact the entire classroom can be evaluated using predictive analytics that tracks

student learning and achievement from the vast amount of interaction data collected

by LMS. Majority of these studies analyze student interaction data in a speciﬁc course

to understand student learning behaviors and teaching methods.

While course level predictions are suitable for supporting instructor level decision mak-

ing; however, if intervention is on student level behaviors such as study habits or self-

regulation skills, it is beneﬁcial to look at student-centered indicators so that interven-

tions may be more targeted and cost-effective. Developing student-centric models that

analyze student LMS interactions across courses in a college/university setting will

help address the issues with course speciﬁc models. This study is the ﬁrst step in devel-

oping feature extraction and modeling methods from LMS login data that are scalable

across semesters and transferable across different undergraduate ﬁelds. Developing

these early performance prediction models alone does not help improve student learn-

ing as identifying self-regulation components like time management and other student

engagement factors play a crucial role in recommending methods to improve their cur-

rent behaviors.

LMS systems also enabled researchers to study time management strategies, a compo-

nent of self-regulation adopted by students (Jo et al., 2015;Ahmad Uzir et al., 2020).

In contradiction to traditional time management studies that utilize self-reported ques-

tionnaires, student time management strategies are analyzed based on login and sub-

mission patterns in LMS systems. However, only few studies focused on the relation-

ship between time management strategies identiﬁed in LMS systems with the biologi-

cal nature of human functionality. Therefore, these earlier modeling efforts are limited

to predicting student performance as a function of their login patterns and fell short

of identifying patterns in time management that support improving student academic

improvement. These earlier studies also focused on student behaviors related to their

time compliance with assignments, including procrastination behaviors and ways to

support the earliness of student work (Ilves et al., 2018;Edwards et al., 2009;Martin

et al., 2015). These studies reported that early starters have better learning outcomes,

but there is little emphasis on what times these students work during a day and how

these learning time patterns affect their performance. Inspired by these earlier studies

and comparatively similar studies in biology about human activity/productivity dur-

ing different times of the day, referred to as chronotype/chronological analysis, this

study analyzes the relationship between student activity on LMS at different times in

a day and their performance by employing clustering methods. In addition to this, the

student chronotype clusters are added to LMS behaviors and demographics as features

to predictive models to predict student performance.

Earlier research in the area of Learning Analytics and Educational Data Mining focused

on the relationship between self-regulation and student performance through LMS sys-

tems used student interaction features as proxy variables for self-regulation (Dabbagh

and Kitsantas, 2013;Landrum, 2020). These studies showed a signiﬁcant correlation

between student login patterns and their academic performance. However, it is still

challenging to design interventions as correlations do not necessarily mean a meaning-

ful cause and effect between student login behaviors and academic performance. In

this work, we introduce a novel framework to perform causal inference and discovery

of multimodal student data collected from LMS and student administrative system. In

addition, we enrich the feature set by introducing new approach to extract login behav-

iors at a student level. Finally, this work also introduced concepts from human chrono-

biology to analyze student login behavior on an hourly basis (Chronotype analysis) to

identify activity patterns at different times in a day and their relationship to academic

performance. The proposed correlation/predictive modeling framework (Mandalapu

et al., 2021), chronotype analysis and causal inference framework goes beyond tradi-

tional educational data mining approaches to answer the following research questions

that are under-studied in current literature.

2. RESEARCH QUESTIONS & OUTLINE

RQ1. How student learning activity at different times in a day (chronotypes) relate to

their academic performance?

RQ2. How student-centric characterization of LMS activity correlates with academic

performance?

RQ3. What are the causal relationship between student-centric characterization of LMS

activity and academic performance?

In the related work Section 3, we discuss earlier research works that use LMS system

data to predict student academic achievement. Section 3.1, we details research work

that focuses on the evolution of chronotype on the basis of human biology and in the

following Section 3.2 explains how chronotypes are linked to individual cognitive abil-

ities and academic achievement.The ﬁnal related work sections 3.3 and 3.4 explains

earlier research work in correlation and causation to identify relationship between stu-

dent LMS interaction data and their academic performance.

Section 4provides the details of data set used to perform analysis in this study. The

methodology section 5details the steps taken to extract student aggregate login features

in 5.1. The correlation studies that focuses on machine learning modeling, and post hoc

LIME based explanations are detailed in Section 5.2.1 and Section 5.2.2. The section

5.3 details the causal analysis and inference tools and methods adopted in this study.

The results of the study related to student chronotype analysis are listed in section 6.1.

Section 6.2.1 explains the results of predictive models by comparing their predictions

based on multiple evaluation metrics, this section also details various correlation based

explanation methods that details the relationship between student demographics, LMS

interaction features and academic performance. The causal discovery and inference

results are detailed in section 6.3. Finally in Section 7, we detail the key ﬁndings and

contributions made by this study to the existing literature in subsections ?? and ??.

3. RELATED WORK AND BACKGROUND

Universities and colleges around the world adopted LMS systems, such as Moodle and

Blackboard, to provide onsite, hybrid, and online courses based on their capabilities

to support communication, content creation, administration, and assessment(Alokluk

et al., 2018;Berechet and Georgescu, ) LMS systems build and swiftly distribute indi-

vidualized learning materials and information in addition to automating and centraliz-

ing a variety of administrative processes including setting up and managing student ac-

counts, establishing a syllabus, assignments, assessments, and grading, etc.(Shchedrina

et al., 2021). These methods also encourage the reuse of instructional materials. Instruc-

tors can design content structures, distribute it in a sequential order, limit access, orga-

nize group activities, monitor student activity, load and replace learning materials, and

give feedback on assessments using the systems. LMS systems use a variety of login

roles based on user classiﬁcation thanks to cutting-edge database software created by

Oracle, IBM, and Microsoft that emphasizes interconnection, data independence, and

security. These roles will permit instructors to create new content or privately address

student issues and create discussion boards to capture student knowledge on speciﬁc

topics.

The relationship between the use of LMS and student academic accomplishments has

been examined in numerous papers in the ﬁelds of Educational Data Mining and Learn-

ing Analytics. According to Vengroff and Bourbeau’s (Vengroff and Bourbeau, 2006)

study, undergraduate students beneﬁted from having more content available in LMS.

They also conclude that students who used LMS regularly did better in exams than

their peers who have minimal interactions. In their study, Dutt and Ismail (Dutt and

Ismail, 2019) found that keeping track of the LMS resources that students use encour-

ages the creation of fresh approaches to learning that improve student achievement.

Additionally, they examined the thresholds for student involvement elements like self-

assessment quizzes, exercise time, discussion boards, and performance results. An-

other study by Lust et al. (Lust et al., 2011) investigated how differently students used

various LMS capabilities, such as the amount of time spent on web-links, web lectures,

quizzes, feedback, discussion board entries, and messages read. The ﬁndings of this

research made a signiﬁcant contribution to the creation of adaptive and creative recom-

mendation systems. In their research, Hung and Zhang (Hung and Zhang, 2012) also

discovered trends based on six indices that represented student effort: the frequency

with which students access course materials, the number of LMS logins, the sum of

interactions in discussion threads, the number of synchronous discussions, the number

of posts read, and the course’s ﬁnal grades.

When investigating the relationship between students’ online behavior on the LMS and

their grades, Dawson et al. (Dawson et al., 2010) found a substantial difference between

high and low performing students in the number of online sessions visited, overall time

spent, and the number of posts in discussion boards. A multinomial logistic regression

model was created by Damainov et al. (Damianov et al., 2009) based on the amount

of time spent in the LMS. This study discovered a substantial correlation between the

amount of time students spent studying and their grades, particularly for those who

had grades between D and B. Other studies emphasized how frequently students ac-

cessed course materials in the LMS, as opposed to measuring time spent online. A

study by Baugher et al. (Baugher et al., 2003) found that regularity in student hits is a

reliable predictor of student performance compared to the total number of hits. In their

study, Chancery and Haque studied the student interaction logs of 112 undergraduate

students and discovered that those with lower LMS access rates had lower grades than

their counterparts with greater access rates. Biktimirovan and Klassen (Biktimirov and

Klassen, 2008), who observed a signiﬁcant correlation between student hit consistency

and success, provided additional support for this study. In their study, which measured

access to numerous LMS features, it was discovered that the only signiﬁcant predictor

of student achievement is access to assignment solutions. However, these studies tend

to be more descriptive than predictive.

3.1. CIRCADIAN RHYTHMS & CHRONOTYPES

Humans have a 24-hour internal clock running in the background that determines

when to sleep and when to be productive. These 24-hour cycles are referred to as cir-

cadian rhythms. The behavioral manifestation of these circadian rhythms is termed

chronotype (Reppert and Weaver, 2002;Dibner et al., 2010). Understanding an individ-

ual’s chronotype helps identify their routine and provides insights about their highly

active and productive time. Earlier research argues that circadian rhythms differ be-

tween individuals as a group of clock genes conditions them (Bell-Pedersen et al., 2005).

Nonetheless, these are not ﬁxed and can vary during an individual’s lifetime.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Student-centricModelofLearningMan-agementSystemActivityandAcademicPer-formance:fromCorrelationtoCausationVarunMandalapuUniversityofMarylandBaltimoreCountyvarunm1@umbc.eduLujieKarenChenUniversityofMarylandBaltimoreCountylujiec@umbc.eduSushrutaShettyUniversityofMarylandBaltimoreCountysshetty2@umbc.edu...

展开>> 收起<<

Student-centric Model of Learning Man- agement System Activity and Academic Per- formance from Correlation to Causation.pdf

共43页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Student-centric Model of Learning Man- agement System Activity and Academic Per- formance from Correlation to Causation

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: