부산대학교 도서관

Abstract Online learners participate in various educational activities including reading, writing, watching video tutorials, online exams, and online meetings. During the participation in these educational activities, they show various engagement levels, such as boredom, frustration, delight, neutral, confusion, and learning gain. To provide personalized pedagogical support through interventions to online learners, it is important for online educators to detect their online learners’ engagement status precisely and efficiently. This paper presents a review of the state of the art in engagement detection in the context of online learning. We classify the existing methods into three main categories—automatic, semi-automatic and manual—considering the methods’ dependencies on learners’ participation. Methods in each category are then divided into subcategories based on the data types (e.g., audio, video, texts for learner log data etc.) they process for the engagement detection. In particular, the computer vision based methods in the automatic category that use facial expressions are examined in more details because they are found to be promising in the online learning environment. These methods are nonintrusive in nature, and the hardware and the software that these methods use to capture and analyze video data are cost-effective and easily achievable. Different techniques in the field of computer vision and machine learning are applied in these methods for the engagement detection. We then identify their challenges of engagement detection and explore available datasets and performance metrics for engagement detection, and provide recommendations for the future to advance the technology of engagement detection for online education.