Presentation
Adaptive Multimodal AI Tutoring: Using Real-Time Cues to Personalize Feedback and Support Engagement in Learning
SessionThursday Poster Session
DescriptionLearner engagement is widely regarded as a critical factor that influences learning success. In online learning, the absence of in-person interaction increases the risk of attention drift and boredom, which underscores the need to monitor and support engagement. Recent advances in generative AI enable systems to infer learners’ cognitive and affective states from multimodal cues, supporting more adaptive instructional responses. However, little research has examined how such systems can dynamically adapt both the type and timing of feedback based on learners’ moment-to-moment engagement states inferred from multimodal signals. This work presents an adaptive multimodal AI-driven tutoring system that infers learners’ states by interpreting real-time visual, auditory, and behavioral cues. Based on the inferred learner state, the AI tutor determines when and how to intervene to sustain engagement. The system is structured as a closed-loop cognitive architecture: perception (capturing real-time multimodal cues), decision (aggregating the multimodal inputs into four affective metrics), and action (delivering feedback based on the inferred state by mapping each metric to a feedback type and timing strategy). This work presents a high-fidelity, interactive multimodal AI tutoring system that illustrates the feasibility of integrating multimodal cues to enable adaptive instructional feedback and engagement-aware intervention in online learning contexts.
Event Type
Poster
TimeThursday, October 22nd5:30pm - 6:30pm PDT
Location