Presentation
Assessing Affective Signals in LLM-Driven Human–AI Training Interactions
DescriptionLarge language models (LLMs) are increasingly used to create emotionally intelligent AI-driven non-player characters (AI-NPCs) for interactive law enforcement training. Yet evaluating the behavioral and affective fidelity of these agents remains challenging, particularly in scenarios that require socially meaningful and emotionally coherent interaction. This study examines affective signals in LLM-driven human–AI training interactions using a VR-based de-escalation scenario with AI-NPCs designed to vary in cooperative behavior and neurodiversity profile. Participants interacted with AI-NPCs in an immersive virtual environment and completed post-interaction emotion ratings using the standardized Discrete Emotion Questionnaire, while dialogue transcripts were analyzed using a text-based emotion recognition pipeline based on EmoBERTa, a speaker-aware, transformer-based emotion recognition model. Results suggest that AI-NPC dialogue showed detectable differences in affective expression across designed conditions, especially for cooperative versus noncooperative interaction styles. However, emotions inferred from participant language showed only limited alignment with participants’ self-reported affect. These findings highlight both the value and limitations of computational emotion analysis for evaluating AI-based training systems. These findings suggest that, rather than replacing user-reported experience, language-based affect inference may serve as a complementary tool for assessing whether AI-NPCs express intended social and emotional cues in scalable training environments.
Event Type
Lecture
TimeWednesday, October 21st4:30pm - 4:50pm PDT
Location