Presentation
Triangulating Unmoderated Usability Testing Methods: Lessons from Evaluating Educational Games
DescriptionThis experience report studies the strategic triangulation of fully unmoderated usability testing methods to evaluate educational games. Our team designed, piloted, and implemented a multimethod usability study of six environmental educational games delivered through Canvas to 45 undergraduate students. Across three pilot deployments, we refined instructions, stabilized screen-recording procedures, and addressed participant discomfort with remote think-aloud protocols by expanding opportunities for written feedback.
The study integrated three different unmoderated UX methods: screen recordings to capture behavioral data, think-aloud protocols to capture cognitive data, and post-session surveys to capture attitudinal data. Structured comparison across these data streams revealed that each method captured distinct aspects of the user experience. For example, recordings exposed navigation breakdowns, think-aloud data revealed conceptual misunderstandings, and surveys often reflected positive retrospective impressions despite observable friction during interaction.
Our findings demonstrate that divergence across behavioral, cognitive, and attitudinal measures is analytically valuable while conducting these tests. This report offers practical guidance for UX researchers and HFE professionals designing scalable unmoderated usability evaluations and proposes triangulation as a framework for improving interpretive validity in remote multimethod UX research.
The study integrated three different unmoderated UX methods: screen recordings to capture behavioral data, think-aloud protocols to capture cognitive data, and post-session surveys to capture attitudinal data. Structured comparison across these data streams revealed that each method captured distinct aspects of the user experience. For example, recordings exposed navigation breakdowns, think-aloud data revealed conceptual misunderstandings, and surveys often reflected positive retrospective impressions despite observable friction during interaction.
Our findings demonstrate that divergence across behavioral, cognitive, and attitudinal measures is analytically valuable while conducting these tests. This report offers practical guidance for UX researchers and HFE professionals designing scalable unmoderated usability evaluations and proposes triangulation as a framework for improving interpretive validity in remote multimethod UX research.
Event Type
Lecture
TimeTuesday, October 20th3pm - 3:20pm PDT
Location
Similar Presentations