Presentation
RISE: Pragmatic Evaluation of Human-AI Reliance Calibration
DescriptionThis presentation will introduce the Reliance in Intelligent Systems Evaluation (RISE) framework, a practical structure for assessing how people use AI advice in human-AI teaming contexts. It will discuss why technical performance alone is insufficient for evaluating AI-enabled systems, particularly in safety-critical work, and why reliance calibration is an important human factors concern.
The presentation will outline how RISE organises reliance measures into key categories, including raw AI use, appropriate and inappropriate reliance, verification and monitoring behaviour, trust, confidence, workload, and task outcomes. It will also describe a proposed minimum reporting standard for human-AI reliance analysis, intended to help researchers and practitioners compare designs, identify misuse or disuse risks, and interpret whether AI support is being used safely and effectively.
The presentation will outline how RISE organises reliance measures into key categories, including raw AI use, appropriate and inappropriate reliance, verification and monitoring behaviour, trust, confidence, workload, and task outcomes. It will also describe a proposed minimum reporting standard for human-AI reliance analysis, intended to help researchers and practitioners compare designs, identify misuse or disuse risks, and interpret whether AI support is being used safely and effectively.
Event Type
Lecture
TimeTuesday, October 20th11:40am - 12pm PDT
Location