Presentation
What is the Best Word to Describe AI Uncertainty: Confidence, Certainty, Reliability, or Accuracy?
DescriptionAI systems are becoming increasingly common in decision-making tasks. However, these systems often generate probabilistic responses, resulting in uncertainty in their outputs. Understanding this uncertainty can help users make robust decisions, such as when to trust the agent and whether to seek more information. Interpreting uncertainty is not always straightforward, though; past work shows there is variability in how people interpret probabilistic information (e.g., weather forecasts). This study builds on that research by analyzing how people interpret probabilistic AI responses. In this survey-based experiment (N = 632), participants interpreted AI uncertainty statements. The word used to describe the uncertainty (confidence, certainty, reliability, or accuracy) varied between trials. Participants often treated confidence and certainty as indicators of how sure the agent was about its output, whereas interpretations of reliability and accuracy were more variable. These findings can guide how AI systems communicate uncertainty, supporting more accurate user interpretations and robust decision-making.
Event Type
Lecture
TimeTuesday, October 20th11:30am - 12pm PDT
Location