Presentation
Joint Activity Testing for Evaluating Event Detection in Dynamic Human-AI Systems
DescriptionReliable methods to evaluate the potential benefits and risks of human-AI systems in high-risk domains remain an acknowledged research shortcoming. Joint Activity Testing (JAT) has shown considerable promise to reveal these benefits and risks. However, applications of JAT have been currently restricted to static scenarios, limiting the applicability to dynamic settings. In this work, we expand JAT techniques to incorporate a sequence of decisions in a temporally extended task: detecting active satellite maneuvers. We report the results of a pilot study to assess the feasibility of JAT for evaluating human-AI systems in space situational awareness. Our results show distinct challenge–performance relationships across scenarios, suggesting the value of JAT for assessing dynamic joint human-AI event detection tasks.
Contributors
Event Type
Lecture
TimeFriday, October 23rd11am - 11:20am PDT
Location