Presentation
Brainstorming with a Bot: Upgrade or Overhype?
DescriptionAs large language models (LLMs) become embedded in collaborative workflows, their impact on human creativity remains unclear. To understand this, we introduce a scalable, process-level framework for evaluating creativity at the level of individual contributions within text-based interactions, operationalizing participation, novelty, and appropriateness using semantic and linguistic measures validated against expert human ratings. Results show that working with an LLM leads to more diverse ideas, but lower participation and appropriateness. In comparison, working with a person leads to greater active participation, appropriateness, and higher cumulative idea development. Participants also feel less confident and less positive about the results when working with an LLM. These findings offer practical advice for designing conversational agents that better support human creativity by balancing idea diversity with steady participation, appropriateness, and ongoing idea development. The framework enables continuous evaluation and comparison of conversational systems, offering a scalable basis to assess how models influence human creative behavior.
Event Type
Lecture
TimeWednesday, October 21st3pm - 3:20pm PDT
Location
Similar Presentations