Presentation
Reading Between the Lines: A Scoping Review of Large Language Models in Conversational Data Analysis
SessionThursday Poster Session
DescriptionLarge language model based tools, such as ChatGPT, are increasingly being used to help researchers analyze human-generated conversational data including interviews and focus groups. These tools can reduce the time required for qualitative analysis by performing repetitive manual tasks such as coding and identifying themes or patterns across large sets of text. Nevertheless, the use of such tools poses questions on issues related to accuracy, transparency, privacy and trust.
This scoping review outlines the way in which transformer-based language models have been used in recent studies to facilitate qualitative analysis. The review included studies published between 2020 and 2025 that used LLMs for tasks related to coding, thematic analysis, classification, summarisation or interpretation. GPT models were the most commonly used, especially for tasks related to thematic analysis. Some papers prompted models to perform the whole analysis independently, while others treated them as supplementary tools for human led analysis.
The review also found major differences in how studies reported model versions, prompts and validation methods complicating reproducibility of the findings. Overall, the usage of LLMs in qualitative research is promising; however, stronger reporting, validation and ethical guidelines are needed.
This scoping review outlines the way in which transformer-based language models have been used in recent studies to facilitate qualitative analysis. The review included studies published between 2020 and 2025 that used LLMs for tasks related to coding, thematic analysis, classification, summarisation or interpretation. GPT models were the most commonly used, especially for tasks related to thematic analysis. Some papers prompted models to perform the whole analysis independently, while others treated them as supplementary tools for human led analysis.
The review also found major differences in how studies reported model versions, prompts and validation methods complicating reproducibility of the findings. Overall, the usage of LLMs in qualitative research is promising; however, stronger reporting, validation and ethical guidelines are needed.
Event Type
Poster
TimeThursday, October 22nd5:30pm - 6:30pm PDT
Location
Similar Presentations