Thematic analysis with open-source generative AI and machine learning: a new method for inductive qualitative codebook development

Abstract This work aims to answer one central question: to what extent can open-source generative text models be used in a workflow to approximate steps in thematic analysis in social science research? To answer this question, we present the Generative AI-enabled Theme Organization and Structuring (GATOS) workflow, which uses open-source machine learning techniques, natural language processing tools, and generative text models to facilitate aspects of thematic analysis. To establish evidence of validity of the method, we present three case studies applying the GATOS workflow, leveraging these models and techniques to inductively create codebooks similar to traditional procedures using thematic analysis. We show that the GATOS workflow can identify themes in the text that were used to generate the original synthetic datasets. We conclude with a discussion of relevant considerations, the implications of this work for social science research, and the tradeoffs of using open-source generative text models to facilitate scalable qualitative data analysis.