Yasmin A. Carruthers, Johannes Heim
Abstract What makes artificially generated podcasts feel natural, and what are their current limitations? In this exploratory study, we compare podcasts generated by Google’s NotebookLM Audio Overview feature with naturally produced podcasts to explore two key ingredients of human dialogue, turn-taking timings and contextually appropriate role behavior. Humans excel at predicting where a previous speaker ends their turn, which allows them to chime in with minimal overlap or gap between speaker turns. Humans also follow unwritten rules of conversation to avoid redundancy, provide relevant information, and adhere to socially appropriate behavior within the dynamics of an unfolding conversation. These key ingredients to human conversation pose significant challenges to artificial dialogue generation despite recent advancements in speech recognition, appropriateness of content delivery, and voice production. NotebookLM overcomes some of the traditional pitfalls of spontaneous dialogue generation through avoiding endpoint prediction by including turn-taking and backchanneling in its script writing. It does so by strictly adhering to avoidance of overlaps or gaps. This is unlike what we observe in human podcast dialogues. Where humans show an acute awareness of the changing dynamics between hosts and guests in conversation, and even overlap to show engagement, AI shows little flexibility. We argue that naturalness in dialogue requires a contextually appropriate and addressee-oriented approach to respond to how a conversation unfolds and not just getting the timings right. A novel human intervention feature shows that pre-determining role behavior and turn taking only goes so far in generating natural dialogue: NotebookLM clearly hits its limits when responding to unpredictable intervention. Variable turn-taking timings and adapting conversational role behavior in spontaneous conversation remain unmatched human skills that are indispensable for natural dialogues.