Reduced linguistic coherence in psychosis defies semantic similarity accounts and relates to altered large-scale cortical hierarchy

Coherence in speech is clinically significant in mental disorders but remains difficult to quantify. We tested the widely-held assumption that semantic similarity metrics derived from large language models capture human-rated coherence. Across three large neurotypical datasets in different languages, semantic similarity failed to correlate with human ratings, while six other metrics, especially the probability-based metrics, showed significant but weak correlations. In an additional English dataset of 94 individuals, including healthy controls and patients with schizophrenia spectrum disorders (SSD), speech coherence was reduced in SSD. Incoherence in the drug-naïve first-episode samples related to altered whole-brain intrinsic functional gradients, and to the probabilistic metric of perplexity in speech. Together, these findings call into question semantic similarity as a proxy for coherence, motivate greater emphasis on probabilistic predictability measures for evaluating coherence, and substantiate the perspective of spontaneous speech as an overt readout of an alteration of hierarchical cortical organization in schizophrenia.