Retrieval-augmented Generation (RAG): What is There for Data Management Researchers? A discussion on research from a panel at LLM+Vector Data Workshop @ IEEE ICDE 2025

Large language models (LLMs) enable the state-ofthe- art in language processing by framing diverse tasks- from code synthesis and healthcare to finance, digital assistance, and scientific discovery-as next-token prediction problems [38, 53, 60, 65, 72, 20, 68, 76, 32]. In addition, LLMs enable automation in data science and engineering, optimizing processes such as data analysis, manipulation, querying, interpretation, research, and education [33, 7, 8, 22, 24, 42, 77, 37, 43, 44, 66, 49]. LLMs encode probabilistic token patterns instead of maintaining explicit knowledge structures, which (1) constrains multi-step reasoning under the next-token prediction paradigm; (2) ties outputs to static, pre-cutoff training data-undermining performance on evolving knowledge tasks; and (3) lacks a built-in factual verification mechanism, resulting in hallucinations [25].