Shiyi Xu
The integration of Large Language Models (LLMs) into clinical workflows has blurred the boundary between passive decision-support tools and systems capable of autonomous diagnostic reasoning. Current regulatory instruments-including the US FDA Clinical Decision Support Software (CDSS) guidance and the EU Artificial Intelligence Act (Regulation EU 2024/1689)-classify these systems primarily as "decision support," yet the non-deterministic outputs and emergent reasoning of generative AI introduce liability complexities these frameworks were not designed to address. This hypothesis and theory article examines the regulatory gap surrounding LLM-associated clinical errors. Drawing on regulatory precedents, case law, and a representative scenario of polypharmacy mismanagement, we analyze the risks of automation bias and the structural inadequacy of the Learned Intermediary Doctrine when applied to generative AI. We advance the autonomy-liability correspondence hypothesis: legal liability should be allocated as a monotonic function of a system's structural autonomy from meaningful human review. From this we derive a "Three-Tiered Liability Escalation Framework" (T-LEF) that allocates accountability among AI developers, healthcare institutions, and clinicians according to measurable degrees of autonomy. Arguing that traditional Explainable AI is technically insufficient for foundation models, we propose Clinical Algorithmic Audit Trails (CAAT)-a cryptographically secured, privacy-preserving traceability infrastructure-as a complement to explainability, uncertainty estimation, human-factors evaluation, and post-market surveillance, alongside a structured Safe Harbor provision to support compliant innovation. Because paediatric patients-from neonates to adolescents-constitute a population in which these failure modes are both more probable and more consequential, we give paediatric care explicit focus: under-representation of paediatric data in training corpora, weight- and development-based dosing across narrow therapeutic windows, and children's inability to detect or contest erroneous recommendations jointly intensify these concerns, and a dedicated section examines how the T-LEF, the Clinically Deceptive Hallucination (CDH) evidentiary test, and CAAT apply to paediatric deployment. We acknowledge that the T-LEF and CAAT remain prospective proposals requiring empirical validation, stakeholder consultation, and refinement before adoption. This framework is offered as a structured contribution to regulatory deliberation, not a finalized implementation blueprint.