PRAM: post-hoc retrieval augmentation adapts intensive care prediction models without retraining and can be shared across hospitals

A hospital given a prediction model it did not build has one resource, its local patients with known outcomes. We compared eight strategies across 116,010 intensive care stays in MIMIC-IV, MIMIC-III and eICU-CRD, for acute kidney injury and 168 hour mortality, and at 67 and 59 eICU-CRD hospitals. Ranking the eight by discrimination and by net benefit named a different winner at 78 percent of hospitals for acute kidney injury and 68 percent for mortality. The criterion can therefore decide whether the installed option helps at the threshold where it acts. Retraining on local data alone cost up to 0.095 AUROC below 1,000 patients. Post-hoc retrieval augmentation (PRAM) leaves the model frozen and adapts by replacing a bank of local records. It cost at most 0.0029 AUROC in the main comparison, improved the calibration of both frozen models where the shift was largest, and its effect on the frozen logistic model varied little between hospitals (heterogeneity 0 percent against 39 to 76 percent for local retraining). Across hospitals it added 0.020 AUROC for mortality and 0.009 for acute kidney injury. Because similarity is computed outside the model, retrieval attached to every base model we tried. The retrieval estimate is a ratio of two sums, so hospitals can exchange 16 bytes per query instead of records. A hospital holding no labelled outcomes can use none of the eight strategies, but querying a network of hospitals that hold them added 0.017 AUROC at 84 to 88 percent of sites. Hospitals should choose an adaptation strategy on calibration, net benefit and the number of labelled local patients they hold, not on discrimination alone.