科研速览 · Science Skim继续刷下去 · Keep skimming →
◇ arXiv2026-09-11· cs.IR

A Historical Corpus Is Not a Historical System: Auditing Hindsight Leakage in Stateful Data Discovery

Yixi Zhou, Fan Zhang, Sikun Wang, Yingfan Xu, Haipeng Zhang

原始摘要(英文原文)· Original abstract
Offline replay should estimate what a discovery system could retrieve at a historical point, yet freezing the corpus leaves interaction memory unconstrained. We formalize point-in-time (PIT) discovery through historical state $(D_t, θ_t, M_{< i})$ and introduce a paired replay that changes only memory availability. The protocol constructs PIT and full-stream Future views from behavior-only traces and audits selected entries with a Temporal Violation Rate. Across three table-text domains, two stream regimes, two retrievers, and five seeds (216,000 rows), Future inflated Asset Recall@100 by 2.62-5.24 points; all 12 paired intervals excluded zero. With behavior-only trace memory, PIT underperformed the no-memory Stateless condition; Future masked 32.7-48.4% of that harm. For a simulated positive-feedback cache, PIT added 4.65-18.96 points over Stateless while Future added another 4.11-9.58 points. On five timestamped FreshStack topics, Future exceeded PIT by 2.72 points [1.75, 3.71]. Historical evaluation must version and validate memory with the corpus.
读原文 · Read the paper ↗

AI 追问PRO

登录后使用 AI 追问

讨论区

登录后参与讨论

相关论文 · Related

A Historical Corpus Is Not a Historical System: Auditing Hindsight Leakage in Stateful Data Discovery — 科研速览 Science Skim