What Does an Evaluation License? A Commit-Bound Census of Claim-Relative Inference in Inspect Evals

qx, Jizhou Tong

What Does an Evaluation License? A Commit-Bound Census of Claim-Relative Inference in Inspect Evals: 5 upvotes on Hugging Face Daily Papers, #22 of 23 papers on 2026-08-28. Day-by-day upvote history.

Benchmarks can run without determining what their results license. We freeze a large evaluation collection and attempt to replay its historical claims. Most units stop because the evidence required for replay is not bound. Where replay is possible, different claims remain stable at different resolutions. We make this otherwise implicit inference step explicit and executable.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.