The Research Agent Eval Plan

Suppose the model behind the upgrade research agent is replaced tomorrow. Which numbers would you need to see before you let engineers act on its answers again?

A short list would include whether it still finds every breaking change, whether its citations still point to the right passages, and whether it is right on every run rather than on only some. It would also ask what each answer costs and whether a comment on an issue thread can still steer it. The earlier lessons of this design produced each of those numbers