Chapter Assessment: LLM Judges

This review covers LLM judges for the job ticket assistant, from how a judge is asked a question to how the invented detail judge was written and revised. It also covers the three reported biases, validation against the dispatcher's labels, the corrected pass rate, and pairwise win rates. The bank holds 20 questions, and each attempt draws ten. Most questions describe a decision the team faces with a judge, and several ask you to compute a rate. Keep these at hand: the split of 160 labels into 60 and 100, the TPR of 0.92 and TNR of 0