Grokking LLM Evals
0% completed
Counting and Ranking Failure Modes
Two failure modes from the job ticket sample have exactly the same count: 5 traces each out of 120. In one, a customer wrote that water was dripping through the hall ceiling, and the ticket said same day. In the other, a customer forgot to give the address of a rental apartment, and the reply did not ask for it.
The second failure costs a dispatcher a phone call and the customer a few minutes. The first leaves water coming through a ceiling for hours while the ticket waits in the same-day queue. A list sorted by count alone would place them side by side.
.....
.....
.....
Like the course? Get enrolled and start learning!
Reading Progress
0%