Grokking LLM Evals
Vote

0% completed

​

Labeling Expected Behavior

A customer writes: "The heater has been acting up since the start of winter, and now it won't turn on at all." The job ticket assistant summarizes it as "Heater has had intermittent problems for several months and no longer turns on."

Two dispatchers label the summary for invented detail. The first marks it as a pass, because the summary restates what the customer said in other words. The second marks it as a fail, because the customer never said "several months" or "intermittent". Both are experienced and both read carefully

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%


Vote for new content