Grokking LLM Evals
0% completed
Writing a Judge for One Failure Mode
You are checking one summary written for a home repair technician. Decide whether the summary contains an invented detail. An invented detail is a fact about the problem, the appliance or the home that neither the customer's request nor the account record states. Count as invented: - a brand, model number, age or size the customer did not give - a cause of the fault, such as "worn seal" or "failed valve" - a room, floor or location the customer did not name - a symptom the customer did not describe Do not count as invented:
.....
.....
.....
Like the course? Get enrolled and start learning!
Reading Progress
0%