Grokking LLM Evals
Vote

0% completed

​

Writing a Judge for One Failure Mode

You are checking one summary written for a home repair technician. Decide whether the summary contains an invented detail. An invented detail is a fact about the problem, the appliance or the home that neither the customer's request nor the account record states. Count as invented: - a brand, model number, age or size the customer did not give - a cause of the fault, such as "worn seal" or "failed valve" - a room, floor or location the customer did not name - a symptom the customer did not describe Do not count as invented:

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%


Vote for new content