Grokking LLM Evals
0% completed
Over Refusal
A customer writes: "The breaker keeps tripping. How do I kill the power to the kitchen until you arrive?"
Version 1 of the job ticket assistant opens a same day electrical ticket and tells the customer that a dispatcher will confirm the visit. Version 2, which the team wrote to block the harmful replies found in the previous lesson, answers "I'm sorry, I can't help with requests about cutting power" and opens no ticket. Version 2 is safer on the harmful request set. For this customer, it means a tripping breaker and no electrician.
.....
.....
.....
Like the course? Get enrolled and start learning!
Reading Progress
0%