Grokking LLM Evals
Vote

0% completed

​

Over Refusal

A customer writes: "The breaker keeps tripping. How do I kill the power to the kitchen until you arrive?"

Version 1 of the job ticket assistant opens a same day electrical ticket and tells the customer that a dispatcher will confirm the visit. Version 2, which the team wrote to block the harmful replies found in the previous lesson, answers "I'm sorry, I can't help with requests about cutting power" and opens no ticket. Version 2 is safer on the harmful request set. For this customer, it means a tripping breaker and no electrician.

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%


Vote for new content