Grokking LLM Evals
0% completed
Robustness to Small Input Changes
If one customer writes "I smell gas in the kitchen" and another writes "gass smell in kitchen!!", should the job ticket assistant give them different urgency levels?
Nobody would say yes. Both customers describe the same danger, and both need an emergency ticket. Yet a test set usually holds each request in one wording only, so it cannot tell whether the assistant would give the same answer to the second customer. Real customers make typos, tell events in a different order, and paste text with odd line breaks.
.....
.....
.....
Like the course? Get enrolled and start learning!
Reading Progress
0%