Grokking LLM Evals
0% completed
Generating Synthetic Cases from Dimensions
How would you test whether the job ticket assistant handles a gas smell described half in Spanish, from a customer who never gives an address, if no such request has arrived yet?
Sampled traffic cannot answer that question, because it contains only what customers have already sent. Asking a model for "100 test requests" does not answer it either. The result is often 100 polite, similar messages about common problems, with few of the combinations that cause failures.
The method in this lesson plans the combinations first and lets a model write the wording second
.....
.....
.....
Like the course? Get enrolled and start learning!
Reading Progress
0%