Grokking the Anthropic Forward Deployed Engineer Interview
0% completed
Grading With Code, People, and Model Graders
Three kinds of grader can score an evaluation set, and each has a clear limit:
- Code compares an output with a known answer in milliseconds and gives the same result every time. It cannot judge whether a memo explains a difference clearly.
- People can judge any output, including tone and clarity. They are slow, they cost the most, and two of them do not always agree.
- A model grader reads an output with written criteria and scores it in seconds. It can judge memos, but it can be wrong in ways nobody notices until it is checked against people.
.....
.....
.....
Like the course? Get enrolled and start learning!
Reading Progress
0%