Grokking LLM Evals
Vote

0% completed

​

Relevance Labels for Retrieval

A technician asks the repair guide assistant: "How do I reset the high limit switch on the WH-50 water heater?" The retriever returns five chunks. Which of them answer the question?

  • Chunk 1, from the WH-50 manual: the reset steps, numbered one to four.
  • Chunk 2, from the WH-50 manual: safety warnings that come after the reset steps.
  • Chunk 3, from the WH-40 manual: the reset steps for an older model with a different switch.
  • Chunk 4, from an internal repair guide: how to tell a tripped switch from a failed one.
  • Chunk 5, from the WH-50 manual: the warranty terms.

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%

Vote for new content