Grokking LLM Evals
0% completed
Faithfulness and Groundedness
import json SPLIT = """Split this answer into short, separate factual claims. Return a JSON list of strings, one claim per item. Answer: {answer}""" CHECK = """Passages from repair manuals: {passages} Claim: {claim} Using only the passages, is the claim SUPPORTED, CONTRADICTED or NOT_MENTIONED? Write one sentence of reasoning, then the label alone on the last line.""" def call_model(prompt): # Send the prompt to your model provider and return the reply text. # This needs your own API key, so run it on your own machine. raise NotImplementedError
.....
.....
.....
Like the course? Get enrolled and start learning!
Reading Progress
0%