Grokking LLM Evals
Vote

0% completed

​

Faithfulness and Groundedness

import json SPLIT = """Split this answer into short, separate factual claims. Return a JSON list of strings, one claim per item. Answer: {answer}""" CHECK = """Passages from repair manuals: {passages} Claim: {claim} Using only the passages, is the claim SUPPORTED, CONTRADICTED or NOT_MENTIONED? Write one sentence of reasoning, then the label alone on the last line.""" def call_model(prompt): # Send the prompt to your model provider and return the reply text. # This needs your own API key, so run it on your own machine. raise NotImplementedError

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%


Vote for new content