Grokking LLM Evals
Vote

0% completed

​

The Bootstrap

Write the result of each test case on a card and put all 300 cards in a box. Draw one card, write down what it says, and put it back. Repeat this 300 times, and you have a new list of 300 results. Some cards appear twice or three times in it, and some do not appear at all.

That new list is what a second test set of the same size might have looked like. Draw a few thousand such lists, compute the same number on each one, and the spread of those numbers shows how much the original number could have varied. This is the whole method, and a computer does the drawing in under a second.

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%


Vote for new content