Leaderboards and Arena Rankings

"The questions within these benchmarks are not open-ended, hindering the ability to capture the flexible and interactive use found in real-world settings."

The authors of Chatbot Arena wrote that sentence about fixed benchmarks with answer keys, such as those in What a Public Benchmark Measures. Their answer was a public leaderboard built from people's own questions and votes. It has no fixed question list to leak, no answer key to contain errors, and no maximum score to reach.