Many Comparisons and Slices

An engineer tries 20 rewordings of the job ticket assistant's prompt on the 300 development cases. The best one passes 254 cases, against 240 for the current prompt, and its paired test gives a p-value of 0.035. From that result, the team can promise three things:

• On these 300 cases, the best variant passed 14 more cases than the current prompt.
• If this variant had been the only one tried, a gain this large would rarely appear by chance.
• The 15 cases the variant broke can be listed and read one by one.

It cannot promise three others