Experiment Data for Data Engineers

A furniture store is testing a new one-page checkout against its current three-step checkout. The plan sends half of the users to each page. After 14 days, 9,760 users have seen only the current page at checkout (the control), and 10,240 have seen only the new page (the treatment). A fair split of these 20,000 users would put 10,000 on each side, so the new page has 240 users more than planned.

Chance alone can make a split uneven, so the question is how uneven