Code Models

A common belief about AI-written code is "If it passes its tests, it works." The belief has a sound basis because a passing test is real evidence, and tests catch many mistakes. However, a test checks only the cases that someone thought to write down.

Consider one change. An engineer asks an AI assistant to add coupon support to a checkout service. The first test run crashes on a function that doesn't exist, and the engineer fixes it in a minute. After that, every test passes, including the tests the assistant wrote