Design a Code Assistant

Frame the Problem: Two Products With One Name

"Design GitHub Copilot" is really two designs, with opposite constraints.

Separate inline completion from chat assistance before choosing models or designing the request paths.

1. Inline completion. This is the ghost text that appears as you type.

It fires on nearly every pause in your keystrokes, and it has to show something useful in 200 to 400 ms total.

A 200 to 400 ms end-to-end target usually favors a small, optimized model. Benchmark candidates under realistic load instead of assuming that every large model is too slow.