Rate Limiting Expensive Compute

Two customers each make 60 requests in one minute.

One customer sends short questions and receives short answers. The other sends large documents and requests long outputs. The request counts match, but the compute use differs substantially.

A request limit protects request-handling overhead. It does not, by itself, protect expensive inference. The API must limit the resource being consumed, reserve for output that does not yet exist, and account for a stream that holds capacity until its terminal event.