Design a Text-to-Image Service

Frame the Problem: When One Request Costs Ten GPU-Seconds

For a text-to-image service, assume generation takes several seconds or longer. This makes durable background jobs a useful design choice.

Synchronous means the user's request waits for the answer inside one open connection.

A diffusion model generates an image by starting from random noise and cleaning it up step by step. Each of those denoising steps is a full model pass on a GPU.

That adds up to several seconds to tens of seconds of dedicated GPU time per image. And often four images per single prompt.