0% completed
Step 2: Requirements
"Generate unique IDs" sounds complete, but it leaves three questions open: must IDs be sortable, may a client make its own, and how slow can a response be before callers suffer? Settling those first gives the four functional requirements below.
Functional requirements
- Generate unique IDs on demand. Clients call an API for a new ID at any time, and no two calls ever return the same ID, even from different services or regions.
- Support time-sortable and opaque IDs. A client chooses the format per request through the
typefield in Step 4. A time-sortable ID reflects creation order; an opaque ID reveals no time or sequence. Both are unique across the whole system. - Simple client API. One REST or gRPC endpoint,
POST /v1/ids, returns new IDs with little overhead. A batch request, for example 100 IDs in one call, lets a client that needs many IDs pay the call overhead once. - No offline client ID generation. Every ID comes from the service. A mobile app cannot run the algorithm offline, because it has no machine number, and two devices generating independently would eventually collide.
The last item is a restriction rather than a feature, and agreeing on it early removes the need to merge or reconcile IDs created by disconnected clients.
Non-functional requirements
These requirements, listed in the diagram, determine how the service is built.
High throughput. The system sustains at least 100,000 IDs per second, and handles peak bursts of 200,000 to 500,000 by adding nodes. Each generator node serves tens of thousands per second.
Low latency. Making an ID takes a few processor instructions, so almost all of a caller's wait is network. Within a region, the service answers in under 5 ms at p99 (99 of every 100 requests complete within that time). A client using the library variant from Step 5 makes no network call and waits microseconds. Every other client gets a faster answer only from a closer node, so routing to the nearest region matters more than any optimization inside the service.
Treat a p99 above 10 ms as a fault rather than a busy period. Nothing in this service does enough work to be slow, so a slow response points to a garbage collection pause, a saturated network card, or a failing node.
High availability and fault tolerance. The target is 99.99 percent uptime with no single point of failure. When a server or a whole datacenter fails, generation continues, with clients failing over to another region if necessary. Other services call this one before they can insert a row, so an outage here stops writes everywhere.
Global uniqueness and a consistent format. The design partitions the ID space so two datacenters never produce the same value. An ID made in Europe has the same structure and length as one made in the United States.
Scalability. Capacity grows by adding generator nodes, in existing or new regions, without major reconfiguration. When traffic doubles, we deploy more nodes; in the rare case that the ID layout itself runs short, we rebalance its bit fields. Neither change is a redesign.
Ordering guarantees. When one time-sortable ID is created after another, it usually has the higher value. "Usually" is the correct word, because clock skew, the difference between two machines' clocks, can invert IDs created a few milliseconds apart on different nodes. Callers accept that: a feed sorted by ID shows two posts from the same instant in either order, and nobody can tell. Duplicates are different. A clock that drifts or jumps may disturb the order, but it must never give two callers the same ID.
ID size. Time-sortable IDs are 64-bit integers, which databases store and index efficiently. Opaque IDs may be 128 bits, the size of a UUID.
Reading Progress
0%