Unique ID Generator: System Definition

Step 1: System Definition

Insert a row into a database and something has to give it an identifier. On one machine that is easy. The database counts up, and every new row takes the next number.

Now run that same code on many servers, across several regions. Each counts up on its own, so two will hand out the same number. This case study designs the service that prevents that.

A Globally Unique Identifier (GUID) generation system is a service that makes unique identifiers at scale. Many services and data centers ask it for IDs at the same time. No two IDs it hands out are ever the same.

Those IDs can be database primary keys, event or message IDs, or object handles. Any place that needs one distinct label per entity can use them.

The system supports two ID formats: opaque IDs and ordered IDs.

An opaque ID is a random or pseudo-random value, like a UUID. It carries no order and no meaning. That makes it useful when an ID must not reveal anything, or when a unique label is all that is needed.

An ordered ID encodes time or a sequence. A newer ID is larger than an older one, whether compared as numbers or as strings. That makes ordered IDs useful for sorting, and the sort order roughly follows creation order.

Two phrases in that definition need a precise meaning. "Global uniqueness" means an ID made on any node, at any time, differs from every other ID. "Web-scale" means very high request rates, hundreds of thousands per second, across many regions and services.

The diagram below shows the main entities and how a request flows between them. A few key terms follow it.

UUID (Universally Unique Identifier). A UUID, called a GUID in Microsoft terms, is a 128-bit value used to label objects. Common versions like UUIDv4 are made from random or pseudo-random numbers. They are opaque: by design they carry no time or order, only a statistically unique random value.

With 2^128 possible values, collisions are practically negligible. Even at 1 billion UUIDs per second for 100 years, the chance of a single collision is only about 50 percent. That is why a UUID generator checks nothing before returning a value. UUIDs need no coordination between generators.

That freedom has a cost. UUIDs do not sort naturally, and they are relatively long.

Snowflake ID. A 64-bit ordered ID format introduced by Twitter. It is made of time bits, machine bits, and sequence bits. The time bits make these IDs sort by creation order. Many distributed ID schemes, like Instagram's and Discord's, are variants of this pattern.

ULID and KSUID. Newer GUID formats that sort correctly as strings. They combine a timestamp with random bits. The result is an ordered ID that needs no coordination: 128 bits for ULID, 160 for KSUID.

Monotonic. Here it means an ID sequence that only goes up within one scope. An ID made later has a higher value than one made earlier. Full global monotonic order is hard to guarantee without a single source of time. But each generator, or each timeline, produces a monotonic sequence of its own. That is the strongest ordering promise this design can make.

Database auto-increment keys. Many databases can auto-increment a numeric primary key. That gives unique, sequential IDs, easy to sort by creation order within one table. In a distributed system with many databases or shards (pieces of one database spread over machines), one shared sequence is hard to coordinate.

Traditional auto-increment is a single-node solution. Used globally, it becomes a single point of failure and a bottleneck at high throughput. Every request updates the same row, and one failure stops every ID. Separate sequences per shard produce duplicates across shards, unless each shard is given its own pre-assigned range.

Use cases. This system is used whenever many distributed parts need unique references without a central bottleneck. For example, web servers make database keys for new records, microservices tag events for tracing, and IoT devices label sensor readings. None of them risks a collision.

Offering both formats covers a wide range of needs. Opaque IDs give unpredictable, secure identifiers. Time-ordered IDs give sorted, timeline-friendly keys.

Next: Step 2, where the requirements are defined.