Designing a URL Shortening Service like TinyURL

Designing a URL Shortener (video)

Now compare your first design with the video. Look for decisions that follow from a specific requirement.

Step 1: Clarify the Requirements

💡 A design is only correct for a specific set of requirements. Confirm the scope before drawing the architecture.

Suppose the interviewer says, "Design TinyURL." Do not start with servers or databases.

First, ask what the service must do. Then ask which qualities matter most.

Functional requirements describe what users can do:

1. Given a long URL, the service returns a short link.
2. Following a short link redirects the user to the original URL.
3. Users can choose a custom alias.
4. Links expire after a default period. Users can also choose an expiry time.
5. Link owners can view click statistics.

Analytics belongs in the main list. It changes two later decisions.

First, we must observe every click. Second, click processing must not slow down redirects.

Extended requirement

A link can be private. Only permitted users may open it.

Non-functional requirements describe how the service should behave:

1. High availability. Existing links should keep working when parts of the system fail.
2. Low redirect latency. A redirect adds an extra network request, so it must be fast.
3. Hard-to-guess links. Random guesses should rarely find another user's link.

We will not design billing, paid plans, link previews, or destination scanning. These features are outside the agreed scope.

The requirements now give us tests for every later choice. For example, a predictable key would fail the third non-functional requirement.

Step 2: Estimate the Scale

💡 Estimate only the numbers that may change the design. Round the inputs so the arithmetic stays easy to check.

We need two starting assumptions:

1. How many new links are created each month?
2. How many redirects happen for each new link?

Assume 500 million new links per month and 100 redirects for every write.

Every estimate below follows from these two numbers.

Request load

A day has about 100,000 seconds. This rounded value makes the request-rate calculation simple.

500M links / 30 days           = about 17M writes per day
17M / 100,000 seconds          = about 170 writes per second
Rounded write rate             = 200 writes per second
200 writes x 100 redirects     = 20,000 reads per second
20,000 reads x 3 peak factor   = 60,000 peak reads per second

The service is read-heavy. Redirects happen about one hundred times more often than link creation.

Storage

Assume that we keep each link for five years. Also assume that one stored record uses about 500 bytes.

500M x 12 months x 5 years     = 30 billion links
30 billion x 500 bytes         = about 15 TB

Fifteen terabytes will not fit on one database machine. We will need to divide the data across machines.

Bandwidth

Each request moves a small amount of text.

200 writes x 500 bytes         = about 100 KB per second in
20,000 reads x 500 bytes       = about 10 MB per second out

Ten megabytes per second is modest. Bandwidth does not require a special design here.

Cache size

Popular links receive most of the traffic. Assume that 20 percent of requested links produce 80 percent of redirects.

20,000 x 100,000 seconds       = about 2 billion reads per day
20% x 2 billion x 500 bytes    = about 200 GB

This is an upper bound. Many requests open the same popular links, so the real hot data set will be smaller.

What the estimates tell us

Three results affect the architecture:

• Thirty billion rows require partitioning. One machine cannot store all the data.
• The 100-to-1 ratio makes caching valuable. A cache can remove most reads from the database.
• Bandwidth is not the limit. The system moves URLs and small records, not large files.

The purpose of estimation is not precision. It is to discover which parts of the system need special treatment.

Step 3: Define the API

💡 The API turns requirements into concrete operations. Missing fields or responses often reveal a missing design decision.

We need four main operations: create a link, follow it, read its statistics, and delete it.

Create a short link

POST /urls
Authorization: Bearer <token>        optional for public links
Idempotency-Key: <unique value>

body: {
  original_url,
  custom_alias?,
  expires_at?,
  visibility?
}

201 Created: {
  short_url,
  short_key,
  expires_at,
  visibility
}

An idempotency key identifies one creation request. Retrying the same request must not create a second short link.

The client does not send a trusted user_id. The gateway validates the token and derives the user identity from it.

If a custom alias already exists, return 409 Conflict. If the input is invalid, return 400 Bad Request.

Follow a short link

GET /{short_key}

302 Found:       Location: <original_url>
401 Unauthorized: sign-in is required for this private link
403 Forbidden:    the signed-in user cannot open this link
404 Not Found:    the key does not exist
410 Gone:         the link has expired

Use 302 Found rather than 301 Moved Permanently. Browsers may cache a 301 and stop sending later clicks to our service.

A 302 keeps every click visible. This preserves the analytics requirement from Step 1.

Read click statistics

GET /urls/{short_key}/analytics?from=<time>&to=<time>
Authorization: Bearer <token>

200 OK: {
  total_clicks,
  clicks_by_day,
  top_referrers,
  top_countries
}

Only the link owner may read these statistics. The counts may lag by a short time because analytics can be eventually consistent.

Delete a link

DELETE /urls/{short_key}
Authorization: Bearer <token>

204 No Content

Only the owner may delete a link. A later redirect to that key must not return the old destination.

Limits

Rate-limit creation per account. Rate-limit anonymous creation and redirects per IP address.

These limits stop one caller from consuming the key pool. They also make large-scale key guessing harder.

The API now covers every functional requirement. We can use it to decide what data the system must store.

Step 4: Define the Data Model

💡 Start with the reads and writes. They tell you which fields and indexes the system needs.

The main lookup is simple. A redirect receives a short key and needs the original URL.

Each URL record stores:

• short_key: the primary key used by redirects.
• original_url: the destination returned in the redirect.
• user_id: the owner of the link.
• created_at: when the link was created.
• expires_at: when the link stops working.
• visibility: whether the link is public or private.

Private links also need permission data. Custom aliases use the same short_key field as generated keys.

Click events do not belong in this record. Updating a click counter on every redirect would make one popular row a write bottleneck.

Choose the database from the access pattern

The database must store billions of small records. Most requests read one record by its primary key.

A distributed key-value or wide-column database fits this pattern. DynamoDB and Cassandra are common examples.

These databases divide records across many machines. They also support fast primary-key lookups without joins.

The user_id field does not require a relational database. The redirect path never joins the URL record with a user table.

The service needs one secondary access pattern: list the links owned by one user. An index on user_id can support that query.

Store click events separately. Analytics needs large scans and grouped counts, while redirects need single-key lookups.

One database does not need to serve both patterns.

Step 5: Draw the High-Level Design

Start with the two user actions. One action creates a link, and the other follows it.

These actions form separate paths through the system.

The write path

The write path handles about 200 requests per second.

1. A request enters through the API gateway.
2. The shortening service obtains an unused short key.
3. The service inserts one URL record.
4. The service returns the complete short URL.

The write rate is modest. The hard problem is giving every write a unique key.

The read path

The read path handles about 20,000 requests per second.

1. A request enters through the API gateway.
2. The redirect service looks for the key in the cache.
3. On a cache miss, it reads the URL database.
4. It returns a redirect to the original URL.

Reads outnumber writes by one hundred to one. The cache and read replicas therefore belong on the read path.

The two paths may begin as one application. We can deploy them separately when their scaling and availability needs become different.

Step 6: Go Deep

Choose the key length

Each generated key uses characters from a fixed alphabet. A larger alphabet gives more keys at the same length.

Suppose we use the 62 characters in A-Z, a-z, and 0-9.

Six characters provide about 56.8 billion possible keys. Our estimates require 30 billion stored links.

30 billion / 56.8 billion = about 53 percent

More than half of the six-character keyspace would be occupied. A random guess would often find a real link.

That result fails the hard-to-guess requirement from Step 1.

Process a redirect

The redirect service follows these steps:

1. Look for the short key in the cache.
2. Read the database after a cache miss.
3. Check whether the record exists.
4. Check its expiry time and permissions.
5. Return the redirect.
6. Publish a click event without waiting for analytics.

Each failed check has a different response:

• Return 404 Not Found when the key never existed.
• Return 410 Gone when the link has expired.
• Return 401 Unauthorized when a private link requires sign-in.
• Return 403 Forbidden when the signed-in user lacks permission.

Otherwise, return 302 Found with the destination in the Location header.

Choose between 301 and 302

Both responses redirect the browser. Their caching behavior is different.

A 301 says the redirect is permanent. Browsers may cache it and stop contacting our service.

This reduces traffic, but later clicks disappear from our analytics.

A 302 keeps the redirect temporary. Each click still reaches the redirect service.

Use 302 because click statistics are a requirement. Without per-click analytics, 301 would reduce more traffic.

Cache popular links

A cache stores frequently used records in memory. Memory reads are much faster than database reads.

Each cache entry contains the original URL, expiry time, and visibility data. The redirect service can complete every check from this entry.

On a cache miss, read the database. Then place the returned record in the cache.

URL records are easy to cache because their destinations never change. A delete operation removes the cache entry, and every hit checks the expiry time.

Begin with the 200 GB estimate from Step 2. Adjust the size after measuring the real cache hit rate.

Use least recently used, or LRU, eviction. LRU removes the entry that has gone unused for the longest time.

Cache missing and expired keys for a short time too. This is called negative caching.

Negative caching stops repeated requests for dead keys from reaching the database.

Partition the URL data

Thirty billion rows need many database machines. Partitioning divides the rows among those machines.

Hash the short key and map the result to a partition. Random keys will spread evenly across the partitions.

Use consistent hashing to reduce data movement when machines change. Only a small part of the key range moves after adding a machine.

Range partitioning is less useful here. Redirects never request an ordered range of short keys.

Custom aliases can also make first-character ranges uneven. Hash partitioning avoids this imbalance.

Replicate each partition

Keep at least two copies of every partition on other machines.

Replicas protect the data after a machine failure. They can also serve reads and reduce load on the primary copy.

Managed databases like DynamoDB and Cassandra perform partitioning and replication for the application.

Remove expired links

Checking the whole database for expired rows would create constant work. Use lazy cleanup instead.

• When a redirect finds an expired link, return 410 Gone and schedule its removal.
• Run a low-priority background job to remove expired links that receive no traffic.

Some expired rows may remain stored for a while. The redirect service still refuses to serve them.

Never reuse an expired short key. Old emails, bookmarks, and printed links may still contain it.

Reusing that key could send an old link to a new owner's destination. The large keyspace makes reuse unnecessary.

Record analytics outside the redirect path

Click processing must not increase redirect latency. Use an asynchronous event pipeline.

The redirect service returns 302 Found first. It then publishes a click event to a queue like Kafka.

Each event may contain:

• the short key,
• the event time,
• the referrer,
• the user's country,
• and the user agent.

Analytics consumers read events in batches. They write grouped counts into a store designed for analytics queries.

The event stream is manageable:

20,000 events per second x 200 bytes = about 4 MB per second

The queue separates redirects from analytics. A slow analytics consumer creates a backlog, but redirects continue working.

Click counts become eventually consistent. They may appear seconds or minutes after the real click.

This delay is acceptable for reports. Fast redirects are more important than immediate counters.

Control abuse

Abuse means using the service beyond its allowed limits.

Rate-limit link creation per account and per IP address. This stops one script from filling the database or exhausting prepared keys.

Rate-limit redirects per IP address too. This makes repeated guessing slower.

The API gateway applies these limits before requests reach the services.

Check access to private links

Access control decides whether one user may open a specific link.

Store the visibility flag and permission data with the URL record. The redirect service can then check access without calling another service.

Keep abuse controls and access control separate. They solve different problems.

Step 7: Find Bottlenecks and Failure Points

A complete design also explains how it behaves when a part becomes slow or unavailable.

Cache failure

A large cache failure sends many requests to the database. This sudden load can overwhelm the database.

Partition and replicate the cache. Restore traffic gradually while failed cache nodes recover.

Database failure

A failed database machine should not remove a partition. Promote a healthy replica and rebuild the missing copy.

Redirects may read from replicas. New writes may pause briefly while the database selects a new primary.

KGS failure

The standby KGS replaces a failed primary. Shortening servers continue using their local blocks during the change.

If every block becomes empty, only new link creation stops. Redirects do not depend on KGS.

Analytics backlog

A slow consumer increases the number of events waiting in the queue. Click reports become older, but redirects remain fast.

Add consumers when the backlog grows. The queue keeps the events until consumers process them.

Keep the redirect path independent

The redirect path depends only on the cache and URL database.

KGS, analytics, cleanup, and destination screening can all fail without stopping existing links.

This separation is the most important availability property in the design.

Putting It Together

Read the final diagram as three flows.

Create flow

1. The API gateway checks authentication and rate limits.
2. The shortening service takes a key from its local block.
3. The service writes the URL record to the database.

Redirect flow

1. The redirect service checks the cache.
2. It reads the database only after a cache miss.
3. It checks expiry and access rules.
4. It returns 302 Found.

Background flow

The redirect service publishes click events to the queue. Analytics consumers process them later.

KGS refills key blocks, and the cleanup job removes expired rows. Neither task sits inside the redirect request.

Where AI Fits in This Design

The core URL shortener does not need AI. Key generation, caching, and redirects follow fixed rules.

AI can help with destination screening, which is outside the core scope. Short links hide their final destinations and can be used for phishing.

A background worker can send each destination to a classifier. A classifier is a model that labels content, such as safe or suspicious.

Run this check after link creation. Do not place the model inside the redirect path.

The worker stores the result as a flag on the URL record. The redirect service already reads flags for expiry and visibility.

Model errors need a safe response. A false positive marks a safe page as suspicious.

Show a warning page or request human review after a positive result. Do not silently delete the link.

💡 In the interview: spend most of your time on key generation and redirects. Explain why six characters fail the stated security requirement. Then explain why KGS moves collision checks out of user requests. If asked about redirects, connect 302 directly to click analytics.

Key takeaway: this system has about one hundred reads for every write. Use seven-character base62 keys because six characters are too easy to guess at this scale. Pre-generate keys and give them to servers in blocks. Cache popular URL records before a partitioned and replicated database. Return 302 so every click reaches the service. Send click events through a queue, so analytics never delays a redirect. Keep background services outside the redirect path, so existing links remain available during other failures.