How would you design a global image CDN with regional replication?

A global image CDN delivers pictures from edge locations close to users while authoritative copies live in multiple regions. The core goal is simple. Keep hot objects near users, fetch from the nearest healthy region on cache miss, and control freshness with versioned URLs. This lowers latency, smooths traffic spikes, and keeps origin load and egress spend under control during real world peaks.

Why It Matters

Image heavy products win or lose on time to first pixel. Regional replication trims cross ocean hops and reduces tail latency. It also improves availability because a single region outage does not take media offline. In a system design interview this topic tests knowledge of cache keys, origin shield, replication lag, TTL policy, failure handling, and cost modeling. In production the payoff is higher conversion, better retention, and healthier margins.

How It Works Step by Step

1. URL schema with immutability

• Encode identity and immutability into the path app media id version format width height
• Use version bumps for edits so old versions remain cacheable
• Add a short lived token for access control and hotlink prevention when needed

2. Upload and write path

• Clients upload to the closest edge or to a regional write endpoint using signed requests
• The write region stores the original in durable object storage and records metadata in a small catalog keyed by content id
• If the product needs variants or thumbnails, kick transforms to workers via a queue and produce AVIF, WebP, and JPEG in common sizes

3. Regional replication strategy

• Enable cross region replication from the write region to read regions using storage level replication rules
• Treat replication as eventual and design read logic that can fall back to the write region
• Use object metadata or a compact manifest so any region can validate the latest version and variants

4. Edge routing and cache behavior

• Use anycast DNS so users land on the nearest CDN edge
• Compute a cache key with resource id, version, requested size, and requested format
• On a miss, fetch from an origin shield near the chosen read region to coalesce requests and protect buckets

5. Nearest healthy region selection

• Prefer the closest region that has the object
• If the replica is missing, fall back to the write region and cache the response at the shield
• Maintain a small health feed that marks regions as degraded and route away when needed

6. Freshness and invalidation

• Choose long TTL for immutable versions and rely on version bumps for updates
• For rare critical fixes where the URL cannot change, push an explicit purge to the CDN and shield
• Support serve stale while revalidate so users see fast responses while freshness is restored in the background

7. Format and size negotiation

• Use accept header hints to return AVIF or WebP when the client supports them
• If generating on demand, cache the derivative at the shield and optionally persist it in the regional bucket to avoid repeated compute

8. Access control and privacy

• For private images use short lived signed URLs
• Keep secrets out of the path and place the signature in the query string
• For public assets do not let the signature reduce cache hit ratio by leaking into the cache key

9. Observability and safety

• Track edge and shield hit ratio, replication lag per region, origin read QPS, and tail latency
• Define SLOs for p95 latency per continent and overall success rate
• Run automated drills that simulate a read region loss and confirm expected failover behavior

10. Cost model and guardrails

• Most spend is bandwidth egress and origin operations on misses
• A high shield hit ratio plus smart size grids reduce miss traffic
• Cap the number of on demand sizes and garbage collect cold variants to control footprint

Real World Example

Consider a social photo app with users in North America, Europe, and South Asia. The write region processes uploads and moderation. Storage replication copies the object to regional buckets in Europe and Asia. A user in Paris requests a profile picture. The request hits a nearby edge and on a miss the edge asks the EU shield. If the EU bucket has the object the shield serves and caches it. If replication has not completed the shield falls back to the write region, caches the response, and subsequent EU requests become hot. If the EU read region is marked unhealthy routing switches to the next closest region. Versioned URLs avoid broad purges. Dashboards show hit ratio and replication lag so the team can tune TTL and shield size.