Grokking System Design Fundamentals
Ask Author
Back to course home

0% completed

Vote For New Content
Beyond CAP Theorem
Table of Contents

Contents are not accessible

Contents are not accessible

Contents are not accessible

Contents are not accessible

Contents are not accessible

CAP is a simplified model that only considers a binary state (partition or no partition). In real systems, even when there’s no partition, there are still trade-offs — notably between consistency and latency/performance. To address this, researchers have proposed extensions and refinements to CAP.

PACELC Theorem

The most well-known extension is the PACELC theorem, introduced by Daniel Abadi around 2010. PACELC expands on CAP by asking: not only what happens during a Partition (P), but also what happens Else (E) when there’s no partition. In formal terms, PACELC says:

  • If a partition (P) occurs: the system must choose either Availability (A) or Consistency (C) (this is essentially the CAP theorem trade-off during a failure).
  • Else (no partition): the system must choose between Latency (L) and Consistency (C).

In shorthand, “if P then A or C; else L or C.” This acknowledges that even in normal operation, there’s often a tension between giving responses quickly (low latency) and doing extra work to ensure consistency. For example, waiting for data to replicate to many nodes might ensure strong consistency but adds latency to each request. Or you can respond faster from one node’s data (lower latency) at the cost of maybe not being fully up-to-date (slightly weaker consistency).

PACELC thus fills in the gap that CAP leaves: CAP is silent when the system is running well. PACELC says even with no failures, you’re either prioritizing speed or consistency. It effectively adds a second dimension to the design choices.

Let’s illustrate PACELC with examples:

  • Cassandra under PACELC: We said Cassandra is AP in CAP terms. Under PACELC, we classify it as PA/EL – Partition tolerant + Available, and Else (when no partition) it prefers low Latency over full consistency. Cassandra’s design optimizes for fast writes and reads (low latency) rather than blocking to synchronize every replica on each request. So it’s Available under partition, and in normal times it’s still favoring performance (with eventual consistency). In PACELC notation: PA/EL.

  • MongoDB under PACELC: MongoDB was CP in CAP. Under PACELC, it’s often described as PC/EC – Partition tolerant + Consistent, and Else Consistent. That means even with no partition, MongoDB (by default) chooses consistency over latency (it funnels writes through one primary and replicates synchronously to secondaries). During a partition, it sacrifices availability to keep data consistent (only a majority partition can accept writes). So MongoDB is PC/EC by default configuration.

  • Google Spanner: Google Spanner is a globally-distributed SQL database. It aims to be strongly consistent and highly available, by using synchronized clocks (TrueTime) to minimize uncertainty. According to PACELC, Spanner can be seen as PC/EC (much like MongoDB – it prioritizes consistency in all cases). In CAP terms, Spanner is technically a CP system (it will sacrifice availability if a major partition occurs), but through engineering (and controlling clock drift) Google made partitions rare and short, so in effect Spanner behaves like a CA system for practical use. Spanner chooses consistency even if that means a bit more latency (transactions wait out clock uncertainty), thus it fits PACELC’s PC/EC category (favoring consistency both during and outside partitions).

The PACELC theorem reminds us that performance (latency) is part of the equation. A system that is strongly consistent even without failures might incur higher response times, while a system that gives snappy responses might be doing so by not waiting on cross-node consensus.

PACELC Theorem
PACELC Theorem

c. CRDTs and Hybrid Systems

Convergent Replicated Data Types (CRDTs) are data structures designed to allow multiple replicas to be updated independently and converge to a consistent state without requiring coordination. CRDTs can help system designers achieve both strong eventual consistency and high availability. By combining CRDTs with other techniques, it is possible to build hybrid systems that provide tunable consistency guarantees, enabling applications to make trade-offs based on their specific requirements.

d. Application-specific trade-offs

The CAP theorem and its extensions provide valuable insights into the fundamental trade-offs in distributed systems design. However, it is crucial to remember that real-world systems often involve more complex and application-specific trade-offs. As a system designer, it is important to understand the unique requirements and constraints of your application and make informed decisions about the trade-offs that best meet those needs.

Summary

While the CAP theorem has been foundational to understanding the trade-offs in distributed systems, it is essential to explore and consider more nuanced models and techniques to design systems that effectively address the challenges and requirements of modern applications.

.....

.....

.....

Like the course? Get enrolled and start learning!

Table of Contents

Contents are not accessible

Contents are not accessible

Contents are not accessible

Contents are not accessible

Contents are not accessible