Fault Tolerance vs. High Availability

Both promises use redundancy, which means keeping extra copies of important parts. The difference is how the copies are used, and what happens in the seconds after a failure.

How Fault Tolerance Works

A fault-tolerant system is designed so that a failure needs no recovery step that users can see. The copies are already doing the work when a part fails.

Active copies. Several copies of each part do the same work at the same time. When one copy fails, the others are already serving users. Nothing has to start, and nothing has to switch.

Synchronous replication. With synchronous replication, a write counts as complete only after it is saved on at least two machines. If one machine fails right after the write, the data still exists on another one. This is how fault-tolerant systems avoid losing data.

Voting. Some systems run the same calculation on three or more computers and compare the results. The system uses the majority answer. A computer that fails, or gives a wrong answer, is simply outvoted. Aircraft flight control systems work this way, and the quorum chapter explains the same idea for databases.

Redundant hardware. Single machines are built with spare parts inside them. Examples are two power supplies and two network connections. Another example is RAID, which stores data across several disks so that one disk can fail without data loss.

Automatic failover. When a part fails, the system moves work away from it immediately. No person has to act.

All of this costs more. Every part runs as two or three copies, all the time. Synchronous writes are slower, because each write waits for another machine. And the design is harder to build, test, and run.

How High Availability Works

A highly available system also has copies. But it accepts a short pause while it detects a failure and recovers from it.

Redundancy and clustering. Services run as a cluster, which is a group of servers that work together as one system. If one server fails, the others in the cluster keep working.

Load balancing. A load balancer spreads requests across healthy servers. When health checks show that a server has failed, the load balancer stops sending it traffic.

Rapid recovery. A standby copy takes over after a failure. For a database, a replica is promoted to become the new primary. Detecting the failure and switching usually takes seconds, sometimes a minute or two.

Asynchronous replication. Many highly available databases use asynchronous replication. The primary confirms a write first and copies it to the replicas a moment later. Writes are faster, but if the primary fails before the copy arrives, the most recent writes can be lost.

High availability is usually described in nines. For example, 99.99 percent availability allows about 52.6 minutes of downtime per year. The Availability lesson explains the nines and failover in detail.

The main benefit is cost. High availability gives most of the protection for much less money and complexity. It balances cost against the level of availability that the product actually needs.

The Five Key Differences

1. Objective. Fault tolerance aims for continuous operation, so a failure never becomes visible to users. High availability aims for as much uptime as possible, with fast recovery when something fails.

2. Approach. Fault tolerance uses active copies, synchronous replication, and automatic failover, so there is nothing to recover. High availability uses redundancy, load balancing, and fast failover to a standby.

3. Downtime. Fault tolerance allows no downtime, even during a failure. High availability accepts brief interruptions, often seconds to a minute.

4. Cost and complexity. Fault tolerance is more expensive and more complex, because it needs full copies that are always active. High availability is more cost-effective.

5. Data integrity. Fault tolerance keeps all data safe, even during a failure. High availability puts uptime first, so a small amount of recent data can be lost in some failures.

Choosing Between Them

Choose fault tolerance when downtime or data loss can cause serious harm. This is why it belongs in critical systems in finance, healthcare, and aviation. A payment that disappears, a heart monitor that pauses, or a flight computer that stops for 20 seconds is not acceptable.

Choose high availability when a short interruption is acceptable and cost matters. An online store that fails over in 30 seconds loses a few page loads. Users retry, and the business continues.

Two questions help decide.

1. Can a few seconds of downtime cause serious harm or a large loss?
2. Is losing the last few seconds of writes unacceptable?

If the answer to the first question is yes, the part needs fault tolerance. If only the second answer is yes, protect the data with synchronous replication, and use high availability for the rest.

Most systems use both, for different parts. In an online store, the payment records need fault tolerance for the data, with synchronous replication. The product catalog needs high availability. The recommendations service can simply degrade gracefully when it fails. Choosing per part gives strong protection where it matters, without paying for it everywhere.

Key Takeaways

• Fault tolerance keeps a system functioning correctly, without interruption, when parts fail. There is no downtime and no data loss.
• High availability keeps a system operational for a very high percentage of the time. It accepts brief interruptions and recovers quickly.
• Fault tolerance uses active copies, synchronous replication, voting, and automatic failover. It costs more and is more complex.
• High availability uses clusters, load balancing, and fast failover. It is more cost-effective, but recent writes can be lost in some failures.
• Choose fault tolerance for critical systems in finance, healthcare, and aviation. Choose high availability when short interruptions are acceptable.
• Real systems mix both and choose for each part.

These two promises are not two levels of the same thing. They are different trade-offs between cost, downtime, and data safety. That completes this chapter. Next is a Flashcards Review of the key terms, followed by the Chapter Assessment.

Practice Questions

Try each question first, then open the answer.

1. A hospital runs a system that shows live patient heart data to nurses. An online bookstore runs its product pages. Which system needs fault tolerance, and which one needs high availability?

<details> <summary>Show answer</summary>

The patient system needs fault tolerance, and the bookstore needs high availability. If the heart data pauses for 20 seconds, a nurse could miss a dangerous change, so any downtime can cause serious harm. If the bookstore pauses for 20 seconds, a few customers reload the page. A short interruption is acceptable there, and high availability costs much less.

</details>

2. A database uses asynchronous replication, and its replica is about 3 seconds behind. The primary confirms 50 orders, then crashes 2 seconds later. The replica becomes the new primary. What happens to those orders, and what would prevent it?

<details> <summary>Show answer</summary>

Those 50 orders can be lost. The primary confirmed them, but the replica was 3 seconds behind, so they had not been copied yet. Synchronous replication prevents this, because a write is confirmed only after a second machine has saved it. The cost is slower writes.

</details>

3. A highly available service takes 15 seconds to detect a failure and 15 seconds to switch to a standby. It fails 4 times in a 30-day month. How much downtime does it have, and does it meet a 99.9 percent target?

<details> <summary>Show answer</summary>

2 minutes of downtime, so it meets the target. Each failure causes 15 + 15 = 30 seconds of downtime. Four failures give 4 x 30 = 120 seconds, which is 2 minutes. A 99.9 percent target allows 43.2 minutes in a 30-day month. This service even meets 99.99 percent, which allows 4.32 minutes.

</details>

4. An aircraft uses three flight computers that vote on every result. Why not use one active computer and one standby computer instead?

<details> <summary>Show answer</summary>

A standby needs time to detect the failure and take over, and that pause is not acceptable in flight. With voting, all three computers work at the same time. If one fails, the other two are already producing the answer. Voting also catches a computer that gives a wrong answer without crashing, because the other two outvote it.

</details>

5. A new company wants fault tolerance for every part of its app, including search and recommendations. What is the problem with this plan, and what would you suggest?

<details> <summary>Show answer</summary>

It pays for far more protection than most parts need. Fault tolerance needs full active copies, slower synchronous writes, and a more complex design everywhere. A better plan is to choose for each part. Use fault tolerance for data that must never be lost, like payments. Use high availability for the rest, and let features like recommendations degrade gracefully.

</details>