Stateful vs Stateless Architecture

Stateless Servers: The Usual Default

In a stateless design, the state is kept in one of three places.

• A shared store. A database, or a cache like Redis, holds each user's session, which is the record of who a logged-in user is. The request carries a session ID, usually in a cookie, and any server can look the session up. This is the most common approach.
• A signed token. The client carries a token that proves who the user is. The server checks the token's signature, so it does not need to look anything up. But a signed token is hard to cancel before it expires, so these tokens are usually short-lived.
• The request itself. Everything needed to answer the request is inside the request. This is what people mean when they say REST is stateless.

An online banking application shows why this is confusing. The bank clearly remembers who you are. But your session is in a shared store, found by a cookie, so any application server can serve your next request. The system has state, but the servers do not.

What Stateless Gives You

• Any server can handle any request. No server holds data that the others lack, so the servers are interchangeable.
• Simple scaling. A new server can take traffic immediately, because nothing needs to be copied to it first.
• Cheap failures. If a server crashes, the load balancer sends its requests to other servers. No sessions are lost, and nobody is logged out.
• Safe deployments. The team can restart servers one at a time, because no server holds anything that must be kept.
• No sticky sessions. A sticky session is a load balancer rule that sends each user back to the same server every time. It is only needed when state is kept inside one server. Sticky sessions make load uneven, and when that server crashes, its users lose their data.

What Stateless Costs You

Stateless servers are not free. The state has to live somewhere, and that has costs.

• Repeated data in every request. The client must send authentication details and context with every request. For example, suppose a signed token is 1 KB, and the service receives 2,000,000 requests per day. The tokens alone add about 2 GB of data per day.
• An extra lookup. With a shared store, a server may read the session on every request. Inside one data center, a Redis lookup usually takes about a millisecond. That is small, but it is not zero.
• A new critical component. If the shared session store goes down, no server can find any session. So the store must be highly available too.

For most web and API servers, these costs are much smaller than the benefits. That is why stateless is the usual default.

Stateful Servers: The Exception

A stateful server keeps data in its own memory between requests. This makes it resource intensive, because each session uses memory on that server. For example, 100,000 sessions at 50 KB each use about 5 GB of one server's memory.

A few real cases justify stateful servers. The reason is usually the same: the state changes too fast to store somewhere else, or the open connection is itself the state.

• Chat and live-update gateways. A WebSocket is a long-lived connection between a client and one server, used for live updates. The open connection cannot be moved or shared. A message for a user must reach the exact server that holds that user's connection. The Polling vs. Long-Polling vs. WebSockets vs. Webhooks lesson covers these options.
• Real-time game servers. A match changes many times per second. Writing every action to a database would be far too slow. So the match state stays in memory, and players are routed to the server that runs their match.
• Older applications with sessions in memory. Some older systems keep each user's session in the application's own memory. These systems need sticky sessions to work at all.
• Stream processors and coordination services. A stream processor like Apache Flink keeps running totals in memory as events arrive, and saves them regularly. Coordination services like ZooKeeper and etcd hold shared data on purpose.

What Stateful Costs You

Keeping state inside the server gives speed, or makes a live connection possible. In exchange, the team takes on extra problems.

• Routing. Something must know which server holds which user's state. That usually means a registry, which is a shared map from each user to their server, or consistent hashing.
• Failure. When the server crashes, its memory is lost. To survive that, the state must be copied to other servers, which is a lot of extra work.
• Deployments. The team cannot simply restart the server. It must first move users away, hand the state over, or accept an interruption.
• Memory use. Every active user uses memory on a specific server, so capacity must be planned per server.

For example, a chat app has 500,000 connected users, and each gateway server holds 50,000 connections. The app needs 10 gateways. When a message arrives for user 42, the system looks up the registry. It finds that user 42 is connected to gateway 3, and sends the message there.

All of these problems can be solved. But they are extra work that a stateless design does not need. That is why stateless is the default, and stateful is the exception.

Stateless vs. Stateful

Choosing Between Them

• Web and API servers: make them stateless. Keep sessions in a shared cache or a signed token. This should be your starting answer in almost every design.
• Open connections, like chat or live updates: use a stateful gateway layer. Keep that layer thin, and keep everything behind it stateless.
• State that changes many times per second, like a game: use stateful servers, with routing that sends each user to the right server.
• Never keep ordinary web sessions in server memory. Doing that forces sticky sessions, and every crash logs users out.

Using This in an Interview

Say early that the application tier is stateless, and say where the state went. For example: "The application servers are stateless. Sessions are stored in Redis, so I can add or lose a server without logging anyone out."

If your design uses WebSockets, mention the exception yourself. Say that the gateway servers are stateful, because each one holds open connections.

A common follow-up question is: "How does a message reach the right gateway?" Answer with a registry that maps each connected user to the gateway holding their connection. When a user reconnects to a different gateway, the registry is updated.

Key Takeaways

• State is data a system remembers between requests. Stateless does not mean there is no state. It means the server does not keep that state in its own memory.
• In a stateless design, every request carries the information the server needs, or a key to find it in a shared store.
• Stateless servers are interchangeable, which makes scaling, deployments, and recovery from failures simple. This is the default for web and API servers.
• Stateless has costs too: repeated data in every request, an extra lookup, and a shared store that must stay available.
• Stateful servers are resource intensive, and need routing, replication, and careful deployments.
• Stateful servers are justified when state changes too fast to store elsewhere, or when the open connection is the state.

Moving state out of the servers is what makes adding and removing servers easy. The next lesson, Serverless Architecture vs Traditional Server-based, takes this idea further, to code that runs without servers you manage.

Practice Questions

Try each question first, then open the answer.

1. A user logs in on server 1, which stores the session in its own memory. The load balancer sends the user's next request to server 3. What happens, and what are two ways to fix it?

<details> <summary>Show answer</summary>

Server 3 has no session for the user, so the user appears logged out. One fix is to store sessions in a shared store like Redis, so any server can read them. This is the better fix. Another fix is sticky sessions, which send the user back to server 1 every time. But sticky sessions make load uneven, and the user still loses the session if server 1 crashes.

</details>

2. A stateful server holds 100,000 active sessions, and each session uses about 50 KB of memory. How much memory do the sessions use, and what happens when the server crashes?

<details> <summary>Show answer</summary>

About 5 GB, and all 100,000 users lose their sessions. 100,000 x 50 KB = 5,000,000 KB, which is about 5,000 MB, or 5 GB. That memory exists only on this server. When it crashes, the sessions are gone, so those users must log in again and may lose unsaved data.

</details>

3. A service uses 1 KB signed tokens, and receives 2,000,000 requests per day. How much extra data do the tokens add, and what does the service get in return?

<details> <summary>Show answer</summary>

About 2 GB per day. 2,000,000 x 1 KB = 2,000,000 KB, which is about 2 GB. In return, the servers do not need to look up a session for each request. They only check the signature, so any server can handle any request. The downside is that a signed token is hard to cancel before it expires, so it should be short-lived.

</details>

4. A chat app has 500,000 connected users. Each gateway server can hold 50,000 WebSocket connections. How many gateways are needed, and how does a message for one user reach the right gateway?

<details> <summary>Show answer</summary>

At least 10 gateways, plus a registry. 500,000 / 50,000 = 10, so the app needs at least 10, and a few spares for failures. When a user connects, the registry records which gateway holds that user's connection. When a message arrives, the system looks up the registry and sends the message to that gateway.

</details>

5. A team wants to deploy a new version of its stateful game servers while matches are running. What is the problem, and what can the team do?

<details> <summary>Show answer</summary>

Restarting a server would end every match running on it. The match state exists only in that server's memory. The team can first stop sending new matches to the old servers, and wait for the running matches to finish. Then it restarts those servers with the new version. Another option is to hand each match's state over to a new server, which is more complex.

</details>