Notification System: Scalability and Performance

Step 8: Scalability and Performance Strategies

Steps 5 through 7 drew the system. This step asks whether it carries the load that Step 3 measured.

Step 3 left us two numbers. The first is about 150 million notifications a day. The second is a fan-out event, one publish that turns into tens of thousands a second. Fan-out means one incoming event becoming many outgoing messages.

Those two numbers make different problems. A steady daily volume needs enough machines. A burst needs a way to absorb it.

Each strategy below answers one of them. It does so by spreading load, running work in parallel, or removing a bottleneck.

Horizontal scaling and load balancing. Every part of this design grows the same way, by adding machines rather than by buying a bigger one. Run many Notification Service instances behind a load balancer, so API requests spread evenly across them.

The channel workers scale the same way. If email needs more throughput, add email workers, and each one reads from the queue partitions. The queue itself, Kafka for example, runs as a cluster of brokers and splits its data across them.

Each added machine adds about the same capacity, so no single machine has to be very powerful. One rule follows from that. Nothing should sit where all traffic must pass through one instance.

Efficient queueing and parallelism. A high-throughput queue like Kafka takes messages in fast and hands them out fast. Large Kafka clusters handle millions of messages a second, so 100k a second is fine with a proper setup.

Tune the partition count to get the parallelism you need. Then batch where you can: the Notification Service can send messages to Kafka in batches, and consumers can commit offsets in batches. That cuts the overhead per message.

And because the design is asynchronous, the API never waits on the slowest part of delivery.

Caching and fast data access. Preferences and templates are read on every notification. Two reads per send at tens of thousands a second would be the busiest query in the system. Both change rarely, so both cache well.

A cache like Redis holds preference records in memory, so no send touches the database for them. That cuts the preference check to microseconds. Templates can be cached in memory inside the Notification Service the same way.

Notification history is the exception. The whole history is far too large to cache. A user who checks often could have their last N notifications kept in a fast in-memory store.

Database partitioning and sharding. Sharding means splitting one dataset across several databases, each holding a part of it. Shard the notification logs by user id or by region. User-based sharding puts a range of ids, or one region, into its own database. Then no single database takes all the writes for 150 million notifications a day.

Partition the logs by time as well: one partition or table per day or per month. Be clear about what this buys. Writes go to the newest partition, so this does not spread the write load.

What it makes cheap is deleting old data. Drop the whole old partition in one operation, instead of deleting rows one by one. Old partitions are mostly static and can move to cheaper storage. Queries over a recent range also read less.

Preferences may not need sharding in a scalable NoSQL store. In SQL, partition them by user as well. The rule is the same everywhere. No single node or small cluster carries the whole 150 million a day from Step 3. And none carries the tens of thousands a second that a fan-out event turns it into.

Load balancing at external providers. Everything so far scaled by adding our own machines. The last stage does not, because the email, SMS, and push providers belong to somebody else. Those calls still need balancing.

Use several SMTP endpoints or several connections, so one slow connection does not stall a worker. Cloud services like SES and FCM scale on their side, but they have rate limits. Watch those limits and ask for higher quotas early.

For SMS through Twilio, use several phone numbers or a messaging service pool to send in parallel. If one provider cannot take the peak, or throttles, add another and split the traffic. That is the same fallback Step 7 described for reliability, used here for throughput.

Next: Step 9, which says what happens when each of these parts fails.