Designing Facebook’s Newsfeed

User and Entity hold the profiles. A User row is one person: name, email, and a few dates. An Entity row is a page or a group, and its Type column records which of the two it is. Both tables grow only when new accounts appear, so they stay small.

The UserFollow table is the one to study. EntityOrFriendID can point at a user or at an entity, and Type says which one. The feed query in step 6 filters on that column. One row records one follow. The average reader from step 2 therefore adds about 500 rows here. With 300 million users, UserFollow outgrows every other table in the schema.

FeedItem is the post record that step 2 priced at about 1 KB: the text plus its metadata. UserID names the author, and the optional EntityID names the page or group it was posted in. NumLikes is a like counter. Ranking in step 6 reads it when it scores a post.

Media files are not in any of these tables. A Media row holds only the file's metadata and its Path, a reference to the file itself, which lives in blob storage. A FeedMedia row links one post to one media row. A post with three photos therefore has three FeedMedia rows.

The last record in the diagram lives in memory, not in the database. It is the pre-generated feed: one entry per user, holding that user's ready feed items in order. It is derived data: every item in it is a copy built from the tables above. If it is lost, the tables can rebuild it. Step 6 defines its exact structure, keyed by UserID, and shards it across the 1,500 cache servers from step 2.

Step 5: Draw the High-Level Design

At a high level, this problem divides into two parts: generating a feed and publishing it.

Feed generation. A newsfeed is generated from the posts of the users and entities that a user follows. Say we receive a request to generate the feed for a user named Jane. We perform these steps:

1. Retrieve the IDs of all users and entities that Jane follows.
2. Retrieve the latest, most popular, and most relevant posts for those IDs. These are the candidate posts for Jane's newsfeed.
3. Rank these posts based on their relevance to Jane. This ranked list is Jane's current feed.
4. Store this feed in the cache and return the top posts (say 20) to be rendered on Jane's screen.
5. When Jane reaches the end of her current feed, the client fetches the next 20 posts from the server, and so on.

Notice that we generated the feed once and stored it in the cache. What about new posts from people Jane follows? If Jane is online, we need a way to rank those new posts and add them to her feed. The simple way is to rerun the steps above at short intervals, then notify Jane that newer items are waiting. A timer alone cannot meet the 5-second target from step 1, so step 6 adds a faster path for live updates.

Feed publishing. Whenever Jane loads her newsfeed page, she requests and pulls feed items from the server. When she reaches the end of her current feed, she pulls more. For newer items there are two options. The server can notify Jane, and she pulls. Or the server can push the new posts to her. Step 6 compares these options.

Components. At a high level, the Newsfeed service needs these parts:

1. Web servers: maintain a connection with the user. This connection carries data between the user and the server.
2. Application servers: run the workflow of storing new posts in the database servers. Other application servers retrieve the newsfeed and push it to the end user.
3. Metadata database and cache: store the metadata about Users, Pages, and Groups.
4. Posts database and cache: store metadata about posts and their contents.
5. Video and photo storage, and cache: blob storage for all the media included in the posts.
6. Newsfeed generation service: gathers and ranks all the relevant posts for a user, generates the newsfeed, and stores it in the cache. This service also receives live updates and adds newer feed items to any user's timeline.
7. Feed notification service: notifies the user that newer items are available for their newsfeed.

The diagram below is the high-level architecture of the system. In it, User B and User C follow User A.

Read the diagram from the poster's side. User A creates a post, and the application servers store it in the posts database. Users B and C follow A, so the newsfeed generation service adds A's post to their stored feeds. The feed notification service then tells B and C that new items are waiting.

Step 6: Go Deep

Generating the feed

Take the simplest case first. The newsfeed generation service fetches the most recent posts from all the users and entities that Jane follows. The query would look like this:

(SELECT FeedItemID FROM FeedItem WHERE UserID in (
    SELECT EntityOrFriendID FROM UserFollow WHERE UserID = <current_user_id> and type = 0(user))
)
UNION
(SELECT FeedItemID FROM FeedItem WHERE EntityID in (
    SELECT EntityOrFriendID FROM UserFollow WHERE UserID = <current_user_id> and type = 1(entity))
)
ORDER BY CreationDate DESC 
LIMIT 100

The two SELECT statements come from the two kinds of follow. The first finds posts by followed users, matching on UserID. The second finds posts in followed pages and groups, matching on EntityID. UNION merges the two result sets into one list, which is then sorted by creation date.

This design has four problems:

1. It is very slow for users with a lot of friends and follows. We have to sort, merge, and rank a huge number of posts on demand.
2. We generate the timeline when the user loads their page. That adds latency to every page load.
3. For live updates, each new post causes feed updates for all followers. That could create high backlogs in the newsfeed generation service.
4. For live updates, pushing (or notifying about) newer posts to users could create very heavy loads. That is especially true for people or pages that have a lot of followers.

The first two problems share a cause: we do the work while the user waits. To fix that, we pre-generate the timeline and store it in memory.

Offline generation. Dedicated servers continuously generate users' newsfeeds and store them in memory. When a user requests their feed, we serve it from the pre-generated, stored copy. The feed is compiled in the background on a regular basis, not on load.

Whenever these servers generate the feed for a user, they first check when that user's feed was last generated. Then they generate new feed data from that time onwards. We store this data in a hash table. The key is UserID. The value is a struct like this:

Struct {
    LinkedHashMap<FeedItemID, FeedItem> feedItems;
    DateTime lastGenerated;
}

We store the FeedItemIDs in a structure like a LinkedHashMap or a TreeMap. Both keep the items in order and let us jump to any item by its ID. That combination is what pagination needs. When a user wants more feed items, the client sends the last FeedItemID it currently shows. We jump to that ID in the map and return the next batch of items from there.

How much feed to keep in memory

How many feed items should we store per user? Start with 500, then adjust from the usage pattern. Suppose one page of a feed shows 20 posts, and most users never browse more than ten pages. Then 200 posts per user covers almost everyone. A user who wants more posts than we hold in memory is served by a query to the backend servers.

20 posts per page x 10 pages     = 200 posts per user

Should we generate and keep feeds for all users? Many users do not log in often, and their pre-generated feeds waste memory. There are two options. 1) The simple option is an LRU cache. LRU (evict the least recently used entry) removes users who have not accessed their newsfeed for a long time. 2) A smarter solution learns each user's login pattern and pre-generates their feed just before they are likely to be active. Useful inputs are the time of day a user is active, and the days of the week they open their feed.

Publishing the feed

Publishing means getting a new post into the feeds of everyone who follows the poster. Pushing one post to all followers is called fan-out. So the push approach is called fan-out-on-write, and the pull approach is called fan-out-on-load.

Keep the two roles from step 2 apart here, because they are easy to confuse. The cost of fan-out-on-write depends on the poster. One post becomes one write per follower the poster has. The size of a feed depends on the reader. It is built from every account the reader follows. Our average reader follows about 500 accounts, which is manageable. But some posters have millions of followers, and that is where fan-out gets expensive.

1. Pull model, or fan-out-on-load. Keep all recent feed data in memory. Users pull it from the server whenever they need it. Clients can pull on a regular schedule, or manually. This has two problems. New data is not shown until the client issues a pull request. And it is hard to find the right pull interval. Most pulls return an empty response when there is no new data, which wastes resources.

2. Push model, or fan-out-on-write. Once a user publishes a post, we immediately push it to all their followers. The advantage is on the read side. When fetching a feed, we do not have to go through the reader's follow list and fetch posts for each account. That significantly reduces read operations. To receive pushes, each user maintains a long poll request with the server. Long polling means the client keeps a request open, and the server answers it only when new data exists. The problem is a poster with millions of followers (a celebrity user). One post means the server has to push updates to millions of people at once.

3. Hybrid. Combine fan-out-on-write and fan-out-on-load, and choose per poster. Stop pushing posts from users with a very high number of followers. Only push posts from users who have a few hundred (or thousand) followers. For celebrity users, let their followers pull the updates. Disabling fan-out for those few posters saves a huge number of resources. When a reader loads their feed, the system takes the stored list and merges in recent posts from the few celebrities they follow.

Two more variants are worth knowing. Once a user publishes a post, we can limit the fan-out to only their online friends. And we can combine "push to notify" with "pull for serving". The server pushes a small signal that new posts exist, and the client pulls the posts. A pure push or a pure pull model is less versatile than these combinations. Push vs Pull Architecture covers the general trade-off, and Designing Twitter applies it to a timeline.

How many feed items can we return per request? Set a maximum for one request. A typical page is 20 items, and the API caps count at 200. Within that limit, let the client specify how many it wants. A mobile client and a desktop client show different numbers of posts.

Should we always notify users about new posts? Notifying is useful. But on mobile devices data is relatively expensive, and pushed data can consume unnecessary bandwidth. So at least for mobile devices, we can choose not to push the data. Instead, users "Pull to Refresh" to get new posts.

Ranking the feed

The simplest ranking is by creation time. Modern ranking algorithms do a lot more, to make sure "important" posts rank higher. The idea is to select key signals that make a post important, then combine them into a ranking score. A signal is a feature of a post that we can measure.

Useful signals include the number of likes, comments, and shares, the time of the update, and whether the post has images or videos. A score computed from these features is enough for a simple ranking system. A better system evaluates itself constantly. It checks whether changes improve retention (how often users come back) and ad revenue.

Partitioning the data

Sharding posts and metadata. We have a huge number of new posts every day, and the read load is extremely high. So we distribute the data across many machines to read and write it efficiently. For the databases holding posts and their metadata, use the same sharding design as Designing Twitter.

Sharding feed data. The feed data lives in memory, and we partition it by UserID. We pass the UserID to a hash function, which maps the user to one cache server. That server holds all of the user's feed objects. We never expect to store more than 500 FeedItemIDs for one user, so a user's feed always fits on a single server. To get a user's feed, we query only one server. For future growth and replication, use consistent hashing. Consistent hashing means adding or removing a server moves only a small part of the data.

Step 7: Bottlenecks and Failure Points

The interviewer does not expect this design to be limitless. They expect you to know where it ends.

• The read path depends on the feed cache. Every feed request goes to one cache server, found by hashing the UserID. If that server is down and not replicated, every user hashed to it loses their feed. Replicate the feed cache.
• Fan-out cost is set by the poster with the most followers. The hybrid removes the celebrity case from the push path. Without it, one celebrity post would occupy the newsfeed generation service for everyone.
• Pre-generated feeds lag reality. A new post reaches a follower's feed only after fan-out or the next refresh. That lag is what the 5-second target measures.
• 150 TB of memory is the largest cost in the design. Storing 200 items instead of 500, and evicting inactive users, are the two ways to shrink it.

Where AI Fits in This Design

After the main design, interviewers often ask one more question: where would AI fit in this system?

This lesson already ranks the feed with a score built from signals: likes, comments, shares, recency, and whether a post has media. A learned model is the natural next step, predicting which posts a reader will interact with. The interview point is not the model. It is what the model needs from the system.

A model needs its signals at feed-build time, and it needs them fast. Counting likes across the posts database on every build is exactly the work the pre-generation design exists to avoid. The signals get their own store: a table or cache, keyed by post, updated as events arrive, read in bulk when a feed is built. Engineers call this a feature store. It is a genuinely new component, with its own freshness and failure questions, and naming it is what separates a real answer from "add ML".

Everything else stays. Fan-out still decides which posts reach a feed; the model only orders them once they are there. The celebrity rule, the per-user feed cache, and pagination work unchanged.

The lesson already names the second use: learning each user's login pattern, so feeds are pre-generated just before they are needed. The shape is the same: the model schedules the work, and the pipeline does it.

💡 In the interview: spend your time on feed generation and feed publishing, and move quickly through everything else. Say early that building the feed at request time cannot meet the 2-second target, and move to pre-generation. The moment the interviewer is listening for is when you name the celebrity problem and switch to the hybrid. Push for ordinary posters, pull for accounts with millions of followers. Keep the poster and the reader apart when you explain it. Fan-out cost follows the poster's follower count, not the reader's follow list. If you are asked what breaks first, name the cache server that holds a user's only copy of their feed.

Key takeaway: a newsfeed is a read-heavy system where every read merges posts from hundreds of accounts. About 17,500 feed requests per second, each over about 500 followed accounts, cannot be built at request time. Pre-generate each feed offline and keep about 200 to 500 items per user in a LinkedHashMap keyed by UserID. Page through it from the last FeedItemID the client saw. Deliver new posts by fan-out-on-write for ordinary posters and by pull for celebrities, because fan-out cost grows with the poster's follower count. Rank with signals like likes, comments, and recency. Shard the feed cache by UserID with consistent hashing, so a feed read touches one server. Keep media in blob storage, so a cached post stays about 1 KB.