News Feed High Level System Design (small and crisp)

News Feed System Design

We have mainly three internal services :

1. Post Service : Publish Post
2. Fanout Service : Generate news feed for friends
3. News Feed Service : Retrieve news feed for user

Post Service (POST /v1/user1/feed)

Steps :

1. Application call post api to Post service
2. Create post and persist data into post cache and post database
3. Create post event (eventId, postId, userId) and publish to queue

Fanout Service

Steps :

1. Consumes event from queue
2. Fetch user friends ids from Graph DB
3. Create event with <post_id, user_id> and publish to queue (here user id is friend id)
4. Fanout worker fetch the data from queue and create entry into news feed cache like below

Within the fanout service, we can employ either a fanout write (push) model, a fanout read (pull) model, or a hybrid approach combining both. The fanout write model facilitates real-time generation of news feeds and operates swiftly. However, it faces challenges with hot keys since it may generate new feeds for inactive or rarely active users. To address this issue, we can utilize the fanout read model, which generates news feeds at runtime when users log in. Although this approach avoids hotkey problems, it is comparatively slower as the news feed is not precomputed.

News Feed Service (GET /v1/user1/feed)

Steps :

1. Application make call to retrieve news feed for user to news feed service
2. News feed service makes call to news feed cache (we store only postId and userId in cache to utilise minimum memory)
3. Makes call to post cache to fetchpost details (like post content, images etc)
4. Makes call to user cache to fetch user details (like profile name, account details etc)
5. Generate full news feed and return.

I hope you enjoyed the blog. Happy Learning !!