System Design : Design messaging/chat service like Facebook Messenger or Whatsapp
HTML-код
- Опубликовано: 28 июн 2024
- System design : Design messaging or chat service like facebook messenger.
redis.io/topics/cluster-tutorial
docs.datastax.com/en/cassandr...
en.wikipedia.org/wiki/WebSocket
I haven't even finished the video, and I have to thank you. This has to be the best explanation on chat messages.
Very helpful tutorial. Covers quite a few aspects of the problem. One tutorial I would like is how do you give estimates of server request and memory needed for the design question.
Thanks for the video! There is a little error about BOSH and long polling explanation BOSH and long polling is the same and what you describe as long polling is regular polling
yeah ... exactly, the explanation for long polling is wrong in the video.
yup
This is the best video till date on chat service. Thank you
Complex but very informative. I was thinking the same about storing a timestamp to know when a user has unread messages or not.
Great to see you doing these system design questions. !! Keep them coming :)
definitely
This one deserves a thumb up!
way better than grokking system design and some other chat messaging system design videos! Thank you so much!
Great job. One of the best design videos I've seen.
Thank you Sir. This is a great explanation for beginners like me.
Great explanation.. It all just fit in my mind.. The teaching style is so nice.. that everything just slips in brain.
one of the best ones on chat services
Tushar, you are a good teacher. Easy to understand. Thank you.
Hey Tushar first of all thank you very much for these videos, they're invaluable. I'm having a interview in the coming month that I know for a fact will feature a systems design question and your series have helped me shaped my thinking about design.
Also if you're looking for further ideas, I think something involving geo-spacial data like Uber will be a fun video to do
+Netherblood I m glad this is helping and believe me that question is in my list.
Hi @Tushar, thanks for clarifying steps in thinking through system design process - very helpul for interview prep. You did mention right at the outset that you will not be going into security aspects of the design... are you planning to make new videos on this topic? I'm hoping: do interviewers still avoid such complicated issues as security during interviews? I'm concerned there are so many layers to consider - between middleware components, in transit, user/device level, coord of Auth/Login/session-init, mobile vs web app, ubiquitous handoff between devices/browsers in continuing/rejoining chat sessions, also considering storage of session details in cache and sync with cassandra - there are so many more complicated requirements these days... would appreciate your thoughts and advice on how to streamline all these various scenarios into suitable interview-timeline-constrained solution presentation.
Thanks for the great video, Tushar!
Thanks for the video, very information. In your introduction to System design video you mentioned about API design. I was hoping in your next video you first define the API supported by this design before going into the details.
Great Video, we want more people like you.
The video is really useful. I liked it. However, I'd suggest a small change. Instead of having two separate tables for unread and read messages, I'd just create a column ReadFlag in the Conversation table. The boolean value in the column will indicate if the message was read or unread.
Could have been done that way because of performance reasons.
excellent tushar.... becoming an hard-core fan of you :P
The idea of storing sessions is really cool.
One more system design question request - design a system where alexa orders a list of products on Amazon
you really make design and coding simple, very nicely explained.
Thank You Tushar Sir and you are the hope for people like me.
I am going to use Cassandra because this is my favorite database. Great.. :)
Tushar, your videos are awesome.
Thank you so much for this video. Its been really helpful.
very well presented overview, thanks :)
In the case of WebSockets, In order to transfer the request to the same machine, Load Balancers will have to keep a map for the user-machine pair in memory. Or it can also query redis, but this needs to be done every single time. even for ping operations and that would be really heavy!
Excellent video, thanks!
Thanks Tushar for coming up with this new design algo series.. if possible can you also make a video on designing stock application and uber service.. I have seen these questions being asked in some of the tech giant companies in SF bay area..
Much useful for me , thnx a lot..
Best chat system design I saw so far! Thank you@Tushar. Will it be possible for you to design how Facebook notification system works?
Some very interesting designs there Tushar. Nice! I have some questions and suggestions though.
How can N2 talk to N3 when there is no web socket connection with them? How does N2 know to send N3 a message? Is the communication between N2 and N3 handled at application level? I think there should be MQ implementation here because server can't be used to send messages to other servers. Do you think it can scale well?
yes servers will talk to each other using MQ.
Awesome !
nice job, keep making more. they are very helpful.
Question - Is there a reason why we cannot collapse conversation table and unread table into one? It would seem to me that we could simply keep a status flag and timestamp flag in conversation table to indicate whether/when a message has been read. That would eliminate the need of having to read from both unread and conversation table when the system tries to resend messages. I could see that one pro to keeping them separated is that it will keep the # of entries in unread table at a minimum and therefore improve the query performance. However, I think this may be offset by the fact that to retrieve an unread message, we would have to query both the unread table and the conversation table. Just a thought.
Interesting one, you are a good teacher
superb video man. hats off to you
I think we can discuss more about the concurrency aspect and fault tolerance. It would great to know about how a chat system is essentially a multi-leader distributed systems with each chat thread being more like a table being updated concurrently by two or more chat participants. Concurrency for the redis cache/ cassandra, would inevitably bring up things like partitioning / sharding strategies for the database.
There can be more on the discussions on blocking users, monetisation aspect including push notifications.
The recently implemented feature of businesses using the messenger platform to push ads and reach with consumers is very innovative. In these cases there are broadcast messages, and can be dealt with in different ways.
Chat bots are one more interesting thing coming up with Messenger.
Overall I believe this video is very good and interesting, and is helping me a lot to understand how to tackle system design interviews. Thanks a lot for keeping these videos coming..!
+Anish Chakraborty good feedback. Cassandra does inherent partitioning based of partition key which in this case is user IDs and conversation id. Reason I stayed away from that was to keep things somewhat simple. Your observations are very accurate and you are ready to take this to next level.
awesome explanation ...
Very nice, wonderfully explained. Thanks a lot for sharing it Tushar.
One alternative thought, do you think there could be a bottleneck when a Node gets a message from the client, as it is supposed to do two things
a. Write the message into the DB.
b. Send the message to the right node (who is currently handling the recipient client).
Could this be an issue in case of scale like FB messenger.
Instead of directly handle the incoming message what if the Node1 just write the incoming message to the enterprise stream (say Kakfa). The alternative workflow would be
UserA -> Node1 -> ChatStream -> [Node2 picks up cause it was listening to this specific topic] Node2 -> UserB
There would be a separate service altogether which will pick the messages and write into the Conversation table, similarly Node2 picks up the message as it was listening to this specific Topic and send to the recipient client (using WebSocket).
The Node1 and Node2 will just communicate with the client and with the ChatStream and nothing else.
It will be good to know your thought over this alternative approach.
Note: I can already see the complexity of the deciding the Topic, because if we are going to create Topic for every user that would an billions of Topic, hence we have to group the active users or using other better way to create a smaller number of Topics, so the stream doesn't become insane.
Its a fair point. But remember you can add as many nodes(N1, N2) as you like. Its totally horizontally scalable in that sense.
i was about to comment the same approach of using some distributed queue like Kafka or AWS SQS.
I think this design is more in sync with the recommendations of scalability, as we should refrain from allowing two nodes to talk to each other. When the node2 was trying to send the message to node3, node3 could have died, customer could have left and reconnected a websocket connection somewhere else (potentially moving it to node1), etc. Using a chatstream of sorts makes this easy to manage.
Completely agree with you. The approach in the video is not scalable.
@@haseebshahid5148 This solution in the video is highly scalable. A message queue like Kafka would not work for two reasons (1) messages would be delivered unordered (unless each user has their own shard which for 1bn users is impossible) and (2) communication would be async so if user1 wrote a message to user2 there would be some delay (perhaps miniscule but perhaps not) before user2 sees it. I don't see anything wrong with nodes talking to each other? If N2 needs to communicate with N1 and N1 goes down, it can either return an error to the user (failed to send) or retry with redis to see the new server managing the user to send the message to.
Amazing you are the boss
thank you very much for valuable post
Excellent video. Loved it,.
I really like your videos @Tushar and they are very comprehensive. Had one suggestion on the messaging design though, can you look into the design aspect of using Kafka or any other message Service , can be hosted in a cloud etc.
Sorry for multiple posts, but I am working on a similar design and I have ran into use cases that these did not address. In many cases, it is possible that same user A is connected to our service using two different clients. E.g. Desktop/Phone. In those cases, the service needs to handle such scenarios where you need to add a deviceId or similar to redis. Also, you need to echo any message you receive to all users connected in conversation. A hi sent by user A to user B should send hi to all devices of userA and B. This way, a message typed by userA in desktop will show up in his phone as message sent as well.
Thanks for the video.
BOSH is the same as HTTP long polling, I think your explanation of Long pulling is incorrect.
You explained polling for long polling.
:)
Great video, a bonus for the links too.
Hi Tushar, Thanks for the wonderful video. I see that the tables like unread or read should contain message encryption id for a specific message is read or not.
Very nice explaination
Thanks for the good video! Do you think that use RethinkDB could be a good option as well?
Hey, @Tushar!
Your videos are pretty great with only one issue of a bad Microphone that keeps hissing in my head hours after watching your content. I'd suggest you to get a shotgun mic if the budget allows, there's an awesome post by Wistia on Mics for videos like yours. Or go for the do**bag mic :P the one that hangs around on your face.
+Ahmad Awais I do use shotgun mic. I will read the post. Thanks for feedback.
That's strange. Why is there so much white noise in the background then? Seriously, you have one of the best videos on these topics but the mic make them very hard to listen to.
I really agree, it is really irritating to listen more than 5 mins of the video because of the sound.
Hi. Long polling that you described is more like "periodic pulling". Long pulling needs to be hold by the server.
Thanks for the good work Tushar. One clarification, it is not clear from the video what is the definition of conversation and thus conversationID.
Thank you Tushar - Super interesting talk. One thing I was puzzled with: How can I do a search if the messages are encrypted. Per facebook they do not have the applications' private keys
Messages would be decrypted at our end
Good explanation, searching chat is something i'd like to know more, suppose if there are a ton of messages from a lot of users in general, bringing them all to the client and decrypting and then searching on the client is not efficient.
that was a wonderfully explained.thank you. but i would like to understand the diagram on BD how does it work there in.
You mention at the start this is your third design video, I went to your channel to find the other two, and there is a lot of videos there, is there some "difficulty order" or any other order type that you could make a playlist for us to watch videos in some logical order, simpler to more complex, would be appriciated. Thanks
Hey Tushar love your videos man. Important thing, as per this design, we cannot send a notification. (I know notification is not a part of our feature yet, but could be thought of as an important feature to be added as the project evolves)
Explanation:
-> We are tying up "online" and "socket being open". So web-socket is open only when you are online.
-> I think even if your app is closed and you are not online, your web-socket should be open.
at 12:50, user A tries to send message to user C but user is not online and can't get a notification. He will directly receive message when he opens app and a socket connection is established.
very good sir, thank you
Hi Tushar,
The video is very informative. Can you please share your views on how to handle - delete a message from user/group's chat
Great video! You mentioned only the encryption key is sent and will be reused for reply, then how would the encryption and decryption happen on FE(device)? I imagine there must be persisted somewhere to be looked up through encryption key?
hi @tushar, I just have a problem with chat search as the chat is encrypted so how does it search in encrypted chat in server side.
Thanks for the great explanation. Question - In the picture sending flow (23:30), if UserA sends an encrypted picture to N2, how would N2 get a thumbnail from it without looking in to it?
could be another blob url, or byte-array. It's design
Images are not encrypted!
@@karthikmucheli7930 If a small image is base64 encoded, couldn't it be encrypted then?
Hi Tushar, great video thanks for doing this! In your example, for the conversation table, do you mean Cassandra would partition based on conversationID as partition key? Meaning the conversation table is shared and would be accessed by all nodes.
Any conversation will be be shown to all users who are on that conversation. I presume this mean no user has their own conversation table for him/herself. Does the conversation table need FromUser and ToUser column? If, not where do you keep a mapping of a user and all the conversations they are part of? Also I presume the unread/read table are partitioned based on recipient user key?
is the conversion table kept in Casandara? isn't the read costly and impact the speed?
can't you store it in Redis as hash key value?
Very well explained!
I need some discussion my thoughts. I was thinking of a non-session oriented approach to this. UserA makes a connection to the load balancer for every message, sends his message and then close the connection. Any of the server (N1, N2 ...) can receive it. That will then make a connection to UserB, send the message and the close the connection.
Advantages:
-There won't be any inter-server communication. Imagine a group of 100 people (managed by say 50 servers). Someone sends an image to that group. That image will now be downloaded only by single server (instead of 50).
- We don't need to maintain user-session relationship in Redis.
Disadvantages:
- Handshaking overhead for every message.
- Hard to tell if a user is online.
Hey @Tushar, how about this approach , where we maintain only one table called conversations table, and in that one field could be state (READ, UNREAD, etc). For example, let's say we have a table with following schema (sender, receiver, encrypted msg, key, time, state, .... ). When someone opens a new chat window, the server makes a call to the db, select all those conversations between these 2 where time > t or may be keep an upper count of the messages sorted by time. Do you see any bottlenecks here.
Tushar thanks for the Video. It’s awesome as always.
One query, on what basis we remove user info from redis..
Good question. There are few options.
1) Never remove the user. As there are a limited number of users like 1 billion if we want we can keep them there permanently.
2) ttl(time to live) based removal. Remove it if user has not heartbeated in 1 hour. Very easy to do on cache like redis.
3) Let redis LRU cache do its removal. It will remove least used stuff first which is basically user who has heartbeat oldest.
as per current cases in whatsapp, by default we can go with option 1 when user A goes offline, user B should still be able to see last seen time. If user had not subscribed for last seen option, then remove with option 2.
Also, we need to enable catch write to disk in case any of the redis node goes down.
Hello Tushar, great explanation. Can you please explain a bit more about the communication between the nodes? Like when N2 receives info from user A, how does it talk to N3? Since all these nodes are behind the load balancer, these are all multiple instances of the same application. Could you detail the internal design in this area?
This video incorrectly talks about a design which is highly coupled between the Servers. In reality, a session's microservice is responsible for sending the messages to those said servers which just holds a TCP connection. These servers' only job is to just send information to the clients it is currently holding on. Evidently this is a cleaner architecture which removes the de coupling. No systems are designed where it is important one server talks to another server which does the "same thing".
really good one.
Would you be able to create a video of a live news feed system which will display latest feeds from all around the world such as Linkedin's newsfeed.
Very nice video. Can you please clarify how User B "tells" User A that a message is read? Also should there be a column in conversation
table to capture the read time so that this info is available for historical reasons?
One question on unread table, it may be too much by using delete, we should be easily maintain a last online timestamp for every user, and read from Cassandra once user gets online since last online timestamp. Also I think you can partition the receiving message and send messages in database. Which means every message can be replicated to sent user and received user. Then when do search, the search should be faster
Very well explained tushar, can you guide me a bit on internal working of how would we handle notifying to User3 who is not online that user2 has sent him a message?
Some thoughts: Did you consider how to keep multiple different devices in sync? Is the device storing state? What if Someone has 10 years of messages in the DB and then gets a new phone? How does the DB deal with that read request? What about storms of abnormal queries like New Years Eve when everyone is activating conversations that are a year old and skips the cache? When does data get tombstoned from Redis so that you can determine that a user is no longer online? Immediately after failed heartbeats? Or is there a job that does it every N hours based on last heartbeat?
This guy knows signal in 2017, we are using it in 2021.
Good design overall, but not modular enough. Consider making the image service a standalone service.
hi @tushar - please tell me what will be the partition keys and clustering keys in cassandra tables
isn't distribution system very critical part of this? how DB is going to sharded and how to maintain the consistency and speed from DB? how the system needs to scale out? what you are doing (which i am learning a lot) is how the system works like your algo tutor which I really appreciate a lot.
Very nice
I didn’t skipped ad.
a load balancer will only play a role before the initial HTTP GET (Upgrade) request is made, i.e. before the one and only TCP connection involved in said WebSocket connection creation is established between the two communication end points. Thereafter, the TCP connection stays established and cannot become "redirected" by a network device in between.
I think better approach is:
when UserA wants to talk to UserB, A initiates conversation using a separate Conversation Service which records participants and returns Conversation Id.
Then this conversation Id can be used for Redis Pub/Sub.
Servers of both user A and user B subscribe there and start listen.
In this case it doesnt matter at all to where each client is connected to and we dont have to keep that weird server names table.
User A sends a message -> it is stored in DB and added to Pub/Sub.
Server of User B reads the message from Pub/Sub and send to user B.
If server of User B went offline - subscription is lost, same as user connection. It's fine.
Upon reconnect user will send us last offset and we will deliver unread messages and subscribe to pub/sub again.
In similar way we can think about push notifications..
Use websocket, similar to sticky session, is less desirable for such high-traffic system. HTTP long polling is less sexy but can be horizontally-scaled much better since we can make it stateless. Each Node exposes a webhook for other Nodes to post to: hey right now UserB is connecting to you and there is indeed a message for UserB. The UserB-Node addressing can be done by Redis like what you proposed here.
How to make file transfer application like share it . can you explain the design of share it in next videos please?
Video is very helpful . Could you please make a video on 1) System Design of Stock exchange. 2) System Design of Payment Gateways like payu...Thanks
Hi Tushar thanks for very informative content. My question is why did you choose Cassandra? Is there any specific use case that Cassandra handles efficiently compared to other database?
+Rajat Khandelwal great question. I should have clarified on that. First it's distributed and scales well. Second it's amazing for storing time series data like we have in this case. It's easy to do query like give me everything for given conversation id since time t5.
Tushar Roy - Coding Made Simple Thanks 😃
Excellent
I understand the best approach is probably WebSockets when we are talking about browsers. What about Native Apps (iOS/Android). Do we do Virtual Polling (poll often) ?
In the HTTP route, if UserA sends a message to User B, the message goes to N1. Can N1 as part of the Ack back to UserA, also add payload from other users trying to send messages to UserA ? (or is that bad overloading) ?
Thanks. Great videos.
if you are using native apps then you can do whatever heck you want. Best would be a custom protocol on top of keep alive tcp connection (which is bidirectional in nature).
Thank you Sir
Hi Tushar, Its great video thanks, but i have one question why do we need an unread and read table or we can add a new column (status) along with text and save the current status of the message (read or unread)?
Nice video
How can we use XMPP in this? Does it make implementations of features like online/available/typing easier?
It does. But different xmpp server implementations have different segments of xmpp protocol implemented. So not all xmpp servers support the same features. Then again I haven't seen xmpp implementations in 5 years so things might be different now
super........
Great walk through. Would've liked to see where cassandra fits in?
the tables he mentioned are in cassandra only
Hi thanks for the video. It was very informative. One question though:
Assuming that 1 million people are online, will it be possible for the hosts(N1,N2 ...) to maintain 1 million open socket connections?
Since a socket has port and IP, and given that the ports are limited for each host, will maintaining opened web sockets scale?
I think that servers should periodically logout users in 2 cases - long period of inactivity or the number of connections exceeds a certain limit.
they would spin up more servers to handle the additonal clients, hence n1, n2, ...nX
These hosts are provided on demand, with autoscale. They probably wouldn't need that much disk space or computing power, maybe some memory to handle transfering huge amount of files to the blob storage.
Exactly the question I had in mind.
This guy says he got 600k simultaneous websocket connections with 1 AWS machine using Node: blog.jayway.com/2015/04/13/600k-concurrent-websocket-connections-on-aws-using-node-js/. So even if you could only get 100k, spinning up 10 machines (to get 1M) isn't crazy expensive for any company who has that many active users.
Tushar, thanks for this awesome video. However i have same query like,
1. how group chat can be implemented
2. how can we use more than one instance of redis for availability purpose.
3. The server name you stored in redis , is this for scalability purpose of websocket connections ?
Thanks.
1) in User table you can have userId, groupId combination instead of userA, userB combination. It also stores conversation_id and encryption key. In addition we can have group table for membership info about the group. Does that make sense?
2) Redis cluster comes out of box with all that. Nothing special needs to be done. redis.io/topics/cluster-tutorial
3) Server info is to find which server is managing the client.
Sure, its very helpful, appreciated the help!
make sense ..
Thanks For the video. Nice explanation. Can you also describe how to design A FastPass for Amusement Park?
@tushar But load balancing purpose is defeated if we go for a sticky session