plse dont decrease the length, we want these in depth explanations, we are always learning something new and your explanations are great, watching your videos i actually started to think about tradeoffs and what can we use
+1, as long as you note the moments you’re going into a deep dive that you wouldn’t necessarily do in an interview, I’d love to see more discussion like you did on why Cassandra is better setup for high write load than pSql.
This is so amazing that I don't have words to appreciate the details of the video. Your content is by far the most accurate and some thing that is done with the correct intent to teach.
@@hello_interview the beauty of the system design question is the way you solve one part of the whole question can be applied to another question: for example, splitting a read-heavy and write heavy service into multiple instance and scale them independently. so please keep it as what it is right now. And at the end of the day, we are preparing when watching, time is not the limit, but knowledge is.
You have the ability to articulate things in a very nuanced manner whilst still being quite comprehensive. I am someone, trying to level up as a software engineer from senior to staff level. And I find your content absolutely enriching. Keep educating, you have all the skills needed for an "Exceptional Software Development Educator".
You guys do amazing work. In all honesty I think you guys are all great but Evan is something else! Last week I had like 3 days to prepare for the system design portion of my onsite and literally all I did was find the videos that Evan made, try the problem myself for 1-2 hours, and then watch the video. I ended up getting an offer! Evan's style and even the way he talks (if you can somehow subconsciously copy it) is exactly what interviewers are looking for it seems!
Thanks A LOT Evan! I appreciate the high-quality content-going beyond the introductory level-that you and your team are producing. Thank you for your hard work and dedication. Wishing you continued success in your business. Cheers.
Hands down the best approach to system design interviews compared to everything else i have watched on youtube. Love your systematic approach. Pointing out what is relevant, what isnt, what points are signals to judge your level etc. Great work!
I have never commented on any video but this video truly deserves my word. This was probably the best system design video I have watched. It gives such details that are required when you want to level up your skills (Senior SA/design or Staff SA/design). Loved the details and I wished all areas had been covered in depth. I could have easily watched two more hours. Please go in depth in all areas even though I think it could get repetitive for you. But you are doing the community a favor by diving deep.
Your videos play out like an action thriller. I am hooked from the beginning to the end and I learn so much from each video. The tangents are the essence of your tutorials so please do keep them coming!
Evan, even you might thought this video is too long from your end, but from my perspective, this one, surely is in the TOP2(Uber if you ask) videos of yours. Really really appreciate your dedicated work and sharing: you mentioned MANY different ways of achieving a same thing. Even some of the ways isn't taken in this problem, they may be taken in some other cases, I really enjoy watching you evaluating different approaches. We know this certainly takes you lots of time and energy, so thank you again for putting everything together. Thank you!
I would swipe right on this video a hundred times 😍 perfect balance of breadth and depth. the context on level expectation is super helpful. Looking forward to the next video; would particularly love an infra-focused one after this product-focused topic!
"We might need to select a partitioning key for Redis to ensure that the reciprocals show up on the same node" I never thought I'd ever have goosebumps from a system design explanation
Hi Evan, Yet another masterpiece. I loved the depth you went into. I would like to see same or more depths in videos even if it becomes a 75-90 mins video. Please keep them coming!
Kudos to Evan and team. As a senior engineer I find this channel so useful for System Design. The delivery framework works like a charm. Could you guys please start the series for OOAD, because that is where I get stuck sometimes in picking up the correct design pattern. Overall I like the content of the channel and the pace at which each and every aspect is covered. Cheers. And just like GOT drops the hint, you have dropped the hint of next system design video "Distributed Job Scheduler".
Lol you are great, I stopped the video half way to comment this. The way you explained the eventually consistency problem where two users swiped each other at the same but Cassandra being eventually consistent will prevent them to ever know and they will not be able to find the love they are seeking lol. Amazing Stuff. Requesting you to create more and more content. You are a legend!
Amazing lessons! These are pure gold. I am deeply grateful for all the hard work you are doing. I would have one suggestion if you have time and energy. For me it is quite interesting also to deep dive in topics like api gateway and load balancing. It would be great to have a deep dive on it. Many many thanks for your effort, work and dedication. Keep going please. Cheers!
Hey Evan. I am a former FAANG staff engineer and I am interviewing actively now. Your content is the best out there. The deep dives are the best. They've helped me a lot. Please continue creating content. And if there is a Patreon or something I can contribute to then please let us know.
Keeping going Evan (dude in the video) :p, really love your deep dives of the video and system design in general. Love your content and always wait for the new one. videos are lengthy but content is gold.
I've enjoyed the system design interview videos a lot! Could you potentially do a video on behavior interviews in the future? I know you have an article on the website but a video would be super helpful!
This is one of the best channels teaching system design out there. Can you please do a system like google docs as well? Please please please please please. 😁
Hey Evan, your videos are amazing! They make system design look so easy and open my mind to think in different directions. I really appreciate your efforts. I’m eagerly waiting for more videos! :)
Don't forget the S3 storage for profile photos! Dating app without pictures is incomplete from a product perspective. Other than that little oversight, fantastic walk through. Thank you!
I'm really enjoying your videos. Well-structured, and with a variety of example systems, the repetition is really useful. You are reducing the intimidation factor, too! Thank you thank you for the "leveling" comments, that makes these videos so much more useful, and also helps us to judge for ourselves where we fall in that rubric. I love your writeups, too! One question I have is how much time per "section" would you spend in a real interview? Like, 5 min, 10 min, etc.
Cassandra does have row level atomic operations. If we modify the schema to have Swipe table contain decisions from both swipes, we have the transactional properties that we need.
Very nice deep dive! 👍 I was thinking, how about if we don't keep log of swipes but rather organize them in some kind of user's lookup state (which would consolidate likes and dislikes), may be in redis as well? Then we would pipe swipes through message queue, match with and update lookup states, and then send notification on successful match. This should help to avoid consistency issues as well. And at end we can add like-flag to users in stack response (if we already got their like for the current user before), so we can get that instant match-on-like directly on client side.
@Evan Great video! I have a question about the consistency solution with Postgres you proposed around 53:00. You mentioned having a single row for swipes with columns like user1, user2, user1Decision, user2Decision. However, wouldn't this approach still lead to potential inconsistencies if two users swipe on each other simultaneously? For example, if A right swipes B and B right swipes A at the same moment, wouldn't they create two separate rows with different user1 and user2 values? I'm curious how you would address this scenario to ensure consistency. Thanks! Also, I was thinking about a potential solution. What if we always ordered the user IDs consistently when inserting, like alphabetically or numerically? Then we could use a unique constraint on (user1, user2) and an UPSERT operation. This way, regardless of who swipes first, we'd end up with a single row for both users' decisions. Might solve the consistency issue without needing two separate rows. Just a thought!
Thanks a lot for another great video. The tangents are amazing. I would love to see you going into same/more depth. It's easy to scroll through a section if it is too much. An opposite is impossible ;)
Great video and I really appreciate the leveling guidance. The whole time I was thinking of the “showing the same profile twice” issue, so I’m glad that was covered at the end. I still think it was worthy of covering up front though in the functional requirements, because seeing the same profile repeatedly seems like it would annoy users. One other bit of feedback: the font used is hard to read on mobile, especially when zooming out to a wider view. I get that I’m trying to view a video with a big diagram on a tiny screen and may have to squint, but the font is making the readability worse.
Thanks, Evan! This is amazing. Quick question. The calculation you did for the average user swipes per second, would we need to do similar calculations in a Meta interview?
Hi Evan, thank you for sharing the high-quality video! One quick question: in the step of drawing the design diagram, when do you use one-directional arrows between boxes? When do you use the bidirectional arrows? I am looking forward to your reply. Thank you!
First of all, Thanks a ton, this is really helpful, the delivery framework and deep dive is great, it makes everything easy to follow. Regarding using Cassandra, could it be replaced by DynamoDB and consistent reads. If i understand dynamodb correctly then it will solve the issue without any write scaling penalty, please correct me. Also the redis cache solution for consistency, what would be the TTL for this cache, I think it should be a greater than the time for cassandra write to propogate to all the cluster nodes something like 60 seconds at max. Please clarify. Suggestions The tangent explanations which made this video longer can be a separate ~15 minute videos. Please consider tagging system, Atlassian asks this a lot.
Another great video by the HelloInterview team :) A point of clarification - it was mentioned earlier that swipes would require strong consistency, but then later on it was mentioned that Cassandra is ideal as a database of choice given its stronger write performance. However, Cassandra in the context of CAP theorom is an eventually consistent database. Are we saying that Cassandra is still a great choice because the level of consistency in Cassandra can be tuned to be either strong or weak? Thank you!!!
50:34 I would not recommend SAGA pattern here. 2 Phase commit is a better option here I believe (if we have to deal with distributed transactions). SAGA pattern usually implies that different operations are happening (with possible rollback) on business layer with eventual consistency. While 2PC tries to abstract away the distributed transaction as atomic operation on business layer. Moreover if we are ok with eventual consistency here, instead of 2PC, we can try to utilise CDC from Redis to Cassandra (would improve write throughout for users)
Loving the videos. I have a couple system design rounds coming up despite the roles being for frontend. How deep in terms of infra/backend knowledge do I need to go? I've never worked with most of these technologies like redis/cassandra/CDC/locks/partitioning etc. so it feels like im mostly just regurgitating knowledge from youtube videos rather than real work experience. Are system design interviews for frontend candidates focused more on browser caching, tokens, debouncing, throttling etc. or is it identical to backend interviews?
As far as I remember, we wanted to have swipe db as strongly consistent. We used cassandra as it is optimized for writes and write vol for swipe is far greater than set profile/getStack. But cassandra by itself prioritizes availability over consistency. So , how can we achieve initial cap requriement.
Could you please explain why you decided not to configure Cassandra to have a quorum for reads and writes (R + W > N)? Was it mostly because it will affect write throughput?
Great content. An advice, could we use a normal font rather than the current one? Because that is hard and require more time to recognize the text. Thanks!
Thanks for the video! When implementing the Redis solution, wouldn’t availability of cache resources be an issue? Now every swipe event will cause a check in the cache for the inverse swipe. If redis is single-threaded, wouldn’t that overload the cache?
thanks for your sharing, great content there. regarding the cassandra for write, I am assuming the row keys are user ids, but how about the columns? is the column also user id? then how do we deal with the case where column reach 200millions? I just check that with gpt that the performance would degragaded after the column reach 200m. are we missing the assumption that the system will handle less than 200m users?
Hi, Really Thank you a lot. Just curious, Is it possible for you to do some deep dive about different databases and How should we pick the right database in different cases. Thank you very much.
I'd like to record myself the same way you have shot these videos (screen share of excalidraw and popout view of the candidate) so that I can practice. What setup and tool do you use? BTW great videos! Keep up the good work!
Why is it only searching by location in the postgres profile DB that's inefficient? how is checking "a < longitude < b and c < latitude < d" more inefficient than "25 < age < 35 and gender == F and ..."
the age query in your example is equally as inefficient. The 'gender = F' is fast because we'd have an index on gender. Even with an index on numeric fields, its still a range, so you need to scan.
How do we decide who is user1 and user2 for the 3rd approach to maintain consistency? Imagine user 1 and user 2 match at the same time how will both users check if an entry already exists in the swipe db?
Hi, Evan! Thanks for the great video; it was really insightful. However, I'm still not sure why the Redis solution works. I understand that you're using an atomic write (swipe) and read (check if the inverse swipe exists), which should ensure that two such simultaneous write-plus-read operations wouldn't interleave.cHowever, couldn't it be the case that Redis is also eventually consistent? I mean, Redis could be using its own replicas with asynchronous replication. So, the user swiping later might not be able to see the inverse swipe if it hasn't been written to the replica they're reading from yet. The same issue could happen with the Postgres solution-while we are now writing to a single row, it could still happen that they might be reading from a replica that has stale data. Or is it that initiating a transaction forces data to be read from the master node, eliminating the possibility of stale reads? Could you please clarify these points?
@@hello_interview Thanks for replying, but I think you meant reading to the leader and not writing to it. Since, I guess, writes are always to the leader. It's the read that could create an issue. However, if you start forcing to read from the leader all the time, you defeat the purpose of replicas. I guess it's one of the scenarios where you can trade availability for consistency.
Thanks for the amazing video! I'm a little confused why we need both Redis and Cassandra? If we're writing all the likes to Redis anyway, do we really need them in the db? Would Cassandra just be more for the dislikes/left swipes?
Wouldn't a PUT or PATCH request be more appropriate for setting profile details since the requirement was to update those preferences, not create the profile with those preferences?
what about having a Swipe capture service and then pass the data to distributed messaging system like kafka and then consumers consume and does the matching (match service) and send a notification ?
Thanks for the wonderful explanation. I have a question, let say building a simple website and if we don't want to create api to connect the frontend with the database then what is the other possible way, is it possible to read and write data between database and website?
when you propose option3 with postgres instead of the Cassandra, did you cover the downside as you need to have more nodes to support the write per second?
one question i have , for the getstack api our main problem is the two dimensional nature of location, so can we introduce a redis cache which will store only lat lon, and leverage geohashing to get proximate users, when getstack call will come we will first bring out all the users proximate to this one and then we will search in the profile database with other filters writng quereies. can you plse tell what will be the tradeoffs in this approach
Yah any geospatial index works. Redis is fine, could replace elastic search with it. Just means you can’t do the other filters in the same db. So I’d opt for either Elasticsearch or just extensions on Postgres.
plse dont decrease the length, we want these in depth explanations, we are always learning something new and your explanations are great, watching your videos i actually started to think about tradeoffs and what can we use
+1
+1
+1
+1, as long as you note the moments you’re going into a deep dive that you wouldn’t necessarily do in an interview, I’d love to see more discussion like you did on why Cassandra is better setup for
high write load than pSql.
Super detailed and enjoyed learning about trade offs. Learnt a lot.
This is so amazing that I don't have words to appreciate the details of the video. Your content is by far the most accurate and some thing that is done with the correct intent to teach.
💙
I would say these videos are not long enough! More and longer deep dives please. And also side tracks like quickly explaining Cassandra. ❤
🫡
@@hello_interview the beauty of the system design question is the way you solve one part of the whole question can be applied to another question: for example, splitting a read-heavy and write heavy service into multiple instance and scale them independently. so please keep it as what it is right now. And at the end of the day, we are preparing when watching, time is not the limit, but knowledge is.
Your videos are literally the BEST system design content out there! Concise, clear, and complete! Great stuff!
You have the ability to articulate things in a very nuanced manner whilst still being quite comprehensive. I am someone, trying to level up as a software engineer from senior to staff level. And I find your content absolutely enriching. Keep educating, you have all the skills needed for an "Exceptional Software Development Educator".
You guys do amazing work. In all honesty I think you guys are all great but Evan is something else! Last week I had like 3 days to prepare for the system design portion of my onsite and literally all I did was find the videos that Evan made, try the problem myself for 1-2 hours, and then watch the video. I ended up getting an offer!
Evan's style and even the way he talks (if you can somehow subconsciously copy it) is exactly what interviewers are looking for it seems!
“Even the way he talks” 😂 idk about that , but hyped you got the offer!! - Evan
Oh man, i have been waiting for a new video on this playlist for a long time!! 🙂
Thanks for your patience! 🫶🏻
@@hello_interview No problem. I have an interview coming up with Amazon (L5), preping for it. This is very helpful.
Good luck, you've got this!
I would like to thank you for your decision to teach system design for free. Really it’s worth to spend my valuable time on hello interview video🤞🏻
i am HOOKED. consistency in the context of Tinder ...just awesome. Great job Evan!
you ROCK 😛
This is the best HLD content available on youtube.
The way you explain complex things so elegantly is just amazing!!!
Can’t thank you enough for the amount of valuable depth in these videos.
Thanks A LOT Evan! I appreciate the high-quality content-going beyond the introductory level-that you and your team are producing. Thank you for your hard work and dedication. Wishing you continued success in your business. Cheers.
Appreciate you so much!
Hands down the best approach to system design interviews compared to everything else i have watched on youtube. Love your systematic approach. Pointing out what is relevant, what isnt, what points are signals to judge your level etc. Great work!
One of the few places on RUclips helping staff design interview, thanks for helping us
🫡
I have never commented on any video but this video truly deserves my word. This was probably the best system design video I have watched. It gives such details that are required when you want to level up your skills (Senior SA/design or Staff SA/design). Loved the details and I wished all areas had been covered in depth. I could have easily watched two more hours. Please go in depth in all areas even though I think it could get repetitive for you. But you are doing the community a favor by diving deep.
Very kind :) glad you liked it!
Your videos play out like an action thriller. I am hooked from the beginning to the end and I learn so much from each video. The tangents are the essence of your tutorials so please do keep them coming!
Haha you're too nice! Comparing a system design breakdown to an action thriller is something I never thought i'd hear.
Evan, even you might thought this video is too long from your end, but from my perspective, this one, surely is in the TOP2(Uber if you ask) videos of yours. Really really appreciate your dedicated work and sharing: you mentioned MANY different ways of achieving a same thing. Even some of the ways isn't taken in this problem, they may be taken in some other cases, I really enjoy watching you evaluating different approaches. We know this certainly takes you lots of time and energy, so thank you again for putting everything together. Thank you!
This was really nice to read 🫶 appreciate you!
I would swipe right on this video a hundred times 😍 perfect balance of breadth and depth. the context on level expectation is super helpful. Looking forward to the next video; would particularly love an infra-focused one after this product-focused topic!
Coming right up!
"We might need to select a partitioning key for Redis to ensure that the reciprocals show up on the same node"
I never thought I'd ever have goosebumps from a system design explanation
This dudes explanations are so perfect, well organized and easy to understand for people like me who are more around the mid level, thank you!
Hi Evan,
Yet another masterpiece. I loved the depth you went into. I would like to see same or more depths in videos even if it becomes a 75-90 mins video. Please keep them coming!
video length is good, it's all important info with different approaches, never felt overwhelmed!
Kudos to Evan and team. As a senior engineer I find this channel so useful for System Design. The delivery framework works like a charm. Could you guys please start the series for OOAD, because that is where I get stuck sometimes in picking up the correct design pattern. Overall I like the content of the channel and the pace at which each and every aspect is covered. Cheers.
And just like GOT drops the hint, you have dropped the hint of next system design video "Distributed Job Scheduler".
The QnA section speacialy in Deep dive is realy awesome and really helps. Thanks
This content is very high quality, well done.
Amazing video. I am a mid level engineer. And the staff level engineer's standpoint helped understand how they perform and whats expected from them.
Lol you are great, I stopped the video half way to comment this. The way you explained the eventually consistency problem where two users swiped each other at the same but Cassandra being eventually consistent will prevent them to ever know and they will not be able to find the love they are seeking lol. Amazing Stuff. Requesting you to create more and more content. You are a legend!
😂 ♥️
I have my amazon interviews coming up and your videos have been an amazing resource. Thank you!
Amazing lessons! These are pure gold. I am deeply grateful for all the hard work you are doing. I would have one suggestion if you have time and energy. For me it is quite interesting also to deep dive in topics like api gateway and load balancing. It would be great to have a deep dive on it. Many many thanks for your effort, work and dedication. Keep going please. Cheers!
Hey Evan. I am a former FAANG staff engineer and I am interviewing actively now. Your content is the best out there. The deep dives are the best. They've helped me a lot. Please continue creating content. And if there is a Patreon or something I can contribute to then please let us know.
Awww man you rock! No need re-patreon. Tell a couple friends about hello interview and we’re more than even. Cheers
@@hello_interviewI have already told a few friends and will continue telling more people :)
I cannot believe we have this information available for free!!!
🆓🫡
Keeping going Evan (dude in the video) :p, really love your deep dives of the video and system design in general. Love your content and always wait for the new one. videos are lengthy but content is gold.
🫶🫶
I've enjoyed the system design interview videos a lot! Could you potentially do a video on behavior interviews in the future? I know you have an article on the website but a video would be super helpful!
Definitely!
This is one of the best channels teaching system design out there. Can you please do a system like google docs as well? Please please please please please. 😁
Thank you so much for this video ! Appreciated the level of detail-it wasn’t too long at all
I like the way you explain how to do the system design interview. Very clear and concise!!!
🫡
Hey Evan, your videos are amazing! They make system design look so easy and open my mind to think in different directions. I really appreciate your efforts. I’m eagerly waiting for more videos! :)
Don't forget the S3 storage for profile photos! Dating app without pictures is incomplete from a product perspective. Other than that little oversight, fantastic walk through. Thank you!
Beb wake up. Another hello interview video just dropped.🚀
😂😂
Evan, I could watch 5 hours of video if you do it. Your content is simply brilliant, every time I watch, I fell like I learned for weeks 😅
🥲💙
Commenting, so whenever someone likes it, I will come and rewatch this. Thanks Evan!
Putting a reminder to like this every day to pump our view counts.
I'm really enjoying your videos. Well-structured, and with a variety of example systems, the repetition is really useful. You are reducing the intimidation factor, too! Thank you thank you for the "leveling" comments, that makes these videos so much more useful, and also helps us to judge for ourselves where we fall in that rubric. I love your writeups, too!
One question I have is how much time per "section" would you spend in a real interview? Like, 5 min, 10 min, etc.
Totally! We have some recommendations here: www.hellointerview.com/learn/system-design/in-a-hurry/delivery
Super useful, don't shorten.
Guys, thank you! Good length, good material, some points a bit controversial but I like your style! Thank you!
Controversy sells 😉 kidding, but glad you like it!
Cassandra does have row level atomic operations. If we modify the schema to have Swipe table contain decisions from both swipes, we have the transactional properties that we need.
Such solid content. Thanks man!
Very nice deep dive! 👍 I was thinking, how about if we don't keep log of swipes but rather organize them in some kind of user's lookup state (which would consolidate likes and dislikes), may be in redis as well? Then we would pipe swipes through message queue, match with and update lookup states, and then send notification on successful match. This should help to avoid consistency issues as well. And at end we can add like-flag to users in stack response (if we already got their like for the current user before), so we can get that instant match-on-like directly on client side.
Make them of 2 hours, if anything. Super good!!!
@Evan Great video! I have a question about the consistency solution with Postgres you proposed around 53:00. You mentioned having a single row for swipes with columns like user1, user2, user1Decision, user2Decision. However, wouldn't this approach still lead to potential inconsistencies if two users swipe on each other simultaneously?
For example, if
A right swipes B and
B right swipes A at the same moment,
wouldn't they create two separate rows with different user1 and user2 values? I'm curious how you would address this scenario to ensure consistency. Thanks!
Also, I was thinking about a potential solution. What if we always ordered the user IDs consistently when inserting, like alphabetically or numerically? Then we could use a unique constraint on (user1, user2) and an UPSERT operation. This way, regardless of who swipes first, we'd end up with a single row for both users' decisions. Might solve the consistency issue without needing two separate rows. Just a thought!
Thanks a lot for another great video. The tangents are amazing. I would love to see you going into same/more depth. It's easy to scroll through a section if it is too much. An opposite is impossible ;)
True!
The Best system design videos
😊
The beta system design videos viewer 😉
This resource is a gold!
🏅💙
Hey Evan, could you please please please make a video on News feed as well. 🥺 Love your content.
Great video and I really appreciate the leveling guidance. The whole time I was thinking of the “showing the same profile twice” issue, so I’m glad that was covered at the end. I still think it was worthy of covering up front though in the functional requirements, because seeing the same profile repeatedly seems like it would annoy users.
One other bit of feedback: the font used is hard to read on mobile, especially when zooming out to a wider view. I get that I’m trying to view a video with a big diagram on a tiny screen and may have to squint, but the font is making the readability worse.
Fair point! Heard this a couple times. Will try a different font and more zoom next time
After watching this video, I'm 100% sure I want Evan as my manager lmaoo he looks very serious, kind and maybe also very funny
Only maybe funny 😝
Great content as always Evan!
Amazing content. Thank you very much♥
PLEASE start using the dark mode for excalidraw, i watch these at night to prep haha.
Your explanation is amazing, thanks a lot for content
Learned a lot through this video. 🔥Thanks a lot.
Very nice video, length is great, if one can't sit through an hour long video, probably he is not ready for this topic anyway.
Thanks, Evan! This is amazing. Quick question. The calculation you did for the average user swipes per second, would we need to do similar calculations in a Meta interview?
Thank you your explanations are the best, please also do one for Netflix I got asked that recently and its the most interesting.
Check our RUclips write up. Pretty similar.
Our love is in the air, and you have it all!
♥️♥️♥️
Hi Evan, thank you for sharing the high-quality video! One quick question: in the step of drawing the design diagram, when do you use one-directional arrows between boxes? When do you use the bidirectional arrows? I am looking forward to your reply. Thank you!
Tbh I don’t focus on it much. I’m sure I have inaccuracies there. Yo be correct it would be whether the flow of data is bi- or uni-directional
Very very nice video.
Thanks for great video, one question on swipe db, what is the partition key based on as two users are required to form the key.
excellent! thank you very much
First of all, Thanks a ton, this is really helpful, the delivery framework and deep dive is great, it makes everything easy to follow.
Regarding using Cassandra, could it be replaced by DynamoDB and consistent reads. If i understand dynamodb correctly then it will solve the issue without any write scaling penalty, please correct me.
Also the redis cache solution for consistency, what would be the TTL for this cache, I think it should be a greater than the time for cassandra write to propogate to all the cluster nodes something like 60 seconds at max. Please clarify.
Suggestions
The tangent explanations which made this video longer can be a separate ~15 minute videos.
Please consider tagging system, Atlassian asks this a lot.
Another great video by the HelloInterview team :) A point of clarification - it was mentioned earlier that swipes would require strong consistency, but then later on it was mentioned that Cassandra is ideal as a database of choice given its stronger write performance. However, Cassandra in the context of CAP theorom is an eventually consistent database. Are we saying that Cassandra is still a great choice because the level of consistency in Cassandra can be tuned to be either strong or weak? Thank you!!!
Evan talks about it at 44:06 - nice!
Keep watching :) we get into that
Thank you for such a great video
50:34
I would not recommend SAGA pattern here.
2 Phase commit is a better option here I believe (if we have to deal with distributed transactions).
SAGA pattern usually implies that different operations are happening (with possible rollback) on business layer with eventual consistency. While 2PC tries to abstract away the distributed transaction as atomic operation on business layer.
Moreover if we are ok with eventual consistency here, instead of 2PC, we can try to utilise CDC from Redis to Cassandra (would improve write throughout for users)
Loving the videos. I have a couple system design rounds coming up despite the roles being for frontend. How deep in terms of infra/backend knowledge do I need to go? I've never worked with most of these technologies like redis/cassandra/CDC/locks/partitioning etc. so it feels like im mostly just regurgitating knowledge from youtube videos rather than real work experience.
Are system design interviews for frontend candidates focused more on browser caching, tokens, debouncing, throttling etc. or is it identical to backend interviews?
Can’t speak too confidently to this, never given a front end interview myself. Checkout thegreatfrontend though. Great stuff over there!
Your content is dope, Thank you
You’re dope 🫡
Isn’t redis eventual consistent when using clusters? Or would you use a single instance? How would you scale that?
Thank you so much for this amazing contents. Really helpful.
Which is this software you are using for depicting the design?
Excalidraw
As far as I remember, we wanted to have swipe db as strongly consistent. We used cassandra as it is optimized for writes and write vol for swipe is far greater than set profile/getStack. But cassandra by itself prioritizes availability over consistency. So , how can we achieve initial cap requriement.
i think it is replied in 'deep dive'. I did deep dive next day.
Could you please explain why you decided not to configure Cassandra to have a quorum for reads and writes (R + W > N)? Was it mostly because it will affect write throughput?
Yah great call out. Thats exactly why. It basically neutralizes the reason we chose it in the first place.
Great content. An advice, could we use a normal font rather than the current one? Because that is hard and require more time to recognize the text. Thanks!
Next time!
Could you pls elaborate more on lat/long problem - when you say we are using indexes to query two dimensional data?
Thanks for the video! When implementing the Redis solution, wouldn’t availability of cache resources be an issue? Now every swipe event will cause a check in the cache for the inverse swipe. If redis is single-threaded, wouldn’t that overload the cache?
In memory! Redis clusters can handle hundreds of millions of reads a second
thanks for your sharing, great content there.
regarding the cassandra for write, I am assuming the row keys are user ids, but how about the columns? is the column also user id? then how do we deal with the case where column reach 200millions? I just check that with gpt that the performance would degragaded after the column reach 200m. are we missing the assumption that the system will handle less than 200m users?
Hi, Really Thank you a lot. Just curious, Is it possible for you to do some deep dive about different databases and How should we pick the right database in different cases. Thank you very much.
Will add it to the list!
I'd like to record myself the same way you have shot these videos (screen share of excalidraw and popout view of the candidate) so that I can practice. What setup and tool do you use? BTW great videos! Keep up the good work!
Excalidraw and screen studio!
Great video... thanks! Liked Subscribed Commented! Keep up the great work!
Appreciate you!
Why is it only searching by location in the postgres profile DB that's inefficient? how is checking "a < longitude < b and c < latitude < d" more inefficient than "25 < age < 35 and gender == F and ..."
the age query in your example is equally as inefficient. The 'gender = F' is fast because we'd have an index on gender. Even with an index on numeric fields, its still a range, so you need to scan.
I can not believe this video only has 694 thumb up......
How do we decide who is user1 and user2 for the 3rd approach to maintain consistency?
Imagine user 1 and user 2 match at the same time how will both users check if an entry already exists in the swipe db?
Hi, Evan!
Thanks for the great video; it was really insightful. However, I'm still not sure why the Redis solution works. I understand that you're using an atomic write (swipe) and read (check if the inverse swipe exists), which should ensure that two such simultaneous write-plus-read operations wouldn't interleave.cHowever, couldn't it be the case that Redis is also eventually consistent? I mean, Redis could be using its own replicas with asynchronous replication. So, the user swiping later might not be able to see the inverse swipe if it hasn't been written to the replica they're reading from yet.
The same issue could happen with the Postgres solution-while we are now writing to a single row, it could still happen that they might be reading from a replica that has stale data. Or is it that initiating a transaction forces data to be read from the master node, eliminating the possibility of stale reads?
Could you please clarify these points?
You need to make sure you're writing to the leaders to ensure consistency in both cases, but as long as you've done that you're fine.
@@hello_interview Thanks for replying, but I think you meant reading to the leader and not writing to it. Since, I guess, writes are always to the leader. It's the read that could create an issue. However, if you start forcing to read from the leader all the time, you defeat the purpose of replicas. I guess it's one of the scenarios where you can trade availability for consistency.
Thanks for the amazing video! I'm a little confused why we need both Redis and Cassandra? If we're writing all the likes to Redis anyway, do we really need them in the db? Would Cassandra just be more for the dislikes/left swipes?
Because Cassandra is eventually consistent. So two matches coming in at around the same time may go without trigging a match.
Wouldn't a PUT or PATCH request be more appropriate for setting profile details since the requirement was to update those preferences, not create the profile with those preferences?
thanks, just awesome
what about having a Swipe capture service and then pass the data to distributed messaging system like kafka and then consumers consume and does the matching (match service) and send a notification ?
Love in the air❤❤❤
hehe 😍
Thanks for the wonderful explanation. I have a question, let say building a simple website and if we don't want to create api to connect the frontend with the database then what is the other possible way, is it possible to read and write data between database and website?
Yeah, most apps start this way. Hello Interview is actually set up like this :)
What would you say if someone considered messaging a match as a requirement & "Conversation" and "Message" as core entities in an interview?
For write throughput and consistency, why not using dynamo consistency
Same considerations as the Postgres trade off mostly. Could work, just also more expensive
when you propose option3 with postgres instead of the Cassandra, did you cover the downside as you need to have more nodes to support the write per second?
Yah think it’s mentioned. At least by way of “more expensive”. The implication there is more hardware
@@hello_interview but the challenge is much more than that? As we need to shard/replicate the db ?
@@Richard-yw9ifthere are middleware services like Citus which handle this, i think they even have a managed service or a k8s opertor for it.
one question i have , for the getstack api our main problem is the two dimensional nature of location, so can we introduce a redis cache which will store only lat lon, and leverage geohashing to get proximate users, when getstack call will come we will first bring out all the users proximate to this one and then we will search in the profile database with other filters writng quereies.
can you plse tell what will be the tradeoffs in this approach
Yah any geospatial index works. Redis is fine, could replace elastic search with it. Just means you can’t do the other filters in the same db. So I’d opt for either Elasticsearch or just extensions on Postgres.
@@hello_interview got it thanks for the reply🙌🙌
Could you add some LLD Machine Coding System Design. Probably any structure for solving them.