I second. I wonder what does it take to make it popular. There should be something simple. I am watching a few other channels. Nobody covers it to this level. May be more videos required. on short topics. Say 10-15 mins .
Fantastic series of videos. I've worked on very large distributed caches (multiple terabytes, thousands of servers) and some other fun stuff to mention to the interviewer is: - Failure modes besides just plain qps. Specifically, Redis will be overwhelmed by connection churn if it receives thousands of new connections per second. Twemproxy, on the other hand can handle this just fine. I have not tested this with Redis Cluster, but I suspect Redis Cluster also suffers from this. - Auditing: With GDPR and California's analog to this, the legal department often is interested in how long we store data and where. For the former case, they may want a global TTL, or failing that, metrics and alerts when the cache TTL is too high. For the latter case, we may need to separate out data from users in various countries at the client level. - Maintaining client libraries: How do we make sure everybody is on the newest version of the library? How do we manage multiple implementations? For Redis Cluster, this is a big problem- it turns out different clients have very different characteristics when it comes to detecting failures of nodes. Using a sidecar proxy like Envoy can be a big help as we can manage retries and health checks in a more principled manner. - Testing: Frequently service owners don't think about how their service behaves with a degraded cache. Chaos monkey is a step in the right direction, but it is a blunt tool. Instead we can add knobs to return a percentage of errors, or delay requests, allowing us to really understand how we need to improve the service- for example retry intervals.
@@SystemDesignInterview Unfortunately you have so few videos, I've been watching them over and over before my system design interview. Top K heavy hitters is my favorite one, probably watched it 5 times already.
You know a video is good, when you have to pause the video constantly and process the information. Thank you so much for your time and effort on these high quality and thoughtful lectures!
That is absolutely true. I usually watch other videos in 1.5x time so I complete them fairly fast but for this video I have to pause it many times to take notes and process information so it takes more than the video's length (easily 2x).
Discovering this channel in 2022, I wish there will be more updates. This is one of the best system design learning resources, and it's free. Every sentence is so on point. Thank you very much!
your videos are detailed in just the right places. Probably testament to your years of industry/academic system design work. As a mid-level (5 YOE) engineer, the trade-offs, optimizations and advanced details are just what I need to excel in my career. Can't thank you enough.
Thank you David for a detailed feedback. Not the first time I hear good words from you. I have a concern that my videos may be a bit complicated for less experienced engineers, as some more basic concepts are not well covered in videos. I rely on the fact that people know them. Which may not be a correct assumption. I would like to create videos that cover those more basic concepts. Will see. [A call to everyone, not just David] If you think that some areas deserve a more thorough coverage, or on the contrary, some explanations are too deep and should be trimmed, please let me know. Your feedback is always appreciated.
By far the best example of a structured approach to work out a system design problem. And in this particular case, a better solution than other videos on RUclips, Grokking and Interview Cake.
@@SystemDesignInterview you deserve it. I took another look at your code and have a question about a minor detail. Do you really need the key in each node? I think that keeping the value should be enough. Through the key, you get the reference to the node in the hashmap, and from there the value.
Hi Diego. Good question! Storing key in the node helps when we need to delete an entry from the cache when it is full. And more specifically, for this line: _map.remove(tail.getKey())_
I've watched all the popular channels on YT for system design and this one is without question the best one. Thank you so much for creating these videos. I hope you make more.
hands down, the best distributed cache explanation on RUclips. Every other video talks about caching and not distributed caching. This is a complete crash course on distributed caching.
This is the only channel where it feels like if we were to actually build something how we would do it. Other channels are mostly yeah put some load balancers, a reverse proxy bla bla bla. The way you go in-depth about each thing and make us aware of the tradeoffs is truly incredible. I watched the view counter and heavy hitter and came here, and surprisingly my thoughts ran exactly as parallel as you. Looks like I am learning. Thank you so much Mikhail.
Another great video, I can't say the importance of details in system design and as far as I've seen only your videos cover it. Don't get swayed by other 'scratching the surface' videos, keep up the good work!
I'm short on words how good Mikhail explains things. Initially I was just going over the video without making any notes. Then once I started making notes, I realized there are tons of information in everything single component or design choice he explains.
I liked the way you started from simple local system and evolved to distributed cache by solving all the challenges on the way. Your videos are very informative!!
This is a complete video and in just 34 minutes. The hard work you have put in this video is evident. You have covered many scenarios that can be reused in other system design questions.
Watched other youtube videos too, but this one is the best. It starts with the requirements, then talks about LRU cache and even it's quick code implementation. Then talks about how cache host is selected (like Consistent Hashing) and how Cache Client selects the Cache Hosts (talks about all 3 ways). Also, nicely explained with diagrams. Thanks a lot.
This is by far the best demonstration of designing a distributed cache. I really appreciate the way you brought up the the solution to a bigger problem in a progressive approach. Loved it. Keep growing.
Wow. I'm currently studying system design and have to say this is by far the best resource I've run into and I've paid for multiple services and classes on system design. Thank you for creating these awesome videos, you definitely have a +1 follower.
Hey Daniel. I also liked this channel very much. Unfortunately it stopped receiving uploads for now. Do you mind to share from the services and classes you've paid each ones you liked the most? So I can continue my studies from here to somewhere else.
I have watched too many video on this. Yours is simply the best so far. Some just scratch the surface, others just dive straight into complex technicalities. This is class apart. I wish you upload more stuff in the future!! A big thanks again...
Great explanation! I hope you will continue with posting more System design interview videos. The fact that you started from the naive approach to a better solution I consider is a big thing to consider all pros/cons!
This channel is highly underrated!. This is the very best explanation of distributed cache I have ever come across. It's simple and touches every core components through a concise explanation. Big thanks Mikhail!, expecting more uploads.
I like your style of repeating concepts throughout several videos. It helps reinforce them and allows the viewer several viewing opportunities to let it sink in over different videos (versus just watching the same video over and over to absorb something, which can get boring).
One of the best system design videos. Actually much better than most of paid video contents. The creator of this video clearly demonstrated his deep understanding to the topic.
Got exactly this question on a system design interview. Thanks a lot! I was properly prepared. This is the best set of lectures on system design I've ever saw.
Hi Mikhail. This is the first video that I watched on the channel and I have to say it's the best resource out there. I love the way you progress to a better solution at each step. Keep up the good work. I'll use this information while implementing a distributed cache in my project. Thanks.
The details and trade-offs you talk about clearly indicate that this information is acquired through a lot of practical experience. A lot of hard work has gone into this from your end. Thank you for sharing your knowledge. Look forward to more videos from your channel.
I had read about Partitioning and Replication in isolation and their tradeoffs but was looking for how it comes together in a real-world application. Every second of this talk is pure gold :) I had gone through other System design articles/talks but they all just scratched the surface and was unsatisfied. The way you have progressively enhanced the solution is very apt and I really like the references for further reading on consistent hashing. Kudos!
Thank you very much, Swapnil, for such a detailed feedback! Glad you liked the video! Let me know if you have any questions. About this or any other video on the channel.
Your channel and explanation of System design for various topics is so neat and elegant. Even before you dive into the design aspect of the question, the identification of the functional and the non functional requirements itself is so useful. Truly yours is a very under rated channel. Hope to see many more videos from you.
You are so awesome. THis is the only channel which gives a easy to complicated solution instead of explaining how redis works withs its huge complicated architecture which is not useful for system design interviews.
The content that you've put is extremely high in quality, leaving nothing to chance. I haven't seen such pure content since DDIA. Many thanks for making it available for free.
Excellent video! Quality of content is top notch. I spent hours reading up on each components before I came across your videos. You are doing an amazing job!
Although it’s covered at a high level, the video covers a wide range of factors to be considered when designing the system, very informative and helpful to work out an all rounded solution. Highly recommended!
This video provides an excellent balance of high-level concepts and diving deep into details. An incredible resource to find, and vastly better than many paid options out there on systems design.
You channel is the best. I have seen so many system design videos so far. But the way you cover smallest of the details is awesome. You deserve claps from side. 👏
He is one of the best, I have seen so far. Full details, with integrated parts of system, covering all the aspects. Awesome!!! Please make some more videos like this.
I guess he is the only who covered and connected the dots between LRU cache implementation and how to transition that to a distributed hash table problem. The content is amazing !!!
After watching many videos on consistent hashing, I am glad that I came across yours. The way you explained the process was simple and easy to understand. I really appreciate your efforts in putting together quality content.
Beautiful video. Thanks a lot, Man! I recently finished my masters in Distributed Systems and this video helped me consolidate my otherwise Distributed Knowledge ;)
By some reason RUclips hides valid comments. I can only see such comments in the inbox, but there is no way for me to reply. Let me re-post such comments on behalf of people who submitted it. From @krfong krfong Hi Mikhail, thank you for taking the time and effort to make these videos. I'm currently preparing for interviews, and your videos have helped reinforce my knowledge, in some parts, and reveal my gaps in others, of how to design distributed systems components. 😊 THANK YOU!! I'd like to address your question above - whether or not you cover the right amount of breadth and depth in your videos. First, as I understand the audience that you're teaching to (and let me know if I've got the wrong impression) - new grads and swes who are not yet sr. or put another way, those who have conceptual knowledge of distributed systems, but are not yet able to synthesize this information and apply them to real-world applications, and also, those who do not yet have the experience of designing/explaining conceptual models in an interview format. If, in fact, you are creating for these kinds of people, I'd like to say that you should continue to teach for them because your videos address a (much-needed-to-be-filled) gap in the space of accessible distributed systems learning materials. Current accessible web content like Guarav Sen's and Tushar Roy's videos on system design, reading materials such as Martin Kleppman's "Designing Data Intensive Applications", Mikito Takada's book "Distributed Systems for Fun and Profit", the High Scalability blog, and seminal papers in distributed systems, all are not able to teach distributed systems from the angle that you do. What those resources are unable to address (and what you've been able to do) is to synthesize distributed system concepts and demonstrate how to apply them to designing distributed system components in an interview format. Your videos have been able to put a language, a loose-script, if you will, of how to have a conversation with others about distributed systems; that is vital for those who are learning how to design distributed systems in a considered way. I think you have a good balance of breadth and depth. To offer a suggestion, people could benefit from your videos if you discuss how your designs' address questions like failover scenarios, how the design addresses r/w skew, dirty r/w and lost updates, and linearizability (or consistency) implications towards the end of your videos. Lastly, I'd like to ask a question, are your videos representative of how you'd suggest one should discuss building a system design component in an interview? If anything, what have you left out of your videos that candidates should consider in a system design interview, generally speaking and in specific to this video, or the notification or distributed rate limiting ones? I love your work, thank you, and awesome stuff!!
Hi @krfong krfong. I can imagine amount of effort you put into writing this feedback. Very insightful and informative. Appreciate it a lot! Let me try to answer your questions. I am sure you know it already that there is no one single format for answering system design questions. With coding questions things are much more clear. In a nutshell: we start with asking clarifying questions, evaluate different options for the algorithm, write the code and explain our thought process along the way, test and make sure we covered edge cases, estimate time and space complexity. But with system design, interview may go in many different directions. That is why I try to touch various concepts and go deeper into more important of them (on my mind). So, there is no one single format. But we can talk about expectations. For less senior engineers, an interviewer wants to see how you think. What challenges you see, what options you consider. Remember, that nobody expects ideal design after 45-60 minutes interview. Engineers spend months (depending on the size of the problem, of course) to design systems properly. It usually takes several iterations. And if a candidate can analyze requirements, evaluate options and make progress in the right direction, this is a good indicator that when join the company, engineer will apply the same critical thinking and come up with good design during her day-to-day job. When she has more time and help from more senior engineers. And to develop this critical thinking we need knowledge and exposure to software engineering problems. And this is also one of my goals, to share knowledge and how to apply it to solving problems. For senior engineers expectations are higher. It is assumed you already have knowledge and know how to analyze problems. And it is no longer enough to demonstrate a solution that just works. We need to demonstrate how to optimize things and, eventually, how to reduce the cost. All kinds of cost: operational, dollar (e.g. hardware), cost of extensibility in the future. It is important to design systems that not just scalable and highly available (this is a given by default), but systems with close-to-optimal cost. Simplicity always wins. If this topic of system design interview expectations is interesting to my viewers, please let me know. I will make a separate video.
Regarding topics I left out of the videos, this list is not short )) Each concept mentioned in videos may easily become a topic of its own. But I am sure you expect a more definitive answer from me. Let me name some important topics I left out: - For Distributed Message Queue: how to store messages on disk. Here we should mention sequential reads/writes, page cache, embedded databases. - For Notification Service: how delivery to subscribers work, how failures are handled. Here we should mention websockets. - For Distributed Cache: a big topic of memory allocation. These are some of the examples. Also, I do not talk much about hardware, memory, network throughput estimations. Will try to close this gap one day and devote videos to those topics.
Your videos are the best I've ever seen on RUclips. You've made understanding these complex components so much easier. Thank you so much! Hope you keep creating more awesome content.
Very well explained videos. Like everyone mentioned, this channel is awesome. Thank you @System Design Interview. Sad to see no more new videos coming from this channel.
Love the encouragement to dive into details. Easy to see how someone involving technical details instead of relying on managed services to handle it would make them stand out in an interview. Thanks!
the moment he said there are 2 consistency models: Eventual consistency (like gossip, epidemic) and stronger (with 2PC, consensus), I knew this is a golden content.
It has so many details in every min of video. I really appreciate the craft. I have watched every video many times. Waiting for more like: Google drive, Uber, Netflix system design.
Oh my... this is the best ever system design! If I am the interviewer I will be begging the recruiter to hire you, man... The way you layout the problem statement, evolve and improve the system, revisit the requirements and discover further is just amazing. I can't believe you covered so much in just a half-hour.
@8:29 : Our Implementation of LRU cache will have HashMap and Doubly LinkedList. In Which HashMap will hold the keys and address of the Nodes of Doubly LinkedList . And Doubly LinkedList will hold the values of keys.
for the problem of uneven distribution of keys among servers in consistent hashing we can use aliasing to mitigate that. So each server can have any number of aliases hashed to positions on the ring. The name of the alias is based on the server id suffixed with some string value. Key hashes that map to a server alias are serviced by the actual server.
Thank you, Haytham, for these details! Well explained. *A note for viewers*: this concept is also known as "virtual nodes" or "vnodes". Further interested in this topic, please take a look at this great post: medium.com/@dgryski/consistent-hashing-algorithmic-tradeoffs-ef6b8e2fcae8
This is my first comment on any video. What an incredible job you did in this video, I loved how you introduced the problem and suggested their solutions. Not to miss explained in very simple and efficient way. Thank you!
Great Video. Very well explained the relevant Ifs and Buts. Its the best so far on RUclips for Distributed Cache. Thanks a lot. Waiting for more videos.
Wow, I think this channel has the best explanation of the available system design problems, which is able to provide a step-by-step explanation of how we get to the final design. Some other channels just slap the final design almost right away, which can be overwhelming and sometimes leave its sub systems inadequately explained. Please add more system design problems, keep up the great work!
This is an excellent video for understanding the inner working of distributed cache. It starts from a very basic implementation and builds on top of it and clearly explains all concepts of distributed cache and major issues. Very helpful video.
Holy crap! These videos are awesome. It is very systematic but allows for other potential discussion. Thanks and keep creating more content. Will definitely recommend to my friends.
Excellent application and explanation of distributed systems concepts - Partitioning, Replication, Configuration management, Consistent hashing along with pros and cons.
Amazing content. So much to-the-point and valuable information packed in 34 minutes. This one video is equivalent to multiple hours of videos from other channels out there! Many thanks Mikhail.
this is the most underrated channel, kudos to your way of explaining things in succinct manner. Please put more videos
Thank you, Vicky, for the kind words! Working on more videos.
I second. I wonder what does it take to make it popular. There should be something simple. I am watching a few other channels. Nobody covers it to this level. May be more videos required. on short topics. Say 10-15 mins .
@@nirumani put it on LC discussion board
Fantastic series of videos.
I've worked on very large distributed caches (multiple terabytes, thousands of servers) and some other fun stuff to mention to the interviewer is:
- Failure modes besides just plain qps. Specifically, Redis will be overwhelmed by connection churn if it receives thousands of new connections per second. Twemproxy, on the other hand can handle this just fine. I have not tested this with Redis Cluster, but I suspect Redis Cluster also suffers from this.
- Auditing: With GDPR and California's analog to this, the legal department often is interested in how long we store data and where. For the former case, they may want a global TTL, or failing that, metrics and alerts when the cache TTL is too high. For the latter case, we may need to separate out data from users in various countries at the client level.
- Maintaining client libraries: How do we make sure everybody is on the newest version of the library? How do we manage multiple implementations? For Redis Cluster, this is a big problem- it turns out different clients have very different characteristics when it comes to detecting failures of nodes. Using a sidecar proxy like Envoy can be a big help as we can manage retries and health checks in a more principled manner.
- Testing: Frequently service owners don't think about how their service behaves with a degraded cache. Chaos monkey is a step in the right direction, but it is a blunt tool. Instead we can add knobs to return a percentage of errors, or delay requests, allowing us to really understand how we need to improve the service- for example retry intervals.
@@SystemDesignInterview Unfortunately you have so few videos, I've been watching them over and over before my system design interview. Top K heavy hitters is my favorite one, probably watched it 5 times already.
10/10 would recommend to anyone who wants to know about caching
Thank you for the feedback, Amardeep! Really appreciate it.
You know a video is good, when you have to pause the video constantly and process the information. Thank you so much for your time and effort on these high quality and thoughtful lectures!
That is absolutely true. I usually watch other videos in 1.5x time so I complete them fairly fast but for this video I have to pause it many times to take notes and process information so it takes more than the video's length (easily 2x).
Discovering this channel in 2022, I wish there will be more updates. This is one of the best system design learning resources, and it's free. Every sentence is so on point. Thank you very much!
Discovered this channel at the end of 2023 and it’s still top channel.
Hey from 2024
Thanks.
~ 10:45 Local Cache Implementation
Stepping into the distributed world:
10:46 ~ 12:42 Dedicated Cache Cluster vs Co-located Cache
12:43 ~ 14:05 Choosing a cache host: naive approach
14:06 ~ 16:04 Choosing a cache host: consistent hashing
16:05 ~ 17:25 Cache Client
17:26 ~ 20:00 Maintaining a list of cache servers
20:01 ~ 22:13 Summary
22:14 ~ 25:52 Achieving high availability
25:53 ~ 32:17 What else is important: consistency, data expiration, local and remote cache, security, monitoring and logging, cache client, consistent hashing
32:18 ~ 34:33 Summary
The most thorough system architecture rundown with plenty of examples.
Thank you for the feedback, ErrorFr33!
Every time I need to start interviewing again, I keep coming back to your videos. They are are simply the best!🏆🏆🏆
This channel is a goldmine. Shame there's only 7 videos, I could watch this stuff for days on end
I have seen so many system design videos on distributed cache and i can tell you nobody has explained better than this video.
your videos are detailed in just the right places. Probably testament to your years of industry/academic system design work. As a mid-level (5 YOE) engineer, the trade-offs, optimizations and advanced details are just what I need to excel in my career. Can't thank you enough.
Thank you David for a detailed feedback. Not the first time I hear good words from you.
I have a concern that my videos may be a bit complicated for less experienced engineers, as some more basic concepts are not well covered in videos. I rely on the fact that people know them. Which may not be a correct assumption. I would like to create videos that cover those more basic concepts. Will see.
[A call to everyone, not just David] If you think that some areas deserve a more thorough coverage, or on the contrary, some explanations are too deep and should be trimmed, please let me know. Your feedback is always appreciated.
By far the best example of a structured approach to work out a system design problem. And in this particular case, a better solution than other videos on RUclips, Grokking and Interview Cake.
Thank you, Diego, for the feedback. Much appreciated!
@@SystemDesignInterview you deserve it. I took another look at your code and have a question about a minor detail. Do you really need the key in each node? I think that keeping the value should be enough. Through the key, you get the reference to the node in the hashmap, and from there the value.
Hi Diego. Good question!
Storing key in the node helps when we need to delete an entry from the cache when it is full. And more specifically, for this line: _map.remove(tail.getKey())_
@@SystemDesignInterview Oh yes.. it makes sense. I missed that. Thanks.
Clear, concise, and crisp. Feels like hitting a gold mine in the world of junk
The coverage of videos is both deep and wide. Comprehensive! this guy leaves no stone unturned.
Took me 3 hours to finish the video. Each and every statement had so much information to process. Absolutely amazing video.
Glad it was helpful, Heisenberg! Thanks for the feedback!
I must say, underrated channel. This deserve lot more views. The uniform approach you followed in all videos, just awesome. Thanks a lot.
Thank you, Subba, for the kind words! Much appreciated.
I've watched all the popular channels on YT for system design and this one is without question the best one. Thank you so much for creating these videos. I hope you make more.
hands down, the best distributed cache explanation on RUclips. Every other video talks about caching and not distributed caching. This is a complete crash course on distributed caching.
This is by far the best system design explanation I've ever heard. It's clear and so very organized, which makes it easy to follow. Please make more!!
Appreciate the kind words, Saffaura!
This is gold. Addresses many small details that other videos just glossed over
This is the only channel where it feels like if we were to actually build something how we would do it.
Other channels are mostly yeah put some load balancers, a reverse proxy bla bla bla. The way you go in-depth about each thing and make us aware of the tradeoffs is truly incredible.
I watched the view counter and heavy hitter and came here, and surprisingly my thoughts ran exactly as parallel as you. Looks like I am learning.
Thank you so much Mikhail.
The best system design demo i've even seen, with detailed and sound explanation behind each step in approaching the problems. Thanks!
Another great video, I can't say the importance of details in system design and as far as I've seen only your videos cover it. Don't get swayed by other 'scratching the surface' videos, keep up the good work!
I'm short on words how good Mikhail explains things. Initially I was just going over the video without making any notes. Then once I started making notes, I realized there are tons of information in everything single component or design choice he explains.
I liked the way you started from simple local system and evolved to distributed cache by solving all the challenges on the way.
Your videos are very informative!!
Glad you liked!
This is the most informative, detailed, comprehensive, concise, and easy to follow video I've ever seen. Thank you so much for making this video.
This is a complete video and in just 34 minutes. The hard work you have put in this video is evident.
You have covered many scenarios that can be reused in other system design questions.
Hi Mukesh. Thanks a lot for the feedback!
Watched other youtube videos too, but this one is the best. It starts with the requirements, then talks about LRU cache and even it's quick code implementation. Then talks about how cache host is selected (like Consistent Hashing) and how Cache Client selects the Cache Hosts (talks about all 3 ways). Also, nicely explained with diagrams. Thanks a lot.
You covered almost all perspectives of the interviews. Really liked the flow and simplicity.
This is by far the best demonstration of designing a distributed cache. I really appreciate the way you brought up the the solution to a bigger problem in a progressive approach. Loved it. Keep growing.
Wow. I'm currently studying system design and have to say this is by far the best resource I've run into and I've paid for multiple services and classes on system design. Thank you for creating these awesome videos, you definitely have a +1 follower.
Thank you, Daniel, for all the kind words!
Hey Daniel. I also liked this channel very much.
Unfortunately it stopped receiving uploads for now.
Do you mind to share from the services and classes you've paid each ones you liked the most?
So I can continue my studies from here to somewhere else.
One of the most detailed explanation of distributed cache. That includes the way it should be approched with, in the interviews. Thanks a lot
Speaking logical facts with a straight face is a next level skill.
I have watched too many video on this. Yours is simply the best so far. Some just scratch the surface, others just dive straight into complex technicalities. This is class apart.
I wish you upload more stuff in the future!!
A big thanks again...
Glad you liked it, Pratyush. Appreciate the feedback!
Great explanation!
I hope you will continue with posting more System design interview videos. The fact that you started from the naive approach to a better solution I consider is a big thing to consider all pros/cons!
Hi Dumitru. Appreciate your feedback! Working on more videos. If you like the format, I hope the next video (in progress) will resonate well with you.
This video is incredible !!
I have never seen a more complete structured solution to a system design question.
Great work, Kudos !!
This channel is highly underrated!. This is the very best explanation of distributed cache I have ever come across. It's simple and touches every core components through a concise explanation. Big thanks Mikhail!, expecting more uploads.
Glad you like the video, Nithin. Thank you for the feedback! More videos to come.
I like your style of repeating concepts throughout several videos. It helps reinforce them and allows the viewer several viewing opportunities to let it sink in over different videos (versus just watching the same video over and over to absorb something, which can get boring).
This is one of the best videos available on the internet for system design of distributed cache like Redis.
One of the best system design videos. Actually much better than most of paid video contents.
The creator of this video clearly demonstrated his deep understanding to the topic.
The BOSS of System of Design! Please keep making more videos. You are awesome! Thank you !
You are awesome )) Thanks for sharing the feedback,
Manoj.
Got exactly this question on a system design interview. Thanks a lot! I was properly prepared. This is the best set of lectures on system design I've ever saw.
Excellent explanation! This channel is very underrated the quality of the videos are far better than any other in the internet.
The way you explained system designs in your channel is awesome. Appreciate your work. Please put more videos.
Hi Mikhail. This is the first video that I watched on the channel and I have to say it's the best resource out there. I love the way you progress to a better solution at each step. Keep up the good work. I'll use this information while implementing a distributed cache in my project. Thanks.
Great explanation. The ending of these videos helps identifying how to solve real world problems using the content in the videos.
Thank you, @cnanavat, for all your feedbacks!
The details and trade-offs you talk about clearly indicate that this information is acquired through a lot of practical experience. A lot of hard work has gone into this from your end. Thank you for sharing your knowledge. Look forward to more videos from your channel.
I had read about Partitioning and Replication in isolation and their tradeoffs but was looking for how it comes together in a real-world application. Every second of this talk is pure gold :) I had gone through other System design articles/talks but they all just scratched the surface and was unsatisfied. The way you have progressively enhanced the solution is very apt and I really like the references for further reading on consistent hashing. Kudos!
Thank you very much, Swapnil, for such a detailed feedback! Glad you liked the video! Let me know if you have any questions. About this or any other video on the channel.
@@SystemDesignInterview you should upload more videos on system design. You have a lot to teach.
Your channel and explanation of System design for various topics is so neat and elegant. Even before you dive into the design aspect of the question, the identification of the functional and the non functional requirements itself is so useful. Truly yours is a very under rated channel. Hope to see many more videos from you.
I am glad I bumped into this channel before my interviews. Man what explanation. Just super
You are so awesome. THis is the only channel which gives a easy to complicated solution instead of explaining how redis works withs its huge complicated architecture which is not useful for system design interviews.
The content that you've put is extremely high in quality, leaving nothing to chance. I haven't seen such pure content since DDIA. Many thanks for making it available for free.
Excellent video! Quality of content is top notch. I spent hours reading up on each components before I came across your videos. You are doing an amazing job!
Thank you, Trupti.
Excellent explanation among tens of Videos I have seen on RUclips on this topic. Better than paid courses online.
Although it’s covered at a high level, the video covers a wide range of factors to be considered when designing the system, very informative and helpful to work out an all rounded solution. Highly recommended!
Thank you for the feedback, eeyore345!
Very well put together. I loved how you started simple and then expanded on it as more requirements were added one by one. Keep up the good work.
Thank you, Pratibha, for the feedback! Much appreciated!
Best video for anyone who wants to understand the basics of distributed caching
I like how you keep it high level and summarize periodically. Like the structure.
Thank you for the feedback, vetiஅரவிந்த! Glad to know that both the structure and the approach resonate well with you.
This video provides an excellent balance of high-level concepts and diving deep into details. An incredible resource to find, and vastly better than many paid options out there on systems design.
10:50 - make cash distributed
15:05 - consistent hashing
22:15 - Achieving high availability
You channel is the best. I have seen so many system design videos so far. But the way you cover smallest of the details is awesome. You deserve claps from side. 👏
Glad you liked it, Keerthikanth! Appreciate the feedback.
He is one of the best, I have seen so far. Full details, with integrated parts of system, covering all the aspects. Awesome!!! Please make some more videos like this.
Thank you for the kind words, Dharmendra. More videos to come.
I guess he is the only who covered and connected the dots between LRU cache implementation and how to transition that to a distributed hash table problem. The content is amazing !!!
Please add more videos.. you have been the best channel on SD I have seen so far.
This is one of the best system design related videos I've seen on youtube. Thank you
I'm amazed by the amount of information introduced in the video. Like other people said, kudos to you!
After watching many videos on consistent hashing, I am glad that I came across yours. The way you explained the process was simple and easy to understand. I really appreciate your efforts in putting together quality content.
I never get an offer when I smile and try to leave a positive impression in my interviews. I need to learn to be more like this guy.
this is probably best system design interview video i've watched. i learnt so much, thank you!!
By far the best resource I found on Distributed Caching. What detail, man. Hooked!!
Glad you liked it, Garry! Appreciate the feedback!
A very well crafted video. This should serve as the benchmark for system design interviews.
this is The only channel that gives you insights of how to do system design interviews.
Beautiful video. Thanks a lot, Man! I recently finished my masters in Distributed Systems and this video helped me consolidate my otherwise Distributed Knowledge ;)
such a clarity of thought on every single component and design choice, loved the depth !
You have amazing skills of explaining concepts, you have truly mastered it. I leaned a lot by watching your videos. Keep posting.
By some reason RUclips hides valid comments. I can only see such comments in the inbox, but there is no way for me to reply. Let me re-post such comments on behalf of people who submitted it.
From @krfong krfong
Hi Mikhail, thank you for taking the time and effort to make these videos. I'm currently preparing for interviews, and your videos have helped reinforce my knowledge, in some parts, and reveal my gaps in others, of how to design distributed systems components. 😊 THANK YOU!! I'd like to address your question above - whether or not you cover the right amount of breadth and depth in your videos. First, as I understand the audience that you're teaching to (and let me know if I've got the wrong impression) - new grads and swes who are not yet sr. or put another way, those who have conceptual knowledge of distributed systems, but are not yet able to synthesize this information and apply them to real-world applications, and also, those who do not yet have the experience of designing/explaining conceptual models in an interview format. If, in fact, you are creating for these kinds of people, I'd like to say that you should continue to teach for them because your videos address a (much-needed-to-be-filled) gap in the space of accessible distributed systems learning materials. Current accessible web content like Guarav Sen's and Tushar Roy's videos on system design, reading materials such as Martin Kleppman's "Designing Data Intensive Applications", Mikito Takada's book "Distributed Systems for Fun and Profit", the High Scalability blog, and seminal papers in distributed systems, all are not able to teach distributed systems from the angle that you do. What those resources are unable to address (and what you've been able to do) is to synthesize distributed system concepts and demonstrate how to apply them to designing distributed system components in an interview format. Your videos have been able to put a language, a loose-script, if you will, of how to have a conversation with others about distributed systems; that is vital for those who are learning how to design distributed systems in a considered way. I think you have a good balance of breadth and depth. To offer a suggestion, people could benefit from your videos if you discuss how your designs' address questions like failover scenarios, how the design addresses r/w skew, dirty r/w and lost updates, and linearizability (or consistency) implications towards the end of your videos. Lastly, I'd like to ask a question, are your videos representative of how you'd suggest one should discuss building a system design component in an interview? If anything, what have you left out of your videos that candidates should consider in a system design interview, generally speaking and in specific to this video, or the notification or distributed rate limiting ones? I love your work, thank you, and awesome stuff!!
Hi @krfong krfong. I can imagine amount of effort you put into writing this feedback. Very insightful and informative. Appreciate it a lot! Let me try to answer your questions.
I am sure you know it already that there is no one single format for answering system design questions. With coding questions things are much more clear. In a nutshell: we start with asking clarifying questions, evaluate different options for the algorithm, write the code and explain our thought process along the way, test and make sure we covered edge cases, estimate time and space complexity. But with system design, interview may go in many different directions. That is why I try to touch various concepts and go deeper into more important of them (on my mind).
So, there is no one single format. But we can talk about expectations. For less senior engineers, an interviewer wants to see how you think. What challenges you see, what options you consider. Remember, that nobody expects ideal design after 45-60 minutes interview. Engineers spend months (depending on the size of the problem, of course) to design systems properly. It usually takes several iterations. And if a candidate can analyze requirements, evaluate options and make progress in the right direction, this is a good indicator that when join the company, engineer will apply the same critical thinking and come up with good design during her day-to-day job. When she has more time and help from more senior engineers. And to develop this critical thinking we need knowledge and exposure to software engineering problems. And this is also one of my goals, to share knowledge and how to apply it to solving problems.
For senior engineers expectations are higher. It is assumed you already have knowledge and know how to analyze problems. And it is no longer enough to demonstrate a solution that just works. We need to demonstrate how to optimize things and, eventually, how to reduce the cost. All kinds of cost: operational, dollar (e.g. hardware), cost of extensibility in the future. It is important to design systems that not just scalable and highly available (this is a given by default), but systems with close-to-optimal cost. Simplicity always wins.
If this topic of system design interview expectations is interesting to my viewers, please let me know. I will make a separate video.
Regarding topics I left out of the videos, this list is not short )) Each concept mentioned in videos may easily become a topic of its own. But I am sure you expect a more definitive answer from me. Let me name some important topics I left out:
- For Distributed Message Queue: how to store messages on disk. Here we should mention sequential reads/writes, page cache, embedded databases.
- For Notification Service: how delivery to subscribers work, how failures are handled. Here we should mention websockets.
- For Distributed Cache: a big topic of memory allocation.
These are some of the examples. Also, I do not talk much about hardware, memory, network throughput estimations. Will try to close this gap one day and devote videos to those topics.
Your videos are the best I've ever seen on RUclips. You've made understanding these complex components so much easier. Thank you so much! Hope you keep creating more awesome content.
Thank you for the feedback, Akshat! Much appreciated.
YOUR CHANNEL IS SO MUCH UNDERATED SIR!!! I SWEAR I'LL TRY MY BEST TO MAKE SPREAD A WORD ABOUT YOUR CHANNEL!!!!!
Very well explained videos. Like everyone mentioned, this channel is awesome. Thank you @System Design Interview. Sad to see no more new videos coming from this channel.
Love the encouragement to dive into details. Easy to see how someone involving technical details instead of relying on managed services to handle it would make them stand out in an interview. Thanks!
The best video I have seen covering a lot of topics on distributed systems.
Thanks for the video. This is the best system design channel on youtube! Please add more system design videos if time permits.
Thank you, Prasad, for the feedback! I work on more content.
Totally Agree. This is gold! You explain the complexities so clearly and without overloading the narrative with buzzwords. Thank you!!
the moment he said there are 2 consistency models: Eventual consistency (like gossip, epidemic) and stronger (with 2PC, consensus), I knew this is a golden content.
It has so many details in every min of video. I really appreciate the craft. I have watched every video many times.
Waiting for more like: Google drive, Uber, Netflix system design.
Really glad you liked the content, Omprakash! And appreciate the feedback! Topics you mentioned are in my TODO list. Thanks.
By watching this video, I have learned approaches for how to deal with hot shards, consistency, etc. Excellent work. Thank you very much
Thank you, Lian, for the feedback! Glad to be helpful!
Oh my... this is the best ever system design! If I am the interviewer I will be begging the recruiter to hire you, man...
The way you layout the problem statement, evolve and improve the system, revisit the requirements and discover further is just amazing. I can't believe you covered so much in just a half-hour.
@8:29 : Our Implementation of LRU cache will have HashMap and Doubly LinkedList. In Which HashMap will hold the keys and address of the Nodes of Doubly LinkedList . And Doubly LinkedList will hold the values of keys.
for the problem of uneven distribution of keys among servers in consistent hashing we can use aliasing to mitigate that. So each server can have any number of aliases hashed to positions on the ring. The name of the alias is based on the server id suffixed with some string value. Key hashes that map to a server alias are serviced by the actual server.
Thank you, Haytham, for these details! Well explained.
*A note for viewers*: this concept is also known as "virtual nodes" or "vnodes". Further interested in this topic, please take a look at this great post: medium.com/@dgryski/consistent-hashing-algorithmic-tradeoffs-ef6b8e2fcae8
Thank you for uploading this. Very informative and so far the best tutorials available online.
So many promised videos, please please make them. This is The Best system design channel on youtube.
This is my first comment on any video. What an incredible job you did in this video, I loved how you introduced the problem and suggested their solutions. Not to miss explained in very simple and efficient way. Thank you!
Great Video. Very well explained the relevant Ifs and Buts. Its the best so far on RUclips for Distributed Cache. Thanks a lot. Waiting for more videos.
Thank you, Ashish, for the feedback. Really appreciate it!
Wow, I think this channel has the best explanation of the available system design problems, which is able to provide a step-by-step explanation of how we get to the final design. Some other channels just slap the final design almost right away, which can be overwhelming and sometimes leave its sub systems inadequately explained.
Please add more system design problems, keep up the great work!
This video worth every second. I felt satisfied by watching the whole video. Prolific content
Thank you for the feedback! Glad you liked!
I was always struggle with HLD cache design. Always though it would be LLD but you explain in way that clears all my thoughts. Great work man :)
This is an excellent video for understanding the inner working of distributed cache. It starts from a very basic implementation and builds on top of it and clearly explains all concepts of distributed cache and major issues. Very helpful video.
Holy crap! These videos are awesome. It is very systematic but allows for other potential discussion. Thanks and keep creating more content. Will definitely recommend to my friends.
That is all I can ask for, Jay. Let's spread the knowledge! Thank you very much for the feedback!
I think you are brilliant. Please share your knowledge on a regular basis. Hope to see more videos in this channel.
Excellent application and explanation of distributed systems concepts - Partitioning, Replication, Configuration management, Consistent hashing along with pros and cons.
Amazing content. So much to-the-point and valuable information packed in 34 minutes. This one video is equivalent to multiple hours of videos from other channels out there! Many thanks Mikhail.