System Design Interview - Step By Step Guide
HTML-код
- Опубликовано: 27 ноя 2019
- Please check out my other video courses here: www.systemdesignthinking.com
Topics mentioned in the video:
- Stages of a typical system design interview: functional requirements (API), non-functional requirements, high-level design, detailed design, bottlenecks and tradeoffs.
- Why requirements clarification is so important.
- What questions to ask the interviewer.
- How to design API.
- Non-functional requirements to consider: scalability, performance, availability, consistency, cost.
- How to define a data model.
- How to scale a SQL database.
- Apache Cassandra high-level architecture.
- Data processing concepts: checkpointing, partitioning, in-memory aggregation, deduplication cache, dead-letter queue, embedded database, state management.
- Data ingestion pipeline concepts: blocking vs non-blocking I/O, buffering and batching, timeouts, retries, exponential backoff and jitter, circuit breaker pattern, software vs hardware load balancing, load balancing algorithms, DNS, health checking and high availability of load balancers, partition strategy, hot partitions, client-side and server-side service discovery, single leader replication and leaderless replication, textual vs binary data formats.
- Data retrieval pipeline concepts: time-series data, data rollup, hot storage, cold storage.
- Types of performance testing: load testing, stress testing, soak testing.
- Health monitoring.
- Audit systems.
Inspired by the following interview questions:
Google (www.careercup.com/question?id...) 
Наука
Continued from someone's comment :
1:30 Why you need to ask your interviewer questions? Why requirements clarification is so important?
3:19 What we may want to ask about? Requirements clarification (User/Scale/Performance/Cost)
7:14 Define functional requirements-API
9:19 Define non-functional requirements
10:32 High-level architecture
11:56 Define Data model (what we store)
16:09 Where we store: List of non-functional requirements
17:20 Where we store: How SQL database handle these requirements
20:40 Where to store: About NoSql: Cassandra
25:00 How we store: data modelling for sql v nosql
29:03 Data processing path
30:51 Data processing: Data aggregation basics
34:42 Data procesing : Processing service detailed design
41:08 Data Ingestion pipeline
1:03:00 Data Retrieval path
1:06:35 Data flow simulation
1:08:26 Technology stack
1:13:22 Bottleneck, tradeoffs and more
1:09:06 Summary
Kudos to the creator. Happy Learning.
Can this comment be pinned above?
1:19:07 Summary
I recently interviewed at 2 top companies and I just nailed the system design interviews. In total, I had to go through 4 such 45 min interviews. I have around 10 years of engineering experience but my background is not strictly distributed systems. I am good at DS/algo but system design was my achilles' heel. Mikhail made it simpler for me to catch up. I did read a lot of books to understand how systems like Kafka, Zookeeper, Cassandra etc. are implemented. I had lots of bits and pieces of information from lots of sources. This channel helped me put all that understanding together in a coherent way. I also followed the structure described by Mikhail in the interviews and it helped me keep the discussion on point. The interviews were difficult, interviewers asked tough questions but listening to these videos multiple times helped me calm my nerves.
I haven't heard back from those companies yet but initial signs look positive. We will see how it pans out. Regardless of the result, I feel satisfied with my preparation and a big chunk of credit goes to Mikhail.
Why am I writing such a big comment here? Well, I want to obviously thank Mikhail but I want people to realize how valuable this channel can be for interview preparation. Please try to make it more popular by sharing it with your friends.
Hi primespace! Did I pay you to write this post? ))
To be serious, I am really glad for you. To know that these videos helped you both sharpen your knowledge and give you a boost in confidence, makes me feel happy.
Regardless of the result, you should be proud of yourself. Watching these videos thoroughly multiple times and reading through a lot of resources is a hard work. I know it. So, give yourself a credit.
If you feel inclined, please let us know the final result of your interviews.
@@SystemDesignInterview Hahaha! No money was involved and it is coming straight from heart :). It should be the other way around. People should be paying you for this excellent content. I compared it with paid content like "Grokking the system design interview" and your content is far more practical and miles ahead.
Yes, I will share the outcome once I hear back from them.
One unique pattern I observed in these interviews was that the interviewers forced me to design a system which does not need to scale and will run only on one machine. They wanted to discuss the threading model, how would I use mutexes and locks etc. It almost seemed like they wanted me to keep things very simple and instead of saying big things, come up with simple solutions like - I will use SQLite because requirements are simple. Just something to watch out for. People shouldn't fall in the trap of assuming the interviewer is asking to design a highly scalable system. I almost fell for it but corrected myself in one interview. I said things as simple as I'd simply store the data in XML file if it is only 100 users and the interviewer was very happy to hear my response. Once we spent 15 minutes on very simple stuff then only they moved to scaling aspects. Surprisingly they drilled me on things like how DNS works, how TCP handshake happens and what is SSL etc. These questions were asked at random times while I was talking about something unrelated. In any case, these were easy enough to answer so all good.
@@SystemDesignInterview The results came back. Yes from both. Still has to match teams and officially get offers but so far so good. Thank you again, Mikhail! It would not have been possible without your help.
Well done, primespace, well done.
@@abby297in thank you for sharing your experience!! did you actually know the answers for those networking layer-specific questions and do you think familiarity with the OSI model would be useful to study at all?
So you created a channel, made 6 videos, showed us how a system design interview video should really be like, then dropped the mic and left :-) I am really sad that there aren't any more videos, but at least I hope the reason is that you are in a very happy place, and you don't have time anymore, because you enjoy your life :-) In any case, thanks a lot for the videos that you did make! There are other decent videos on system design, but none matches the ones on your channel
He has a course. Has some more detailed content
@@arun_kannan what's the name? I'd love to look it up
@@arun_kannan I would also like to know what it is 🙂
ok, course bought 🙂link is now in the channel description, from what I noticed
@@avraamtolmidis5188how is it? Is the 1st module good?
I'm still waiting for the 2nd module before I buy it. He mentioned on Leetcode that he's 60% done with the 2nd module.
By far the only person who talks about system design after mastering it in youtube!
Mikhail, you should win a RUclips Creator award for this channel. The content is just so exceptional.
Hey man, please continue releasing videos. It's the best material I've seen so far.
The quality and clarity is amazing
@@dhawang.gayash4080 popppoppopppppooppppppoipooiipipioqqqqqqqqqqqqq
Unfortunately no new content.
This is pure gold. I have slept several times, woke up, rewinded several times, made notes, still watching. You are a legend. Hats off to you, you spent 1.23 hours talking non stop on a serious and important topic, covering everything. You explained several topics in so simple language which was hard to grasp. One was data consistency and availability and other was Eventual consistency and many more. Thanks once again. May God Bless you.
don't look at it as a 83 min video, it contains enough contents to use a week to digest it :) it is absolutely fantastic, comprehensive, understandable, and maybe the best video in System Design Interview on RUclips and anywhere else. Thanks for all the efforts to providing these contents.
I hope you continue creating content. This is the kind of content that needs to be out there more. It's not something we all learn on the job, in college, or even in our spare time with pet-projects. (Who has the money or time to toy with distributed systems of this complexity and scale as a side-project to learn about them? Practically no one!) Yet, it's something that is part of interviews now very regularly even for people with a few years of experience even if they'll never touch a system like this in their entire career. Interesting to learn about at least - just so dense though. Each little service or idea could be an entire chapter in a textbook itself and you've covered well over 30 or 40 of these in this video.
This is hands-down the best system design interview prep channel. I failed many system design rounds until I watched this video, and nailed the next one.
I watched a number of system design interview prep videos. This one really stands out and I wished I had watched this one earlier. He seems to be a real architect and knows the real stuff, unlike some other fluffy ones. Great, hats off, Mikhail!
1:30 Why you need to ask your interviewer questions? Why requirements clarification is so important?
3:19 What we may want to ask about? Requirements clarification (User/Scale/Performance/Cost)
7:14 Define functional requirements-API
9:19 Define non-functional requirements
10:32 High-level architecture
11:56 Define Data model (what we store/Where we store)
after 12 minute you closed video
@@moscowhits3227 lol
How can give up after 12min? Later on is the real hard part
Just watch the video, this is pure gold
The best video on the system design I have seen so far. Thank you for your effort on making such a detailed and structured video!
This is awesome!
I really miss this video during my preparation to my first interview I had failed.
This video really made me believe in myself. Thank you Mikhail, your way of explaining such complicated topic as a system design is the best I have ever seen.
Truly THE best video on System design. It is so detailed and thorough. It was an amazing learning video. Thank you, Mikhail! Appreciate all your effort in putting this up.
Best System Design Interview Preps. Builds up the system slowly and explains in breadth and depth. I would pay for this
Thanks god I found this channel by accident while I was preparing for my upcoming system design interview. Best material one could find on the web, very structured, detailed and covers a lot of topics. I didn't get the offer but system design was my best interview and this video (and others) really helped me to lead conversation and structure my thoughts. I really hope that you would continue making videos because it's the best resource for preparation out there. Thanks a lot!
I'm so touched by your words when you said, "If you are still with me watching this video you should be proud of yourself. Seriously. (82:02)" because I felt really proud.
Thank you so much, Mikhail
It took me a few days to get through this video. As a software engineer with 8 years of experience, you have very clearly demonstrated to me that I know nothing.
Thanks
D:
Cram time...
The best channel for system design interview questions. 🔥
Glad you like the channel, Ashish. Thank you for the feedback.
I almost never comment on RUclips videos, but I just had to let you know how helpful this was. I can't even imagine how much time and effort was put into this. So detailed and explained beautifully with such clarity, thank you!
I wanted to thank Mikhail for the amazing job he does and for the top-quality content he makes on this channel. I just got an offer from Amazon and I owe my success at the system design interview to the content of this channel. Keep up the good work!
Спасибо!
Congrats, Maxim! Happy to hear that!
Thank you Mikhail ! I went through these high quality videos multiple times and these enabled me to clear multiple system design rounds including a dream offer from a faanG company. Thanks again.
I just found this channel, I don't know why you stopped making videos but you are godsent. I learned more in these 5-6 videos than I have learnt in 100s of pages of books and hours of videos. This is just gold. If you find time please do make more!.
I've been in the interview loops several times in my 8year software engineering career and as part of interviewing, I've followed multiple different system design interview preparation sources. This video is the best source of information that one needs when preparing for a system design interview. The way the concepts are put in a clear, concise and defined manner is commendable. Really appreciate the hard work put behind creating such an informative video.
Such a great material! Tons of complicated information and always straight to the point.
Thank you for helping us becoming a better engineers!
As a beginner, I have to say that this is the best and easy to understand tutorial, watching several times along the learning process helped a lot . Appreciated.
I think you are the only qualified person I've found to talk on this topic on RUclips. So many people draw boxes with zero justification, depth or discussion of tradeoffs. Thank you for the tremendous effort
You are a superb presenter and COACH. (I am very tough on people who claim to be teachers but can't do it effectively so this is intended as the highest compliment.)
Obviously you also know the material widely and deeply, and yet understand how to organize it for people who don't have your skills.
Thank you
The best video I've seen so far about the system design question approach! Watch over and over again. Thank you from your fan :-)
You are back, Memfis! Glad to see you. And thank you for the feedback!
This is one of the most comprehensive work done ever to explain system design fundamentals. Thanks a lot and I really wish that you keep posting such wonderful videos.
I have never designed a system before, but going through this video has made me feel like I am ready to design someone's system.
Thank you, Mikhail, this video is well put together.
Finally, I have completed the video, two days. I enjoyed every single piece of information. Thank you so much.
I worked in support for 12 years and trying to get into SRE. Your System design videos are LIT. Please continue the good work. I already learned quite a bit from this video. The technique is neat.
Much appreciate this upload. I think I speak for anyone who's watched this video fully that this is a thoroughly informative and practical video covering the salient spectrum of a complex topic.
Very well done and looking forward to more videos or resources. Many thanks!
This is the best 1h23min I've ever spent on youtube. You are so detailed, so clear. I am absolutely humbled but your delivery and the depth of consideration you bring. Please keep at it, keep this simple, factual format.
Awesome work! Just want to show my respect and gratitude here. I would definitely recommend viewing this channel and read the book multiple times to fully understand how to pick an idiomatic distributed system design for different scenarios. Wish to see more high-quality videos on this channel! Thank you very much for sharing!
Thanks to you I landed a job in Manga. Words can't express my gratitude. We miss you!
We miss you! Still watch you video occasionally to refresh the memory. Please produce more videos!
I'm preparing for the incoming interviews, got to say that your system design videos are top among the best youtube can offer. Would love to hear more from you.
Yesterday I gave my first FAANG interview for Amazon, I watched all of your videos "at least" twice. It was a great help. THANK YOU!
Hope it went well! Wish you all the luck at the interviews, Komal!
One brilliant thing about this video is that that the teacher is literally designing another Kafka when talking about partitions piece of the system(the one which is used by partition service). Rather than abstracting that piece by just calling it Kafka, the author is talking about the barebones structure of that piece thereby also teaching us how Kafka itself works(on some level).
I'm preparing for a Senior TPM interview at Amazon next week, and your videos are hands down the most helpful videos on youtube. Thanks for all your hard work!
Hi Sean! Appreciate the feedback! And wish you all the luck on your interview!
I cannot thank you enough for putting these system design videos up, they helped me a ton while preparing for my SDE2 interview at Amazon and developing the right mind set for system design . I'd like to share the good news with you that I landed my dream job offer with your help. This channel is indeed the best system design interview prep on youtube. Thank you again !
Always glad to hear stories like this! Thank you, Ethan, for sharing it.
It is your hard work that helped you land your dream job. Wish you all the best in your new role at Amazon!
Hey man, do you have any tips for gaining the technical expertise to be a sde2? (Other than leetcode and system design)
This is mind blowing. I didn't expected this kind of system design tutorial on RUclips. Your style to explain HLD and then deep dive into each component, Its awesome and hidden gem. Hats off to you. I know it takes lots of time to create such content but please try to upload more videos on system design.
Appreciate the feedback, Dheeraj! Thank you.
exactly
This channel is the best of all available on RUclips for System Design.
Glad to hear back from you, Algorithm Implementer! Have not seen you for a while.
You are back! Thank you very much! Just failed system design on facebook. I hope this video helps me with the next interview! You are the best!
Hi!
First of all, do not be discouraged that the interview didn't go as expected. Interview is a lottery. Preparation may help to increase our chances a lot. But never to 100%. Keep pushing and don't give up!
Second, thank you for the feedback! Let me know if you have any questions. I am here to help.
Master of System Design is back.
Missed you all as well! ))
I am actually looking for a Lead Design Engineer to build a scalable distributed systems with a minimum server capacity for 1 billion users.
Must be willing to travel, manage production team, and be prepared for a fast progression of project integration.
If you're not CTO capable, not looking long term big money with Equity potential. Well....it's not a good fit.
I need someone who wants to build something amazing.
Why... because It's an amazing project.
If you know anyone please reply me.
@@CestuiQueTrustBeneficiary-KING I am interested. My email id is prgupta8891@gmail.com
This video explains *so many terms and concepts*. I wish this video existed when I first joined AWS Kinesis in 2015, because I kept hearing all these terms and not understanding them. Such a tremendous resource, thank you for this monumental effort!
This awesome video matched the name 'SystemDesign Interview'. I would keep watching all series
Thanks! Even though I'm a new grad I have a system design interview to prepare for.. this video is very helpful!
You are a life saver 🙌 you made me so confident in my interview. I got asked a question that was here and I attacked it out of instinct.
Took me 5 hours to take notes and get through this video, but it was worth it. Literally every second is important
Михаил! Thanks so much for doing this. I have an interview with FB in a couple of weeks and your tutorials are really helping with the structure and approach. Really appreciate this! Спасибо огромное!
Миша, thank you a lot for the feedback! Wish you luck on your interviews!
Получилось пройти?
This is absolutely the best one and everyone should watch to get a grasp on the System Design interviews. We are very thankful to Mikhail to make this and available to everyone free.
Glad you liked it, thunrou. Thank you for the feedback!
wow.. this is perhaps the best video I have seen on System Design Interview
This is trully the best channel on system designs. By far! Thank you for all your effort, it is greatly appreciated!
Удачи с проектом!
Подписался на канал. Будем ждать новых видео :)
I have been waiting for almost 6 months. Finally you are back. Definitely the best system design video I have watched so far.
Hi ichbinas. Yes, this video took me a while to prepare. Glad you liked it!
Amazing content: high quality, relevant and easy to understand. Please make more. It’s worth paying for, thank you for keeping the education free
Hi Gregory. Appreciate the feedback! Thanks.
The quality of the content of your videos is incredibly high.
Very well done, a great cover of different topics and a very good and detailed explanation of components and tradeoffs.
Keep up the good work!
This is the single best video on system design on youtube thus far. Please keep making more!
Appreciate your feedback, Santosh.
Best system design channel I've ever seen on youtube, thanks so much, I'm so excited that I fount it. Great work!
the Best system design guide out there, the perfect video stressing on the way to think, rather than what to do, the recaps were aptly timed too. I owe you big time.
Glad to know that, Aravamuthan! Thank you for the feedback.
By far the best video on system design interview questions, it saved at least 10 hours of effort. Great part is that you can apply these same concepts in a lot of questions , because of the depth that you covered. Liked and Subscribed !!
Glad to be helpful!
He did System Design of System Design Interview👏
Watching the first 6 mins of the vid took me 30 mins. This is Gold.
Truly one of the best videos I have come across for System Design concepts and approach. Please continue posting videos of such quality.
Thank you, Ashish!
Even a video like this is getting 27 dislikes ? From competing SD channels ?
Dude awesome work. You are really helping me aggregate information in a cohesive manner.
Hands down.. one of the best videos on system design I came across so far..
Each word he says is gold mine and every sentence deserves to be seperate video for other youtubers to explain.
No kidding, I pause to write down the gold nuggets of information. lol
Best system design video ever. Thanks, Mikhail.
Looking forward to more :)
It is rare to have your attention span > 15 mins, I was skeptical to start the long video. I'd like to personally share with everyone, this is a well-drafted & organized video! The content is atomic, well-stitched and flows seamlessly throughout duration. Keep producing more such content, Kudos!
The best system design interview channel on RUclips!
You are setting the bar high here, Q )) But I will try to keep it high.
Thank you for the feedback! Much appreciated!
I just wanted to say that I came across this channel by coincidence after looking at many system design channels and your videos are the best.
I rarely comment but this channel is a hidden gem and I wanted to thank you Mikhail for the time you put into making these videos.
Thank you, Adel Ali, for all the kind words! Appreciate the feedback!
The structure of your content is simply amazing!!! How the dots get connected can be visualized much better!!!
seriously we need to be united and force him to release more videos, this work is precious then gold
Thanks Mikhail for this gem of a video. You are doing a great service to the software community. We would all love to see you releasing more content, if possible.✌️
Mikhail, thanks for your insights and detailed walkthrough on the system design topic. It was one of the key sources that I referred and it really helped to get an SDM role with Amazon. Thanks a TON
Very high quality video. I'm currently preparing for system design interviews and your channel is highly informative. I particularly enjoy your usage of diagrams, since I am a visual learner. Keep up the great work!
Best System design video. I liked the way you methodically went through the entire flow and explained the trade offs at each step. Please keep up the good work. You are awesome!!!
Thanks!
Spent hours learning this video😃 The knowledge is explained very clearly and the content is perfectly designed. Thanks so much for the wonderful tutorials!
This is so far the best content I found about system design interview!
This was an awesome video for system design approach !!! I've seen few other videos, but this one beats them all. Exactly what i was looking for. Very well structured. Thank you very much !
Thank you for the feedback, Nrithya!
Thank you Mikhail for the awesome content! Your's is simply the best channel for system design interviews. Pls keep more videos coming.
Every sentence this gut says is gold.
Thank you Mikhail for making an amazing channel. Yes I agree this channel beats all channels and does provide a systematic way to deal with system design problems. It is like a superset of all along with clear and well organized approach. Please keep making such videos.
Thank you for the feedback, Mangesh!
This is one of the best videos to start with system design basics.
Thanks a lot Mikhail for creating these System Design videos. I've really found all of your videos very helpful and easy to follow. Building on all the approaches that you've shared via your channel, I was able to crack the Google's System Design interview round. I couldn't have imagined it a few months ago that I could clear Google's System Design rounds. Thanks again and do share more and more videos with us :)
Best system design video I ever came across. Thank you so much!
This is an excellent video. Even premium services can't provide such a high quality content video on System Design. Kudos :)
😱 This is literally a shock-content.
Awesome, Brilliant, Many thanks, Mikhail! Please keep making videos, I can imagine how much time and effort you spent on this, but still, it was a pleasure watching all the videos from the first second!
Ps. appreciate you little jokes you put in those videos :)
After reading so many youtubers for sys design, this is THE best one.
Wow! This video pretty much covered all the Systems concepts I studied in my Masters. Thanks.
I have a System Design Interview tomorrow afternoon - Im cramming using this course. Definitely will help a lot. Thanks!
Best system design videos! Thank you Mikhail!! Looking forward to more!!!
awesome videos!!! noticed that you mentioned this channel will have other relevant videos in this video. I'm excited about that!
Best video that covers the breadth and enough depth of the distributed systems. Thanks a lot
Thank you, Mikhail, for your exceptional content! I learned a lot from you.
Please make more such videos. I have learnt more from this video than the whole of my experience years combined.
By far the best video on System design i have seen on RUclips till now , thank you sir for sharing such valuable information in an organized and detailed manner.
Thank you, Pranov, for the feedback! Glad you liked the video!
@@SystemDesignInterview Please make more videos, like the different types of database and advantage and disadvantage of them. The content is really great !
What a great video! Looking forward to those newer videos covering more concepts of sysdes!