21: Distributed Locking | Systems Design Interview Questions With Ex-Google SWE
HTML-код
- Опубликовано: 26 апр 2024
- There seems to be some sort of distributed consensus that people lock their car doors whenever they see me walking near them
Наука
Наука
I am a software engineer at Amazon, and your videos have helped me a lot. I was recently able to get senior and principal level offers from Oracle, Microsoft, and a few others. Thanks again. Appreciate it.
Holy moly that's awesome!! Congrats on your offers!!
Hey, can we connect on LinkedIn?
Once you read Designing Data-Intensive Applications all the videos becomes a great source to revisions. Great quality videos, keep up the good work!!!!
The intro is hilarious😂p.s. congrats on the 30k milestone!
Excellent job Sir
Thanks for the content! Preparing for an upcoming Mid level role, hopefully your videos will help me
Good luck!
You are doing awesome work
The content is great, but I always come for the golden nugget in the description
I churn out nuggets in the description and on the toilet
hi jordan thanks for those vidoes very helpful,
but something it's clear for me how does s3 handles the fencing tokens? doesnt the appliation requires another layer for solving this out?
S3 is a bit of a bad example because we don't own s3. But imagine you own whatever data source you're sinking to, you can just build this into the logic there.
congrats, whta introo
Your intros are always weirdly funny :) Question: in your final design, is the queue also written to followers? i.e. If Leader were to go down, would followers know that B is waiting for A? How would the websocket be restored by A?
Yep, the queue is written to followers. If the leader goes down, the clients will notice it, and reach out to the other links of nodes in the zookeeper cluster to get the address of the new leader and connect there.
Hey , I love the way you teach . Can you also do this concepts in hands-on project in spring boot or something , so that we can improve our coding and also learn how to test such scenario in real Life
This is something I've thought about, but realistically would take me a very long time to do haha. In my current state, it's unlikely I can, but maybe if I have some significant life changes
Nice Video but i have one query. In case of distributed consensus, how reads are done for lock ?. If it is read from replica which is not upto date, it can lead to a problem. I have also watched your raft videos, which gives me impression that distributed consensus provides lineraliziability but not strong consistency in reading.
You read from the leader. This is slow, but it's still fault tolerant, because we have the ability to perform a fail over if the leader goes down.
LE'S GO KNICKS!
Hey Jordan , Could you please create a video on stock exchange system design, that would majorly focus on the users getting a notification on the stocks they have subscribed if the stock values go up or down based on some parameters in real time.
Thanks
Will do eventually!
In the Distributed Conses, we had 5 nodes(1 leader, 4 followers). When leader went down, we chose the follower that was up to date. What if as soon the follower was chosen as a leader, it went down. Now we have 2 nodes with old data and 1 with new. What happens in this case?
The one with new data must become the leader. If it goes down, we can't proceed as we can no longer reach a a majority of nodes.
What editor are you using for drawing? Do you also use any pen based device?
Apple pencil + oneNote
@@jordanhasnolife5163 thanks for your response 🫡
No Flink and CDC used? Jordan, are you ok?
No someone said my videos were too unrealistic for interviews on reddit and now I'm in a deep state of depression
@@jordanhasnolife5163 I think that's true, but I don't necessarily think it's bad. Your videos aren't really structured in an interview style. You go into a lot of depth in your designs. It would be unrealistic to draw the sometimes huge designs your show at the end of your videos in the space of an interview - especially somewhere like Meta where you realistically only have 35 minutes of design time, but it's beneficial to see everything as inspiration for how you might deep dive in different areas. In a real interview you may only deep dive into a couple of the areas you show.
Hi Jordan, really appreciate the content, is it possible for you to share your ipad notes. It is difficult to follow and revise your content without the notes and making the entire notes while following the video is time consuming.
It would be really helpful if you could share your hand written notes from ipad (maybe it is not perfect but still a better reference than nothing) which we could keep as reference to follow your content. As we go through the video, we could add our own comments or notes on it to make it more clear. Please consider.
Planning on doing this in bulk after finishing my current series, this will be in the next 1-3 months.
what if you have a large amount of connections on the leader node? How do you deal with that situation?
I'm assuming this is for many different locks, you basically have to partition them across many zookeeper clusters.
@@jordanhasnolife5163 does that mean this leader + several followers just act like one zookeeper node. And for horizontal scale up we need more zookeeper nodes with sharding(and every node have their own leader and followers)?
@@user-kf1ul3uh2w You are correct
Thanks for the great video. I need your help with one of my task. I will become your patreon if you help. In my current company I have received one task in which I have to execute queries in the order they were originally executed. I have a list of queries and their original start and end times. So to execute them again in the same order we need to build dependency graph. How we can build this dependency graph.
Query 1: start time 1 end time 3
Query 2 start time 2 and end time 5
Query 3 start time 4 end time.
Qry 2 can start after qry 1 has started. Query 3 can be started after 1 finished and 2 started
My implementation is not efficient as I am checking for every query all the query started before it and storing the dependencies in list
Doesn't really make much sense to me considering the start times and end times. But look up topological sorting. Make a graph of the dependency relationships, and run a topological sort. This will tell you when you can schedule a given task, at which point you can run a second job that looks at currently scheduled tasks and whether their start time has passed.
I don't have a patreon, send it to charity.
What is the source of truth for all this?
I'm not sure what you're asking - do you mean my sources?
@@jordanhasnolife5163 Thanks for replying! I meant where did you learn about all this? Is there a comprehensive resource or is this just result of your years of experience in tech?
@@sid4579 Well considering that I don't have many years of experience in tech, I'm going to say that I did not learn anything that way. I'm simply just aggregating any information that I can find across anywhere on the internet. If there was a comprehensive resource for it, I don't think I'd be making these videos in the first place, as I myself am attempting to be a comprehensive resource for it.
@@sid4579 Martin Kleppmann's book and RUclips videos cover a lot of this.