Hi Arpit, Got some questions. If you can answer it will be awesome. 1. Consistent hashing will be used for writing the data as well to DB right? 2. What happens if we remove one node from a ring of 4 & that removed node contains some data. So how to determine which Node we should transfer our data from that removed node? 3. How the cache will be updated in real time? Does it got updated when user create/update a rating, or it will fetch data from DB? Cache is of limited size so what would be the best approach for storing data: I think it should be LFU. What your views? 4. How to route the request in case a node goes down? Are we adding that item (another node from other AZ1) to consistent hashing ring or how?
1. Yes. It spits out data ownership. 2. Consistent hashing spits out that info 3. Depends on the usecase not one correct answer here. LFU with exponential decay can also work fine. 4. That's the recovery handler written in the routing layer (API servers) in this case. Or a separate component.
Hi Arpit, thanks for this channel. I was utilising it to fullest before I discontinued because of lack of dedication and consistency. And now when I visit videos section, I'm f**ked up how to cover all these gems. Not sure to bing watch them on weekends or to keep them aside for a while and start covering daily videos consistently.
Hi Arpit, Thankyou for such an amazing explanation. I wanted to ask you a question. Is it possible that our hash space gets skewed up at a certain point like the files are inserted on the left side of the hash space and the storage nodes are inserted on the right or vice versa? If yes, so wouldn't it result in the celebrity problem? I might be missing something here. Would love to hear your thoughts. Thankyou
Although hash functions are used in such a way such that we close to uniform distribution but in case if such a condition happens you can shift some data to another node but yeah this will require some changes in the review service hashing logic.
Hi Arpit. If we use NOSQL here what DB would you suggest a Cassandra or MongoDB? I know Cassandra is write efficient. But what about reads? And personally which DB would you choose? @AsliEngineering
Do we shard databases ourselves? Like in dynamo db, sharding is handled on its own. There is a provision I guess to shard but thats not usually used by anyone. Can you give some real world examples of where consistent hashing is done nowadays? (given db requirements are handled by cloud service providers and autoscaling, provisioning and sharding is done by the database automatically?
When we are creating new ring, how do we know which requests need to be relocated because we would be moving some keys from the older shard to new shard. How to find which key?
What will be the structure of the redis key and value? Also, for DB sharding, we don't generally recommend sharding for rdbms systems right? because joining data across multiple shards are costly? How does this get managed? Shouldn't we use cassandra here?
Cross sharing is frowned upon but if you are using shards on the basis of accommodation id you will never have to do cross shards. I think he used sql in the example because that's what booking also does. Also Cassandra is a good fit generally when there is more write to read ratio here the case is different. Although I am not sure about prematerialized view performance wrt to any other Nosql DB. I think that is the deciding factor here
what if routing decision is made by db router instead of consistent hashing algo written in routing service, i think db written routing alog is better than our own consistient hashing algo
So it turned out to saying that even reviews on your System-Design-Master-Class are playing a vital role. (You will come to know what I meant to say in your next cohort :) )
Hi aprit, i have one question that all this data migration that we have to do how is it done?. Do we write our own scripts for it to first migrate and then inform services about new node . What is the right way of handling this
As he already said there are read replicas. You can have leader election Master goes down slave takes over and yes consistency can be comprised for such use cases as availability and latency has been prioritised
very good explanation ! does writing service will scale ? as we have only one master node , or Cassandra ( multiple writers) will be good choice for that ? , as we are not looking for acid compliant behaviour
A Cassandra cluster would be a great choice. +1 but since he mentioned trade offs I think we can trade off consistency for availability (so maybe 2 write masters which will be eventually consistent)
00:19:00 Thank You… If we use asynchrous replication.. How do we handle replication lag ? For example, Imagine If I have written a review for hotel which goes to master and then immediate read query went to replica which has’t recieved replica data yet… In the userinterface, User might think something went wrong with his submitted review and they will resubmit it too ? How do we handle this situation ?
@@AsliEngineering so while booking any cab via any app like Uber, Ola etc. They all share a OTP for the ride which keeps on changing. But rapido keeps a constant OTP for users which is 4 digits so how they are managing large scale
I thought about it, but that diagram is really very very very high level. THey are literally just a bunch of microservices. If I create a video on it, no one will learn anything concrete from it. It will just be a list of microservices and guessing what it does. Say, I create a 20 min video on it, that will definitely get me views, but will add nothing to the viewer. Thank you so much for suggesting this, but I really do not want to waste the time of anyone watching my video. I like to keep the video information dense and useful. Hope you understand.
I believe all these complexities will be abstracted by distributed database providers like DynamoDB, ScyllaDB, or CockroachDB.
Hi Arpit, Got some questions. If you can answer it will be awesome.
1. Consistent hashing will be used for writing the data as well to DB right?
2. What happens if we remove one node from a ring of 4 & that removed node contains some data. So how to determine which Node we should transfer our data from that removed node?
3. How the cache will be updated in real time? Does it got updated when user create/update a rating, or it will fetch data from DB? Cache is of limited size so what would be the best approach for storing data: I think it should be LFU. What your views?
4. How to route the request in case a node goes down? Are we adding that item (another node from other AZ1) to consistent hashing ring or how?
1. Yes. It spits out data ownership.
2. Consistent hashing spits out that info
3. Depends on the usecase not one correct answer here. LFU with exponential decay can also work fine.
4. That's the recovery handler written in the routing layer (API servers) in this case. Or a separate component.
@@AsliEngineering Thank you :)
liked your explanation Arpit. Thanks a ton
thanks🙏
Very Informative video Arpit...Thanks :)
In Japan, last year 2023, some hotels owners did large lawsuit agajnst Booking, because of payment delay. Booking side execuse was System trouble.
Hi Arpit, thanks for this channel. I was utilising it to fullest before I discontinued because of lack of dedication and consistency. And now when I visit videos section, I'm f**ked up how to cover all these gems.
Not sure to bing watch them on weekends or to keep them aside for a while and start covering daily videos consistently.
30 minute a day ! thats how I do it .. will catch up eventually
Hi Arpit, Thankyou for such an amazing explanation.
I wanted to ask you a question.
Is it possible that our hash space gets skewed up at a certain point like the files are inserted on the left side of the hash space and the storage nodes are inserted on the right or vice versa? If yes, so wouldn't it result in the celebrity problem?
I might be missing something here. Would love to hear your thoughts. Thankyou
Although hash functions are used in such a way such that we close to uniform distribution but in case if such a condition happens you can shift some data to another node but yeah this will require some changes in the review service hashing logic.
Hi Arpit. If we use NOSQL here what DB would you suggest a Cassandra or MongoDB? I know Cassandra is write efficient. But what about reads? And personally which DB would you choose? @AsliEngineering
Excellent explanation .
Do we shard databases ourselves? Like in dynamo db, sharding is handled on its own. There is a provision I guess to shard but thats not usually used by anyone. Can you give some real world examples of where consistent hashing is done nowadays? (given db requirements are handled by cloud service providers and autoscaling, provisioning and sharding is done by the database automatically?
depends on the database you are using. in managed db we need not do anything but if you are self hosting then yes.
@@AsliEngineering got it, thanks!
How would data integrity be handled when shards are added or removed?
When we are creating new ring, how do we know which requests need to be relocated because we would be moving some keys from the older shard to new shard. How to find which key?
Changes happen atomically all at once.
What will be the structure of the redis key and value? Also, for DB sharding, we don't generally recommend sharding for rdbms systems right? because joining data across multiple shards are costly? How does this get managed? Shouldn't we use cassandra here?
Cross sharing is frowned upon but if you are using shards on the basis of accommodation id you will never have to do cross shards. I think he used sql in the example because that's what booking also does. Also Cassandra is a good fit generally when there is more write to read ratio here the case is different. Although I am not sure about prematerialized view performance wrt to any other Nosql DB. I think that is the deciding factor here
when the new ring is being prepared, are reads/writes served by old ring? Is there any downtime associated?
nope. no downtime needed.
what if routing decision is made by db router instead of consistent hashing algo written in routing service, i think db written routing alog is better than our own consistient hashing algo
I had the exact same question, why do we want to recreate what database could already do, routing its requests to corresponding shards
Thanks
Thank you so much for the kind gesture :)
08:50
it is two regions and not AZ
So it turned out to saying that even reviews on your System-Design-Master-Class are playing a vital role. (You will come to know what I meant to say in your next cohort :) )
heheheh. suspense :) looking forward to it :)
Hi aprit, i have one question that all this data migration that we have to do how is it done?. Do we write our own scripts for it to first migrate and then inform services about new node . What is the right way of handling this
Yes. We write the scripts.
What if the master shard goes down during write re and quest. I guess here we could avoid strong consistency for available
As he already said there are read replicas. You can have leader election Master goes down slave takes over and yes consistency can be comprised for such use cases as availability and latency has been prioritised
very good explanation !
does writing service will scale ? as we have only one master node , or Cassandra ( multiple writers) will be good choice for that ? , as we are not looking for acid compliant behaviour
A Cassandra cluster would be a great choice. +1 but since he mentioned trade offs I think we can trade off consistency for availability (so maybe 2 write masters which will be eventually consistent)
Best best best awsome
00:19:00 Thank You… If we use asynchrous replication.. How do we handle replication lag ? For example, Imagine If I have written a review for hotel which goes to master and then immediate read query went to replica which has’t recieved replica data yet… In the userinterface, User might think something went wrong with his submitted review and they will resubmit it too ? How do we handle this situation ?
You will ensure Read Your Write consistency by redirecting the critical reads to the master.
@@AsliEngineering Read Your Write consistency, is this any specific case we should wonder about while using master-slave replication?
Hi! Nice video can u share your thoughts on how Rapido is keep a constant OTP of 4 digits for all users.
Have never used Rapido. So no idea about it.
@@AsliEngineering so while booking any cab via any app like Uber, Ola etc. They all share a OTP for the ride which keeps on changing. But rapido keeps a constant OTP for users which is 4 digits so how they are managing large scale
@@bharatarya7929 then OTP are not unique just a random number assigned to a user.
Great use case and amazing presentation.
Wondering, any specific reason for using relational db?
Can't we solve it using NoSQL db?
You can so long as it fits the usecase. No hard bounds.
Can we get twitter design video based on the diagram shared by Elon Musk ?
I thought about it, but that diagram is really very very very high level. THey are literally just a bunch of microservices.
If I create a video on it, no one will learn anything concrete from it. It will just be a list of microservices and guessing what it does.
Say, I create a 20 min video on it, that will definitely get me views, but will add nothing to the viewer.
Thank you so much for suggesting this, but I really do not want to waste the time of anyone watching my video.
I like to keep the video information dense and useful.
Hope you understand.
@@AsliEngineering Sure. That makes sense :)